Recap: Android Components

1. Recap: Android Components Application= Set of Android Components Intent • Activity • UI Component typically corresponding to one screen. • BroadcastReceiver • Responds to notifications or status changes. Can wake up your process. • Service • Faceless task that runs in the background. • ContentProvider • Enable applications to share data.

2. Android Programming Lecture 10 Multimedia

4. Audio Video MP3 MIDI PCM/WAVE AAC LC etc… H.263 H.264 AVC MPEG-4 etc… The Android multimedia framework includes support for playing variety of common media types, so that you can easily integrate audio, video and images into your applications.

5. Media Playback • Android multimedia framework includes support for playing variety of common media types, so that you can easily integrate audio, video and images into your applications. • You can play audio or video from media files stored in your application’s resources (raw resources), from standalone files in the file system, or from a data stream arriving over a network connection, all usingMediaPlayerclass

6. Media Playback Guide • http://developer.android.com/guide/topics/media/mediaplayer.html

7. Media Player • One of the most important components of the media framework is the MediaPlayer class. An object of this class can fetch, decode, and play both audio and video with minimal setup. It supports several different media sources such as: • Local resources • Internal URIs, such as one you might obtain from a Content Resolver • External URIs (streaming) • Supported Media Formats are: • RTSP (RTP, SDP) • HTTP/HTTPS progressive streaming • HTTP/HTTPS live streaming • 3GPP • MPEG-4 • MP3

8. Step 1: Create the raw folder in the project and put the media file into the folder Step 2: Configure the media player object to start the media playback // Create an instance of MediaPlayer and load the music MediaPlayermediaPlayer= MediaPlayer.create(this, R.raw.music); // Start the media playback mediaPlayer.start(); To replay the media, call reset() and prepare() To pause, call pause() To stop, call stop() MediaPlayer class reference: http://developer.android.com/reference/android/media/MediaPlayer.html

9. Media Player • Whit this example you can play an audio file as a local raw resource from res/raw/ directory: • You can also load a local content through an URI Object • You can also play a content from a remote URL via HTTP streaming. MediaPlayermediaPlayer = MediaPlayer.create(context, R.raw.sound_file_1);mediaPlayer.start(); // no need to call prepare(); create() does that for you Uri myUri = ....; // initialize Uri hereMediaPlayer mediaPlayer = new MediaPlayer();mediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC);mediaPlayer.setDataSource(getApplicationContext(), myUri);mediaPlayer.prepare();mediaPlayer.start(); String url = "http://........"; // your URL hereMediaPlayermediaPlayer = new MediaPlayer();mediaPlayer.setAudioStreamType(AudioManager.STREAM_MUSIC);mediaPlayer.setDataSource(url);mediaPlayer.prepare(); // might take long! (for buffering, etc)mediaPlayer.start();

10. Read Media Information

11. Callback Functions

12. Task at home Follow this tutorial to make a cool sound equalizer: http://www.101apps.co.za/articles/perfect-sound-using-the-equalizer-effect-a-tutorial.html

13. Audio Recording

14. Audio Capture • You can record audio using the MediaRecorderAPIs if supported by the device hardware • Emulator does not have the capability to record audio and video MediaRecorder class reference: http://developer.android.com/reference/android/media/MediaRecorder.html

15. Media Recorder • Create a new instance of android.media.MediaRecorder. • Set the audio source using MediaRecorder.setAudioSource(). You will probably want to use microphone with MediaRecorder.AudioSource.MIC • Set output file format using MediaRecorder.setOutputFormat(). • Set output file name using MediaRecorder.setOutputFile(). • Set the audio encoder using MediaRecorder.setAudioEncoder(). • Call MediaRecorder.prepare() on the MediaRecorder instance. • To start audio capture, call MediaRecorder.start(). • To stop audio capture, call MediaRecorder.stop(). • When you are done with the MediaRecorder instance, call MediaRecorder.release() on it to release the resource. http://developer.android.com/guide/topics/media/audio-capture.html

16. privatevoidstartRecording() { • // 1. Create a new instance of android.media.MediaRecorder • MediaRecordermRecorder= newMediaRecorder(); • // 2. Set the audio source using MediaRecorder.setAudioSource() • // In this example, use microphone to get audio • mRecorder.setAudioSource(MediaRecorder.AudioSource.MIC); • // 3. Set output file format using MediaRecorder.setOutputFormat(). • mRecorder.setOutputFormat(MediaRecorder.OutputFormat.THREE_GPP); • // 4. Set output file name using MediaRecorder.setOutputFile(). • mRecorder.setOutputFile(“/sdcard/myaudio.3gp”); • // 5. Set the audio encoder using MediaRecorder.setAudioEncoder(). • mRecorder.setAudioEncoder(MediaRecorder.AudioEncoder.AMR_NB); • // 6. Call MediaRecorder.prepare() on the MediaRecorder instance. • try { • mRecorder.prepare(); • } catch (IOException e) { • Log.e(LOG_TAG, "prepare() failed"); • } • // 7. To start audio capture, call MediaRecorder.start(). • mRecorder.start(); • } • privatevoidstopRecording() { • // 8. To stop audio capture, call MediaRecorder.stop(). • mRecorder.stop(); • // 9. call MediaRecorder.release() on it to release the resource. • mRecorder.release(); • mRecorder = null; • }

17. Things to Remember: Manifest The application needs to have the permission to write to external storage if the output file is written to the external storage, and also the permission to record audio. These permissions must be set in the application's AndroidManifest.xml file, with something like: <manifest . . . > . . . <uses-permission android:name="android.permission.RECORD_AUDIO"/> <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" /> . . . </manifest>

18. SoundPool • Multiple sounds can be played from raw or compressed source • A SoundPool is a collection of samples that can be loaded into memory from a resource inside the APK or from a file in the file system • More efficient than media player in terms of CPU load, latency • Adjustable playback frequency • Each sound can be assigned a priority • Support for repeat mode SoundPool class reference: http://developer.android.com/reference/android/media/SoundPool.html

19. Creating Sound Effects • SoundPool is designed for short files which can be kept in memory decompressed for quick access, this is best suited for sound effects in apps and games • Media Player is designed for longer sound files or streams, this is best suited for music files or larger files. The files will be loaded disk each time created is called, this will save on memory space but introduce a small delay (not really noticeable) MediaPlayer SoundPool Refer to the video for a comparison between the two classes https://www.youtube.com/watch?v=bcYX5fa_jFc

20. Constructs a SoundPool with the maximum number of simultaneous streams and Audio stream type. SoundPoolsoundPool = new SoundPool(4, AudioManager.STREAM_MUSIC, 0); intsoundId = mSoundPool.load(this, R.raw.b1, 1); soundPool.play(soundId, 1f, 1f, 1, 0, 1f); Load the sound from the specified resource. Play the music clip • soundID – Sound ID returned by the load() function • leftVolume – Left volume value (0.0 ~ 1.0) • rightVolume – right volume value (0.0 ~ 1.0) • priority – stream priority (0 = lowest priority) • loop – loop mode (0 = no loop, -1 = loop forever) • rate – playback rate (0.5 ~ 2.0, 1.0 = normal playback) SoundPool class reference: http://developer.android.com/reference/android/media/SoundPool.html

21. Task at home In assignment 1 game, ask the player to record the short sound, and use the sound in the game, like when the ball hits the paddle and bounces

22. Video

23. VideoView • VideoViewis a View that has video playback capabilities and can be used directly in a layout • It is a View working with media player • The VideoView class can load images from various sources (such as resources or content providers), takes care of computing its measurement from the video so that it can be used in any layout manager, and provides various display options such as scaling and tinting • We can then add controls (play, pause, forward, back, etc) with the MediaController class.

24. Video View • Add VideoView component on layout file • Set content on VideoView • setVideoURI(...) • setVideoPath(...) • Use start(), stop(), etc. to control video. main_layout.xml MainActivity.java

25. Callback Functions • setOnPreparedListener -> onPrepared() • Called when VideoView is prepared for play. • Use when waiting contents on online to be prepared. • setOnCompletionListener-> onCompletion() • Called when VideoView playback is completed

26. Using Native App • Invoke the video playback by using the common intent • Remember, your app is now in the background. • Intent intent = new Intent(Intent.ACTION_VIEW, Uri.parse(srcPath));intent.setDataAndType(Uri.parse(srcPath), "video/mp4");startActivity(intent);

27. Camera

28. Camera • The Android framework includes support for various cameras and camera features available on devices, allowing you to capture pictures and videos in your applications. • The Android framework supports capturing images and video through the Camera API or camera Intent.

29. Camera • Camera • This class is the primary API for controlling device cameras. This class is used to take pictures or videos when you are building a camera application. • MediaRecorder • This class is used to record video from the camera. • Intent • An intent action type of MediaStore.ACTION_IMAGE_CAPTURE or MediaStore.ACTION_VIDEO_CAPTUREcan be used to capture images or videos without directly using the Camera object.

30. Image Capture Intent • Capturing images using a camera intent is quick way to enable your application to take pictures with minimal coding. An image capture intent can include the following extra information private static final intCAPTURE_IMAGE_ACTIVITY_REQUEST_CODE = 100;private Uri fileUri;@Overridepublic void onCreate(Bundle savedInstanceState) {super.onCreate(savedInstanceState);setContentView(R.layout.main);// create Intent to take a picture and return control to the calling applicationIntent intent = new Intent(MediaStore.ACTION_IMAGE_CAPTURE);fileUri= getOutputMediaFileUri(MEDIA_TYPE_IMAGE); // create a file to save the imageintent.putExtra(MediaStore.EXTRA_OUTPUT, fileUri); // set the image file name // start the image capture IntentstartActivityForResult(intent, CAPTURE_IMAGE_ACTIVITY_REQUEST_CODE);} http://developer.android.com/guide/topics/media/camera.html

31. Image Capture Intent • MediaStore.EXTRA_OUTPUT- This setting requires a Uri object specifying a path and file name where you'd like to save the picture. This setting is optional but strongly recommended. If you do not specify this value, the camera application saves the requested picture in the default location with a default name, specified in the returned intent's Intent.getData() field. private static final intCAPTURE_IMAGE_ACTIVITY_REQUEST_CODE = 100;private Uri fileUri;@Overridepublic void onCreate(Bundle savedInstanceState) {super.onCreate(savedInstanceState);setContentView(R.layout.main);// create Intent to take a picture and return control to the calling applicationIntent intent = new Intent(MediaStore.ACTION_IMAGE_CAPTURE);fileUri= getOutputMediaFileUri(MEDIA_TYPE_IMAGE); // create a file to save the imageintent.putExtra(MediaStore.EXTRA_OUTPUT, fileUri); // set the image file name // start the image capture IntentstartActivityForResult(intent, CAPTURE_IMAGE_ACTIVITY_REQUEST_CODE);}

32. Retrieve Results @Overrideprotected void onActivityResult(intrequestCode, intresultCode, Intent data) {if (requestCode == CAPTURE_IMAGE_ACTIVITY_REQUEST_CODE) {if (resultCode == RESULT_OK) {// Image captured and saved to fileUri specified in the Intent • Bitmap thumbnail = data.getParcelableExtra("data"); Toast.makeText(this, "Image saved to:\n" +data.getData(), Toast.LENGTH_LONG).show(); } else if (resultCode == RESULT_CANCELED) {// User cancelled the image capture} else {// Image capture failed, advise user} }if (requestCode == CAPTURE_VIDEO_ACTIVITY_REQUEST_CODE) {if (resultCode == RESULT_OK) {// Video captured and saved to fileUri specified in the IntentToast.makeText(this, "Video saved to:\n" +data.getData(), Toast.LENGTH_LONG).show(); } else if (resultCode == RESULT_CANCELED) {// User cancelled the video capture} else {// Video capture failed, advise user} }} Receive the result with the file system location of the new Image Receive the result with the file system location of the new Video.

34. Media Router • As users connect their televisions, home theatre systems and music players with wireless technologies, they want to be able to play content from Android apps on these larger, louder devices. Enabling this kind of playback can turn your one-device, one-user app into a shared experience that delights and inspires multiple users.

35. Media Router https://www.youtube.com/watch?v=_NGB10uN6OI

36. Recommended Reading • Android Developer Site: Media and Camera • http://developer.android.com/guide/topics/media/index.html • Vogella Tutorial: Handling Media with Android • http://www.vogella.com/tutorials/AndroidMedia/article.html