Your Metro style app, video and audio, Part 1

1. PLAT-775T Your Metro style app, video and audio, Part 1 Gabe Frost Senior Program Manager Lead Microsoft Corporation

2. demo Music & Video apps It’s easy to build tailored playback experiences with rich file metadata.

3. Agenda • Delivering great tailored audio and video experiences • Media platform • HTML5 Media Elements • Programming Model • A word on formats • Recap You’ll leave with examples of how to • Create apps with rich, integrated media experiences • Incorporate media into your apps in new ways

4. Building apps with great audio and video experiences is easy.

5. demo Weather app Use media to make apps more visually rich and engaging.

6. With media APIs + Media Elements • Take advantage of graphics hardware and devices with little domain knowledge • Easy to build tailored user experiences with built-in UI and controls • Simple to use media from sources other than web servers, like Local or HomeGroup PC libraries and network attached storage • Demanding scenarios like streaming, 3D and DRM are simplified because of the powerful and extensible underlying media platform

7. Media platform Windows Metro Style App <audio src=“…”> <video src=“…”> WinRT (Windows.Media) Streaming Capture Transcode Playback/Preview Media Foundation Audio/Video Source Video Decoder Video Effect 1 Video Effect N Video Encoder Audio/Video Sink Audio Decoder Audio Effect 1 Audio Effect N Audio Encoder

8. HTML5 Media Elements

9. Standard HTML5 <audio> and <video> Select methods and attributes • play() • pause() • load() • audioTracks • readyState • controls • currentTime • duration • error • src • playbackRate • preload • volume • … • W3C HTML5 Spec • You can create rich media experiences by combining with CSS and JavaScript and DOM events • Fully hardware accelerated • Performance, battery life and reliability

10. Standard HTML5 <audio> and <video> HTML Video Tag Events • abort • canplay • canplaythrough • durationchange • emptied • ended • error • loadeddata • loadedmetadata • loadstart • pause • play • playing • progress • ratechange • seeking • seeked • suspend • stalled • timeupdate • volumechange • waiting

11. Example: HTML5 Audio and Video Elements • <!-- Attributes allowed on both tags: src, preload, autoplay, • loop, controls. Video tags also support poster, width, height, muted. Both support many events (e.g. volumechanged) --> • <audio id="myAudioTag"src="mySong.wma"preload="none"/> • <audio src="mySong.m4a"autoplay loop controls /> • <video id="myVideoTag"src="myVideo.mp4"preload="none"/> • <video src="myVideo.wmv"autoplay loop controls /> • <video src="myVideo.mp4"autoplay muted /> • <video src="myAudio.mp3"poster="myImage.jpg"controls />

12. Example: HTML5 Audio and Video Elements • <audio id="myAudio"src="http://www.contoso.com/track1.mp3"autoplay /> • <script type="text/javascript"> • var audio = document.getElementById("myAudio"); • // Handle when the audio track has reached the end • audio.addEventListener("ended", function() { • // Switch the audio src to a new track • audio.src = "http://www.contoso.com/track2.m4a"; • }); • </script>

13. Example: Local files with Media Elements • <video id="player" controls autoplay /> • <script type="text/javascript"> • // Invoke a file picker for the user to select media from the file system • var picker = new Windows.Storage.FileOpenPicker(); • picker.pickSingleFileAsync(). • then(function(file) { • // Create a URL to represent the local file • varurl = URL.createObjectURL(file); • document.getElementById("player").src = url; • }); • <script>

14. Example: Accessing file metadata • // Programmatically select the file (could also be from home network storage) • varfile = new • Windows.Storage.KnownFolders.musicLibrary.getFileAsync("track.mp3"). • // From the file, extract media-specific properties • then(function(myFile) { • myFile.properties.getMusicPropertiesAsync(). • then(function(musicProps) { • var album = musicProps.album; • varartist = musicProps.artist; • vartitle = musicProps.title; • // etc. • }); • });

15. Windows 8 enhancements • 3D video • Audio and video effects and extensibility • Audio output selection such as Bluetooth • Background audio • DRM such as PlayReady • Low-latency support for communications & games • Play To for streaming to TVs & audio systems • Zoom and mirror mode for video

16. Programming Model

17. Windows.MediaAPIsMuch more than playing video and audio Capture Snap a picture, record video and audio Devices Input and output audio devices such as Bluetooth Playlists Work with audio playlists PlayTo Stream media to a television or audio system Windows.Media. Protection Use PlayReady or other content protection systems Convert music or video to other formats or resolutions, trim, or make effects permanent Transcoding Insert or remove well-known video effects such as Stabilization VideoEffects

18. demo Streaming to devices with Play To HTML5 websites using IE10, Music, Video and Photo apps work today. Simple to support in your Metro style apps

19. Example: Stream media with Play To (JS) • <video id="videoplayer" src="http://www.contoso.com/clip.mp4" controlsautoplay/> • <script type="text/javascript"> • // Step 1: Obtain PlayToManager object for app’s current view. • varptm = Windows.Media.PlayTo.PlayToManager.getForCurrentView(); • // Step 2: Register for the sourcerequestedevent (user selects Devices button). • ptm.addEventListener("sourcerequested", function(e) { • var request = e.sourceRequest; • // Step 3: Specify the media to be streamed (to filter devices) • var deferral = request.getDeferral(); • request.setSource(document.getElementById("videoplayer").msPlayToSource); • deferral.complete(); • // The media will then be streamed to the device chosen by the user in the UI. • }); • </script>

20. Example: Stream media with Play To (C#) • <MediaElement x:Name="videoplayer" Source="http://www.contoso.com/clip.mp4" • AutoPlay="true" /> • // Step 1: Obtain PlayToManager object for app’s current view. • PlayToManagerptm = Windows.Media.PlayTo.PlayToManager.GetForCurrentView(); • // Step 2: Register for the SourceRequested event (user selects Devices button). • ptm.SourceRequested += (PlayToManager sender, PlayToSourceRequestedEventArgs e) => { • request = e.SourceRequest; • // Step 3: Specify the media to be streamed. • PlayToSourceDeferral deferral = request.GetDeferral(); • request.SetSource(videoplayer.PlayToSource); • deferral.Complete(); • } • // The media will then be streamed to the device chosen by the user in the UI.

21. demo Camera Capture UI It’s easy to snap a photo or record video by invoking the built-in capture UI.

22. Use the camera to snap a photo • // Application manifest capabilities required to access camera and microphone • <Capabilities> •     <DeviceCapability Name="webcam" /> • <DeviceCapability Name="microphone" /> • </Capabilities>

23. Use the camera to snap a photo • <imgid="preview" /> • <script type="text/javascript"> • // Invoke the camera capture UI for taking a photo • varcaptureUI = new Windows.Media.Capture.CameraCaptureUI(); • captureUI.captureFileAsync(Windows.Media.CameraCaptureUI.Mode.photo). • then(function(capturedItem) { • if (capturedItem) { • // Display the photo • document.getElementById("preview").src = • URL.createObjectURL(capturedItem); • } • }); • </script>

24. demo Simple Encode app Convert media files into different formats, resolutions or trim to a specific timeframe.

25. Transcode to a different format • varoutputFile; • varvidLib = Windows.Storage.KnownFolders.videosLibrary; • var transcoder = new Windows.Media.Transcoding.MediaTranscoder(); • // Choose a popular output format and quality • var profile = Windows.Media.Capture.MediaEncodingProfile. • createMp4(Windows.Media.Capture.VideoEncodingQuality.vga); • // Create the file to transcode into, and select the input file to read from • vidLib.createFileAsync("output.mp4"). • then(function(result) { • outputFile = result; • return vidLib.getFileAsync("test.wmv"); • // Transcode the file • }).then(function(inputFile) { • return transcoder.transcodeFileAsync(inputFile, outputFile, profile); • }).then(function() { // log "done" });

26. Simple trim • varoutputFile; • varvidLib = Windows.Storage.KnownFolders.videosLibrary; • var transcoder = new Windows.Media.Transcoding.MediaTranscoder(); • // Specify the start and stop time defining the portion to keep • // Create the file to transcode into, and select the input file to read from • vidLib.createFileAsync("EditedVacation.mp4"). • then(function(result) { • outputFile = result; • return vidLib.getFileAsync("vacation.mp4"); • // Trim the file: profile isn’t changing so profile argument isn’t needed • }).then(function(inputFile) { • return transcoder.transcodeFileAsync(inputFile, outputFile); • }).then(function() { // log "done" }); transcoder.trimStartTime = 900; transcoder.trimStopTime = 6000;

27. A word on formats

28. Media Formats and Codecs System-wide • File format: MP4 (preferred) or ASF • Video: H.264 or VC-1 • Audio: AAC, MP3 or WMA • Broad support across consumer hardware: great performance and battery life Per-app • Your Metro style app can package private plug-ins for its own use (e.g. adaptive streaming)

29. Recap

30. With media APIs + Media Elements • Take advantage of graphics hardware and devices with little domain knowledge • Easy to build tailored user experiences with built-in UI and controls • Simple to use media from sources other than web servers, like Local or HomeGroup PC libraries and network attached storage • Demanding scenarios like streaming, 3D and DRM are simplified because of the powerful and extensible underlying media platform

31. Related sessions • [PLAT-776T] Your Metro style app, video and audio, Part 2 • [APP-788T] Integrating stunning media experiences in XAML • [PLAT-777T] Capturing personal photos, video, and audio in Metro style apps • [HW-715T] Building a great Metro style device app for your camera • [PLAT-778T] Media fundamentals of a communications app • [PLAT-783T] Extending the media platform using input, output and processing plug-ins • [HW-177T] Building great networked media devices for Play To apps • [APP-408T] Integrating with the Windows device experience • [HW-329T] Understanding Wi-Fi Direct in Windows 8

32. Further reading and documentation • Design guidelines for Play To receivers • Streaming media to devices using Play To • Play To & media streaming for Metro style apps • Using audio hardware acceleration for ultra-low-power consumption.

33. thank you Feedback and questions http://forums.dev.windows.com Session feedbackhttp://bldw.in/SessionFeedback

34. © 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

35. BACKUP

36. Use the camera to snap a photo (C#) • // Define capture element in XAML • <captureElementx:Name="preview" Width="320"Height="240"Margin="10,5,10,5" /> • // Initialize • mc = new MediaCapture(); • Windows.Media.Capture.InitializeOperationopInitialize = await mc.initializeAsync(); • // Assign the source • preview.Source = mc; • // Start the preview • Windows.Media.Capture.StartPreviewOperationopStartpreview = await • mc.StartPreviewAsync();

37. Transcode to a different format (C#) • // Select the input file to read from • StorageFileinput = await • KnownFolders.VideosLibrary.GetFileAsync("input.wmv"); • // Create the file to transcode into • StorageFileoutput = await • KnownFolders.VideosLibrary.CreateFileAsync("output.mp4"); • // Choose a popular output format and quality • MediaEncodingProfileprofile = • MediaEncodingProfile.CreateMp4(VideoEncodingQuality.Vga); • // Transcode the file • MediaTranscoder transcoder = new MediaTranscoder(); • await transcoder.TranscodeFileAsync(input, output, profile);

38. Simple trim (C#) • // Select the input file to read from • StorageFileinput = await • KnownFolders.VideosLibrary.GetFileAsync("input.mp4"); • // Create the file to transcode into • StorageFileoutput = await • KnownFolders.VideosLibrary.CreateFileAsync("output.mp4"); • MediaTranscoder transcoder = new MediaTranscoder(); • // Specify the start and stop time defining the portion to keep • // Trim the file: profile isn’t changing so profile argument isn’t needed • await transcoder.TranscodeFileAsync(input, output); transcoder.TrimStartTime = 900; transcoder.TrimStopTime = 6000;