Syncing Audio, Video, and Animations in Silverlight: A Comprehensive Guide

1. Syncing Audio, Video and Animations in Silverlight Dan Wahlin

2. Session's Twitter Hash Tag #MIX10Sync

3. Agenda • What Problem Are We Solving? • Sync Options • Using Media Markers • Using Attached Properties • Creating Media Behaviors

4. What Problem Are We Solving?

5. The End Goal Animation 1 Begins Animation 2 Begins Animation 3 Begins …. Video 1 Plays Video 1 Stops Audio 1 Plays

6. Sync Options Empty Storyboard Media Markers Attached Properties Media Behaviors

7. Sync Options Empty Storyboard Media Markers Attached Properties Media Behaviors

8. Media Markers • Markers provide a way to raise events at specific times as audio or video plays • Supported by Expression Encoder • MediaElement supports markers: • Markers property • MarkerReached event • Useful for media-driven syncing of audio, video and animations

9. Creating Media Markers 2 3 4 1

10. Exporting Media Markers 5

11. Media Marker XML • Marker XML file created by Expression Encoder: <?xml version="1.0" encoding="utf-8"?> <Markers> <Marker Time="00:00:02" Value="Media1" GenerateKeyFrame="True" GenerateThumbnail="False" /> <Marker Time="00:00:04" Value="Media2" GenerateKeyFrame="True" GenerateThumbnail="False" /> </Markers>

12. Using Media Markers • Adding to the Markers collection: void MainPage_Loaded(object sender, RoutedEventArgs e) { XDocument doc = XDocument.Load("Markers.xml"); var markers = from marker in doc.Descendants("Marker") select new TimelineMarker { Text = marker.Attribute("Value").Value, Time = TimeSpan.Parse(marker.Attribute("Time").Value) }; foreach (var marker in markers) { this.MyMedia.Markers.Add(marker); } this.MyMedia.MarkerReached += MyMedia_MarkerReached; this.MyMedia.Play(); }

13. Demo: Using Media Markers

14. Sync Options Empty Storyboard Media Markers Attached Properties Media Behaviors

15. Attached Properties • Animations can target Attached Properties: <DoubleAnimationStoryboard.TargetProperty="(Canvas.Top)" /> • Custom Attached Properties can be used to sync animations with audio/video • As an Attached Property value changes (due to animation) an event can be raised

16. Creating an Attached Property • Using DependencyProperty.RegisterAttached() public class StoryboardTimer {public static readonlyDependencyPropertyMillisecondsProperty = DependencyProperty.RegisterAttached( "Milliseconds", typeof(double), typeof(StoryboardTimer), new PropertyMetadata(0.0, new PropertyChangedCallback(OnMillisecondsChanged)) ); }

17. Animating an Attached Property <StackPanel x:Name="LayoutRoot" sync:StoryboardTimer.Milliseconds="0"> <DoubleAnimation x:Name="MillisecondsAnimation" Storyboard.TargetName="LayoutRoot" Duration="00:00:15" From="0" To="15000" By="50" /> Storyboard.SetTargetProperty(MillisecondsAnimation, new PropertyPath(StoryboardTimer.MillisecondsProperty)); StoryboardTimer.MediaKeyFrameTriggered += MediaTimeline_MediaKeyFrameTriggered; void MediaTimeline_MediaKeyFrameTriggered(...) { SyncMedia(e.Milliseconds); //Play or stop media }

18. Demo: Using Attached Properties

19. Sync Options Empty Storyboard Media Markers Attached Properties Media Behaviors

20. Media Behaviors • Behaviors provide a flexible way to sync media with animations and perform special effects • Supported in Visual Studio and Expression Blend • Add new "behavior" to an existing element to extend its functionality • Can be attached directly to a Storyboard when media needs to be synced with animations

21. The MediaTimelineBehavior • MediaTimelineBehavior simplifies syncing media and animations: • Attach to Storyboard • Define media keyframes declaratively, in external file or in code • Monitors Storyboard's current time • Automatically starts, stops or pauses media at appropriate times based upon media keyframes

22. Creating the Behavior • Creating the MediaTimelineBehavior: public class MediaTimelineBehavior : Behavior<Storyboard> { protected override void OnAttached() { base.OnAttached(); _Storyboard = this.AssociatedObject; //Create DispatcherTimer object } public void Begin() { //Start Storyboard and DispatcherTimer } protected override void OnDetaching() { base.OnDetaching(); //Unhook events and stop DispatcherTimer } }

23. Using the MediaTimelineBehavior <Storyboard x:Name="SyncStoryboard"> <i:Interaction.Behaviors> <sync:MediaTimelineBehavior> <sync:MediaTimelineBehavior.MediaKeyFrames> <sync:MediaKeyFrameTargetName="Media1" KeyTime="00:00:02" MediaAction="Play" /> <sync:MediaKeyFrameTargetName="Media1" KeyTime="00:00:05" MediaAction="Stop" /> <sync:MediaKeyFrameTargetName="Media2" KeyTime="00:00:03" MediaAction="Play" /> <sync:MediaKeyFrameTargetName="Media2" KeyTime="00:00:06" MediaAction="Stop" /> </sync:MediaTimelineBehavior.MediaKeyFrames> </sync:MediaTimelineBehavior> </i:Interaction.Behaviors> <!-- Storyboard Animations --> </Storyboard>

24. The SlowMotionVideoBehavior • SlowMotionVideoBehaviorallows a video to be played in slow motion for a specific duration: <MediaElement x:Name="Media1" AutoPlay="True" Volume="0" Source="Wildlife.wmv"> <ic:Interaction.Behaviors> <sync:SlowMotionVideoBehavior x:Name="slowmotionBehavior" SlowMoStartTime="00:00:01" Duration ="00:00:30" /> </ic:Interaction.Behaviors> </MediaElement>

25. Demo: Using Media Behaviors

26. Questions?

27. Contact Info Blog and Code http://weblogs.asp.net/dwahlin Twitter @DanWahlin Email: dwahlin@theWahlinGroup.com