Combining Media

1. 8 Combining Media

2. Key Points • There are two models for combining elements of different media types: page-based and synchronization-based • Hypermedia is a page-based model that generalizes hypertext to include other media types. The World Wide Web is an example of a hypermedia system.

3. Key Points • Helper applications are used by Web browsers to display media types they cannot handle themselves. Plug-ins are used to extend the range of types that can be displayed within the browser environment. • The object element is provided for embedding arbitrary media in HTML 4 documents, but img and embed are more commonly used.

4. Introduction • There are two models currently in use for combining elements of different media types • Paged-based • Synchronization-based

5. Paged-based • Paged-based • In a two-dimensional arrangement that resembles the way text and images are laid out in books and magazines. • Time-based elements, such as video clips and sound, are embedded in the page as if they were images, occupying a fixed area; controls may be provided to start and stop playback. • Linking mechanism • Linked paged-based multimedia productions are known as hypermedia. • World Wide Web

6. Synchronization-based • Makes time the central organizing principle • Presented as a sequence, like a slide show • Transitions, such as dissolves and wipes, may be used to go from one element in the sequence to the next. • Parallelism • Several video clips may be shown at the same time, perhaps overlaid against a s6tatic image, or a sound track may play during an animation.

7. Interactive • Scripting: author to write simple programs • Page-based multimedia • Temporal organization can be added • Synchronization presentation • linking

8. Reasons for Using Actions • Reason for associating actions with events • To provide interactivity • Users can control the system’s behavior. • Users can direct the flow of information that they receive. • Events that occur at specific times • Allow time-based behavior and synchronization to be introduced into systems • Actions in timeline • Provide non-linearity

9. Multimedia Scripting • Scripting language • Perform computations on the attributes of objects • Create new objects • Affecting their appearance and behavior • Triggered by event

10. WWW Client-side Scripting • Server-side • Enable an HTTP server to communicate with other resources, such as databases, and incorporate data obtained from them into its response. • Web pages dynamically form time-varying data • Client-side • Appearance and behavior of Web pages • Allowing code has been downloaded from Internet to run on your system. • Scripts cannot • access any local resources such as files • make any network connections • Their interaction with server is limited to requesting new resources and posting information form HTML forms. • Secure but limited • Provide feedback to user

11. HTML and Hypermedia • Link: href attribute • Helper applications • Plug-in • OBJECT: HTML 4.0, preferred way • Netscape: EMBED

12. OBJECT • OBJECT has content. • The content is displayed only user agent is unable to display the object. • Support arbitrary media • PARAM, name and vale • Controller = false, autoplay=true, loop=true

13. Synchronization-based presentations • Pure hypermedia: no temporal structure • Timelines for authoring • Slide show packages • Transitions: wipes, dissolves, ripples, page turns • Timeline-based multimedia • More complex • Macromedia Director • Cast, Cast members, Score • Sprites can be animated using key frames. • Sprite properties can be animated.

14. Limitations • Web browsers are not designed to fully support multimedia features • Limited coordination of media elements

15. The Problem • Lots of Bits • Images, audio and video are beyond Internet design specs • Results in space/ time constraints at: • the server • the network( s) • the client • Not All Bits are Equally Important • Time between samples often more important than bits in sample, • for example lip synchronization (but not always...) • Content may be Distributed Across Network • Need to synchronize presentation • Objectives • Add synchronization to the Web • Allow interoperability • Use declarative format, preferably text — thus XML

16. Solution • Use of a single timeline for all media • Creation of time-based multimedia delivery over the web • Synchronization • Coordination

17. What Is SMIL? • SMIL stands for Synchronized Multimedia Integration Language • SMIL is pronounced "smile" • SMIL is a language for describing audiovisual presentations • SMIL is easy to learn and understand • SMIL is an HTML-like language • SMIL is written in XML • SMIL presentations can be written using a text-editor • SMIL is a W3C standard • SMIL 2.0 August 2001 • http://www.w3.org/TR/smil20/

18. What Is SMIL?

19. ASimplified SMIL Example <smil> <body> <seq repeatCount="indefinite"> <img src="image1.jpg" dur="3s" /> <img src="image2.jpg" dur="3s" /> </seq> </body> </smil>

20. What Can SMIL Do? • SMIL can be used to create Internet or Intranet presentations • SMIL can be used to create slide show presentations • SMIL has been described as the Internet answer to PowerPoint • SMIL presentations can display multiple file types (text, video, audio...) • SMIL presentations can display multiple files at the same time • SMIL presentations can display files from multiple web servers • SMIL presentations can contain links to other SMIL presentations • SMIL presentations can contain control buttons (stop, start, next, ...) • SMIL has functions for defining sequences and duration of elements • SMIL has functions for defining position and visibility of elements

21. SMIL Files • A SMIL file contains all the information necessary to describe a multimedia presentation • SMIL files are stored with the extension *.smil • A SMIL file contains the following: • The layout of the presentation • The timeline of the presentation • The source of all multimedia elements

22. How to Play a SMIL File? • You will need a SMIL player. Different SMIL players can be found • RealOne Platform by RealNetworks has full support for SMIL 2.0 • SMIL Player by InterObject supports the SMIL 2.0 Basic Profile • With Internet Explorer 5.5 or later, SMIL elements can be inserted into HTML files • This way any SMIL presentation can run over the Internet as standard HTML files

23. SMIL in HTML • Adding References • Adding a "time" namespace definition to the <html> tag • Adding an <?import> element to import the "time" namespace • Adding a <style> element to define the class "time" <html xmlns:time="urn:schemas-microsoft-com:time"> <head> <?import namespace="time" implementation="#default#time2"> <style>.time {behavior: url(#default#time2)}</style> </head> <body> <time:seq repeatCount="indefinite"> <img class="time" src="image1.jpg" dur="3s" /> <img class="time" src="image2.jpg" dur="3s" /> </time:seq> </body> </html>

24. SMIL Timing • time formats: • hh:mm:ss.f • number [h|min|s|ms] • wallclock (YYY-MM-DDThh:mm:ss+zone) • Duration • The duration (dur="5s") attribute • When To Start? • The begin (begin="2s") attribute

25. SMIL Sequence • <seq> - the most common SMIL element - defines a sequence • The <seq> element can have a number of attributes: • begin, dur, repeatCount <html xmlns:t="urn:schemas-microsoft-com:time"> <head> <?import namespace="t" implementation="#default#time2"> <style>.t {behavior: url(#default#time2)}</style> </head> <body> <t:seq repeatCount="indefinite"> <h2 class="t" dur="1s"> I will display for one second</h2> <h2 class="t" dur="2s"> I will display for two seconds</h2> <h2 class="t" dur="3s"> I will display for three seconds</h2> </t:seq> </body> </html>

26. SMIL in Parallel • Objects inside the <par> element will be played at the same time • Attributes: begin, dur, endsync= "first"|"last"|id(clip), repeatCount <html xmlns:t="urn:schemas-microsoft-com:time"> <head> <?import namespace="t" implementation="#default#time2"> <style>.t {behavior: url(#default#time2)}</style> </head> <body> <par> <t:audio src="liar.wav" repeatCount="indefinite" type="wav" /> <t:seq repeatCount="indefinite"> <h2 class="t" dur="1s"> I will display for one second</h2> <h2 class="t" dur="2s"> I will display for two seconds</h2> </t:seq> </par> </body> </html>

27. SMIL Transitions • IE 6 supports transitions based on the SMIL 2.0. Transitions are implemented with the element <transitionfilter> • Attributes: type, begin, mode, from, to • Transition Filters • fade, barnDoorWipe, barWipe, clockWipe, ellipseWipe, fanWipe, irisWipe, pushWipe, slideWipe, snakeWipe, spiralWipe, starWipe <t:transitionfilter targetelement="keyb" type="clockWipe" begin="keyb.begin" dur="2s" /> <img id="keyb" class="t" src="pic_keyb.jpg" dur="4s" width="128" height="107" />

28. SMIL Media Elements • <animation> <audio> <brush> <img> <param> <ref> <text> <textstream> <video> • Attributes: erase src type <html xmlns:t="urn:schemas-microsoft-com:time"> <head> <?import namespace="t" implementation="#default#time2"> </head> <body> <t:video src="http://www.ananova.com/about/vap_windows_check.wmv" repeatCount="indefinite" type="wmv" /> </body> </html>

29. Reading List • Check:http://www.ccse.kfupm.edu.sa/sukairi/swe423/course_resources/8-Combining Media/