Application-Specific Constraints for Adaptive Multimedia Presentation Generation
This paper explores the automatic generation of multimedia presentations using the Cuypers multimedia generation engine. It discusses the significance of both quantitative and qualitative constraints in multimedia design. The work highlights the insufficiency of quantitative constraints alone and suggests the integration of qualitative constraints for more expressive and intuitive high-level specifications. The authors introduce a constraint logic programming approach combining Prolog with constraint solving to effectively manage conflicting requirements. The paper concludes with future directions for enhancing multimedia generation through best-first techniques and knowledge management strategies.
Application-Specific Constraints for Adaptive Multimedia Presentation Generation
E N D
Presentation Transcript
Application-specific constraints for multimedia presentation generation Joost Geurts, Jacco van Ossenbruggen and Lynda Hardman CWI Amsterdam email: Joost.Geurts@cwi.nl
Talk overview • Generating multimedia automatically • Cuypers multimedia generation engine • Multimedia and constraints • Quantitative constraints • Qualitative constraints • Cuypers demo • Conclusion, future directions
Generating adaptive multimedia • Content • Large multimedia database • System profile • PC, PDA, WAP • Network profile • Modem, Gigabit • User profile • Language, Interests, Abilities, Preferences Too costly to author manually
Automatic multimedia generation • Designer does not specify complete presentation……but only specifies requirements • System automatically finds a solution which meets requirements • How should the requirements be specified? • Declarative constraints
Traditional use of constraints • Constraint solving used for problems with: • Many variables • Large domains • Based on domain reduction paradigm • Quantitative constraints • Integer domain • Reduction by arithmetic relations • Greater than (>) • Less than (<) • Equals (=)
Drawbacks of quantitative constraints • Too many (trivial) solutions that differ by: • 1 pixel position, or • 1 milliseconds in timing • Not sufficiently expressive e.g. cannot specify “no overlap” constraint • Too low level e.g. A.X2 B.X1
Solution: qualitative constraints • Example “A not overlap B”, “B after C” • Advantages: • More intuitive • More expressive • Smaller domains • Qualitative solutions translate automatically to lower level quantitative constraints • New problem: What if constraints are insoluble?
Solution: Constraint Logic Programming • Combine Prolog unification and backtracking with constraint solving • Use Prolog rules to generate constraints • Backtrack when constraints are insoluble
Cuypers generation engine • Multiple layers: • Prolog rules to generate constraints • Qualitative constraints translate to quantitative constraints • Solution of both constraints provides sufficient information for final presentation
Cuypers demo: scenario • Client:User is interested in Rembrandt and wants to know about about the “chiaroscuro” technique • Server: Query database • Server: Generate constraints according to: • System profile • User profile • Network profile • Server: Solve constraints / revise constraints • Server: Generate SMIL presentation • Client: Play presentation
Conclusions • Quantitative constraintsare insufficient for automatic multimediapresentation generation. Also need • Qualitative constraintsto allow intuitive and effectivehigh level specification, and • Backtrackingfor revising specific constraintsthat cause the entire set to fail
Future directions • Best-first instead of depth-first • Choose “best” among possible solutions • Needs evaluation criteria • Improve knowledge management • Make design knowledge declarative and explicit • Preserve metadata in final presentation • Use standardized and reusable profiles
Need to make trade-offs • Semantics • Convey message • Aesthetics • Clear / nice layout • Resources • Screen size, bandwidth • Dimension may result in conflicting goals
Quantitative Constraints % csp(+Ids, -Boxes) csp([IdA,IdB],[box(IdA,[x1:AX1, …]), box(IdB,[x1:BX1,…])]) :- % get values maxX(MaxX), maxY(MaxY), height(IdA,HeightA), widtht(IdA,WidthA), … % define domains [AX1,AX2,BX1,BX2]::[0..MaxX], [AY1,AY2,BY1,BY2]::[0..MaxY], % set width & height AX2 – AX1 #= WidthA, AY2 – AY1 #= HeightA, … % constraints AX2 #< BX1, % A left-of B AY1 #= BY1, % A top-align B, … true.
Multimedia and Constraints • Constraint Logic Programming • Domain reduction • Backtracking • Unification (matching rules) • Qualitative Constraints • Non-integer domain • Allen’s 13 temporal interval relations in three dimensions
Qualitative Constraints • Example: • Two images, A,B • A left or right of B • A not above or below B
Qualitative Constraints % csp(+Ids, -Graph) csp([IdA, Idb], [edge(IdA,IdB,x,NoOverlap),…]) :- % define domains NoOverlap :: [b,b-,m,m-], Overlap :: [d,d-,s,s-,f,f-,e], % constraints edge(IdA,IdB,x,NoOverlap), % B not-overlap A edge(IdA,IdB,y,Overlap), % B overlap A true.
Qualitative Constraints • Reasoning • Inverse: edge(A,B,D,Value) <=> inverse(Value,RValue),edge(B,A,D,RValue). • Equality edge(A,B,D,V1), edge(A,B,D,V2) => V1 #= V2 • Transitive edge(A,B,D,VAB), edge(B,C,D,VBC) => tr(VAB,VBC,VAC), % rule generation algorithm edge(A,C,D,VAC). • Translation rules to quantitative domain edge(A,B,D,b) => node(A,D/2,V2), node(B,D/1,V1) V1 #< V2.
Problems in generating multimedia • Text documents are flexible • Add page, scrollbar, • Template models • Wrap text around images • Multimedia documents are less flexible • No pages or scrollbars, no line-breaking or hyphenation • Not based on text-flow • Feedback needed • Linear process model does not work for multimedia
Quantitative Constraints • Example: • Two images, A,B • A left-of B • A top-align B
Cuypers generation engine • Rhetoric/Semantic • Sequence, Example • Communicative devices • Bookshelf, Slideshow • Qualitative Constraints • A before B • Quantitative Constraints • A.X2 < B.X1 • Presentation • SMIL
