390 likes | 397 Vues
This overview discusses the concept of Active Capture and its challenges in design and implementation. It highlights various applications and the need for toolkit-level support in design and implementation. It also introduces a visual language and tools for rapid prototyping and Wizard-of-Oz prototypes.
E N D
ACAL – Active Capture Automation Language Ana Ramírez Advisors: Marc Davis, Jen Mankoff GUIR 25 February 2004 UC Berkeley - Garage Cinema Research - Group for User Interface Research
Overview • What is Active Capture • Challenges in • design • Implementation • Support at toolkit level for • design • implementation
Motivation • Systems that direct human actions • Keep awake system • Sports instruction (golf swing) • Automated health screening • Video Door locks • Interactions that control timing of interaction
Active Capture Direction/ Cinematography Capture Interaction Active Capture Human- Computer Interaction Computer Vision/ Audition Processing
Active Capture Direction/ Cinematography Capture Interaction Active Capture Human- Computer Interaction Computer Vision/ Audition Processing
Active Capture Direction/ Cinematography Capture Interaction Active Capture Human- Computer Interaction Computer Vision/ Audition Processing
Active Capture Direction/ Cinematography Capture Interaction Active Capture Human- Computer Interaction Computer Vision/ Audition Processing
Active Capture Direction/ Cinematography Capture Interaction Active Capture Human- Computer Interaction Computer Vision/ Audition Processing
Implemented Applications See Video at: www.cs.berkeley.edu/~anar/presentations/ImplementedApps.mpg
Head Turn Recognizer • Uses • Gross motion detector • Eye detector • Looks for • No eyes and No motion followed by • Motion followed by • Eyes followed by • No motion • Uses mediation if something goes wrong.
Exercise • Given raw materials • User actions • System actions • Recognizers system uses • Write down a representation of the interaction of the head turn recognizer. • Work in groups of two
Designing Active Capture Applications • Describe path of “righteousness” • Describe what to do if something goes wrong (mediation) • Use good mediation techniques • Progressive assistance • Freshness • Graceful failure
Path of “righteousness” is not obvious Tedious to write Expression of time flow cumbersome Head Turn Application
Challenges • Difficult to represent • Control process with feedback (mediation) • Timing • Strict and non strict ordering
Goals of ACAL • In general • Natural to describe Active Capture applications. • Support mediation strategies • Include time flow primitives • Support brain storming • Visual “language” • Support rapid prototyping • Support for Wizard-of-Oz prototypes • Support implementation • Make path of “righteousness” apparent in code • Be able to prove an implementation will reach the “done” state.
Current Status of ACAL • Visual “Language” • Started with Ka-Ping Yee in Marc Davis’ class on Multimedia Information in Spring 2003 • Toolkit level support • Main focus this semester • Support Wizard-of-Oz protoyping • Future work • Link visual language, toolkit and wizard-of-oz support together. • Future Work
Current Research Areas • Design Guidelines • Jeff Heer, Nathan Good, Ana Ramirez, Marc Davis, Jen Mankoff. “Presiding Over Accidents: System Mediation of Human Action” CHI’04 • Language Support • ACAL • New Application • “Say Cheese”
Visual “Language” • Path of “righteousness” • Observations • Commands • Capture • Time Constraints
Visual “Language” • Path of “righteousness” • Observations • Commands • Capture • Time Constraints
Visual “Language” • Path of “righteousness” • Observations • Commands • Capture • Time Constraints
Visual “Language” • Path of “righteousness” • Observations • Commands • Capture • Time Constraints
Visual “Language” • Path of “righteousness” • Observations • Commands • Capture • Time Constraints
Visual “Language” • Path of “righteousness” • Observations • Commands • Capture • Time Constraints
Visual “Language” • Add mediation for case when actor is looking at camera before turn.
Visual “Language” • Add freshness to mediation
Language Design Process Two key challenges: • Control-oriented vs. time-oriented representation • Absolute vs. relative time relationships
Control vs. Time • State machines and procedural programs describe control flow well • ...but they visualize time poorly • Timeline representation allows concurrency to be fully expressed • ...but decisions and control flow don’t fit easily on a timeline
Control vs. Time • Hybrid visual representation: timelines with flow arrows
Absolute vs. Relative Time • Horizontal scale on timeline implies particular lengths of intervals • Problem: sometimes want ordering; sometimes want specific intervals • Solution: arrangement on timeline yields ordering; min/max specifiers constrain time intervals
Flexibility in Ordering • Each point on a track specifies “true”, “false”, or “don’t care” Example: doesn’t matter whether when waving or speaking begins or ends, as long as both happen at some point within a 5-second period waving speaking < 5 sec
Lessons Learned • Difficult to balance between: • Control-oriented vs. time-oriented representation • Absolute vs. relative time relationships • Difficult to manage complexity • Important to be able to see path of “righteousness” • Easy to get mediation wrong.
Future Work • Visual Language • Better solution to absolute vs. relative time relationships challenge • Better support for mediation strategies • Implementation support • “Say Cheese” • Automated health screening
Questions anar@cs.berkeley.edu
ACAL Design Goals • Natural to describe Active Capture applications • Support key strategies for mediation • Progressive Assistance • Graceful Failure • Freshness • Include time flow primitives • Support brainstorming process
ACAL • Toolkit level support for applications with: • Mediation • Complex timing • Rich media input and output
Future Applications • “Say Cheese” • Folk Computing • Support remote, more frequent medical screening.
Methodology • Theoretical • Active Capture Design Space • Mediation strategies / guidelines • Practical • Reverse engineer implemented applications • Design a new application