a CAPpella

1. Paper Prototype Test • Preferred vertical view, but… Questions Display for sound level Problem: value in line graph goes from left-to-right instead of up-and-down. Solution: implementation is horizontal • Preferred strips version, but cells delineated events Solution: • Horizontal or vertical view? • Visualization: + = • How to use • Start system and demonstrate. • Stop system; system stores events in logs. • Use interface to view and select relevant events. • Feed relevant events to learning algorithm. • Go back to step 1 or 3. • No preference for transitions or intervals Playerpanel Event views Results Classificationtable • Select transitions or intervals An interval A transition Actionview Boolean-valued events Integer-valued events Events w/ double-precision values Events w/ multiple points RFID events Actions Interactive Use Feature Selection Train Classify Interactive Use Feature Selection Train Classify Feature Selection Manual Correction Feedback to Designer cells strips Time series of number of people in a room dots a CAPpella Ian Li CSE, University of Washington SUPERB 2003 Advised by Prof. Anind Dey Acknowledgements • Prof. Anind Dey, Jason, Raffay, Holly, Gary, Scott K., Jimmy, Prof. Jen Mankoff, Bola, and SUPERB Program Prototyping Context-Aware Applications by Demonstration a CAPpella Building Context-Aware Applications • Tool for building context-aware applications by demonstration • Uses IML model • By writing code • Context Fabric (Hong et. al., 2001) Context Toolkit (Dey, 1999) • For developers, not for non-programmers User Interface • Viewing logs of events • By visual programming • iCAP (interactive Context-Aware Application Prototyper) (Sohn & Dey, 2003) • Visual language to create rules • Provides playback of raw data (e.g., video) from demonstration • Selecting relevant events Selecting individual events Selecting multiple events Programming-by-Demonstration • Allows global and detailed view of events with zoom slider • Gives feedback about the classifications of a demonstration • Allow users to “act out” what the context-aware application should do • Example: programming a context-aware application to turn on the lights in a room Future Work • Evaluate user interface • Integrate user interface and machine-learning system • Consider order of events • Ex.: Door bell rings must come before person coming in • Improve visualization of classification (color code?) Machine Learning • Interactive machine learning (IML) model • Classical machine-learning (CML) model • Go inside room. • Turn on the lights. • Repeat several times, until the application learns that every time you go inside the room, it should turn on the lights. (Fails & Olson, 2003) • Two things to do this: • Machine-learning system • User interface design • Learning algorithm does both feature selection and learning • User corrects mistakes of learning algorithm • Assumes training is done once, so it is slow • Selecting features is hard