Truong and Abowd, Georgia Institute of Technology, 2004

1. INCA: A Software Infrastructure to Facilitate the Construction and Evolution of Ubiquitous Capture and Access Applications Truong and Abowd, Georgia Institute of Technology, 2004

2. Goal • To build a toolkit to facilitate creation of capture and access applications -> rapid prototyping

3. Method • Identify a well defined class of applications to support • Identify relevant design abstractions for these applications • Develop an infrastructure according to these abstractions • Implement this infrastructure • Develop ‘interesting and complex’ applications to validate infrastructure

4. Motivation • People have bad memories • Manual capture and access are incomplete and inconvenient • Current research cannot leverage past work because lack of generalized platform • Also hinders evaluation and iteration process

5. Related Work in Several Domains • Classroom • eClass, Cornell Lecture Browser, MANIC, AutoAuditorium, STREAMS, Authoring on the Fly, Audio Notebook, StuPad, DEBBIE • Meetings • DOLPHIN, TeamSpace, Tivoli, Dynomite, FiloChat • Other • Forget-Me-Not, Conference Assistant, Comic Diary

6. Lessions Learned from eClass • Need for flexibility (esp. in information storage and access methods) • System must be able to evolve • Temporal vs. Functional division of components

7. INCA (finally) • Supports Key Functionalities • Capture data • Storage • Transduce (data format/type conversion) • Access to multiple, related, or integrated streams of info via context-based queries • Supports multiple instances of each functionality

8. Capture and Tagging • Data represented as raw bytes with tagged attributes (metadata) • Capture device advertises availability of data (?) • Tagger object adds metadata to data from capture device

9. Storage • Each StorageModule stores one type of data • Specifies list of attributes of data it’s interested in • Listens for data availability (capture function calls store callback) • Announces to other components what type of data it houses

10. Storage (cont) • Repository is the generalized StorageModule • Is extendable

11. Access • Components can subscribe to captured data • Capture function does ‘handle’ callback to actually get data • Request creates context-based query

12. Transducing (File Conversion) • Module advertises what kinds of file formats it accepts and what it converts to • Automatically run by the runtime system • Typical conversions: text to speech, video to image frames (and vice versa)

13. Other Goodies • Attribute-triggered automated garbage collection • ObserveModule and ControlModule • Library of reusable components to support capture of audio, video, web visits and ink • Network layer that handles synchronous and asynchronous events • Registry maintains list of components

14. Diagram of layers

15. INCA in Action! • Used to create application for classroom system • CaptureModules: BoardSurface, WaveCapturer, WebMemex, eScanner • StorageModule: Repository • Uses ControlModule and ObserverModule to determine available capture services • AccessModule: (instantiated by JSP) AudioPlayer, NotesPlayer, ExtendedSurface • Also created a system for collection and analysis of behavioral data of children with autism

16. Evaluation (?) • Many other applications built using INCA, at Georgia Tech and Universidade de Sao Paulo

17. Questions • In terms of evaluation, is it good enough to know that INCA was used by others to build applications? What about ease of use? Performance issues? • Can a prototyping tool that involves end-users (like a CAPpella) or non-CS people (like Topiary) be created to achieve the same purpose?

18. Sidenote on Autism • “Although autism is defined by a certain set of behaviors, children and adults can exhibit any combination of the behaviors in any degree of severity. Two children, both with the same diagnosis, can act very differently from one another and have varying skills.” • “…whatever the diagnosis, children with autism can learn and function productively and show gains with appropriate education and treatment.” http://www.autism-society.org/

19. Designing Capture Applications to Support the Education of Children with Autism Hayes, Kientz, Truong, White, Abowd, Pering, Georgia Institute of Technology, 2004

20. Overview • Caregivers and educators of Autistic children record data to evaluate effectiveness of therapy • Introduced 3 prototypes for capture and access • Found that end-user iteration was key to capture and access applications for this particular domain

21. Three Prototypes • Walden Monitor: caregiver wears a camera and records notes on a TabletPC • Abaris: sensors built into the environment, notes taken in a form on a TabletPC, data is synchronized (as best as it can) • CareLog: child wears a personal server, and any caregiver present can record notes (stored on PS) using any wireless enabled device

22. Constraints (Social Concerns) • Support the ‘Care Cycle’? • Allow users to control amount of data and type of data recorded while allowing them to capture rich data? (not get bogged down with data) • Effort required to use system (inconvenience) • Privacy and control of data • Cost

23. Evaluation Metrics (Technical Concerns) • Integration of manually and automatically captured data • Level of distribution • Data analysis and visualization

24. Care Cycle • Diagnosis based on observation • Goal setting • Learning and behavior modification • Evaluation of progress based on observation • (start over again)

25. Balancing Need for Rich Data vs. Capturing Too Much Data • Need to pair narrative with discrete data • Difficulty parsing narratives • Difficulty retrieving and analyzing rich data (which supplies the narrative)

26. Reducing Inconvenience • Technology should not distract caregivers from their primary jobs – caring for and supporting a child with autism

27. Privacy and Control of Data • Schools don’t want to be liable • Video requires parental consent • Integrated classrooms – CWA students with regular students

28. Cost • Taking care of a CWA is costly already

29. Integration of data • Integration and synchronization of data streams is important

30. Distribution • Storage important for privacy concerns • Also has repercussions on cost • Distribution means flexibility in interfaces (capture devices and analysis tools)

31. Data Analysis • Use the data to inform decisions on therapy • Want to find trends

32. Conclusion • End-user Iteration is key • Users should be able to iteratively configure their systems to balance these constraints • Distributed modular systems are best suited for iteration

33. Questions • Did they adequately discuss the constraints and how each prototype fulfilled/violated them? (ex. Privacy issues – who’s privacy should the application protect? How will applications deal with control of data?) • Are there other constraints that they didn’t consider? • Are there other flaws with the prototypes that the writers did not consider?