1. Context-Aware Computing John Canny HCC Retreat 7/5/00

2. Consequences of Ubiquitous Computing and Calm Technology: • We want to have constellations of devices working for us, somehow inferring and supporting our activities.

3. Mining Tacit knowledge from activity: • Activity mining involves content and context: the inter-relatedness of people and information objects.

4. ABC: Activity-Based Computing • Activities are clusters with: • Users • Documents • Tools • Realizations can be • GUIs • speech, etc. • Nearness encodes awareness.

5. ABC: Inspiration • Based on “activity theory” from psychology (Vygotsky, Leont’ev, Engestrom, Kutti). • Activities are high-level behaviors directed toward some end, usually with others. • Recent studies (Gruen ‘96) support the value of the activity model, but: • activities are usually not distinguishable instantaneously, they interleave and overlap. • Activities can persist over different time frames and occur in different places.

6. ABC: What is it good for? • Streamlining interaction: disambiguation, menu customization. • Prefetching files, prestarting devices. • Awareness (who else is working on the project now) • Attention management - graded awareness of other activities. • Pro-active sharing in group work. • Infering document utility. • Human expertise location. • Re-contextualizing documents: authorship, roles, backgrounds, discipline-specific vocabulary...

7. ABC: A lo-fi prototype (Danyel Fisher)

8. ABC: Contextual data sources • Who: • Direct communication: 1-1 email, phone, F2F. • What: • Topical discussions, forums, F2F meetings. • Document writing, reading, search, markup. • When: • The current time; time windows for activities. • Where: • A “place” which has meaning for the users activities.

9. ABC: Representation User 1 Mail User1 Read Document5 Start User 2 Project Device7 Write User 3 Program12 Document3 Markup Algorithms are SVD and other pattern analysis schemes

10. Knowledgescapes (Heyning Cheng) • Knowledgescapes is a search engine that uses activity logs only - it doesn’t look at document content:http://indios.cs.berkeley.edu/knowledgescapes.html • Activity data is used in knowledgescapes to infer document relevance/quality. Results: • Performance is fair with text queries • Performance is very good with related document queries • In progress: Infer user expertise from document selections.

11. Knowledgescapes prototype Search Engine Query(terms) Document1 Document2 Information Need Document3 Document4 Rankings from reading time. Treat as probability of interest or E(interest)

12. Another inspiration: LSA Word 1 Passage 1 Passage 2 Word 2 Passage 3 Word 3 Passage 4 This structure is resolved by SVD into latent semanticcategories which better model the document content.

13. Another inspiration: LSA Word 1 Passage 1 Concept 1 Passage 2 Word 2 Concept 2 Passage 3 Concept 3 Passage 4 Word 3 Decomposition of the linear may from words to passages into two linear maps, with latent concepts in between.

14. Content and context mining • Many successful content analysis schemes are based on LSA (Latent Semantic Analysis). • Widely used “context” analysis schemes (from social network analysis) use similar algorithms. • We are developing a latent-variable method which combines evidence about content and context to model activities. We will use it for: • Document prefetching for unseen documents • Document authority/quality estimates • Naming activities • Building personalized thesauri

15. Context and content analysis User 1 Term 1 Read Document1 User 2 Write Term 2 Document5 User 3 Term 3 Document3 Markup Use document access data and LSA andFactor both maps into latent categories

16. Sensors for the HCC lab • Sensor networks: • Microphone arrays • Camera arrays • Small scanners • Output • Wall projectors, PDAs, the UPM • Networking using USB and IrDA • Sensor proxies for outside access via Jini or HTTP