Technology for the Aging A Collaborative Effort

1. UMassAmherst Technology for the AgingA Collaborative Effort Phebe Sessions Julie Abramson Mary Olson Edward Riseman Allen Hanson Roderic Grupen Erik Learned-Miller MERL Candy Sidner Supported by the National Science Foundation

2. Collaborative Effort • Collaborative Research Effort • Social scientists and computer scientists • Elderly representatives and their stakeholders • caregivers • family • local and state agencies, etc. • Two Part Talk: Social Science and Technology

3. Enhancing Control and Empowerment for the Elderly through Assistive Technology Dr. Phebe Sessions Smith College School for Social Work Social Science Components November 4, 2005 Washington, D.C.

4. Outline • Issues in collaboration • Theoretical framework: ecosystemic and social constructionist • Gerontological literature and traditions • Research plan

5. Issues in Collaboration UMass Computer Sciences/Smith SSW Why social work? • Pragmatism • Commitment to underserved populations • Community-building skills Challenges of collaboration • Novelty and not knowing • Different languages

6. Theoretical framework • Ecosystemic and Social Constructionist • Strengths perspective • Collaboration, dialogue • Qualitative methods in research

7. Gerontological literature and traditions • Aging and the deficit perspective: “declinism” • Positive aging and core significance of control • Learning issues and aging • Social networks and aging • Ethical considerations

8. Research plan • Qualitative methods • Year One: Focus groups with elderly, families, and service providers • Year Two: Implementation of technology in common room • Year Three: Revision of technology based on Year Two findings

9. Technology Components • Umass Vision and Robotics groups • Professor Emeritus Edward Riseman (PI) • Professor Allen Hanson (co-PI) • Professor Roderic Grupen (co-PI) • Mitsubishi Electronic Research Laboratory • Dr. Candace Sidner

10. Assisted-Living Clientele Speech Synthesis Medical Structure Interface Tracking Staff Professionals Graphics Scripts Vision Speech Human Services Interface Speech Understanding Activity Modeling Legal Conversational Interaction Lifestyle/Event Coordination Care-Givers Family Family/Community Interface Technology Overview SMITH ISSUES: privacy access client control MERL UMASS

11. Interaction: Appointment Management IA: Hi, Maisie. Senior: Hi. IA: later this morning at 11 am you have a dentist appt. Senior: Ok IA: You will have to take the bus so you need to leave here an hour before and walk to the bus stop. That means leaving about 10 am. Senior: I want to leave later. IA: Well, the bus takes 20 minutes. How long do you need to get to the bus stop? Senior: 15 minutes IA: The bus comes every 15 minutes, so when do you think you should leave? Senior: 10:15 IA: Okay. How about I remind you at 10 am to get ready? Senior: Good. IA: Good bye for now.

12. Technical Goals • An effective API to the client’s environment: • Independent Living • Social/family • Medical • Financial • Social services • Legal • ….. • Plug and play for rapid development and deployment • Interface driven user interactions • System interfaces • Nimble, reconfigurable, robust, minimally intrusive, controllable

13. The Client-Care Sandbox Nimble & Reconfigurable

14. Component Technologies • Fault tolerant script based interfaces: Collagen plus GUI • Minimal intrusion • Robust • Speech and speech understanding • Command and control • Voice Analysis - e.g. stress • Visual interfaces • Virtual Reality • Tracking • Change Detection • Environment • Object locations • Motion/expected tracks, deviation from normal activity • People • Gait and postural models • Speech patterns

15. Collagen: Collaborative Agents • Principals: Candace L. Sidner and Charles Rich • Java middleware for collaborative interface agents • User and agent collaborate to complete predefined tasks • Speech or text-based discourse with agent • mouse, touch screen, others • Provides friendlier, more familiar interface for inexperienced users • Many existing applications • http://www.merl.com/projects/collagen Mitsubishi Electric Research Laboratory

16. * focus stack plan tree Collagen communicate observe observe interact interact Collaborative Interface Agent * SharedPlans per Grosz, Sidner, Lochbaum, Kraus, et al.

17. Collagen Example: Calendar Application Dialog Fragment • User: “Let’s create a new appointment.” • Agent: activates Create Appointment window. • User: enters the appointment’s date and description. • User: “What’s Next?” • Agent: “What time is the appointment?” • User: “11 AM.” • Agent: sets the time of the appointment to 11 AM. • Agent: “I’m going to save this appointment.” • User: “OK.” • Agent: “We have created a new appointment!”

18. Speech Recognition • Available engines • IBM Viavoice • Microsoft Speech SDK • CMU Sphinx Engine 2 and 4 • Mode of application • Continuous speech recognition mode • Speaker dependent • Low accuracy • Voice commands mode • Speaker independent • Higher Accuracy • Still susceptible to noise

19. Managing Speech Recognition Technology • Reduce noise • Pressure-gradient/Noise-canceling Microphone • Sound card/USB pod • Continuous recognition • Training is essential • Do not expect high accuracy • Approach • Design dialog script carefully • Build in a fault tolerant mechanism • Always have a “take me to an operator” option • Constrained uses: • voice command mode • voice stress analysis

20. Local Experimental Environment • “Smart” Room • Distributed sensor network • Human and robotic residents • Complex environment : camera locations

21. Tracking • Distributed sensor network • Sensor management • Selection, reasoning across sensors • Real-Time fault-tolerant approach • Fault containment units • Hardware, Software • Hierarchical • Wrappers around a resource policy • Augmented by fault and context reporting mechanisms • Dynamic restructuring • Supports • Activity modeling • Identification • Unusual event detection

22. Tracking Supports Virtual Worlds

23. Immersive Virtual Worlds • Provide several levels of interaction styles • Client control over interaction style and level of access of remote visitors (privacy) • Avatar plus virtual world • Avatar plus real world • Person plus virtual world • Person plus real world (Registered) Transparent Objects

24. Rapid Viewpoint Changes • Camera context switches e.g. surveillance • Disconcerting, difficult for user “Snap” Transitions “VR Smoothed” Transitions

25. Mixing Virtual and Real Worlds • Events in real world mapped to virtual world • Temporal events synchronized to virtual world • Transparency - look through objects • Form of access control

26. Conclusions • Starting up • Conjectures on the future • Empowering the elderly: they decide • Social etiquette enforced through virtual worlds • Client controls level of access and degree of privacy • Built on • Modern social science theory • Existing technology base • Client-care Sandbox model for rapid prototyping • Particulars determined through focus groups