“Curious Places”

1. “Curious Places” Kathryn Merrick, Mary Lou Maher Rob Saunders A Room that Adapts using Curiosity and Supervised Learning October, 2007 Key Centre of Design Computing and Cognition, University of Sydney

2. Overview • Adaptable, Intelligent Environments • Curious Supervised Learning • A Curious, Virtual, Sentient Room • Limitations and Future Work

3. Adaptable, Intelligent Environments • The computer for the 21st century • Hundreds of computers per room • Computers come and go • (Weiser, 1991) • Adaptability is important at two levels: • The middleware level • The behaviour level

4. Adaptable Middleware • Resource management and communication • Adaptability has been widely considered at this level • Real time interaction • Presence services • Ad hoc networking Intelligent Room Project Gaia BlipSystems

5. Adaptable Behaviour • Adapting behaviour to human activities • Supervised Learning • The “Neural Network House” • Data mining • Considered in fixed domains • How can we achieve adaptive behaviour in response to changing hardware or software?

6. Adaptability by Curiosity and Learning • Curiosity adapts focus of attention to relevant learning goals • Learning adapts behaviour to fulfil goals • Curious reinforcement learning • Curious supervised learning MySQL Database Agent Projector Rear projection screen Curious Information Display Curious Research Space PC Agent Bluetooth blip nodes

7. Supervised Learning • “Learning from examples” • A supervised learning problem P can be represented formally by: • A set S of sensed states • A set A of actions • A set X of examples Xi = (Si, Ai) • A policy π : S A

8. Complex, Dynamic Environments • Contain multiple learning problems • P = {P1, P2, P3…} • Learning problems in P may change over time • Addition of new problems • Removal of obsolete problems

9. Modelling Curiosity for Supervised Learning • Aim to focus attention on states, actions and examples from a subset of problems • Works by filtering • Identify potential tasks to learn or act upon • Compute curiosity values • Arbitrate on what to filter • High curiosity may trigger learning or action • Low curiosity does not S(t), X(t) Curiosity Observations and events Task Selection Curiosity Value Arbitration X(t) S(t) Learning Action

10. W(t) sensors S(t), X(t) Y(t-1) C M= { Y(t) U π(t) } Y(t) X(t) S(t) π(t-1) L π(t) SL A π(t) A(t) effectors T(t) The Curious Supervised Learning Agent • Past states, examples and actions are stored in an experience trajectory Y • Experiences may influence curiosity

11. A Curious, Virtual, Sentient Room • A university meeting room in Second Life • Seminars and Meetings • Tutorials • Skype-conferencing

12. Virtual Sensors and Effectors • Blip System • Lights • Floor Sensors • SmartBoard and Chairs

13. Meta-Sensors and Meta-Effectors • Agent does not communicate directly with sensors and effectors • Agent has a ‘sensor of sensors’ and an ‘effector of effectors’ • BlipSystem provides an up-to-date list of current sensors and effectors and acts as an intermediary for communication

14. The Curious Room Agent • Computational model of novelty used for curiosity • Table-based supervised learning using associations • Learns accurately but • Unable to generalise

15. Behaviour of the Curious Place • Avatar enters  Lights go on • Avatar sits  SmartBoard on  Lights off

16. Preliminary Evaluation • ~6 repetitions by human controlled avatars required for learning • Can adapt to new devices • Can adapt simple behaviours to form more complex sequences

17. Limitations • Current prototype is proof-of-concept only, no significant empirical results yet • Issue of if/when/how to ‘forget’ behaviours • Is an interface required for manual editing or override of learned behaviours?

18. Future Work • Further work on curiosity models • Design a suite of experiments to test attention focus in • Environments of increasing complexity • Dynamic environments • More complex tasks