CREATING COMMON PRESENCE FOR A MULTIENTITY RESCUE TEAM

1. CREATING COMMON PRESENCE FOR A MULTIENTITY RESCUE TEAM Jussi Suomela, Jari Saarinen and Aarne Halme Helsinki University of Technology Publication of PeLoTe-project Project funded by the European Community under the IST programme Future and Emerging Technologies Jussi Suomela

2. IntroductionPeLoTe = Building Presence through Localization for Hybrid Telematic Systems Scenario • Human and Robotic entities (HE, RE) explore common area • Both provide continuous mapping data from environment • Mapping information is processed to a common presence for both entities • Applications: Rescue, military, planetary, etc. Jussi Suomela

3. Motivation • Multiple entities working in the same environment for a common goal • Entities are autonomous, but different in nature • How to fuse the information from entities in a way that would benefit all entities? •  Common Presence Jussi Suomela

4. Presence - Being where? Different entities have different locations and tasks • Physical Presence – “Sense of being here/there” • Social Presence – “Sense of being together” • Common Presence – “”Sense” of being here/there together” Jussi Suomela

5. Common Presence • Common presence is a dynamic model - shared by diverse entities - that represents knowledge about a working environment, objects in it and their mutual relationships, as well as goals and the status of task. • Common presence is an interface, not only an environmental model • Transferring information between entities • Providing a common space - ”presence”, which all entities can understand Jussi Suomela

6. Common Presence in PeLoTe • Centralized spatial model that includes concepts related to the fire fighting • In PeLoTe all working entities are sharing the same frame of reference • In essence, the common presence in PeLoTe creates and maintains the shared situational awareness Jussi Suomela

7. Operator • Telepresent in the rescue site • Plans and supervises the task • Controls all the entities • Updates the Common Presence model • GUI for mission planning, entity control and telepresence. Verbal communication with human entites PeLoTe GUI Jussi Suomela

8. Human entity • Present and ”locally telepresent” in the rescue site • Explores the site and provides verbal information to the operator • Wearable sensor system provides location and automatic mapping data to the common presence system Human interface with map including the navigated route (left) and real-time scanner data with proximity alarm (right). Jussi Suomela

9. Fitting human in CP • Common presence and the reality have to match  POSITION! • Personal Assistance System • Communication • Display • PeNa for human positioning • inertial navigation • stride length measurement • laser scanner for odometry and SLAM • own US/RF based localization beacons Jussi Suomela

10. Robot entity • Present and ”locally telepresent” in the rescue site • Robot’s presence is the virtual model of the CP • Follows autonomously the planned track • Provides continuous mapping data to the CP • Can carry localization beacons Jussi Suomela

11. Based on Standard Rescue Map (SRM), a 2D polygon map and an object database Future objective: all buildings with straight alarm connection have one Shows the navigable area and important objects Objects in dynamic layers based on type and importance Continuous updating Sensor mapping Verbal comments, video clips Presence model is filtered to entity specific form Zoomable map for humans geometric polygon map for robots Updated CP model provided realtime to the entities Building the presence Jussi Suomela

12. System telematics • In real rescue task the communication is the most critical system • In PeLoTe communication was built on TCP/IP based client server architecture • The core is PeLoTe server located in the operator room • WLAN supported with direct radio modem and walkie talkie communication Pelote communication, TC means the operator and PAS is the personal assistance system meaning the support system of the exploring humans Jussi Suomela

13. PeLoTe experiments • End user demonstrations • Introducing the system to professionals • User tests • -6 teams with system, 6 without • - Evaluation with questionnaires, observations and performance measures Jussi Suomela

14. Results - questionnaire • In all cases the PeLoTe teams reported higher feeling of presence than control teams • Exception was suspension of disbelief (the people trust more to own senses than information provided by system) • SSM = spatial situation model • SoD = Suspension of disbelief • SP_SL = Spatial presence : self localisation • VSI = Visual spatial imagery Jussi Suomela

15. Results - Performance Performance table for control teams • Performance measures are in average much better for PeLoTe teams than for control teams. • Only exception is time, which is due to the different actions they made during mission (such as teleoperating robot through dangerous areas) Performance table for PeLoTe teams Jussi Suomela

16. Results - Observations • People with no – or little background on computers were able to use the system, control robots and successfully execute the mission • Operator was able to track and map the situation, give control paths to robot and human. • PeNa user was able to stay a way from the dangerous areas, find and rescue victims, teleoperate robot locally and follow instructions given by operator. • The stress level of PeLoTe teams were much lower than with traditional teams. • The PeLoTe teams had much better memory of what happened during the mission (situational awareness) Jussi Suomela

17. Conclusions • Common presence is a tool to change information between cooperative human, robotic and possible other entities • Presence provides communication, support and virtual world model for entities exploring in changing and partially unknown environment where conditions may limit the perception • System was demonstrated in a rescue task but can be applied in military, construction etc. • System was evaluated to increase the shared situational awareness • The future work is to continue the theoretical formulation of the common presence Jussi Suomela