Migration from Teleoperation to Autonomy via Modular Sensor Bricks

1. Migration from Teleoperation to Autonomy via Modular Sensor BricksJuly 12, 2005 Roselyne Barreto Imaging, Robotics, and Intelligent Systems Laboratory Department of Electrical and Computer Engineering, The University of Tennessee, Knoxville, TN.

2. Outline • Presentation of Thesis Concept • Motivation • Scenarios • Modular Autonomy • Mobility brick • New ANDROS Mapping (Mobility Brick 2) • JAUS compliant ANDROS • Summary and Conclusions

3. Original OCU Robot http://www.irobot.com/governmentindustrial/ http://www.remotec-andros.com/ http://www.foster-miller.com/t_robotics.htm Basic Teleoperation • Teleoperated robots: • ANDROS • PackBot • Talan • SafeBot • Remote Rider (segway)

4. Basic Teleoperation • Most of those robots are designed for hazardous mission • They are teleoperated because human supervision is necessary in many delicate operations • They can include a camera of a GPS sensor or another sensor occasionally but it all has to be physically attached to the robot; once attached the operation of the “weapon” has to be simple )(e.g. “fire weapon 1”) • They can carry at most 2 extra sensors • For practical and commercial reasons most robots do not adapt to sensors that are not built in or easily attached

5. Autonomy • Some sensors are common for autonomous robots • Range sensor • Sonar • Visual sensor • Encoders • Most autonomous robots include one or more of those sensors • The sensors are built in the robots. They are used to navigate or perform specific assembly tasks

6. Motivation: Integration • Robot autonomy is still a huge challenge for researchers • It is still limited to navigation or very simple tasks • For hazardous tasks the operator cannot be put out of the loop • Hence the motivation here is to keep the teleoperation feature of those very robust robots and add some degree of autonomy to simplify the task of the operator and occasionally relieve him of doing simple tasks

7. Motivation: Integration • The system will be composed of three different states • Proprietary teleoperation (from manufacturing company) • Teleoperation from a computer - more freedom to • Connect with sensor bricks during teleoperation • To have simple automated (preprogrammed ) tasks • To remote log in an on-board computer for teleoperation • To allow autonomous function • Not mentioning the reduction in size, weight and user-friendliness • Full or partial autonomy • Hence the goal is to integrate those three states and make the transition between states as easy as possible

8. Motivation: Modular Autonomy • Moreover most robots are limited to a few sensors because they have to be physically connected to those sensors • Sensor bricks do not have to be actually attached to the mobility brick which widens the possibilities of inspection • The bricks can easily be stackable on the body, or be stationed overhead or in the corner of a room… • Most importantly, they are easily interchangeable as robotics is heading towards modularity

9. Scenarios: vehicle inspection Using a mirror on the stick involves a lot of dexterity and still only covers 30 to 40% of the under vehicle surface to be inspected. The operator can still miss some information (thermal, 3D…). Most importantly the operator is exposed to any potential threat during the inspection. The mobility brick can easily scrutinize the entire under carriage. It can carry more than one brick to thoroughly inspect the car and no one is in danger. Scenario: The operator drives the brick to the car and the robot performs an autonomous under vehicle inspection

10. Scenarios: vehicle inspection Scenario: Send a robot to do an autonomous driver interrogation and trunk inspection

11. Scenarios: scouting mission Sending a mobility brick to take a picture of a specific target; the robot needs the visual brick for acquisition purposes and the range brick for navigation purposes http://www.irobot.com/governmentindustrial/

12. Other Applications • Visual Homing: The robot connects to a visual brick and uses it for navigation • Path planning: the robot connect to the range sensor and uses it for navigation • Civil Support Team: After nuclear, chemical and biological accidents they have to send man in heavy equipment for no more than an hour and rotate • A robot can go with several bricks at the time for more than an hour • This preserves the teleoperation but from a computer which allows connection to several sensors at the time • The operator can rely on more than just human vision while collecting data

13. Teleoperation Original OCU Computer Interface Computer Integration Sensor Bricks Autonomy Mobility Bricks Migration from Teleoperation to Autonomy via sensor bricks The general idea is to transform a teleoperated vehicle into a semi-autonomous to autonomous system • Preserve the original control unit • Move to teleoperation from a computer • Move to an improved computer control unit by integrating data from modular sensor bricks • Transform the robot into a mobility brick by adding intelligence (on-board computer)

14. Range Computer Integration Visual Thermal Nuclear Mobility Brick Chemical Modular Autonomy The concept of modular autonomy involves a principal processing unit, modular sensor bricks and one or more mobility bricks • The computer integration includes teleoperation from a computer when needed • When the main control unit is set on autonomy, the mobility brick takes over and uses the sensor bricks to “see” or perceive its environment • The main control unit has the authority to stop any operation at any time

15. Modular Autonomy

16. Modular Autonomy

17. Communication Pre-processing Sensor Communication Pre-processing Mobility Power Power Sensor/Mobility Brick • The sensor brick concept developed in the IRIS lab includes 4 blocks, the communication, the pre-processing, the acquisition and the power blocks • The acquisition block is the sensor itself (range, nuclear, thermal…) • Similarly we define a mobility brick as consisting in communication, pre-processing, power unit with a mobility platform instead of the sensor • The mobility platform is a basic motorized vehicle which can be teleoperated or part of the autonomous brick • Can be seen as the primitive driver for JAUS Sensor Brick Mobility Brick

18. 802.11 RS 232 802.11 RS 232 Communi- cation Pre- processing Mobility Main Control Unit Power Mobility Brick To transition from the Remotec ANDROS to a mobility brick • Map the original control unit and incorporate the captured signals into a computer program • Connect a computer the robot • Eliminate the RS 232 wireless communication • Communicate through 802.11 between the on-board computer and the main station

19. Mobility Brick 1 • I have tested the system on the ANDROS 1 and a tablet PC • Doug helped me design and build a temporary part to physically attach a box containing a laptop to the robot ANDIBOT 1

20. Mobility Brick 1 ANDIBOT 1

21. Mobility Brick 2 • I have mapped and decoded ANDROS 2 • The results are what we expected: • Longer string (more capabilities) • From 21 characters to 35 characters • Faster baud rate (faster robot) • From 1200 to 9600 • Obviously the initial code does not work directly on the new robot, but the changes are less than one could expect

22. Front/Rear tracks Torso Vehicle Drive Mobility Brick 2 • Most function are still aligned • One additional function (wrist pitch) 0A000C2000908D80C0Æññ 0A00182000907F7F3F0000FFFF02CF02þññ Check sum works the same

23. Mobility Brick 2 • Most important changes between the two robots are the length of the strings and the baud rate • Most body function, vehicle drive and settings are controlled by the same characters (same position) • New functions are controlled by the additional characters to the right of the string • However at this point the robot does not respond to the MFC program • It might be that the new robot is trying to identify the origin of the commands (fiber optic cable? Wireless? controller?) and does not recognize the computer

24. JAUS JAUS – Joint Architecture for Unmanned Systems JAUS is a technical architecture that is concerned with the data structure of unmanned systems which are comprised of software elements, the externally visible properties of those elements, and the relationships among them. JAUS Reference Architecture Part 1

25. JAUS JAUS System Topology JAUS Reference Architecture Part 1

26. JAUS • System level: grouping of more subsystems – ensemble of sensor/mobility bricks • Subsystem level: an independent distinct unit – each brick • Node level: assembly of hardware + software units that has a specific complete role within a subsystem – preprocessor • Component level - lowest level: software unit that provides a well-define service or set of services (executable task or process) – wireless system. All the components currently supported by JAUS are already define • Command and control components • Communication components • Platform components • Manipulator components • Environmental sensor components

27. JAUS • Platform components include several components such as Global Pose Sensor, Local Pose Sensor, Velocity State Sensor, Local Path Segment Driver and others • Currently the one platform component we are really interested in is the primitive driver • It performs basic driving and all platform related functions including operation of common platform devices such as the engine and the lights • This is where the mobility brick fits in • Currently the ANDROS robot with a computer with basic vehicle drive and track functions with basic speed and light settings and capabilities to connect to other components can be seen as a primitive driver

28. JAUS defined Messages Primitive Driver Actuator commands JAUS • JAUS is a component based, message passing architecture • To be JAUS compliant the brick has to be connected to and recognize other JAUS compliant components • In other words it has to receive JAUS messages from other components • This is the key to implementing a JAUS compliant mobility brick

29. JAUS System Commander Global Pose Sensor Reflexive Driver Range Sensor Velocity State Sensor JAUS Compliant Mobility Brick Primitive Driver Example of JAUS System: Reflexive Tele-Operation Block Diagram Note: We could use our brick as we want and also be JAUS compliant

30. Summary and Conclusions • Thesis will be the Transition from Teleoperation to Autonomy via Modular Sensor and Mobility Bricks • One mobility brick has been implemented and is being demo in DC • Demo of brick • Demo of automatic trunk inspection • Mapping of new ANDROS is done but the robot is currently not accessible • A brick can be JAUS compliant by including the capability to receive JAUS defined messages from other components

31. Future Work • Fix bug in first GUI and improve macros section for Demos • Investigate the string transmission problem in new robot • Find out how to implement a JAUS compliant system • Design a better Mobility Brick System

32. Thank You! Questions?