Monitoring, Modelling, and Predicting with Real-Time Control

1. Monitoring, Modelling, and Predicting with Real-Time Control Dr Ian Oppermann Director, CSIRO ICT Centre

2. Understanding the world … in real time It is so natural for us. Nothing is “natural” in Robotics How do we do it in “real time”? We focus on four components. Sensing Perception, Control Actuation CSIRO.Monitoring, Modelling, and Predicting with Real-Time Control

3. Understanding the world : A Robot’s Viewpoint Real time operations performed, from sensing to actuation, within the timeframe required for effective operation in a dynamic environment CSIRO.Monitoring, Modelling, and Predicting with Real-Time Control

4. Globally Consistent Real-time Perception Globally Consistent Model (map, trajectories etc.) SLAM Sensing (Lidar. Radar, Stereo) Semantics (Path, Obstacles) Long Term Mapping The difference between Localization and MOT is the difference between recording the pose of ourselves versus recording the pose of somebody else The difference between MOT and CD is in the time scale and whether the object is transitory or permanent CSIRO.Monitoring, Modelling, and Predicting with Real-Time Control

5. Mobile Mapping: What’s available now? Big, Bulky and Expensive • Rooftop-mounted sensors • 2D lidars with high-end GPS/INS • Cost: $ X00,000 Limited Environments • Reliance on GPS which is challenged in urban canyons, underground, near large infrastructure, forests, mines Not Real Time • Processing straightforward if accurate position is known at all times but NOT REAL TIME CSIRO.Monitoring, Modelling, and Predicting with Real-Time Control

6. What problems to address? • Cost • move from expensive hardware to software • Accuracy • improve location estimation • loop closure (knowing where you have been) • A dynamic environment • Long term versus short term changes in environment • Speed • Real time means being able to sense, aggregate, decide or re-plan in the time frames affected by limits of safety, fuel, task completion constraints CSIRO.Monitoring, Modelling, and Predicting with Real-Time Control

7. Mapping and Location Improving accuracy, reducing cost • No GPS • No odometry • No encoders • No IMU (inertial measurement Unit) Data Association Robust Optimization 3D Map created with spinning 2D LIDAR on Bobcat With Scan Matching (without any additional sensors) CSIRO.Monitoring, Modelling, and Predicting with Real-Time Control

8. Loop Closure – Improving accuracy Data Association Robust Optimization 2D LIDAR on moving 4WD With Place Recognition (loop closure) CSIRO.Monitoring, Modelling, and Predicting with Real-Time Control

9. Loop Closure Reliability and Accuracy over long times and long distances 2 Lasers No GPS No odometry No encoders No IMU No Calibration CSIRO.Monitoring, Modelling, and Predicting with Real-Time Control

10. Object Tracking Long term Tracking Under different conditions Passive actuation CSIRO.Monitoring, Modelling, and Predicting with Real-Time Control

11. Example : Mapping Jenolan Caves CSIRO.Monitoring, Modelling, and Predicting with Real-Time Control

12. Example : Mapping Jenolan Caves Sample Point Cloud CSIRO.Monitoring, Modelling, and Predicting with Real-Time Control

13. Example : Mapping Jenolan Caves Watertight Surface Generation CSIRO.Monitoring, Modelling, and Predicting with Real-Time Control

14. Example : Mapping Jenolan Caves Chifley Cave Surface Model CSIRO.Monitoring, Modelling, and Predicting with Real-Time Control

15. Example : Mapping Jenolan Caves Registered Point Cloud CSIRO.Monitoring, Modelling, and Predicting with Real-Time Control

16. Example : Mapping Jenolan Caves Point Cloud Overlay CSIRO.Monitoring, Modelling, and Predicting with Real-Time Control

17. Example: Bringing it all together CSIRO.Monitoring, Modelling, and Predicting with Real-Time Control

18. Example : Airborne Terrain Mapping and Static Object Avoidance CSIRO.Monitoring, Modelling, and Predicting with Real-Time Control

19. Assistive Reactive Augmented Reality Local Tele-Operation Manual Virtual Real Autonomy Robot Supervisory Proactive Global Augmented Virtuality Shared Autonomy Mixed-Reality Tele-Robotic Communications Latency Intelligent Behavior Extent of Knowledge The Future?Connecting the Virtual to the Real (Robot) User Interface Machine Autonomy CSIRO.Monitoring, Modelling, and Predicting with Real-Time Control

20. The Future? Convergence of Technology Visualization (Advanced Graphics) Control Training Simulation (Physics Engines) Realistic Interaction with Environment Realistic Visualization of Environment Gaming (Human Factors) Realistic Simulation of Environment. Planning CSIRO.Monitoring, Modelling, and Predicting with Real-Time Control

21. Simulation and Computation Spectrum Multiple Scenarios Off-line Analysis CSIRO.Monitoring, Modelling, and Predicting with Real-Time Control

22. MBWA WiMAX Backhaul Network WiFi/MiMo Mesh Access Network Peer-to-Peer Peer-to-Peer Whole of system view CSIRO.Monitoring, Modelling, and Predicting with Real-Time Control

23. The Challenge: Unified User Interface Command & Control Whole of Mine Planning Tele - Robotic SCADA by Paul Bourke CSIRO.Monitoring, Modelling, and Predicting with Real-Time Control

24. Summary • Automation and Teleoperation in Realtime Environments require: • Knowing state of environment [Mapping] • Knowing where you are [Localization] • SLAM provides: • Localization in GPS denied areas • Mapping over large scales • Independent real-time sensor • Integration with other sensors (GPS,IMU etc) • Future Realtime Automation will require: • Pervasive Tracking • SLAM-MOT (SLAM with Moving Object Tracking) • Collaborative / Distributed real time mapping • Deal with uncertainty, latency, trust and different sensing modalities • Life long mapping and long term map management • Scalable (semantic mapping) • Maps get better with age, rather than “blurry CSIRO.Monitoring, Modelling, and Predicting with Real-Time Control

25. CSIRO – more than 80 years of achievements CSIRO.Monitoring, Modelling, and Predicting with Real-Time Control

26. Thank you Contact Us Phone: 1300 363 400 or +61 3 9545 2176 Email: enquiries@csiro.au Web: www.csiro.au For further information: Dr Ian Oppermann Director, CSIRO ICT Centre Email: ian.oppermann@csiro.au Web: www.csiro.au/ict With Thanks to Dr Rob Zlot Dr Mike Bosse Dr Elliot Duff Dr Jonathan Roberts