Outline

1. Technologies for Mobile ManufacturingSanjiv Singh/Reid SimmonsRobotics InstituteCarnegie Mellon UniversityFebruary 2008

2. Outline • Motivation • Objectives • A Simple Example • Autonomous Assembly • Tele-operated Operation • “Sliding” Autonomy • More complicated examples • Key Technologies • Relative Position Estimation • Coordinated Control of Mobile Manipulation • Task Control Architecture

3. Terrestrial Construction • Many different tasks • Complimentary entities • Big plan that is constantly refined

4. Objectives • Enable heterogeneous multiple robots to coordinate complex assembly tasks • Emphasis on tasks that can not be done by single robots • Enable flexible human-robot interaction during assembly • Deal with unanticipated contingencies • Reduce need for operator Candidate Tasks: • Assemble multi-element, compliant structure • Brace structure for strength • Cable structure

5. Planning Planning Executive Executive Behavioral Control Behavioral Control Previous Work: Distributed Architectures Planning Executive Behavioral Control

6. Execute Sequentially Execute Sequentially Execute Concurrently Previous Work: Multi-Robot Synchronization • Enable agents to allocate and synchronize tasks; detect and handle each others exceptions Task A Task C Robot 1 Task B Robot 3 Robot 2

7. Coordinated Assembly • Three heterogeneous robots • Crane has large workspace, high weight capacity • Manipulator has fine control • Roving eye provides high degree of resolution • Independent robot operation without accurate inter-robot calibration

8. Multi-Robot Testbed

9. A Simple Example: Fully Autonomous Dock a single beam into two upright connectors with mm tolerance

10. Dock beam in near stanchion Push beam over far stanchion Grasp beam Dock beam Contact beam Push beam Combined State-Machine for Dual-End Dock Lower beam Align beam over near stanchion Turn far end into view Align beam over far stanchion Lower beam into far stanchion CRANE Align MM at near stanchion Stow arm Align MM at far stanchion Stow arm Turn 180˚ MOBILE MANIPULATOR Watch crane Watch MM Watch dock Move away From MM Move to far stanchion Watch MM Watch crane Watch push ROVING EYE *Sequential connections for watch tasks not shown for clarity.

11. First Results

12. Failures • First dock succeeds 70% of the time • Complete second dock succeed only 6% of the time • 20% of the time, initial conditions are not set • Common errors: • Mobile Manipulator might over or underturn • Beam gets caught on groove • Arm gets caught on beam • Fiducials are blocked • Actuator deadband causes infinite loop • Visual servoing fails • Crane actuator slip causes offset error

13. Mature Autonomous System • Setup • Completely autonomous • 50 trials • Typical Failures • Electrical failure on MM • Software crash • Near collision due to errors in visual perception • Obscured fiducial • MM lost grip on beam • Assembled portion broke apart • Speed • µ=9.9min, =1.6min

14. Teleoperated System • Setup • 50 trials (total) with four robot-experienced users • Several robot-specific GUIs • Teleoperation using visual feedback from Roving Eye • Typical Failures • Visual feedback created “tunnel” vision • Stereo vision did not provide users with good depth perception • Experienced one network and one electrical failure • Speed • µ=12.5min, =4.0min

15. Sliding Autonomy: Adding an Operator • Fully autonomous operation has many failure modes • Not enough bang for the buck to automate some operations • Would like seamless method to switch between operator and system • Operator should be able to take over either control or monitoring of task • Three modes of human interaction: • Pre-assigned task • Intervention • Exception Handling

16. Sliding Tasks • Mobile Manipulator • First dock • Turn • Second dock* • Roving Eye • Visual search • Turn • Crane • Second dock*

17. Results w. Sliding Autonomy • Setup • Several task-specific GUIs • Limited adjustable tasks • Feedback available from any autonomous task • 50 trials • Results • Discretionary-Intervention Successes • Mandatory-Intervention Successes • Failures due to damaged hardware & network failure

18. Outline • Motivation • Objectives • A Simple Example • Autonomous Assembly • Tele-operated Operation • “Sliding” Autonomy • More complicated examples • Key Technologies • Relative Position Estimation • Coordinated Control of Mobile Manipulation • Task Control Architecture

19. More complicated example #1

20. Modes • Teleoperated: User controls each robot in turn through keyboard and mouse • Autonomous: Hit start and step back • System Initiative: System asks for help when needed • Mixed Initiative: System initiative + Operator intervention

21. Modes • Teleoperated: User controls each robot in turn through keyboard and mouse • Autonomous: Hit start and step back • System Initiative: System asks for help when needed • Mixed Initiative: System initiative + Operator intervention

22. 732 [227] 94% (16) 52 [16] 500[182] 100% (16) 27 [21] Tele-Op System Initiative 516[125] 89% (35) 0 529 [148] 94% (16) 29 [13] Mixed Autonomous

23. More complicated example # 2 • Extended scenario involves planning because • constraints make it difficult to script a plan • Recovery from failure might require many steps

24. University of Maryland Space Systems LabNeutral Buoyancy Tank Ranger EASE Truss Assembly Space Shuttle Cargo Bay

25. “Roving Eye” “Crane” “Mobile Manipulator” internet Ranger Arms at U Maryland Operator at CMU Trestle - U Maryland SSL Cooperation

26. Outline • Motivation • Objectives • A Simple Example • Autonomous Assembly • Tele-operated Operation • “Sliding” Autonomy • More complicated examples • Key Technologies • Relative Position Estimation • Coordinated Control of Mobile Manipulation • Task Control Architecture

27. Sensing Location • Need to localize parts wrt to robot during operation so robot can plan motion and adapt to any variations in starting conditions & performance • Complicated because the robot base is not stationary • Method 1: No global reference frame. Relative position (between parts & between robot and part) is determined via fiducials • Advantages: flexible, low infrastructure • Disadvantages: accuracy can be low unless high fiducials are sensed with high resolution, computationally expensive • Method 2: Establish global reference frame. Parts and Robots are all registered in common frame. • Advantages: high accuracy, low computation requirements • Disadvantages: high infrastructure costs, must guarantee line of sight from fixed infrastructure

28. Sensing Relative Position • Visual Fiducial allows determination of ID & 6 DOF displacement between camera and fiducial. • Fiducials have some redundancy, can work even if the fiducial is partly obscured. • Main computational expense is in detecting fiducial in the scene. • Accuracy increases as camera gets closer to fiducial

29. Tracking Fiducials (with occlusion)

30. Other kinds of Fiducials • Active Fiducials can be used.

31. Sensing in a Global Reference Frame • Transmitters fixed to infrastructure • Receivers on items that move • Requires synchronization between receivers http://www.indoorgps.com/

32. Mobile Manipulation • Want to place the robot end effector accurately in a large workspace. Could do this by coupling manipulator & mobile base. • Coordination of base and arm motions of Mobile manipulators is complicated because of redundant degrees of freedom degrees of freedom. • Further considerations: • Want to keep the arm from getting close to singularities. • Want to control end-effector but want to ensure that the base meets the constraints. • Arm and base have very different response

33. Mobile Manipulation Resolved motion control withArm motion only-- SMALLER WORKSPACE Resolved motion control with Coordinated Arm and Base-- LARGER WORKSPACE

34. Implementation on CMU MM

35. Projecting into the Null Space (Example1)

36. Projecting into the Null Space (Example2)

39. Offline Planning to decouple Base & Arm Motion Each grid cell gets a score based on how much of the path and how well the it can be covered with the base at that point.

40. Seam Following Motion sped up by 4x