Robust Distributed Task Allocation for Autonomous Multi-Agent Teams

1. Robust Distributed Task Allocation for Autonomous Multi-Agent Teams Ph.D. Candidate:SameeraPonda Thesis Committee: Prof. Jonathan P. How, Prof. Mary L. Cummings, Prof. Devavrat Shah

2. Motivation • Modern missions involve networked heterogeneous multi-agent teams cooperating to perform tasks • Unmanned aerial vehicles (UAVs) – target tracking, surveillance • Human operators – classify targets, monitor status • Ground vehicles – rescue operations • Key Research Questions: • How to coordinate team behavior to improve mission performance? • How to hedge against uncertainty in dynamic environments? • How to handle varying communication constraints?

3. Problem Statement • Objective: Automate task allocation to improve mission performance • Spatial and temporal coordinationof team • Computational efficiency for real-time implementation • Key Technical Challenges: • Combinatorial decision problem (NP-hard) – computationally intractable • Complex agent modeling (stochastic, nonlinear, time-varying) • Constraints due to limited resources (fuel, payload, bandwidth, etc) • Dynamic networks and communication requirements • Robustness to uncertain and dynamic environments 25 40 10 • Problem Statement: • Maximize mission score • Satisfy constraints • Decision variables: • Team assignments, Service times 25 25 30

4. Planning Approaches • Optimal solution methods are computationally intractable for large problems • Typically use efficientapproximation methods [Bertsimas ’05] • Most involve centralized planning [Bertsimas ’05] • Base station plans & distributes tasks to all agents • Requires full situational awareness • High bandwidth, slow reaction to local changes • Motivates distributed planning [Sariel ‘05, Lemaire ‘04] • Agents make plans individually & coordinate with each other through consensus algorithms [Olfati-Saber ‘07] • Faster reaction to local information • Increased agent autonomy • Key questions for distributed planning: • What quantities should the agents agree upon? • Information / tasks & plans / objectives / constraints • How to ensure that planning is robust to inaccurate information and models? 25 25 25 40 10 40 10 40 10 25 25 25 25 25 25 30 30 30

5. Distributed Planning • Main issues: Coupling & Communication • Agent score functions depend on other agents’ decisions • Joint constraints between multiple agents • Agent optimization is based on local information • Key challenge: How to design appropriate consensus protocols? [Johnson ‘10] • Specify what information to communicate • Create rules to process received information and modify plans • Performance guarantees – is distributed problem good representation of centralized? • Convergence guarantees – will algorithm converge to a feasible assignment? • Centralized Problem: • Maximize mission score • Satisfy constraints • Decision variables: • Team assignments, Service times • Distributed Problem: • Maximize mission score individually • Satisfy constraints • Decision variables: • Agent assignments, Service times

6. Distributed Planning – CBBA • Consensus-Based Bundle Algorithm (CBBA)[Choi, Brunet, How ‘09] • Iterations between 2 phases: Bidding & Consensus • Core features of CBBA: • Sequential greedy task selection – Polynomial-time, provably good approximate solutions • Guaranteed real-time convergence even with inconsistent environment knowledge 1 Phase 1: Build Bundle & Bid on Tasks (individual agents) 2 Phase 2: Consensus (all agents) All agents consistent? 3 Yes N No • Key Contributions – extensions to CBBA framework: • Time-varying score functions (e.g. time-windows of validity for tasks) • Guaranteeing connectivity in limited communication environments • Robust planning for uncertain environments

7. CBBA with Time-Windows • In realistic continuous-time missions, have time-varying task scores Score Score Score Time-window Time-critical Peak-time e.g. rendezvous, special ops e.g. rescue ops, target tracking e.g. monitor status, security shifts Arrival Time Arrival Time Arrival Time • Extended CBBA to continuous-time domains[ACC 2010] • Task optimization involves decisions on task assignmentsand task service times • Preserves convergence properties • Embedded the algorithm into dynamic planning architecture • Real-time simulation framework for dynamic missions • Experimental flight tests for UAV/UGV teams • Demonstrates real-time feasibility

8. Cooperative Distributed Planning • Often have fleet-wide hard constraints on assignments • Agent assignments coupled through joint team constraints • Example: Maintaining network connectivity in dynamic environments • Often have limited communication radius, line-of-sight requirements • As agents move around environment – dynamic networks, potential disconnects • Several issues: • Some tasks rely on continuous connectivity (e.g. streaming live video) • Cannot perform consensus, cannot deconflictplans • How to include network connectivity constraints into distributed planner? 25 10 40 25 25 30 Disconnected Network

9. Example: Baseline Scenario • Motivating example – Surveillance Mission around base station • UAVs travel to tasks and stream live video back to base station • Successful task execution relies on continuous connectivity • Limited comm radius (RCOMM) 0 30 10 10 No connectivity! 0 15 No connectivity!

10. Example: Network Prediction • Conservative solution – predict network connectivity violations • Drop tasks if disconnects will occur • Only execute tasks in local vicinity – conservative 30 10 10 15

11. Example: Planning with Relays Relay Relay • Can use some agents as communication relays! • Coordinated team behavior leads to higher mission performance • Goal: Develop cooperative planning algorithms to coordinate team 30 30 10 10 15

12. CBBA with Relays • CBBA with Relays [JSAC 2012, Globecom 2011, Infotech 2011, Globecom 2010] • Generate CBBA assignments • Predict network over mission duration • Repair connectivity by creating relay tasks • Key features: • Explicit consideration of dependency constraints • Predict network topology only at select mission-critical times – avoids discretizing time • Leverages information available in CBBA consensus phase • Preserves polynomial-time and convergence guarantees • CBBA with Relays improves performance • Agents accomplish higher value tasks • Guaranteed network connectivity • Demonstrated real-time applicability Real-time experiment Field experiment iRobot Create Pelican quad

13. Distributed Planning Under Uncertainty • Uncertainty in planning process • Inaccurate models (simplified dynamics, parameter errors) • Fundamentally non-deterministic processes (e.g. sensor readings, stochastic dynamics) • Dynamic local information changes • Can hedge against uncertainty to improve planning • Robust planning involves several challenges • Optimal solutions computationally intractable – increased dimensionality of planning problem • Non-trivial coupling of distributions – analytically intractable • Current approaches involve many limiting assumptions Target Identification Mission Agent Schedule Late! Tasks Distribution for Operator Target Identification Figure from [D. Southern, Masters Thesis, 2010] Time • Key questions: • How to propagate uncertainty through planner to generate agent assignments? • How to distribute planning given additional complexity due to uncertainty? • How to ensure real-time performance and computational tractability?

14. Distributed Planning Under Uncertainty • Chance-Constrained CBBA – Extended CBBA to incorporate risk into planning process [ACC 2012] • Model coupling using numerical approx (sampling) • Preserves polynomial-time • Probabilistic performance guarantees for given risk • Key features: • Improved CBBA to handle non-submodular score functions (e.g. stochastic scores) [CDC 2012] • Approximate distributed agent risk given mission risk using Central Limit Theorem assumption • Improved performance under uncertainty • Higher scores within allowable risk • Distributed approximation on par with centralized • Current work is exploring dynamic aspects • Dynamic risk allocation • Model learning using Nonparametric Bayesian techniques [GNC 2012]

15. Conclusion • Distributed task allocation strategies for autonomous multi-agent teams • Extended CBBA algorithm to include time-varying score functions • Addressed cooperative planning in comm-limited environments using relay tasks • Presented robust risk-aware distributed extensions to deterministic planning • Acknowledgments: • Prof. Jonathan How for his invaluable advice and support • My committee members Prof. Cummings and Prof. Shah • My collaborators and colleagues at ACL, esp. Luke Johnson and Andrew Kopeikin • Aero/Astro faculty and staff • Graduate Aero/Astro friends!