Zach Ramaekers Computer Science University of Nebraska at Omaha Advisor: Dr. Raj Dasgupta

1. A Coalition Game-Based Algorithm for Autonomous Self-Reconfigurations in Modular Self-Reconfigurable Robots Zach Ramaekers Computer Science University of Nebraska at Omaha Advisor: Dr. Raj Dasgupta

2. Modular Self-Reconfigurable Robots (MSRs) – What and Why • An MSR is a type of robot that is composed of identical modules • The modules connect together to form larger robots capable of performing complex tasks • Why MSRs? • Inexpensive and Simple • Highly Adaptable • Three main types of MSRs: Chain, Lattice, Hybrid

3. ModRED (Modular Robot for Exploration and Discovery) • Novel 4 degrees of freedom design • Gives improved dexterity • Allows maneuver itself and get out of tight spaces’ • ModRED sensors and actuators • Arduino processor (for doing computations) • IR sensors (for sensing how far obstacles are) • Compass (which direction am I heading) • Tilt sensor (what is my inclination) • XBee radio (for wireless comm.) • Designed by Dr. Nelson’s group, Mechanical Engineering, UNL Simulated robot in Webots CAD diagram of robot

4. ModRED Movements in Fixed Configuration All these movements are in a fixed configuration

5. Problem Addressed: Dynamic Self-Reconfiguration by MSRs Why does an MSR need to reconfigure dynamically? Problem Statement: How can an MSR that needs reconfigure (e.g., after encountering an obstacle) determine which other modules to combine with, and the best configuration to form with those modules? ... in an autonomous manner.

6. Coalition Games for Dynamic MSR Reconfiguration Dynamic self- reconfiguration Game Theoretic Layer (Coalition Game) • We propose a novel, coalition game theory based approach to address the problem of MSR self-reconfiguration • A coalition game gives a set of rules that determine what teams a group of humans (or robots) will form between themselves • Teams are guranteed to be stable: once teams are formed no one will want to change teams Mediator Controller Layer (Gait-tables) Movement in Fixed Configuration

7. Coalition Games for Dynamic MSR Reconfiguration Dynamic self- reconfiguration Game Theoretic Layer (Coalition Game) • We propose a novel, coalition game theory based approach to address the problem of MSR self-reconfiguration • A coalition game gives a set of rules that determine what teams a group of humans (or robots) will form between themselves • Teams are guranteed to be stable: once teams are formed no one will want to change teams Mediator For our scenario: Each module of an MSR is provided with embedded software called an agent that does the coalition game related calculations Controller Layer (Gait-tables) Movement in Fixed Configuration

8. Coalition Games for Dynamic MSR Reconfiguration Dynamic self- reconfiguration Game Theoretic Layer (Coalition Game) • We propose a novel, coalition game theory based approach to address the problem of MSR self-reconfiguration • A coalition game gives a set of rules that determine what teams a group of humans (or robots) will form between themselves • Teams are guranteed to be stable: once teams are formed no one will want to change teams • Our problem: How can we determine these teams or partitions or coalitions for our MSR problem? Mediator For our scenario: Each module of an MSR is provided with embedded software called an agent that does the coalition game related calculations Controller Layer (Gait-tables) Movement in Fixed Configuration

9. Determining the Partitions • Enumerate all possible partitions that includes all the agents - called coalition structures Example coalition structures with 4 agents {1} {2}{3}{4} {1, 2}{3}{4} {1} {2 ,3}{4} V(CSi) = u(1) + u(2) + u(3) + u(4) V(CSi) = u(1,2) + u(3) + u(4) V(CSi) = u(1) + u(2,3) + u(4) • Each coalition structure is associated with a value • V(CSi) = sum of utilities of each coalition in CSi • All the possible coalition structures are represented as a coalition structure graph

10. Coalition Structure Graph (CSG) Possible partitions of 4 modules (agents) Problem: Find the node (coaliltion structure) that has the highest value in this graph

11. Coalition Structure Graph (CSG) Possible partitions of 4 modules (agents) Problem: Find the node (coaliltion structure) that has the highest value in this graph • Finding the optimal coalition structure node in this graph is not easy! • CSG has w(nn) nodes, the search problem is hard (NP-complete) • Approximation algorithm used to find the optimal CSG node (Sandholm 1999, Rahwan 2007, etc.): in exponential time.

12. Dynamic Reconfiguration under Uncertainty Ridge in planned path We are here I need to form another configuration Which modules should I join with? And we are here

13. Modeling Uncertainty in Coalition Formation Possible Coalition Structures {1} {2}{3}{4} {1, 2}{3}{4} {1} {2 ,3}{4} V(CSi) < u(1,2) + u(3) + u(4) (subadditive) V(CSi) = u(1,2) + u(3) + u(4) (additive) V(CSi) > u(1,2) + u(3) + u(4) (superadditive) {1, 2}{3}{4} {1, 2}{3}{4} V(CSi) = u(1,2) + u(3) + u(4) (additive reward value) • CSG with uncertainty • Conventional CSG

14. Modeling Uncertainty in Coalition Formation Possible Coalition Structures {1} {2}{3}{4} {1, 2}{3}{4} {1} {2 ,3}{4} • Dealing with uncertainty: Markov Decision Process (MDP) provide a mathematical model for robots or agents to determine their actions in the presence of uncertainty V(CSi) < u(1,2) + u(3) + u(4) (subadditive) V(CSi) = u(1,2) + u(3) + u(4) (additive) V(CSi) > u(1,2) + u(3) + u(4) (superadditive) {1, 2}{3}{4} {1, 2}{3}{4} V(CSi) = u(1,2) + u(3) + u(4) (additive reward value) • CSG with uncertainty • Conventional CSG

15. MDP-Based CSG Solution Conventional CSG without uncertainty Modified CSG with uncertainty

16. Generate Coalition Utility Values Generate Coalition Structure Graph Set of modules information Run MDP Traversal to Find Optimal CS Run Value Iteration and Determine Policy Optimal Coalition Structure Algorithm to find optimal coalition structure • Pruning – used to reduce the number of nodes that are searched by the algorithm in the coalition structure graph • Three strategies explored: Keep the optimal and two least optimal children; keep the optimal, median, and least optimal children; keep three random children

17. Simulation Results

18. Conclusions and Future Work • Developed coalition game theory based algorithm for MSR self-reconfiguration • Validated to work on accurate model of MSR called ModRED within Webots robotic simulator • To the best of our knowledge • First application of game theory to MSR self-reconfiguration problem • First attempt at combining planning under uncertainty (MDP) with coalition games* • Future work • Investigate distributed models of planning under uncertainty (MMDPs, DEC-MDPs, etc.) • Simulation of exploration and coverage on realistic terrains • Integrate with hardware of ModRED robot *: Suijs et al.(1999) defined a framework called stochastic coalition game, but using a different model involving agent types, which wasn’t validated empirically.

19. ModRED Self-Reconfiguration Simulation Demo