Dynamics of Learning & Distributed Adaptation

1. Dynamics of Learning & Distributed Adaptation Santa Fe Institute: James P. Crutchfield, P.I. • Multi-Agent System Science (MASS) • Dimension • Agents learn complex environment ab initio • Synchronization of agent to environment • Agents adapt to nonstationary environment • Strategies for agent-agent coordination • Metrics for large-scale MASs • Statistical Complexity: • Amount of structure & organization in environ’t • Individual agent knowledge v. group knowledge • Mutuality: Architecture of information flow • Lyapunov Spectra: Degrees of stability and instability • Causal Synchrony: Detect coherent subgroup behavior • CAHDE REF • ACFC: Adapting to instabilities in air flow control • AirOps: Emergence of spontaneous leadership • Solution: • Interacting reinforcement and -machine learning agents solve a group task • Approach: Pattern Discovery: Beyond pattern recognition Design & analysis based on sound principles of learning Metrics for cooperation in large-scale systems • Results To Date Predictive theory of agent learning: Quantify agent modeling capacity Data Set Size v. Prediction Error v. Model Complexity Pattern Discovery: The “Aha” Effect Incremental learning algorithm Quantify structure in environment: How structure leads to unpredictability for agent Define synchronization for chaotic environments: Predict required data and time to synchronize Periodic case solved in closed form Transient information: New metric of synchronization Dynamics of reinforcement-learning agents: Nash equilibria v. oscillation v. chaos Dependence on system architecture and initial state • Future Plans (6 months out) • New problems: • Continuous-state and continuous-time agents • Adaptation to active, pattern-forming environments • Dynamical theory of how learning and adaptation occur • Anticipated results: • Monitor emergence of cooperation in agent collectives • Measure mutuality in interacting reinforcement learners • Test on in-house autonomous robotic vehicle collectives • Analytical tools: • Predict whether or not group cooperation can occur • Agent intelligence versus group size • Prediction of the rate of adaptation during collective task • Prototype models: Solvable MAS systems • Software tools: • Ab Initio Learning Algorithms • Library for Estimating MASS Metrics • Enterprise Java Platform for Robot Collectives