1 / 43

Pursuit Evasion Games (PEGs) for Multiple UUVs

Pursuit Evasion Games (PEGs) for Multiple UUVs. UUVs have the potential to provide an effective defense against submersible threats to military and civilian assets

gossman
Télécharger la présentation

Pursuit Evasion Games (PEGs) for Multiple UUVs

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Pursuit Evasion Games (PEGs) for Multiple UUVs • UUVs have the potential to provide an effective defense against submersible threats to military and civilian assets • The strategies and protocols for their operation are at least as big a challenge as the design and construction of the vehicles themselves • This project address the strategies and coordination protocols necessary to enable this technology • Defense against enemy subs is the number one FNC that is called for in every briefing since 2000.

  2. Bear UUV Being Built byLt Tulio Celano III, USN 8 ft X 10 in., 400 lbs. displacement, upto 100 ft. depth, Top speed 12 knots, effective cruising speed 5 knots, endurance at 5 knots is 45 hours, Modular design

  3. BEAR 1-UUV, Nov 2004 Showing Modules with Mast Up Celano Machine Shop Ballast Pump and Ballast Module

  4. Prior PEG Experience • PEG experience in: • Unmanned aerial and ground vehicle (UAVs and UGVs) • Two and three dimensions. • Symmetrical and asymmetrical games • Proven tool: Nonlinear Model Predictive Trajectory Control (NMPTC) • Explicitly addresses nonlinear systems with constraints on operation and performance • A cost minimization problem in the presence of state and input constraints • Control resulting in the minimum cost is determined over a model predicted horizon • Previously demonstrated in rotary-wing and fixed-wing UAVs

  5. NMPTC: Cost Function Definition • Cost function is defined by:

  6. NMPTC: Cost function illustration

  7. NMPTC: Cost function minimization • A gradient decent method is used to minimize the cost function • Initialization with previous result reduces the number of iterations required • Usually 3-4 iterations are required • The number of iterations in limited to prevent overruns in real-time • In rapidly changing situation this can result in suboptimal solution • Sudden changes may take several time steps • However, this is alright since the situation is changing and unpredictable

  8. Previous UC Berkeley PEG Experiments • Berkeley Aerobot Project (BEAR) • Goal: to build a coordinated, intelligent network with multiple heterogeneous agents • 11 Rotorcraft-based unmanned aerial vehicles (UAVs) • 5 Unmanned ground vehicles (UGVs) • Shipdeck simulator (landing platform) • Stochastic Pursuit-Evasion Games (PEG) • Self-localization • Target detection • Map building • Pursuit policy • Trajectory generation • Control / Action

  9. Previously: PEGs with 4 UGVs and 1 UAV • Sub-problems for Pursuit Evasion Games • Sensing • Navigation sensors -> Self-localization • Detection of objects of interest • Framework for communication and data flow • Map building of environments and evaders • How to incorporate sensed data into agents’ belief states • probability distribution over the state space of the world(I.e. possible configuration of locations of agents and obstacles) • How to update belief states • Strategy planning • Computation of pursuit policy • mapping from the belief state to the action space • Control / Action

  10. PEG Experiment with UAV/UGVs • PEG with four UGVs • Global-Max pursuit policy • Simulated camera view • (radius 7.5m with 50degree conic view) • Pursuer=0.3m/s Evader=0.5m/s MAX

  11. Evaluation of Policies for different visibility Capture time of greedy and glo-max for the different region of visibility of pursuers 3 Pursuers with trapezoidal or omni-directional view Randomly moving evader • Global max policy performs better than greedy, since the greedy policy selects movements based only on local considerations. • Both policies perform better with the trapezoidal view, since the camera rotates fast enough to compensate the narrow field of view.

  12. Evader’s Speed vs. Intelligence Capture time for different speeds and levels of intelligence of the evader 3 Pursuers with trapezoidal view & global maximum policy Max speed of pursuers: 0.3 m/s • Having a more intelligent evader increases the capture time • Harder to capture an intelligent evader at a higher speed • The capture time of a fast random evader is shorter than that of a slower random evader, when the speed of evader is only slightly higher than that of pursuers.

  13. SEC Capstone Demo: Fixed-Wing PEGs • Capstone Demonstrations were proposed to highlight and test the technologies developed in the SEC program • One was a fixed-wing UAV flight test • 6 participant technology developers (TDs) • Honeywell, Northrop Grumman, U Minnesota, MIT, Stanford, and UCB/U Colorado/CalTech • System Integrator was Boeing • OCP would be software framework • Autonomous T-33 trainer as UAV surrogate • Piloted F-15 as wingman/opponent • 13 month schedule May 03 – June 04 • UCB Contribution: Fixed-Wing PEGs

  14. Demo: UCB PEG Scenario • 20 – 60 min. games confirm NMPTC feasibility at real-time • Evader goal: get to final waypoint or avoid evader • Pursuer goal: ‘target’ evader • Pursuer and evader restricted to same performance limits • Scenarios: • UAV as evader • UAV can become pursuer OCP Experiment Controller Snapshot T-33: Evader (yellow) F-15: Pursuer (blue) Target cone definition (θ=10˚,d=3 nm) Left: F15 not behind UAV, middle: F15 not pointed at UAV, right: F15 behind AND pointed at UAV

  15. Flight Test 1 (UAV as evader)

  16. Flight Test 2 (UAV as evader/pursuer)

  17. UUV PEGs: Multiplayer Games • In littoral waters the pursuit evasion game consists of an enemy submarine attempting to cross a line of UUVs which are protecting an asset • The enemy submarine has a speed advantage over the blue force UUVs • UUVs play a role in between a sensor web and a group of pursuers • Research aimed at determining new approaches to teaming for multi-player games. Current literature focuses exclusively on either Nash or Stackleberg solutions

  18. Multiplayer PEG Challenges • The research challenge includes extending the strategies to: • Large multi-player teams • Asymmetric platform characteristics • Limited communications • High level of uncertainly

  19. UUV PEG: Approach • The UUV PEG involves two distinct phases: • Detection phase • Maximize chances of detection constrained by: • Area to cover • Number of UUVs available • Possible evader strategies • Capabilities of UUVs and evader (sensors and noise signatures) • Response phase • Maximize chances of catching the pursuer constrained by: • Capabilities of UUVs and evader (speed, manueverability and communications) • Number of UUVs available • These two phases also depend on each other as both must succeed • How to share resources to maximize overall chance of success • How to overlap the strategies: detectors are responders as well

  20. Multiplayer PEGs: Proposed Solution • A close analogy is football or other team games: • Multi player • Initial (global) strategies well defined • Limited (local) coordination after the snap • What can we learn? • How can we apply this? • How far does the analogy go?

  21. Multiplayer PEGs • Preseason (Off-line precomputed strategy) • Play book: • Evaluate strategies and configurations that will maximize chance of success based on best estimate of other team’s tactics • Practice and preseason games: • Test playbook and find problems • Game time (On-line adaptive strategy) • Choose play based on best knowledge and experience • Line up (in best detection configuration, not necessarily static) • Execute the play • Active and reactive actions (respond to detected evader) • Local communication • Adapt to evolving behavior • Learn from experience, repeat as necessary (Learning by Doing)

  22. Pre-game: Strategies for Detection • Maximize the chance of detecting the evader • Tradeoffs • Movement of the pursuer: • Moving quickly covers more area, but • This makes it easier for evader to see the pursuer and avoid the pursuer • Sensors: • Using passive sonar reduces the range of detection • Using active sonar reveals the pursuers location • Number of pursuers in detector role: • Increases chance of detection

  23. Basic principle: Defense in depth • May not be the optimal with limited resources, • for instance if there are not enough UUVs to ensure detection of the evader by the front line.

  24. Options: Zone Defense • Would this leave seams for an evader to exploit if they have superior sensors, for instance? • Is communication necessary to make such a zone defense effective? • Is there an alternative?

  25. Options: Channeling the evader • A coordinated, heterogenous detection strategy. • For instance, some pursuers could use a very active strategy that exposed them to intentional detection by the evader, with the intension of “channeling” the evader towards other more passive pursuers.

  26. Pursuer Strategies: Capture • Speed disadvantage means that simply optimizing the detection probability is not sufficient • Reachability of UUVs must be known • Communication and coordination will be necessary to overcome speed disadvantage

  27. Defining the Problem:Basic UUV PEG Scenarios • Scenario 1 • Single evader infiltration • Objectives • Red: pass through game area undetected • Blue: detect red team only • Scenario 2 • Single evader attack • Objectives • Red: get within weapons range of some objective • Blue: prevent red attack on objective • Capabilities • Red: limited number of torpedoes available to attack target or blue team UUVs • Red: suicide attacks only, must get within effective range • Scenario 3 • Multiple evader attack • Objectives & Capabilities • Same as Scenario 2

  28. UUV Characteristics in General • Performance • Speed and acceleration • Maximum rate of turn • Maximum rate of ascent/descent (not symmetric in general) • Maximum depth • Sensors • Effective range in passive detection mode • Effective range in active detection mode • Deployable sonar buoys • Communications • Effective communication range (variable in general)

  29. UUV Characteristics, cont. • Method of attack, including: • Self detonation, effective range and perhaps effectiveness as a function of range • Missile (i.e. torpedo) capabilities • Counter measures, including: • Sonar buoys • Noise canisters • Noise signature as a function of: • Speed • Acceleration • Rate of turn • Rate of ascent/descent • Sensor mode • Communication mode • And others

  30. The First Problem Definition • Based on “Scenario 2”: • Single evader attack, many defenders • Objectives • Red: get within weapons range of some objective • Blue: prevent red attack on objective • Capabilities • Red team: • Limited number of torpedoes available to attack target or blue team UUVs • 3 times speed advantage over the Blue (pursuer) team • Blue team: • Suicide attacks only, must get within effective range • 3 times manuever (turn rate ) advantage over the Red team • Limited communication range • Passive and active sensors available

  31. The First Problem Definition, cont • Characteristics • Speed • 3X advantage to Red Team • Maneuver advantage • 3X to Blue Team • Detection a function only of • speed, • communication use • Distance • Sonar • Active & passive available • Communications available for each team: • range a function of power • detectability also a function of power

  32. Detection: Strategy Comparison

  33. Detection: Monte Carlo results • Goals: • Statistical model as a function of configuration, spacing, etc. • Test strategies Line abreast Staggered (Recall: detection function)

  34. Capture: Strategies based on reachability (Mitchell’s Level Set Toolbox) From Airplane example Still even, turn rate reduced to 1/3 Pursuer speed reduced to 1/3 Pursuer speed reduced to 1/3, turn rate increased by 3

  35. Combined the Strategies: Advances in Game Theory • These PEGs fit Game Theory descriptions as: • mixed strategy • simultaneous move • multiplayer • coordinated games • games with incomplete information • Specific tactics can be evaluated to find the equilibria and optimal strategies in certain situations, e.g.: • the best search patterns within a single zone for one UUV, or for two adjacent zones/UUVs • Statistical methods or Monte Carlo methods could be used to determine the changes of success for each player

  36. Communication Between UUVs • The issue of the short range of underwater communications with multiple players is very similar to the problem of ad hoc wireless networks of motes devices • This experience is directly adaptation to the multiple, coordinated UUV scenarios and used to disseminate information within the team regarding: • Detection of an evader • Likely detection by the other team • Coordination instructions • Strategic commands • etc. • This is integral into the simulation environment

  37. Future Research • We are building a group of UUVs at the USNA in Annapolis. These are also being shared with NUWC, Newport • We will develop a theory of multi-player pursuit evasion games with off-line strategies (using robust optimization, the level set toolbox), on-line modifications of the strategy using Model Predictive Control, and an outer-loop of learning • Expect to have simulation results by June 05, experiments for single UUV by June 05, multiple UUV experiments by June 06

  38. UCAR Transition: NMPC in a Complex 3-D Environment Potential function NMPC

  39. UCAR-Obstacle Sensing • Map-based approach • - Obstacle map is measured and stored in the computer • - Upon request, the nearest obstacle coordinate is passed to the MPC unit • - Sensing is always perfect, thus reducing any risks due to sensing failure or any unpredicted control behavior

  40. Obstacle Sensing using Laser Scanner Scanner Control Computer Tilt Mount Position Command • - Encoder • Servo • Micro controller Encoder reading - PIII 700MHz PC104 module Measured range data Ground Station Measured range data • 361meas/scan Minimum range data • Real-time 3D Visualization Vehicle state Light weight 2D Laser Scanner Flight Computer • Real-time optimization Reference trajectory • GPS+INS • PIII 700MHz • PC104 module Navigation data Minimum range data MPC Engine

  41. November 2004 Flight Tests

  42. Multi-Player Games: The Play of the Big Game 1982

More Related