Computational and Information Sciences Directorate Army Research Laboratory (ARL)

1. Data Mining Combat Simulations: an Emerging Opportunity Barry A. Bodt babodt@arl.army.mil (410) 278-6659 Computational and Information Sciences Directorate Army Research Laboratory (ARL) The U.S. Army’s Corporate Laboratory

2. Motivation • Simulation and statistical analysis are underutilized in helping the commander’s staff to analyze courses of action. • Battle results are infinite in scope, yet the outcome of any one battle is defined by a unique set of battlefield interactions. • Key is to recognizing those interactions through development of more informative performance measures unique to the scenario at hand.

3. Approach Use statistical methods and combat models to create a methodology that identifies non-traditional metrics for plan evaluation.

4. Background Military Decision Making Process • Focus on wargame • Disciplined rules • Synchronization matrix COA

5. Joint Tactical Operation Center, Qatar

6. Network Centric Warfare Communicate… Smart Logistics On-board Diagnostics Soldier Health Sensor information …

7. Information Requirements in NCW The key to any analysis is the set of measures used to represent the performance and effectiveness of the alternatives being considered. We are relatively good at measuring the performance of sensors and actors, but less adept at measuring command and control. Command and control, to be fully understood, cannot be analyzed in isolation, but only in the context of the entire chain of events that close the sensor-to-actor loop. To make this even more challenging, we cannot isolate on one target, or even a set of targets but need to consider the entire target set. Furthermore, network centric warfare is not limited to attrition warfare … It is not sufficient to know how many targets are killed, but exactly which ones and when… Ref: Network Centric Warfare, 2002

8. Simulation Data • Scenario development • OneSAF lay down of forces • OneSAF modified output • Data supporting modeling

9. Scenario BMP-2 BMP-2 BMP-2 T-72M T-80 Town T-72M T-72M T-72M T-80 T-72M T-80 T-80 Company Objective

10. OneSAF Screen Dump

11. Automated Data Collection • OneSAF Modifications OBJECT_ID: 100A31 X = 24396.82 Y = 25828.75 Z = 755.72 Vehicle Authorized Undamaged Catastrophic Firepower Mobility Damage Damage Damage M2 1 0 1 0 0 Equip/Supplies: Current Lvl Resupply Lvl Avg Per Veh 25mm HE (M792) 625.00 625.00 625.00 25mm APFSDS-T (M919) 325.00 325.00 325.00 TOW (TOW) 0.00 5.00 0.00 7.62mm MG (M240) 2340.00 2340.00 2340.00 Fuel (Fuel) (gallons) 171.00 174.00 171.00

12. OneSAF ModificationKiller/Victim Scoreboard • Firer and Target Identity and Location • Type of Ammo • Range • Outcome Time Stamp 1010070890 Vehicle ID 1076 Firer ID 1087 Projectile 1143670848 Firer Position: X = 220217.00 Y = 146765.00 Z = 12.37 Target Position: X = 222454.38 Y = 149117.80 Z = 9.99 Vehicle 1076: Hit with 1 "munition_USSR_Spandrel" (0x442b0840) Comp DFDAM_EXPOSURE_HULL, angle 19.53 deg Disp 0.889701 ft Kill Thermometer is: Pk:1.00, Pmf:1.00, Pf:0.90, Pm:0.80 Pn:0.80 RANGE 3246.773576 r = 0.990835 kill_type = MF 1076 100A41 vehicle_US_M1 1087 100A23 vehicle_USSR_BMP2

13. Data Supporting Classification Models • Response – mission accomplished (success) if an undamaged platoon occupies objective at battle end (MA) • other responses include MBT and “Eric” strength and forces on objective • 228 OneSAF runs • 3 situational snapshots per run • 10% blue ammo expended • 25% blue ammo expended • 40% blue ammo expended • 429 data points per run (143 per stopping time) • Number of K, M/F, F, and M kills • Ammunition levels • Number of hits delivered • Range of hits • Number of side hits delivered • Distance to objective • Number of Blue on objective Data Matrix 228 x 434

14. Model Performance Slice 1 ~ 2000m Or ~ 5 ½ minutes Slice 2 ~ 4000m Or ~ 10 minutes Correctly Classified Loss: 71% Win: 68% Overall: 70% Correctly Classified Loss: 82% Win: 77% Overall: 80% Slice 3 ~ 5800m Or ~ 20 minutes Pred Obs 0 1 Pred Obs 0 1 Correctly Classified Loss: 88% Win: 82% Overall: 85% 0 85 34 0 98 21 Pred Obs 0 1 1 35 74 1 25 84 0 105 14 1 20 89 Company Objective Company Objective Company Objective

15. Method Comparison Percent Correct Classificationby Stopping Time and Method

16. Advantages • Support prediction for COA performance evaluation • Provide models identifying key battle parameters for a given engagement, influencing both COA development and commander’s critical information requirements • Input to CCIRs • Input to contingency plans • Input to tolerances for synchronization

17. Implementation Models Reach back Advantages -computational power (ARL 9th) -more complex analyses Disadvantages -latency -can’t smell gunpowder DistributedAdvantages -cheaper boxes (250 OneSAFboxes used at Ft. Leavenworth) -closer to action Disadvantages -depth of a field analysis -automation required

18. Why Aren’t We Already Doing This? A few reasons … • Computer simulation focus has been mainly strategic or oriented toward acquisition. Tactical application has been limited. • Simulations did not have high enough fidelity for tactical application. • Simulations were unstable. • Computing resources were inadequate. • Necessary communication of inputs had not been imagined. • Simulation creators do not always talk to statisticians.

19. Improvements Here and On the Way • Stability • Power Point force laydown of forces • MS Word OPORD • Terrain, weather wizzards • Composable simulations • After Action Report data • Man-in-loop allowed • Sensor advances • Communication advances • Computation speed and cost

20. Catching On?

21. Next Up Wei-Yin Loh, Regression Tree Analysis of Battle Simulation Data David Kim, Robust Modeling Based on L2E Applied to Combat Simulation Data Warren Liao, Discovery of Battle States Knowledge from Multi-Dimensional Time Series Data