Alan Steinberg and Robert Pack Space Dynamics Laboratory/Utah State University Presented at

1. Pixel-Level Fusionof Active/Passive Data for Real-Time Composite Feature Extraction and Visualization Alan Steinberg and Robert Pack Space Dynamics Laboratory/Utah State University Presented at NATO IST-036/RWS-005 Halden, Norway 10-13 September 2002

2. Concept of Pixel-Level Fusion • Active Sensor – LADAR • Passive Sensor – Electro-Optic (EO) Radiation • Fused into One Data Set RANGE DATA PASSIVE IR FROM MULTI - FUSED RANGE IMAGE IMAGE DATA CHANNEL LADAR +

3. Enables Observation Through Partial Obscuration

4. Individual Pixel Individual Pixel r1 Cloud Layers Upper Canopy r2 Lower Canopy r2 r1 r3 r3 r4 r4 r5 Ground Clutter Employment Concept

5. Fused LADAR/EO Image 1(Co-Registered and Georeferenced at the Pixel Level) LADAR Image EO Image

6. Fused LADAR/EO Image 2(Co-Registered and Georeferenced at the Pixel Level) LADAR Image EO Image

7. Orthoprojection of the Two 3D Images Combined in Geographic Space Second Image First Image

8. Profile Views in 3D Viewer First LADAR/EO Image First and Second LADAR/EO Images Combined

9. Visualization Concept -Data Volume Exploration and Clutter Removal

10. Terrain (per DTED or detected ground) Ground Plane 3D Images of Cluttered Environment Image Plane Vegetation Canopy

11. Predict Ground Image Plane Bottom of Canopy Layer (Terrain + 3m) Terrain (per DTED or detected ground) Ground Plane

12. Slice Volume at Bottom of Canopy Image Plane Bottom of Canopy Layer (Terrain + 3m) Terrain (per DTED or detected ground) Ground Plane Residual Shadowed Regions: PD 0

13. Example of Canopy Slicing Oblique EO Scene Orthoprojected Using LADAR – Trees Removed Ground with Trees Removed Residual Shadowed Region: PD 0

14. Two Shots Combined – Trees Removed Corresponding EO Image Residual Shadowed Region: PD 0

15. Target Hidden Under Tree Hidden Target

16. Target Exposed After Tree Removal Exposed Target

17. Conclusions (1):Pixel-Level Fusion of Active/Passive Data for Real-Time Composite Feature Extraction & Visualization • Pixel-Level fusion of LADAR + EO data facilitates target detection & recognition in highly occluded environments • Clutter – tree canopies, clouds, etc. – can be removed through geometric filters

18. Visualization Implication

19. Some Visualization Implications • Target Detection & Recognition • Probabilities of Detection • Probabilities of Target Presence (prior & posterior) • Exploitation of Negative Data • Temporal Evolution of Estimation and Expectations • Out-of Sequence Data • Situation Projection • Expected Updates

20. Expectations Map Prior or Posterior Probability Density (e.g. Track Projection Uncertainty) Target Detectability Map Likely false alarm pD(x,t+n|t,A) p(x,t+n|t) x= Target state (type & location, etc.) t, t+n= Update times A = Action plan (sensor route, sensor controls, processing controls, etc.) 0 - 0.25 0.25 – 0.50 Likely missed detections 0.50 – 0.75 0.75 - 1.00 p(x,t+n|t)pD(x,t+n|t,A) Prior & Posterior Detection Probabilities; e.g. Presentation of Negative Data

21. Some Visualization Implications • Target Detection & Recognition • Probabilities of Detection • Probabilities of Target Presence (prior & posterior) • Exploitation of Negative Data • Temporal Evolution of Estimation and Expectations • Out-of Sequence Data • Situation Projection • Expected Updates

22. Revised Estimates of Track Histories (MTI + Imagery) MTI Reports Imagery Reports MISSILE ASSEMBLY FACILITY Time ELEMENTARY SCHOOL Latency Issues:Out of Sequence Data Initial Estimates of Track Histories (MTI only) MISSILE ASSEMBLY FACILITY Time ELEMENTARY SCHOOL

23. Adaptive Information ExploitationProcess • Acknowledge uncertainties • Estimate length of time for which decision can be deferred • Estimate probability of resolving ambiguity in time • Given current collection plan • By redirecting current resources • By adding resource • Estimate Cost & Net Utility of Candidate Action Plans • Select & Implement Revised Plan

24. Current Estimate of Present Situation -40 -30 -20 -10 0 +10 +20 +30 +40 Visualizing Time-Varying Situation Estimation & Expectation (1 of 3) Situation Time Estimated History Estimated Present Situation Projection Situation Report Time (Plan A) Received Reports Expected Coverage Tgt Selection Decision Tgt Engagement Decision

25. Current Projection of Future Situation -40 -30 -20 -10 0 +10 +20 +30 +40 Visualizing Time-Varying Situation Estimation & Expectation (2 of 3) Situation Time Estimated History Estimated Present Situation Projection Situation Report Time (Plan A) Received Reports Expected Coverage Tgt Selection Decision Tgt Engagement Decision

26. Expected Update of Present Situation -40 -30 -20 -10 0 +10 +20 +30 +40 Visualizing Time-Varying Situation Estimation & Expectation (3 of 3) Situation Time Estimated History Estimated Present Situation Projection Situation Report Time (Plan A) Received Reports Expected Coverage Tgt Selection Decision Tgt Engagement Decision

27. Conclusions • Pixel-Level fusion of LADAR + EO data facilitates target detection & recognition in highly occluded environments • Clutter – tree canopies, clouds, etc. – can be removed over time through geometric filters • Enables visualization of time-varying situation estimation & expectation • Presentation of Detection Probabilities (prior & posterior): e.g. Presenting Negative Data • Presentation of Temporal Evolution of Estimates and of Expectations • Out-of Sequence Data • Situation Projection • Expected Updates & Information Evolution

28. Thank YouMange Takk!