1. Manifold Alignment for MultitemporalHyperspectral Image Classification H. Lexie Yang1, Dr. Melba M. Crawford2 School of Civil Engineering, Purdue University and Laboratory for Applications of Remote Sensing Email: {hhyang1, mcrawford2}@purdue.edu July 29, 2011 IEEE International Geoscience and Remote Sensing Symposium

2. Outline • Introduction • Research Motivation • Effective exploitation of information for multitemporal classification in nonstationary environments • Goal: Learn “representative” data manifold • Proposed Approach • Manifold alignment via given features • Manifold alignment via correspondences • Manifold alignment with spectral and spatial information • Experimental Results • Summary and Future Directions

3. Introduction N>>30 • Challenges for classification of hyperspectral data • temporally nonstationary spectra • high dimensionality 3 2 1 2001 2001 2002 2003 2004 2005 2006 N narrow spectral bands June July May May May May June

4. Research Motivation • Nonstationarities in sequence of images • Spectra of same class may evolve or drift over time • Potential approaches • Semi-supervised methods • Adaptive schemes • Exploit similar data geometries • Explore data manifolds Good initial conditions required

5. Manifold Learning for Hyperspectral Data • Characterize data geometry with manifold learning • To capture nonlinear structures • To recover intrinsic space (preserve spectral neighbors) • To reduce data dimensionality • Classification performed in low dimensional space Original space Manifold space 3rd dim Spectral bands n Spatial dimension 6 5 4 3 2 1 Spatial dimension 1st dim 2nd dim

6. Challenges: Modeling Multitemporal Data • Unfaithful joint manifold due to spectra shift • Often difficult to model the inter-image correspondences Data manifold at T2 Data manifolds at T1 and T2 Data manifold at T1

7. Proposed Approach: Exploit Local Structure Assumption: local geometric structures are similar Approach: Extract and optimally align local geometry to minimize overall differences Locality Spectral space at T2 Spectral space at T1

8. Proposed Approach: Conceptual Idea (Ham, 2005)

9. Proposed Approach: Manifold Alignment • Exploit labeled data for classification of multitemporal data sets Samples with class labels Samples with no class labels Joint manifold

10. Manifold Alignment: Introduction • and are 2 multitemporalhyperspectral images • Predict labels of using labeled • Explore local geometries using graph Laplacian and some form of prior information • Define Graph Laplacian • Twopotential forms of prior information: given features and pairwise correspondences [Ham et al. 2005]

11. Manifold Alignment via Given Features Minimize Joint Manifold Given Features

12. Manifold Alignment via Pairwise Correspondences Minimize Correspondences between and Joint Manifold

13. MA with spectral and spatial information • Combine spatial locations with spectral signatures • To improve local geometries (spectral) quality • Idea: Increase similarity measure when two samples are close together Weight matrix for graph Laplacian: where spatial location of each pixel is represented as

14. Experimental Results: Data Three Hyperion images collected in May, June and July 2001 May, June pair: Adjacent geographical area June, July pair: Targeted the same area May June July

15. L L L I1, I2 I1 I2 Experimental Results: Framework Graph Laplacian Prior information Joint manifold Given features Classification with KNN Correspondences Develop Data Manifold of Pooled Data

16. Manifold Learning for Feature Extraction • Global methods consider geodesic distance • Isometric feature mapping (ISOMAP) • Local methods consider pairwise Euclidian distance • Locally Linear Embedding (LLE): (Saul and Roweis, 2000) • Local Tangent Space Alignment (LTSA): (Zhang and Zha, 2004) • LaplacianEigenmaps (LE): (Belkin and Niyogi, 2004) (Tenenbaum, 2000)

17. MA with Given Features • Baseline: Joint manifold developed by pooled data 79.21 77.29 77.88 76.31 (May, June pair)

18. MA Results – Classification Accuracy • Evaluate results by overall accuracies

19. Results – Class Accuracy May, June pair Typical class Critical class (Island Interior) Critical class (Riparian) (Woodlands)

20. Summary and Future Directions • Multitemporal spectral changes result in failure to provide a faithful data manifold • Manifold alignment framework demonstrates potential for nonstationary environment by utilizing similar local geometries and prior information • Spatial proximity contributes to stabilization of local geometries for manifold alignment approaches • Future directions • Investigate alternative spatial and spectral integration strategy • Address issue of longer sequences of images

21. Thank you. Questions?

22. References • J. Ham, D. D. Lee, and L. K. Saul, “Semisupervised alignment of manifolds,” in International Workshop on Artificial Intelligence and Statistics, August 2005.

23. Backup Slides

24. Local Manifold Learning for Feature Extraction (s,f) • Local geometry preserved via various strategies for embedding • Popular local manifold learning methods • Locally Linear Embedding (LLE): (Saul and Roweis, 2000) • Local Tangent Space Alignment (LTSA): (Zhang and Zha, 2004) • LaplacianEigenmaps (LE): (Belkin and Niyogi, 2004) • Pairwise distance between neighbors computed using Gaussian kernel function - O(pN2) method • Embedding computed to minimize the total distance between neighbors

25. LE: Impact of Parameter Values • Parameter values for local embedding • s obtained via grid search • k, p obtained empirically BOT Class 3, 6 BOT Classes 1-9

26. Alignment Results: Typical Class • Island Interior

27. Alignment Results: Critical Class • Critical class: Riparian

28. Alignment Results: Critical Class • Critical class: Woodlands

29. MA Results – Classification Accuracy • Evaluate results by overall accuracies Labeled Class (Subset Data) Classified via Pooled Data Classified via Given Features Classified via Correspondences May, June pair

30. MA Results – Classification Accuracy Classified via Given Features (Spectral + spatial) Classified via Correspondences (Spectral + spatial) Labeled Class (Subset Data) Classified via Given Features (Spectral) Classified via Correspondences (Spectral) May, June pair