1 / 37

Towards a Meaningful MRA for Traffic Matrices

Towards a Meaningful MRA for Traffic Matrices. D. Rincón, M. Roughan, W. Willinger. IMC 2008. Outline. Seeking a sparse model for TMs Multi-Resolution Analysis on graphs with Diffusion Wavelets MRA of TMs: preliminary results Open issues. Context: Abilene. 12 nodes (2004). STTL. NYCM.

osborn
Télécharger la présentation

Towards a Meaningful MRA for Traffic Matrices

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Towards a Meaningful MRA for Traffic Matrices D. Rincón, M. Roughan, W. Willinger IMC 2008

  2. Outline • Seeking a sparse model for TMs • Multi-Resolution Analysis on graphs with Diffusion Wavelets • MRA of TMs: preliminary results • Open issues

  3. Context: Abilene 12 nodes (2004) STTL NYCM CHIN DNVR WASH SNVA IPLS KSCY ATLA LOSA ATLA-M5 HSTN Abilene topology (2004)

  4. Example: Abilene traffic matrix

  5. Traffic matrices • Open problems • Good TM models • Synthesis of TMs for planning / design of networks • Traffic prediction – anomaly detection • Traffic engineering algorithms • Traffic and topology are intertwined • Hierarchical scales in the global Internet apply also to traffic • Time evolution of TMs • How to reduce the dimensionality catch of the inference problem? • Our goals • Can we find a general model for TMs? • Can we develop Multi-Resolution machinery for jointly analyzing topology and traffic, in spatial and time scales?

  6. Can we find a general model for TMs? • Our criterion: the TM model should be sparse • Sparsity: energy concentrates in few coefficients (M << N2) • Tradeoff between predictive power and model fidelity • Easier to attach physical meaning • Could help with the underconstrained inference problem • Multiresolution analysis (MRA) • “Classical MRA”: wavelet transforms observe the data at different time / space resolutions • Wavelets (approximately) decorrelate input signals • Energy concentrates in few coefficients • Threshold the transform coefficients  sparse representation (denoising, compression) • Successfully applied in time series (1D) and images (2D)

  7. How to perform MRA on TMs? • Traffic matrices are 2D functions defined on a graph • 2D Discrete Wavelet Transform of TM as images • Uniform sampling in R2 • TMs are NOT images! – the intrinsic geometry is lost • Graph wavelets (Crovella & Kolaczyk, 2003) • Spatial analysis of differences between link loads -anomaly detection • Drawbacks of the graph wavelets approach • Non-orthogonal transform - overcomplete representation • Lack of fast computation algorithm

  8. How to perform MRA on TMs? Operator T • Diffusion Wavelets (Coifman & Maggioni. 2004) • MRA on manifolds and graphs • Diffusion operator “learns” the underlying geometry as powers increase – random walk steps • Amount of “important” eigenvalues -vectors decreases with powers of T • Those under certain precision are related to high-frequency details, while those over  are related to low-frequency approximations W1 V1 W2 V2 W3 V3

  9. Cv2 5 CW2 3 CW1 2 How to perform MRA on TMs? Operator T • Diffusion Wavelets (Coifman & Maggioni. 2004) W1 V1 W2 V2  Eigenvalues (low to high frequency)

  10. Diffusion Wavelets and our goals • Unidimensional functions of the vertices F(v1) can be projected onto the multi-resolution spaces defined by the DW. • Network topology can be studied by defining the right operator and representing the coarsened versions of the graph. • But Traffic Matrices are 2D functions of the origin and destination vertices, and can also be functions of time: TM(V1,V2,t)

  11. 2D Diffusion wavelets Operator T • Extension of DW to 2D functions defined on a graph • F(v1,v2) • Construction of separable 2D bases by “projecting twice” into both “directions” • Tensor product • Similar to 2D DWT • Orthonormal, invertible, energy conserving transform WW1 VW1 WV1 VV1 WW2 VW2 WV2 VV2 WW3 VW3 WV3 VV3

  12. 2D Diffusion wavelets Operator T • Extension of DW to 2D functions defined on a graph WW1 VW1 WV1 VV1 WW2 VW2 WV2 VV2

  13. MRA of Traffic Matrices • More than 20000 TMs from operational networks • Abilene (2004), granularity 5 mins • GÉANT (2005), granularity 15 mins • Acknowledgments: Yin Zhang (UTexas), S. Uhlig (Delft), • Diffusion operator: • A: unweighted adjacency matrix • “Symmetrised” version of the random walk – same eigenvalues • Double stochastic (!) • Precision ε = 10-7

  14. 1 1 2 1 4 2 0 0 1 12 12 10 3 5 2 6 2D Diffusion wavelets – Abilene example V0 12 V4 6 W1 V1 W5 V5 W2 V2 W6 V6 W3 V3 W7 V7 W4 V4 W8 V8 # eigenvalues at each subspace Wj = WVj + VWj + WWj

  15. 2D Diffusion wavelets – Abilene example STTL SNVA DNVR LOSA KSCY HSTN IPLS ATLA CHIN NYCM WASH ATLA-M5

  16. 2D Diffusion wavelets – Abilene example

  17. 2D Diffusion wavelets – Abilene example DW coefficients Abilene 14th July 2004 (24 hours) Time (5 min intervals) Coefficient index (high to low freq)

  18. 2D Diffusion wavelets – Abilene example • How concentrated is the energy of the TM? • Wavelet coefficients for the Abilene TM • 12 x 12 = 144 coefficients, low- to high-frequency Coefficients – high to low frequency

  19. Compressibility of TMs

  20. Stability of DW coefficients

  21. Coefficient rank – Abilene March 2004 Time (5 min intervals) Coefficient index Rank signature

  22. Rank signature – anomaly detection?

  23. Conclusions and open issues • Representation of TMs in the DW domain • TMs seem to be sparse in the DW domain • Consistency across time and different networks • Ongoing work • Develop a sparse model for TMs • How the sparse representation relates to previous models (e.g. Gravity) ? • Exploit DW’s dimensionality reduction in the inference problem • Exploring weighted / routing-related diffusion operators • Exploring bandwidth-related diffusion operators • Introducing time correlations in the diffusion operator • Diffusion wavelet packets – best basis algorithms for compression • DW analysis of network topologies

  24. Thank you ! Questions?

  25. Extra slides

  26. Géant 23 nodes (2005)

  27. AS1 AS2 AS3 Network/AS PoPs Access Networks Context: topology • Spatial hierarchy

  28. Multi-Resolution Analysis • Intuition: “to observe at different scales”

  29. Multi-Resolution Analysis • Approximations: coarse representations of the original data

  30. V0 W1 V1 V2 W2 W3 V3 Multi-Resolution Analysis • Mathematical formalism • Set of nested scaling subspaces (low-frequency approximations) generated by the scaling functions • The orthogonal complement of Vi inside Vi+1 are called detail (high-frequency) or wavelet subspaces Wi, generated by waveletfunctions

  31. Multi-Resolution Analysis • Scaling functions: averaging, low-frequency functions • Wavelet functions: differencing, high-frequency functions

  32. Multi-Resolution Analysis (2D) • Separable bases: horizontal x vertical • Example: 2D scaling function

  33. Wavelet transform example • 2D wavelet decomposition of the image for j=2 levels • Vertical/horizontal high/low frequency subbands

  34. Our approach • Can we develop Multi-Resolution machinery for analyzing topology and traffic, in spatial and time scales? • Classical 1D or 2D wavelet transforms are not an option • We need a new graph-based wavelet transform! • Graph wavelets for spatial traffic analysis (Crovella & Kolaczyk 03) • Diffusion wavelets (M. Maggioni et al, 06)

  35. The tools: Graph wavelets • Graph wavelets for spatial traffic analysis (Crovella & Kolaczyk 03) • Exploit spatial correlation of traffic data • Sampled 2D wavelets

  36. The tools: Graph wavelets • Graph wavelets for spatial traffic analysis (Crovella & Kolaczyk 03) • Link analysis • Definition of scale j: j-hop neighbours

  37. traffic j=1 j=3 j=5 The tools: Graph wavelets • Graph wavelets for spatial traffic analysis (Crovella & Kolaczyk 03) • Anomaly detection in Abilene

More Related