Towards Unbiased End-to-End Network Diagnosis

1. Towards Unbiased End-to-End Network Diagnosis Name: Kwan Kai Chung Student ID:05133720 Date: 18/3/2007

2. Why Network Diagnosis • Several Months before, Taiwan earth crack • Surely a Disaster to Taiwan, cause death • Surely a Disaster to Hong Kong, cause…..

3. Why Network Diagnosis • Damage to major physical link, of course can discover immediately • How about small/insignificant/local damage, or non-physical problem (heavy traffic by infected PC) • Problem shooting need information on network traffic/link status

4. Why Network Diagnosis • By knowing Traffic information, user know which ISP to pay (though they may never get real information) • Local ISP know how to choose higher level ISP, and troubleshooting • Overlay Service provider and provide more intelligent network choosing method for better service • So we desire, and provide research, aims to improve diagnosis mechanism, provide better visibility on network information. More Fast, more accurate, more adaptive to fast network growth, and more efficient in narrow the problem location range

5. Why so hard • Internet is non-centralized • Physical equipments (routers) are maintained by different organizations, and they don’t willing to cooperate • In business sense, hardware modification method is not a short term prefer solution • Software only solution meet limited from hardware by leak of information in segment the network • Some method are not usable as they depends on protocol (e.g. ICMP) that banned as security reason • Asymmetric behavior of Network

6. Diagnosis Method • Classification method - End – to –End Require a number of observer locate in distributed location • Router Response base Single location observer, ping/traceroute method, estimate network status by send message and observe response

7. What is Bias • Most method make some assumption to network behavior • These assumption induce non-zero expect value error • These are error bias and biased assumption

8. Least-biased End-to-End Network Diagnosis (LEND) • Aims to provide Unbiased Diagnosis method • Use End-to-End Method • Identify segments that can infer it’s property by End-to-End Measurement

9. Current two Minimum Assumptions • End-to-End measurement can infer the end-to-end properties accurately • The linear system between path and link level properties assumes independence between link-level properties • These two assumption may also biased!

10. Algebraic Model s: Number of links v: Vector with s elements, element j is 1 if the jth link is in the path, else 0 lj: loss rate of the jth link xj: log(1-lj) r: Number of path G: A matrix with row as a path and column as link bi: log(1-pi), i is the path index

11. Minimal Identifiable Link Sequence (MILS) • Link • LEND we conside physical link only, it is actually a cable between routers/End System • MILS A minimum path segment with loss rate that can be uniquely identified by end-to-end path measurements

12. MILS • In some situation, some property of link within path cannot be identify • MILS tell which segments enable to identify it’s property • Give more information for narrow the possible problem location

13. MILS • MILS may linear dependent • MILS may be a sub-segment of other MILS • MILS is consecutive sequence of link • MILS cannot be composed by other MILSes • MILSes can be expressed as linear combination of end-end paths • MILSes does not share end-points • MILSes must in the path space

14. LEND phases • 2 Phases • Infer loss ratio of all end-end path • Identify all MILS and their loss ratio

15. Identify MILSes • For each path exhaustively enumerate the link sequence in size increasing order • Check minimality by check if current checking sequence share start link with previous discovered MILS • Check identifiable by the fact that MILSes must in path space, equation help use to check this fact

16. Identify MILSes and Properties

17. Identify MILSes and Properties • Xg != X, Xg do not give a real link loss rate (as suppose loss ratio of individual link cannot be measured) • But loss ratio given by • Is suppose to provide real loss ratio of corresponding MILS • To lower overhead network traffic, b (depends on End-End loss ratio) can keep unchange for 1 hour

18. Extend To Directed Graph • Previous Calculation model network as undirected graph • Not suitable as network behavior is asymmetric • Cannot extend directly, without change, all MILSes are End-End path, cannot further segment

19. How to extend to Directed Graph • Make use of “Good Path Algorithm” • No Negative loss rate • Without remove good link, we can identify all link and their loss rate as 0 • Sometime act low loss rate path as good path can provide diagnosis granularity with tradeoff of accuracy

20. Update of Network Topology • 4 Changes: add/delete bad path, add/delete good path • Update of the orthonormal basis Q according to 1 column/1 row change in path space have a smaller Big O then re-initialize new Q • MILSes re-identify after update Q

21. Remarks • LEND provide information on which segment can uniquely identify their property • Can work with other statistical method to estimate property of un-identifiable segments

22. Advantage of LEND • Provide additional information for narrow the problem location range • No Hardware modification need for all routers • Low number of assumption, less bias and more accuracy

23. Disadvantage of LEND • Require distributed observer • Diagnosis granularity depends on observer distribution (location with more observer population will get better granularity)