1 / 73

Inter-Domain Routing Convergence Issues, Impacts and Improvements

Inter-Domain Routing Convergence Issues, Impacts and Improvements. Inter-Domain Routing Convergence Issues Path exploration, misconfiguration, timers, … Impact on Data Plane Data Forwarding transient forwarding loops transient loss of reachability, even during “ failure recovery ”

kato-colon
Télécharger la présentation

Inter-Domain Routing Convergence Issues, Impacts and Improvements

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Inter-Domain Routing Convergence Issues, Impacts and Improvements Inter-Domain Routing Convergence Issues Path exploration, misconfiguration, timers, … Impact on Data Plane Data Forwarding transient forwarding loops transient loss of reachability, even during “failure recovery” Possible improvements EPIC: limit effect of path exploration R-BGP: pro-active “back-up” BGP paths (optional) Readings: Please do the required readings CSci5221: Inter-Domain Routing Convergence Issues and Improvements

  2. Inter-Domain (BGP) Routing Convergence Issues • BGP can converge very slowly • Some measurement studies show it may take up to 15 min! • Many potential causes ! • Some key factors • Routers use MRAI (min. route advertisement interval) to damp and ensure “routing stabilities” • BGP uses path vector, route updates processed and propagated one router at a time! • Routers may “change mind” when receiving new updates • This may lead to so-called the “path exploration” problem that may lead to slow convergence CSci5221: Inter-Domain Routing Convergence Issues and Improvements: Part 1

  3. Delayed BGP Convergenceand Path Exploration AS Path prevents loops..... do not help with convergence “path exploration”, after failure, delays convergence When a link fails (or is repaired), routers “go through” a sequence of paths before selecting a “converged” path Inherent dependencies in advertised “path vectors” Router’s best path is an extension of a neighbors’ best path. Which extends a best path from one of its own neighbors. And so on…… We will look at the path exploration problem in more detail later (slide 30 onwards); first we look at two measurement studies! 3

  4. Classification of Routing Events • Tup: a previously unavailable route is announced. • This represents a route repair • Tdown: a previously unavailable route is withdraw • This represents a route failure • Tshort: an active route with a longer ASPath is replaced with a new route announcement with a shoter ASPath • This represents both a route repair and failover • Tlong: an active route with a shorter ASPath is replaced with a new route announcement with a longer ASPath • This represents both a route failure and failover CSci5221: Inter-Domain Routing Convergence Issues and Improvements: Part 1

  5. Labovitz Study: Experimental Results CSci5221: Inter-Domain Routing Convergence Issues and Improvements: Part 1

  6. Labovitz Study: Experimental Results (cont’d) CSci5221: Inter-Domain Routing Convergence Issues and Improvements: Part 1

  7. Labovitz Study: End-to-End Measurements CSci5221: Inter-Domain Routing Convergence Issues and Improvements: Part 1

  8. Wang et al Study [W+06]:More In-depth Analysis of Impact of BGP Dynamics on Data Plane Motivation: • Real-time services have made high availability of end-to-end Internet paths of paramount importance. • low packet loss rate, low delay, high network availability, and fast reaction time • Internet path failures are widespread • can last as long as 10 minutes • Degraded end-to-end path performance is correlated with routing dynamics.

  9. Questions to Be Answered • How routing changes result in degraded end-to-end path performance? • What kinds of routing dynamics cause the degraded end-to-end performance? • How factors such as topological properties, or routing policies affect performance degradation?

  10. Wang et al Study [W+06] • Study end-to-end performance under realistic topologies. • Investigate several metrics to characterize the end-to-end loss, delay, and out-of-order packets. • Characterize the kinds of routing changes that impact end-to-end path performance. • Analyze the impact of topology, routing policies, MRAI timer and iBGP configurations on end-to-end path performance.

  11. Methodology • A multi-homed prefix • BGP Beacon prefix: 192.83.230.0/24 • Controlled Routing Changes • Failover events: Beacon changes from the state of having both providers to the state of having only a single provider. • Recovery events: Beacon changes from the state of having a single provider for connectivity to the state of having both providers. Provider 1 Provider 2 Provider 1 Provider 2 Provider 1 Provider 2 Failover event Recovery event Beacon Beacon Beacon

  12. Active Probing • From 37 PlanetLab hosts to the Beacon host (a host within the Beacon prefix) • Back-to-back traceroutes • Back-to-back pings • UDP probing (50msec interval) • Data plane performance metrics host B host A Internet host C Provider 1 Provider 2 Beacon host

  13. Representativeness • Connectivity of Destination Prefixes • SS: Single-homed prefixes via a single upstream link • SM: Single-homed prefixes via multiple upstream links • MS: Multi-homed prefixes via a single upstream link • MM: Multi-homed prefixes via multiple upstream links • Routing tables from one tier-1 ISP on January 15, 2006

  14. Representativeness (contd.) • Multi-homed destination prefixes Peer link ISP 2 ISP 3 ISP 1 Customer link Customer link destination

  15. Representativeness (contd.) • Multi-homed destination prefixes with multi-upstream links ISP 2 ISP 1 ISP 2 ISP 1

  16. Packet Loss Failover Recovery • Loss burst: consecutive UDP probing packets lost during a routing change event.

  17. Correlating Packet Loss with Routing Failures • ICMP replies • temporary loss of reachability (!N or !H) • forwarding loops (exceeded TTL) • Routing failures • temporary loss of reachability and transient routing loops • Correlate loss bursts with ICMP messages • time window [-1 sec, 1 sec] • Underestimate the number of loss bursts due to routing failures • missing ICMP packets.

  18. An Example planet02.csc.ncsu.edu experiences packet loss on July 30, 2005

  19. Loss Bursts due to Routing Failures • Failover events: 76% packets lost • Recovery events: 26% packets lost Failover Recovery

  20. Loss Burst Length • loss burst length can be as long as 480 packets for failover events, and 180 packets for recovery events Loss burst length Failover events Recovery events

  21. Multiple Loss Bursts • Multiple loss bursts after the injection of a withdrawal message or an announcement. Failover Recovery

  22. Location of Lost Bursts (Failover events) • Location of the first lost bursts caused by routing failures. • From ISP 2’s BGP updates: • Routing failures do occur and are not visible from ICMP messages due to short duration. • From another AS’s BGP updates, and Oregon RouteView • Routing failures are cascaded to other ASes.

  23. Location of Lost Bursts (Recovery events) • Location of the first lost bursts caused by routing failures. • BGP updates from ISP 2 • 12 withdrawals over 724 recovery events

  24. How Routing Failures Occur (Failover)? Prefer-customer routing policy: routes received from a provider’s customers are always preferred over those received from its peers. Provider 1 Provider 2 Peer link 0 R2 R3 R4 R5 0 0 2 0 0 1 0 R1 R6 0 0 Customer link Beacon AS 0

  25. How Routing Failures Occur (Failover)? (contd.) No-valley routing policy: peers do not transit traffic from one peer to another. 1 0 2 0 1 0 R8 R7 R9 2 0 1 0 Provider 3 Peer link R2 R3 R4 R5 Peer link 0 0 0 2 0 0 1 0 R1 R6 0 0 Provider 2 Provider 1 Beacon AS 0

  26. How Routing Failures Occur? (Recovery) iBGP constraint: a route received from an iBGP router cannot be transited to another iBGP router Provider 2 Withdraw (2 0) R1 R2 R4 Provider 1 1. Path 0 => R3 recovery. 2. R3 sends the path to R2 path (0) Path (0) 3. R2 sends a withdrawal to R1 R3 4. R3 sends the recovery path to R1 0 5. R1 regains its connection to the Beacon Beacon AS 0

  27. Summary of Findings • During failover and recovery events • Routing changes impact packet loss significantly. • Multiple loss bursts are observed in 60% of events. • Routing changes can lead to long packet round-trip delays and reordering. • Loss bursts explained by routing failures last longer than those unidentified ones. • Loss bursts caused by forwarding loops last longer than those caused by loop-free routing failures.

  28. Conclusions • During failover and recovery events • routing failures contribute to end-to-end packet loss significantly. • Routing policies, iBGP configuration and MRAI timer values play a major role in causing packet loss during routing events. • Degraded end-to-end performance can be experienced by a diverse set of hosts when there is a routing change. • Accommodate routing redundancy may eliminate majority of identified path failures.

  29. Recap: Impact of BGP Dynamics=> loss of connectivity on data plane! • Route changes cause up to 30% packet loss for more than 2 minutes [Labovitz00] • Routing events can cause multiple loss bursts, and one loss burst can last for up to 20s [Wang06] • Popular and unpopular prefixes experience losses due to BGP dynamics [Wang05] • VoIP outages are highly correlated with BGP updates [Kushman06]

  30. Delayed BGP Convergenceand Path Exploration AS Path prevents loops..... do not help with convergence “path exploration”, after failure, delays convergence When a link fails (or is repaired), routers “go through” a sequence of paths before selecting a “converged” path Inherent dependencies in advertised “path vectors” Router’s best path is an extension of a neighbors’ best path. Which extends a best path from one of its own neighbors. And so on…… 30

  31. What is Path Exploration (cont’d) When a link fails, a set of dependent paths becomes invalid (or obsolete) Removed one by one from the system Router selects and propagates it Receives withdrawal Selects next best path (possibly invalid) Receive withdrawal, repeat till no more invalid paths Each route update (for same prefix) delayed by MRAI timer at each router MRAI: minRouteAdvertiseInterval 1997 Merit/Labovitz Study: 15 minutes to converge! 31

  32. Path Exploration Example 4210 4 8 210 2 7 9 5 74210 75210 76310 0 1 10 3 6 Network in a steady state 310 6310 32

  33. Path Exploration Example (cont’d) 75210 76310 W 4210 4 210 8 2 7 9 5 74210 75210 76310 0 1 10 3 6 6310 310 33

  34. Path Exploration Example (cont’d) Paths 75210 and 76310 both contain the “problem edge” 10. 2 additional messages to force 7 to flush “bad paths”. Number of “spurious messages” increases with the “richness” of connectivity ….. 8 4 9 2 7 7210 74210 7510 75210 72510 …… 5 0 1 3 6 34

  35. Toy Example: Worst Case Scenario 35

  36. Impact of Path Exploration (cont’d) Delays a router from picking valid, alternate paths Have to first go through all invalid paths Large scale packet losses in a short duration Core routers process millions of packets a second Minimum convergence time is Ω(Lh) L is “longest” path of the network L >> D: D is “diameter” of the network h is message processing time at a node Experimental results (in 1997) show up to 15 minutes to converge! terrible for “streaming”, network-gaming, etc 36

  37. Quantifying Impact Position in the hierarchy affects the amount of path exploration seen 37

  38. Causes for Path Exploration Invalid paths are selected, propagated, then withdrawn Routers waste time processing “stale information” Delay convergence to valid, perhaps less preferred, alternate paths Key Issue: How to distinguish invalid paths from valid paths Difficult in BGP: AS Paths -- high level, abstract 38

  39. Naive Solutions Fail TAG withdrawals: When router generates withdrawal, tag it with cause/location 4 2 7 5 0 1 3 6 74210 75210 76310 WDRAW: (2,1) failed 39

  40. Naïve Solutions Fail (cont’d) AS Paths do not describe (or reflect) internal AS topology. When an internal edge fails, which AS Path affected? [10] or [210]? 4 7 5 0 1 3 6 2 40

  41. Naïve Solutions Fail (cont’d) Link between 3.2 and 6.1 fails. 6.1 generates a withdrawal and tags with <3,6> Should 6.3 remove all paths containing <3,6> ? 4 2 7 5 0 1 3.2 6.1 3.3 6.3 6.2 AS 3 AS 6 41

  42. AS PATHS: High Level Connectivity AS 3 AS 11536 AS 81 AS 1239 AS 217 AS 217 and AS 3 receive the same AS PATH [11536 1239 81] Underlying physical paths are disjoint 42

  43. EPIC --- Our Solution Exploit Path dependencies to Invalidate Paths To avoid path exploration: When linkfails, a set of dependent paths becomes invalid All the dependent paths must be removed from the system Dependent paths cannot be described using only AS Paths AS Paths are annotated with additional information (forward edge sequence numbers - fesn) Can capture path dependencies Can distinguish valid and invalid paths 43

  44. Forward Edge Sequence Numbers When AS Path being advertised to an external AS neighbor, include fesnof “forward” external edge fesn = edge identifier + sequence number Edge <X,Y> AS X AS Y 44

  45. Forward Edge Sequence Numbers (cont’d) Defined per destination, for every AS-AS edge When AS X sends a route to AS Y, the fesn(X:Y, n) is attached If route already has a previously attached fesn, new fesn is prepended to it -- fesnList AS Y (X:Y, n) AS Z (X:Z, m) AS X (X:Y, n) AS W 45

  46. Identifying Invalid Paths: External Failure 4 3 6 2 1 5 Invalid path-stem is (exactly) fesnList received at AS 3 46

  47. Identifying Invalid Paths: Internal Failure 4 3 6 2 1 5 Invalid path-stem is (exactly) fesnListexported from AS 3 to AS 4 47

  48. fesn Management When a link fails, its fesndoes not change Same value carried in withdrawals When <X,Y> is repaired: AS X increments the sequence number Subsequent route announcements carry “updated”fesn So a larger fesn always corresponds to “newer” information 48

  49. fesnList Propagation Same AS Path, distinct fesnLists 4 [4210] {(0:1, 7)(1:2, 7)(2:4, 14)(4:7, 11)} 11 2 7 14 [10] {(0:1, 7)(1:2, 7)} 5 0 1 [0] {(0:1, 7)} 14 3 6 7 10 7 3 [10] {(0:1, 7)(1:3, 3)} 3 7 49

  50. fesnList Propagation 4 2 7 5 0 1 3 6 After the routes are processed at all nodes Routing Table at AS 7 50

More Related