Measuring ISP topologies with Rocketfuel

1. Measuring ISP topologies with Rocketfuel Ratul Mahajan Neil Spring David Wetherall University of Washington ACM SIGCOMM 2002

2. Motivation • To understand Internet structure and design. • How ISP router-level topologies are designed. • Can’t get the real maps. • Backbone maps often available in marketing form. • Severely lacking in router-level detail.

3. ISP topologies for research • Could extract from a Whole-Internet map:eg. Skitter, Mercator, Lumeta. • Paper’s Philosophy: • By focusing on an ISP, can get better precision. • ISPs publish enough information to reconstruct maps. • End goal is more accurate maps for research.

5. Terminology • Each POP is a physical location where the ISP houses a collection of routers. • The ISP backboneconnects these POPs, and the routers attached to inter-POP links are called backboneor core routers. • Within every POP, accessrouters provide an intermediate layer between the ISP backbone and routers in neighboring networks.

7. Points of Presence and Backbone

9. Rocketfuel’s Backbone Map They aren’t telling us everything…

10. Rocketfuel Methodology • ISPs release “helpful” information: • BGP - which prefixes are served • Traceroute - what the paths are • DNS - where routers are and what they do • Build detailed maps: • Backbone • POPs • Peering links

11. Traceroutes • Publicly available traceroute servers • Challenge: To build accurate ISP maps using few measurements • Brute Force Method • 784 vantage points to 120,000 allocated prefixed in BGP table • Queried every 1.5 minutes: 125 days to complete a map.

12. Directed probing • Capitalize on routing information • Identify traceroutes which transit the ISP network Example : AS 7 Dependent Prefixes: 4.5.0.0/16 Insiders : 4.5.0.0/16 Up/down traces: AS 11 to 1.2.3.0/24

13. Path Reductions T1 and T2 enter the ISP at the same point on the way to the same destination Paths to P1 and P2 leave the ISP at the same point Next-hop AS Reduction Ingress Reduction Egress Reduction

14. Reduction Effectiveness • Brute force : 90-150 million traceroutes required • BGP directed probes : 0.2-15 million traceroutes required • Executed after path reduction : 8-300 thousand traceroutes required

15. Location and Role Discovery • Where is this router located? use DNS names S1-bb11-nyc-3-0.sprintlink.net is a Sprint router in New York City use connectivity information if a router connects only to router in Seatles, it is in Seattle • What role does this router play in the topology? only backbone routers connect to other cities use DNS names s1-gw2-sea-3-1.sprintlink.net is a Sprint gateway router

16. Alias resolution problem

17. Alias resolution solution • Send a packet to each interface to solicit responses. • Previous work - responses have the same source: Routers often set source address to outgoing interface • New approach – • responses have nearby IP identifiers: • IP ID is commonly set from a counter. • Alias resolution optimization • Sort by DNS name - find aliases quickly • Cluster by return TTL - rule out many addresses • ALLY found 2.8 times as many

18. IP ID method • x<y<z, z-x small  likely aliases • If |x-y|>200  Aliases are disqualified, third packet is not sent

19. ISP MAPS

20. AT & T

21. Sprint

22. Level 3

23. Telstra

24. POP Structure

25. Completeness • Validation with ISPs • Good to excellent • Hesitant to reveal customer data • Scanning IP addresses • Comparison with Routeviews • Number of BGP adjacencies • Worst case 70% • Skitter • Seven times as many links and routers

26. Impact of reductions • Ingress and Egress reductions

27. Next HOP ASs • Specially beneficial for Insiders

28. Analysis • POP Sizes • All skewed • Most routers present in ten largest POPs • Sprint: 60% POPs : less than 20% of Sprint routers

29. Router Degree Distribution • Small range in data • Layer 2 switches unaccounted

30. Peering Structure • Advantage here: Where and how many places do two ISPs connect • Highly skewed for all ISPs: