Interdomain Issues for IP networks

1. Interdomain Issues for IP networks Henning Schulzrinne (with lots of borrowed slides...)

2. Overview • Architecture review • Interdomain routing • Multicast • VPNs • Interdomain QoS • signaling • charging and settlements • Interdomain application signaling • Carrier selection and multihoming

3. Architecture review • Classical view of ISP food chain • Tier-1, tier-2

4. 357 Mbit/s USA &Canada 56’241 Mbit/s 19’716 Mbit/s Asia-Pacific Europe 2’638 Mbit/s 468 Mbit/s 171 Mbit/s LatinAmerica & Caribbean Arab States, Africa 127 Mbit/s Inter-regional Internet backbone Source: TeleGeography Inc., Global Backbone Database. Data valid for Sept. 2000.

5. Examples of carriers • Tier 1: UUNet, Cable & Wireless (C&W), Sprint, Qwest, Genuity, AT&T • Tier 2: America Online, Broadwing, @home • Tier 3: RCN, Verizon, Log On America

6. Definitions • Peering: exchange of data between ISPs on a sender-keeps-all basis • Access provider (IAP): provide dial-up and leased line access, buy Internet access from tier-1/2 providers • Transit: Using ISP A to reach customers of ISP B, C, ... • Hot potato routing: find earliest exit point to destination network  asymmetric routes

7. NAP or IXPs Internet eXchange Points • "An Internet Exchange (IX)acts as a junction between multiple points of Internet presence. Here, peers are able to directly connect to each other to exchange local Internet traffic. Typically, the IX owns and operates the switching platforms used to interconnect the various users/subscribers." • Also known as Metropolitan Area Exchanges (MAEs) • see http://www.telegeography.com/ix/ • governed by Multi-Lateral Peering Agreements (MLPA)

8. Austria - The Vienna Internet eXchange (VIX) Belgium - Belnet (BNIX) Cyprus - The Cyprus Internet eXchange (CyIX) Denmark - Danish Internet eXchange (DIX) Lyngby Finland - Finnish Commercial Internet eXchange (FCIX) Helsinki) France - Paris Internet eXchange (PARIX) France - French Global Internet eXchange (SFINX) Germany - The Deutsche Central Internet eXchange (DE-CIX) Frankfurt Greece - The Athens Internet eXchange (AIX) Ireland - The Internet Neutral eXchange (INEX) Italy - The Milan Internet eXchange (MIX) Italy - NAP Nautilus (CASPUR) Luxembourg - The Luxembourg Internet eXchange (LIX) Netherlands - The Amsterdam Internat eXchange (AMS-IX) Norway - Norwegian Internet eXchange (NIX) Portugal - The Portuguese Internet eXchange (PIX) Scotland - Scottish Internet Exchange (ScotIX) Spain - El Punto Neutral Espanol (ESPANIX) Sweden - The Netnod Internet eXchange (D-GIX) Switzerland - The Swiss Internet eXchange (SIX) Switzerland - Geneva Cern (CIXP) Switzerland - Zürich Telehouse Internet Exchange (TIX) United Kingdom - The London INternet eXchange (LINX) United Kingdom - Manchester Network Access Point (MaNAP) United Kingdom - London Network Access Point (LoNAP) Bulgaria - The Sofia Internet eXchange (SIX - GoCIS) Czech Rep. - Neutral Internet eXchange (NIX) Prague Latvia - The Global Internet eXchange (GIX) LatNet Romania - The Bucharest Internet eXchange (BUHIX) Slovakia - The Slovak Internet eXchange (SIX) Ukraine - The Central Ukrainian Internet eXchange Some European IXPs

9. The view from elsewhere • Looking glass sites show BGP routes:

10. RADB $ whois -h whois.radb.net AS14 aut-num: AS14 as-name: COLUMBIA descr: Columbia University in the City of New York Network Operations Academic Information Systems 612 West 115th Street New York, NY 10025 admin-c: CU-NOC tech-c: CU239-ORG import: from AS1785 action med=100; # ApTh commodity accept ANY import: from AS701 action med=200; # UUnet commodity accept ANY import: from AS14:AS-ISPPEERS action pref=10; # private ISP peers accept <^PeerAS+$> import: from AS14:AS-NNPEERS action pref=10; # private NN peers accept <^PeerAS+$> import: from AS145 action med=75; # vBNS I2 accept ANY AND NOT {0.0.0.0/0} import: from AS11537 action med=50; # Abilene I2 accept ANY AND NOT {0.0.0.0/0} import: from AS3754 action med=100; # NYSERNet I2 accept <^AS3754+ AS11537+> AND NOT {0.0.0.0/0}

11. QoS • Interdomain SLAs are rare (or non-existent) • Large difference between inter- and intradomain performance?

12. Interdomain QoS Issues • Request authentication • Uniform service levels – my "gold" is your "bronze"... • Payment • NJ Turnpike? • Gardenstate Parkway?

13. Interdomain QoS metric

14. Carrier selection • Allow selection of carrier • Easy for multi-homed sites • but everything else requires loose source route – but what IP address? • will work in both directions

15. Interdomain multicast • Any-source multicast (ASM) has many operational problems: • PIM-SM/DM are only intradomain • PIM-SM complex • RP has scaling and reliability problems • interdomain never got off the ground • no deployed multicast address allocation mechanism • spam problem – anybody can send to group

16. Interdomain multicast • Single-source multicast (SSM) • source-filtered  IGMPv3 • {S,G} as group • avoid address allocation • match many applications: • Internet radio/TV • conferences with single active source

17. Distributed Denial of Service (DDOS) • Need packet tracing (in progress) • Need push-back to filter DOS stream • at source • or close to source • Authentication of filter request to prevent malicious blackouts

18. Settlements • = payments between providers • long history in telephone network • e.g., 4.6b US$ in 2000 net settlements

19. Total Accounting Rate (TAR) • Traditional conceptual cost of connecting a call from country A to country B • Each end contribute the building cost of half circuit to the midpoint • Based on “cost” of early tiny capacity submarine cable • Settlement A to B at 1/2 TAR S. Cheng/ITU

20. Total Accounting Rate (TAR) - cont’d • Same Rate for the opposite direction • Apply to all PSTN services • When the accounting rate change in one direction the other direction must follow S. Cheng/ITU

21. TerminationRate • Usually based on cost of terminating call by destination carrier • Accounting rate may not be the same for the other direction • Accounting rate in each direction can change independent of each other • May deliver traffic at mid-point or FOB on either end of circuit. S. Cheng/ITU

22. Sender keeps all (Peering) • Sender keeps all revenue from calling party • No settlement between carriers • Applicable if average cost and traffic volume are virtually identical in each direction • Usually based on half circuit ownership S. Cheng/ITU

23. US domestic telephony settlements • Doesn't quite fit SKA • Long-distance company collects • Pays fixed charge/minute to originating and terminating local exchange carrier (LEC)

24. $19.95 per month subscription $7.50-$10.50 Wholesale PoP Access $2.00 - $3.00 Customer Care $3.50-$7.50 margin per customer $3.00 amortised customer marketing Where does the money go? Typical US ISP cash-flow Source: Adapted from Paul Stapleton, ISP$ Market Report, Boardwatch Magazine.

25. Settlements-based traffic Settlements-based traffic PTO = Public PTOs A & Bsplit the cost ofthe int’l circuit Telecommunications Operator Delivers traffic PTO B PTO A Pays settlement fees Retains Collects Terminates Collects revenues revenues traffic traffic User 1 User 2 User 3 User 1 User 2 User 3 For accounting rate traffic, a direct bilateral relationship is established between the origin and termination operators. Intermediate transit operators are compensated from the accounting rate which is usually split 50:50. PTO B retains net settlement. ……...

26. Internet Peering traffic (Web) ISP = Internet PTO B pays the full cost ofthe int’l circuit Services Provider One-way (thick pipe) ISP A ISP B Two-way (thin pipe) Requestsand terminates traffic Collects Exchanges revenues traffic Web 1 Web 1 Web 1 User 1 User 2 User 3 For Internet Peering traffic, ISP B pays for both halves of the International circuit(s) which are used for peering with ISP A. ISP B also pays for traffic exchange. ISP B may pay for the circuit directly, or in conjunction with one or more PTOs.

27. Settlements and Peering: What’s the difference? • Settlement-payment traffic • Substantial revenue transfers, from core to periphery of network • Promotes “organic” network growth • So, Operators generating less traffic than they receive have an incentive to keep prices high • Peering traffic • Some revenue transfers, from periphery to core of network • Promotes “spontaneous” network growth • So, ISPs generating less traffic than they receive have an incentive to force prices down

28. Internet traffic flows are highly asymmetric • Public switched telephone service • Traffic flows are bilateral and broadly match value flow in that caller, who initiates the call, also pays for it • Call-back reverses the direction of the call, from a statistical viewpoint, but caller still pays & benefits • Traffic flows unbalanced between developed and developing countries • Public Internet service • Traffic flows are multi-lateral: A single session may poll many countries • Web-browsing is dominant form of traffic: traffic flow is dominantly towards user who initiates the call. Web traffic highly asymmetric • Newer forms of Internet traffic (telephony, push media, streaming video etc) reverses traffic flow to be from user which initiates the call

29. Interdomain AAA • Roaming user identified by NAI (RFC 2486), e.g., alice@example.com • Find DIAMETER AAA via NAPTR + SRV • Generic problem different: • User Alice@A from ISP A visits ISP B • ISP B needs to determine whether Alice is a valid customer of A • Alice needs to authorize B to query A • Needs to get authorization for maximum € amount •  very similar to credit card authorization!

30. Clearinghouse models • e.g., iPass or GRIC for roaming dial-up and wireless users • member company charges subscribers • gets access to other dial-up ports via clearinghouse • gets reimbursed for "visitors" • GSM roaming is not a good model • no price transparency • inefficient routing

31. SIP interdomain • Designed to find proxies by request URI • Authentication and anonymity are issues: • how can callee ascertain identity of random caller? • how can caller know that she's talking to the right person? • trust provider to remove privacy-compromising information

32. BGP problems • Trust • Need route filtering: In April 1997, a small ISP in Florida made a mistake in configuring the router that joined its small network to Sprint. This ISP, known as AS number 7007, allowed all the routes it learned from Sprint using BGP to be exported back to Sprint as its own routes. This is easy to do, because BGP implementations can take routes from IGP and convert them into EGP routes. In this case, the IGP converted CIDR routes into classful routes. The Sprint BGP speaker wasn't filtering properly either and began sending out updates that added AS7007 as the correct route for a portion of every CIDR block (essentially, the first class C, 24-bit-long network prefix). This misinformation first spread through Sprint's network, then to neighboring NSPs, including ANS, MCI, UUNet, and others. Many routers crashed because their routing tables suddenly doubled in size (an additional route was added for each CIDR block), and the routing instability spread throughout the Internet. Remember that, when a router crashes, it drops its BGP connection with its peer, which then sends out an update withdrawing all the routes announced previously by the crashed router. (Network Magazine, March 2002)

33. Alternatives to improving routing • "Resilient Overlay Networks" (Andersen/Balakrishnan/Kashoek/Morris2001) • application-layer routing with one hop • Multihoming: • treat networks like cheap PCs • 99.5% reliability² 99.9975% reliability

34. Multihoming problems • Need either an ASN or two IP address ranges • Only for larger networks  don't allow advertisements for /24 • Network impact: two /22 entries for each subnet • Alternative: NAT • doesn't help reachability of servers advertised in DNS

35. BGP Issues Geoff Huston

36. Why measure BGP? • BGP describes the structure of the Internet, and an analysis of the BGP routing table can provide information to help answer the following questions: • What is changing in the deployment environment? • Are these changes sustainable? • How do address allocation policies, BGP and the Internet inter-relate? • Are current address allocation policies still relevant? • What are sensible objectives for address allocation policies?

37. Techniques • Passive Measurement • Takes measurements from a default-free router at the edge of the local network • Easily configured • Single (Filtered) view of the larger Internet • What you see is a collection of best paths from your immediate neighbours eBGP Local AS Measurement Point

38. Techniques • Multiple Passive measurement points • Measure a number of locations simultaneously • Can be used to infer policy AS2 AS1 AS3 Measurement Points

39. Techniques • Single passive measurement point with multiple route feeds • Best example: • Route-views.oregon-ix.net • Operating since 1995 • 42 peers • Uses eBGP multihop to pull in route views

40. Techniques • Active Measurement Tests • Convergence • Announcement and withdrawal Reporting Points Monitoring Unit AS1 Route Injection Point Internet AS2

41. Interpretation • BGP is not a link state protocol • There is no synchronized overview of the entire connectivity and policy state • Every BGP viewing point contains a filtered view of the network • Just because you can’t see it does not mean that it does not exist • BGP metrics are sample metrics

42. BGP Table Growth BGP Table Growth – 12 year history

43. BGP Table Growth – 2 year history

44. BGP Table Growth – 2 year & 6 month trends

45. BGP Table Growth – Projections

46. Prefix distribution in the BGP table

47. /24 is the fastest growing prefix length

48. /25 and smaller are the fastest growing prefixes in relative terms

49. Prefixes by AS • Distribution of originating address sizes per AS • Address advertisements are getting smaller Non-Hierarchical Advertisements Number of AS’s Prefix Length

50. Multi-homing on the rise? • Track rate of CIDR “holes” – currently 41% of all route advertisements are routing ‘holes” This graph tracks the number of address prefix advertisements which are part of an advertised larger address prefix