Adaptive Routing

1. Adaptive Routing Proshanto Mukherji CSC 457: Computer Networks University of Rochester

2. Introduction • Networks are not static. They are subject to three classes of change: • Topologies change as nodes are added and removed • Traffic patterns change cyclically • Overall network load changes • So, routing algorithms that assume that the network is static don’t work in this setting

3. A B C “I can reach C with cost 10” B 30 Rotten Algorithms 1: DV 50 20 10

4. “I see B (cost 10), and A (cost 50)” “I see C (cost 10), and A (cost 20)” A B C 50 A C 20 10 B Host Next Cost B B 20 C B 30 Rotten Algorithms 2: LS 50 20 10

5. Diameter of graph Number of edges Number of hosts Edges per host Analyses • Distance Vector: • No. of messages: O( |E|.diam ) • Size of each message: O( n ) • Link State • No. of messages: O( n.|E| ) • Size of each message: O( e ) • Each generates O( n.|E| ) = O( n3) bytes of traffic! • Perhaps ok for single execution, but not every time the load on the network changes • When will you run the algorithm?

6. A New Model • We want instead to think of the routing protocol as always active • Model the network as a dynamical system in the topology and link costs

7. Q-Routing • Qx(d, y)is the time that node x estimates it will take to deliver a packet to node d via its neighbor y • When y receives the packet, it sends back a message (to node x), containing its (i.e. y’s) best estimate of the time remaining to get the packet to d, i.e. • t = min(Qy(d, z))over all z neighbors( y ) • x then updates Qx(d, y) by: • [Qx(d, y)]NEW = [Qx(d, y)]OLD + .(s+q+t - [Qx(d, y)]OLD ) RTT: x  y Time spent in queue at x new estimate old estimate

8. message message to d x y w min(Qy(d, zi)) = 13; RTT = s = 11 [Qx(d, y)] += (0.25).[(11+17) - 20] 22 Q-Routing (contd.) to d Qy(d, z1) = 25 Qy(d, z2) = 17 estimated RTT = 3 Qy(d, ze) = 70

9. message to d x y w Q-Routing (contd.) to d message Qw(d, z1) Qw(d, z2) Qw(d, ze) Use this path next time

10. Ants

11. Ants Online: Routing Tables • Each router x in the network maintains for each destination node d a list of the form: <d, <y1, p1>, <y2, p2>, …, <ye, pe>>, where y1, y2, …, yeare the neighbors of x, and p1 + p2 + …+ pe = 1 • This is a parallel (multi-path) routing scheme • This also multiplies the number of degrees of freedom the system has by a factor of |E| pi= P(msg to d forwarded via yi)

12. Ants Online: Generation Every destination host hdperiodically generates an “ant” to a random source host hs • An “ant” is a 3-tuple of the form: < hd, hs, cost> • cost is a counter of the cost of the path the ant has covered so far

13. 1+ p 1+ p normalizing sum of probabilities to 1 Ants Online: Updation When a router x receives an ant < hd, hs, cost> from neighbor yi, it: • Updates cost by the cost of traversing the link from xtoyi (i.e. the cost of the link in reverse) • Updates entry for host (<hd, <y1, p1>, <y2, p2>, …, <ye, pe>>) p = k / cost, for some k pi = pi+ p for j i, pj = pj

14. Ants Online: Propagation (1) • Two sub-species of ant: • Regular Ants: P( ant sent to yi ) = pi • Uniform Ants: P( ant sent to yi ) = 1 / e • Regular ants use learned tables to route ants • Uniform ants explore randomly

15. Ants Online: Propagation (2)

16. Q-Routing vs. Ants • Q-Routing only changes its currently selected route when the cost of that route increases, not when the cost of an alternate route decreases • Q-Routing involves overhead linear in the volume of traffic in the network; ants are effectively free in moderate traffic • Q-Routing cannot route messages by parallel paths; uniform ants can

17. Summary • Routing algorithms that assume a static network don’t work well in real-world networks, which are dynamic • Adaptive routing algorithms avoid these problems, at the cost of a linear increase in the size of the routing tables • Q-Routing is a straightforward application of Q-Learning to the routing problem • Routing with ants is more flexible than Q-Routing

18. Finis that’s all, folks