Enhanced ECMP and Large Flow Aware Transport draft-yong-pwe3-enhance-ecmp-lfat

1. Enhanced ECMP and Large Flow Aware Transportdraft-yong-pwe3-enhance-ecmp-lfat {lucyyong,yangpeilin}@huawei.com IETF77, Anaheim CA

2. traffic flows ECMP Paths Why Enhanced ECMP • Internet Traffic show traffic pattern specialty • Hash based ECMP can not evenly disperse traffic flows over ECMP paths under such traffic pattern • Hashing dispersion is simple and stateless, • if flow IDs are random enough, hashing evenly disperses the number of flows on paths • But it does not mean the even traffic volume on the paths • Simulation shows uneven load under internet traffic • Uneven load over ECMP paths results • Congestion happens on a path while other are partial idle • Low network efficiency IETF77, Anaheim CA

3. Enhanced ECMP Proposal • Apply different treatments on small flows and large flows • Use hash to disperse all small flows over ECMP paths • Use a table to map a small set of large flows to ECMP paths • Simple load balance algorithm can effectively compensate unbalanced paths caused by hashing • Mapping table automatically refresh to remove non-live flows • A very small set of large flows will not give BIG burden to device and not cause scalability concern IETF77, Anaheim CA

4. Large Flow Recognition • Larger flows in the context of this draft are top of rate ranked flows • Only needs very small percentile of large flows in the method • The rest flows are referred to as small flow in this draft • Several ways to recognize a large flow from the received traffic, this subject is out of scope of this document • However, a large flow applies to the entire network • Not necessary to perform large flow recognition at every node if PE can insert a large flow indication in packets, i.e. only PE perform the recognition process • P nodes just inspect the indication on received packets IETF77, Anaheim CA

5. Simulation Work • We analyzed Internet Traffic captured by Caida (http://www.caida.org/data/monitor) • Make a traffic generator that generates Internet traffic • 2% large flows take up 30% of traffic volume • 98% small flows take up 70% of traffic volume • Flow rate for small or large flows are randomly generated • Apply generated traffic to 4 ECMP paths by using existing ECMP and enhanced ECMP approach, respectively • Run it again over 10 ECMP paths IETF77, Anaheim CA

6. Result for Four ECMP Paths ECMP can get ~10% volume difference between paths Enhanced ECMP obtains <1% volume difference between paths IETF77, Anaheim CA

7. Result for Ten ECMP Paths ECMP can get ~15% volume difference between paths The more ECMP paths, the worse Hash perform Enhanced ECMP obtains <1% volume difference between paths IETF77, Anaheim CA

8. Large Flow Indication • Draft-ietf-pwe3-fat-pw introduces the flow label in PW packets, • Flow Label • Be inserted at ingress PE • Be removed at egress PE • Be used by P nodes in ECMP process to get flow granularity dispersion • Locate at the bottom of label stack and only used by ECMP only • Suggest to embed a large flow indication in Flow Label to facilitate enhanced ECMP process at P IETF77, Anaheim CA

9. Large Flow Indication • Flow Label has the same format as MPLS label • Traffic Class bits are not defined in the flow label and not used by current ECMP • Propose to use one TC bit for large/small flow indication IETF77, Anaheim CA

10. TC Bits Usage Conflict? • Traffic Class bits in MPLS label is used for DiffServ • This only applies to PW label and LSP label not Flow Label • DiffServ function uses the top label TC bits • Flow Label is never on the top of label stack • DiffServ is for differentiated service treatments • Large and Small Flow Classification is different flow treatments in ECMP process • Load balance improvement -> Conclusion: there is not conflict on TC bit usage Can anyone think of a reason to use DiffServ in the Flow Label? IETF77, Anaheim CA

11. Ingress PE Process • Insert flow label for each received packet • Perform Large Flow Recognition • Large flow criteria can be configured by operator • Insert large flow indication in the packets for the large flow • Set the default as a small flow • Egress PE trims off the flow label before forwarding to AC • Same process as described in FAT-PW IETF77, Anaheim CA

12. One More Thing To Address • In PSN, Flow Label, PW Label, LSP Label may occur at BOS • Packet Separation Process needs to distinguish Flow Label from PW Label and LSP Label • Suggest to use TTL value 0 to differ them • Flow Label has S bit =1 and TTL =0 • Packet Separation Process • Use first nibble to differ IP packets and Labeled packets • Use S bit and TTL value to determine Flow Labeled packets • Use TC bit to determine Small and Large Flow Packets • Error Tolerance • If FL is sent to the forwarder due to an error, TTL=0 will force the packet to be dropped immediately IETF77, Anaheim CA

13. Applicability • A single large flow in PW • IP packets • LSP traffic with entropy label or application label • Transport over LAG • MS-PW Full Backward Compatibility • Proposed method supports full compatibility (described in the draft) IETF77, Anaheim CA

14. Acknowledgement Authors like to thank Stewart Bryant, Frederic Jounay, Simon Delord, Raymond Key for their review and comments Next Step We like to hear people comments and advices in moving to next step IETF77, Anaheim CA