Industrial Issues in Software Performance

1. Industrial Issues in Software Performance Aman Shaikh AT&T Labs - Research ashaikh@research.att.com June 24, 2008 7th International Workshop on Software and Performance Princeton Acknowledgement: Alan Glasser, Kobus van der Merwe,Jennifer Yates

2. Introduction • Communication service providers • Various services: voice, video, data • Various ways of connecting to customer • Ethernet, Frame Relay, ATM, Wireless, … • Trend towards supporting all communication over IP-based core network • Today’s IP networks and services • Thousands of equipments and systems have to work in tandem • Routers, switches, servers, line cards … • Provisioning, configuration, trouble-shooting, maintenance, capacity planning, billing, … • Huge amalgamation of hardware and software Industrial Issues in Software Performance

3. Challenges of Managing Networks • Lack of management support • Absence of standardization in network management • Solutions often develop in an organic fashion • Scale • Hundreds of offices, thousands of routers, hundred thousands of line cards and transport facilities • Heterogeneity • Different kinds of equipments • Example: routers, switches, servers, line cards, … • Different vendors and models for same equipment • Example of router vendors: Cisco, Juniper, Alcatel, … • Cisco router models: CRS, GSR, 7200 series, … • Variety of service offerings with wide variety of features Industrial Issues in Software Performance

4. Challenges of Managing Networks • Constant churn • Customers, services, technology • Explosive growth • Internet traffic is predicted to double every two years • http://newsroom.cisco.com/dlls/2008/ekits/Cisco_Visual_Networking_Index_061608.pdf • More stringent requirements and expectations • Service delivery model of IP is best effort • Works for original applications of the Internet: file transfer, e-mail and newsgroup • Applications such as VoIP, IPTV, gaming require strong performance guarantees from the network • Deliver packets reliably within a certain time bound • Security • Internet’s intrinsic model: “Trust every packet and do not charge for it” Industrial Issues in Software Performance

5. P P P P P P P P C C C C C C C C P P P P P P P P P P P P C C C C C C C C C C C C PE PE PE PE PE PE PE PE PE PE E E E E E E E E E E Tier-1 Service Provider Network DWDM systems OC-48 or OC-192 DWDM PoP Intercity PoP Customer facing PE interfaces PoP Metro • Rough Numbers • 100s of offices • 100s of Ps, 1000s of PEs, 10000s of CEs • 100,000s of transport facilities LEC CE CPE Access CE CPE PoP: Point-of-Presence P: Backbone (core) Router PE: Provider Edge Router CE: Customer Edge Router Industrial Issues in Software Performance

6. Mgt 4 Mgt 3 Mgt 1 Mgt 2 Ticket Management abc xyz abc xyz abc 11 7 19 Mgt sys xyz 26 25 xyzr abc def abc 5 1 24 20 5 ghi xyz abc 7 29 abc1 ISE xyz 13 23 14 xyz 7 26 gef abc abc abc2 24 def abc abc abce xyz 6 xyz abc3 ghi abc 27 7 xyz abc abc 25 26 abc xyz abc4 xxx 11 def 7 11 13 13 xyz abc xxx abc5 6 abc ghi 5 Transport 18 System for xyz abc 3 Portals 7 xxx 7 yyz abc xyz 21 def Task xyzs 14 1 xxx def abc ghi 11 To abc abc 30 ghi 9 2 13 9 abc 29 abc xyz nms 30 1 abc xyz CFKB xyz 9 abc Platform for task xyz xyz abc 11 2 Platform for doing xyz 9 6 ghi 9 xyz abc occ xxx xyz xyz xyz 11 1 7 mno xyz ghi def def abc xxx xxx 30 abc abc abc ABC Portal abc abc 29 1 abc abc xxx xxxx xyz def 11 abc abc platform for xyz xyz xyz yyy ghi xyz abc abc xyz abc abc abc abc abc abc 11 System X zzzzz abc xyz abc NF-TA def CAPRI xyz xyz def xyz xxx! 11 xyz def abce ghi Platform for abc xyz To another sys abc abc ghi 11 6 xyz xyz 23 9 abc DB4 abc abc abct 28 18 def DB3 abc 28 xyz abc abc ghi 9 DB2 xyz 3 11 9 7 7 xyz abc 11 DB1 Platform for xyz xyz abc 7 abc 7 7 xyz 11 def xyz xyz 7 abc xxx 27 1 7 abc 19 abc xyz Platform xyz abc 20 xyz xyz abc abc xyz def abc abc 19 yyy abc def 21 xyz def abc 20 xyz abc abc xyz ghi abc ghi abc 21 19 xyz abc def abc abc abc 20 3 abc dsagjag abc ghi Database 2 Database 3 Database 1 Dga;ljag;lkj Databases ghi abc def The Software above it! Taken from SIGCOMM 2005 Tutorial by Aman Shaikh & Albert Greenberg Industrial Issues in Software Performance

7. Implications for Software Requirement • Need to translate network requirements into software requirements • Example: what does five-9’s of availability mean for software on a router? • Need to determine performance requirement for tomorrow, not today • Accommodate for growth that occurs during expected life-time of a software Industrial Issues in Software Performance

8. Implications for Software Development • How to write reliable and robust software? • Systems need to work 24 x 7 • Under non-deterministic and unpredictable environment • Systems should handle errors and overload gracefully • Systems should support hit-less upgrade • How to write scalable and extensible software? • Software often lives longer than originally anticipated • How to integrate software with existing systems? • Integration with legacy software often a challenge • Implication for performance • Cater to lowest common denominator • Exploit opportunities available for parallelism Industrial Issues in Software Performance

9. Implications for Software Testing • How to generate network-like conditions during testing? • Enumerating all possible conditions is infeasible • However, “untested feature interaction” can lead to problems • Creating network scale is nearly impossible • Simulators and emulators do exist, but do not always scale to the extent of large real networks Industrial Issues in Software Performance

10. My World-View • Knowing performance requirements upfront is vital • But in real world, this is often not possible • Or performance requirements change/evolve over time • Due to growth • Longer than expected shelf-life of software • How to cope with evolving performance requirements? • Analyze how software is likely to scale with input parameters • Those Big-Oh and Big-Omega’s are important • Use tools to determine actual performance • Humans not very good judge of performance and bottlenecks • Have a modular design • Cleanly separate interface and implementation • (Re)-use existing libraries/modules • Document, document, document, … • So that someone else can pick up the threads • Or you yourself don’t feel lost when you re-visit your code Industrial Issues in Software Performance