Measurement tools and the adoption of Abilene Observatory

1. Measurement tools and the adoption of Abilene Observatory APAN Cairns meeting, July 2004 Jin Tanaka & Takatoshi Ikeda APAN-JP NOC

2. Outline • Background • Measurement Tools in APAN-JP • Selection and Provision of Measurement data • Deployment plan of Observatory

3. Background • Various measurement tools are installed independently by research projects in APAN-JP. • Operators can’t grasp the whole measurement tools. • Some demonstrations are being done without any notifications. • Multiple measurement tools sometimes deal with the same data simultaneously. • We can see the cases that the same data are measured with multiple machines. • Many researchers have paid attention to throughput data and routing information as the research material. • Operators are investigating the network condition in spots. • Operators are required to provide theresearchers not only with connectivity but with more advanced services as needed. • Operators are required to collaborate with the researchers, just before the events or the demonstrations.

4. Members of APAN-JP NOC <Categories> <Job Functions> Experiments & Advanced Services Network Researchers Basic Services including Performance Network Engineers 24 x 7 Operators Trouble Tickets

5. Measurement Tools in APAN-JPThroughput -1- *¹http://www.web100.org/

6. Measurement Tools in APAN-JPThroughput -2- *² http://www.gtrc.aist.go.jp/gnet/

7. Measurement Tools in APAN-JPRouting * http://www.zebra.org/

8. Measurement Tools in APAN-JPLatency *⁴http://people.ee.ethz.ch/~oetiker/webtools/smokeping/

9. Measurement Tools in APAN-JPRouter

10. Measurement Tools in APAN-JPNetflow

11. Measurement Tools in APAN-JPSyslog

12. Measurement Tools in APAN-JPUsage

13. Selection and Provision of Measurement Data • Sort out datarequired for researchers • Throughput • Iperf, BWCTL • Latency • OWAMP,RRD • Netflow • cflow,flowtools to be provided by the Observatory system that NOC members will coordinate. Advanced backbone services Evaluation ＆ Selection • GNET-1 • IGP Routing information • Tcpdump … etc. • to be adopted into the Observatory system as “new” common tools. • to be left as • the private tools. Collocated systems assisted by NOC Engineers

14. Selection and Provision of Measurement Data Potential “New” Common Tool • ~Example~ • GNET-1 • Developed by AIST (The National Institute ofAdvanced Industrial Science and Technology, Japan) • GNET-1 is a gigabit network testbed, which consists of four Gigabit Ethernet interfaces, four fast andlarge amount of SRAMs and a large-scale FPGA connected to the them. • Function • Wide area network emulation • Network instrumentation • Traffic shaping • Traffic generation Powerful tool for developing network-aware grid software like GridMPI and Gfarm

15. Selection and Provision of Measurement Data TransPAC/APAN Observatory Systems • We are planning to provide the services of two categories: • Advanced backbone data • Backbone data collected by NOC using equipment located in TransPAC/APAN nodes. Data item and data format will become the same as those of Abilene observatory. • Collocation Systems assisted by NOC engineers • Data collected by respective research projects using their own equipments collocated in XP. • Accounts should be provided for NOC operators and engineers. • Permission of exporting data should be authorized by the projects. Just started by the Joint TransPAC and APAN-JP NOC members.

16. Selection and Provision of Measurement DataBackbone Data Collection • Necessity for backbone measurement • Advanced Backbone measurement is required for improving NOC services. • Advanced data enables to allocate the network resources effectively. • It helps operators to find “where is a bottle-neck?” • It activates network researches with advanced backbone data. NOC operators can’t coordinate the E2E measurement. Congestion of measuring activities will be relieved. Observatory Backbone Measurement E2EMeasurement

17. Selection and Provision of Measurement DataProposal of TransPAC/APAN Observatory • Measurement should be done in the best conditions • Observatory systems should be installed in the best environments of TransPAC/APAN network. i.e. To be set as close to the router interfaces as possible, using the best interface cards of Jumbo Frame. • NOC should maintain the systems and export data to authorized researchers. • A good common platform for performance measurement should be installed at each XP. • Common measurement method should be developed • Common tools are installed, providing the common data. • Data should be formatted in the global standards. i.e. GGF, piPEs, IPPM(IETF), IRPM …etc.

18. Selection and Provision of Measurement DataFormation made by Observatory Observatory system will grow up R&E networks! Software, Papers, etc. NOC Basic Service Helpful in operation NOC Advanced Service Researchers Data with authentication Observatory • Developed by NOC researchers & maintained by NOC engineers/operators • Common Tools with High Priority • Output data in standard format can get measurement data of the global networks and work together with foreign researchers.

19. Deployment plan of Observatory Observatory system could be deployed across the TransPAC circuits. This deployment is to : • get data over the long latency link • cooperate with the Abilene Observatory

20. Deployment plan of Observatory • Location • Japan • Tokyo XP • U.S. • To be decided by Indiana University folks Chicago/Indianapolis Los Angeles TokyoXP Average RTT 190ms

21. Deployment plan of Observatory • Machines • There are ４ machines in each side for observatory. (NMS1-4) The configuration is the same as that of Abilene Observatory : • optimized for observatory based on their experience • made easy to cooperate with the Abilene Observatory • managed by NOC • Hardware Specification • Chassis : SuperMicro SuperServer 6013P • CPU : Dual 3.06GHz Intel Xeon • Memory : 2 x 1GB DDR266 • Disk : 2 x SCSI 73GB (RAID1)(NMS2~4） • 3 x SCSI 73GB (RAID5)(NMS1） • NIC : Dual 10/100/1000 on-board • Intel 1000base-SX （NMS1)

22. Deployment plan of Observatory • Roles of NMS • Routing, Router, Syslog and Usage data will be evaluated for the deployment. • NICs on all servers are for Gigabit Ethernet.

23. Deployment plan of Observatory • Network Layout (TokyoXP)

24. Deployment plan of Observatory • Rack Front View

25. Deployment plan of Observatory • Rack Rear View

26. Deployment plan of Observatory • Network Layout (U.S.) • Indiana University folks will talk about the network layout. (For reference) • Current Situation of StarLight site • No L3 device • Create VLAN for the observatory over the TransPAC circuit. It is L2 connectivity between TokyoXP and StarLight site • In the future, L3 device might be installed, and reconfiguration of network layout will be studied. • Power supply • AC power unprotected • at the cost of circuit reliability -- 意味を教えて！

27. Current Status • Current Status (TokyoXP) • The machines has installed the OS and connected the network • Ready to collect data • Netflow • Ready to collect a Netflow data with Juniper router • Problem with the Procket routers: The Procket routers create the UDP data of MTU size for netflow, IP packet containing UDP header and IP header exceed the MTU size. Hence fragmentation is occurred and flow-tools cannot receive packets correctly. ->will be fixed in the next software version • Throughput and Latency • Ready to collect throughput and latency data with the peering machine located in Chicago, managed by John-san. • Need to tune some parameter for good throughput data (buffer , flow-control ) • Our L2 switch didn’t perform wire-rate switching. Some packets was discarded at input port while testing a high bit rate . we will investigate in detail .

28. Roadmap • under discussion with Indiana University, but the following schedule is envisaged: • 2004.7 Netflow & piPEs (temporal) Website was formed: http://www.jp.apan.net/NOC/Observatory/ • 2004.8 start to develop the real system • 2004.10 Connection with Abilene Observatory • 2005.1 Presentation at APAN Bangkok meeting

29. Deployment into APAN • We expect that Observatory system will be widely deployed in APAN. TransPAC Observatory APAN Observatory

30. References • Abilene Observatory • http://abilene.internet2.edu/observatory/ • http://loadrunner.uits.iu.edu/~neteng/nms/ • Tools • web100 http://www.web100.org/ • GNET-1 http://www.gtrc.aist.go.jp/gnet/ • zebra http://www.zebra.org/ • Smokeping http://people.ee.ethz.ch/~oetiker/webtools/smokeping/ • Flow-tools http://www.splintered.net/sw/flow-tools/ • OWAMP http://e2epi.internet2.edu/owamp/ • Iperf http://dast.nlanr.net/Projects/Iperf/

31. Thank you • Please feel free to contact us if you have any questions and comments • Jin Tanaka –tanaka@kddnet.ad.jp • Takatoshi Ikeda –tk-ikeda@kddlabs.jp