1 / 64

Instrumentation and Measurement Background

Instrumentation and Measurement Background. GENI I&M Motivation. To conduct useful and repeatable experiments, GENI experimenters require extensive and reliable instrumentation and measurement capabilities to gather, analyze, present and archive Measurement Data (MD ).

clea
Télécharger la présentation

Instrumentation and Measurement Background

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Instrumentation and Measurement Background

  2. GENI I&M Motivation • To conduct useful and repeatable experiments, GENI experimenters require extensive and reliable instrumentation and measurement capabilities to gather, analyze, present and archive Measurement Data (MD). • In addition, the GENI operations staff require extensive and reliable instrumentation and measurement capabilities to monitor and troubleshoot the GENI suite and its constituent entities. Some of this data will be made available to experimenters, to help them conduct useful and repeatable experiments. FIBRE KOM - 29/10/11

  3. Relationship to GENI Architecture The Instrumentation and Measurement WG focuses on the instrumentation and measurement infrastructure that will be deployed and used in GENI. FIBRE KOM - 29/10/11

  4. GENI I&M Services Goals (1/3) • Provide broad data gathering, analysis and archival capability that is sufficient for scientific mission, operations, and success of the infrastructure. • Remove the burden on researcher to become a system and network measurement infrastructure expert so that researcher can better focus on the science in the experiments • Measure details of GENI behavior with high precision and accuracy in a ubiquitous, extensible, highly available, secure, and integrated manner without adversely impacting experiments. FIBRE KOM - 29/10/11

  5. GENI I&M Services Goals (2/3) • Provide drill-down performance transparency of system and network resources at hop, link, path and slice levels in terms of availability, health status, and diagnosis of perceived as well as impending problems. • Allow and make-it-easy for various user groups to access and control functions involving interactions between I&M sub-services encompassing resources such as instrumentation taps in the network, time sensors, software-based and hardware-based measurement probes, router/switch MIBs, and (short-term/long-term) measurement data archives. FIBRE KOM - 29/10/11

  6. GENI I&M Services Goals (3/3) • Provide performance transparency of the status of the individual I & M sub-service components and their interfaces with other sub-services to ensure correctness of measurements provisioned. • Through tight integration with the CFs, provide mechanisms to handle security, privacy and access control of measurement data archives to allow access only to authorized users, and also provide different data views based on authorization privileges. • Make it easy for data providers (e.g., instrumentation device designers, experiments who implement custom instrumentation devices, etc) to supply data (within legal and ethical limits) to other entities. FIBRE KOM - 29/10/11

  7. GENI I&M User Groups • Experimenters • Opt-In Users: users who participate or “opt-in” to a GENI experiment to utilize resources, applications or services. • Central (i.e., GMOC) Operators • Aggregate Providers and Operators • Archive Providers and Operators: Groups that catalog indexes of GENI-related measurement datasets in a repository and provide tools for users to share, annotate, search and cite the measurement datasets. • Researchers that use Archived Measurement Data: Users that utilize the measurement datasets provided by Archive Providers in order to test hypotheses, and promote reproducible research. FIBRE KOM - 29/10/11

  8. GENI Key I&M Services • Measurement Orchestration (MO) service • Measurement Point (MP) service • Measurement Information (MI) service • Measurement Collection (MC) service • Measurement Analysis and Presentation (MAP) service • Measurement Data Archive (MDA) service FIBRE KOM - 29/10/11

  9. GENI I&M Services Overview Experimenter Operator Wide range of MP Services MO MAP MP MI MP MC MAP MDA MPs MC Researcher

  10. Measurement Orchestration (MO) Service FIBRE KOM - 29/10/11

  11. Measurement Point (MP) Services FIBRE KOM - 29/10/11

  12. Measurement Point (MP) Services FIBRE KOM - 29/10/11

  13. Measurement Point (MP) Services FIBRE KOM - 29/10/11

  14. Measurement Information (MI) Service FIBRE KOM - 29/10/11

  15. Measurement Collection (MC) Services FIBRE KOM - 29/10/11

  16. Measurement Analysis and Presentation (MAP) Service FIBRE KOM - 29/10/11

  17. Measurement Data Archive (MDA) Service FIBRE KOM - 29/10/11

  18. Experimenter Gathering MD from his/her Slice Experimenter MO MP MC MAP MP MC MDA

  19. Experimenter Gathering MD from his/her Slice FIBRE KOM - 29/10/11

  20. Operator Gathering MD from GENI Infrastructure Operator MAP MI MP MP

  21. Operator Gathering MD from GENI Infrastructure FIBRE KOM - 29/10/11

  22. Experimenters Gathering MD from their Slice and from GENI Infrastructure FIBRE KOM - 29/10/11

  23. Experimenters Gathering MD from their Slice and from GENI Infrastructure FIBRE KOM - 29/10/11

  24. GENI I&M Architecture Overview:Types of Services • Type 1 Service: Service contained within a slice. • Type 2 service: Common service platform plus multiple slivers dedicated to multiple experiments. • Type 3 service: Common service with shared data provided to multiple experiments. • Type 4 Service: MDA service with a portal for sharing data FIBRE KOM - 29/10/11

  25. perfSONAR PERFormance Service Oriented Network monitoring ARchitecture

  26. perfSONAR as a Middleware FIBRE KOM - 29/10/11

  27. Analysis/Visualization Infrastructure Data Services Information Services Measurement Points User GUIs Service Lookup Web Pages Measurement Archives Topology NOC Alarms Service Configuration Transformations Auth(n/z) Services perfSONAR Architecture Overview FIBRE KOM - 29/10/11

  28. perfSONAR • Base network measurement schema • OGF Network Measurement Working Group (NM-WG) • Topology Schema • OGF Network Markup Language (NML-) WG • Includes Topology Network ID • perfSONAR Protocol Documents • OGF Network Measurement and Control (NMC-) WG FIBRE KOM - 29/10/11

  29. MonIPÊ Infrastructure FIBRE KOM - 29/10/11

  30. Visualização dos Resultados

  31. Results visualization

  32. Results visualization • Backbone transition from 2.5 to 10Gbps FIBRE KOM - 29/10/11

  33. perfSONAR-MDM • Developed within the scope of the GÉANT project • Written in Java, was designed as a monitoring solution for LHC (Large Hadron Collider). • Available in Debian or RPM packages • https://forge.geant.net/forge/display/perfsonar/Home FIBRE KOM - 29/10/11

  34. perfSONAR-MDM 3.3 Components • RRD Measurement Archive • SQL Measurement Archive • SSH/Telnet Measurement Point • Lookup Service • Authentication Service and Authorization Service • OPPD (previously called BWCTL) Measurement Point) • E2EMon Measurement Point FIBRE KOM - 29/10/11

  35. perfSONAR-PS • http://psps.perfsonar.net/ • Collaboratively developed by: • ESnet, Fermilab, Georgia Tech, Indiana University, Internet2, SLAC, The University of Delaware • Written in Perl. Available for independent deployment or through the Performance Toolkit (pS-PT). FIBRE KOM - 29/10/11

  36. pS-PT • perfSONAR Performance Toolkit (pS-PT) • Current version: 3.2.1 • LiveCD or Net-Install based on CentOS 5.5 • Easy to deploy • Boot with the instalation CD • http://psps.perfsonar.net/toolkit/ • User-friendly Web Interface • https://pspt.nuperc.unifacs.br FIBRE KOM - 29/10/11

  37. FIBRE KOM - 29/10/11

  38. perfSONAR-OS Services • SNMP Measurement Archive • perfSONAR-BUOY • perfAdmin GUI • Lookup Service • Topology Service • Status Measurement Archive • PingER Services • PingER GUI • LS Registration Daemon FIBRE KOM - 29/10/11

  39. pS-PT • Metrics • Throughput • Delay • Ping • SNMP Utilization • Cacti graphs with host performance information • Scheduled tests: • Throughput, one-way delay, and ping. FIBRE KOM - 29/10/11

  40. LAMP • Leveraging and Abstracting Measurements with perfSONAR • perfSONAR is a multi-domain performance monitoring framework, which defines a set of protocols standards for sharing data between measurement and monitoring systems. • Objectives • Collaborate on defining a common but extensible format for data storage and exchange. • Use perfSONAR NM-WG schema as starting point • Develop a representation of GENI topolgy to be used to describe measurements and experiment configuration • Collaborate on a common GENI I&M architecture FIBRE KOM - 29/10/11

  41. perfSONAR-based GENI I&M • Operations & Management: • Federated GENI is perfect match to perfSONAR • perfSONAR SNMP MA setup to query ProtoGENI switches on Internet2 • Experiments: • perfSONAR architecture is modular and extensible • Integration with GENI will need • Measurement Portal • Measurement Orchestration FIBRE KOM - 29/10/11

  42. OnelabMonitoring

  43. MySlice FIBRE KOM - 29/10/11

  44. TopHat (Topát) • http://www.top-hat.info/ • PlanetLab Europe’s active measurement component • Part of OneLab’s effort: global federated environment • TopHat provides PlanetLab applications with a topology monitoring service for the entire lifecycle of an experiment. • Uses DIMES and ETOMIC for specialized Services FIBRE KOM - 29/10/11

  45. Experiment Lifecycle Support • Experiment set up: • Assist users in choosing the nodes for their experiments based on measured characteristics of the network • Run time: • Provides info to support adaptative applications and experiment control. • Post-morten data: • Data are archived for further analysis. FIBRE KOM - 29/10/11

  46. Topology Information • Sources at IP level: • BGP feeds: • RouteViews, Team Cymrum, and PWhoIs • Active measurements: • ScriptRoute (PL native) • Other: eg, perfSONAR FIBRE KOM - 29/10/11

  47. TopHat and perfSONAR • Measurement tools: • “perfSONAR offers a uniform set of tools from a set of peer entities” • TopHat federates heterogeneous systems • Focus: • perfSONAR focus on troubleshooting across network boundaries • TopHat focus on experiment support • TopHat could benefit from accessing router information via perfSONAR (queue lengths and packet drop rates). FIBRE KOM - 29/10/11

  48. TopHat Architecture perfSONAR FIBRE KOM - 29/10/11

  49. Core API • Get: • request information and measurements about nodes and paths • Filter: • filters a ser of nodes or paths • SetCallback: • Configures conditions on which the system will react by triggering a callback function FIBRE KOM - 29/10/11

  50. DIMES • www.netdimes.org • distributed scientific research project, aimed at studying the structure and topology of the Internet, with the help of a volunteer community (similar in spirit to projects such as SETI@Home). • Traceroute measurement module • Large number of vantage points (ca. 1700 measurement agents) FIBRE KOM - 29/10/11

More Related