1 / 9

Content-Aware Device Benchmarking Methodology (draft-hamilton-bmwg-ca-bench-meth-05)

In November 2010, the BMWG convened during IETF-79 in Beijing to discuss the benchmarks for new classes of network devices operating above the IP layer. The proposed methodology aims to more accurately reflect device performance in real-world scenarios, building on existing frameworks like RFC2647 and RFC3511. Key agenda points included methodology drafts, terminology updates, and a solicitation for volunteer participants. The goal is to establish repeatable tests that predict device performance under simulated network conditions while addressing outstanding questions related to traffic composition and security.

dawson
Télécharger la présentation

Content-Aware Device Benchmarking Methodology (draft-hamilton-bmwg-ca-bench-meth-05)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. BMWG Meeting IETF-79 Beijing November 2010 Mike Hamilton mhamilton@breakingpoint.com BreakingPoint Systems Content-Aware Device Benchmarking Methodology(draft-hamilton-bmwg-ca-bench-meth-05)

  2. Agenda • Charter Update • Now What? • Proposed Schedule • Goals Reset • Goals/Non-Goals • Proposed Schedule • Outstanding Questions • Draft-05 highlights

  3. Charter Update New classes of network devices that operate above the IP layer of the network stack require a new methodology to perform adequate benchmarking.  Existing BMWG RFCs (RFC2647 and RFC3511) provides useful measurement and performance statistics, though they may not reflect the actual performance of the device when deployed in production networks.  Operating within the limitations of the charter, namely blackbox characterization in laboratory environments, the BMWG will develop a methodology that more closely relates the performance of these devices to performance in an operational setting. In order to confirm or identify key performance characteristics, BMWG will solicit input from operations groups such as NANOG, RIP and APRICOT.

  4. Now What? • Define strategy for getting the work done • Terminology Draft • Methodology Draft • Call for volunteers • Would like to setup weekly meetings

  5. Proposed Schedule • December 2011 • Terminology Draft for IESG Review • Methodology Draft for IESG Review • December 2010 • -00 terminology draft • January 2011 • -00 methodology draft • March – June 2011 • Solicit WG feedback • June 2011 • Start WGLC

  6. Goals Reset • Create a series of benchmark tests to MOST accurately predict device performance under realistic conditions FOR A SPECIFIC SIMULATED NETWORK • RFC 2544 Quotes Page 11, Section 18, “Multiple Frame Sizes” • “The distribution MAY approximate the conditions on the network in which the DUT would be used.” • “The authors do not have any idea how the results of such a test would be interpreted other than to directly compare multiple DUTs in some very specific simulated network”

  7. Goals/Non-Goals • Goals • Repeatable Results • Random/Repeatable • Testbed to testbed • Compare Multiple DUTs • Non-Goals • Replace RFC 2544/3511

  8. Outstanding Questions • Traffic Composition • Basket? • Methodology for selecting? • Updatable? • One network is different from another every day • Security? • Is it worth the trouble? • Malformed Traffic • What information is necessary to create repeatable ‘application layer’ flows? • Type of flow, size of flow, etc • Others?

  9. Draft-05 Highlights and Reasons • “Shell” Methodology • More reproducible • Added back ‘security’ • Why shy away when application layer already difficult • Maintain ‘fuzzing’ aspect • Random but repeatable

More Related