Introduction to System-on-Chip Functional Verification

1. Introduction to System-on-Chip Functional Verification Taking On The Verification Challenge With Better Performance, Capacity, and Productivity Avery Design Systems2 Atwood LaneAndover, Ma 01810978 689 7286 www.avery-design.com

2. Seminar Agenda • The state of functional verification today • Scope and scale • Current best practices • Technology • Methodology • Current problems and challenges • System-On-Chip • Large-scale systems designers • The future of functional verification • Industry Trendlines • Avery’s perspective and vision • Avery solutions and products

3. What is Driving Functional Verification? • Verification requirements grows at a multiple of Moore's Law • 10X for ASICs • 100X for ASIC-based systems and SOCs which include embedded software • Verification Complexity = ƒ(Architectural Complexity, Amount of Reuse, Clock Frequency, System Software Worst-Case Execution Time, Engineer Experience) 10B 100M 1M 2002 Verification cycles 1996 1990 Effective gate count 100K 1M 10M Verification productivity level increases lags all other aspects of design!

4. Design Complexity 30% 70% Verification Design Time to Market Growing Design Verification Investment... • Increasing design complexity imposes a thorough functional verification process • Verification needed at all levels : Block - Module - System

5. Embedded System /SOC Host System Embedded System Application Embedded OS/Firmware Device Drivers Hardware Devices How SOCs and Embedded Systems Exponentially Expand the Scope and Scale of Verification ? System Applications Operating System Device Drivers Hardware Devices System Verification Target Host System ASB-PCI Bridge • Multiple core functions • On-chip interfaces protocols • Multiple off-chip interface protocols • Embedded processors, firmware, OS

6. What Kinds of Verification Are Being Used? • Architectural verification Custom performance modeling, Algorithm models (MatLab), Formal Model Checking • Implementation verification • Circuit-level Spice, Fast Spice, Symbolic simulation, Formal Equivalency • Gate/Structure Logic Simulation, Formal Equivalency, HW Emulation • Timing Static Timing Analysis • RTL Logic Simulation, Formal Model Checking, HW Emulation • Manufacturing verification • At-speed Logic Simulation • Manufacturing defects ATPG, BIST

7. The SOC Verification Process Architecture Spec Performance Analysis Algorithm/Architecture Validation ASIC Functional Spec Verification Test Plan Test Assertions Test Streams Functional Coverage Monitors Result Checkers Protocol Verifiers Reference Models/Golden Results If you start verification at the same time as design with 2X the number of engineers, you may have tests ready when the ASIC RTL model is done RTL Modelling Verification Target Configuration

8. A Closer Look at Implementation Verification • System verification: Does the overall design work? - Run application : check end result • Module verification: Does the communication between blocks work? - Check blocks synchronization, latency… • Block verification: Does an input stream generate the expected output behavior? - Check block functionality and corner cases.

9. Finding/fixing bugs costs in the verification process System Time to fix a bug Module Block Design integration stage • Chip NREs increasing making respins an unaffordable proposition • Average ASIC NRE ~$122,000 • SOC NREs range from $300,000 to $1,000,000 RESPIN

10. Current verification practice • High quality verification is only done at system level • Self-checking methodology is standard • At the end of the system verification a high confidence level is reached • Module level verification is typically NOT done • It is felt that too much work is involved and little pay-back • Nobody is really responsible for module verification

11. Too much verification / Debug needs to be done at system levelResulting in a long verification process (70% of the design cycle time) Current verification practice (2) • Block level verification is done “ad-hock” • Often stimuli-only and visual inspection of simulation results • Structural coding style instead of behavioral • Testbench derived from RTL implementation and not specification • Testing what is implemented and not what is specified • Testbench typically not well structured / everything in a flat process • difficult reuse, hard to read and maintain; hard to document

12. System verification • Should be used as an overall application validation • Not suited for finding RTL implementation bugs • Simulation end-state validation • Do I get the correct end state results? • The sequence of operation is not relevant • Signal activity is not relevant • Typical approach • Run application / test case and check that at the end I get the expected memory / register image • It is too hard to define the sequence of operation; a cycle accurate circuit model is typically not available. • Engineering type • Application specialist; not necessarily RTL design / VHDL literate.

13. System verification prerequisites • Test plan • Application requirements • Overall performance tests • Extensive block and module verification COMPLETED • Finding / debugging RTL bugs at system level is extremely time consuming • RTL designers might not be available to support debug process • Long iteration cycles • Re-verify block and module functionality before restarting system verification • Implies new synthesis runs Finding Bugs late in the design process can substantially delays tape-out. Bugs will/should be found but their number should be minimized by extensively verifying Blocks and Modules

14. Mn M2 M1 M3 Module Verification • Required to verify inter-block communication / sub-system functionality • Module-level testbench characteristics • Bus protocol generator and checkers • Reuse of module-level generators B1 B5 B3 ... B2 B4

15. Module verification goals • Verify module operation • Verify inter-block communication • Verify module performance • sufficient cache-memory and FIFOs size • sufficient inter-block bus bandwidth • optimal processing power partitioning • Pre-requisite : Block-level verification COMPLETED • Engineering type • Senior designer / Lead designer • Verification engineer

16. Data flow2 Checker 1 Checker 2 Data flow1 Reset Block Wait Access Read Write Block verification • Used during RTL design to verify block operation • Block-level testbench characteristics • Custom designed to verify basic operation & corner cases • Cycle accurate and timing accurate for post-synthesis analysis • Complements static timing analysis

17. Block verification goals • Verification goal • Debug RTL implementation • Verify implementation is according to spec • Verify block post-synthesis netlist • Engineering type • RTL designer • Possibly not the same person who did the RTL implementation • Testbench architecture • simple data flow • one process for each data stream • Process • define test cases before RTL implementation • develop test cases before or during RTL implementation

18. Best practice for a Block and Module testbench • Easy and fast to write for RTL designer • use HDL language at behavioral level • Self-checking : mandatory for efficient verification • Visual inspection is error-prone especially in debug cycle • Hierarchically structured • allows abstraction: dataflow - operation - signal • Reusable • direct reuse of (protocol) generators and checkers from block to modules • reuse from project to project when same basic protocol is used : • ARM AMBA Bus, ATM-UTOPIA, PCI, SONET,...

19. What’s Considered Average Today? Applications Total Regression Single Test Cycles Cycles Length (Cycles) • Voice over Packet (VoP) chip-set embedded solution • G.7XX, G.168, DSL, IP • 1-2M gates • Long-haul optical transport system • 40 Gbps per wavelength • 3.2 Tbps fiber capacity • 622 MHz (SONET/SDH 192), 322K bits/frame (STS-192) • 25M gates / wavelength • Terabit router • OC-3 to OC-48, ATM • 30 M gates • Intel P4 microprocessor • 45 M transistors 1010 107 105 1013 1010 107 1012 109 106 1016 1012 106

20. Functional Testing Methods

21. Functional Testing Methods Cont’d • Simulation-based • Directed testing • Generate testcases for each test assertion and related test parameters • Interfaces exercised independently • Block, module, and system-level • Pseudo-random testing • Generate testcases for one or more test assertions concurrently • Test parameters selected randomly from enumerated list • Multiple interfaces exercised concurrently • Block, module, system-level • Auto-directed testing • Automatic functional test generation • Deterministic process using SAT solver engine(s) • Guided by functional coverage targets • Formal methods • Formal model checkers • Prove design complies with design properties • Or not, generate counter examples • Hybrid, Semi-formal checkers • Verifies specific assertions/properties in the context of actual simulation • Plus, searches limited state space around seed state (assertion triggers) for non-compliance and generates counter examples

22. Measuring Progress of Verification • How do you know you’re getting close? • Test pass rate • Bug occurrence rate • How do you know you’ve testing everything • HDL code coverage (line, state, state arcs, expression) • Micro-architectural features exercised (normal, corner cases) • IO and System Protocols Exercised

23. Putting together the SOC Verification Environment Puzzle

24. So What’s The Answer?

25. Industry Trendlines (and Hype) • Design productivity • C++ based design will supercede HDL-based design • Faster simulation through abstraction • “…Using the same language for HW and SW must make HW-SW co-design and verification easier” • “…Software engineers can lend a hand designing the hardware” • Behavioral synthesis • FPGAs will replace ASICs • “I’ll finish the design and verify it in the lab!” • Verification productivity • High-level Verification Languages (HVLs) are the only way to implement transaction-level verification • Formal model checking is on the verge of becoming practical

26. What is the Industry Doing? • Use new language for verification other than design • Synopsys’ Vera, Verisity’s e-language, Accellera’s SUGAR • Use C or C++ for design and verification • Synopsys’ SystemC, Cadence’s TestBuilder • Next generation “Verilog” HDL and simulator • SystemVerilog Language Standard builds on IEEE-1364-2001 • Co-design’s Superlog combines C constructs and Verilog together • Extend current Verilog simulators • Avery’s TestWizard adds verification features to Verilog IEEE-1364-1995, VHDL 1076 (VCS, NC-Verilog, MTI) Draft standard by June 2003 Withering? Draft standard by June 2003

27. Pros and Cons for New Language Approach • Language is specific to address verification • But 80% of verification features are the same as design features • Verification engineers have their own language • Divides design and verification into 2 separate groups • Performance penalties to pass data between verification and design • Difficult for verification to get design data

28. Pros and Cons of C and C++ Approach • Everyone knows C, a lot know C++ • Multi-bit operators can be easily be overloaded as C++ operators • You don’t care about timing, X, and design contains no bus • Fast simulation, some people claim that simulation is 100X faster than Verilog • But the design description level was different between the two • There is no inherent speed advantage if models are described at the same level • Difficult to handle large amount of parallelism • Pay nobody • Difficult to correlate design bugs to C or C++ model • Difficult to do incremental refinement of the design • Synthesis and other tools are not mature • Cannot accept legacy code

29. Pros and Cons of Next Generation Approach • It’s not Verilog, it’s not C • No backwards compatibility • Long standards approval process • Requires widespread compliance by EDA vendors • Verification-centric enhancements do not have same priority as design and synthesis-centric enhancements

30. Verification Crisis is Not Solvable by Raising Abstraction Level Alone • Verilog and C are at the same abstraction level with only data structure limitation • Design’s today only use a very limited subset, restricted by Synopsys synthesis subset • Systematic exhaustive testing of designs is against human nature Verification crisis is only solved with breakthroughs in methods, technologies, process

31. The Avery Vision • Tackle the 100 M gate systems verification challenge through better • Productivity • Performance + Capacity • Economics • Innovate new verification technologies and flows addressing RTL, HW-SW systems, and IP • Build on customers investments in tools, language standards, and engineering “know how”

32. The Avery StrategyDefining Principles for Effective Functional Verification • Transaction-oriented verification • Raise test generation abstraction and tools to match rise in architectural complexity • Reusable verification IP • Standards-based design increases design and interoperability and designer productivity • System integration issues • System-level simulation performance/capacity • Early HW-SW integration • Design management • Build on existing tools and methods • Use only existing standard implementation languages

33. The Avery SolutionRequired Elements for Effective Functional Verification • Verilog/VHDL/C-based testbench automation • Up to 3 X better efficiency using native Verilog and C/C++ • Test development • Coverage analysis • Assertions and protocol verification • Distributed parallel simulation • 5-10 X increase in performance and capacity vs. stand-alone HDL simulators through parallel simulation processing

34. The Avery Solution Cont’dRequired Elements for Effective Functional Verification • Hardware-Software co-verification • Earlier system SW integration and compliance validation • Integrated HDL and C/C++ verification flow for IP • C/C++, CycleC, SystemC, Cynlib, and SpecC • Leverage all the best EDA tools and solutions • Access to best-in-class simulators

35. TestWizard Is…

36. TestWizard is a Complete Transaction-level Verification Solution • Transaction-based Verification • Decouple testcase content and signal-level interfaces • Apply pseudo-random test case generation to generate more thorough set of testcases • Eliminate manual result checking and use only dynamic self-checking tests and transaction database logging • Assertion-based Verification • Use automatic protocol assertion checking to ensure system compliance during every simulation • Coverage-based Verification • Formally measure functional test coverage vs. design architecture • Check all properties & scenarios have been tested

37. Key Benefits of TestWizard • Compared to conventional HDL methods • High-level structured methods and psuedo-random generation generates comprehensive workloads to test design • 3X more productivity • Up 10X better code density • Closed-loop method achieves 100% functional coverage • Assertions provide “always on” validation of protocols

39. TestWizard Verification is … • Transaction modeling using complex data structures • Test generators generate random transaction sequences based on test parameters • Transactors stream transactions in/out of DUT • Result checking is dynamic self-checking using transaction database queries • Assertion-based verification checks protocol compliance “on the fly” • Closed-loop functional & assertions coverage analysis adapts traffic shape for more effective & efficient testing

40. #1: Think Transactions • Container of basic units of data and control that a system works on • Requires complex datatypes to implement • Decouple tests/transactions and signal-level interfaces

41. TestWizard Supports Complex Datastructures • Records • Hierarchical (i.e. Records of records) • Record Pointers • Dynamically allocated/deallocated • Dynamic field sizing • Unions support composite types • Supports reuse strategies • Extend base records • Integrated with other TestWizard functions • Randomization • Default and instance overriding of fields • Record streaming functions simplify BFM/transactors • Transaction database logging and SQL-like queries to simplify transaction-level verification • List types and list access functions • Mutable and hierarchical provides unstructured data aggregation • Supports string, numeric constants and numeric ranges

42. Examples of Complex System Transactions: Internet Protocols

43. #2: Random Traffic Shaping Parameters Protocols, #packets, interval time, IP src/dst addresses, MPLS labels, Time-to-live, hop-limit, MPLS label stack depth, LDP messages

44. TestWizard Supports Intelligent Random Testing in Verilog/VHDL • Automated stream generation modes • Directed • Constrained Random • Boundary case pseudo-random • Weighted Distributions • Related case pseudo-random • Cyclic pseudo-random • Filter functions • Enumerated value ranges • Uses list types for value set definitions

46. #3 Transaction Database Logging • Stores transaction records • Supports multiple transaction record types • Timestamps • Automatically manages database by size and age

47. #4: Dynamic Result Checking • Transaction Database required to verify complex transaction processing like: • Out-of-order transactions • Multi-protocol processing • Modified headers • QoS and SLA • Supports complex searches • Record Type, Field Type, Field Value, Time

48. #4a: Dynamic Checking Examples • Does the router generate ICMP Type 11 (timeout messages) back to IP source address when TTL and HLIM reaches zero • Predictor stores IP pkt in Port #0 egress “expect xlog” db and starts timeout assertion thread • Router egress emits ICMP Type 11 packet • Dynamic checker examines “actual” egress pkt and compares to “expected” egress pkt; eventual match or timeout assertion violation • Is IPv6 QoS met for certain specified flow labels • Does router perform correct label stack operations • Does router correctly respond to LDP label map requests for good and bad labels

49. #5 Assertions • “On the Fly” protocolverification of System-level and Interface protocols • Powerful & concise temporal property semantics to Verilog/VHDL • Repeating sequences • Temporal ranges • Conditional sequences • Logical or/and • Highly reusable • Comparable to Sugar (Accellera FPSL WG) • Sugar-to-TestWizard translator available soon

50. #5 Assertions cont’d Property: For all read or write transactions in the address range: ‘hAA:-hBB, data transfers of 1-16 bytesare allowed. Data transfers can completeout-of-order. On transaction completiondetection, byte counter must match initial transaction header byte count TestWizard assertions support complex Overlapping sequences using threads and tags