Current Status and Challenges of SoC Verification for Embedded Systems Market

1. Current Status and Challenges of SoC Verification for Embedded Systems Market November, 2003 Chong-Min Kyung, KAIST, Daejeon, Korea

2. Various Embedded Mobile Systems Data Processing Consumer Media Center Audio Desktop PC DVC DTV NotebookPC DSC DVD Game Console MP3Player SmartPhone PDA CreditCard CarNavigation CellularPhone Internet Telematics Automotive & etc. Communication Bluetooth

3. Difficulties in Embedded Systems Design(What’s special in ES design?) • Real operating environment of ES is difficult to reproduce. -> In-system verification is necessary. • Total design flow of ES until the implementation is long and complex. -> Verification must be started early. • Design turn-around time must be short as the life-time of ES itself is quite short. -> Short verification cycle

4. Most Important Issues in Embedded System Verification • 1. Verify Right : • Always make sure you have correct specifications to start with. (Frequent interaction with SPECIFIER, customer, marketing, etc.) • In-System Verification • Check Properties in Formal Techniques. • 2. Verify Early • System-level, Heterogeneous Models, SW-HW • 3. Verify Appropriately • HW-SW Co-simulation • 4. Verify Fast • Hardware Acceleration, Emulation

5. Strongly Required Features of Embedded System Verification • Accommodate Multiple Levels of Design Representation • Exploit Hardware, Software and Interfacing mechanisms as Verification Tools

6. Agenda • Why Verification ? • Verification Alternatives • Languages for System Modeling and Verification • Verification with Progressive Refinement • SoC Verification • Concluding Remarks

7. Trend of Verification Effort in the Design • Verification portion of design increases to anywhere from 50 to 80% of total development effort for the design. 1996 300K gates Code Verify (30 ~ 40%) Synthesis P&R Code Verify (50 ~ 80%) Synthesis P&R 2000 1M SoC Verification methodology manual, 2000- TransEDA

8. Percentage of Total Flaws • About 50% of flaws are functional flaws. • Need verification method to fix logical & functional flaws From Mentor presentation material, 2003

9. Another recent independent study showed that more than half of all chips require one or more re-spins, and that functional errors were found in 74% of these re-spins. • With increasing chip complexity, this situation could worsen. • Who can afford that with >= 1M Dollar NRE cost?

10. Bug Fixing Cost in Time • Cost of fixing a bug/problem increases as design progresses. • Need verification method at early design stage Cost of Fixing a Problem Behavioral Design RTL Design Gate Level Design Device Production Verification methodology manual, 2000 – TransEDA

11. Verification Performance Gap; more serious than the design productivity gap • Growing gap between the demand for verification and the simulation technology offered by the various options. Verification Performance Gap SOC HW simulation acceleration (<10KCPS) Cycle-base simulation (<1KCPS) Verification complexity Simulation performance Event-base simulation (<10CPS) Complex ASIC Medium ASIC Small ASIC Design complexity System-on-a-chip verification, 2001 – P.Rashinkar

12. Completion Metrics; How do we know when the verification is done? • Emotionally, or Intuitively; • Out of money? Exhausted? • Competition’s product is there. • Software people are happy with your hardware. • There have been no bugs reported for two weeks. • More rigorous criteria; • All tests passed • Test Plan Coverage • Functional Coverage • Code Coverage • Bug Rates have flattened toward bottom.

13. Verification Challenges • Specification or Operating Environment is Incomplete/Open-Ended. (Verification metric is never complete like last-minute ECO.) • The Day before Yesterday’s tool for Today’s Design. • Design productivity grows faster than Verification productivity.

14. Agenda • Why Verification ? • Verification Alternatives • Simulation • Hardware-accelerated simulation • Emulation • Prototyping • Formal verification • Semi-Formal (Dynamic Formal) verification • Languages for System Modeling and Verification • Verification with Progressive Refinement • SoC Verification • Concluding Remarks

15. Overview of Verification Methodologies Prototyping Faster speed, closer to final product Emulation Hardware Accelerated Simulation Simulation Basicverificationtool Semi-formal Verification Formal Verification Bigger coverage

16. Software Simulation • Dynamic verification method • Bugs are found by running the design implementation. • Thoroughness depends on the test vector used. • Some parts are tested repeatedly while other parts are not even tested. Other parts are not even tested. Testbench DUV a = 1; #20 b = 1; $display (“status is = %d”,c); ... Some part of the design is tested repeatedly.

17. Software Simulation • Pros • The design size is limited only by the computing resource. • Simulation can be started as soon as the RTL description is finished. • Set-up cost is minimal. • Cons • Slow (~100 cycles/sec) ; Speed gap between the speed of software simulation and real silicon widens. (Simulation speed = size of the circuit simulated / speed of the simulation engine) • The designer does not exactly know how much percentage of the design have been tested.

18. Hardware-Accelerated Simulation • Simulation performance is improved by moving the time-consuming part of the design to hardware. • Usually, the software simulation communicates with FPGA-based hardware accelerator. Hardware Accelerator Simulation environment Testbench Module 2 is synthesized & compiled into FPGAs Module 0 Module 1 Module 2

19. Hardware-Accelerated Simulation • Pros • Fast (100K cycles/sec) • Cheaper than hardware emulation • Debugging is easier as the circuit structure is unchanged. • Not an Overhead : Deployed as a step stone in the gradual refinement • Cons (Obstacles to overcome) • Set-up time overhead to map RTL design into the hardware can be substantial. • SW-HW communication speed can degrade the performance. • Debugging of signals within the hardware can be difficult.

20. Hardware-Accelerated Simulation • Challenges • Overall speed depends on the communication between simulator and hardware. • Execution time decomposition in a typical case of a PCI-based hardware accelerator ; • SW simulator + PLI/FLI + Driver overhead : 38%  It is desirable to reduce the driver call overhead. • PCI overhead : 44%  Can be reduced by using DMA data transfer. Hardware Accelerator 18% SW Simulator + PLI/FLI + Device Driver 38% PCI overhead 44%

21. Emulation • Imitating the function of another system to achieve the same results as the imitated system. • Usually, the emulation hardware comprises an array of FPGA’s (or special-type processors) and interconnection scheme among them. • About 1000 times faster than simulation. Prototyping Emulation Hardware Accelerated Simulation Simulation

22. Emulation • User logic design is mapped to emulation board with multiple FPGA’s or special processors. • The emulation board has external interconnection hardware that emulates the pins of final chip. Emulation hardware with multiple FPGAs Logic design Design mapping & > & + External pins

23. Emulation • Pros • Fast (500K cycles/sec) • Verification on real target system. • Cons • Setup time overhead to map RTL design into hardware is very high. • Many FPGA’s + resources for debugging  high cost • Circuit partitioning algorithm and interconnection architecture limit the usable gate count.

24. Emulation • Challenges • Efficient interconnection architecture and Hardware Mapping efficiency for Speed and Cost • RTL debugging facility with reasonable amount of resource • Efficient partitioning algorithm for any given interconnection architecture • Reducing development time (to take advantage of more recent FPGA’s)

25. Prototyping • Special (more dedicated and customized) hardware architecture made to fit a specific application. Prototyping Emulation Hardware Accelerated Simulation Simulation

26. Prototyping • Pros • Higher (than emulation) clock rate (over 1M cycles/sec) due to specific design of prototyping board. • Components as well as the wiring can be customized for the corresponding application. • Can be carried along. (Hardware Emulation? Forget it!) • Cons • Not flexible for design change (Every new prototype requires a new board architecture. / Even a small change requires a new PCB.)

27. A Prototyping Example Switch board • Prototype of 4-Port Gigabit Ethernet Switch • Two Xilinx Virtex-E 2000 FPGAs are on FPGA board. • Four FPGA boards are used. • Processor board contains PCI bridge and MPC860 microprocessor. Xilinx FPGA Processor board PCI bridge MPC860 microprocessor Courtesy of Paion, Inc.

28. Overview of Verification Methodologies • Formal verification • Application of logical reasoning to the development of digital system • Both design and its specification are described by a language in which semantics are based on mathematical rigor. • Semi-formal verification • Combination of simulation and formal verification. • Formal verification cannot fully cover large designs, and simulation can come to aid in verifying the large design. Simulation Semi-formal Verification Formal Verification More complete verification

29. Formal Verification • Objective • Check properties of model with all possible conditions • Pros • Assures 100% coverage. • Fast. • Cons • Works only for small-size finite state systems. • Uncomfortable due to culture difference (E.g., engineers are not familiar with the use of temporal logic used for “property” description in Model Checking)

30. Formal Verification Techniques • Symbolic Simulation • Equivalence Checking • Theorem proving • Model Checking

31. Symbolic simulation • Object • Compute an expression instead of a value • Advantage • More coverage per simulation • Disadvantage • Not good with state machines • Does not support temporal logic • Industrial success story • Symbolic trajectory evaluation • Tackle switch level design (Randal E. Bryant – COSMOS)

32. X a X X a X X X a X X X X a Ex) 4-bits shift register • Specification • If ‘a’ is applied to din, then four cycles later, ‘a’ appears at dout a din dout d0 d1 d2 d3

33. Formal Verification : Equivalence Checker • Equivalence checker compares the golden model with the refined model. • Functional representations are extracted from the designs and compared mathematically. • Pros • Exhaustive design coverage • Very fast • Cons • Memory explosion • Tools such as LEC (Verplex), Formality (Synopsys), FormalPro (Mentor) supports Equivalence checking. ? GoldenModel RefinedModel =

34. Equivalence checking • Object • Checks equivalence of two models • RTL vs. gate • Before optimization vs. after optimization • Before test insertion vs. after • Reference model vs. implementation • Advantage • Guarantee functional equivalence of two models for all input values • Disadvantage • Needs golden reference model • Targets implementation errors rather than design bugs

35. Theorem proving • Deductive verification • Use axioms and proof rules to model the system (formal system). • State the property to be verified as a theorem of this formal system. • Derive this theorem with the help of a theorem-prover which generates rules derivable from axiom and premises. • Useful for verifying algorithm • Disadvantage • Very hard to automate. • Requires user interaction. • Deriving the formal system can be quite cumbersome. • Requires an expert to use the theorem-prover. • Industrial success story • AMD K7 floating point verification • Intel instruction decoder verification

36. Ex) CMOS inverter • Element model (axiom, definition) • pwr(p) ≡ (p=T) • Ptran(g,a,b) ≡ (!g  (a=b)) • Ntran(g,a,b) ≡ (g  (a=b)) • gnd(p) ≡ (p=F) • Inverter Circuit model (premise) • Inv(i,o) ≡∃p1,p2 . pwr(p1) & Ptran(i,p1,o) & Ntran(i,o,p2) & gnd(p2) • Specification (theorem) • Inv(i,o) ≡ (o = !i) pwr p1 i o p2 gnd

37. Formal Verification : Model Checking • Model checking verifies that the design satisfies a property specified using temporal logic. • Computational Tree Logic • Specify the temporal relationship among states in FSM with temporal operators; • A (always), E (exists) – path quantifier • G (global), F (future), X (next), U (until) – temporal modality ? ComputationalTree Logic Design 

38. Model checking • Object • Check properties of model with all possible conditions • Advantage • Can be fully automated • If the property does not hold, a counter-example will be generated • Relatively easy to use • Problem • Works (well) only for finite state systems. • Needs abstraction or extraction which tend to cause errors

39. Ex) Highway traffic light control • Specification • Safety • Avoid collision – The light is not green in both directions at the same time • Liveness • A car on the farm road eventually crosses the intersection Farm road High way

40. Formal Verification • Challenges • The most critical issue of formal verification is the “state explosion” problem. • The application of current formal methods are limited to the design of up to 500 flip-flops. • Researches about complexity reductions are : • Reachability analysis • Design state abstraction • Design decomposition • State projection

41. Semi-Formal Verification - Assertion • Assertion-based verification (ABV) • “Assertion” is a statement on the intended behavior of a design. • The purpose of assertion is to ensure consistency between the designer’s intention and the implementation. • Key features of assertions • 1. Error detection : If the assertion is violated, it is detected by the simulator. • 2. Error isolation : The signals related to the violated assertion are identified. • 3. Error notification : The source of error is reported to the user.

42. Semi-Formal Verification - Assertion • Example of assertion-based bug detection Identify signals related to the violated assertion PCI DMA Controller event devsel : if (FRAME=0) [1..4] (DEVSEL=0) assert(devsel); PCI “devsel” assertion is violated! Report to the user!!

43. Assertion Example - Arbiter req[0] req[2] Arbiter ack[0] ack[2] req[1] req[3] ack[1] ack[3] for(i=0; i<arbit_channel; i++) { event b_req_start[i] : posedge req[i]; event e_req_granted[i] : if en then if (b_req_start) then ( #[min_lat .. max_lat] (grants[i] || !reqs[i]) ); event e_granted_only_if_req[i]: if (!reqs[i]) then !grants[i]; event e_highest_grant[i] : if (grants[i]) then (priority[i] == highest_value); } assert(e_req_granted[i]); assert(e_granted_only_if_req[i]); assert(e_highest_grant[i]); } request is asserted After request, grant should be eventually asserted within allowed latency Grant is asserted only when request is asserted. The granted one has the highest priority. Set assertion on the above events.

44. Semi-Formal Verification - Assertion • Simulation Quality of assertion-based verification Simulation with assertions Efficiency of assertion Number of bugs found Formal verification Simulation Time, Effort Setup testbench Describe assertions By IBM in “Computer-Aided Verification” 2000

45. Semi-Formal Verification - Coverage • Coverage-directed verification • Increase the probability of bug detection by checking the ‘quality’ of stimulus • Used as a guide for the generation of input stimulus Test Plan (Coverage Definition) Directives Random Test Generator Test Vectors Coverage Reports Coverage analysis Simulation

46. Semi-Formal Verification - Coverage • Coverage metrics for coverage-directed verification • Code-based metrics • Line/code block coverage • Branch/conditional coverage • Path coverage • Circuit structure based metrics • Toggle coverage • Register activity • State-space based metrics • Pair-arcs : usually covered by Line + condition coverage • Spec.-based metrics • % of specification items satisfied

47. Semi-Formal Verification - Coverage • Coverage Checking tools • VeriCover (Veritools) • SureCov (Verisity) • Coverscan (Cadence) • HDLScore, VeriCov (Summit Design) • HDLCover, VeriSure (TransEDA) • Polaris (Synopsys) • Covermeter (Synopsys)

48. Semi-Formal Verification • Pros • Designer can measure the coverage of the test environment as the formal properties are checked during simulation. • Cons • The simulation speed is degraded as the properties are checked during simulation. • Challenges • There is no unified testbench description method. • It is difficult to guide the direction of test vectors to increase the coverage of the design. • Development of more efficient coverage metric to represent the behavior of the design.

49. Speed Comparison Speed (Cycles/sec, log scale) 10MHz 1~10MHz 500KHz 1MHz 100kHz 100 kHz 10 kHz 50-70Hz 100Hz 0 kHz Software Simulation Hardware- Accelerated Simulation (from Quickturn/Dynalith Presentation) Hardware emulation (from Quickturn presentation) Prototyping Semi-formal (Assertion-based verification)

50. Design Complexity