Formal Verification at IBM: Applications and Technology Overview

1. Viresh Paruthi, Jason Baumgartner IBM Systems and Technology Group, Austin TX, USA Formal Verification at IBM:Applications and Technology Overview

2. Outline • Introduction • Formal Verification @ IBM • Technology • Conclusion Ru!eBase SixthSenseedition

3. Introduction to Hardware Verification • Numerous types of verification relevant to hardware design • Also timing analysis, circuit analysis, protocol analysis, … always @(posedge clk) begin if ( r ) then p <= 0 else p <= p+1; end if; end Netlist Schematic HDL Layout IC Equivalence Checking Equivalence Checking Wafer Test Layout vs Schematic

4. Introduction to Hardware Verification • We focus upon functional verification and equivalence checking • Though the techniques we discuss may also be applied to architectural models, protocol models, software-like models, … • As long as they are synthesizable always @(posedge clk) begin if ( r ) then p <= 0 else p <= p+1; end if; end … IEEE Standard 754-2008 Netlist HDL Equivalence Checking Functional Verification

5. Avenue N Avenue E I=(--,nC) NG I=(--,C) I=(--,--) O=(G,R) NY EY O=(Y,R) O=(R,Y) O=(R,G) I=(--,--) I=(C,--) EG I=(nC,--) Traffic Light Controller (TLC)

6. Verification Step 1 • Unit testing, a.k.a. designer verification • Check for intended behavior against a selection of input vectors I=(--,nC) NG I=(--,C) I=(--,--) NG ? I=(--,nC) O=(G,R) I=(C,--) NY EY NG EY NG O=(R,Y) O=(Y,R) I=(--,--) NY EG I=(--,C) O=(R,G) I=(--,--) ? I=(--,--) I=(C,--) EG I=(nC,--) • Investigate streams of state transitions and outputs • Light weight check, low “coverage”

7. Verification Step 2 – Specification • Properties – True for every possible execution • Safety – Nothing bad happens • N/S and E/W lights should never be GREEN at the same time • None of the lights should transition from GREEN directly to RED • Liveness – Something good eventually happens • Both N/S and E/W lights will turn GREEN eventually “Verification is only as good as the specification”

8. Verification Step 3 • Simulation • Systematically check the design satisfies its specification • Against directed or random input streams I=(--,nC) ? NG I=(--,C) I=(--,--) NG ? I=(--,nC) O=(G,R) NY I=(--,C) I=(--,--) I=(C,--) NY EY NG EY NG O=(R,Y) O=(Y,R) I=(--,--) NY EG I=(--,C) O=(R,G) I=(--,--) ? I=(--,--) I=(C,--) EG I=(nC,--) • A “(random) walk” of the state space of the design • Incomplete – cannot rule out presence of bugs

9. A (32-bit) B (32-bit) Add / Mul / Div / Rot… C (64-bit) Source: Comprehensive Functional Verification, Wile, Roesner, Goss We done…? Let’s take a simple real world ALU Number of patterns to verify a single op: 18,446,744,073,709,551,616 A super-fast simulator running at 1 MHz or 1 million patterns/sec will take 584,942 years to verify a single op! June 1994: Pentium div bug costs $$$$$$$ in parts replacement cost to Intel

10. NG NG I=(--,nC) I=(--,nC) NY NY I=(--,--) I=(--,--) EY EY Verification Step (4) • Formal Verification • Rigorously prove the design satisfies its specification EG NG I=(--,--) I=(--,nC) I=(C,--) EY I=(--,nC) NG NY EG I=(--,nC) I=(--,--) NG I=(--,--) EG I=(nC,--) I=(--,C) I=(--,--) NG O=(G,R) I=(C,--) NG EG NY I=(--,C) I=(--,--) NY EY I=(--,--) O=(R,Y) O=(Y,R) I=(nC,--) I=(C,--) EG O=(R,G) I=(--,--) I=(C,--) EG I=(nC,--) EG I=(nC,--) Step 2 Step 4 Step 0 Step 1 Step 3 • Exhaustive exploration of the state space of the design • Complete – conclusively establishes correctness / absence of bugs

11. EY EY Verification Step (5) • Semi-formal Verification • Alternate between visiting one-state-at-a-time and exhaustive search • Resource bounded formal exploration amplifies simulation results ? NG I=(--,nC) I=(--,nC) NG ? I=(--,C) I=(--,--) NG NY EG I=(--,--) I=(--,nC) I=(C,--) O=(G,R) I=(--,--) I=(--,--) EG NG I=(nC,--) NY EY NG O=(R,Y) O=(Y,R) ? I=(nC,--) I=(C,--) EG O=(R,G) I=(--,--) I=(C,--) EG I=(nC,--) EG I=(nC,--) • Incomplete, high(er) coverage • Happy medium

12. Verification Step 6 – Debug! • Liveness property fails • Reveals a recurring path in the logic where E/W light never goes GREEN • Because a legal path exists where no car arrives on E/W route I=(--,nC) NG I=(--,C) I=(--,--) I=(--,nC) O=(G,R) NY EY NG NG O=(R,Y) O=(Y,R) O=(R,G) I=(--,nC) I=(--,--) I=(C,--) EG I=(nC,--) • The driver needs to specify a “fairness” constraint • The E/W light have a car (C) arrive every so often

13. Formal Verification Concepts Design-Under-Test entity .... end ...; architecture.... Driver Checker .... ... end ..... • Randomness / Non-determinism • Foundation of the exhaustive exploration with considering all (legal) valuations • Every input at every step of the evaluation • In contrast to simulation which applies a single value at every step • “Passive” Testbench • Define legal behavior of the inputs • Encode stimulus generator (logic’ally) • Specify “constraints” (assumptions) to rule out illegal stimulus • Risks “over constraining” – mask out valid stimulus compromising completeness • In contrast to “active” testbench in simulation – applies stimulus every cycle “Verification is only as complete as the driver” Testbench

14. FV Task Life Cycle Understand Design-Under-Test (DUT) function Write the driver / environment (constraints) Write checkers and verify to validate environment Write checkers and assertions (spec) and verify DUT Regress

15. TLC is Arbitration! • Arbiters • Restrict access to a shared resource • E.g. bus access where all requestors cannot be serviced concurrently Requests Arbiter Fairness Arbiter Grants • Verification goals: • Starvation, Mutual Exclusion / Collision

16. TLC “Systems” • Tandem operation • Dynamic coordinated control • Sensors, Adaptive timers • Traffic flow • Synchronization • Vehicles / Time • Back-pressure

17. System-level FV Core #1 Core #2 Core #8 L2 / L3 Cache L2 / L3 Cache L2 / L3 Cache ........ Bus Arbiter MC MC • Arbitration of large number of requests from various sources • A multi-stage arbitration organized as a reduction trees • Liveness check is insufficient, accurate request-to-grant delay crucial aspect • Ascertain performance, throughput… • FV techniques are able to accurately measure worst case performance • Decouple fairness logic from core arbitration • Check grant of requests “boundedly”

18. Lower level Black-boxed Chip Unit 1 Unit m . . Wrapper 1 Leaf level Macro 1 Macro 2 Macro n . . . Design hierarchy FV Scalability Smarts • Reductions • Include least amount of relevant logic • E.g. L3 hangs – include a few CO/CI machines • Localization – carve out logic sufficient to verify spec • E.g. one-hot mux-selects need logic up to source • Abstractions • Replace actual logic with behavioral with sufficient detail to verify • Caveat: Abstractions by definition are “leaky” • E.g. verify overall bus arbitration with abstracting on-chip traffic • Decomposition • Divide-and-conquer • Verify components in isolation and compose results • Requires verifying interface protocol rigorously Vehicles/ Time

19. FV Planning Candidates for FV application • Protocol verification • Mission-critical components/aspects • Early-RTL debug and exploration • Logic with low DV coverage • Clock-domain crossings • Logic with liveness, fairness requirements • Control-intensive units • Aiding DV to hit elusive coverage goals • Sequential equivalence checking • Recreating lab bugs

20. Outline • Introduction • Formal Verification @ IBM • Evolution • Verification Methodology • Sequential Equivalence Checking • Technology • Conclusion Ru!eBase SixthSenseedition

21. Formal Verification Evolution @ IBM Paruthi FMCAD 2010 Middle Ages The Future… Early Times Modern Era 2000 2002 2006 2012 Avoid duplicate work Reusable methodologies / IP Automation, automation… Stay tuned! Advent of SFV, Parallel, SEC Larger logics verified; higher coverage Same “look and feel” as simulation SEC key to many newer methodologies Applied to small logics (~100s of registers) Manual Intensive w/ dedicated resources Required setting up of complex drivers Large scale FV application Integrated approach / DLV Out-of-the-box methodologies High speed, capacity toolsets SFV: Semi-formal verification SEC: Sequential Equivalence Checking DLV: Designer-level Verification FV Capacity = Usability High tool capacity has enabled profound methodology impact

22. Formal Verification at IBM • Vision: Bring FV to the masses • Common infrastructure → Trivial learning curve, resource savings • Shared / reusable verification IP → High ROI, tight integration • High scalability → Improved productivity Amortize development cost → Higher value proposition • Synergistic application alongside other verification disciplines • Focused on the same problems

23. Verification Technology RTL Driver/Checker PSL et al. (VHDL, Verilog) Assertions Language Compile Physical VLSI Design Tools / Model Build Custom Design Test Program Generator (GPro, X-Gen) Cycle-Based Model Constraint Random C++ Testbench Testbench Boolean Software Simulator (MESA) (Semi-) Formal Verification (RuleBase SixthSense Edition) Equivalence Check (Verity) Hardware Accelerator (Awan) Hardware Emulator

24. IBM Systems and Technology Group Integrated Approach: Design i1 . . . o MUX . . . in sn s1 • Assertion-based Verification (ABV) Designer-level Verification (DLV) • Require designers to capture assumptions as verif objects (checkers) • Accelerated debug, faster IP integration, documentation… • …and perform basic verification leveraging those • High ROI: Improved productivity / cost / schedule, efficient use of resources • Reuse events (checkers, coverage) • Proof design events with FV FV events / assumptions cross-checked One-hot Complete Driver Simple Driver Enables Stimulus Enables Integrated Checking Assertions Vhdl Simulation Semi- Formal H/W Accel Verification Assertion-Based Verification Designer-Level Verification Block-Level Verification

25. Integrated Approach: Verification • Better synergy with other verification disciplines • Formal plans drawn collaboratively with design and simulation teams • Optimized testplans via detailed reviews with simulation team • Unified view of verification “coverage” inclusive of simulation and formal • Minimize duplicate work in verif disciplines Book verification of logics in formal • LRUs, Arbitration, Debug Bus, Mux-based networks…

26. Verification Progression VPO Level Hardware Emulation Hardware / Firmware Verification VBU Level Inter-chip interactions System Level Pervasive verification Protocol analysis Recreate bring-up fails Hardware Acceleration Chip Level Hardware Verification Element Level Starvation free arbitration Software Simulation Unit Level Defined interfaces End-to-end check (e.g. FPUs) Block Level Formal Verification “Deep dive” FV Obtain proofs Find corner case bugs VBU = Virtual Bring-Up (chip) VPO = Virtual Power-On (system)

27. Driver Driver Block 2 Driver Driver Block 1 Block 1 Checker (Properties) Block 2 Checker (Properties) (Sub-) Unit Driver Driver Block 1 Block 2 Testbench Components Block 1 Checker (Properties) Block 2 Checker (Properties) Design Components Scaling Formal Testbenches • Wide-spread adoption of FV requires scalability to simulation-sized testbenches • Easier to specify well-documented functional units vs. components thereof • Simpler (constraints-based) drivers – higher productivity • Create “Templates” – blueprint to verify a certain type of logic • A cook-book approach / recipe to check complex RTL implementations • Predictable, portable, repeatable, teachable… • L2, MC, LSU, ISU…

28. High-level Modeling Support entity e1 is port (i1: in std_ulogic_vector(0 to 3); we,re: std_ulogic; o1: out std_ulogic_vector(0 to 3)); end;architecture e1 of e1 is signal ff: hl_fifo(fifo(0 to 3)(0 to 3)); begin process (ALL) begin if (we = '1') thenfifo_push(ff, i1); end if; if (re = '1') thenfifo_pop(ff, o1); end if; end process; end; • Raise level of abstraction of the testbench specification • Provide rich set of convenience functions as VHDL support library • Parameterized functions encapsulate commonly used logic constructs • Clocks generation (e.g., oscillator), edge detection (falling, rising) • Vector processing functions – one hot, parity, hamming distance… • Waveform drivers (wave, pulse), counters, delays, FIFO… • PSL (VHDL) events managed as part of unified event management support

29. Formal Verification IP Off-the-shelf IP to improve verification productivity and quality • Code reusability – faster verification times • Formalization of the spec – improved accuracy and coverage • Standardization – implementation independent packagearbiter_rules_and_driver_pkg is propertynot_bounded_starve(constN;booleanrequest, grant)is never{request;notgrant[*N]}!; … end package arbiter_rules_and_driver_pkg; Sequence start Next clock N repetitions Sequence end Checker VHDL: use vhdl_packages.arbiter_rules_and_driver_pkg.all; … assert not_bounded_starve(N, request, grant);

30. Combinational Equivalence Check (CEC) Sequential Equivalence Check (SEC) Logic 1 Logic 1 d1 init {x0, x1, …} {0, 0, …}? 0? Logic 2 d1 x d2 init s Logic 2 0? • Supports arbitrary design changes (I/O equivalent) • Retiming, power saving, redundant logic… • Explores sequential behavior of the designs • Computationally more complex than CEC • Requires 1:1 state elements mapping • Cannot handle sequential behavior • Validates next-state functions and outputs • Verify transistor- / gate-level design against RTL d2 IBM Systems and Technology Group Equivalence Checking

31. : Simulation Assertions Sequential Equivalence Checker Proof of Equality Mismatch Trace Outputs Initialized OLD Design =? Inputs Initialized NEW Design Lower levels black boxed Chip Lower levels black boxed Unit 1 Unit m . . . Lower levels black boxed Wrapper 1 Leaf level Macro 1 Macro 2 Macro 3 Macro n . . . Design hierarchy Sequential Equivalence Checking Sequential Equivalence Check (SEC) Hierarchical application scales to large enough design partitions OLD Design Supports arbitrary design changes (I/O equivalent) Explores sequential behavior of designs Retiming, power saving, redundant logic… NEW Design Initialization Data Input Constraints Game changing application of the technology End-2-end verification of entire chips Invaluable productivity advantage, resource savings

32. Outline • Introduction • Formal Verification @ IBM • Technology • Core proof / falsification techniques • Transformation-Based Verification (TBV) • Research and development • Supported languages and environments • Conclusion Ru!eBase SixthSenseedition

33. Netlist-Based Verification • A stateis a valuation to the latches / registers in the design • Initial values define the initial states • Next-state functions define the state transitions • Reachable states: which may be transitioned to from initial states • A bad state refers to one violating a property: a bug Combinational logic Input stimulus 0 1 0

34. Netlist-Based Verification: Goals • A counterexample trace, from an initial state to a bad state • Or a proof that no such trace exists • Proofs generally requires a reachable state computation Initial States

35. Verification with RuleBase SixthSense Edition Environment, Driver DUV Spec Assertions, Properties + + ? [1:n] Fail + Counter example Pass + Witness Pass vacuously Bounded pass

36. Supported languages • Hardware description languages • Verilog, VHDL, Mixed • GDL (mainly for protocol verification) • Assertion languages • PSL: standalone checker or embedded in HDL • SVA • Verification directives • Assertions, Coverage points • Assumptions • Full liveness, fairness, restrict support • Support for various trace browsers • High capacity via Transformation-Based Verification

37. Simulation (Validation vs Verification) • A “random walk” through the state space of the design • Stimulus can be random (constrained, biased); from manual testcases; from testcase generators • Very robust set of tools & methodologies available • The original method for functional verif; FV came decades later • Scalable: applicable to designs of any size • Explicit one-state-at-a-time nature severely limits attainable coverage • Cannot yield proofs; fails to expose all bugs, tedious to cope with Incomplete coverage of huge designs

38. Acceleration • Like simulation, though uses reconfigurable hardware (eg FPGAs) vs software-interpreted models • Often 10000+ faster than simulation • Enables deep evaluation such as virtual power-on • Though accelerators are expensive ($1M+) • Somewhat size-gated: model must fit available architecture • Performance degrades if testbench cannot be evaluated purely in synthesized model

39. Formal Verification (FV) • FV refers to exhaustive verification: proof capable • Use symbolic vs explicit analysis, e.g. BDDs or SAT solvers • Enables astronomically higher coverage than simulation: 2>>1000 vs 2<40 • Generally requires analysis of reachable states: size limited • Though many advanced techniques exist to overapproximate reachable states, reduce design size, abstract irrelevant behavior • Great capacity gains over past decades, though still size-limited Complete coverage of smaller designs

40. Semi-Formal Verification (SFV) • Refers to using symbolic algorithms in an incomplete way • Bounded model checking: any bug within k steps of initial states? • Semi-formal extensions: within k steps of other reachable states? • Offers astronomical coverage of symbolic algos on larger designs • Powerful bug-hunter • Though still size-limited vs. simulation and acceleration

41. prove! Transformation-Based Verification • Encapsulates engines against a modular API • Transformation engines, proof engines, falsification engines • Modular API enables maximal synergy between engines • Each (sub)problem may be addressed with an arbitrary sequence of algos • Motivation: every problem is different; different algorithm sequences may be exponentially more / less effective on a given problem • Incrementally chop complex problems into simpler problems, until tractable for core verification algos • Recall exponential relation between verification complexity and design size

42. Counterexample consistent with original design Combinational Optimization Engine 119147 regs Transformations are completely transparent to the user – internally used to enable exponential speedups! All verification results are in terms of original design Phase Abstraction Engine 79302 regs optimized, phase abstracted trace optimized trace Localization Engine Semi-Formal Engine Interpolation Engine Semi-Formal Engine Induction Engine optimized, phase abstracted, localized trace Retiming Engine … … … Transformation-Based Verification Design + Driver + Checker 140627 registers Problem decomposition via synergistic transformations 1320 regs Parallel algo exploration, (sub)problem solution 189 regs

43. Example Engines • Combinational rewriting • Sequential redundancy removal • Min-area retiming • Sequential rewriting • Input reparameterization • Localization • Target enlargement • State-transition folding • Circuit quantification • Temporal shifting + decomp • Isomorphic property decomp • Unfolding • Speculative reduction • Symbolic sim: SAT+BDDs • Semi-formal search • Random simulation • Bit-parallel simulation • Symbolic reachability • Property-directed reachability • Induction • Interpolation • Invariant generation • Array abstraction • Expert System Engine orchestrates parallel optimal engine selection • If there's a useful model checking algorithm in the world, RuleBase SixthSense Edition probably has it

44. Example Transform 1: Retiming • Retiming eliminates state elements by moving them across gates • Moving a state element across a gate time-shifts its behavior • Very effective at reducing overhead of pipelined designs • 62% reduction attained on POWER4 netlists

45. Example Transform 2: Redundancy Removal • Redundancy is prevalent in verification testbenches, e.g.: • Deliberate logic replication to reduce delay (due to placement issues) • Disabled pervasive logic such as scan-chains • Redundancies between design + checker • May be eliminated through redundancy removal

46. Example Transformation 3: CEGAR-Based Localization • Abstraction that reduces netlist size via cutpointing internal gates • Counterexample-Guided Abstraction Refinement • Begin with an arbitrary small abstraction • Perform verification on the abstraction • Proof obtained on abstract model? Pass • Counterexample found? Check if valid w.r.t. original netlist • Yes? Fail • No? Refine the abstraction; goto 2

47. Misc Transforms Refactoring Input Reparameterization ODC-based reductions “Dependent register” reductions BDD-based rewriting Speculative reductions Constraint-based simplification Target enlargement Time-shifting abstraction Property decomposition … Lookup-Table Based Rewriting

48. prove! Transformation-Based Verification Generality • Allows arbitrary sequencing of engines • Localization followed by retiming, rewriting, redundancy removal – then localization! • Many of these are synergistic, e.g. • Localization cutpoints enhance retiming • Transforms qualitatively alter the localized design, enabling improved reductions • Some transforms have no counterpart to original netlist, e.g. • Retiming a localized netlist yields reductions unachievable in original netlist • TBV is a unique, patented capacity benefit of RuleBase SixthSense Edition! No other tool is as flexible

49. Example Experiments MMU Initial* BRN AXE CUT AXE CUT RET BRN CUT ESE Registers 124297 67117 698 661 499 499 133 131 125 PASS ANDs 763475 397461 9901 8916 5601 6605 16831 4645 1300 1038 sec Inputs 1377 162 1883 809 472 337 1004 287 54 386 MB ERAT Initial* BRN EQV RET BRN SEQ AXE:50 Registers 45637 19921 419 337 273 257 FAIL ANDs 316432 167619 3440 2679 1851 1739 2831 sec Inputs 6874 68 63 183 126 126 884 MB BRN: Combinational Rewriting RET: Min-area retiming AXE: Localization CUT: Reparameterization ESE: Reachability EQV: Sequential Redundancy Removal

50. Model Checking Capacity vs Time* Invariant- Based IC3 Interpolation Abstraction-Refinement Transformation- Based Verif SAT-Based BMC BDD-Based Model Checking Explicit-State Model Checking Scalable Equivalence Invariants Semi-Formal Verif Partitioned BDDs RuleBase: SixthSense Edition has the highest capacity of any tool on the market Property checking and sequential equivalence checking * Not guaranteed capacity;1) some tiny problems are unsolvable! 2) includes bounded proofs Very incomplete list; cumulative capacity trend leverages earlier innovations + SW engineering