Advanced Debugging Techniques Using Maximum Satisfiability

1. Improved Design Debugging using Maximum Satisfiability Sean Safarpour, Hratch Mangassarian, Andreas Veneris Mark H. Liffiton, Karem A. Sakallah University of Toronto University of Michigan

2. Outline • Introduction & Motivation • Debugging with max-sat • Debugging with approximate max-sat • Experiments • Conclusion

3. Introduction & Motivation • Verification is a major pain in VLSI design cycle • check correctness against specs • identify corner cases • evaluate test coverage • Automated solutions exist • When design fails, find error source • manual debugging process • time consuming specification specification design specification verification specification yes pass? pass? no debug debug

4. Introduction & Motivation • What is “debugging” • identifying error source/locations in design • i.e. lines of code in the RTL (Verilog/VHDL) • Debugging is a verification bottleneck • manual task • consumes up to 50% of verification effort • Automated debugging or error localization • finds source of error in circuit automatically • use information from verification (testbench, traces, etc.) • current techniques not practical in industry

5. Introduction & Motivation • Why is debugging hard to automate? • Problem Complexity grows O(n# errors) • many gate-level errors in high-level designs • Algorithms work on gate/Boolean models • circuits are big: > 100,000 gates • Algorithms must model sequential behavior • traces are long: > 1000s of clock cycles • expensive using time frame expansion • Debugging techniques over past decades • Simulation, path tracing, BDDs • SAT & QBF techniques: last 5 years • promising results

6. Erroneous design Erroneous design Automated Debugger Automated Debugger potential error locations potential error locations Simulation & Expected traces Erroneous design Introduction & Motivation • What are the inputs to automated debuggers? • Erroneous design (one or multiple gate/high-level bugs) • Simulation Trace: stimulus to observe erroneous behavior • Correct/Expected Response: trace provided from specs

7. Introduction & Motivation • Previous SAT & QBF techniques • How can erroneous circuit be modified to satisfy the expected response under the given stimulus? • Insert instruments in circuit/CNF such that satisifying assignment => error locations • Find all error location tuples up to N • Observation • The union of the constraints is UNSAT • removing clauses corresponding error locations makes the problem SAT • Maximum Satisfiability problem

8. a a 0 Correct response 0 c c 1 1 b b e e 1 1 d d Correct circuit (!a+c) (!b+c) (a+b+!c) (c+!e) (d+!e) (!c+!d+e) (!a) (b) (d) (!a) (b) (d) (!e) (!e) = SAT Debugging example Simulation trace 1 0 Erroneous circuit (!a+c) (!b+c) (a+b+!c) (c+!e) (d+!e) (!c+!d+e) = UNSAT

9. Debugging with max-sat • Removing clauses satisfies the formula • can only “remove” clauses corresponding to circuit constraints • identify bugs in gates (not stimulus/response) • max-sat algorithms do exactly that • Requirements • find all gate-level errors up to cardinality N • must map clauses to gates • develop concept of error cardinality

10. Debugging with max-sat • Maximal Satisfiable Subset (MSS) • generalization of max-sat (maximum cardinality) • adding any extra clause makes formula UNSAT • Minimal Correction Set (MCS) • complement of MSS • Can find all MCS of up to size k • Can group clauses to approximate MCSes • groups y1, y2,..., yn • every clause C becomes C’= (y1 -> C)

11. a 0 c 1 b e 1 d (!a) (b) (d) (!e) Debugging with max-sat • How to debug gates instead of clauses? • define a group for each gate in circuit • can only reason about satisfiablity at gate granularity • How to get all solutions? • block previous solutions and solve again • cannot remove blocking clauses 1 (!a+c+!yA) (!b+c+!yA) (a+b+!c+!yA) (c+!e+!yB) (d+!e+!yB) (!c+!d+e+!yB)

12. Debugging with max-sat • What about sequential designs andmultiple vectors? • Time-frame expansion or Iterative Logic Array unfold/replicate combinational circuit in time • Multiple stimulus/response are used to increase debugging resolution • All clauses derived from same combinational gate are in same group

13. Replication Example a a 1 a 2 a 3 1 c 1 c 2 c 3 c b e 1 2 3 b 0 b 0 b 0 e 1 e e 2 3 d 1 1 1 d 1 d d 2 3 1 1 1 (a+!c+!yA) (b+!c+!yA) (!a+!b+c+!yA) (c+!e+!yB) (d+!e+!yB) (!c+!d+e+!yB) (a1+!c1+!yA1) (b1+!c1+!yA1) (!a1+!b1+c1+!yA1) (c1+!e1+!yB1) (d1+!e1+!yB1) (!c1+!d1+e1+!yB1) (a2+!c2+!yA2) (b2+!c2+!yA2) (!a2+!b2+c2+!yA2) (c2+!e2+!yB2) (d2+!e2+!yB2) (!c2+!d2+e2+!yB2) (a3+!c3+!yA3) (b3+!c3+!yA3) (!a3+!b3+c3+!yA3) (c3+!e3+!yB3) (d3+!e3+!yB3) (!c3+!d3+e3+!yB3)

14. Replication Example a a 1 a 2 a 3 1 c 1 c 2 c 3 c b e 1 2 3 b 0 b 0 b 0 e 1 e e 2 3 d 1 1 1 d 1 d d 2 3 1 1 1 (a+!c+!yA) (b+!c+!yA) (!a+!b+c+!yA) (c+!e+!yB) (d+!e+!yB) (!c+!d+e+!yB) (a1+!c1+!yA) (b1+!c1+!yA) (!a1+!b1+c1+!yA) (c1+!e1+!yB) (d1+!e1+!yB) (!c1+!d1+e1+!yB) (a2+!c2+!yA) (b2+!c2+!yA) (!a2+!b2+c2+!yA) (c2+!e2+!yB) (d2+!e2+!yB) (!c2+!d2+e2+!yB) (a3+!c3+!yA) (b3+!c3+!yA) (!a3+!b3+c3+!yA) (c3+!e3+!yB) (d3+!e3+!yB) (!c3+!d3+e3+!yB)

15. Debugging with approximate max-sat • Can group clauses from single gate together • single group represents a gate-based error • Can group “same” gates from different vectors and different time frames together • single group represents a gate-based error anywhere • What about other types of groupings? • based on modules/functions (higher level) • based on topology • arbitrarily

16. Debugging with approximate max-sat • Example based on topology group 2 group 1 group 3 group 4

17. Debugging with approximate max-sat • Empirical results show: max-sat is very efficient at finding approximate solutions • the larger the groups the faster • resolution vs. performance trade-off • SAT-based debugger is good for exact solutions, not for approximate • faster than max-sat for exact solutions • Exploit the strengths of both approaches • use approximate max-sat as preprocess/filter • use SAT-based debugging on filtered problem

18. Debugging with approximate max-sat • Two step debugging framework with max-sat • divide and conquer approach • approximate max-sat finds coarse solutions • exact debugger refines solutions erroneous design exact error locations over-approximation of suspects approximate max-sat debugger exact debugger vectors/traces grouping

19. Experiments • Circuits from ISCAS’85/89, ITC’99, opencores.org • error inserted: random function change • AND, OR, NAND, NOR, XOR swamped • simulation to find error • captured stimulus and correct trace • 1GB memory limit, 1 hour timeout • Clauses group by • gate • replication • circuit topology • max-sat solver from U of Michigan • SAT-based debugger from U of Toronto

20. Experiments • How many clauses or gates to group together • bigger/fewer groups -> faster the max-sat portion, more suspects • smaller/more groups -> few suspects, faster exact debugger • Empirical results show groups of 20 gates are effective

21. Experiments • sample experiments • max-sat very fast on grouped problem • exact debugger very fast on problem with eliminated suspects • result: orders of magnitude faster

22. Experiments • Number of solved instances and runtime • proposed framework against SAT-based debug only

23. Experiments • Runtime comparison: • proposed framework against SAT-based debug only

24. Conclusion • SAT-based debugging is a natural max-sat problem • formulate max-sat problem for circuits • gate grouping • replication grouping • find all error suspects • proposed framework combining strengths of max-sat and SAT-based debugging • order of magnitude improvements achieved