Finding and Fixing Faults

1. Finding and Fixing Faults Barbara Jobstmann, Stefan Staber, Andreas Griesmayer, and Roderick Bloem Graz University of Technology Alpine Verification Meeting, October 2005

2. Dr Seuss, Did I Ever Tell You how Lucky You Are? Fault Localization and Correction

3. More Motivation Debugging consists of • Detecting failure • Localizing fault • Correcting fault Manual Localization & Correction takes significant time • Bugs fixes at very end of design cycle (high risk) Important problem, but little research! Fault Localization and Correction

4. Previous Work • Understandability of counterexamples • Clarke et al 95: original work • Ravi, Somenzi, Jin: decision points, length and “width” of counterexample • Comparing good and bad traces to find suspicious points • Groce, Zeller, Ball and Rajamani • Diagnosis & Repair for • combinational circuits [various] or • sequential circuits with very simple fault models [Wahba&Borrione] Fault Localization and Correction

5. Our Idea • Find (and correct) faulty component • But: what is a component • You tell me! • Expressions, gates are typical choices • Ideas presented here work for any component model Fault Localization and Correction

6. Outline • Localization • Finds suspect components • Works with a set of traces • BMC-like • Correction • Localizes fault and finds repair that is good for all inputs • About as fast as BDD-based model checking (with some heuristics) Fault Localization and Correction

7. Localization: Setting • Identify components responsible for a failure • Input • Faulty design • Finite failure traces • LTL specification (liveness aspects are ignored) • Output • Set of suspect components • Suspicion is for the given traces Fault Localization and Correction

8. Localization: Idea • Given a failure trace • Approach • Unroll the circuit; introduce “abnormal predicates,” fixing inputs to failure trace • Unroll the LTL property using expansion rules • Combine circuits & property • Call a SAT-Solver and find valid assignment for the variables (notably the abnormal predicates) • Mix of BMC [Biere] and Model-Based Diagnosis [Reiter 87, De Kleer and Williams 87] Fault Localization and Correction

9. Localization: Example Property G(req  (ack  X ack)  (ack  X ack)) fails for two consecutive requests (failure trace: req = 1; req = 1) (We get no acks; G2 should be G1  D1) initial state D0=0, D1=0 Fault Localization and Correction

10. 1: Unroll, Introduce Predicates Components: G1: not ABG1 -> (outG1t0 = in1G1t0 + in2G1t0), not ABG1 -> (outG1t1 = in1G1t1 + in2G1t1) G2: not ABG2 -> (outG2t0 = in1G2t0 * in2G2t0), not ABG2 -> (outG2t1 = in1G2t1 * in2G2t1) in1G1t0 = 1 (failure trace t0: req = 1), etc. Fault Localization and Correction

11. Step 2: Unroll Property G((req  (ack  X ack))  (ack  X ack)) Note: Free inputs on the right can be set to 1 (or left free): represent liveness part Fault Localization and Correction

12. Step 3: Combine unrolling of circuit property violated representation of property Fault Localization and Correction

13. Step 4: SAT-Solver Remove constraints Property satisfied Fault Localization and Correction

14. The Formula With a SAT solver: • Single fault: SAT(cex(k)  circuit(k)  property(k)  oneAbnormal  valid=1) • Two faults: SAT(cex(k)  circuit(k)  property(k)  twoAbnormal  valid=1) • 0/1 ILP (PBS): Minimize |abnormal| subject to cex(k)  SAT(circuit(k)  property(k)  valid=1 Fault Localization and Correction

15. Correction

16. Correction Localize faulty component and find correction • Given • Faulty (finite state) system • LTL specification • (No trace) • Goal • Localize fault and find repair for all inputs: replace component c by function f(s,i), where s is state, i input. • Exact results for invariants • Heuristic for other LTL properties (exact solution very expensive) Fault Localization and Correction

17. Correction: Idea • Infinite two-person games (= controller synthesis = AI planning) • Approach • Extend the faulty system to a game • Find a strategy • Extract a correction (like symbolic model checking) Fault Localization and Correction

18. Correction: Example Property G(req  (ack  X ack)  (ack  X ack)) Fault Localization and Correction

19. 1: Build Game System • Environment decides input values • System decides • Faulty component at the begin of the play • New value at each time step initial state D0=0, D1=0 Environment Fault Localization and Correction

20. 1: Build Game System choice Environment Fault Localization and Correction

21. 1: Build Game Fault Localization and Correction

22. 2: Find Strategy Fault Localization and Correction

23. Step 4: Extract a Correction Table for G2 Simplest implementation: G2 = ¬D1 Note: our spec is incomplete! 0 0 Fault Localization and Correction

24. Computing the strategy • Like BDD-based model checking, a little more expensive • Replace EX by • MX(A) = { s’ | i c s in A: s  (s,i,c,s’) } • For invariants, compute MG A = Y. A  MX(Y) For LTL, we use heuristics • Avoid doubly exponential blowup for deterministic automaton • build a NBW automaton • heuristic: trade completeness for efficiency • Prevent adding new state bits • Find a memoryless strategy • Is NP complete • heuristic: trade completeness for efficiency • Compute strategy by Emerson-Lei algorithm with MX instead of EX Fault Localization and Correction

25. Correction: Experience • about 10 handpicked examples • correction works well • one specification needed updating • one repair was simpler than the original • Implementation needs work Fault Localization and Correction

26. Conclusion • Localization • Find gates that may be incorrect • Based on BMC (with one extra variable per component) • Correction • Find faulty component & correction • A bit harder than BDD-based model checking • One more slide… Fault Localization and Correction

27. Workshop: Verification & Debugging Proposed workshop Associated with CAV’06 Topic: getting rid of the bugs you found (I’m done. Thanks!) Fault Localization and Correction