Verification & Validation

1. Verification & Validation By: Amir Masoud Gharehbaghi Email: amgh@mehr.sharif.edu

2. Outline • Overview • Selective History • Theorem Proving • Model Checking • Hardware Verification • Assertion-Based Verification • Conclusions

3. What is Verification (Validation) • Functional Verification: • Task of establishing that a given design accurately implements the intended behavior

4. Verification&Validation Techniques • Simulation-based • Apply inputs to design, simulate (or run), and check the results • Formal • Mathematically proof the correctness of system against the properties • Semi-formal • Combine simulation and formal verification

5. V&V Techniques Comparison • Simulation-based • Easy to use • Fast • Low coverage • Formal • Perfect coverage • Not easy to use • Not applicable for large designs

6. Using Formal Methods • Writing formal specifications • Proving properties about the specification • Theorem proving • Deriving implementation from a given specification • Refinement • Verifying properties for a given implementation • Property checking

7. Selective History • Early 1960’s • Suggestions: McCarthy and Dijkstra • Late 1960’s and Early 1970’s • Proof systems: Floyd-Hoar, Boyer-Moore,… • Late 1970’s • Temporal Logic for reactive systems (Pnueli, …) • Early 1980’s • Model checking (Clarke, Emerson, …) • Late 1980’s • Symbolic model checking using BDDs

8. Selective History (cont.) • 1990’s -> mostly hardware • Non-BDD based model checking • Satisfiability • Equivalence checking • Symbolic simulation & symbolic trajectory evaluation • 2000’s • Assertion-based verification • Software model checking • Probabilistic verification • Automated theorem proving • Hybrid systems verification

9. Theorem Proving • Formally specify the system in a logic system • Formally specify the properties of system • Prove the correctness of properties of system in a proof system

10. Theorem Provers & Logic Systems • First-Order Logic • ACL2 • Nqthm • High-Order Logic • HOL • PVS

11. Temporal Logic • First-Order Logic + Temporal Operations • Linear Temporal Logic (LTL) • Computational Tree Logic (CTL)

12. LTL Temporal Operations • X: next • F: finally • G: globally • U: until

13. LTL Example • p  X p • alert  F halt • G (alert  F halt ) • G (alert  (alarm U halt ) )

14. CTL Path Operations • A: always • E: there exists • Combine with temporal operations of LTL: • AX, AF, AG, AU • EX, EF, EG, EU

15. CTL Example • AG p • AF halt • E ( alaram U halt ) • AX alarm  EF close

16. Properties • Safety • Something never occurs. • Liveness • Something will ultimately occur. • Reachability • Some particular situation can be reached. • Fairness • Something will (not) occur infinitely often. • Properties are checked under certain conditions

17. Model Checking • Model is a state machine (or automata) • Property is defined in a temporal logic • CTL model checking • O(|Q| * |p|) • LTL model checking • O (|Q| * 2^|p| ) • |Q|: number of states • |p|: number of sub-formulas in property

18. State Explosion Problem • Both in LTL and CTL: • An automata is generated (explicitly) • Number of states grow exponentially

19. Symbolic Model Checking • Symbolically (implicitly) represent states (and transition between states) • Use Binary Decision Diagram (BDD) to represent state variables • Uses CTL properties

20. Reduced Order BDD (ROBDD) • A directed acyclic graph (DAG) with two leaf nodes (1,0) • Represent Boolean functions • Compact • Canonical • Efficient operations (linear or quadratic) • Simple to use

21. Satisfiability (SAT) Checking • Satisfiability Checking: • Check existence of a combination of values for a Boolean function that function is 1 • Check that ~f is unsatisfiable

22. Bounded Model Checking • Search for counter example by unfolding system in time until a bound is reached. • Use SAT checkers • What about unbounded model checking?

23. Symbolic Simulation • Simulate with symbolic (not explicit) values. • Inputs: expressions • Outputs: expressions • Originally based on BDD.

24. Symbolic Trajectory Evaluation • Check properties of A=>C form. • A: input variables’ values over time • C: expected output variables’ values over time • Symbolically simulate with given input values (A). • Check that expected results (C) to be compatible (subset of) simulated output results.

25. Equivalence Checking • Check equivalent behavior between two designs • Same level of abstraction • Different levels of abstraction • Combinational • Sequential

26. Equivalence Checking Methods • Combinational • Compare the canonical representation of two circuits. (may be not feasible) • Use SAT checker • … • Sequential • Find equivalent FFs and Compare combinational circuits between them. • Construct the multiplicative state machine and check the equivalency of outputs in all states. • Bounded model checking

27. Assertion-Based Verification • Assertion: property • Do property checking during simulation • Embed in design • Check in run-time

28. Assertion Languages • OVL: Open Verification library • PSL: Property Specification Language • Formerly “Sugar” • SystemVerilog • …

29. Coverage • Percentage of design covered during simulation • Code • Statement • Path • Condition • … • Signal • …

30. Conclusions • Verification is a serious bottleneck for current designs • Up to 80 percent of design time • Formal methods cannot be applied to real designs • Simulation cannot guarantee correctness of designs • Embedded system verification containing Hw/Sw requires new techniques