Symbolic Passive Testing - Application to an industrial case study (Diamonds project)

1. Work supported by DIAMONDS (ITEA 2) Project Symbolic Passive Testing - Application to an industrial case study (Diamonds project) PramilaMouttappa, StephaneMaag and Ana Cavalli Institut Mines-Télécom/Télécom SudParis RCIS 2013 Industrial Day - 31st May, 2013 Paris, France

2. Overview Motivation Our approach IOSTS basedmodels Parametric trace slicing Formalpropertyevaluation Prototype results Conclusions Symbolic Passive Testing

3. ConformanceTesting IUT Active Tester Verdict Test generation Specification / Requirements Test Cases ACTIVE TESTING Passive Tester Trace Verdict IUT P.O. Specification / Requirements User PASSIVE TESTING Télécom SudParis Modèle de présentation

4. Problem Statement • Limitation of Passive Testing - For checkingboth the data and control part of the protocol. - Enumeration of data values. • ONLY CONTROL PART Invariant : Req / Ack Verdict = True Symbolic Passive Testing

5. Problem Statement (contd.) (ii) CONTROL + DATA PART Invariant : Req(A) / Ack(B) Verdict = False or Inconclusive Symbolic Passive Testing

6. Motivations • To reduce false positive verdicts • To avoid data enumeration • To model eventual behaviordeviations • To study the scalability of very long traces (i.e., if the trafficcollection is continuous) Télécom SudParis Modèle de présentation

7. Our approach ... • Functional and Security attackscanbemodeled – IOSTS • Enumeration of data values is not required – SYMBOLIC • Data relationshipbetween messages – REDUCES FALSE POSITIVE VERDICTS • New Passive Testing Technique - INTEGRATION of Parametric Trace SlicingandSymbolicExecution Symbolic Passive Testing

8. Framework of our approach – Automotive case study 3 2 1 Symbolic execution tree Specification / Requirement’s 4 5 Symbolic execution trace and state details IOSTS model - Functional/Security property Guard-conditions table Bluetooth trace Symbolic Passive Testing Tool : TestSym-P 6 7 Verdicts Pass / Fail / Attack – Pass / Inconclusive ?   Symbolic Passive Testing

9. Sequencediagram - Bluetooth Connectivity & Bluestabbingattack [1] “Bluetooth specification version 2.0 + edr [vol 0],” 1999. [2] D. Browning and G. Kessler, “Bluetooth hacking: A case study,” in Proceedings of the Conference on Digital Forensics, Security and Law, 2009, pp. 20–22. Symbolic Passive Testing

10. IOSTS Model S3 G2:{loc_name ≠vloc_name} ! hci-change-local-name (loc_name0) A3: {vloc_name:=loc_name0} S3.1 Symbolic Passive Testing

11. SymbolicExecution (SE) of IOSTS Symbolic Passive Testing

12. Parametric Trace Slicing • Trace Monitoring technique • Trace slices – Each parameter instance observed in the trace. • The events corresponding to a particular parameter instance are grouped in the order they appear in the trace in a particular trace slice. Symbolic Passive Testing

13. Parametric Trace Slicing - Example Consider a sample Bluetooth trace, • !hci-inquiry ?hci-inquiry-complete !hci-create-connection(bd addr1 ) !hci-create-connection(bd addr2 ) ?hci-connect-complete(bd addr1 ) Symbolic Passive Testing

14. Integration of SE and Parametric Trace Slicing • SymbolicExecution (SE): • The path of the symbolicexecutiontree– represents the property to betested on the trace. • Valuation of the parameters – symbolic values. • Parametric Trace Slicing: • Trace analysis • Valuation of the parameters – concrete values. • Trace slices put togetherconstitutes the implementation trace. • Path of the SE tree – checkedagainsteach trace slice to obtain the Verdict. Symbolic Passive Testing

15. Evaluation - IOSTS property on the trace slices • Control portion : Control portion(SE) = Control portion(slice) goto 2. • Data portion: • Substitute concrete data values in place of symbolic values. • Satisfiability of each state in path of the SE - Guard condition check. • AttackSeq : 0 (PropertySequence), 1 (AttackSequence) else≠ Symbolic Passive Testing

16. Final Evaluation – Property on the Implementation trace • PASS • if (Every (Verdict(tsi) = Pass)) • Attack-PASS • if (Exists(Verdict(tsi) = Attack-Pass)) • FAIL • if [(Exists(Verdict(tsi) = Fail)) ^ (Exists(Verdict(tsi)≠ Attack-Pass))] • INCONCLUSIVE, otherwise. Symbolic Passive Testing

17. Prototype Model – TestSym-P Symbolic Passive Testing

18. Prototype – Trace Parsing Symbolic Passive Testing

19. Prototype - Trace Slicing Symbolic Passive Testing

20. Prototype – Evaluation Output Symbolic Passive Testing

21. Evaluation Results P - Pass F - Fail AP - Attack Pass I - Inconclusive [3] PramilaMouttappa, StephaneMaag and Ana Cavalli, "IOSTS based Passive Testing approach for the Validation of data-centric Protocols", 12th International Conference on Quality Software (QSIC 2012), Xi’an, China, 27-29 August 2012. [4] PramilaMouttappa, StephaneMaag and Ana Cavalli, “Improving Protocol Validation by an IOSTS-based Passive Testing approach”, 9th Workshop on System Testing and Validation (STV 2012), Paris, France, 24th October 2012. [5] PramilaMouttappa and StephaneMaag and Ana Cavalli, “Monitoring based on IOSTS for testing functional and security properties: Application to an Automotive case study”. To be published, 37th Annual International Conference on Computers, Software and Applications (COMPSAC 2013), 22-26 July 2013, Kyoto, Japan Symbolic Passive Testing

22. Innovations with respect to STOA • From our knowledge, there are currently no works tackling Passive testing/Monitoring based on IOSTS without any awareness on the states of the execution traces, moreover, • the integration of symbolic execution of IOSTS and Slicing technique for Passive Testing, • dealing with symbolic values eliminates the necessity of enumeration of all data values, • the approach enables testing functional and vulnerability / attack patterns by passive testing. Symbolic Passive Testing

23. Future Works • Online testing - i.e., evaluation of properties as the implementation is being run. • Time constraints - that are needed in the definition of severalproperties. Symbolic Passive Testing