1 / 16

Advanced Symbolic Testing for Key-Value Store Libraries Using QuickCheck Feature

This lecture covers the principles of symbolic testing for data structures using QuickCheck, focusing on the `dict` key-value store. Learn how to create and manipulate dictionaries, apply black-box testing techniques, and generate symbolic cases to verify properties like unique keys. We'll delve into generators, evaluate symbolic function calls, and explore the shrinking capabilities of QuickCheck. Discover how to model the behavior of dictionary operations and design robust tests for data representations, while providing debugging strategies for identifying failures.

susan
Télécharger la présentation

Advanced Symbolic Testing for Key-Value Store Libraries Using QuickCheck Feature

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. FIGYELEM! Software Testing with QuickCheck Lecture 2 Symbolic Test Cases

  2. Testing Data StructureLibraries • dict—purelyfunctionalkey-value store • new()—createemptydict • store(Key,Val,Dict) • fetch(Key,Dict) • … • Complex representation… just test the API • ”black box testing” • In contrast to testing e.g. dict invariants

  3. A Simple Property • Perhaps the keys are unique… • Cannot test this without a generator for dicts prop_unique_keys() -> ?FORALL(D,dict(), no_duplicates(dict:fetch_keys(D))).

  4. Generatingdicts • Black box testing  use the API to createdicts • Wesimulatelazyevaluationwhereweneed it dict() -> oneof([dict:new(), ?LET({K,V,D},{key(),value(),dict()}, dict:store(K,V,D))]). dict() -> oneof([dict:new(), ?LET({K,V,D},{key(),value(),dict()}, dict:store(K,V,D))]). dict() -> ?LAZY( oneof([dict:new(), ?LET({K,V,D},{key(),value(),dict()}, dict:store(K,V,D))]) ). A choicebetween generators Infinite recursion! key() -> oneof([int(),real(),atom()]). value() -> key(). Constant time

  5. Let’s test it! • Black box testing: we need to knowhow this wasconstructed!

  6. Symbolic Test Cases • {call,M,F,Args} represents a functioncall M:F(…Args…) symbolically. • eval(X) evaluatessymboliccalls in a term X dict() -> ?LAZY(oneof([ {call,dict,new,[]}, {call,dict,store,[key(),value(),dict()]} ])). Test case is nowsymbolic prop_unique_keys() -> ?FORALL(D,dict(), no_duplicates(dict:fetch_keys(eval(D)))).

  7. Let’s test again! Shrinks the values, but not the structure! Wecanseeexactlyhow the failingcasewasbuilt up!

  8. Shrinking the structure • QuickCheck has manyshrinking operations; hereweuse • Binds X1, X2… to valuesgenerated by G1, G2… • GeneratesResult • Can shrink to any of the Xi ?LETSHRINK([X1,X2,…], [G1,G2,…], Result(X1,X2,…))

  9. dict() with Shrinking • ?LETSHRINK makes shrinkingrecursivetypesvery easy dict() -> ?LAZY(oneof( [{call,dict,new,[]}, ?LETSHRINK([D],[dict()], {call,dict,store,[key(),value(),D]})])).

  10. Let’s test and shrink! Nice shrinkingresult -1 and -1.0, eh?

  11. Testing vs. an Abstract Model • Howshoulddict operations behave? • The ”same” as a list of key-value pairs • Use the proplistslibrary as a reference • Make comparisons in the ”model” world Returns list of key-value pairs model(Dict) -> dict:to_list(Dict).

  12. Commuting Diagrams Dict Dict + {K,V} dict:store(K,V,…) model model List [{K,V} | List] model_store(K,V,…) Hoare: Proof of Correctness of Data Representations, 1972

  13. Testing store prop_store() -> ?FORALL({K,V,D}, {key(),value(),dict()}, begin Dict = eval(D), model(dict:store(K,V,Dict)) == model_store(K,V,model(Dict)) end). model_store(K,V,L) -> [{K,V}|L].

  14. Next Steps… • Writesimilarproperties for all the dict operations • Extend the dict generator to useall the dict-returning operations • Eachproperty tests many operations • …and, of course, correct the specification!

  15. Debuggingproperties • Why is a propertyfalse? • Weneedmore information! • ?WHENFAIL(Action,Property) performs Action whenreporting a failingcase • Typicaluse: equal(X,Y) -> ?WHENFAIL(io:format("~p /= ~p~n",[X,Y]), X==Y).

  16. Exercises

More Related