A Survey of Model Checkers

A Survey of Model Checkers John C. Sloan and Venkat Raghav -- Florida Atlantic University --

Contents: • Introduction – Sources and suitability • Case Study – Volunteer Supercomputing • LTSA – Labelled Transition System Analyser • CPN Tools – Colored Petri Net Tools • SPIN – Simple Promela INterpreter • UPPAAL – UPPsala and AALborg Universities • Summary – comparisons and contrasts

Sources: • V. Raghava, "A Comparison of Model Checking Tools for Service • Oriented Architectures", MS Degree Thesis, Florida Atlantic University, • Boca Raton, Florida USA (December 2007), Advised by T. M. Khoshgoftaar. • J. C. Sloan, T. M. Khoshgoftaar, and V. Raghav. "Assuring Timeliness of • Volunteer e-Science Service-Oriented Architecture“. Computer, 41(8):56- • 62, August 2008. • J. C. Sloan and T. M. Khoshgoftaar, “From Web Services Artifact to a • Readable and Verifiable Model”, IEEE Transactions on Services • Computing, 2(4):277-288, October 2009.

Suitability: • Four model checkers will be compared • .. from highest to lowest level of abstraction • Evaluate their architectural expressiveness – how closely • the tool’s modeling formalism matches how a user thinks. • .. ultimately subjective (ie: which is better Mac or PC?). • Case study provides a benchmark for comparison. • Apply the six elements of an SOA to the case study: • On one side of a portal a (i)provider (ii)registers with a • (iii)portal (or registry), the types of (iv)services the provider • is willing to offer, • At the same time, a (v) requestor asks the portal for • services, causing the portal to dynamically • (vi) discover service providers that can offer solutions. • Once discovered, the work units initiated from the requestor • are processed by the providers, with the portal possibly • coordinating the distribution of results.

Case Study: • BOINC – Berkeley Open Infrastructure for Network Computing • It is a deployment platform for Volunteer Supercomputing • Provides an opporchancity for individual PC users to donate unused • CPU cycles to a worthy cause, like: • SETI@Home – Search for Extraterrestrial Intelligence (cringe) • Einstein@Home – Particle Physics • climatePrediction.net – duh.. • How it works @ your home computer • When your screensaver comes up, it runs a BOINC core client that • communicates with the web portal to execute a work unit. • How it works @ the web portal • Phase 1 – Home user as service provider,registers with the portal • Phase 2 – Portal’s download manager sends a work unit to provider • Phase 3 – Portal receives results uploaded from home computer. • Repeat Phases 2, 3 until user changes scheduling preferences. The view from home and the view from the portal are partial .. A more unified view of this case study is needed.

Case Study: • As an SOA, this case study involves the six elements of an SOA: • The home user, acting on behalf of a service (i)provider, • (ii)registers with a Volunteer Supercomputing (iii)portal • (like Einstein@Home), the machine type, operating • system, and scheduling preferences which together • describe the (iv)services (ie: CPU cycles) the provider • is willing to offer, • At the same time, a (v) requestor from the scientific • community (ie: a Particle Physicist), asks the portal for • services like CPU cycles for some machine architecture, • causing the portal to dynamically (vi) discover service • providers that can offer solutions. • Once discovered, the work units initiated from the requestor • are processed by the providers, with the portal coordinating • the process. • In most other SOA’s the broker piece is a passive registry • and not an active portal that coordinates activities after • the provider and requestor hook up.

Case Study – Modeling the system • We first consider the routine interactions between these four parties; • Human user • Service provider (aka BOINC core client) hooked to the screen saver • Web service portal that functions as a service registry and • coordinator of tasks for a given project (or consortium thereof) • 4. Requestor in the research community. Requestor Portal Provider Human • Each model checker has its formalism that enables the verification • engineer to model interactions in a way that is machine verifiable! • LTSA uses message sequence charts (MSC’s) • CPN Tools uses colored Petri nets. • SPIN uses source code text written to a modeling language like • PROMELA. • UPPAAL uses timed automata.

Case Study – Specifying properties Each model checker supports how the user specifies properties in the same way … through a text box. Once both model and properties are specified .. .. then push a button! Either an affirmative response or counterexamples will be given.

LTSA – Architecture WS-BPEL composition GUI client MSC-Draw Conversion tool FSP compilation Property verification Trace violations Properties specification

LTSA – modeling interface

LTSA -- Message Sequence Chart (MSC) Human Provider Portal Scheduler D/L Mgr U/L Mgr logout P H A S E #1 Available Roger Here’s my schedule Roger P. #2 Give me a work unit Here is your work unit P. #3 These are my results Thank you – you have been credited login

LTSA -- Message Sequence Chart (MSC) Human Provider Portal Scheduler D/L Mgr U/L Mgr We will be modeling only Phases #2 and #3 using the remaining model checkers, since they operate at a more detailed level of abstraction logout P H A S E #1 Available Roger Here’s my schedule Roger P. #2 Give me a work unit Here is your work unit P. #3 These are my results Thank you – you have been credited login

LTSA – Sample output

LTSA – Labeled transition system output

LTSA – in its full glory (all Eclipse plugins enabled)

LTSA • Advantages: • Conceptually simple modeling formalism – low learning curve • High level of abstraction avoids details • Enables representing looser peer-coupled choreographies • as well as the tighter coupled orchestrations. • Disadvantages: • High level of abstraction avoids details • Captures notion of sequence only • Does not express notions of channels and tokens • Does not express notions of real-valued time • No apparent means of expressing timing properties • in terms of some temporal logic.

CPN Tools – Architecture Forthcoming.

CPN Tools – without makeup Wrong .. Needs to capture mutual exclusion between user using PC and BOINC core client running.

CPN Tools – with makeup userLoggedIn userLoggedOut userLogsIn userLogsOut Wrong .. Needs to capture mutual exclusion between user using PC and BOINC core client running. providerAvailable providerAwaitingWorkunit portalRelaysWorkunit portalStagingWorkunit requestorSendingWorkunit portalAwaitingResults providerUnblockedByPortal requestorInteractsWithPortal providerInitialized providerRunning providerAwaitingAck providerReceivesAck providerRelaysResults requestorReceivingResults portalReceivingResults portalSendingAck portalRoutesResults

CPN Tools – inhibitor arcs (which makes it Turing expressive) Now THIS CPN subnet implements the idea that once the user logs off, the BOINC core client is enabled. Once logged in, BOINC client gets disabled!!!

CPN Tools • The following slide is a hand rendering of the previous two slides in • PowerPoint, where the subnet in the previous slide is simply • abstracted as an inhibitor arc. • We will be modeling routine operation of BOINC Phases #2 and #3 • (not the initialization phase #1). • We will be explaining the detailed operation by playing a token • game with the following Petri net, starting with its initial marking.

Portal Research Community 18 10 Collecting Work unit Distributing result A researcher interacts with portal Provider 15 11 12 13 14 Portal sending work Portal Receiving result Receive work unit Send result Running Human 5 9 PC idle 17 8 16 Portal receives request from researcher and assigns work 4 1 Portal posts/ distributes results Awaiting work Awaiting ACK Logout Login 2 Human using Provider PC 6 19 20 7 3 Sending ACK Wait until Provider PC becomes idle Available for work Initialized Receive ACK

CPN Tools • Advantages: • Familiar extension of Petri nets (colors implement inhibitor arcs) • Almost keyboard-free user interface – beyond the Mac and PC! • Moderate level of abstraction • Can represent tokens traveling over channels • for a close-up view of system interactions. • Can express inhibitor arcs (to model mutual exclusion • between PC user login and Provider de-activation • Can express data types of messages over each channel • Disadvantages: • Unfamiliar user interface requires getting used to • Moderate level of abstraction still does not express notions of • real-valued time

SPIN – Architecture Properties In LTL Promela code Spin GUI client pan.c C compiler Counter- examples a.out

SPIN – source code sample

SPIN – sample output Event viewer Message sequence chart

SPIN • Advantages: • Familiar text-based user interface • Promela – the programming language of Spin is Turing expressive • Moderate level of abstraction • Can represent tokens traveling over channels • for a close-up view of system interactions. • Can express data types of messages over channels • Disadvantages: • Frustratingly inexpressive text-based user interface • Why can’t I just draw a picture! • Moderate level of abstraction still does not express notions of • real-valued time • UNIX orientation makes it hard to install on Windows PC’s. • Program artifacts are freeform text that can be unstructured • making it hard to translate model to source code. • the other 3 model checkers work with XML, making model • to source code conversion more feasible

UPPAAL – motivation • But why UPPAAL? • With only the notion of sequential time and • Without the notion of real-valued time, then • There is no ability to represent relative • frequencies of events • As the following slides will show

Portal Research Community 18 10 Collecting Work unit Distributing result A researcher interacts with portal Provider 15 11 12 13 14 Portal sending work Portal Receiving result Receive work unit Send result Running Human 5 9 PC idle 17 8 16 Portal receives request from researcher and assigns work 4 1 Portal posts/ distributes results Awaiting work Awaiting ACK Logout Login 2 Human using Provider PC 6 19 20 7 3 Sending ACK Wait until Provider PC becomes idle Available for work Initialized Receive ACK

UPPAAL – motivation (continued) • What happened? • In untimed models like LTSA CPN Tools and Spin • the previous events are perfectly valid. • But they don’t make sense • Common sense would dictate that a PC usually • executes dozens of work units while the PC user • is logged off for the evening. • UPPAAL and timed automata to the rescue!

UPPAAL -- Architecture GUI client Model and properties Verifyta server Counter- examples

A Survey of Model Checkers

A Survey of Model Checkers

Presentation Transcript

Checkers solved!

Delta Debugging and Model Checkers for fault localization

Wordlist 37 A Game of Checkers

Checkers solved!

HPC Checkers

Checkers

Security Checkers

Checkers solved!

Pressure Checkers

A Survey of ICASSP 2013 Language Model

Checkers

CHECKERS

Tuning SAT-checkers for Bounded Model-Checking

Modeling AADL Flows in Real Time Model Checkers

Checkers Cheaters

Tuning SAT-checkers for Bounded Model-Checking

A Survey of Boosting HMM Acoustic Model Training

Tuning SAT-checkers for Bounded Model-Checking

A Survey of Protein Folding in HP Model

A Survey on Programming Model

Tuning SAT-checkers for Bounded Model-Checking

BOGOR – A Flexible Framework For Creating Model Checkers