Modular Verification of Concurrent Assembly Code with Dynamic Thread Creation and Termination

Modular Verification of Concurrent Assembly Codewith Dynamic Thread Creation and Termination Xinyu Feng Yale University Joint work with Zhong Shao

Motivation • Proof-carrying code (PCC) • In principle: verify any property on any code • Real binaries & no loss of efficiency • Embedded OS, device drivers… • All safety & liveness properties… • Formal, machine-checkable proofs • In reality: only works for sequential code Can concurrent code ever be supported by the PCC framework ? NJPLS@Stevens

Challenges • Challenges for Proof-carrying concur. code • A general framework for concurrent assembly code verification • Lack of structures (e.g. cobegin/coend blocks) • Specification/proof generation • Spec inference, proof assistant, theorem prover • Concurrent assembly code verification • No directly applicable logic • Traditional Hoare-logic: only sequential code • Type Systems: no Concurrent Typed Assembly Language (TAL) NJPLS@Stevens

Previous work • Rely-Guarantee (R-G) Method • Shared memory concurrency • Thread modular verification • Only for higher-level code: cobegin/coend • CCAP[Yu&Shao, ICFP’04] • The first PCC framework supporting concurrent assembly code • R-G method • Only support static threads • P1 || … || Pn NJPLS@Stevens

Concurrency Programming • cobegin/coend • S::=…| cobegin P1 || P2 codend | … • Higher-level, well-structured • Only support properly nested concurrent code • fork/join • S::=…| tid := fork f(a) | join tid | … • More flexible: improperly nested code • OSes/Java/… NJPLS@Stevens

Our Contributions • A new PCC framework: CMAP • Verification of general properties • Dynamic thread creation/termination • Generalize the Rely-Guarantee method • Modular verification • Realistic features • Multiple instantiations of thread code • Thread argument passing, thread-local data NJPLS@Stevens

Outline of This Talk • Background: the Rely-Guarantee Method • Challenges for Dynamic Thread Creation/Termination • Our Approach • The CMAP Framework • Conclusion and Future Work NJPLS@Stevens

G1  A2 G2  A1 S1 S2 S3 S4 S5 The Rely-Guarantee Method Thread 1 (A1,G1) Shared Memory Thread 2 (A2,G2) A1: S2 – S3, S4 – S5,… G1: S1 – S2, S3 – S4,… A2: S1 – S2, S3 – S4,… G2: S2 – S3, S4 – S5,… NJPLS@Stevens

The Rely-Guarantee Method • Thread + Thread Environment • Rely and Guarantee • A, G: State  State  Prop • Thread Modularity • Non-Interference (interface compatibility): • i,j. ij  Gi  Aj • Safety of each thread • Ti: (Ai, Gi) NJPLS@Stevens

GCD Example [Yu&Shao’04] Thread1: while(a<>b){ if(a > b) a := a-b; } Thread2: while(a<>b){ if(b > a) b := b-a; } NJPLS@Stevens

Concurrency Programming • cobegin/coend • S::=…| cobegin P1 || P2 codend | … • Higher-level, well-structured • Only support properly nested concurrent code • fork/join • S::=…| tid := fork f(a) | join tid | … • More flexible: improperly nested code • OSes/Java/… NJPLS@Stevens

Static and Dynamic Threads f(a) . . . “Static Threads” fork f(a1) fork f(an) fork f(a2) … “Dynamic Threads” NJPLS@Stevens

Challenges • First attempt • Check NI between all static threads • Ti: (Ai, Gi) • i,j. ij  Gi  Aj • Too rigid to handle changing env. NJPLS@Stevens

T3 C D Challenges: Changing Env. I • A-B: initialize data d • no other threads will change d • A: d = d’ • B-C: collaborate with T3 to process d • T3 may change d • Still do not allow other threads change d • C-D: T3 terminates • No other threads can change d T1 T2 A B Use pc to mark stages? NJPLS@Stevens

T3 … C D Challenges: Changing Env. I Global data: int data[100] T1 T2 main: int i:=0; while (i<100){ data[i]:=f(i); fork child(i); i++; } A B NJPLS@Stevens

Challenges: Changing Env. II • T2 and T3 have no overlap in their lifetime • non-interference between all threads? • Only check those that overlap? • How to specify the overlapping? T1 T2 T3 NJPLS@Stevens

Challenges: multiple instantiations (Aa, Ga) f(a) . . . GaiAaj GaAa? fork f(a1) fork f(an) fork f(a2) (Aa1, Ga1) (Aa2, Ga2) (Aan, Gan) NJPLS@Stevens

Challenges: Modularity (A1, G1) (A2, G2) T1: . . . jmp f T2: . . . jmp f Certify once, use everywhere? f: . . . exit NJPLS@Stevens

Our Approach (1) • Problems for checking NI of static threads • Changing environment • Multiple instantiations • Modularity issues • CMAP: “lazy checking” • At each step, all live (dynamic) threads do not interfere NJPLS@Stevens

…  (A0, G0) (An, Gn) each ti satisfies (Ai, Gi) WF(Q, ): Our Approach (2) … Q t0 tn How to track the changing thread queue? NJPLS@Stevens

 Q WF WF   Our Approach (3) • Borrow ideas from typechecking data heaps (as in TAL): Initial condition: 0 . WF(Q0, 0) Q'  ::=add | sub | jd f |… | exit | fork | yield ' NJPLS@Stevens

exit ti Q\{ti} Q WF! (Ai,Gi) \{(Ai, Gi)}  Our Approach (4) • Thread Termination: exit t Q WF (A,G)  NJPLS@Stevens

Our Approach (5) • Thread Creation: fork f(a) fork t’ t t Q Q WF WF ? (A,G)  • t'does not interfere with Q • tdoes not interfere with the new env. NJPLS@Stevens

    fork t Q{t’} WF? ? {A’’,G’’} (A’,G’) Our Approach (6) G  i Aii Gi  A t Q WF (A,G)  G'' i Aii Gi  A'' G A G'  (i Ai)A'' (i Gi)G'' A' A G NJPLS@Stevens

Our Approach (7) • Queue Extension WF(Q{t}, {(A, G)}) WF(Q{t',t},{(A’’, G’’), (AG’’, GA’’)}) fork f(a) A  A’’, G’’  G NJPLS@Stevens

Our Approach (8) • Queue Update WF(Q{t}, {(A, G)}) WF(Q{t}, {(A’, G’)}) AA’, G’G; t: (A’, G’) NJPLS@Stevens

Our Approach (9) (A1, G1) (A2, G2) Certify once, use everywhere? T1: . . . jmp f T2: . . . jmp f AiA, GGi (A, G) f: . . . exit NJPLS@Stevens

Our Approach (10) • Check static threads Lazy Check • Changing Env.  Changing (A, G) • Multiple instantiation  Not care • Modularity  Certify only once • General Enough • Language (higher-level/assembly) • Thread Model (preemptive/non-preemptive) NJPLS@Stevens

Example – Unbounded Thread Creation Global data: int data[100] main: int i:=0; while (i<100){ data[i]:=f(i); fork child(i); i++; } void child(x:int){ data[x] = g(x, data[x]) } NJPLS@Stevens

Example – Unbounded Thread Creation • Specification of Child: • Ax: • Gx: • Non-interference between children: NJPLS@Stevens

Example – Unbounded Thread Creation • How to specify the main thread? main: int i:=0; while (i<100){ data[i]:=0; fork(child, i); i++ } Do we need a G such that: But main cannot satisfy such a G! NJPLS@Stevens

(A, G) (A, G) (A, G) (A’, G’) (A, G) main: int i:=0; while (i<100){ data[i]:=0; fork(child, i); i++ } NJPLS@Stevens

The CMAP Framework • The abstract machine • The verification logic • Specification language • Inference rules • Soundness proof • Example programs • Unbounded dynamic thread creation • Readers/Writers problem • Lock-free program • All implemented in Coq! NJPLS@Stevens

f1: I1 … (data heap) H R R R f2: I2 I I I (dyn. queue) Q … 0 1 2 … (code heap) C add … fork h yield exit r1 r2 r3 … rn h1: I1 (register file) R h2: I2 … (state) S ::=(H,R) (thrd entries) T (instr. seq.) I (program) P ::=(C,T,S,Q,I) The CMAP Framework - Machine NJPLS@Stevens

The CMAP Framework The paper on CMAP (Feng&Shao ICFP’05): http://flint.cs.yale.edu/publications/cmap.html NJPLS@Stevens

Conclusion • Problems for unbounded dynamic thread creation • Changing environment (fork/exit) • Multiple instantiation of thread code • No previously known modular verification method • Our approach • INV: active threads in the system do not interfere • Combine the type-based proof technique with R-G method • Unify thread’s assumption/guarantee with env.’s guarantee/assumption • Thread modularity + code/proof reuse • The CMAP framework and its Coq implementation NJPLS@Stevens

Future Work • Certified Thread Libraries • fork, yield, exit • join, lock, monitors • Surface language • Higher-level specifications • Partially infer A and G • Certifying compilation to CMAP Where is the threads ? User-level thread + thread lib. NJPLS@Stevens

Thank you! NJPLS@Stevens

Modular Verification of Concurrent Assembly Code with Dynamic Thread Creation and Termination