Enhancing Unit Testing for Concurrent Programs: A Novel Probabilistic Approach

Testing Concurrent Programs, A 7-Minute Jargon-Free Introduction Thesis Writing Seminar Mathias Ricken Rice University February 25, 2010

Program bugs Slow computers Computer Annoyances

Unit Testing • Test program parts, not the whole program • Test parts individually, then together • Smaller parts  larger parts • Unit tests are programs • Compare expected result to actual result: assertEquals(12, multiply(3,4));

Moore’s Law Adopted fromSutter 2009

Existing Approaches Fail • Simple Unit Testing Frameworks • Example: JUnit • Assume program is single-threaded • Many errors go undetected Main thread: detected Other thread: not detected

Problem: Non-Determinism • Synchronization points • Information exchanged between threads • Order of threads reaching sync. points is non-deterministic

Problem: Non-Determinism • Synchronization points • Information exchanged between threads • Order of threads reaching sync. points is non-deterministic • Program could succeed in one order but fail in another

Comprehensive Tools: Costly • Model checking or schedule-based execution • Examples: JPF, ExitBlock • Test all arrangements • Too costly t = # of threads s = # of sync. points N = # of arrangements Longer than the universe exists!(if N=1  1 minute)

Probabilistic Approach • Insert random delays before synchronization points to modify the order • Run tests many times with different delays

Probabilistic Approach • Insert random delays before synchronization points to modify the order • Run tests many times with different delays delay

(Proposed) Contributions • Probabilistic Approach • Random delays at synchronization points • Call Graph Analysis • Determine which parts of the program may interact • Only insert delays where necessary • Improved JUnit Framework • Exception handler for all threads • Ensures that all errors in all threads are detected

Evaluation • Tested DrJava with Improved JUnit • Number of unit tests: 900 • Previously unknown problems: 20 • Slowdown: ~1 percent • Other parts still under development

Conclusion • Unit testing of concurrent programs will become more important • Concutest helps… • by improving JUnit’s abilities • by using probabilistic techniques • by using call graph analysis to reduce the number of synchronization points

Image Attribution • Left image on Computer Annoyances:Adapted from Microsoft • Graph on Moore’s Law:Adapted from Herb Sutter 2009 • Top image on Multi-Cores and Concurrency:Adapted from Brian Goetz et al. 2006, Addison Wesley • Bottom image on Multi-Cores and Concurrency:Caption Fridays

Extra Slides

Multi-Cores and Concurrency • Recent trends • Clock speed stagnated • Transistors still increasing • Reason: multi-core CPUs • Implication • Must use concurrency to benefit from modern CPUs • Concurrent programming is difficult

Number of Arrangements • Number of schedules (N) • t: # of threads, s: # of slices per thread • Note: s is # of slices (between synchronization points)

Number of Arrangements Product of s-combinations For thread 1: choose s out of ts time slices For thread 2: choose s out of ts-s time slices … For thread t-1: choose s out of 2s time slices For thread t-1: choose s out of s time slices Writing s-combinations using factorial Cancel out terms in denominator and next numerator Left with (ts)! in numerator and t numerators with s!

Enhancing Unit Testing for Concurrent Programs: A Novel Probabilistic Approach

Télécharger la présentation

Plus de contenu associé