QVM

1. An Efficient Runtime for Detecting Defects in Deployed Systems QVM Matthew Arnold Martin Vechev Eran Yahav

2. Motivation:Dynamic analysis for Debugging Testing • High overhead tolerable • Deep properties relating to program correctness • Production • Low overhead is mandatory • Very limited information

3. Motivation:Dynamic analysis for Debugging Testing • High overhead tolerable • Production • Very limited information • Production • Low overhead is mandatory Testing Deep properties relating to program correctness

4. QVM Approach Virtual Machines are mainstream Java, C#, Perl, Python, PHP, etc… Runtime support to improve program correctness Array bounds checking, GC, etc VM implementations focus on performance Advanced dynamic optimization technology QVM vision Start using VM technology to improving program correctness More advanced checking Efficient analyses Ubiquitous: simply turn on a VM flag

5. But Why Modify the VM? VM Disadvantages Portability Complexity Why not use: Aspects, JVMTI, java.lang.Instrument, bytecode inst. ??? VM Advantages VM only information: Free bits in object header Can walk the heap if we desire (GC) Performance Exploit dynamic optimization Dynamic updating of instrumentation (via OSR, etc) Ease of deployment

6. New Overhead Philosophy Traditional dynamic analysis If I use X, how much overhead will it cost me? QVM: reverse the question I am willing to give you X% overhead What can you do for me? User specifies an overhead budget Goal: give user as much useful information as possible May miss errors, or report “I don’t know” But enables somechecking in scenarios where it is currently infeasible

7. Contributions • Overhead manager (OHM) • Adapts analyses to meet user specified target overhead • Dynamic analyses checking correctness properties • Typestate checking • Object centric sampling • Heap Assertions • QVMI • Overhead aware interface for medium-granularity VM events • Allocations, method calls, class loads,…

8. QVM Architecture Application typestate specs violations report typestate client heap probes client assertions client Clients QVMI observed overhead event filters event callbacks Execution Engine specified overhead OHM adjust sampling rates VM Core QVM

9. QVMI: The QVM Interface • Profiling interface • Similar to JVMPI/JVMTI • Method calls, allocations, etc • Key Difference: filtering on the VM side event filters agent JVMTI event callbacks VM Execution Engine

10. QVMI: The QVM Interface event filters event filters agent JVMTI QVMI event callbacks event callbacks VM Execution Engine Execution Engine • Profiling interface • Similar to JVMPI/JVMTI • Method calls, allocations, etc • Key Difference: filtering on the VM side

11. QVMI: The QVM Interface At VM startup QVM profiling agents register with VM When compiling a method JIT queries QVM agents “Does invocation of method foo() require a call back?” If not, no callback is compiled into code Ensures no overhead for uninteresting events At runtime Callbacks go through QVM overhead can be measured

12. Overhead Manager (OHM) QVMI observed overhead event filters event callbacks Execution Engine specified overhead OHM adjust sampling rates VM Core Monitoring: measure overhead incurred by clients Sampling strategy: events callbacks have adjustable sample rate Controller: adjusts sample rate based on measured overhead

13. Overhead Manager Challenges Fine grained timers critical Large number of small timings is difficult Must have notion of “total application time” We use Linux getrusage Multi-threaded apps have issues See paper Analyses must be able to be “turned off” OK to miss bugs But must not produce meaningless results

14. Origin-specific sampling Randomly distributed sampling produces undesirable results Execution frequency Code eventA (…) eventB (…) eventC(…)

15. Origin-specific sampling Origin-specific sampling Maximizes code coverage Execution frequency Sample Counter Counter Reset Code 0 1 eventA (…) 0 1 eventB (…) 37 100 eventC(…)

16. QVM Clients Analyses Overhead of all clients controlled by Overhead Manager • Typestate property checker • Ordering of API calls • Heap Probes • Query properties of the heap • Java Assertions

17. Client 1: Typestate Property Checker b * dispose* | release* disposed else undisposed Objectallocation err Objectdeath *

18. Typestate Property Checker Simple to implement via QVMI Events used Object Allocation, method invocation, object death Sampling typestate is problematic Ex: File Open  Close High problem of sampling close but not open Solution: object-centric sampling

19. Object Centric Sampling assert (…) tracked tracked T t = new T() assert(…) • Tracked objects marked using bit in object header • Bit checked before executing callbacks

20. Use Case Example: Azureus Over 160 million downloads

21. Azureus Resource Leaks • Typestate checker for undisposed GDI resources • Actual QVM report: QVM ERROR: [Resource_not_disposed] object [0x98837030] of class [org/eclipse/swt/graphics/Image] allocated at site ID 2742 in method [com/aelitis/azureus/.../ListView.handleResize(Z)V] died in state [UNDISPOSED] with last QVM method invoked [org/.../Image.isDisposed()Z].

22. Client 2: Heap Probes • Heap Probes • Allow programmer to query properties of the heap • isShared(Object o1) • Do two or heap objects point to o1 • isThreadOwned(Thread t, Object o) • Is o reachable from only thread t only • Uses components of a parallel GC to evaluate heap queries • Worst case: requires traversal of entire heap • Probe sites automatically sampled by overhead manager

23. Azureus Example OS Resources imgView OS Resources class ListView extends ... { private Image imgView = null; // ... protected void handleResize(booleanbForce) { // ... if (imgView == null || bForce) { imgView = new Image(listCanvas.getDisplay(), clientArea); lastBounds = new Rectangle(0, 0, 0, 0); bNeedsRefresh = true; } else { // ... } // ... } }

24. Possible Fix OS Resources imgView OS Resources protected void handleResize(booleanbForce) { // ... if (imgView == null || bForce) { if(imgView != null && !imgView.isDisposed()) { assert(!QVM.isShared (imgView)); imgView.dispose(); } imgView = new Image(listCanvas.getDisplay(), clientArea); lastBounds = new Rectangle(0, 0, 0, 0); bNeedsRefresh = true; } else { // ... } // ... }

25. Experimental Evaluation

26. Overhead Manager: stabilization

27. Overhead Manager

28. Leak Detection Results

29. Sampling coverage (5% budget)

30. Summary • Recap • Adaptive overhead controller • Clients: typestate, assertions, heap probes • QVMI • Found and fixed bugs several real applications • Future Work • Improve efficiency of heap assertions • Concurrent or incremental evaluation • Overhead manager • Tighter overhead guarantees

31. The End