Dynamic Memory Management for Programmable Devices

1. Dynamic Memory Managementfor Programmable Devices Sanjeev Kumar & Kai Li Princeton University

2. D D CPU R R CPU Mem Mem D D Programmable Devices Main CPU Main Memory Bus Network Card Disk Network Move functionality from main CPUs to devices Dynamic Memory Managagement for Programmable Devices

3. ESP: A Language for Programmable Devices [PLDI’01] • Domain Specific Language with Three Goals • Ease of Programming • Ease of Debugging • Low Performance Overhead • Message-Passing Style Concurrency • An ESP program has concurrent processes communicating with each other by sending objects on channels. Dynamic Memory Managagement for Programmable Devices

4. Explicit malloc/free Fast Unsafe Automatic Garbage collection More CPU & memory Safe Concurrent program Safe Limited CPU & memory Fast How to Support Dynamic Memory Management in ESP? Dynamic Memory Managagement for Programmable Devices

5. Outline • Motivation • Dynamic Memory Management in ESP • Experimental Evaluation • Conclusions Dynamic Memory Managagement for Programmable Devices

6. ESP Approach • ESP supports explicit management • Explicit management is difficult • Global property of program • ESP makes this a local property of each process • Use a model checker to check correctness • Check each process separately • ESP supports safety through model checking Dynamic Memory Managagement for Programmable Devices

7. ESP Approach Contd. • How do you make memory allocation correctness a local property? • How to you use a model checker to verify memory allocation correctness? • How do you implement this efficiently? Dynamic Memory Managagement for Programmable Devices

8. Making Allocation Correctness a Local Property • Processes communicate only using channels • No shared mutable objects • No global variables • Only immutable objects can be sent on channels • Making allocation correctness a local property • No shared objects • Objects are sent over channels by value • Requires a “Deep Copy” of the object being sent • Pointer sharing not preserved during “Deep Copy” • Okay because ESP does not allow circular data structures Dynamic Memory Managagement for Programmable Devices

9. Each process is responsible for freeing its objects Making Allocation Correctness a Local Property Cont’d channelchannel1: recordT; channelchannel2: arrayT; process example { while( true) { $r: recordT = {4,true}; out( channel1, r); in( channel2, $a); free(r); free(a); } } • Objects received over channels are treated like allocation Dynamic Memory Managagement for Programmable Devices

10. Model Checking • State-space exploration of concurrent programs • Try all possible scheduling options • Advantages • Automatic • Produces counter example • Disadvantages • Computationally expensive (exponential) • ESP currently uses the Spin model checker Dynamic Memory Managagement for Programmable Devices

11. Verify Memory Safety Of Each Process pgm.C help.C pgm.ESP p1.spin test1.spin ESP Compiler Generate Firmware pN.spin testN.spin Using Model Checking to Verify Allocation Correctness Dynamic Memory Managagement for Programmable Devices

12. Extracting a Model to Verify Allocation Correctness • ESP processes and channels are translated into Spin processes and channels • Keep track of allocated objects • On allocation and assignment: Mark the pointer to the object “Live” • On free: Mark all pointers to the object “Dead” • On access: An assertion to check if the object being accessed is “Live” • All assertions are verified during model checking • This ensures memory allocation correctness Dynamic Memory Managagement for Programmable Devices

13. Efficient Code Generation • Semantically, objects are not shared by processes • In the generated code, objects are shared between the processes • Avoids object copying • Maintains referencecounts for each object • Involves a number of subtle details Dynamic Memory Managagement for Programmable Devices

14. Outline • Motivation • Dynamic Memory Management in ESP • Experimental Evaluation • Conclusions Dynamic Memory Managagement for Programmable Devices

15. High-performance communication Bypass OS for data transfers Used Myrinet network cards Gigabit network 33 MHz CPU, 1 MB memory VMMC firmware Implemented using ESP 8 Processes and 19 channels 500 lines of ESP and 3000 lines of C Application Main CPU Data OS Network Card VMMC Firmware Network VMMC Firmware using ESP Dynamic Memory Managagement for Programmable Devices

16. Using Model Checking to Verify Allocation Correctness • Checking VMMC firmware using Spin • Found a bug in an earlier version • Found all bugs that were deliberately inserted • including memory leaks • Could perform model checking exhaustively • Each process can be checked separately • Avoided state-space explosion Dynamic Memory Managagement for Programmable Devices

17. Using Model Checking to Verify Allocation Correctness Cont’d Dynamic Memory Managagement for Programmable Devices

18. Performance Overhead • Additional overhead of reference counting • Difference between • Total memory management cost in ESP • Cost of using a simple malloc/free interface • Measure the overhead along three common execution paths in the VMMC firmware Dynamic Memory Managagement for Programmable Devices

19. Performance Overhead Cont’d Dynamic Memory Managagement for Programmable Devices

20. Outline • Motivation • Dynamic Memory Management in ESP • Experimental Evaluation • Conclusions Dynamic Memory Managagement for Programmable Devices

21. Conclusions • Dynamic Memory Management in ESP • Supports explicit memory management • Allocation correctness is local to a process • Model checking used to verify correctness • Reference counting used for efficiency • VMMC firmware • Could verify allocation correctness exhaustively • Low performance overhead for reference counting • Between 1.5% and 5.35% Dynamic Memory Managagement for Programmable Devices

22. Questions? Dynamic Memory Managagement for Programmable Devices