Efficient Software-Based Fault Isolation R. Wahbe, S. Lucco, T. E. Anderson, S. L. Graham

1. Efficient Software-Based Fault IsolationR. Wahbe, S. Lucco, T. E. Anderson, S. L. Graham Xiaoliang Zhang CS533 Concepts of Operating Systems

2. Motivations • Why a modular operating system? • Extensibility: Application programs can incorporate independently developed software modules. (e.g. Micro-kernel systems, Microsoft OLE ) • Hardware-based protection. • What’s the problem among cooperating modules? • Faults in extension code can render a software system unreliable, or even corrupt permanent data) • How to provide fault isolation among modules? • Place each module in its own address space • But this solution incurs prohibitive context switch overhead for tightly-coupled modules. • E.g. RPC cost with hardware-based protection. • Alternative solution: Sandboxing (Software-based protection).

3. What is sandboxing? • An assembly-language-level software approach to implement fault isolation within a single address space • Load the code and data for a distrusted module into its own fault domain, a contiguous region of memory within the application’s address space. • Modify the object code of a distrusted module to prevent it from writing or jumping to an address outside its fault domain. • An cross-fault-domain RPC interface (much cheaper than RPC) is used for inter-fault-domain communications.

4. Which systems need sandboxing • Sandboxing makes faster communication between fault domains. • But increases the execution time for distrusted modules due to encapsulation. • May seem to be counter-intuitive • Slowing down the common case (normal execution) to speed up the uncommon case (cross domain communication) • Good for systems that require frequent communication between fault domains. • Micro-kernel systems, User-programmable high performance I/O systems, User-level examples (e.g. Quark Xpress desktop publishing system) etc.

5. What is a fault domain? • An application’s virtual address space is divided into segments, aligned so that all virtual addresses within a segment share a unique pattern of upper bits, called the segment identifier. • A fault domain consists of two segments, one for a distrusted module’s code, the other for its static data, heap and stack. • Determined at load time. • Software encapsulation transforms a distrusted module’s object code so that it can jump only to legal targets and write only to legal addresses within its data segment.

6. Segment matching • Static verification • Target address is known at compiling time • Unsafe instruction • Jumps to or stores to an address which cannot be statically verified to be correct segment (until dynamically verified at runtime). • It could corrupt critical data. • Segment matching • Insert checking code before every unsafe instruction. The checking code determines whether the unsafe instruction’s target address has the correct segment identifier.

7. Pseudo code of segment matching Example: An unsafe instruction: Code segment ID = 0011 Store target address = 10001001; Data segment-reg = 0001; shift-reg = 100; dedicated-reg = 10001001 scratch-reg = dedicated-reg>>shift-reg = 1000, segment-reg(0001) <> scratch-reg=1000; A trap is generated to trigger a system error routine outside the distrusted module’s fault domain. If they match, store instruction uses dedicated-reg = 10001001.

8. Why dedicated registers? • Why not simply use scratch-reg: scratch-reg (dedicated-reg)<= (target address>>shift-reg) compare scratch-reg and segment-reg • An instruction can jump to the unsafe store/jump instruction, bypassing the checking instructions. • Dedicated registers are used only by inserted code and are never corrupted by code in the distrusted module.

9. Address Sandboxing • Segment matching can pinpoint the offending instruction. • Sandboxing reduces runtime overhead further, at the cost of providing no info about the source of faults. • Before each unsafe instruction we simply insert code that sets the upper bits of the target address to the correct segment ID.

10. Pseudo code to sandboxing Example: An unsafe instruction: Code segment ID = 0011 Store target address = 10001001; Data segment-reg = 0001; and-mask-reg = 00001111. dedicated-reg <= target-reg & and-mask-reg = 00001001 dedicated-reg <= dedicated-reg | segment-reg = 00011001 store value using dedicated-reg 00011001 in this fault domain will be corrupt! Sacrifice for other fault domains.

11. Optimizations • RISC architectures include a register-plus-offset instruction mode. • “store value, offset(reg)”, whose address offset(reg) uses the register-plus-offset addressing mode. • Sandboxing this instruction requires 3 inserted instructions: • One to sum reg+offset into the dedicated-reg; • Two sandboxing instructions to set the segment ID of the dedicated-reg • Optimize this case by sandboxing only the reg, rather than reg+offset. • Guard zones are created at the top and bottom of each segment.

12. Optimizing stack pointer sandboxing • The MIPS stack pointer is treated as a dedicated register. The stack pointer is only sandboxed when it is set. • We can avoid sandboxing the stack pointer after it is modified by a small constant offset as long as the modified stack pointer is used as part of a load or store address before the next control transfer instruction. • Remove sandboxing sequence from loops, in cases where a store target address changes by only a small offset during each loop iteration.

13. Process resources • Preventing distrusted modules from corrupting resources that are allocated on a per-address-space basis. • Solution 1 (not portable): Modifying the OS to know about fault domains. • Solution 2: Distrusted modules must access system resources only through cross-fault-domain RPC. • A fault domain is reserved to hold trusted arbitration code which determines whether a particular system call performed by some other fault domain is safe.

14. Data sharing • Lazy Pointer Swizzling: A technique to share read-write memory region among fault domains with no additional runtime overhead • Modify the hardware page tables to map the shared memory region into each segment at the same offset. • As the shared memory is accessed, sandboxing automatically translates shared address into the corresponding address within the fault domain’s data segment.

15. Implementation • Approach 1 (not language independent): • Encapsulated by modified compiler • Integrity is verified in loading time (into a fault domain). • Approach 2: • Encapsulate the distrusted module by directly modifying its object code at load time.

16. Fast RPC across fault domain • Control escapes a distrusted fault domain only via a Jump Table (to stub instruction). Talk about it later. • A call-stub and a return-stub are created for each pair of fault domains. • The stub is also responsible for managing machine states and registers. • Fatal errors are handled by UNIX signal facility. Trusted modules may use a timer facility to interrupt execution or determinate a call.

17. Jump Table • Allows a fault domain to safely call the a trusted stub routine outside its domain; that stub then safely calls into the destination domain. • Each jump table entry is a control transfer instruction whose target address is a legal entry point outside the domain. • Because the table is kept in the (read-only) code segment in the fault domain, it can only be modified by a trusted module.

18. Performance Result • Sandboxing incurs an average of 4% execution time overhead on a DECstation 5000/240 and a DEC Alpha 400. • Sandboxing lowers RPC cost by more than an order of magnitude. (On Mach3.0, cross-AS RPC 314 times more than procedure call)

19. Performance Result • Software-based fault isolation avoids hardware context switches, substantially reducing crossing costs.

20. Conclusions • Despite the overhead in software encapsulation, software-based fault isolation will often yield the best overall application performance. • Software-based fault isolation will be the better performance choice whenever the overhead of using hardware-based RPC is greater than 5%.