Enforcing Non-safety Security Policies with Program Monitors

1. Enforcing Non-safety Security Policies with Program Monitors Jay Ligatti (Princeton University); joint work with: Lujo Bauer (Carnegie Mellon University), David Walker (Princeton University)

2. Security Policy Enforcement • News flash:Software sometimes does bad stuff • Bugs • Malicious design • One mitigation is run-time monitoring • Ensure that software adheres to run-time constraints specified by a security policy • Stack inspection, file access control, applet sandboxing, firewalls, resource monitors, …

3. Run-time Program Monitors Untrusted Target Program Monitor Executing System Open(f,“w”) is OK Open(f,“w”) Open(f,“w”) • Monitors enforce policies by: • Interposing between untrusted code and the system executing the untrusted code • Making sure only safe code is executed

4. Research Agenda • What are the limits of program monitors?Which policies can monitors enforce? • Understanding limits helps us when selecting enforcement mechanisms • Allows us to choose the simplest mechanism that enforces a policy • Prevents us from attempting to enforce a policy with an insufficiently strong mechanism

5. Outline • Motivation • Which policies can monitors enforce? • Defining “monitor enforces a policy” • Define policies • Define monitors • Define enforcement of policiesby monitors • Delineating the enforceable policies • Infinite renewal properties • Comparison of renewal, safety, and liveness • Summary

6. Systems and Executions • System = a state machine that transitions states by executing actions • We specify a system according to the possibly countably infinite set of actions it can execute A = { open f, close f } (where f is any file name) • Execution = possibly infinite sequence of actions open f; close f open f1; close f1; open f2; close f2; …

7. Execution Notation • On a system with action set A,A* = set of all finiteexecutionsAω = set of all infiniteexecutionsA∞ = set of allexecutions • The empty sequence (execution) is ● • The concatenation of sequences s and u is s;u • Prefix notation: s≤u (or u≥s) • Means: s is a finite prefix of possibly infinite u • Read: “s prefixes u” (or “u extends s”)

8. Policies [Schneider ’00] • Policy = predicate P on sets of executions • Set of executions SÍA∞satisfiesP iff P(S) • Termination policy • P(S) iff every execution in S is finite • Key uniformity policy • P(S) iff the cryptographic keys used in all executions in S form uniform distribution

9. Properties [Schneider ’00] • Policy P is a property iff $ predicate p on executions such that "SÍA∞ , P(S) Û "uÎS : p(u) • Properties cannot specify relationships between executions • Termination is a property • Key uniformity is not a property • Monitors enforce properties, not general policies

10. Safety and Liveness [Lamport ’77; Alpern, Schneider ’85] • Two types of properties have been studied a lot • Safety: “Bad executions cannot be made good” "sÎA∞ : Øp(s) Þ$s’≤s : "u≥s’ : Øp(u) • Access-control (cannot “undo” illegal accesses) • Liveness: “Finite executions can be made good”"sÎA* : $u≥s : p(u) • Termination and nontermination

11. Outline • Motivation • Which policies can monitors enforce? • Defining “monitor enforces a policy” • Define policies • Define monitors • Define enforcement of policiesby monitors • Delineating the enforceable policies • Infinite renewal properties • Comparison of renewal, safety, and liveness • Summary

12. Operation of Monitors: Accepting an OK Action Untrusted Target Program Monitor Executing System Open(f,“w”) is OK Open(f,“w”) Open(f,“w”) Monitor inputs actions from target and outputs actions to the executing systemHere, input action is safe to execute, so monitor accepts it (makes it observable)

13. Operation of Monitors: Suppressing an Action Untrusted Target Program Monitor Executing System Open(f,“w”) is not OK Open(f,“w”) Input action is not safe to execute, so monitor suppressesit and allows target to continue executing

14. Operation of Monitors: Inserting an Action Untrusted Target Program Monitor Executing System Open(f,“w”) is not OK Open(f,“w”) Close(f,“w”) Input action is not safe to execute, so monitor inserts another action, then reconsiders the original action

15. Modeling Monitors • Model a monitor that can accept, suppress, and insert actions as an edit automaton (Q,q0,t) • Q is finite or countably infinite set of states • q0 is initial state • A complete, deterministic, and TM-decidable function t : Q x A ® Q x (A U {●}) suppress input action current state input action new state action to insert

16. Operational Semantics • A monitor’s transition function defines how it behaves on individual input actions • For the definition of enforcement, we need to consider the way monitors transform entire input executions Monitors are execution transformers Valid output Untrusted input a1;a2;a2;a3;… a1;a2;a2;a4;… Monitor

17. Operational Semantics Judgments • Desired judgment: (q0,s) X ßu • Automaton X starting in state q0transforms input sequence s into output sequence u • Build up to this judgment 1. Single-step judgment (q,s) X ®u (q’,s’) 2. Multi-step judgment (q,s) X Þu (q’,s’) 3. Transforms judgment (q0,s) X ßu

18. Outline • Motivation • Which policies can monitors enforce? • Defining “monitor enforces a policy” • Define policies • Define monitors • Define enforcement of policiesby monitors • Delineating the enforceable policies • Infinite renewal properties • Comparison of renewal, safety, and liveness • Summary

19. Effective Enforcement • Effective enforcers adhere to two principles • Soundness: • Monitors’ outputs (i.e., observable executions) must satisfy the desired property • Transparency: • Monitors must not alter the semantics of valid inputs • Conservative definition: on a valid input sequence s, a monitor must output s

20. Enforcing Properties • Automaton X starting in q0 enforcesp on a system with action set A iff "sÎA∞: $uÎA∞:1. (q0,s) X ßu2. p(u) [Soundness]3. p(s) Þ (s=u)[Transparency]

21. Outline • Motivation • Which policies can monitors enforce? • Defining “monitor enforces a policy” • Define policies • Define monitors • Define enforcement of policiesby monitors • Delineating the enforceable policies • Infinite renewal properties • Comparison of renewal, safety, and liveness • Summary

22. Enforcement Powers Related Work • Previous work has examined the enforcement bounds of monitors [Schneider ’00; Viswanathan ’00; Hamlen, Morrisett, Schneider ’03; Fong ’04] • In the previous models, monitors respond to dangerous actions by halting the target outright • Our framework considers more expressive monitors that can accept, suppress, and insert actions

23. Enforcing Properties with Edit Automata • Powerful enforcement technique • Suppress (feign execution of) potentially bad actions, and later, if the suppressed actions are found to be safe, re-insert them • Limitation: model assumes monitors have same computational power as observing system • In practice, some actions cannot be feigned: • Actions requiring an outside system to execute • Time-dependent actions

24. Renewal Properties • Technique of suppressing dangerous actions allows edit automata to enforce any reasonable infinite renewalproperty • Renewal: “Infinite executions are good iff they are good infinitely often” "sÎAω : p(s) Û {u≤s | p(u)} is an infinite set

25. Example Renewal Property • Check subsystem configuration after rebooting: Whenever reboot is called, it must be followed by a call to checkConf • Is a renewal property: • A valid infinite execution has infinitely many valid prefixes (in which checkConf always follows reboot) • An invalid infinite execution has only finitely many valid prefixes • There must be some invalid prefix where thenext action after a reboot is not checkConf — all successive prefixes are invalid

26. Edit Automata Enforcement(Lower Bound) • Theorem: " properties p such that 1. p is a renewal property, 2. p(●), and 3. "sÎA* : p(s) is decidable, $ an edit automaton that enforces p. Edit automata can enforce any reasonable renewal property

27. Edit Automata Enforcement (Lower Bound) • Proof idea: Technique of suppressing actions until they are known to be safe causes every valid prefix, and only valid prefixes, of the input to be output • Given a renewal property p, construct an edit automaton X that uses this technique • In all cases, X correctly enforces p • If input s has finite length, X outputs longest valid prefix of s • Else if Øp(s) and s is infinite, X outputs the longest valid (finite) prefix of s (since p is a renewal property) • Else if p(s) and s is infinite, X outputs every prefix of s and only prefixes of s

28. Edit Automata Enforcement (Precise Bounds) • Edit automata enforce exactly the set of reasonable renewal properties, plus some extra “almost renewal” properties • Details are in ESORICS ’05 proceedings

29. Outline • Motivation • Which policies can monitors enforce? • Defining “monitor enforces a policy” • Define policies • Define monitors • Define enforcement of policiesby monitors • Delineating the enforceable policies • Infinite renewal properties • Comparison of renewal, safety, and liveness • Summary

30. Definition Reminder • Safety: “Bad executions cannot be made good” • Liveness: “Finite executions can be made good” • Renewal: “Infinite executions are good iff they are good infinitely often”

31. Example Renewal Property, Revisited • Check subsystem configuration after rebooting: Whenever reboot is called, it must be followed by a call to checkConf • Is not a safety property: Øp(reboot) but p(reboot;checkConf) • Is not a liveness property: reboot;a cannot be “made good” when a¹checkConf

32. Safety, Liveness, Renewal All Properties 1 File access control 2 Trivial 3 Eventually audit 4 Check configuration after rebooting 5 Termination 6 Termination + File access control Renewal Safety Liveness 1 2 3 5 4 6

33. Outline • Motivation • Which policies can monitors enforce? • Defining “monitor enforces a policy” • Define policies • Define monitors • Define enforcement of policiesby monitors • Delineating the enforceable policies • Infinite renewal properties • Comparison of renewal, safety, and liveness • Summary

34. Summary • Monitors modeled by edit automata can enforce any reasonable renewal property • Renewal properties include all safety properties, some liveness properties, and some properties that are neither safety nor liveness

35. End Thanks / Questions