Scalable and Effective Test Generation for Access Control Systems

1. Scalable and Effective Test Generation for Access Control Systems Ammar Masood School of Electrical & Computer Engineering Purdue University 11th September, 2006 1

2. Outline • Introduction • Problems and Contributions – Part A • Details of Proposed Solutions – Part B • Conclusion and Future Work 2

3. Motivation and Challenges • Protection of information from unauthorized access or modification and protection against denial of service to authorized users is an important security requirement • Access control is one of the key security service providing the support for secure information access • Desired access control objectives only achieved if the underlying implementation conforms to the policy, hence testing becomes essential • Key challenge: how to devise scalable and effective test generation techniques ? 3

4. Requirement for Testing • A number of vulnerabilities are related to design and/or coding flaws in access control modules of an application* • OSVDB reports 53 vulnerabilities related to access control • NVD which records CVE and CERT advisories reports 859 vulnerabilities with impact “provides unauthorized access” and type “access validation error”, 1440 for any impact • Security Focus reports 80 vulnerabilities for the key word “access control” • Formal verification and static or dynamic program-analysis techniques only guarantee correctness of design • Testing is required to detect any faults in the implementation due to, for example, coding errors and incorrect configuration *Data as of 8/30/06 4

5. Conformance and Functional Testing 5

6. Testing Context 6

7. RBAC is a promising approach for addressing diverse security needs of business organizations Access control in organizations is based on “roles that individual users take on as part of the organization” A role is “is a collection of permissions” Constraints are applied to all the links Role Based Access Control (RBAC) and Temporal RBAC TRBAC extends RBAC by imposing duration constraints on user-role assignments/activations and permission-role assignments 7

8. Outline • Introduction • Problems and Contributions – Part A • Details of Proposed Solutions – Part B • Conclusion and Future Work 8

9. Contributions • RBAC fault model • Test generation for RBAC Systems • A Probabilistic model for fault coverage • An empirical evaluation • Test generation for TRBAC Systems • Behavior modeling of TRBAC systems • TRBAC conformance testing 9

10. 1. RBAC Fault Model • Required to study fault coverage of any test generation technique • Proposed fault model comprises • Mutation-based (simple) faults • Non-mutation (malicious) faults • Behavioral conformance used to study the fault model 10

11. 2. Test Generation for RBAC Systems • Requirements :- • Effectiveness – fault detection effectiveness measured with respect to RBAC fault model • Scalability – the cost of test generation and execution • Existing research – Chandarmouli and Blackburn functional testing technique for Discretionary Access Control • Effectiveness not considered • Not amenable for fault coverage analysis 11

12. Proposed Solution • Set of conformance testing procedures with varying cost and effectiveness • Procedure A : Complete-FSM based • Procedure B : Heuristics based • Procedure C : Constrained Random Test Selection (CRTS) strategy based • Procedure A is most effective – complete fault coverage for simple faults and a class of malicious faults – and most costly • Cost and effectiveness of Procedures B and C varies with the heuristic considered for test generation and the length of test cases in the CRTS suite 12

13. Proposed Solution (continued) • Functional Testing • Required to ensure that ACUT conforms to all RBAC policies • Proposed methodology is based on policy meta test set • White box coverage criteria used as a feed back mechanism to establish correctness of ACUT functionality • The functional testing technique is generic in that it can be used for TRBAC systems 13

14. 3. A Probabilistic Model for Fault Coverage • Requirement • A mechanism for analytically comparing fault coverage of heuristics and CRTS strategy based test generation techniques • Existing research • Petrenko et. al. use mutation based approach to access fault coverage of tests for FSM’s • One-to-one relation between faults and structural mutants • Not suitable for our analysis because of many-to-many relation between RBAC/TRBAC faults and structural mutants 14

15. Proposed Approach • Coverage matrix used to model relation between FSM and RBAC faults • Faults exhibited randomly across the FSM transitions • Fault coverage analytically studied for two general cases of fault distribution (uniform and non-uniform) • Simulation:- To study fault coverage of test generation techniques for fault distributions achieved as mix of uniform and non-uniform distributions • High coverage of all techniques for uniform case • Coverage drops as distribution limits to complete non-uniform case • Coverage directly proportional to the number of transitions in the test suite 15

16. 4. Empirical Evaluation • To study cost and effectiveness of use of all the procedures in functional testing of an RBAC system • Based on X-GTRBAC prototype system • X-GTRBAC consists of • Policy initializer • Policy enforcer (ACUT) • Fault detection effectivenessmeasured through program mutation and manual injection of malicious faults • Program mutants manually associated with RBAC faults (simple faults) • Cost measured in terms of total number of state queries performed in the execution of a test suite 16

17. Results • Procedure A most effective and most costly • Heuristics and CRTS strategy perform equally well for simple faults but heuristics lag CRTS strategy in detecting malicious faults • Effectiveness of CRTS increases as length of tests included in the suite increases, cost also increases but is significantly less then that of Procedure A • Reasons: • Heuristics by design fail to consider a holistic view of the system • Simple faults are exhibited across much higher number of transitions as compared to malicious faults, thus easier to detect • CRTS randomly select paths of fixed length from complete FSM, thus as length of tests increases there are more chances of inclusion of higher length paths in the CRTS test suite 17

18. Recommendations to Practitioner • Although only Procedure A provides complete fault coverage it could be prohibitively expensive • CRTS strategy provides the balance between cost and effectiveness • Reaffirmation of usefulness of white-box criteria to enhance tests generated using black-box approach • Malicious faults likely to be missed easily by the heuristics • As exhaustive testing not a viable option, functional testing requires white-box criteria as a feed back mechanism to determine the stopping point 18

19. Comparison with Simulation Results • Fault coverage results for the case of uniform fault distribution in the simulation are close to case study results for simple faults • Given a test generation technique, the analytic result of fault coverage for uniform fault distribution may be used as a predictor of its effectiveness in detecting simple faults • Wide disparity between coverage results for the simulation and for the case study for malicious faults • Logical result as malicious faults are injected with malicious intent, thus can not be modeled with uniform distribution 19

20. 5. Test Generation for TRBAC Systems • Require effective and scalable test generation technique • How to measure effectiveness? • TRBAC fault model (extensions in RBAC fault model) • Scalability ? • Determined by the size of the test suite (size of model) • Why can’t existing approaches for test generation be directly used for TRBAC test generation? • Techniques for RBAC system not usable as simple FSM’s cannot capture real-time considerations • Solution – use Timed Input Output Automata (TIOA) to model TRBAC • TIOA based test generation techniques • Symbolic clustering of states – scalable but effectiveness not measurable • State characterization set based (Timed-Wp method) – effective but not at all scalable • TIOA transformation to FSM (se-FSA based) – effective and scalable 20

21. Proposed Approach 21

22. Behavior modeling of TRBAC systems • Requirement • Model correctly specify the behavior implied by the TRBAC specification • TRBAC model (TRBACM) is based on TIOA • Two options in constructing TRBACM • Construct a single monolithic model • Divide the system into parts – compositional construction • TRBACM= URM || PRM • TRBACM is proved to correctly model the TRBAC specification 22

23. TRBAC conformance testing • Key steps • Transformation of TRBACM into se-FSA • Constructing the test tree corresponding to the se-TRBACM • Use of an Integer Programming (IP) based approach to generate the conformance test suite • Fault detection effectiveness • Provides complete fault coverage by virtue of correctness of TRBACMand the correlation between TRBAC, TIOA and se-FSA faults • Heuristics can be used to reduce the model size and thus the size of the corresponding test suite • May result into reduced fault detection effectiveness, can be analytically studied for cases of fault distribution using the probabilistic model 23

24. Outline • Introduction • Problems and Contributions – Part A • Details of Proposed Solutions – Part B • Conclusion and Future Work 24

25. Conformance Relation • Based on behavioral conformance • Specified using the two conditions, which informally imply that ACUT • assigns (deassigns) and activates (deactivates) a role only if such assignment (deassignment) and activation (deactivation) is allowable by the current policy in effect • assigns (deassigns) a set of permissions to (from) a role only if allowable by the current policy in effect • ignores ill-formed requests 25

26. RBAC Fault Model • Conformance between ACUT and ACUT implies absence of any faults in the ACUT i.e. faults in P • Conformance testing of ACUT can thus be considered as verifying that P does not belong to set of faulty policies • RBAC fault model defines the set of faulty policies • Obtained using mutation based approach [Petrenko et.al.] • Three types of operators used for mutating the elements of RBACP • Set mutation operators • Element modification operators • Rule mutation operators 26

27. Malicious Faults • Counter based • A specific count of events leads to fault • I/O based • Faults based on malformed requests • Sequence-based • A specific sequence of events leads to fault 27

28. Conformance Testing Procedures • Behavior implied by a policy expressed as an FSM. • Heuristics applied to scale down the model. • Use the W-method, or its variant, to generate tests from the complete (Procedure A) or scaled down model (Procedure B) or randomly select paths of fixed length from the completemodel (Procedure C) 28

29. 0000 DS11 DS21 AS21 AS11 DS21 DS11 1000 0010 DS11 AS21 AC21 AS11 DS21 AC11 DC11 DS21 DC21 DS11 1100 1010 0011 DS21 DS11 AC21 DC21 AC11 DC11 AS21 AS11 1110 1011 Sample FSM Two users, one role. Only one user can activate the role. Number of states≤32. AS: assign. DS: De-assign. AC: activate. DC: deactivate. Xij: do X for user i role j. 29

30. Heuristics H1: Separate assignment and activation H2: Use FSM for activation and single test sequence for assignment H3: Use single test sequence for assignment and activation H4: Use a separate FSM for each user H5: Use a separate FSM for each role H6: Create user groups for FSM modeling. 30

31. 00 00 AS11 AC11 AC21 DS11 DS21 DC11 DC21 DS21 DS11 AC21 AC11 10 10 AS21 01 01 11 AC21 00 AC11 00 AS21 AS11 DS21 DS21 DS11 DS11 AC21 AC11 10 11 10 11 DC21 DC11 Reduced Models Assignment Machine Activation Machine Heuristic 1 User u1 Machine User u2 Machine Heuristic 4 31

32. Procedure C: CRTS Strategy • Constructs a pool RTi of n random tests. • Lengths of all tests in the pool RTi is same, i.e. i which is selected to be comparable with the length of longest test generated using Procedure A • The total number of tests n is selected based on comparison with the maximum number of tests generated using the heuristics (Procedure B) • Construct five test suites RTi1,…., RTi5 by randomly selecting fixed number p of tests from RTi • p empirically chosen based on economical or statistical criterion 32

33. Probabilistic Model for Fault Coverage • State observability assumed • Based on Coverage matrixCx, x {H0, H1,…, RTi} • Visibility of faults among transitions is given by x=b. Cx where b is a identity row vector of length j • Fault Coverage (FCx) is computed as where 33

34. Boundary Cases of Fault Distribution • |F|=j=|TH0|, such that one-to-one correspondence between faults and transitions, FCx= # of transitions in x/j • If x1 covers more transitions then x2 FCx1 > FCx2 • Single fault f with equal probability of being exhibited across any transition t TH0 • Fault coverage of x is now the probability of detecting f using x 34

35. General Cases of Fault Distribution • Case A: The total number of transition across which each fault f is exhibited is uniformly distributed • Case B: Total number of faults is more than 1, each fault f has equal probability of being exhibited across any transition t TH0 35

36. Simulation • Five cases of fault distribution • Cases 0 and 4 – same as Cases A and B • Cases 1, 2 and 3 – respectively correspond to cases in which 75%, 50% and 25% of faults are uniformly exhibited (as per Case 0) rest as per Case 4 • Metrics used for comparison of testing generation techniques • Average fraction of faults detected • Probability of detecting all faults p(F) • Setup • 10,000 iterations • 5 values of fault density 0.01, 0.05, 0.1, 0.2 and 0.5 36

37. Results : Average fraction of Faults Detected • Common trend for all cases of fault distribution • Expected as faults are independently and identically exhibited • High coverage for all techniques for Case 0 • As fault distribution limits to Case 4, coverage reduces dramatically for techniques with less number of transitions in their test suites 37

38. Results : Probability of Detection of all Faults • p(F) reduces considerably with increase in fault density • Expected as p(F) is the product of probabilities for detection of individual faults • As fault distribution limits to Case 4, the exponential term in p(F) corresponding to Case 4 dominates • No test generation technique other than the complete FSM based, provides guarantee of detecting all faults • Solution – use white box adequacy criteria for test enhancement 38

39. 39

40. Empirical Evaluation : Setup • Study carried out using the proposed functional testing methodology • Stopping criterion – complete coverage of simple faults • Policy meta set – comprises two policies • Meta test sets – corresponding to the three procedures • Test generation techniques used • H3, H4 and H5 heuristics • RT4, RT6, RT10 and RT100 • 100 tests in each test suite RTij 40

41. Empirical Evaluation : Results 41

42. Empirical Evaluation and Simulation Results Comparison 42

43. TRBAC Fault Model • Conformance relation similar to the one for RBAC systems • Addition of a condition to consider temporal conformance • RBAC fault model extended by changing the application of rule mutation operator, result is addition of three temporal faults 43

44. Timed Input Output Automata (TIOA) 44

45. ?AC(u1,r1,t2) L0 L0 URassign(u1,r1)=0, URactive(u1,r1)=0 L1 URassign(u1,r1)=1, URactive(u1,r1)=0 L2 URassign(u1,r1)=1, URassign(u1,r1)=1 x1=t1 !DS(u1,r1) x1=t1 !DS(u1,r1) ?AS(u1,r1,t1) x1:=0 L1 L2 x2=t2 !DC(u1,r1) ?AC(u1,r1,t2) x2:=0 TRBAC Modeling • TRBACM= URM || PRM • URM=URb1 ||ur URb2 ||ur, …,||ur URbk , three types of URb’s corresponding to user-role (UR) pairs with • Explicit assignment information • No explicit assignment and implicit activation • No explicit assignment but implicit activation 45

46. TRBAC Modeling (continued) • PRM=PRb1 ||pr PRb2 ||pr, …,||pr PRbk , two types of PRb’s corresponding to permission-role (PR) pairs with • Explicit assignment information • No explicit and implicit assignment • Example: Three permissions p1, p2and p3 , three roles r1, r2 and r3, r2 I r3 • p2r1 , p3r1 and p1r2 explicit assignment 46

47. Sample TRBACM • Example policy with a user u1 two roles {r1, r2} • Constraint: u1 cannot be simultaneously assigned to both roles • No permissions considered thus TRBACM= URb(u1,r1) ||ur URb(u1,r2) 47

48. ?AS(u1,r1), Set(x1,4) ?AC(u1,r1,t2) 0<x1<4 x1<x2 0<x1 0<x2 4<x1 x1<x2 0<x1<4 0<x2<2 4<x1 0<x2<2 0<x1<4 2<x2 4<x1 2<x2 l1 l0 l0 l1 l0 l2 l0 t1=4 and t2=2 - - 2<x2-x1<4 0<x1-x2<4 2<x1-x2<4 - - L0 Exp(x1,4), !DS(u1,r1) ?AS(u1,r1), Set(x1,4) x1=t1 !DS(u1,r1) x1=t1 !DS(u1,r1) q2 q0 q1 ?AS(u1,r1,t1) x1:=0 ?AC(u1,r1), Set(x2,2) Exp(x2,2), !DC(u1,r1) L1 L2 Exp(x2,2),?AS(u1,r1), Set(x1,4) x2=t2 !DC(u1,r1) q3 q4 Exp(x1,4), !DS(u1,r1) Exp(x1,4), !DS(u1,r1) ?AC(u1,r1,t2) x2:=0 se-FSA q5 Exp(x1,4), Exp (x2,2) !DS(u1,r1) Exp(x2,2) q6 se-FSA Transformation [Khoumsi] • Three types of events • Input events – input actions and/or clock resets • Output events – output actions and/or clock expirations • Complex events – mix of above two 48

49. Test Generation From se-TRBACM • se-TRBACMdeterministic and finite state • W-method can thus be used for test generation • Assumed location observability – tests constructed from test tree (Tr) • Tr constructed so that all terminals correspond to accepting states of se-TRBACM • Tr represents paths in se-TRBACM, • Given a path pt in Tr, A test sequence is constructed by associating all edges ept with monotonically increasing time stamps • Temporal constraints determined by the Set and Exp events along edges of pt 49

50. pt1 How to Construct a Test Sequence? • Corresponding to pathpt1 • The temporal constraints can be represented as • Formulate as an IP to control the minimum resolution dti • For k=0.1 the solution would be • Conformance Test Suite (CTS) constructed by finding feasible time stamps for all test sequences 50