Aggregate Reporting and Data Disclosure Avoidance Techniques

1. Aggregate Reporting and Data Disclosure Avoidance Techniques • Monday, October 29,2012 • Kim Nesmith, Louisiana Department of Education • Adrian Peoples, Delaware Department of Education • Baron Rodriguez, Privacy Technical Assistance Center

2. Overview Louisiana Process and Types of Suppression Delaware Public Reporting Rules and Strategy Contact Information and Resources

3. Louisiana

4. First Steps • Determining a “n” size • Determining a percentage threshold • Limiting student Level Reports and establishing MOUs

5. Next Steps • Determining what is most important • Determining how to handle complementary suppression

6. Complementary Suppression • If you can “back into a number”, the suppression is not effective • When only one number in a row or column is suppressed and the total is present • When all suppressed numbers are 0s and the total is present • If numerator, denominator, and percentage are all present

7. N Size Example

8. N Size Example

9. Percent Example

10. Percent Example

11. Maintaining Transparency • Talking with the requestor • Creative solutions

12. Delaware

13. Delaware Public Reporting Rules Rule of X • Delaware masks all data for a particular demographic if its group size is less than or equal to X • 15 for Assessment, Enrollment, Teacher Quality • 40 for Accountability 5/95 Rule • If demographic performance is calculated to be either at or below 5% OR at or above 95%, Delaware masks the data.

14. Strategy: Level of Implementation Application Database large images Data

15. Maintenance Gradient Application Database Data Level of Maintenance

16. Strategy: Data vs. Effect Suppression Data Suppression • DO NOT SHOW data to any constituent group (e.g. public, administrators, teachers, etc.) • DO NOT ALLOW aggregate data to be used as input to any data-driven decision-making Effect Suppression • SHOW data to appropriate constituent group (e.g. public, administrators, teachers, etc.) • DO NOT ALLOW aggregate data to be used as input to any data-driven decision-making

17. Implementation Example: Database Level/Data Suppression

18. Implementation Example:Database Level/Data Suppression

19. Implementation Example: Application Level/Effect Suppression

20. Biggest Pitfall: Inconsistent Implementation Policy Accountability Assessment Enrollment Teacher Quality • One point of contact • Both policy and data • Small constant team • Long history • Multiple transient contractors • Newcontractor • Bringing new reports to the public

21. Resources/Sessions PTAC State-by-State analysis of public reports: 2PM today in the Burnham room. Please send a representative from your state to receive your sealed copy! Case Study 5: Minimizing Access to PII… Data De-identification: A Glossary of Terms

22. PTAC Guidance FAQ’s Frequently Asked Questions: If I am only publishing aggregate data tables, do I still need to be concerned about disclosure avoidance? What issues should educational agencies and institutions consider to successfully balance privacy protection requirements with data disclosure requirements? Is public reporting of data for small groups (“small cells”) the same thing as a disclosure? What standard is used to evaluate disclosure risk? Does the U.S. Department of Education require educational agencies and institutions to use specific data disclosure avoidance techniques? And many more…

23. Contacts & Additional Resources Contact information: Adrian Peoples, apeoples@doe.k12.de.us Kim Nesmith, kim.nesmith@la.gov Baron Rodriguez, Baron.Rodriguez@aemcorp.com For more information on Aggregate Reporting: Resource 1: Presentation: Protection of Personally Identifiable Information through Disclosure Avoidance Techniques Resource 2: PTAC Privacy Toolkit – Case Studies, etc. Resource 3:Tech Brief #3: Statistical Methods for Protecting Personally Identifiable Information in Aggregate Reporting (DRAFT; Dec 2010)