1 / 18

Enhancing Statistical Disclosure Control for Continuous Data with Edit Restrictions

Explore techniques like additive noise, microaggregation, rounding, and rank swapping to protect continuous microdata with edit constraints, ensuring data utility and confidentiality.

primo
Télécharger la présentation

Enhancing Statistical Disclosure Control for Continuous Data with Edit Restrictions

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SDC for continuous variables under edit restrictions Natalie Shlomo & Ton de Waal UN/ECE Work Session on Statistical Data Editing, Bonn, September 2006

  2. Contents • The problem • Evaluation data • SDC techniques • Additive noise • Microaggregation • Rounding • Rank swapping • Conclusions

  3. The problem • Statistical disclosure control (SDC): microdata need to be protected against disclosure before release • Several SDC-techniques available for continuous microdata • Do not take edit constraints into account • Inconsistent microdata lead to loss of utility and pinpoint potential intruders to protected data • Problem: extend SDC techniques for continuous microdata to take edit constraints into account • Micro edits – record level inconsistencies • Macro edits – overall loss of utility (bias and variance)

  4. Evaluation data • 2000 Israel Income Survey with three continuous variables (gross, net and tax) and one control variable (age) • 32,896 individuals of which 16,232 earned income from salaries • Edits: • E1a: gross≥ 0 • E1b: net ≥ 0 • E1c: tax ≥ 0 • E2: IF age ≤ 17 THEN gross ≤ 6,910 • E3: net + tax = gross

  5. Additive noise • Generate random value and add to value to be protected • Random value can be drawn in several ways, depending on • Aiming to preserve variances or not • A single variable or multiple variables

  6. Additive noise for a single variable using standard approach • Adding standard noise: perturb Y as follows • Y* = Y + e, e drawn from N(0, σ2) • Adding random noise to gross with σ2 = 0,2xVar(gross) resulted in 1,685 failures of E1 and 119 failures of E2 • Adding standard noise in groups • Define 5 equal groupings (quintiles) by sorting • Within each group applying above method resulted in 66 failures of E1 and no failures of E2

  7. Additive noise for a single variable using correlated noise Perturb value Y as follows (Natalie’s trick): • Y* = d1Y + d2e, • d1 = (1- δ2)1/2, d2 = δ for positive parameter δ • e drawn from N((1-d1)/d2 x mean(Y), Var(Y)) • Note that • E(Y*)= E(d1Y) + E(d2e) = E(Y) • Var(Y*) = (1- δ2)Var(Y) + (δ2)Var(Y) = Var(Y) • Linear equations are preserved

  8. Additive noise for multiple variables and linear programming • Perturb each variable Yi separately, resulting in Yi* • Adjust perturbed values Yi* slightly so that all edits become satisfied (LP-trick) • Minimize Σi |Yi* - Yi,final| subject to edit constraints • Yi,final are final perturbed values • Problem is simple linear programming problem

  9. Additive noise for multiple variables using correlated noise Perturb vector Y by applying Natalie’s trick • Y* = d1Y + d2e, • d1 = (1- δ2)1/2, d2 = δ for positive parameter δ • e drawn from N((1-d1)/d2 x mean(Y), Var(Y)) • mean(Y) mean vector of Y; Var(Y) covariance matrix of Y • Means, covariances and equations are again preserved • E(Y*) = E(Y) • E(Var(Y*)) = E(Var(Y)) • Linear equations are preserved

  10. Microaggregation • Replace value to be protected by average value in small group • Reduction in variance due to elimination of “within” variance • Microaggregation can be applied in several ways: • Standard version of microaggregation • Microaggregation followed by adding noise (to preserve original variance) and using linear programming to ensure preservation of linear equations (LP-trick) • Microaggregation followed by adding correlated noise to ensure preservation of linear equations (Natalie’s trick) • Avoids need for LP- trick but does not raise variance to expected level

  11. Results for microaggregation

  12. Rounding • Round value to be protected to multiple of rounding base • Rounding can be applied in several ways: • Random rounding • Controlling totals and additivity • Controlling totals and additivity, and selecting all rounded values within base of original value

  13. Random rounding • Univariate rounding with rounding base b • res(X) = X – largest multiple of b less than X • Round X up with probability res(X)/b and down with probability 1 - res(X)/b • Expectation of rounding is zero • In expectation totals are preserved

  14. Random rounding: controlling totals • Select fraction of res(X)/b random entries to be rounded upward and round the rest downward • total is exactly preserved • gross is calculated as sum of rounded tax and net • gross may jump a base • apply reshuffling algorithm to correct this

  15. Results for rounding

  16. Rank swapping • Sort variable to be protected and construct groupings, select random pairs in each group and swap values between pairs • Different group sizes lead to different results • Evaluation criteria: • AD = Σi |Xi,orig – Xi,pert|/nr • where i is cell in age group (14) x sex (2) x income group (22) • BV = Σj nj (averagej(X) – average(X))2/(p-1) • with j=1,..,p in age group (14) x sex (2)

  17. Results for rank swapping

  18. Conclusion • Standard perturbation methods can be extended so they take (micro and macro) edit constraints into account • “Best” method to protect data set is to some extent subjective choice • Must provide protection against disclosure risk according to tolerable risk threshold • Must provide fit for purpose data according to needs of users

More Related