Aggregate Data and Statistics

1. Aggregate Data and Statistics Wendy Watkins Carleton University Chuck Humphrey University of Alberta Statistics Canada Data Liberation Initiative

2. Outline • What are aggregate data? • Why aggregate? • How to aggregate? • Computing exercise

3. What are aggregate data? Let’s start with the relationship between statistics and data.

4. Statistics and Data Data • numeric files created and organized for analysis • requires processing • not ready for display Statistics • numeric facts/figures • created from data, i.e, already processed • presentation-ready

5. Statistics and Data

7. Statistics and Data

8. Statistics and Data In short, statistics are created from data and represent summaries of the detail observed in the data.

9. What is aggregation? Building on this example, let’s explore aggregation. We see a table with the number of smokers summarized over categories for age, education, sex, geography, and different time points.

10. Categories of Periods Statistics Categories of Sex Categories of Region Age and Education are in the background and display totals.

11. What is aggregation? Aggregation involves tabulating a summary statistic across all of the categories or levels of a set of variables.

12. The summary statistic The summary statistic in this example is the total number of smokers.

13. Variables and categories The variables and their categories are: Region (11): Canada and the ten provinces Age (5) : Total, 15-19, 20-44, 45-64, 65+ Sex (3) : Total, Female, Male Education (4) : Total, Some secondary or less, Secondary graduate or more, Not stated Periods (5) : 1985, 1989, 1991, 1994-95, 1996-97

14. Variables and categories The tabulation consists of determining the combinations of all categories across variables and then counting the number of smokers within each of these combinations. 11 x 5 x 3 x 4 x 5 = 3300 category combinations

15. Tabulating or aggregating One might be wondering if there is a difference between tabulating and aggregating. Usually, they are the same thing.

16. Tabulating = aggregating In creating tables from data, the variables are arranged in various combinations along the columns and the rows.

17. Tabulating = aggregating Placing multiple variables along the columns or rows is called nesting. Tables may have variables nested on both the columns and rows.

18. Categories of Sex nested within Periods

19. Categories of Education nested within Sex Categories of Sex nested within Region

20. A quick summary Up to this point, we have noted that • statistics are created from data • aggregations consist of tabulating statistics within the categories of selected variables • variables may be nested within columns and rows to display these tabulations

21. What are aggregate data? Q: What is the difference between an aggregation or tabulation and aggregate data? A: The display of the aggregation (that is, the structure of the tabulated output).

22. Statistical data structure A statistical data structure is a fixed, two-dimensional matrix with the variables in the columns and cases in the rows. V1 V2 V3 V4 V5 V6 V7 Case 1 Case 2 Case 3 Case 4 Case 5 Case 6 Case 7

23. Statistical data structure Aggregate data require the same type of statistical data structure. With aggregate data, the variables are the cells of a tabulation while the cases are the categories of one or more of the table’s dimensions.

24. What are aggregate data? Here is an example of a tabulation of three variables. One variable consists of five levels and will be used to represent the cases in an aggregate data file. The other two variables make up a two-way table and the cells in this table will be the variables in the aggregate data file.

25. R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 From tabulation to data Dimension 1 5 4 Dimension 3 3 2 1 Dimension 2

26. From tabulation to data Start with dimension 3 and convert this dimension into the rows or cases of an aggregate data file.

27. R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 From tabulation to data Dimension 1 5 4 Dimension 3 3 2 1 Dimension 2

28. R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 From tabulation to data Dimension 1 Dimension 2

29. 1 2 3 4 5 From tabulation to data Dimension 3

30. From tabulation to data Working with the six cells from the tabulation for level 1 of dimension 3, locate these six cells as six variables in the new data structure.

31. R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 From tabulation to data R1C1 Dimension 1 Dimension 2

32. 1 2 3 4 5 From tabulation to data R1C1 Dimension 3

33. R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R2 C1 R3 C1 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 From tabulation to data R2C1 Dimension 1 Dimension 2

34. 1 2 3 4 5 From tabulation to data R1C1 R2C1 Dimension 3

35. R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R3 C1 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 From tabulation to data Dimension 1 R3C1 Dimension 2

36. 1 2 3 4 5 From tabulation to data R1C1 R2C1 R3C1 Dimension 3

37. R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 From tabulation to data R1C2 Dimension 1 Dimension 2

38. 1 2 3 4 5 From tabulation to data R1C2 R1C1 R2C1 R3C1 Dimension 3

39. R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 R2C2 R3 C2 From tabulation to data R1C2 Dimension 1 Dimension 2

40. 1 2 3 4 5 From tabulation to data R1C2 R2C2 R1C1 R2C1 R3C1 Dimension 3

41. R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 R3 C2 From tabulation to data Dimension 1 R3C2 Dimension 2

42. 1 2 3 4 5 From tabulation to data R1C2 R2C2 R3C2 R1C1 R2C1 R3C1 Dimension 3

43. R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 From tabulation to data Dimension 1 Dimension 2

44. From tabulation to data Repeat this for level 2 of dimension 3.

45. R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 From tabulation to data Dimension 1 Dimension 2

46. 1 2 3 4 5 R1C2 R1C2 R2C2 R2C2 R3C2 R3C2 R1C1 R1C1 R2C1 R2C1 R3C1 R3C1 From tabulation to data Dimension 3

47. R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 From tabulation to data Dimension 1 Dimension 2

48. From tabulation to data Now for level 3 of dimension 3, etc.

49. R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C1 R2 C1 R3 C1 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 R1C2 R2 C2 R3 C2 From tabulation to data Dimension 1 Dimension 2

50. 1 2 3 4 5 R1C2 R1C2 R1C2 R2C2 R2C2 R2C2 R3C2 R3C2 R3C2 R1C1 R1C1 R1C1 R2C1 R2C1 R2C1 R3C1 R3C1 R3C1 From tabulation to data Dimension 3