Nordisk Statistikermøde i København 11.-14. august 2010

1. Nordisk Statistikermøde i København 11.-14. august 2010 The archivestatistical method 1960-2010 50 years - A Summaryby Svein Nordbotten Svein Nordbotten

2. The situation 50 years ago • Separate statistical organizational units for limited subject matter domains • Few possibilities to link data within and between established domains • Reuse of micro data expensive and time consuming • Creating longitudinal micro data sets rarely possible • Published statistics followed a narrow and restricted tabulation plan • Analysis limited to aggregated macro data • Statisticians were experts in their subject matter domains • Little exchange of experience and information among different groups Svein Nordbotten

3. Two important events in the 1950s • Electronic, programmed computers made processing of large data sets • more effective for administrative and statistical purposes. • Electronic recording made establishing, maintaining and sharing • comprehensive population/object registers possible. Svein Nordbotten

4. The information technology The UNIVAC 1 computer entered the official statistical scene in the US Census Bureau in the early 1950’s, and when the processing of the 1960 censuses were due, a number of NSIs were prepared with their own computers. When the information technology later merged with communication technology, it changed the time and space dimensions for statistical processing. The new technical potensial inspired to think about all activities of an NSI as interrelated parts of a production system in contrast to the traditional separate statistical processes, and the term statistical production became common. Svein Nordbotten

5. Administrative registers Local object identifiers had been used for a long time in many admistrative processes. These specialized systems were expensive to maintain , represented a vast duplication of efforts and were of limited usability as statistical data sources. The efficiency of shared, public object identifiers were obvious. About 1950 the first steps to establish general, permanent, unique object identifier registers for the economic units as well as for the population were taken in the Nordic countries with active participation by statisticians. Svein Nordbotten

6. Statistical ideas in the 1950s Unique and permanent identifiers permitted compilation of the ’life lines’ of statistical objects including interobject events by data linkage. Models for statistical micro systems named statistical archive systems, statistical file systems, register-based statistical systems, etc. were discussed in international fora from about 1960. The next slides illustrate the popular version of ideas discussed. Ten years later the generic version became known as data base management systems. Svein Nordbotten

7. The 3 basic concepts Objects Time Location 11/2/2014 Svein Nordbotten Svein Nordbotten 7

8. Archive data container Attributes Census attributes Human population Objects Census day Time Svein Nordbotten

9. The 3-dimensional space model Attributes Time plane Object plane Attribute plane Objects Time Svein Nordbotten

10. Latitudinal and longitudinal integration OOOOO Object: C Time: t Link H Link H Object : A Time: t Object: B Time: t Link V Link V Link V Object type A type A Time t Time t Object type A Time t Object:B Time: t-1 Object: C Time: t-1 Link H Link H Object: A Time: t-1 Link V Link V Link V Object A: Time: t-2 Object: C Time: t-2 Link H Link H Object: B Time: t-2 Svein Nordbotten

11. An example Svein Nordbotten

12. Summary of the 50 year old visions Comprehensive official register systems General utilization of systems Fast and high capacity IT Extensive and userfriendly communication Software for data processing, saving, retrieval and communication Data collection independent of specific usage Use of communication network for collection Functional organization Wider statistical coverage by more intensive reuse of data Statistics on-demand Analytical interest in and tools for exploiting the new possibilities Legislation regulating the use of micro data Svein Nordbotten

13. Utilization of public identification systems • Population register systems in general use • Economic unit register systems in general use by administrative organizations • Register systems for geographical location of properties established and in use Svein Nordbotten

14. Sample of present applications Censuses (population, location registers) Demographics (demographic registration) Foreign trade (custom data) Income (taxation data) Social statistics (registration of services) Employment (unemployment registration) Education (registration of students) Health (medical registrations) Crime ( Police, court, prison registrations) Business (Business registers) Svein Nordbotten

15. Methodological challenges The present intensive utilization of administrative data in official statistics has also required work on methods in several fields: Electronic processes for collecting administrative data Quality control of data received from external administrative sources Integrating data from different statistical and administrative sources Evaluating quality of integrated data sets Editing data using associated administrative background data Estimating population parameters using administrative background data Confidentiality and security in systems based on administrative data Statistics on demand from systems based on administrative data Evaluation of costs and benefits from using administrative data Svein Nordbotten

16. Privacy issues • Statistical legislation • Privacy legislation • Misuse of information • Value of knowledge • Regulatory use and research • Solutions? A political question Svein Nordbotten

17. Information privacy law ADM 1 AD1 Statistical law NSI SP Data products Data collection SD ADM 2 AD2 Svein Nordbotten

18. Anticipations/realizations Comprehensive official register systems……………………………………………... . Yes General utilization of systems………………………………………………………………. Yes Fast and high capacity IT………………………………………………………………………. Yes Extensive and userfriendly communication…………………………………………… Yes Software for data processing, saving, retrieval and communication……… Yes Data collection independent of specific usage…………………………………….. ? Use of communication network for collection......................................... Yes Functional organization……………………………………………………………………….. Yes? Wider statistical coverage by more intensive reuse of data…………………. Yes Statistics on-demand…………………………………………………………………………… ? Analytical interest in and tools for exploiting the new possibilities……… Yes? Legislation regulating the use of micro data………………………………………… Yes? Svein Nordbotten

19. What next? Electronically observed data Mobile use of statistics Require legislative, technical and methodoligical preparations. Svein Nordbotten

20. Thank you all for your attention! I look forward to seeing you all in another 50 years. Svein Nordbotten