How Can e-Social Science Promote the Re-Use of Data?

1. How Can e-Social Science Promote the Re-Use of Data? Rob Procter National Centre for e-Social Science rob.procter@ncess.ac.uk www.ncess.ac.uk UPTAP Workshop 2007

2. The e-Science Vision • “e-Science is about global collaboration in key areas of science and the next generation of infrastructure that will enable it.” (John Taylor, former DG, Research Councils) • That infrastructure is the Grid: “ … a software infrastructure that enables flexible, secure, coordinated resource sharing among dynamic collections of individuals, institutions and resources” (Foster, Kesselman and Tuecke) • The Grid is not just an enabler of visionary research, however, but can help researchers in more mundane ways. • But, to be successful, the development of the Grid must be driven by researchers’ needs. • I want to use the opportunity provided by this workshop to gather ideas from you about what those needs are with a specific focus on the (re-)use of data. UPTAP Workshop 2007

3. NCeSS Overview • Launched in May 2004 to develop and promote UK e-Social Science. • Unified Centre with distributed structure: • Co-ordinating Hub: Manchester & UKDA • Seven research Nodes located across UK • Twelve small projects UPTAP Workshop 2007

4. NCeSS Overview • Applications of e-Social Science: • Harnessing new kinds of research infrastructure and tools to tackle substantive problems and promote innovation in research methods • Social shaping: • Usability of new infrastructure and tools • Socio-technical factors in their design, uptake and use • Research and policy drivers, impacts UPTAP Workshop 2007

5. CeSDeMIDE PolicyGrid MoSeS Intelligent Simulation Data MiMeG GeODE GeSRM CQeSS Analysis HeadTalk Data chronicles Disclosure Risk Assessment OeSS AGN enabled interviews GeoVUE Grid-enabled data collection DReSS Learning Disabilities Replayer Entangled Data NCeSS 2006 SocialShaping Tools Infrastructure and services Hub Researchmethods UPTAP Workshop 2007

6. Data archive Data archive HPC Analysis HPC Experiment Computing Analysis HPC Today’s Research Infrastructure • Heterogeneous resources with poor inter-operability and complex administrative arrangements. Study Researcher • Doesn’t scale well and makes re-use and sharing of data and other research resources difficult. UPTAP Workshop 2007

7. Data archive Data archive Study Storage Storage HPC HPC Social scientist Social scientist Social scientist Experiment Computing Analysis Analysis Grid-Enabled Research Infrastructure Grid middleware manages the interactions between users, and heterogeneous and distributed resources, providing seamless integration of data, analytic tools and compute resources. Grid Middle- ware UPTAP Workshop 2007

8. The Grid Dissected • Tools to support collaboration between distributed researchers. • Computational Grids for scalable, high-performance computation. • Data Grids for accessing and integrating heterogeneous datasets. • Sensor Grids for collecting real-time data. UPTAP Workshop 2007

9. Research and Policy Drivers UPTAP Workshop 2007

10. Research and Policy Drivers • The range of research resources on offer to the social science community has never been greater. • These include not only traditional research datasets, but new kinds of social data. • However, the often highly distributed and heterogeneous character of these datasets makes it difficult to exploit them to their full potential. UPTAP Workshop 2007

11. Research and Policy Drivers • The data deluge in social sciences: • WWW archive currently contains 55 billion Web pages or 2 petabytes (2x1015) of data and is growing at the rate of 20 terabytes (20x1012) per month • Administrative and transactional data is generated on increasing scale as by product of our everyday activities: • This data is complex and multi-dimensional UPTAP Workshop 2007

12. Data Grids for Social Science • Data Grids are designed to provide unimpeded and integrated use of distributed, heterogeneous, autonomous data resources. • Grid enabling a dataset creates new opportunities for (re-)use: • enables users to integrate it with other datasets • makes it possible to analyse the dataset using techniques that require the kind of computational power that is only feasible using the Grid (e.g., more complex models, more data points) • standardisation of procedures and mechanisms used to access and update the dataset increase its shareability • Automated analyses (i.e., analyses can be re-run automatically when databases are updated) UPTAP Workshop 2007

13. An Example Data Linkage Problem • Many research questions require combination of data from multiple geo-referenced datasets: • E.g., Linking post coded data to census geography • Conversion of data relating to different geographies to a common target geography is • A complex time consuming task • Requires a range of data handling/processing skills • A major barrier to use! • The data conversion process requires users to perform the following generic tasks: • Extract and download data in different formats from a number of databases using different interfaces • Convert each dataset to the desired target geography using geographical conversion tables • Combine the converted sets into a single dataset for analysis • These generic tasks can be automated. UPTAP Workshop 2007

14. A Solution: ConvertGrid • ConvertGrid provides access to 225 UK-wide geography conversion tables between census, electoral, administrative, postal, health and statistical geographies derived from the AFPD. • Facility to convert a researcher’s data from one set of geographical units to another (e.g., from postcode geography to heath geography). • Extensible system - further conversion tables from any source can be incorporated. UPTAP Workshop 2007

15. ConvertGrid – Data Visualisation Interface High average house price sales but low participation rates Low average house price sales but high participation rates Ten minutes from start to finish • Relationship between average house price sales (Experian) and percentage of 16-19 year olds entering university (Neighbourhood Statistics & Census aggregate statistics). • Contact Keith Cole (keith.cole@manchester.ac.uk) for more information. UPTAP Workshop 2007

16. Review literature and generate hypothesis Find datasets related to proposed area of work Explore datasets and determine suitability Publish papers Build models and execute them Write papers Analyse results and compare with hypothesis Share results and conclusions and discuss with collaborators Supporting the Research Lifecycle UPTAP Workshop 2007

17. Increasing (Re-)Use of Social Data • Removing barriers to more effective use of existing social data collections: • Data providers (e.g., ONS, data archives) • Data users • Many researchers are both generators and users of data: • Preparation of data for submission to data archives is not well rewarded so re-use suffers • Removing barriers to use of new kinds of social data: • Privacy and confidentiality of personal data UPTAP Workshop 2007

18. The Data Provider Perspective • Preparation procedures: • Cleaning the data • Generating derived variables • Re-weighting • Adding metadata • Writing user documentation • Maintenance: • Managing changes in sampling frames, definitions, variables and questionnaire over time • Re-weighting • User support: • Handling queries from users about concepts, meaning and linking waves UPTAP Workshop 2007

19. The Data User Perspective • Discovering appropriate data: • Determining what can be done with the data and how. • Accessing the data: • Are existing provisions, such as VMDLs, for access to confidential data adequate? • Understanding how the data has been used to generate answers to other research questions: • Provenance of results, links to publications • Re-running statistical models, comparing results • Ease and of use and quality of documentation: • User manuals UPTAP Workshop 2007

20. The Data User Perspective • Data preparation: • Selecting variables • Linking waves • Linking data sets • Performing and possibly repeating analysis with different data. • Interpreting and visualising results. • Supporting the research lifecycle. • Collaboration with other users and with data providers. UPTAP Workshop 2007

21. Contacting NCeSS and Getting Involved • rob.procter@ncess.ac.uk • www.ncess.ac.uk • Join our email list: • Participate in events: • Agenda setting workshop on combining and sharing data, January 22nd-23rd, Manchester • Annual conference UPTAP Workshop 2007