Secondary Data Analysis: An Introduction

1. Secondary Data Analysis: An Introduction Dr Juliet Hassard Deputy Director, Centre for Sustainable Working Life Lecturer in Occupational Health Psychology

2. What is secondary data analysis? • Types and sources of data • Opportunities, limitations, and challenges • Ethics • Thinking forward: funding and publishing. Overview of presentation

3. Secondary data analysis: why and what it is?

4. The use of secondary data, or existing data that are freely available to researchers who were not involved in the original study, has a long and rich tradition in the social sciences [1]. • Sociology, economics, etc. • Why collect new data, given the wealth of existing data sets that can be used to answer important questions? • Longitudinal & large sample sizes. • Traditionally, the field of psychology (any many of those within it) have dismissed the importance and value of studies using secondary data. • But times are changing…….

5. To ask and answer important questions. For example, • To understand the longitudinal nature of relationships. • To understand group differences, trends over time? • To explore new and emerging social phenomena. Why use secondary data?

6. More data (and types of data) are being collected (and available!) then ever before. • There is a unique opportunity to explore this ever growing source(s) of data, and to ask important research questions. Why secondary data analysis?

7. Let’s get creative…….. • In small groups of 3-5. Discuss and outlines 4-5 different types of data/ types of information that could be used to investigate an important psychological research question. Types and sources of data….

8. Chat forums Blogs Published business reports Online support groups Second life App technology

9. The UK Data Service • https://www.ukdataservice.ac.uk/ • Census data • International macrodata • Longitudinal studies • Qualitative/mixed methods • UK surveys • The National Data Service • http://www.nationaldataservice.org/about/ • Individual studies may have different access points. • E.g., Whitehall II Study, UCL. • Secondary data is everywhere – some in the public forum. Where do I find data?

10. Online support groups: • COULSON, N.S., 2015. Exploring patient's engagement with web-based peer support for Inflammatory Bowel Disease: forums or Facebook? Health Psychology Update. 42(2), 3-9. • Longitudinal data (Whitehall II survey) • Kouvonen, A., et al . (2011). Negative aspects of close relationships as a predictor of increased body mass index and waist circumference: the Whitehall II study. American journal of public health, 101(8), 1474-1480. • Twitter, Instragram….. • Whiting, R., & Pritchard, K. (2015). “Big Data? Qualitative Approaches to Digital Research", Qualitative Research in Organizations and Management: An International Journal, Vol. 10 Iss: 3, pp.296 - 298 Examples

11. Advantages, Limitations, & Challenges

12. Low response rate High attrition rates Access to high quality measures Small sample size Limited money & resources to collect primary data ‘Traditional’ Challenges in Psychological Research Reliance on convenience samples Limited scope for extensive comparative research (across groups or internationally) Correlation does not equal causation

13. The data has already been collected. • Save time – primary researcher does not have to design study and collect new set of data. • The types of data that are typically collected tend to be higher quality than could be obtained by individual researchers. • Typically longitudinal, have large sample sizes that have been obtained using elaborate sample plans. Advantages Ref: Trzesniewski et al., 2011

14. Learning how to work with, manage and analyse secondary data can provide individual researchers with the raw materials to make important contributions to the scientific literature • … using data sets with impressive levels of external validity. Advantages Ref: Trzesniewski et al., 2011

15. Open-source approach to research • Replicate findings using similar analyses • Encourages careful reporting and justification of analytical decisions. • Allows researchers to test alternative explanations and competing models. • Encourages transparency, which in turns help facilitates good science. Advantages Ref: Trzesniewski et al., 2011

16. The data has already been collected!!! • You may not have all the information on how or why certain types of information was collected. • You may not know of any particular problems that occurred during data collection. • Sometimes you are left wanting more ….. Disadvantages

17. The temptation: a statistical fishing trip. • Great research is driven by a good research question that is strongly underpinned and shaped by theory. • The purpose of analysing data is to refine the scientific understanding of the world and to develop theories by testing empirical hypotheses. • “Mo Money Mo Problems” - Mo Data, Mo Temptations ? • A note about statistical power. Disadvantages Ref: Trzesniewski et al., 2011

18. Considerable time and effort: • is invested by the researcher to understand the nature and structure of a data set. • is needed by the researcher to explain and justify the theoretical and analytical approached used. • Although, I would argue there is real advantages to the time invested in doing this. Disadvantage Ref: Trzesniewski et al., 2011

19. Measures in these datasets are often abbreviated. Often because the projects themselves were designed to serve multiple purposes and to support a multidisciplinary team. • Shortened measures, mix-levels of data, and single items measures. • These datasets often have impressive levels of breadth (many constructs are measured), but often with an associated cost in terms of depth of measurement. • Therefore, measurement issues are ~ therefore ~ one of the major issues in secondary data analysis • These issues often require quite a bit of conceptual consideration & defending in the peer-review process. Disadvantage Ref: Trzesniewski et al., 2011

20. A good grounding in psychometrics and Classic Test Theory. • You need to carefully consider and evaluate the trade-offs in reliability and validity. • You need to defend your position when writing up. • You need to understand how measurement issues frame your findings; and, in turn, your interpretation of your findings. Ref: Trzesniewski et al., 2011 Challenges

21. Creating and managing data files • Data inventory • Research journal • Approach to missing data and data screening procedures • Use of and/or development of constructs • Use of proxy variables • Development & testing of composite measures • Single item measures • Accounting for the data structure in your analysis Practical & Methodological Challenges

22. Case study: An Example

23. The aim of the doctoral thesis was to develop and test a theoretical model seeking to describe the aetiological role of psychosocial processes, in and out of the workplace, in predicting gender-related diversity issues in men’s and women’s health at a structural/population level. • An iterative multi-stage methodology was utilised to develop and test the proposed theoretical model. Modelling Gender-related diversity in psychosocial processes and work-related wellbeing: Pathways and Mechanisms

24. Case Study: Methodology

25. European Working Conditions Survey • Pan-European cross sectional survey of working conditions, worker’s health and safety, and living conditions (n = over 40, 000 workers) • Now on the 6th wave of data collection. • The survey as evolved over time asking more questions. • Survey items are informed and based on contemporary theory • The measures used are not always based on a validated psychometric measures • Single items vs. composite measures? Case study

26. The vast majority of latent conceptual constructs are complex and multifaceted in nature. • Consequently, the use of a single item as a theoretical concept may not yield an accurate, comprehensive, and reliable measurement of the given construct of interest. Case study: Single item measures

27. The guiding premise by many in the scientific community is that multiple responses reflect the “true” response more accurately than does a single response. • Imprecision in measurement is one of the key causes (although not the sole cause) of measurement error. • Measurement error creates ‘noise’ to the observed variables. Case study: Measurement error

28. Inaccurate and unreliable measurement of a concept results in key concerns regarding the overall validity and reliability of the hypotheses tested using this (or these) given measurement(s). • It is generally agreed/ suggested that research findings that are valid, reliable and generalizable, are built on a solid foundation of accurate and consistent measurement. Case study: Implications of poor measurement

29. The primary objective of creating a series of summated (or composite) scales is to avoid the exclusive use of, or dependence on, single item constructs where possible. • The use of several variables as indicators provides an opportunity to represent differing facets of a given concept, with the aim of yielding a more well-rounded perspective and, arguably, a better measurement of the given concept Composite measures

30. Thinking about ethics

31. Researchers need to ask: how was consent obtained in the original study? Where sensitive data is involved, we cannot/ should not assume informed consent. • Given that it is usually not feasible to seek additional consent, a professional judgement may have to be made about whether the use of secondary data violates the contract made between subjects and the primary researchers. • Growing interest in secondary data make it imperative that researchers in general now consider obtaining consent, which covers the possibility of secondary analysis as well as the research in hand. • This is consistent with professional guidelines on ethical practice A note about ethics Heaton, J (1998). Secondary analysis of qualitative data. Social Research Update (issue 22). See: http://sru.soc.surrey.ac.uk/SRU22.html

32. Can you publish secondary data analysis – yes! • Never forget: the central role of theory. • Be detail orientated! • Justifying your research question is important, but you also need to be prepared to justify and outline the logic of your analysis framework and approach. • Understand and reflect on how the research design or any experienced methodological issues of your secondary data may impact or frame the interpretation of your results. Some thoughts on writing up

33. Secondary data analysis is an important and useful research methodology. • There are many benefits and strengths to using secondary sources of data. • But there also important pragmatic and methodological challenges that face researchers. Conclusion

34. Trzesniewski, K. H., Donnellan, M., & Lucas, R. E. (2011). Secondary data analysis: An introduction for psychologists. American Psychological Association. • Vartanian, T. P. (2010). Secondary data analysis. Oxford University Press. • Heaton, J. (2008). Secondary analysis of qualitative data: An overview. Historical Social Research/HistorischeSozialforschung, 33-45. • Hinds, P. S., Vogel, R. J., & Clarke-Steffen, L. (1997). The possibilities and pitfalls of doing a secondary analysis of a qualitative data set. Qualitative Health Research, 7(3), 408-424. Suggested reading

35. Thank you for listening j.hassard@bbk.ac.uk