Statistics and your thesis: How to approach the analysis & avoid common pitfalls.

Statistics and your thesis: How to approach the analysis & avoid common pitfalls. Postgraduate Colloquium Jeromy Anglim

Sources of assistance • Quant Lecturers (Paul Dudgeon; Phil Smith; Richard Bell; Garry Robins; Pip Pattison) • Department Stats Consultant - Jeromy Anglim • j.anglim@pgrad.unimelb.edu.au • Room 1110 • University Statistics Consulting Service • http://www.scc.ms.unimelb.edu.au/ • At no charge, up to five hours of statistical advice to any Masters by research, PhD or MD student in 2004;

Statistics Subjects • 4th Year statistics (sit in) • Excellent course that allows you to learn a whole range of multivariate techniques including data importation, cluster analysis, multidimensional scaling, exploratory and confirmatory factor analysis, correspondence analysis, optimal scaling, canonical correlation, ANCOVA MANOVA, and more. • 512-984 Quantitative Methods for Organisational Psychology • good for anyone wanting to do network analysis or multi-level modelling • 512-933 Categorical Data Analysis • as the name suggests, good for analysing categorical data • chi-square, log-linear modelling, time series, markov chains, survival analysis, correspondence analysis • 512-998 Structural Equation Modelling

Additional Assistance • School of Graduate Studies • http://www.sgs.unimelb.edu.au/services/skills/upskills/computer.html • short courses in SPSS • Introduction to SPSS • May even be able to take Monash courses based on reciprocal program • Private tutoring • http://www.pen.psych.unimelb.edu.au/general/tutors.html

Books and websites • Statistics Books • Tabachnick & Fidell - Using Multivariate Statistics (4th Edition) • Hair, Anderson, Tatham, & Black – Multivariate Data Analysis • SPSS books • Coakes & Steade - SPSS without Anguish (5th Edition)

SPSS Help • Right click on an SPSS table and click results coach for an explanation of the table • Right click an SPSS dialog box for more explanation about the particular option • Press the help button • SPSS syntax guide • Websites • http://www2.chass.ncsu.edu/garson/pa765/statnote.htm • Google search: name of statistics technique • Richard Bell’s 4th year subject page: http://webraft.its.unimelb.edu.au/512422/pub/

Latest Position • Statistical Task Force for the APA • Effect size and power • Honesty in reporting • Simplicity is sometimes better

Making Friends with your data • Wright, D.B. (2003) Making friends with your data: Improving how statistics are conducted and reported. British Journal of Educational Psychology, 73(Mar), 123- 136. • Report exact p values • Report effect sizes • Level of measurement is a decision made by researchers • If you are worried about outliers or bad distributions, run the analyses with and without outliers removed or transformations applied. If you get same results, you should feel more confident • Avoid using median splits • Data = model + error • Tell a story with the data

Some golden rules • Stay close to the data • Link analyses back to theories • Try it both ways and if it doesn’t make a different then it probably doesn’t matter

Stage 1 – experimental design • Statistical Power • G-Power • http://www.psycho.uni-duesseldorf.de/aap/projects/gpower/ • or type ‘g power’ into Google • freeware power analysis software • tips: repeated measures increases power; reduce within group variance, increase size of effect, increase sample size

Stage 1 – Experimental Design • Selecting good instruments (reliability, validity) • Proposing analyses for mini-conference • Ground theory in testable ideas • Get input from your panel, if you are uncertain • Common pitfalls • Multiple groups and regression based models • Poorly designed questionnaires (consider the factor structure from the start, have enough rating levels for the scale)

Stage 2 - Data Entry • Create a data definition document • Sets out: columns used in raw text file; SPSS variable name; label or description; missing values and what they mean; any value labels; • Enter at the item level

Stage 2 - Data Entry • Use variable names that are simple to work with • e.g., s1 to s20 for a 20 item questionnaire • stick to a set of conventions) • Have a data validation strategy • Examine frequencies of variables for plausible values • Examine correlations and see whether they correspond to expectations

Stage 3 – Initial Analyses • Missing values analysis • Take advantage of the new SPSS procedure • Scale computation • Remember to reverse negatively worded items (max + min - score) • Factor analyse • Check reliability statistics (item if scale deleted) • Basic descriptives • Means • Frequencies • distributions • correlations • transformations, outliers, etc.

Stage 4 – Main Analyses • Break hypotheses down in terms of specific variables and level of measurement (interval, ordinal, nominal) • Brainstorm list of possible analyses (decision trees in statistics books can be helpful) • Develop analysis plan • What needs to be done to prepare the data for this analysis? • What do I need to learn?

Stage 5 – Results Presentation • Report some measure of effect size • Statistical decisions are about having justifications for your decisions. Cite respected authors to back up statistical decisions. • Tell a story that highlights what is interesting

Error checking • Get to know your data • Check for data entry errors • Check that the variables are behaving as you would expect

Common Pitfalls • Choosing analyses that are more complex than needed • Too much time to master • Not interpreting appropriately • Violating statistical assumptions • Not error checking data

Questions???

Statistics and your thesis: How to approach the analysis & avoid common pitfalls.