data collection Training Session Yana Litovsky Jeff seaman

1. data collection Training SessionYana LitovskyJeff seaman Kuala Lumpur, GEM Annual Meeting January 16, 2013

2. Data Team Introductions Jeff Seaman GEM Data Consultant Yana Litovsky GEM Data Team Supervisor Niels Bosma GEM Research Fellow Alicia Coduras GEM NES Coordinator Chris Aylett GEM Project Administrator

3. Yana Litovsky GEM Data Team Supervisor Based in the US, Yana Litovsky joined GEM in 2009. Her role involves commutating with National Teams about data related issues as well as collecting and processing the submitted APS data. Before GEM, she spent 4 years at the Harvard Business School as a Research Associate working in the field of Industrial Psychology. She holds an Undergraduate degree from Brandeis University in Psychology and Philosophy.

4. Jeff Seaman GEM Data Consultant Based in the US, Jeff joined GEM in 2007 to work on data quality issues. Jeff holds an undergraduate and multiple graduate degrees from Cornell University and has worked extensively in the areas of technology, survey design, and sampling. He currently serves as the Co-Director of the Babson Survey Research Group. He has served as the Director of the Computing Resource Center and as Associate Vice Provost for Computing for the University of Pennsylvania and as Chief Information Officer for Lesley University.

5. Niels Bosma GEM Research Fellow Based in the Netherlands, Niels Bosma has been a member of the GEM Netherlands Team since 2001 and has been engaged in the GEM Data Team since 2004. Niels Bosma holds an MSc degree in econometrics as well as a PhD on entrepreneurship dynamics and regional economic development at Utrecht University, Department of Economic Geography. He has co-authored the GEM Global Report since 2006, published a number of articles in academic journals and contributed to several books and research reports on entrepreneurship.

6. Alicia Coduras Martinez GEM NES Coordinator Based in Spain, Alicia Codurashas been with GEM since 2000. She is the Technical Director of GEM Spainand manages the NES as a member of the GEM Data Team. Alicia is an Associate Research Professor in the Department of Entrepreneurship and Family Business. She holds a doctorate in political sciences from the University Pompeu Fabra and a degree in economics and business sciences from the University of Barcelona. She publishes regularly in academic reviews and develops entrepreneurship reports for Spanish governmental institutions.

7. Chris Aylett GEM Project Administrator Based in the UK, Chris Aylett has been with GEM for over 5 years. He manages the paperwork from the National Teams, issues invoices, liaises with the accounts department and maintains and develops the GEM website. He is also the secretary to the Board, organizing their meetings and preparing agendas, papers and minutes. Prior to working for GEM, he studied French at University College London, and is a qualified translator.

8. Key agenda items Brief GEM Overview APS Questionnaire Sample Design Weights Proposal Submission Process APS Data Collection Process APS Data Review Process APS Processed Results

9. Brief GEM Overview

10. What makes GEM unique? • not-for-profit academic research consortiumwhich estimates the participation in business start-ups and new firms in different countries • grew from 10 countries in 1998 to almost 70 economies and 200,000 respondents in 2012 • comprised of Adult Population Survey (APS) and National Expert Survey (NES) • the Early Stage Entrepreneurial Activity prevalence rate (TEA index) is one of GEM’s best known measures of entrepreneurial activity • because all national teams use the same procedure and provide data to the same GEM standards, the results provided by different countries can be compared worldwide

11. What are GEM objectives? • to measure differences in the level of entrepreneurial activity between countries • to uncover factors determining the levels of entrepreneurial activity • to identify policies that may enhance entrepreneurship • to set international standards for social survey based research methodologies in entrepreneurship • to cover a greater proportion of OECD and non-OECD nations in the interests of gaining a detailed picture of the world's entrepreneurs and their role in economic development

12. Some APS Facts • the Adult Population Survey (APS) investigates the role of the individual in the lifecycle of the entrepreneurial process and his or her characteristics or actions • the APS is unique because it studies entrepreneurship through peoples’ motives and attitudes as they strive to set up, start and maintain businesses • a representative sample of the adult population is used to measure the entrepreneurial activity that represents the entire country

13. GEM Entrepreneurial Phases

14. APS Questionnaire

15. APS Questionnaire • The APS is comprised of MODULES of different groups of questions • Core APS questions (mandatory) • Special Topic (2013: Entrepreneurship and Well-being)(mandatory) • Innovation Questions (optional) • Network Question (optional) • Youth Questions (mandatory/optional) • Employee Entrepreneurial Activity Questions (optional) • Registration Question (optional) • European Commission Questions (optional) • The modules are distributed within the questionnaire in different BLOCKSfor different target respondents

16. APS Blocks • Introduction: All respondents • Block 1: Nascent entrepreneurs • Block 2: Owner-managers • Block 3: Potential and discontinuing • Block 4: Informal investors • Block 5: Occupation& 2013 Special Topic • Block 6: Reserved (not used in 2013) • Block 7: All respondents (demographics)

17. APS Block 1 INTRODUCTION ENTIRE SAMPLE (CORE) ENTIRE SAMPLE (OPTIONAL) Q1A1 Q1A2 BLOCK1 Yes Don’t know NASCENT-ENTRENREURS (CORE) NETWORK (OPTIONAL) No REGISTRATION QUESTION (OPTIONAL) TEAM-ADDED (OPTIONAL)

18. APS Block 2 Q2A Yes Don’t know BLOCK2 + OWNER-MANAGERS (CORE) Q2C Not Same Business NETWORK (OPTIONAL) No/Same business REGISTRATION QUESTION (OPTIONAL) TEAM-ADDED (OPTIONAL)

19. APS Block 3 & 4 Q3B BLOCK3 Yes POTENTIAL AND DISCONTINUING (CORE) NETWORK (OPTIONAL) No TEAM ADDED (OPTIONAL) BLOCK4 INFORMAL INVESTORS (CORE) TEAM-ADDED (OPTIONAL)

20. APS Block 5 & 7 BLOCK5 ENTIRE SAMPLE (OCCUPATION) (CORE) 2013 SPECIAL TOPIC (MANDATORY) ENTREPRENEURIAL EMPLOYEE ACTIVITY (OPTIONAL) INNOVATION (OPTIONAL) TEAM-ADDED (OPTIONAL) BLOCK7 DEMOGRAPHICS (CORE) VENDOR PROVIDED VARIABLES (CORE) TEAM-ADDED (OPTIONAL)

21. APS Questionnaire Format • Full questionnaire, with codes and skip patterns.

22. Sample Design

23. Sample Design • It is important to understand how the data you are analyzing has been collected. • Critical issues include: • Sample strata? • Any oversamples? • Different methodology?

24. Oversamples • Some national teams do not collect a sample that is proportional to the national population distribution – they add an oversample. • Required to meet the n=2000 minimum not counting any oversamples. • Any analysis needs to account for the inclusion (or not) of the oversample

25. Different Methodology • Sample can be: • Face-to-face + fixed line • Mobile + fixed line • Face-to-face + mobile + fixed line • Each methodology can have its own bias • Age and gender distribution • Education, region, income, propensity to start a business

26. Sample Size • Minimum APS sample size = 2000 • Sample sizes vary by nation • Sample sizes vary by year for each nation

27. Sample Sizes (2012)

28. Sample Size Concerns • Beware when combining data from nations or years with different sample sizes. • Not all questions are asked of all respondents, so be aware of number of respondents for the questions you are examining. • The weighted total for any national analysis will match the unweightedtotal. • Weights do NOT adjust for different sized samples. • Combined APS data files would over-represent Spain.

29. Fraction of Sample

30. Number in N=2000 Sample

31. StatisticalHelp • Statistics Without Borders • Statistics Without Borders is an apolitical organization under the auspices of the American Statistical Association, comprised entirely of volunteers, that provides pro bono statistical consulting and assistance to organizations and government agencies. • Data Without Borders • Data Without Borders seeks to match non-profits in need of data analysis with freelance and pro bono data scientists who can work to help them with data collection, analysis, visualization, or decision support.

32. Weights

33. Weights • All analysis of GEM APS data MUST use weights. • Weights differ by the type of sample for each country. • Weights differ according to the age range for each country. • Weights do NOT account for different sample sizes.

34. Vendor Provided Weights Worksheet in Survey Report to describe how weights are calculated

35. Weights • All GEM APS weights have a mean of 1.0. • Vendor-supplied weights are checked – must match national age/gender distribution. • Strata-based weights must match overall strata sizes. • Very large values for weights (high leverage cases) are not allowed.

36. Types of Weights • RESPONSE WEIGHTS: To achieve a better match of the sample surveyed to that of the overall population. • To reduce the potential bias among the collected data due to different responses rates or normal sampling variability. • Required for every country. • SAMPLE DESIGN WEIGHTS: To address specific design issues for the survey sample. • Example, a sample that is divided by urban and rural, with the urban areas sampled at a higher rate than the rural areas. Without weighting, the sample data would over-represent urban respondents.

37. Response Weights • Provided by survey vendor (using age, gender, and perhaps additional variables), or • Calculated as part of the GEM APS harmonization process using information on the gender of the respondent and their age coded in normal GEM age groups.

38. Sample Design Weights • Required for all countries whose sampling design used different strata or divisions. • Examples: • Sample has been collected using different procedures • Sample has been collected using different sampling densities (an “oversample”)

39. GEM National APS Weights • Weight_L • Used for the population aged 18 to 64 (L = Labor Force). • Weight_A • Used for the entire adult population aged 18 to 99 (A = Adults) • If national team only collected data for ages 18 to 64, then Weight_L and Weight_A are identical.

40. GEM Regional APS Weights • Only provided for teams who request regional-level analysis. • WEIGHT_LR • WEIGHT_AR • Each region has its own weights calculated as if they were the entire nation. • Should be used ONLY for region-to-region comparisons, not for any national-level analysis!

41. Final Lesson • Always use weights for any GEM APS individual-level analysis. • All Global Report results calculated using Weight_L. • Selection of proper weight depends on analysis being conducted (age range, regional or national). • Use national-level weights for all but individual regional analysis.

42. Proposal Submission Process

43. RFP Package • Request for Proposal (RFP) is posted to the GEM consortium web site and an announcement is emailed to all teams in February of each year. • The RFP includes all documents needed by National Team to submit APS/NES proposal, administer the questionnaires, record and submit all data. It also includes valuable reference information. “RFP” refers to the entire document package as well as the Request for Proposal document itself.

44. RFP Package (APS question commentary) Detailed description of each question

45. RFP Package (SPSS data input file) Use provided SPSS Data Input Template to submit final survey data. Data in any other format will not be processed

46. RFP Package (survey report: methodology overview) Provide full details of sample design (usually completed by vendor)

47. RFP Package (survey report: strata & fieldwork) • List and describe sample strata (if no strata used, national information) • Complete fieldwork report for each strata after data is collected

48. Preparing an APS Proposal • Download and carefully read the RFP package • Work with Survey Vendor to decide on appropriate survey methodology and fill out all required RFP documents • Vendors must use the GEM forms, they cannot submit their own forms • Vendors must also submit a Survey Vendor Proposal, for which we provide no template • Customize your questionnaire and SPSS Data Input Template • Decide which optional modules you will include (if any) • Decide if you will add additional questions • Establish you demographic variable answer categories • Translate the questionnaire if necessary

49. Tips for translating APS questionnaire • Work with vendor on translations • National team has more knowledge on the purpose of the questions • Vendors have more knowledge on how translated questions will be received by the respondents • Use back-translations to check if the questions have been translated well

50. Submitting an APS Proposal • Email the completed forms to data@gemconsortium.org(refer to RFP for specific instructions on how to label and email proposal documents). • The GEM Data team will review submitted documents and proposed sampling methodology and send National Team a review of their proposal in a new worksheet added to Survey Report document (G. Proposal Review). • The national team MAY NOT BEGINAPS survey until after their proposal has been officially approved (via email) by GEM Data Team.