Surveys

1. Surveys Week 5 NJ Kang

2. What is a Survey? • It is a research strategy than a method or technique  being ubiquity • The collection of a small amount of data in standardized form from a relatively large number of individuals; and • The selection of samples of individuals from known populations.

3. By Bryman (1989) • Survey research entails the collection of data on a number of units and usually at a single juncture in time, with a view to collecting systematically a body of quantifiable data in respect of a number of variables which are then examined to discern patterns of association.

4. By Bryman (1989) • Survey research entails the collection of data on 1) a number of units and usually at 2) asingle juncture in time, with a view to collecting 3)systematically 4) a body of quantifiable data in respect of 5) a number of variables which are then 6) examined to discern patterns of association. Some times Repeatedly done in different time range In depth interview?

5. Planned change in experiments is absent in surveys. • The experiment is active in that it asks; what happens if this is changed? • It is passive and seeks to describe and analyse, • But in some cases to explore, but • Usually it seeks what the individuals surveyed think or feel about the topic. • The logic of causal inference as used in experimentation is not available.

6. Survey Experiment Active in planning Sees to see what will happen if something is done to the samples Needs causal relationship Explorative Qualified? Samples from known population. • Passive in planning • Seeks to see how people think or feel of a topic without being affected by any cause • Lack of causal relationship • Descriptive, Explanative • Quantification (?) • Samples From know population

7. Hybrid research strategies • Use of the typical features of more than one strategy are combined within a single study. • A sequence of surveys is carried out before, during and after an intervention

8. A combined strategy. • To have a mainly survey based study accompanied by a set of case studies (chosen often on the basis of the survey, illuminate, enrich and bring to life the survey findings). • The role of intensive workshop on the teachers’ perceptions on the use of culture based contents in a 3rd grade high school English textbook .

9. Advantages and Disadvantages of the Survey • Some see the survey as the central real world strategy. • It may be that in non-laboratory situations where experiments are often neither feasible nor ethically defensible, surveys give that reassuring scientific ring of confidence. • Associated with surveys is a satisfyingly complex set of technological concerns about sampling, question-wording, answer-coding, etc.

10. Others views • Surveys as generating large amounts of data of dubious value. • Falsely prestigious because of their quantitative nature, the findings are seen as a product of largely uninvolved respondents whose answers owe more to some unknown mixture of politeness, boredom, desire to be seen in a good light, etc. than their true feelings, beliefs or behaviour.

11. The truthworthiness of the data depends to a • The technical proficiency of those running the survey. • Internal Validity • Cannot seek the samples’ perceptions and feelings of the topic due to the use of • Invalid questions • Lengthy wordings • In comprehensible wordings. External Validity Will be threatened when the sampling is faulty, Can’t generalize our findings Reliability Presenting all respondents with the same standardized questions, carefully worded after piloting,

12. Why Survey?

13. descriptive purposes of surveys • Usually to describe how people think of a certain topic. • It can provide information about the distribution of a wide range of people characteristics, and of relationships between such characteristics. (comparison between gender, age, educational back ground etc.)

14. The interpretive purposes of surveys • It is not easy since the information is typically in the form of correlations • Correlation does not imply causation. • Needs an example. (Jullie’s)

15. Formal Design for Surveys • Purely a descriptive purpose • The simple survey • The panel survey • Rotating sample survey.

16. The simple survey • Ad hoc sample survey • The survey involves collecting the same standardized data from an undifferentiated group of respondents over a short period of time. • Pre-experimental designs; the one–group post-test only design

17. The simple survey • Comparison group survey • As the group of respondents is likely to incorporate naturally occurring variables with several levels. (e.g. correlation between low performance and economic status) • It can be reflected exclusively into the design. • The post-test only non-equivalent groups design. • Moving from description to establishing causation is difficult with this design.

18. The panel Survey • Design data are collected on the same set of respondents at two or more points in time. • As with the simple survey, this can also be characterized as incorporating two or more comparison groups by virtue of the inclusion of naturally occurring variables. • The effects of higher education? (4 yrs.)

19. The panel Survey Disadvantages Advantages Inclusion of a clear temporal sequence in the data obtained. • Being test-wise • Mortality of group members • But can not have causal relationship but useful for additional interpretation of data • E.g. the effect of workshop on teachers’ implying Culture related Contents in ELT.

20. Pseudo-panel design • It is possible to use logic and common sense to produce a time sequence for the effects of a range of variables even when the data are collected cross-sectionally rather than longitudinally. 1st economic status 2nd length of staying abroad 3rd country types. 4th types of institution. 5th individual perceptions/characteristics

21. Practicalities of Design • Who do you ask? Samples • How do you ask?  wordings, types (I or S) • What do you ask?  Questionnaires • What resources do you need? 

22. Mounting a small-scale survey • Sort out the general purpose and the specific information requirements • Your preliminary analysis has indicated that a survey is the most appropriate way of addressing your research questions. • The first is to clarify the research questions • Next develop a range of sub-questions, or subsidiary topics, relating to the central question. • Work out the specific information required in connection with each topic or sub-questions.

23. 2. Construct the questionnaire 3. Determine the population and the sample to be selected 4. After the planning stage

24. Probability Samples • Simple Random Sampling • Systematic Sampling • Stratified Random Sampling • Cluster Sampling • Multi-stage Sampling

25. Simple Random Sampling • Selection at random from a list of the population of the required number of persons for the sample.  a lottery method, random number tables, or a computer can be used. • Can’t produce a simple random sample without a full list of the populations.

26. Systematic Sampling • Choosing a starting point in the sampling frame at random, and then choosing every nth person. (needs fifty out of 2000, then fortieth person is chosen. ) • Needs a full list of the population.

27. Non-probability Samples • Quota Sampling • Dimensional Sampling • Convenience Sampling • Purposive Sampling. Snowball Sampling

28. Stratified Random Sampling • Use of STRATA (different groups, status, gender, etc) share a particular characteristic or characteristics. • Proportionate sampling • Disproportionate sampling

29. Proportionate sampling •  same proportion of samples from the same numbers of population Disproportionate sampling •  different proportion of samples due to the different numbers of population or •  differences in their importance. e.g. 3rd grade learners’ perceptions on foreign English teachers and the correlation between their levels of language proficiency

30. Cluster Sampling • This involves dividing the population into a number of units, or clusters, each of which contains individuals having a range of characteristics. • The clusters themselves are chosen on a random basis. • The sub-population within the cluster is then chosen. • Instead of children but school children.

31. Multi-stage Sampling • Take a random sample of schools; then a random sample of the classes within each of the schools; then from within the selected classes choose a sample of children. • Generate a geographically concentrated sampling. • The generalizability issue is the same as for cluster sampling, but judicious use of sampling at appropriate stages enables one to tailor the scale of the project to the resources available.

32. Non-probability Samples • Purposive samples. • No intention or generalization • Quota Sampling • Dimensional Sampling • Convenience Sampling • Purposive Sampling • Snowball Sampling

33. Non-probability Samples • Small-scale surveys commonly employ non-probability samples. • Usually less complicated to set up and are acceptable when there is no intention or need to make a statistical generalization to any population beyond the sample surveyed. • They can also be used to pilot a survey prior to a probability sample approach for the main survey. • Purposive samples.

34. Quota Sampling To obtain representatives of the various elements of a population, usually in the relative proportions in which they occur in the population

35. Convenience sampling • Cheap and dirty way of doing a sample survey. • Not a representative findings. • Choosing the nearest and most convenient persons to act as respondents.

36. Purposive Sampling • The principle of selection in purposive sampling is the researcher’s judgement as to typicality or interest. • the grounded theory approach. • Used as initial sampling. • For case studies.

37. Snowball Sampling • The researcher identifies one or more individuals from the population of interest. • After they have been interviewed, they are used as informants to identify other members of the population, who are themselves used as informants, and so on.

38. Homework • Design your own survey questions and types of surveys with reasons. • Read 9 and summarize