Basic Sampling Concepts

1. Basic Sampling Concepts (Session 02)

2. Learning Objectives By the end of this session, you will be able to • describe what is meant by sample, target population, sampled (study) population, sampling frame, sampling units • explain what is meant by a representative sample • discuss the importance of sampling to ensure survey results are generalisable – use of probability based samples • describe key issues central to the sampling process

3. Population • The population is the entire group of all entities about which we want information, e.g. • All children below 16 years of age ~ to assess their nutritional levels • All farmers growing maize ~ to learn about total maize production in the country • All schools in a country ~ to learn about educational achievements • Suppose you want to estimate the proportion of adults in rural areas with no primary education. What do you think is your population?

4. More on Populations • Population sizes may be finite or infinite. We will allow the possibility of finite populations in this module. • Describing the population clearly is necessary before sampling is considered. Why do you think this is so? • Need also to distinguish between the target population, as defined above, and the study (or sampled) population. Latter excludes the inaccessible members of the target group e.g. temporary international migrant workers.

5. Samples • A sample is a subset of units drawn from the population. This module is largely on sampling procedures. • Sampling units refer to the entities on which measurements are made during a survey • During planning (and implementation) of a survey, check the extent to which the target population can be (or has been) covered. Recognise that survey conclusions apply only to the population (study popn) which has a chance of inclusion in the sample.

6. Issues related to sampling units Part of the planning of the sampling process includes: • Identifying the sampling unit. Is it • a village, household, farm, school, etc? • Recognising occurrence of units at different levels. • In most surveys, sampling is done at different levels, e.g. district, then village, then household, then household member. Need to be specific about the selection of sampling units and sampling effort at each stage and specify this in the sampling protocol.

7. More on sampling units • Observing units over time • In longitudinal surveys, or monitoring and evaluation surveys, should the same units be chosen for measurement each year? • Unequal sized units • Are all units, whose measurements are being summarised, of the same size? If not, e.g. with business surveys, should measurements be weighted according to size? • Number of units • Is the sample large enough to enable an analysis that addresses the study objectives?

8. Sampling frames • The sampling frame is a list of all sampling units, for example • list of villages in a region • list of households within a village • list of schools in the district • Why do we need a sampling frame? • Tool essential for objectively selecting a sample of units from the population of all units • Different frames are needed if sampling at different hierarchical levels • May develop relevant sub-frames at each stage as sampling progresses down the hierarchy

9. Representativeness • The primary aim in a survey is to make inferences about the target population • For this, need to ensure population is well represented in the sample • Bring in qualitative aspects of the population to ensure there is adequate representation of divisions of the population, e.g. rural/urban, different wealth categories, geographical coverage, etc • May be achieved by sampling from these different sub-groups so that all necessary factors likely to influence survey results are represented • In doing so, ensure selection is such that variability of sampling units within each sub-division is captured, i.e. not just 1 unit per group.

10. Non-representativeness? • Example: in a survey of Kenya, urban Nairobi is represented by a random sample of size 10, which, by chance, includes eight women, five being office workers and four of those employed in Ministry offices. Both the two men, by chance, are unemployed. • Example: a random sample of two areas in Malawi happened to come up with urban Mzuzu and urban Zomba [2 of the 6 largest towns] • In both cases, these (honest) random samples were of very small size, and by chance produced quite untypical results.

11. Generalisability • Also key to survey success is to ensure the sampling is such that survey results can be generalised to the study population. • Generalisability requires taking probability-based samples during the sampling process. • Probability sampling is the general term for methods where sample selection is objectively-based on known chances of inclusion in the sample. • If the probabilities are known and non-zero, they don’t have to be equal: corrections can be made at the data analysis stage.

12. Non-generalisability • Example – a researcher selected her sample from a district, including one farm each with all possible combinations of (i) child-/female-/male-headed; (ii) Christian/Muslim; (iii) dambo/hillside land; (iv) purely vegetable-/mixed/purely maize-growing • Some cases very hard to find e.g. there werefew Muslim, child-headed, purely maize dambo farms. • Sample of 36 does not relate to population proportions: not enough cases to be able to represent the commoner combinations • Researcher’s selections not objective/random and cases only came from one smallish area, maybe untypical. Cannot generalise to country!

13. Summary of issues to consider when sampling • Have the objectives been clearly specified? • Has the target population been clearly defined and the possibility of survey results being applicable to a different ‘study’ population been recognised? • What is the geographical coverage? • What factors are likely to influence survey results – have they been considered in the sampling? • What should be the sampling unit(s) for fieldwork?

14. Summary of issues to consider when sampling - continued • Will the sample results lead to generalisable conclusions? • Will the proposed sampling plan be possible within time and budget limitations? • Is the sampling procedure practically feasible? • Will the adopted sampling scheme provide results that address survey objectives with appropriate measures of precision? Developing a good sampling plan requires time and effort! Full plan should be well documented with full justification!

15. References Pettersson H. (2004) “Design of master sampling frames and master samples for household surveys in developing countries”. Chapter V of the UN Publication An Analysis of Operating Characteristics of Household Surveys in Developing and Transition Countries: Survey Costs, Design Effects and Non-Sampling Errors. Pp.71-94. Available athttp://unstats.un.org/unsd/hhsurveys/index.htm De Vaus, A.D. (2001) “Research Design in Social Research”. Sage Publications, London - for discussion on generalisability Bechhofer, F. & Paterson, L. (2000) “Principles of research design in the social sciences”. Routledge, London – for a discussion on representativeness

16. Some practical work follows …