Download
1 / 24
E N D
Survey Designs “Most people are familiar with surveys. We often receive surveys to record opinions as voters, register approval of consumer products, and to measure opinions about electoral candidates. To many people, survey research is simply a “survey” instrument such as a questionnaire or interview. Although we survey people using in an instrument in educational research, the instrument is only one aspect of a broader procedure in survey design” ( Creswell, 2012, p. 375). EDU 500, Digital Research Presentation, Christina Michalek
What is the definition of survey designs? • Survey research is a popular design in education. • Survey research designs are procedures in quantitative research in which investigators administer a survey to a sample or to the entire population of people to describe the attitude, opinions, behaviors, or characters of the population. • In this procedure researchers collect numbered data using questionnaires or interviews and statistically analyze the data to describe trends about responses.
How do survey designs differ from experimental research? “Survey designs differ from experimental research in that they do not involve a treatment given to participants by the researcher” (Creswell,2012, p. 376). • Survey researchers do not experimentally manipulate the conditions they cannot explain cause and effect as well as experimental researchers can. • Consequently, survey research describe trends in the data rather than explanations. • Survey designs can be compared to correlational designs. In both designs there is a correlation of variables, the focus merely varies.
How did survey research develop? The Beginning of Survey Research Development of the Modern Day Survey • “Early surveys date back to 1817, when Marc Antoine Jullien de Paris designed a 34 page international survey of national education systems” (Creswell, 2012, p. 376). • In the 1890’s children G. Stanley Hall surveyed children. In 1907 social problems and educational planning were examined through survey research. • During the period from World War l to World War ll the modern survey emerged. • It was developed by improving sampling techniques and the success of different scales of measurement. • The founding of pilling and survey organizations, a long with computers, the availability of data archives and storage, funding from the federal government, helped to establish the popularity of surveys in education by midcentury” (Creswell, 2012, p. 377).
When do you use survey research? Survey research is used to describe trends, determine individual opinions and to help identify important beliefs and attitudes. • Surveys provide useful information to evaluate programs or issues in schools. • For example, survey research is used to describe trends in community interests in board issues or sate or national trends about mandatory student uniform policies. • Policy issues are also polled. For example, do students need a choice of schools to attend? • Surveys can be used to follow up with graduates to learn about present careers.
What are the key characteristics to survey design? • Survey researchers sample from a population. • Investigators must collect data through questionnaires or interviews. • Instruments must be designed for data collection. • For effective survey results researchers must obtain a high response rate. • Whether a study is longitudinal or cross sectional, there are key components that must be working together when designing or evaluating a survey.
What are the types of designs that fall within this category? There are only two basic types of research surveys: cross-sectional and longitudinal.
What are the differences between the two types of designs that fall into this category? Cross- Sectional Survey Designs Longitudinal Survey Designs • Cross-sectional survey designs are the most popular survey design in educational research. • The researcher collects data at one point in time. • This is beneficial because it provides current attitudes or practices in a short amount of time. • An alternative to using cross-sectional survey designs is to collect data overtime using longitudinal survey designs. • This involves collecting data about trends with the same population, changes in a cohort group, or subpopulation. • The participants may be different or the same people.
Time of Data Collection Study over time Study at one point in time Longitudinal Cross-Sectional Changes in a subpopulation group identified by a common characteristic overtime Attitudes and practices Trends in the same population over time Community needs Program evaluation Changes in the same people over time Group comparisons Trend Panel National assessment Cohort Types of Cross-Sectional and Longitudinal Survey Designs Figure 12.1 (Creswell, 2012, p. 378)
What is an example of a cross-sectional survey? • Cross-sectional surveys are used to gather information on a population at a single point in time. • An example of a cross sectional survey would be a questionnaire that collects data on how parents feel about Internet filtering, as of September of 2012. • A different cross-sectional survey questionnaire might try to determine the relationship between two factors, like religiousness of parents and views on Internet filtering
What is an example of a longitudinal survey? • Longitudinal surveys are used to collect data over time about trends with the same population, changes in a cohort group or subpopulation, or changes in a panel group of the same individuals. • An example of a longitudinal survey would be research about college graduates and their current occupation 1 ,2 and 5 years after graduation. • Another example would be to follow up with college graduates from a specific program or school to learn their views about their educational experiences.
What are the steps to conducting survey research? “The steps of conducting research follow the general process of research. Survey steps however, address primarily the procedures for collecting data, analyzing data and writing the final report” (Creswell, 2012, p. 403). • Step 1: Decide if a survey design is the best to use • Step 2: Identify the research questions or hypotheses • Step 3: Identify the population, the sampling frame, and the sample • Step 4: Determine the survey design and data collection procedure • Step 5: Develop or locate an instrument • Step 6: Administer the instrument • Step 7: Analyze the data to address the research question or hypotheses • Step 8: Write the report
Step # 1: Decide if a Survey Design is the Best to Use You must first decide if survey research fits the study best. Survey research focuses on trends in a population or to describe the relationship among variables. Times when surveys are most suitable are to assess trends, study characteristics of a population, learn about individual attitudes or beliefs, evaluate the effectiveness of a program, and identify the needs of a community.
Step # 2: Identify the Research Question or Hypotheses You can acknowledge both the research question and the hypotheses in a survey design. Surveys do lend themselves to hypotheses because you will be studying / sampling to draw inferences from a population. Research questions and hypotheses must describe the characteristics / trends of a population of people, compare groups in terms of specific attributes, or relate two or more variables.
Step # 3: Identify the Population, the Sampling Frame, and the Sample The process of survey research begins with selecting the population. This means defining the population, determining the number of people, considering whether or not you can obtain a list of names for the sample. Also, you must select the characteristics of the population that will be sampled (i.e. male / female / both). Preferably using random sampling next the researcher must select the sample. The adequate sample size should be determined using a sampling formula.
Step # 4: Determine the Survey Design and Data Collection Procedures The researcher must determine if the survey will be longitudinal or cross-sectional. This decision relates to the problem or topic being studied. Researchers must also consider whether data collection will be based on questionnaires or interviews. This requires weighing the advantages and disadvantages of each form.
Step # 5: Develop or Locate an Instrument A researcher needs an instrument to collect or measure the variables in a study. It is easier to locate one than create an instrument. Standards of reliability and validity need to be applied to scores from existing instruments before you can select them for use. If a study only addresses a few variables it typically easiest to create an instrument.
Step # 6: Administer the Instrument This step is probably the most time consuming phase. This involves obtaining permission to conduct the study and using procedures for gathering the data. This means training interviewers, preparing questionnaires, continuously following up to ensure a high response rate, checking for bias and preparing the data for analysis.
Step # 7: Analyze the Data to Address the Research Question The analysis stage will reflect the types of research questions and hypotheses that the researcher plans to address in the study. The researcher must note the response rate, check for bias, conduct descriptive analysis, and answering descriptive questions.
Step # 8: Write the Report The survey should be written using the standard quantitative structure. This means writing an introduction, review of the literature, the methods, the results, and the discussion.
What are the ethical issues surrounding survey design? Ethical issues can arise in survey research at distinct points in the research process, such as in collecting data, analyzing results, and in reporting results. • Survey researchers may use incentives to ensure participants, however the incentives should be so large that the studies ethics are called into question. • Good ethical practices also state the researcher should not overstate the benefits of participating and should ensure that the benefits are delivered. • Researchers should not put interviewers in an unsafe environment to gather answers. • Respondents / participants need to be kept safe as well. Confidentiality should be protected. • When the project concludes the researcher is responsible for the destruction of the survey instruments.
How do you Evaluate Survey Research? • Whether the research consists of mailed questionnaires or interviews surveys need to meet high standards of quality. • When conducting a survey there must be a “methods” section that conveys the detailed survey procedures. • For educators who read and use results from survey research there is a checklist of items to look for in a published study to determine quality.
Quality Criteria Checklist (Creswell, 2012) • Describes and specifies the target population / sample. • Identifies how the sample was derived. • Discusses the size of the sample and the means for deriving the sample size. • Uses a type of survey that matches the research questions / hypotheses. • Clearly identified the instruments used in data collection and how they were selected. • Reports information of the reliability and validity of scores from past uses. • Discusses the procedures for administering the instrument. • Mentions appropriate follow up procedures • Provides examples of the questions or questionnaire • Uses data analysis procedures to answer the research question or hypotheses • Writes the study in a scholarly way and identifies potential ethical issues.
Works Cited Creswell, John W. (2011). Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research (4th Edition) Pearson.
