1. Research Seminars in IT in Education (MIT6003) Qualitative Educational Research Design & Practical Concerns II Dr Jacky Pow
2. Agenda Qualitative data analysis
Management of time and resources
Validity and reliability in qualitative research
Practical concerns
Qualitative research critique exercise
3. Qualitative data analysis ???? Operational definition
Behaviour
Phenomena
Unit of analysis (pre-defined in design stage)
Coding (transcription and field notes)
Classifying into Categories
Checking inter-rater’s reliability (if any)
Tallying / encoding
Identifying patterns
Theme development
4. Qualitative data analysis ???? Pilot study (to foresee feasible research directions and testify researchable contexts)
Research rigor: precise measurement, refutability of knowledge claims
Replicability of findings: qualitative researchers do not concern much about this
Identify relationships among variables in quantitative data (if necessary)
5. Management of time and resources Time, personnel and financial support are critical to the success of a qualitative study:
Time (sufficient to track the phenomena as they developed)
Personnel (more than one researchers involve in the same study)(capable of thoroughly and efficiently gathering the needed data)
Resources (for personnel, travel, data analysis, and report writing)
6. Management of time and resources Due to the fluid and flexible nature of qualitative research, additional costs may incur
Mindful of the research scale (not too ambitious but can reveal the phenomena as far as possible)
However, determination of resources needed for the study must often wait until fundamental design decisions have been made
7. Objectivity of qualitative research Objectivity is the simultaneous realization of as much reliability and validity as possible (Kirk and Miller, 1986)
Social research can never be ‘totally objective’ (Babbie, 1992) due to the ‘natural subjectivity’ of qualitative research (Marshall and Rossman, 1995)
Should not be judged in a positivists way (Winter, 1989)
8. Reliability ?? Reliability is the degree to which the finding is independent of accidental circumstances of the research (Kirk and Miller, 1986)
‘Fluidity and flexibility’ of methods (Mason, 1996 )
Qualitative study does not pretend to be replicable and qualitative researchers tend to avoid controlling the research conditions
9. Enhancing reliability in qualitative study Stress the importance of repeatability of observations (Kirk and Miller, 1986)
Careful documentation of procedures for collecting and interpreting data
Ensure that the data is not only appropriate to the research questions, but also thorough, careful, honest and accurate (Mason, 1996)
10. Validity ?? Validity is the degree to which the finding is interpreted in a correct way (Kirk and Miller, 1986)
There are two kinds of validity: internal and external validity
11. Measure of internal validity Internal validity (the degree to which correct conclusions about causal relations can be drawn ) is not relevant to a qualitative study (Yin, 1994), especially in descriptive and exploratory studies
Qualitative research tends not to make such causal assertions (Mertens, 1998)
Trustworthiness of the findings
12. Measure of external validity External validity means the degree of transferability (Lincoln and Guba, 1985) or generalisability (Cohen and Manion, 1989) of the findings
Context-sensitive nature of qualitative research
Cannot have high transferability and generalisability – a general weakness of qualitative research
The stronger the falsification attempts a proposition has survived, the more valid, the more trustworthy the knowledge (Kvale, 1996)
13. Triangulation ????? Aims to enhance validity and trustworthiness of findings / knowledge
Types of triangulation (Denzin & Lincoln, 1994)
Cross-perspective
Cross-data-source
Cross-method
Cross-person
14. Researcher’s role management Issue of entry
Entry negotiation requires time, patience, and sensitivity to the rhythms and norms of a group
Building Trust
Managing the political
Reciprocity
Confidentiality and trust
Ethics
Researchers must demonstrate awareness of the complex ethical issues (teacher as researcher)
The research will not be harmful to the subjects
15. Practical concerns Limited local reference materials and literature (?????????????)
Non-local materials and literature may not directly applicable
Hard to find participants (??????)
Don’t want to be observed or video-taped
The issue of teaching privacy
The problem of Hawthorn Effect – invalid data
Students are not really want to be participated
16. Practical concerns Data Coding problem (??????)
To use the data collected, we need to code them
To code the data, we need a data coding system in place
Since the coding systems in foreign countries may not suit the local context, new coding systems are needed, making the re-use of coding system difficult
Researchers need to design their own coding categories
17. Practical concerns Difficulty in transcribing (??????)
Transcribing data is a tedious job (in general, 1 hour of interview needs 6-7 hours to complete the transcription)
If you do not have the time to do the transcription, you may pay someone to do it for you. But you have to make sure s/he has the patient and ability to get the job done
18. Practical concerns Inexperience in data analysis (?????????)
Inter-rater reliability (lack of experienced raters)
Operational definition are usually unclear or keep changing
Easily get frustrated by the amount of time and energy in putting data into the categories
19. Practical concerns Lack of experienced qualitative researchers (????????????)
Academics and researchers usually received formal quantitative training
Qualitative research in education is new to many academics and researchers as it only be introduced in the last 20-30 years at most
Many qualitative researchers could not complete their research because they were unable to seek help from experienced ones to solve their technical problems in data analysis
