Research Seminars in IT in Education MIT6003 Qualitative Educational Research Design Practical Concerns II

1. Research Seminars inIT in Education(MIT6003)Qualitative Educational Research Design & Practical Concerns II Dr Jacky Pow

2. Agenda Qualitative data analysis Management of time and resources Validity and reliability in qualitative research Practical concerns Qualitative research critique exercise

3. Qualitative data analysis ???? Operational definition Behaviour Phenomena Unit of analysis (pre-defined in design stage) Coding (transcription and field notes) Classifying into Categories Checking inter-rater�s reliability (if any) Tallying / encoding Identifying patterns Theme development

4. Qualitative data analysis ???? Pilot study (to foresee feasible research directions and testify researchable contexts) Research rigor: precise measurement, refutability of knowledge claims Replicability of findings: qualitative researchers do not concern much about this Identify relationships among variables in quantitative data (if necessary)

5. Management of time and resources Time, personnel and financial support are critical to the success of a qualitative study: Time (sufficient to track the phenomena as they developed) Personnel (more than one researchers involve in the same study)(capable of thoroughly and efficiently gathering the needed data) Resources (for personnel, travel, data analysis, and report writing)

6. Management of time and resources Due to the fluid and flexible nature of qualitative research, additional costs may incur Mindful of the research scale (not too ambitious but can reveal the phenomena as far as possible) However, determination of resources needed for the study must often wait until fundamental design decisions have been made

7. Objectivity of qualitative research Objectivity is the simultaneous realization of as much reliability and validity as possible (Kirk and Miller, 1986) Social research can never be �totally objective� (Babbie, 1992) due to the �natural subjectivity� of qualitative research (Marshall and Rossman, 1995) Should not be judged in a positivists way (Winter, 1989)

8. Reliability ?? Reliability is the degree to which the finding is independent of accidental circumstances of the research (Kirk and Miller, 1986) �Fluidity and flexibility� of methods (Mason, 1996 ) Qualitative study does not pretend to be replicable and qualitative researchers tend to avoid controlling the research conditions

9. Enhancing reliability in qualitative study Stress the importance of repeatability of observations (Kirk and Miller, 1986) Careful documentation of procedures for collecting and interpreting data Ensure that the data is not only appropriate to the research questions, but also thorough, careful, honest and accurate (Mason, 1996)

10. Validity ?? Validity is the degree to which the finding is interpreted in a correct way (Kirk and Miller, 1986) There are two kinds of validity: internal and external validity

11. Measure of internal validity Internal validity (the degree to which correct conclusions about causal relations can be drawn ) is not relevant to a qualitative study (Yin, 1994), especially in descriptive and exploratory studies Qualitative research tends not to make such causal assertions (Mertens, 1998) Trustworthiness of the findings

12. Measure of external validity External validity means the degree of transferability (Lincoln and Guba, 1985) or generalisability (Cohen and Manion, 1989) of the findings Context-sensitive nature of qualitative research Cannot have high transferability and generalisability � a general weakness of qualitative research The stronger the falsification attempts a proposition has survived, the more valid, the more trustworthy the knowledge (Kvale, 1996)

13. Triangulation ????? Aims to enhance validity and trustworthiness of findings / knowledge Types of triangulation (Denzin & Lincoln, 1994) Cross-perspective Cross-data-source Cross-method Cross-person

14. Researcher�s role management Issue of entry Entry negotiation requires time, patience, and sensitivity to the rhythms and norms of a group Building Trust Managing the political Reciprocity Confidentiality and trust Ethics Researchers must demonstrate awareness of the complex ethical issues (teacher as researcher) The research will not be harmful to the subjects

15. Practical concerns Limited local reference materials and literature (?????????????) Non-local materials and literature may not directly applicable Hard to find participants (??????) Don�t want to be observed or video-taped The issue of teaching privacy The problem of Hawthorn Effect � invalid data Students are not really want to be participated

16. Practical concerns Data Coding problem (??????) To use the data collected, we need to code them To code the data, we need a data coding system in place Since the coding systems in foreign countries may not suit the local context, new coding systems are needed, making the re-use of coding system difficult Researchers need to design their own coding categories

17. Practical concerns Difficulty in transcribing (??????) Transcribing data is a tedious job (in general, 1 hour of interview needs 6-7 hours to complete the transcription) If you do not have the time to do the transcription, you may pay someone to do it for you. But you have to make sure s/he has the patient and ability to get the job done

18. Practical concerns Inexperience in data analysis (?????????) Inter-rater reliability (lack of experienced raters) Operational definition are usually unclear or keep changing Easily get frustrated by the amount of time and energy in putting data into the categories

19. Practical concerns Lack of experienced qualitative researchers (????????????) Academics and researchers usually received formal quantitative training Qualitative research in education is new to many academics and researchers as it only be introduced in the last 20-30 years at most Many qualitative researchers could not complete their research because they were unable to seek help from experienced ones to solve their technical problems in data analysis