EDU5950 SEM1 2015-16

1. EDU5950 SEM1 2015-16 RELIABILITY ANALYSIS -CRONBACH ALPHA TEST FOR NORMALITY

2. Steps in Conducting Research

3. Consists of assigning numbers or labels to units of analysis to represent the variables/phenomenon under study. The key variables need to be translated into numbers so the researcher can analyze the data using statistics. The process of measurement consists of moving from the theoretical definition of a variable (typically called the construct definition) to the concrete mode of measuring that variable in the research study. Measurement

4. When designing an instrument, keep in mind the following: The conclusions drawn in a research study are only as good as the data that is collected. The data that is collected is only as good as the instrument that collects the data. A poorly designed instrument will lead to bad data, which will lead to bad conclusions. Therefore, developing a good instrument is the most important part of conducting a high quality research study. Measurement (REFER TO TEACHER EFFICACY DATA)

5. Validity is the most important consideration in developing and evaluating measuring instruments. Validity is the degree to which evidence and theory support the interpretations on the meaning of the scores derived from the instrument Content Validity: “based on expert ratings of the items” in the test or measurement Construct Validity: “based on the extent of scores derived from the instrument truly reflect the theory behind the psychological construct being measured. Validity and reliability

6. Reliability refers to “how well we are measuring whatever it is that is being measured (regardless of whether or not it is the right quantity to measure).” • In statistics or measurement theory, a measurement or test is considered reliable if it produces consistent results over repeated testings. -D. Rindskopf, Reliability: Measurement. In: Neil J. Smelser and Paul B. Baltes, Editor(s)-in-Chief, International Encyclopedia of the Social & Behavioral Sciences, Pergamon, Oxford, 2001, Pages 13023-13028. (http://www.sciencedirect.com/science/article/B7MRM-4MT09VJ2XN/1/083e3cc0b8b9d4e027b0ba214dcd9fa3) Reliability

7. Test-Retest reliability – administer a test or instrument to the same group of individuals on two occasions and correlate the two sets of scores • Equivalent-Forms reliability – administer two equivalent forms of test or instrument to the same individuals • Internal-Consistency measures of reliability – test whether all the items in the test or instrument are measuring the same thing. It is a measure of homogeneity of the items. • Split-Half reliability • Kuder-Richardson procedures • Coefficient Alpha (Cronbach Alpha after Lee Cronbach) How can reliability be established?

8. RELIABILITY ANALYSISRefer to MATHEMATICS TEACHERS’ EFFICACY data

9. 1. Recode all negative items

10. 2. CLICK ANALYZE => SCALE => RELIABILITY ANALYSIS(you will get the reliability dialog box)

11. 3. Transfer the measured items to the right box – 14 items for TEACHERS’ EFFICACY scale

13. 4. Select the following icons => continue => ok

18. The pilot study was administered to 31 graduate students undertaking Statistics for Educational Research. The internal consistency estimates, based on Cronbach’s alpha, were satisfactory thus acceptable. Based on 14 items, the Cronbach’s alpha obtained is .891. However further analysis using 12 items (by omitting item 11 AND 13) the internal consistency increased to .904. REPORTING

20. Use these for item analysis; i.e., determin-ing quality of individual items.

21. Test for normality of scores

22. TEST FOR NORMALITY

23. TEST FOR NORMALITY

24. TEST FOR NORMALITY

26. TEST FOR NORMALITY

27. TEST FOR NORMALITY

28. TEST FOR NORMALITY

29. TEST FOR NORMALITY TEACHER_EFFICACY Stem-and-Leaf Plot Frequency Stem & Leaf 1.00 2 . 2 6.00 2 . 577999 15.00 3 . 000012222222224 14.00 3 . 55556677778899 6.00 4 . 001222 9.00 4 . 556677779 6.00 5 . 001333 5.00 5 . 56677 Stem width: 1.00 Each leaf: 1 case(s)

30. TEST FOR NORMALITY

31. TEST FOR NORMALITY

32. TEST FOR NORMALITY

33. Normality can be assessed in SPSS using the Explore option of the Descriptive Statistics menu. In this example we will assess the normality distribution of Scores of Test I. We wish to assess for the sample as a whole. If we want to do separately for subgroup within our sample, moves the variables that define these subgroups into the factor list. exploring data

34. Click Analyze => Descriptive Statistics => Explore. To assessing normality

35. Click and move variables you require statistics, graphs and test for into Dependent List.

36. In Display section, select Both. In the Explore Statistics dialogue, Descreptive are selected by default.ClickContinue.

37. Click on the Plot button. Select Histogram and Normality plots with test. Click on Continue.

38. Click on Option. In the Missing value section, click Exclude cases pairwise. Click continue. Then OK.

39. outputs

41. The descriptive statistics is shown in the tables. To obtain the 5% Trimmed Mean, SPSS removes the top and bottom 5% of your cases and recalculate a new mean value. Compare the original mean with new trimmed mean to see whether some of the more extreme scores are having a strong influence on the mean. If these two mean values are very different, you may need to investigate these data point further. outputs

42. The Kolmogorov-Smirnov statistic assess the normality of the distribution of scores. A non-significant result ( Sig value more than .05 ) indicates normality. The actual shape of the distribution can be seen in the Histograms. For this Histograms, scores appear to be reasonably normally distributed. outputs

43. The Normal Q-Q Plots shows the observed value for each score is plotted against the expected value from the normal distribution. A reasonably straight line suggests a normal distribution. The Detrended Normal Q-Q Plots displayed in the output are obtained by plotting the actual deviation of the scores from the straight line. There should be no real clustering of points, with most collecting around the zero line. outputs

44. EXAMPLE OF WRITE-UP Ho1: There is no significant difference in the mean overall test performance in the learning of the Straight Lines topic between the graphing calculator (GC) strategy group and the conventional instruction (CI) strategy group. The means and standard deviations of the overall test performance for both the GC and the CI strategy groups are provided in Table 4.4. The overall performance test ranged between 0 and 40. Mean overall test performance of the GC strategy group was 16.81 (SD=4.76) while mean overall test performance of the CI strategy group was 12.53 (SD=4.99). An independent t-test analysis showed that the difference in the means were significant, t(38)=2.78, p<.05. The results indicated that there was a significant difference in the mean overall test performance in the learning of the Straight Lines topic between the GC strategy group and the CI strategy group. The magnitude of the differences in the means was considered large based on Cohen (1988) with eta squared =.17. The guidelines proposed by Cohen (1988) for interpreting this value are: .01 = small effect, .06 = moderate effect, .14 = large effect. This finding indicated that the GC strategy group had performed significantly better than the CI strategy group.

45. EXAMPLE WRITE-UP Table 4.3: Independent samples t-test to compare means monthly test performance before experiment in Phase I Table 4.3 shows the results of the analysis. The total monthly test performance was 100. The mean performance for the GC strategy group and the CI strategy group were 59.00 and 59.26 respectively. Levene’s test indicated that the assumption for equal variance has been violated, F=9.95, p<.05. Therefore the reading for the output for the independent t-test was based on the reading for equal variance not assumed. The results of the t-test showed that there was statistically no significant difference between the mean monthly test performance for the GC strategy group and the CI strategy group, t(25.41)=−.049, p>.05. This suggested that the students’ mathematics performance for both groups in the group did not differ significantly.

46. H01: There is no significant difference in the mean overall performance in the learning of statistic between the PBL-Tr, PBL-Web and Conv groups. The means and standard deviations of the overall performance for the PBL-Tr, PBL-Web and the Conv strategy groups and also results of the ANOVA test. The overall performance score ranged between 0 and 100. Mean overall performance of the PBL-Tr group was 70.08 (SD=12.08) while mean overall performance of the PBL-Web group was 80.18 and (SD=17.2) and the mean of overall performance of the Conv strategy group was 56.6 (SD=20.38). An ANOVA test analysis showed that the difference in the means were significant, F(2,61)=6.35, p<.05. The results indicated that there was significant difference in the mean overall performance in the learning of statistics between the three groups. The magnitude of the differences in the means was considered large based on Cohen (1988) with eta squared (ES) = 0.172. The guidelines proposed by Cohen (1988) for interpreting this value are: .01 = small effect, .06 = moderate effect, .14 = large effect. Based on Post-Hoc test the mean of overall performance for Conv group was significantly lower than PBL-Tr and PBL-Web groups. However, PBL-Tr did differ significantly from PBL-Web group at 5% level of significance.