Key Findings of the Macao-China PISA 2006 Scientific Literacy Study

1. Key Findings of the Macao-China PISA 2006 Scientific Literacy Study Sit P.S. & Cheung K.C. University of Macau Macao, China

2. Paper presented at“PISA 2006: The Performance of Educational Systems in Countries and Regions” 3-4 April 2008 Trento, Italy

3. Macao, special administrative region of People’s Republic of China, participated in OECD’s Program for International Student Assessment (PISA) for the first time in 2003. • After 3 years, Macao participated again in 2006. • Macao will participate for the third time in 2009.

4. Part one of presentation:Key findings of the scientific, mathematical and reading literacy of Macao’s 15-year-old students in the PISA 2006 Study

5. Three main kinds of literacy are assessed, namely, reading, mathematical and scientific literacy. The target students assessed are all secondary students who are aged between 15 years three months and 16 years two months at the time of assessment. Most students are studying in the middle grade levels (i.e. grade 8, 9 and 10), whereas some students are studying in the lower or higher grade levels (i.e. grade 7 and 11).

6. Policy implications • Principals claim that they are successful in providing compensatory education. There are needs to examine the extent of grade repetition in the various school types so as to assess its effects on literacy attained at the end of compulsory education. • What if PISA results show that repeaters score higher in terms of literacy scores when compared with countries with similar grade repetition problems. Should they be repeated if they study in other countries?

7. Table 1A: Characteristics of schools in the PISA 2006 Macao sample

8. Table 1B: Characteristics of schools in the PISA 2006 Macao sample

9. Table 1C: Characteristics of schools in the PISA 2006 Macao sample

10. Policy implications of the three stratifying variables • Government schools are minority (only 2 schools in the 15-year-old sample) and they are generally affluent in terms of educational resources. However, they are not first choice for parents to send their children for basic education. Explanation of school type effect is not a simple straight-forward problem.

11. Policy implications of the three stratifying variables • In the light of literacy valued in postmodern society as assessed in PISA, differences in course provision between grammar and prevocational schools for compulsory education need revision.

12. Policy implications of the three stratifying variables • Language of instruction is at issue when literacy is assessed. This is complicated by sampled students as they are predominantly ethnic Chinese, and English schools are valued by parents of higher ESCS. Special care is needed when literacy is compared using different groups of students (i.e. home language, test language and language of instruction issue).

13. Table 2: Number of 15-year-olds sampled and tested in Macao

14. Table 3: Grade distribution of 15-year-olds tested

15. Table 4: Macao 15-year-olds’ literacy performance results

16. Table 5: Distribution of Macao 15-year-olds’ proficiency levels on the literacy scales -- Total sample

17. Table 6: Distribution of Macao 15-year-olds’ proficiency levels on the literacy scales - Males

18. Table 7 : Distribution of Macao 15-year-olds’ proficiency levels on the literacy scales - Females

19. Amongst the 57 participating countries/economies, Macao’s scientific literacy performance was statistically significantly above the OECD average, and Macao ranked between 15 and 20 on the combined science scale. In decreasing order of the mean of scientific literacy score, countries/economies statistically significantly higher than Macao were: Finland, Hong Kong, Canada, Chinese Taipei, Estonia, Japan, New Zealand, Australia, Netherlands, Liechtenstein, Korea and Slovenia. Science Literacy Results – from the International Perspective

20. There are six scientific proficiency levels in the combined science scale. Only 1.4% of the 15-year-olds performed below level 1 of the scientific literacy scale. Only 0.3% of the 15-year-olds performed at this highest level 6 on the combined science scale. Science Literacy Results Explained

21. Figure 1:Percentage of 15-year-olds at different scientific literacy proficiency levels across grades in the Macao sample

22. Figure 2 : Percentage of 15-year-olds at different grade levels across scientific literacy proficiency levels in the Macao sample

23. Figure 3: Literacy performance results in the PISA 2006 Study The sampled school Higher than Macao sample The Sampled School Macao Sample Macao sample Higher than the sampled school The sampled school Comparable Macao sample The Sampled School The Sampled School

24. Low-performing and high-performing schools for Macao • When comparing the literacy performance across schools, it is important to note that the literacy assessed referred to the cumulative educational effects of all schools that the students have previously attended. • Therefore, a low-performing school identified in the PISA Study may not be a poor school. Low-performing students dropped out from one school may be subsequently enrolled in another school and thereby lowering the school’s literacy performance level. • Different schools serve different communities and they have different visions and missions.

25. Figure 4: Literacy performance results in the PISA 2006 Study, broken down by Gender Females outperform males, 14.3% of co-edu school The Sampled School Females The Sampled School Males The Sampled School Females The Sampled School Males Macao Sample Females Gender difference pattern similar to Macao sample, 57.1% of co-edu school The Sampled School Females Males outperform females (except reading literacy), 25.7% of co-edu school The Sampled School Males Macao Sample Males

26. Macao 15-year-olds performance in the three key scientific domain areas Amongst the three content areas of science, Macao’s 15-year-olds performed: • Best in “living systems” • Moderately well in “physical systems” • Less well in “earth and space systems”

27. Macao 15-year-olds performance in knowledge of science vs knowledge about science Knowledge of science Better Than Knowledge about science

28. Amongst the 57 participating countries/economies, Macao’s mathematical literacy performance was statistically significantly above the OECD average, and Macao ranked between 7 and 11 on the mathematics scale. In decreasing order of the mean of mathematical literacy score, countries/economies statistically significantly higher than Macao are: Chinese Taipei, Finland, Hong Kongand Korea. Mathematics Literacy Results – from the International Perspective

29. Reading Literacy Results – from the International Perspective • Amongst the 57 participating countries/economies, Macao’s reading literacy performance was not statistically significantly different from the OECD average, and Macao ranked between 18 and 22 on the reading scale. • In decreasing order of the mean of reading literacy score, countries/economies statistically significantly higher than Macao are: Korea, Finland, Hong Kong, Canada, New Zealand, Ireland, Australia, Liechtenstein, Poland, Sweden, Netherlands, Belgium, Estonia andSwitzerland.

30. Table 8: Macao 15-year-olds performance in science, mathematics and reading in 2003 and 2006 # Number of countries/economies participating in 2003 is 40, whereas this number increased to 57 in 2006. * Major assessment domain on which assessment trend is to be charted on a three-yearly cycle

31. Table 9: Macao 15-year-olds’ scientific attitude results

32. Table 10: Pearson correlation of quality science education indicators with scientific literacy performance scores

33. The ten science quality education indicators (focusing on engagement in science) 1. General interest in science 2. General value of science 3. Self-efficacy in science 4. Self-concept in science 5. Enjoyment of science 6. Instrumental motivation to learn science 7. Awareness of environmental issues 8. Concern for environmental issues 9. Optimism regarding environmental issues 10. Science-related activities

34. Table 11: Selected quality science education indicators for the Macao sample

35. Part two of presentation:Relationships of literacy measures with ESCS (Economic, Social and Cultural Status of the home)

36. Figure 5: Plot of literacy performance with ESCS in the Macao Sample Mathematical Literacy Scientific Literacy Reading Literacy

37. Figure 6: Plot of scientific literacy subscale performance with ESCS in the Macao Sample Identifying Scientific Issues Explaining Phenomena Scientifically Using Scientific Evidence

38. Figure 7: Plot between-school scientific literacy performance with ESCS High-performing school Average-performing school Low-performing school

39. Educational provisions: “Excellence” versus “Equity” • Amongst the 57 participating countries/economies, percentage of variance in performance in science explained by the PISA index of economic, social and cultural status of the home (ESCS) is the lowest. • The overall effect and between-school effect of ESCS on scientific literacy are also amongstthe lowest of all participating countries/economies.

40. Educational provisions: “Excellence” versus “Equity” • Hence, Macao’s basic educational system succeeds in providing equitable schooling opportunities for the student body it served. • The results indicate that Macao educational system not only achieves relatively high academic excellence, but also favorable equity in the distribution of educational opportunities.

41. Issues of equity – An insider’s view • Children of higher ESCS not all can go to the high-performing schools, resulting in that there is a distribution of ESCS amongst the low-performing schools • Children of lower ESCS still have a relatively lower chance to study in the high-performing school

42. Part three of presentation:International comparison of literacy performance

43. Table 12: Top five high-performing East Asian countries/economies

44. Table 13: Non-East Asian countries/economies having higher or comparable literacy performance than Macao 15 8 17

45. Part four of presentation:Follow-up Studies

46. Follow-up Study 1 • By making full use of the student, school and system data, examine factors affecting excellence and equity in literacy performance in the three key domains for 15-year-olds in Macao. • Conditions, processes and contexts facilitative of literacy acquisition for the different types of schools and different kinds of students (e.g. students studying in single-sex versus co-educational schools) should be examined.

47. Follow-up Study 2 • By linking the literacy performance results with previous PISA 2003 Study data, chart the change in literacy performance in the three key domains. • Factors affecting growth and decline of literacy performance should be identified so as to guide educational improvement and curriculum reform.

48. Follow-up Study 3 • By comparing selected high performing East Asian with western countries/ economies, attempt to understand the similarities and differences pertaining to excellence in literacy performance for 15-year-olds in the three key domains.

49. Follow-up Study 3 • East Asian countries/economies such as Chinese Taipei, Hong Kong, Japan, and Koreamay be compared with western countries such as Australia, Estonia, Finland, Canada, Ireland, Liechtenstein, New Zealand and Netherlands. • International comparisons can uncover the secrets of academic excellence of 15-year-olds students of the top five high-performing East Asian countries/economies.

50. Concluding Remark 1 • Although the impact of ESCS on mathematical, scientific and reading literacy are not substantial, elevating homes of low ESCS to higher levels is still needed. • This is expected not only to bring about better educational opportunities and equitable educational results, but also can enhance the mathematical, scientific and reading literacy proficiency levels of the 15-year-olds in Macao.