PISA International Conference

1. PISA International Conference

2. Reading Performance of Hong Kong’s15-Year-Old Students in PISA

3. Contents I. Design of Reading Assessment • The construct of reading literacy • Design of PISA assessment tasks II. The findings • The overall and specific performance of HK students on reading literacy • III. Some Observations

4. The Construct of Reading Literacy in PISA

5. The Construct of Reading Literacy in PISA “the ability to understand, use and reflect on written texts in order to achieve one’s goals, to develop one’s knowledge and potential, and to participate effectively in society” (OECD, 1999)

6. 3 dimensions of reading Process of Reading Content of Reading Context of Reading Reading Literacy in PISA

7. The Process - Five aspects of reading • Retrieving information • Broad understanding • Developing an interpretation • Reflecting on content • Reflecting on form Retrieving information Interpreting Reflecting and Evaluating content and form

8. The Content – 2 categories, 11 types Text Continuous Non-Continuous Descriptive Narrative Expositive Argumentative Injunctive Charts and Graphs Tables Diagrams Maps Forms Advertisements

9. The Context • Personal • Public • Occupational • Educational

10. The Construct of Reading Literacy for PISA Process (retrieving, interpreting, reflecting) Context (Personal, Public, Occupation, Education) Content (Format, Type)

11. The Design of PISA Literacy Assessment

12. The Hong Kong Sample • PISA sampling procedure closely followed • 140 schools • 4405 15-year-old students

13. The assessment materials • 37 stimulus texts • 141 reading tasks • 9 equal sets • Chinese version verified by PISA

14. Formats of Response • Multiple choice questions • Complex multiple-choice items • Closed-constructed responses • Short responses • Open-ended responses

15. Reporting of Reading Literacy Performance • Four different proficiency scales Retrieving information sub-scale Interpreting text sub-scale Reflecting on and evaluating text sub-scale Combined scale • These scores then mapped on to a five-level proficiency scale

16. Descriptors of skills demonstrated at each level of the three reading proficiency sub-scales (Appendix 3)

17. Table 5: Reading literacy score range of the reading proficiency levels

18. The testing material – A sample item Graffiti Question 1: The purpose of each of these letters is to: A explain what graffiti is. B present an opinion about graffiti. C demonstrate the popularity of graffiti. D tell people how much is spent removing graffiti. Answer: B Situation: Public Text format: Continuous Aspect: Interpreting texts Level: 2 PISA scale score: 421

19. HK Students’ Performance Overall

21. At Level 5 At Level 4 At Level 3 At Level 2 At Level 1 Below Level 1 Fig. 2 Percentage of students performing at each of the proficiency levels on the combined reading literacy scale

23. 700 650 600 550 Hong Kong 500 Combined reading score OECD average 450 400 350 300 0 10 20 30 40 50 60 70 80 90 100 Percentile Fig. 3 A comparison of students performance on the combined reading literacy scale

24. Descriptors of performance by students at Level 1 and Below Level 1 • Level 1: “capable of completing only the least complex reading tasks, such as locating a single piece of information, identifying the main theme of a text or making a simple connection with everyday knowledge.” • Below Level 1 : “not capable of the most basic type of reading that PISA seeks to measure……Such students have serious difficulties in using reading literacy as an effective tool to advance and extend their skills in other areas.”

26. Summary of HK students’ overall performance on reading literacy • HK has a relatively high performance (525, Proficiency Level 3, 6th) in reading literacy comparing with other countries • Comparing with other countries, HK has a relatively small percentage (10%) of very proficient readers and poor readers • The high rank of HK is largely due to the large percentage of students achieving Level 3 (33%) & 4 (31%)

27. HK Students’ Performance Specific

28. Table 7: Proportion of students at each proficiency level of the reading sub-scales Note: There may be rounding errors.

29. Summary of HK students’ performance on the reading sub-scales • HK students generally do better than OECD countries on the sub-scales • HK students maintain their good performance on “reflecting and evaluation” • HK high-achieving students gradually improve their performance from retrieving to evaluating, while low achievers showed a reversed pattern of results • HK students’ performances show a greater disparity across the sub-scales than that of the OECD countries

30. Table 8: Mean percentage scores across different text formats *2 items were deleted from the Hong Kong data set for subsequent analysis. This OECD average is calculated based on 127 corresponding items.

31. Table 9: Mean percentage scores across different text types

32. Table 10: Mean percentage scores across different contexts of use

33. Conclusion and Discussion

34. Comparatively speaking, HK lags behind in the number of very proficient readers. • HK students are relatively good at high order reading skills. • HK students are particularly good at argumentative text, but weak at advertisement. • HK students perform better on “educational”, “occupational” than on “personal” and “public” context of reading. • HK education is successful in producing a majority of students with an above-average reading literacy. • Are the results contradictory to our perception? • How good do we want our students to be?