Assessing adult literacy The aim, use and benefits of standardized screening tools

1. Assessing adult literacy The aim, use and benefits of standardized screening tools

2. "You teach a child to read, and he or her will be able to pass a literacy test."

3. Presentation • Research questions • Tools and methods • Why screen? Why not? • What to expect from screening?

4. Research questions Screening tools • What is the aim, use and benefits of screening tools in the process of identifying low literate adults? Standardized screening tools • Practical usability • Broad and standardized screening instrument desirable?

5. Policy on literacy Flemish government • Increase of literacy in the Flemish population • Policy documents • Strategic plan on “Increasing literacy” • Operational plan on “Increasing literacy” • Objective: phased and systematic screening of literacy among adults

6. Tools and methods Three qualitative research methods • A literature survey of literacy skills and the screening of those skills • A document analysis (qualitative content analysis) of 31 existing screening devices • Semi-structured in-depth interviews with 33 key figures from Flanders and The Netherlands • Low literate individuals and representatives • Screening professionals • Policymakers • Academic experts • Professionals working in secondary education

7. What is screening? • Literacy screening implies … • … based on behaviour or performance which may or may not be induced • … the literacy skills of an individual or group • … are assessed and evaluated • … using a benchmark or norm • (in a short protocol)

8. Screening: how? Five types of screening instruments • Test – measuring an induced literacy performance using a device developed beforehand (which may or may not be standardized) • Proxy measurement - mapping out factors that show high correlation with low literacy • Self-assessment – making an estimate of one’s own literacy performance level on the basis of structured questions • Interview / discussion – oral questioning of the extent of literacy on the basis of a questionnaire • Observation – consciously observing behaviour with a view to describing and estimating literacy skills

9. Screening: why (not)? To strengthen adult literacy, the first step is theindentification of low-literacy (on an individual level). Still… not al social domains (civil society organisations, health care, work place organisations, …) seem equally open to assessment by means of a standardized screening tool. Reasons • The interpretation and operationalisation of literacy itself • The adequate functionality norm for literacy (and its effects) • The very limited diagnostic information (and what to do with it?)

10. Multi-literacies Multiple viewpoints • Level of literacy performance (“the degree of literacy” of an individual) • = vertical dimension • The range of contexts and situations in which an individual can function using written language • = … horizontal dimension • Different user perspectives (micro, meso, macro) • Different sorts of literacy, different sorts of text (prose literacy, document literacy, numeracy, digital literacy, …)

11. Multi-literacies

12. Multi-literacies “Literacy as a cover term is so broad it must almost bedefined for each occasion on which it is used.” (Kintgen et al, 1988) “(…) there are no clean cut logical or empirical criteria that can help settle disputes about what functional literacy is or is not. These facts should not be taken as a sign of inadequacy of the definition. The definiendum itself is a fuzzy reality and it should not be presented as if it were not.” (De Glopper, 1992)

13. Multi-literacies Consequences for screening • It seems impossible to screen all aspects of literacy… • together • to the same extent • in the timeframe of a quick screening protocol • Content analysis of existing screening devices: no single instrument focuses on literacy as a whole… they test separate skills (writing, listening, …) or one sort of literacy (often prose and document literacy)

14. The cut-off point (1) The cut-off / norm? • When is a person or group functionally literate? • Where is the boundary between having and not having adequate literacy skills (the ‘at risk’ line) Should we use one? • From a theoretical point of view: • Yes – how can we evaluate literacy skills when there is no norm? It is necessary to identify the ‘at risk’ group opposed to a ‘not at risk’ group • But no – it can never do justice to the complexity of literacy • One or more? • One - for the totality of the population • More than one – for each subgroup or population category

15. The cut-off point (2) The cut-off / norm? • Who should define or specify the cut-off criteria? • Intense societal or public debate • International comparison • … • or the screened individual? • Practical consequences • Too high – a large population “at risk” … targeted policy is impossible … over-problematisation / people talked into believing they have a problem • Too low … a wrongful acceptance of a lack of minimum skills ! Screened individual has to “accept” the cut-off point…

16. Screening: information What information to expect from screening? Realism has to rule • Only a cursory and generalised portrait of performance…(illusory effect) • Offers no evidence or knowledge of the interplay between literacy skills and experiences, no diagnostic information But… • … this makes explicit what otherwise would stay unnoticed or intuitive Important • For most interviewees, screening is just a first step that should not take place without a possible… • Diagnosis • Follow-up • Training

17. Screening: expectations Because research data warn for unrealistic expectations… Any (new) screening instrument should Critical success factors • … exhibit several essential features = demonstrate adequate levels of validity, reliability = it should be neutral, fair, efficient • … should be guided by the particularities of the target group • A well-defined target group is necessary • A group with a uniform needs profile • A group that can be reached for screening (for instance: schools, providers of vocational training courses, …)

18. The use of a screening tool Perhaps the most important element: not the screening tool itself but the way it is used… screening should be • Be part of a formative process • Screening results should motivate the screened individual to embark on an educational process • Start from the needs of the person • Aim: strengthen adult literacy … Not simply counting heads or labelling people ‘at risk’

19. Assessing adult literacy The aim, use and benefits of standardized screening tools Queestions, comments, feedbag, … feedback

20. Download the research report (in Dutch) • www.hiva.be • www.cteno.be

22. Document analysis