ImpaCT 2 & the continuing quest to understand ICT & Attainment

1. ImpaCT 2 & the continuing quest to understand ICT & Attainment Malcolm Hunt, Becta MIICE Conference, November 04

2. Presentation Outline • Key findings from ImpaCT 2 • Post ImpaCT 2 work about ICT & Attainment • Context issues, including ICT Pedagogy • Summary & Conclusions

3. Key findings from ImpaCT 2

4. Pre ImpaCT 2 thinking RESEARCH ISSUES: • Difficult to isolate any single aspect such as ICT from other factors. • Factors are likely to be at school, organisational and pupil levels. • All pupils have access to ICT so control group not an option • Using a large enough sample for statistical analysis, for a representative study • Robust/ cumulative qualitative evidence as well to understand practice

5. ImpaCT 2 Overview • Longitudinal Study over 3 years between 1999 and 2002. • Explored the relationship between networked technologies and attainment • Sample of 2179 pupils from 60 schools:- 30 Primary Schools- 5 Special Schools &- 25 Secondary Schools in England

6. Measures of ICT Use:Levels of Use: from questionnairesTypes of Use: from questionnaires, pupil/teacher logs & visits by researchers Sample:700 pupils at KS 2, 3 and 4 ImpaCT 2 Strands & Design • Strand 1: Considered the relationship between pupils’ use of ICT & their performance in National Tests & GCSEs as a measures of attainment. • Strand 2: Developed and applied a variety of methods to establish how pupils use ICT, in particular out of school. • Strand 3: Explored the nature of teaching and learning involving ICT in various settings Methods used :- Pupil researcher logs- Questionnaire on internet use- Special report of a key learning event with ICT-Concept mapping task 15 Cases studies with a focus on the views of pupils, teachers and parents.

7. How did the researchers find out how ICT affected attainment? • 700 pupils for each key stage • Value-added data compared for ‘high ICT’ and ‘low ICT’ pupils • 13 comparisons – English, mathematics and science at KS2, 3 and 4 and –KS4 history, geography, MFL & D&T.

8. Establishing how ICT affected attainment Positive relative gain Pupil A Expected score Negative relative gain Positive relative gain Pupil B Expected score Negative relative gain Baseline score National Test/ GCSE score Pupil A has a higher test score than Pupil B, but performed worse than expected, and so has a negative relative gain score. Pupil B, by contrast, performed better than expected and so has a positive relative gain score.

9. Key Findings from Strand 1 • In 12 out of the 13 comparisons, ‘high ICT’ pupils achieved on average higher value added than ‘low ICT’ pupils. • Of the 12 comparisons 4 were statistically significant:- English at KS 2- Science at KS 3- Science at GCSE & D&T at GCSE • In some cases the difference in average performance was equivalent to 0.2 of a National Curriculum level at KS 2/3 and 0.5 of a GCSE grade at KS4. One level relates to around 2 years in a pupil’s development – ie they are expected to progress by the order of 0.5 of a level a year

10. ImpaCT 2- Getting behind the data • Researcher judged the amount of ICT use overall to be low. • The number of networked computers rose dramatically during the period from 1999-2001. • Types of ICT use as well as the amount of ICT use play a part in terms of impacts on attainment/learning • A range of other context factors play a part

11. Increasing ICT use Getting Behind the DataEnglish KS 2 - School by school comparisons • Exceptionally good teaching with less use of ICT? • The use of a particular intervention other than ICT? High Low Mean relative gain for schools in order of ICT usage for English

12. Increasing ICT use Getting Behind the DataEnglish KS 2: School by schoolcomparisons • School T uses ICT more often than might be expected in English from the overall level of ICT use, yet English attainment is below the average • Maybe the totality of a pupil’s experience influences achievement, rather then an isolated use in a particular subject • Similar level of ICT use, very different relative gain score High Low Mean relative gain for schools in order of ICT usage for English

13. KS 2 English teachers identified: • The motivating effect of ICT was a common factor:- Captivated pupils who previously found it hard to engage- Positive shift in attitude involvement in learning activities • Types of use included word processing, publishing packages, use of the internet for research, software to aid spelling • Higher quality outputs encouraged greater commitment to writing tasks • Relevant software making the learning of key skills more fun (eg learning spellings) • Increased time for reflection/ refining the content of written work

14. Maddux model of Stages in ICT research Type & nature of the Research Questions • Need to move on from whether ICT has general educational benefit to how and why ICT works • Need to refine our research questions and consider subsets of ICT- By subject- By ICT application eg Email, use of spreadsheets • Focus more on the context of use STAGE 1: Computers produce general educational benefits STAGE 2: Exposure to a specific application produces a specific educational gain STAGE 3: Which and how learners interact with teaching variables when using specific aspects of ICT

15. Asking the ‘how’ & ‘why’ questions about ICT & Attainment

16. ICT & Attainment Literature Review (Cox et al 2003) • Positive effects of specific uses of ICT on pupil attainment in almost all NC subjects • Most substantial in Maths, Science and English, where more uses being made of different resources in these subjects • In Maths and Science learning is most enhanced when the type of ICT use and the measures are closely related to specific skills and concepts.

17. English Word processing and presentations Creativity and developing meaning Maths Programming and modelling Understanding and applying concepts Science Simulations and modelling Scientific reasoning and explanations Geography Simulations andmodelling Concepts and relationships History Multimedia resources Understanding and historical reasoning MFL Simulations and role playing Specific language skills, confidence ‘Deeper’ uses of ICT (Cox et al 2003) Specific Uses Subject Impact

18. ICT & Attainment Literature Review • Impact appears strongest for ICT resources which have been embedded in teachers’ practices for a long time. • Emerging body of evidence about specific ICT communications eg email, WWW, but the effects on attainment is not as consistent or extensive. • The crucial component is the teacher and their pedagogical approaches

19. Factors in ICT Pedagogy & Reasoning (Cox et al 2003) Confidence in using ICT Access to ICT Resources ICT Pedagogy and Reasoning Organisation & classroom practice Teachers subject knowledge Teachers pedagogical knowledge, beliefs & values Planning & preparation beyond the classroom

20. Factors that affect teacher uptake & use of ICT (Mumtaz 2000) • Institutional:eg: Time, technical support, ICT resourcing • Resources:eg: Access to computers in the classroomRange of software availableLack of training and the quality of training • Teacher factors:eg: ICT experience, confidence/competence,Beliefs & theories about subject teaching,Knowledge & perceptions about the benefits of ICTManaging classroom activities/control Personal feelings towards ICTFear of change • Main Barriers • ICT Confidence/ competence • Access to ICT • Training • Time • Technical problems/support

21. Based on 172 practitioners surveyed at BETT 03 and the Education Show Broad agreement with the literature survey

22. Lack of tech. support Access at school Technical Problems Lack of personal access Access at home LACK OF TEACHERCONFIDENCE Fear of things going wrong Lack of Time Lack of teacher competence Lack of ICT Skills Training Lack of Pedagogical Training Lack of exploration Teacher Confidence Direction of arrows represent “could be caused by…” Red arrows show a two way effect. Becta ICT Barriers reporthttp://www.becta.org.uk/research/research.cfm?section=1&id=3310

23. Personal ICT Enablers • Personal access to ICT to allow the development of confidence and competence • Commitment and support from departmental and senior management • Involving whole staff in ICT decision making • Importance and involvement of a support community of peers • Perceived usefulness of ICT and ease of access and use Becta ICT Enablers reporthttp://www.becta.org.uk/research/research.cfm?section=1&id=3310

24. ICT Standards report- • Studied results of OFSTED inspections & QCA data on performance in national tests • ‘ICT enablers’ that contribute to good ICT learning opportunities were:- ICT resourcing - ICT leadership & management - ICT teaching - General School Leadership- General Quality of Teaching Becta’s research on ICT and Standards Is there a ‘threshold’ effect?

25. ICT enablers and ICT learning opportunities (primary) %

26. ICT enablers and ICT learning opportunities (secondary) %

27. A developing model: ICT enablers Improved Outcomes The ‘building blocks’ of enablers that can help a school to make effective use of ICT. ICT used effectively in classrooms for learning Improved learning Increased attainment ICT deployed appropriately ICT learning opportunities NCSL/Becta SLICT courses Leadership ICT leadership ICT resources General teaching ICT teaching NOF Training,HOS initiative,CPD support materials • The ICT in schools programme • CfT and LfT

28. ImpaCT 2: Strand 2 research: Somekh et al 2002 Pupils’ & Teachers’ Perceptions of ICT in the Home Stages in networked ICT implementation TIMESCALE? • Stage 1: Provision of Equipment, infrastructure and support • Stage 2: Teaching ICT skills, often in specialist ICT lessons, • Stage 3: Integration of ICT into the curriculum subjects, including literacy and numeracy During data collection, schools moved from Stage 1 to 2 Only a few teachers in a few schools began to move to Stage 3

29. Home / School Issues • The importance of ICT use in the home and for informal learning is increasing (ImpaCT 2 figs)- Primary School pupils: on average 3 hours a week compared to 1 hour at school- Secondary School pupils: on average 10hours a week compared to 2.5 hours at school (4x) • Home ownership and internet access is increasing yet over a quarter of pupils did not have internet access at home. Potential disadvantage? • Pupils have an extensive awareness ofthe role of computers in today’s world (Type of equipment, use & purpose) ImpaCT 2 Findings Teachers recommended website for use at home & in support of subject specific learning Public access to ICT in public libraries & internet cafés can lack flexibility & be expensiveHow might schools address this? Students of all ages may develop ICT skills more quickly and easily than schools anticipate

30. Emerging Findings from Becta’s Research about ICT & Attainment • Growing body of evidence of an ICT effecteg ImpaCT 2 study, SANDS Report • Beginning to appreciating the use of ICT in specific subjects- Subject ICT traditions are importanteg ICT & Attainment literature reviews • Identifying important elements in the effective embedding of ICT- Institutional conditions are criticaleg Barriers and Enablers research, ICT and Pedagogy research

31. Conclusions • ICT and Standards: ensuring that we can go beyond the research questions as to ‘whether’ ICT has an impact, to ‘how’ and ‘why’ it does. • ICT pedagogy: ‘moving on from the ‘foothills of pedagogy’’ - better understanding ICT use in different subjects and with different technologies- better understanding mixed approaches blending ICT & traditional methods, including Personalised Learning • The embedding of ICT- understanding teacher engagement, CPD needs- for Institutional Development & the E Confident school

32. Contact details Becta Millburn Hill Road Science Park Coventry CV4 7JJ 02476 41 6994 Becta Research web site:http://www.becta.org.uk/research Malcolm Hunt: Assistant Director, Evidence and Evaluation,malcolm.hunt@becta.org.uk

33. ImpaCT 2 & the continuing quest to understand ICT & Attainment Malcolm Hunt, Becta MIICE Conference, November 04

34. Sources:ImpaCT 2: http://www.becta.org.uk/research/research.cfm?section=1&id=539ICT and Attainment Report: http://www.becta.org.uk/research/research.cfm?section=1&id=3119ICT and School Standards: http://www.becta.org.uk/research/research.cfm?section=1&id=538Barriers & Enablers Reports: http://www.becta.org.uk/research/research.cfm?section=1&id=3310