The Impact of NCEA on Student Learning Motivation

1. The Impact of NCEA on Student Learning & Motivation Luanna Meyer Professor of Education (Research) Victoria University of Wellington 4 July 2006

2. This research was supported in part by Contract #397-2148 awarded to VUW from the Ministry of Education Research Team at Victoria University: Luanna Meyer, College of Education John McClure, School of Psychology Frank Walkey, School of Psychology Lynanne McKenzie, College of Education Kirsty Weir, School of Psychology

3. The Impact of NCEA on Student Learning & Motivation Multi-method: Quantitative & Qualitative Student survey, year 10 and years 11-13: item ratings, individual item responses, open-ended questions Student achievement data (years 11, 12 and 13) Student focus groups (year 10 and years 12-13) Student individual interviews (year 11) Parent focus groups Teacher focus groups

4. Major Focus: Student Voice Purposive sample from 20 secondary schools: 15 North Island and 5 South Island 8 Auckland region; 11 urban Auckland, Christchurch, or Wellington; 9 in smaller cities, towns, or rural areas—both islands, Northland and the East Coast Size mix, largest to smallest Diverse student population, 2 Maori immersion 17 coed state, 1 integrated state, 1 boys, 1 girls School decile levels: 5 decile 1-2 schools; 10 decile 3-8 schools; 5 decile 9-10 schools

5. Major Research Questions: What is the connection between student choice and student learning? 2. What is the impact of NCEA design features on student enjoyment and motivation? 3. What is the relationship between student motivation and student achievement?

6. Theory & Research Evidence on Motivation Orientations to Learning If a teacher sees a student struggling with a new task, what should he/she say? Why? (a) Keep trying, it’s easy! (b) That’s very hard—you’ll have to work at it, and I’ll check later to see if you need more help.

7. Theory & Research Evidence on Motivation Orientations to Learning Views of Intelligence: Static: Ability or an individual characteristic that cannot be changed

8. The impact of motivation orientations on learning and achievement: Task values that motivate achievement:

9. Survey of Students in Year 10 and in Years 11-13 Demographic questions: Gender, year in school, student status (international or domestic) and whether the student worked part-time Level of NCEA the student expected to complete before leaving school: Levels 1, 2 and/or 3 Future plans: Three things the student is most likely to do after leaving school, chosen from 12 possibilities including further study, gap year, employment, going overseas, doing sports, just hanging out, starting a family etc

10. The Student Survey Self-rating sections of the survey: Influence on choice of subjects How students think about their learning What students like/don’t like about NCEA, and other assessments Open-ended: 3 things you like and 3 things you do not like about NCEA and the Record of Learning

11. Data from the Record of Learning (Years 11-13) Total credits achieved (AS and US) Total Achievement Standards attained, then by AME Total Unit Standards attained Achievement Standard “grade average” The “grade average” on the Record of Learning (1-100 range) All NCEA qualifications to date (Levels 1, 2, 3) For year 13s, UE achieved: yes or no Ethnicity and fee rebate status Note: Several other variables were also entered, including SNAs and any additional national qualifications attained

12. Survey Participants Year 10: 2,083 respondents 1,976 domestic and 79 international students 29% reported being employed part-time

23. Survey Results: Influences on Subject Choices Utility or Importance: Subject needed to achieve a future goal (e.g., for a career or for University Entrance) External: Subject taken for peripheral reasons (e.g., taken by my friends, it’s easy, it fitted my timetable) Interest: Subject matches personal interest and enjoyment (e.g., I enjoy the subject)

24. Sample Items for Influences on Subject Choice: Utility/Importance: The subject gets me the number of credits I need It is related to what I might study at tertiary in future I need it for University Entrance It is related to a future job or career goal Interest: I’m interested in the subject I enjoy the subject I’m very good at the subject

25. Influences on Subject Choice External Factors: My friends are taking it I like the teacher who teaches the subject Because the subject is assessed by assignments and not final exams The subject is easy It fits my timetable

27. Relationships of Choice Influences to Achievement: Both the Utility/Importance and Interest motives relate positively to students’ GPA and attainment of Merit and Excellence

28. Influences on Student Subject Choices One might have predicted that students accumulating primarily Unit Standards were motivated primarily by Utility/Importance, but instead they appear to be making choices based on External Factors…. What might this mean? How can we encourage students enrolling in Unit Standards to make better decisions?

29. Gender Differences Girls were more likely than boys to attribute subject choices to the Utility/Importance and Interest motives There were however no differences by gender for self-ratings of External influences By Years 12-13, boys shift from selecting subjects based on External influences to basing decisions on Utility/Importance This could be because boys shift to working harder between years 11 and 12-13, or because those who are high on External motivation have left school by year 12-13….

30. How Students Think about Their Learning Doing My Best: Valuing work to attain merit or excellence and aiming to get a good education, not just accumulate credits Doing Just Enough: Doing only what is required to attain needed credits or to get by, influenced more by what friends think and whether study interferes with other things

31. Sample Items for How Students Think about their Learning: Doing My Best: I will strive for Merit or Excellence even when I don’t need this to achieve my goals I aim at getting a good education, not just completing tasks to get credits in NCEA I want credits from school that lead to a good job or career Doing Just Enough: Once I have my 80 credits, I’ll be satisfied What my friends think influences whether I work in school For me, getting achieved will be good enough

32. Relationship between Achievement and How Students Think about their Learning Doing My Best correlates positively with GPA scores, getting Merit and Excellence, and the number of total credits attained. Doing My Best correlates positively to achievement standard credits and negatively to unit standard credits Doing Just Enough is related to lower achievement and more unit standard credits

33. What Students Like/Don’t Like about NCEA and Other Assessments Work Avoidance: Views that favour easy options and doing only the minimum Getting Feedback: Wanting more feedback on one’s work and information throughout a course not just at the end The Excellence Factor: Working to maximise one’s achievement and wanting to demonstrate high achievement

34. Sample Items for What Students Like/Don’t Like about NCEA and the Record of Learning Work Avoidance: Being able to relax after I get my 80 credits Not having to do parts of a course that I don’t like when I don’t need those credits Getting Feedback: Taking subjects where the teacher assesses my work during the course rather than only through a final exam Getting feedback on my work Excellence: Being able to seek more than the minimum credits whenever I wish Having a final end-of-course external exam with a grade scaled so I can compare myself with others

37. Gender Differences? Girls scored higher than boys for the Doing My Best and Getting Feedback factors However: There were no gender differences on Doing Just Enough, Work Avoidance, and Excellence orientations

38. Relationship between Achievement and What Students Like/Don’t Like about NCEA and Assessment: Work Avoidance correlates negatively to students’ grade averages and total credits gained, as well as to Merit and Excellence Excellence factor related positively to all measures of student achievement Getting Feedback correlates to the number of Achieved grades. A diverse group of students valued Getting Feedback—across high and low achieving—but students from low decile schools were most interested in this factor

39. Relationships Across Factors and Predicting Achievement Students motivated by Doing My Best were most likely to select subjects based on Utility/Importance and less likely to report External influences Students motivated by Doing Just Enough were most likely to select subjects based on External influences unrelated to the subject.

40. Is Doing Just Enough Good Enough? If students who are “Doing Just Enough” are acquiring fewer credits—which they are—will a goal of just enough end up being enough? This motivation orientation assumes that students know and can work towards attaining exactly what they need—no more, no less….

41. Is “Doing Just Enough” Good Enough? Unless this motivation orientation is challenged, are we allowing students to settle for less and sell themselves short?

42. Influences on School Performance Students’ performance in school will be influenced by their beliefs about their own abilities (static or incremental) and personal characteristics (motivation orientation, perseverance, etc.) These characteristics can be related to and influenced by gender, culture, family circumstances and economic resources…. But both can also be influenced by: what the school and teachers do, the curriculum, and the teaching and learning process—these are the things we can change!

43. Designing Effective Interventions: We found strong relationships between school performance and motivation orientations reported by the students—consistent with the international research literature.

44. Key Findings from Qualitative Analyses of Focus Groups and Interviews Parents indicated strong support for NCEA in principle while suggesting some areas for improvement Students, teachers, and parents are overwhelmingly positive about the impact of internal assessment on student study/learning behaviours and achievement Students continue to perceive some uneven opportunities across schools for access to more subject choices and study across levels of NCEA Students and teachers perceive inconsistencies in the difficulty of some standards across some subjects There were frequent comments on getting a Not Achieved despite answering Merit and/or Excellence questions—most had never experienced this, but had heard about this anomaly and felt it was illogical and unfair

45. Qualifications Design Issues Areas of Strength: Students and adults are extremely positive about the mix of internal and external assessments, valuing internal to guide learning and workload and valuing external to support perceptions of NCEA’s credibility and towards consistency across schools and subjects

46. Qualifications Design Issues Issues to consider: Some student comments suggest that the 80 credit requirement can become a maximum and encourage a minimalist approach to learning

47. Why Student Voice? Students’ motivational orientations can be changed: We need to consider the impact of interventions on not only immediate school performance but on student’s underlying dispositions to learning—if our goal is to support long term, meaningful student achievement outcomes controlled by the learner and not by others!