PORTFOLIO COMMITTEE ON BASIC EDUCATIONADMINISTRATION OF ANA 201310 SEPTEMBER 2013
Presentation Outline • Introduction • Performance in ANA 2012/13 • Test Development • Learner registration • Printing, packing and distribution • Test administration • Marking • ANA Verification • Support to Learners • Conclusion
Introduction • Annual National Assessment (ANA) is in its third year of full implementation. • ANA provides credible and reliable information for monitoring progress in learner performance towards achieving the national goals of “improving the quality of basic education”. • In the last two years, ANA has provided districts, schools, teachers and parents, with useful diagnostic information on the areas of difficulty which has assisted in the structuring of targeted remedial programmes.
Introduction • Intended Outcomes:- • Serve as a diagnostic tool identifying areas of strength and weakness in teaching and learning. • Expose teachers to better assessment practices. • Provide districts with information to target schools in need of assistance. • Encourage schools to celebrate outstanding performance. • Empower parents by providing them information about the education of their children.
Introduction • ANA 2011: administered in February 2011 and tested learners on the work covered in the previous year. Administered in Grades 1 to 6. • ANA 2012: administered in September 2012 – tested learners on the work done in that year. Grade 9 learners were tested for the first time. Included learners from Grade 3 or 6 in independent schools. • ANA 2013: once again administered in September 2013 for learners in Grades 1 -6 and 9.
Introduction • The credibility of ANA is determined by: • - The assessment instrument • - The administration of the assessment • - The marking and moderation of assessment • - The analysis and feedback. • Parallel to the ANA administered by the school (Universal ANA), is the Verification ANA, administered, marked and processed by an independent agent.
Overall results in Mathematics (Average percentage mark) * Grade 9 was not part of the 2011 ANA
Overall results in Language: Grades 4-6 and 9 (Average percentage mark) * Grade 9 was not part of the 2011 ANA
Test Development All tests for grades 1 to 6 and 9 in Mathematics and Language have been set, moderated, edited, externally reviewed and quality assured. Educators (subject advisory and classroom based) were used for the development of tests. Each grade and subject comprised 2 examiners, a chief examiner, an editor and a moderator.
Test Development Tests were developed in English, based on the test framework and specifications, and then versioned into other languages. The 92 sets of developed tests were piloted in February 2013 in selected schools on samples of learners who were a grade higher than the grade of the test. Double the number of required test items were piloted and the best-functioning items were included in the final test, with some refinement where necessary.
Test Development All tests were moderated, quality-assured and edited before the pilot run and after the pilot study. The Advisory Committee comprising local and international expertise in national assessment, scrutinised the tests and provided inputs on the finalisation of the tests. In response to the concerns raised in 2012, the Grade 9 Mathematics test was particularly subjected to additional scrutiny and extra attention by the specialist members of the Advisory Committee.
Adaptations of Tests Tests were adapted for: - learners who are deaf (language simplification), - learners who are partially sighted (a range of large font sizes); and - learners who are blind (Braille versions).
Learner Registration • In most PEDs learner data uploaded from SASAMS. Mpumalanga and KZN captured afresh. E Cape uploaded 2012 data from GET system. W Cape uploaded from CEMIS. • Registration schedules printed and distributed to each school for verification. • Amendments were captured on the GET system. • Triangulation of GET data with SNAP 2013, workbook data and separate printing statistics collected from individual schools.
PRINTING, PACKING • AND DISTRIBUTION
Printing, Packing and Distribution • Printing, packing and distribution for five PEDs has been outsourced to a service provider appointed by DBE. • Four PEDs i.e. MP, WC, NC and KZN have opted to take responsibility for the printing, packing and distribution. • Distribution points have been established in each of the PEDs, from where tests will be collected by school principals.
Printing ,Packing and Distribution ANA Tests completed 30/06/13 ANA Tests printed 23/08/13 Packing of ANA Tests 30/08/13
Printing ,Packing and Distribution Provincial Warehouse 28/08/13 Distribution Point (220) 04/09/13 Schools 10/09/13
Test Administration • Administration categories: Universal ANA (U-ANA); Verification ANA (V-ANA); Independent Schools • Orientation for test administration conducted in all PEDs in July/August 2013. • Orientation sessions were cascaded to districts to eventually cover all schools. • Test Administration Manual developed and distributed to each school.
DBE Monitors will be deployed to monitor the distribution and test administration of ANA. DBE Monitors comprise: DBE Staff and Independent Monitors appointed by the DBE (retired school principals and subject advisors) (80 monitors) Each province will also monitor the distribution and test administration. Each PED will submit a Daily Report to the DBE. Monitoring of Test Administration
Marking • Marking Guideline discussions were hosted in Pretoria for Grades 3, 6 and 9 on 13 – 15 August 2013. Each PED will host provincial marking guideline discussions. • Marking will take place at school level under the supervision of the school principal. • Senior teacher and HoDs will be responsible for school moderation. • All marking, moderation and mark sheets must be completed by 18 September 2013. • School-based marking will also involve the marking of all grades in special schools.
Centralised Moderation) • Grades 3, 6 and 9 scripts will be sampled for centralised moderation. • Between 3 to 15 scripts per school, per grade , per subject will be sampled. • DBE has developed a Marking Manual for marking and moderation processes. • The PED is responsible for selection, appointment and payment of marking personnel, marking centres, and the overall conduct and management of all marking processes. • Twenty four (24) marking centres and estimated 2400 markers. • Markers are appointed per school based on seniority. • Centralised moderation scheduled from 21 – 28 September 13. • Marks will be captured centrally in the province by a service provider.
ANA Verification • ANA Verification will be conducted by an independent Verifying Agent. • Sample size: • - Grades 3, 6 and 9: 2163 schools nationally (approximately 200 schools per province). • - Grades 1, 2, 4 and 5: 450 schools nationally (50 schools per province) • Monitor the test administration in each of the schools. • Tests collected from schools and marked independently. • Capture and process data.
ANA Verification • Test responses will be captured per question – produce an Item Analysis Report. • Final Independent Report presented to Minister. • Minister to release report in December 2013
Conclusion • The DBE is confident that ANA 2013 will be successfully administered. • Despite the demands placed on the system, based on the magnitude and size of this assessment, the DBE is convinced that ANA is the main driver for educational improvement. • Both PEDs and DBE have extensively used the data emanating from ANA 2011 and 2012 to structure their remedial and intervention programmes. • Progress made in achieving the goals of the Action Plan to 2014 will become evident with the outcome of these tests.