Reducing the Monitoring Burden on Centres: Understanding the Current Environment

Reducing the Monitoring Burden on Centres:Understanding the Current Environment Summer 2006

Contents • Objectives and approach • Executive summary • Centres and awarding bodies: • Understanding how learners access qualifications • Understanding how awarding bodies monitor centres • Understanding the impact of awarding body monitoring on centres • Steps awarding bodies are taking to reduce monitoring activity • Introducing other bodies that monitor or inspect centres / providers • Inspectorates • Funding and planning bodies • Quality marks • Identifying what makes a ‘good’ centre • Defining ‘good’ • Measuring ‘good’ and the impact on monitoring • Other monitoring approaches • In the education environment • In other sectors • Appendix A: Consulted organisations B: Bibliography C: Glossary

Understand how awarding bodies monitor centres Understand how other bodies monitor or inspect centres Analyse other monitoring approaches in and outside of the education sector to understand what can be learnt Scope The geographic areas in scope are England, Wales and Northern Ireland Scotland is looked at in order to understand how monitoring differs and what could be learnt The project scope includes only those qualifications in the National Qualifications Framework (NQF) In Wales, General Qualifications are excluded This document analyses how centres are currently monitored and reviews the approaches of other monitors, both in and outside of the centre environment Objectives Approach • Secondary research: • Awarding body and other organisations’ websites • Industry publications and reports • Awarding body monitoring templates • Regulatory documents • Text line searches • Primary research: • Interviews with organisations representing a wide sample of the stakeholders involved in the centre monitoring process Note: A list of stakeholder interviews/ meetings and a bibliography is contained in the appendix.

Executive summary

Learners access qualifications in centres that are accountable to awarding bodies that are, in turn, accountable to regulators REGULATORY BODY AWARDING BODY CENTRE LEARNER • Seek recognition from the regulator • Seek accreditation for qualification from the regulator • Recognise centre • Approve centre to offer a qualification • Recognise awarding body • Accredit awarding body’s qualification • Seek recognition from awarding body • Seek approval to offer a qualification from awarding body • Offer qualification to learner • Seek qualifications at centre Before a learner can be offered a qualification, the centre needs to be recognised and approved for that qualification by the awarding body, and the awarding body needs to be recognised and the qualification approved by the regulator. Source: www.QCA.org.uk; “Research into the current centre recognition and centre qualification approval arrangements”, MORI, September 2005; “The market for qualifications in the UK”, PricewaterhouseCoopers, April 2005.

There are over 115 awarding bodies in the markets and most centres (78%) have between one and five awarding body relationships Number of awarding body relationships of centres Average number of awarding body (AB) relationships of centresa Commentarya,b • Learners receive training and / or teaching at their centre and usually assessed for their qualification(s) at their centre also • Schools, colleges, training providers and employers can all apply to awarding bodies to be recognised as centres • There are over 115 awarding bodies eligible to offer regulated qualifications in the UK • Most centres (78%) have between one and five awarding body relationships, but a significant minority (8%) have over sixteen relationships • The number of awarding body relationships a centre has is driven by factors such as the range of qualifications a centre offers, personal preference and local demand Centre sample size: 1,000 Centres state that qualification content and reputation are more important when choosing awarding bodies and qualifications than paperwork and processes and fees. Source: (a) “Research into the current centre recognition and centre qualification approval arrangements”, MORI, September 2005; (b) Capgemini analysis based on stakeholder interviews, Summer 2006.

Awarding bodies monitor centres for regulatory, quality and commercial purposes Post-approval (or ongoing) monitoring of centres by awarding bodies Post-approval monitoring of centres is defined as: “The review of, and reporting on, the centre’s quality assurance arrangements by the awarding bodies”a DEFINITIONa The purpose of awarding bodies’ monitoring of centres is to: • Maintain quality and standards in their qualifications • Ensure that they are able to satisfy the regulators • Assure their own reputation PURPOSEb Contact with centres that occurs through some monitoring can also result in: • Support for programme leaders • Relationship building • Ensuring opportunities for learners OTHER OUTCOMESb Awarding bodies and others in the environment, including centres, stress that awarding bodies can provide significant support to programme leaders as part of the monitoring process. Source: (a) “Stakeholder consultation: Centre recognition and centre qualification approval, November 2005 – January 2006”, Centre Recognition Project, 11/2005; (b) Capgemini analysis based on stakeholder interviews, Summer 2006.

Awarding body monitoring activity relates either to assessment or to a centre’s continued adherence to the centre approval criteria How awarding bodies monitor centres Centre responsibilities Awarding body monitoring activity Rationale for monitoring activity • To ensure that standards are aligned within and across centres • Internal assessment and standardisation of learners’ written work • External moderation of internally assessed work • To ensure a centre’s conduct of examinations adheres to awarding body expectations • Conduct of examinations in line with instructions • Visits during examination periods • Analysis of examination outcomes ASSESSMENT RELATED • To ensure quality and consistency of assessment practices and procedures within and across centres • This ensures standards are aligned • Internal assessment of learner competence • Internal verification of internal assessment • External verification of internal assessment and verification • Monitoring of external verifiers • To ensure that the centre is continuing to meet the requirements of the approved centre criteria • By reviewing the centre’s quality assurance arrangements • Adherence to centre approval criteria • Systems audit CENTRE APPROVAL RELATED Awarding bodies tend to define their monitoring by qualification type, but different qualifications are subject to the same types of monitoring if the assessment methods used are the same. Source: (a) “GCSE, GCE, VCE, GNVQ and AEA Code of Practice, 2006/07”, QCA, ACCAC (as was), CCEA, 2006; (b) “NVQ Code of Practice”, QCA, ACCAC (as was), CCEA, 2001; (c) “Proposed centre recognition requirements”, QCA, ACCAC (as was), CCEA, SQA, 2005; (d) Capgemini analysis based on stakeholder interviews, Summer 2006.

On the surface, awarding bodies collect very similar information when monitoring centres Sample of awarding body VRQ assessment means Externally assessed Internally assessed Both internal and external Centre responsibilities Awarding body monitoring activity GQ NVQ A B C D E F G H I Internal assessment and standardisation of written work Sampling of internally assessed work X n/a n/a n/a n/a n/a n/a n/a n/a X n/a Conduct examinations in line with instructions Checking exam conduct X n/a X X X n/a n/a n/a X X X Analysing exam outcomes n/a X X X n/a n/a n/a X Internal assessment of learner competence Internal verification of internal assessment Sampling internal assessments n/a X n/a n/a n/a X X X X X Checking internal verification processes, e.g. sampling plan n/a X n/a n/a n/a X X X X X X Adherence to centre approval criteria Recording details of centre, candidates and qualifications n/a X X X X X X X X X X Understanding changes since the last monitoring check n/a X X X X X Understanding if actions have been implemented n/a X X X X X X X X X Checking quality assurance records are maintained n/a X X X X X X X X X X Health and safety policy n/a X X X X X X Physical resources n/a X X X X X X X X X X Staff qualifications n/a X X X X X X X X X X QUALITY ASSURANCE CHECKS Communications system n/a X X X X X X X X Particular requirements n/a X X X X X X X Appeals process n/a X X X X X X X Equal opportunities policy n/a X X X X X X X X Awarding bodies offering GQs or NVQs seek information according to the code of practice and, for VRQs, where they have more flexibility, appear to seek very similar information. Source: (a) Capgemini analysis based on stakeholder interviews and awarding body documentation, Summer 2006

The level of monitoring a centre receives depends on the qualifications it offers as well as on the number of awarding body relationships it has Monitoring activity Centre scenario External moderation1 Exam visits External verification2 Systems audit ABs3 Visits p.a. A A school: • Offers GQs only • Has 5 unitary body relationships • Applies to each qualification with internally assessed work • Maximum of one visit per exam cycle • n/a • n/a 5 0-1 B A small employer: • Offers one NVQ to 10 employees p.a. • Has a relationship with one awarding body • n/a • n/a • Two external verifier (EV) visits per year including a systems audit • n/a 1 2 C A training provider: • Offers several NVQs and VRQs in engineering • Has two awarding body relationships • n/a • Awarding body 1 may visit during its VRQ exams • (All externally assessed) • For NVQs, 4 EV visits per year – 2 from each awarding body • An additional EV visit from awarding body 2 for its VRQs • (All internally assessed) • Awarding body 1 does a systems audit as part of an EV visit • Awarding body 2 conducts a separate systems audit visit 2 6-7 D A college: • Offers a wide range of GQs, NVQs and VRQs • Has over 40 awarding body relationships • Applies to each qualification with internally assessed work • Receives one visit for all GQs p.a. • May receive one visit per awarding body for other qualifications with exam elements • Two EV visits p.a. per NVQ programme area for each awarding body • Additional 1 or 2 visits for VRQs with internal assessment • Some awarding bodies combine NVQ and VRQ visits • Conducted separately by several of the awarding bodies • Combined with external verifier visits for some others • Some do not do audits 40+ 40+ Each centre has a very different monitoring experiences and, as a result, different interpretations of the ‘burden’ of this monitoring. Source: (a) “AEA, GCE, VCE, GCSE, GNVQ and ELC: Instructions for conducting examinations, 1 September 2005 to 31 August 2006”, JCQ, 2005; (b) “NVQ Code of Practice”, QCA, ACCAC (as was), CCEA, 2001; (c) Capgemini analysis based on stakeholder interviews, Summer 2006. Note; (1) External moderation of internally assessed and standardised written work; (2) External verification of internally assessed and verified learner competence; EV – external verifier; (3) ABs – awarding bodies. Column indicates the number of awarding body relationships the centre has.

Centres commented on the benefits of awarding body monitoring such as support, but feel that awarding body approaches differ greatly RANGE OF CENTRE VIEWS Monitoring is a burden Monitoring is a benefit “I am paying awarding bodies for a service and they are conducing tick box exercises on me that are of no benefit to me.” Training provider “Awarding bodies can provide a lot of help to struggling centres.” Training provider “Some awarding bodies have a very supportive approach whereas others are there to check only.” Training provider “Programme leaders find direct contact with an EV useful.” Examination officer “It is burdensome having lots of awarding body relationships, but we wouldn’t not have a relationship purely on the basis of burden. We want to offer learners the best quality.” Examinations officer “Some of the information they ask for does not relate in any way to the quality and standards of their qualifications” Training provider “All awarding bodies do things differently. All their forms are very different, their visit cycles etc. It would help if these were aligned” College Director “All monitors – not just awarding bodies – ask the same things” Training provider Centres commented that increasing the number of awarding body relationships does increase burden, but they want to offer the best quality qualifications to learners and increased burden can be an acceptable by-product. Source: (a) Capgemini analysis based on stakeholder interviews, Summer 2006.

Many awarding bodies commented that the monitoring they do is not burdensome, and the support they provide can be very valuable RANGE OF AWARDING BODY VIEWS Monitoring is a burden Monitoring is a benefit “There are times where we have helped a failing centre to turn around and subsequently receive a 1 in their ALI inspection” “Some centres feel burdened by awarding bodies because we give them actions, but there’s a reason they need the actions!” “I appreciate that we can only see the impact from our point of view and don’t have a feel for the combined impact of awarding body monitoring.” “What burden?” “Efficient centres have no problem providing awarding bodies with the information they need therefore it’s not burdensome” “The primary role of a verifier is to check assessment set up is robust – QA purpose rather than support.” “Centres have a choice about the number of awarding body relationships they have” “Some of the burden centres feel is as a result of regulatory requirements.” Some awarding bodies felt that tweaks could be made to reduce some of the burden centres feel. Source: (a) Capgemini analysis based on stakeholder interviews, Summer 2006.

The ‘burden’ that awarding bodies place on centres as a result of ongoing monitoring could be reduced by simplifying or integrating the activity Simplify the monitoring process: • Awarding bodies could reduce the monitoring activityper monitoring interaction • i.e. request less information for monitoring purposes or stop some monitoring • Awarding bodies could reduce the frequency of their monitoring activity Integrate the monitoring process across awarding bodies: • Awarding bodies could standardise their monitoring activity • For example, standardising forms they use or the format of documents they request • Awarding bodies could join up their monitoring activity • For example, combining visits or having one organisation monitor on behalf of others These burden reduction methods are not mutually exclusive and can all produce options for consideration. Source: (a) Capgemini analysis, Summer 2006.

Awarding bodies are already taking steps to reduce monitoring burden on centres, with most initiatives being led by JCQ members Simplify monitoring Integrate monitoring Reduce the monitoring activity per monitoring interaction Standardise monitoring across awarding bodies • Common centre approval criteria* • Centres do not need seek approval more than once • Consortium arrangements • Moderation and verification of internal assessment is done across the consortium rather than on each centre • Risk-based approaches – monitoring good centres less (but struggling centres more) • Conducting desk-based or telephone monitoring • Common instructions for examination conduct* • Common access arrangements* • Common centre approval criteria* • Common external verifier reporting* • Common data standards*: • Building the capacity to share data • Common centre numbers • Unique learner numbers Reduce the frequency of monitoring activity Join up monitoring across awarding bodies • Risk based approaches • For exam visits, for example, awarding bodies will target spot checks at the centres they feel pose the greatest risk • The risk assessment is often qualitative (for example, the centre is new, therefore high risk) • Combined exam visits for all qualifications* • One organisation monitors on behalf of others • Common centre approval criteria* • Some awarding bodies effectively approve on behalf of others All of the starred initiatives are part of the JCQ’s Eight Pledges and, with the exception of common exam instructions and access arrangements, apply to JCQ awarding bodies (or some of them) only at present. Source: (a) “Consortium Arrangements for BTEC and Edexcel NVQ Qualifications”, Edexcel, 2004; (b) “Consortium Arrangements for General Qualifications”, Edexcel, 2004; (c) “Application for Centre Consortium Arrangements“, JCQ, 2005; (d) Capgemini analysis based on stakeholder interviews, Summer 2006.

Awarding bodies Inspectorates In addition to awarding bodies, centres are inspected or reviewed by inspectorates and funding and planning bodies, each for different purposes Bodies that monitor centres Funding and planning bodies • Maintain quality and standards in the qualifications • Ensure that they are able to satisfy the regulators • Assure their own reputation • Provide support for programme leaders • Ensure opportunities for learners • Raise quality and standards of education and training by identifying and promoting good practice • Provide an independent evaluation of the quality and standards achieved by a particular centre to inform the government and the public • Ensure that the provision they fund is meeting the demands of learners, employers and local communities for learning and skills • Ensure that the provision they fund is of a high quality and represents value for money • Also, they ensure that providers take ownership of quality and demonstrate their commitment to continuous improvement Purpose of monitoring / inspections Need assurance that: Learners achieve awards that are representative of their competence Need assurance that: Learners’ achievement levels are high and education and training quality is high Need assurance that: Provision meets the needs of the community, it is high quality and represents value for money A centre may be providing education and training to a low standard, but be maintaining the correct level of standards in the assessment of qualifications To promote self-assessment and increase capacity for self-improvement, some centres also choose to adhere to certain external quality frameworks. Source: (a) www.ali.gov.uk; (b) ”The Common Inspection Framework for Education and Training from 2005”, Ofsted and ALI; (c) www.ofsted.gov.uk; (d) “The Common Inspection Framework for Education and Training in Wales”, Estyn; (d) “COMMON FRAMEWORK FOR INSPECTION for The Education and Training Inspectorate”, ETI; (e) “IMPROVING QUALITY :RAISING STANDARDS, Arrangements for the Inspection of Further Education Programmes in Northern Ireland Revised 2003”, ETI, 2003; (f) Capgemini analysis, summer 2006; (g) www.lsc.gov.uk; (h) www.elwa.ac.uk; (i) www.delni.gov.uk; (j) www.deni.gov.uk; (k) www.hefce.ac.uk.

Awarding bodies Inspectorates Awarding bodies are the only monitors to check the appropriateness of assessment decisions or examination conduct Bodies that monitor centres Funding and planning bodies • Moderation of internally assessed work • Visits during examination periods to ensure adherence to instructions • Analysis of examination outcomes • Verification of internal assessment and verification through sampling and speaking with staff and learners • Systems audit to check appropriateness of a centre’s quality assurance processes • This may include conversations with staff and learners • Observing learning • Interviewing learners, staff and other stakeholders • Examining learners' work and documents relating to training, assessment, verification and qualifications • Reviewing self-assessment reports, quality assurance processes, and strategic and operational plans • The provider’s self-assessment report and quality development/ improvement plans • Inspection reports or reports from other visits by the inspectors • Strategic plans • Information from health and safety monitoring, and financial data • Performance against national, regional and local benchmarks for learner success rates Evidence gathered However, all monitors appear to do a quality assurance / systems audit type check. Source: (a) www.ali.gov.uk; (b) ”The Common Inspection Framework for Education and Training from 2005”, Ofsted and ALI; (c) www.ofsted.gov.uk; (d) “The Common Inspection Framework for Education and Training in Wales”, Estyn; (d) “COMMON FRAMEWORK FOR INSPECTION for The Education and Training Inspectorate”, ETI; (e) “IMPROVING QUALITY :RAISING STANDARDS, Arrangements for the Inspection of Further Education Programmes in Northern Ireland Revised 2003”, ETI, 2003; (f) Capgemini analysis, summer 2006; (g) www.lsc.gov.uk; (h) www.elwa.ac.uk; (i) www.delni.gov.uk; (j) www.deni.gov.uk; (k) www.hefce.ac.uk.

Awarding bodies Inspectorates Most monitors, even some inspectorates, monitor or review centres on a annual basis with any visits lasting anything from a few hours to weeks Bodies that monitor centres Funding and planning bodies • Length: ranges from a few hours to a full day • Frequency: some centres will receive no monitoring contact, others will receive multiple visits from many awarding bodies per year • Length: ranges from two days to eight weeks • Frequency: ranges from three to seven years for inspections • However, some inspectorates conduct more frequent visits – for example Ofsted visits colleges to conduct a quality monitoring visit every year • Length: often conducted remotely. Length of visits vary • Frequency: often annually; more for providers with failings either in the quality of their provision, or the rigour of their own internal quality management systems Length and frequency monitoring / provider reviews Inspectorates and funding and planning bodies and a number of awarding bodies also use risk assessments (sometimes informally) to determine the level of contact a centre / provider requires. Source: (a) www.ali.gov.uk; (b) ”The Common Inspection Framework for Education and Training from 2005”, Ofsted and ALI; (c) www.ofsted.gov.uk; (d) “The Common Inspection Framework for Education and Training in Wales”, Estyn; (d) “COMMON FRAMEWORK FOR INSPECTION for The Education and Training Inspectorate”, ETI; (e) “IMPROVING QUALITY :RAISING STANDARDS, Arrangements for the Inspection of Further Education Programmes in Northern Ireland Revised 2003”, ETI, 2003; (f) Capgemini analysis, summer 2006; (g) www.lsc.gov.uk; (h) www.elwa.ac.uk; (i) www.delni.gov.uk; (j) www.deni.gov.uk; (k) www.hefce.ac.uk.

Awarding bodies Inspectorates The monitoring scope varies between awarding bodies and other monitorswith inspectorates and funding bodies looking at total provision Bodies that monitor centres Funding and planning bodies • Awarding bodies are concerned primarily with the programme areas delivering their qualifications • This may include a quality assurance check at the programme level • However, some awarding bodies check quality assurance systems across the centre • Inspectorates are concerned with provision across an entire centre • Inspectorates may only sample a number of programme areas in order to make a judgement on the quality and standards of education and training in a centre • Funding bodies are concerned with provision across an entire centre • Providers that have demonstrated that adequate quality management systems are in place, may receive a very light touch funding body review Scope of monitoring / provider reviews A centre that has received an inspection result of ‘satisfactory’ may have shortcomings or areas for improvement in the one programme area that an awarding body is concerned with. Note that a small number of centres may not be inspected or reviewed by inspectorates or funding and planning bodies meaning awarding bodies may be the only monitors looking at that centre Source: (a) www.ali.gov.uk; (b) ”The Common Inspection Framework for Education and Training from 2005”, Ofsted and ALI; (c) www.ofsted.gov.uk; (d) “The Common Inspection Framework for Education and Training in Wales”, Estyn; (d) “COMMON FRAMEWORK FOR INSPECTION for The Education and Training Inspectorate”, ETI; (e) “IMPROVING QUALITY :RAISING STANDARDS, Arrangements for the Inspection of Further Education Programmes in Northern Ireland Revised 2003”, ETI, 2003; (f) Capgemini analysis, summer 2006; (g) www.lsc.gov.uk; (h) www.elwa.ac.uk; (i) www.delni.gov.uk; (j) www.deni.gov.uk; (k) www.hefce.ac.uk.

Inspectorates and funding and planning bodies use risk based approaches to reduce burden on providers Simplify monitoring Integrate monitoring Reduce the monitoring activity per monitoring interaction Standardise monitoring • Risk based monitoring approaches • The extent of the review is determined based on the previous review and more recent data and reports • The inspectorates and funding bodies have agreements in place to share data • Self-assessment is used to review providers • Desk-based reviews occur rather than visits in person • Providers are usually monitored by one inspectorate and one funding and planning body • Therefore, unlike in the awarding body environment where centres are monitored by many of the same type of bodies, there is little scope for standardisation here • The exception is in England where some providers are monitored by both ALI and Ofsted and LSC and another funding bodies (e.g. Jobcentre Plus) • In these cases, the organisations are working together to standardise their approach Reduce the frequency of monitoring activity Join up monitoring across awarding bodies • Risk based monitoring approaches • Some reviews are removed for providers that perform well, for example: • ALI conducts quality monitoring visits about a year after inspections, but providers that perform well at inspection are not visited • LSC conducts an interim review six months after the annual review, but providers that perform well are not reviewed again • Some bodies inspect or review on behalf of others • ETI conducts all inspections or surveys on behalf of DEL, the funding body in NI • Co-ordinated visits to providers • Where possible LSC conducts Provider Financial Assurance visits in line with inspectorates Inspectorates visiting on behalf of other bodies (as occurs in NI with DEL and ETI) and coordinated visits also reduce the inspection and review burden on providers. Note: (1) Providers in NI are reviewed by ETI, the inspectorate, only and ETI informs the funding body of outcomes.

If a centre can be identified as being ‘good’ then the monitoring activity it is subjected to could be reduced Identifying good and the potential impact on monitoring • In order to understand whether monitoring activity could be reduced per monitor (by seeking less evidence per monitoring interaction or reducing the frequency of monitoring), two questions need to be asked: • What makes a centre ‘good’? • How can a ‘good’ centre be identified? What ‘good’ looks like for each different centre monitor will be explored in turn followed by how ‘good’ is measured to reduce monitoring activity.

When asked what a ‘good’ centre looks like, awarding bodies tended to comment on the efficiency of the centre and its quality assurance processes ‘Good’ centres for awarding bodies • A good centre… • …is one that returns all examination documentation on time and completed correctly • …has standard admin processes across the centre • …feels good when you walk through the door – you can smell it • …has excellent quality management processes in place • …sounds good. Our experience means we just know • …has good internal verifier processes so that not all verification is end-loaded • …has a balance of experienced staff versus new staff and these new staff have training opportunities available to them According to awarding body commentary, identifying ‘good’ tends to be a qualitative measure identified as a result of the experience of the awarding body staff or contractors conducting the monitoring. Source: (a) Capgemini analysis based on interviews with stakeholders, summer 2006.

For inspectorates, ‘good’ centres are places where learners achieve high standards, quality of provision is high and management is effective and efficient ‘Good’ providers for inspectorates • The elements of ‘good’ for inspectorates centre around the three elements of the common inspection framework: • Achievements of learners • Quality of provision • Quality of leadership and management • Quality of leadership and management underpins the achievement of learners and quality of provision, for example: • One of Estyn’s key questions in part 3 of the framework isb: • How well do leaders and managers evaluate and improve quality and standards? • Moreover, when grading, Estyn provides guidance to its inspectors that, if learner achievement and quality of teaching has been graded poorly, management and leadership cannot be rated highlyd • One of ETI’s questions in part three of the framework isc: • How well do leaders and managers set a clear direction leading to high quality education and training? • ALI and Ofsted’s key question in part 3 of the framework isa: • How effective are leadership and management in raising achievement and supporting all learners? For the inspectorates, effectiveness of management underpins learner achievement and teaching quality and for Estyn, low grades for the latter two mean a low grade must be given for management. Source: (a) ”The Common Inspection Framework for Education and Training from 2005”, Ofsted and ALI. (b) “The Common Inspection Framework for Education and Training in Wales”, Estyn; (c) “COMMON FRAMEWORK FOR INSPECTION for The Education and Training Inspectorate”, ETI; (d) Meeting with Ann Keane of Estyn, 08/08/2006. Note: Ofsted and ALI publish one inspection framework

“Providers have prime responsibility for the quality of their provision, and DELLS aims to ensure that providers take ownership of quality and demonstrate their commitment to continuous improvement.”b “The greatest likelihood for improvement, particularly improvement to be sustained over time, occurs when a school has established a positive culture and commitment to professional growth.”c “Primary responsibility for improving the quality of provision rests with the provider”, LSCa “The Government looks to all providers to adopt strategies for securing continuous improvement as many already do. These strategies should be based on self-assessment and action planning.”, ‘Learning to Succeed’ White Paper, 06/1999a “We expect providers to develop their own quality management systems... At the centre of these systems will be regular self-assessment to evaluate quality … and quality development planning to build on strengths and address weaknesses”b “Through rigorous self-evaluation a school can recognise its strengths and put in place processes to ensure that those areas identified as strengths continue to flourish, while other areas in need of improvement are prioritised and developed.”c For funding and planning bodies, ‘good’ centres have strong self-assessment and development processes that enable them to self-improve ‘Good’ providers for funding and planning bodies LSC (England)a,1 DELLS Planning and Funding (Wales)b,2 ETI (reviewing on behalf of DEL) (Northern Ireland)c,3 funding bodies, inspectorates and quality improvement The self-assessment guidance of the bodies was published in the last year and it will take time for all providers to achieve the expected standard. Source: (a) “Quality improvement and self-assessment”, LSC, 05/2005; (b) “Quality handbook”, ELWa, 06/2006; (c) “Together towards improvement”, ETI. Note: (1) LSC – Learning and Skills Council; (2) DELLS – Department for Education, Lifelong Learning and Skills. The funding body was previously ELWa, but this has merged with DELLs; (3) DEL – Department for Employment and Learning.

Awarding bodiesa Awarding body A using ‘standard’ monitoring to determine monitoring level Inspectoratesb,c ALI using one cycle of standard monitoring and pre-inspection analysis to determine monitoring level Funding and planning bodiesc LSC using a pre-review analysis to determine monitoring level In order to reduce monitoring activity, a monitor must conduct one cycle of ‘standard’ monitoring or a pre-monitoring analysis in order to assess ‘good’ Examples of activity to baseline in order to reduce monitoring activity • Awarding body A conducts annual monitoring visits (systems audits)to check that the required quality assurance processes are in place • Centres are graded on a scale of 1-5 with 1 being the top grade • The grade impacts the level of monitoring centres receive the following year • Centres scoring 1 or 2 are offered the opportunity to self-assess the following year • ALI is conducting one cycle of standard monitoring to gather baseline data • After this, the extent of a provider’s inspection is determined by a pre-inspection analysis • This analysis draws on a range of information including: • Previous inspection report or quality monitoring report • Retention and achievement data and other performance information • ALI conducts a Quality Monitoring Visit about a year after inspection • Providers that perform well at inspection are exempt from this visit • Prior to annual planning reviews, providers submit self assessment reports • Providers that demonstrate, through self-assessment, effective management of the risks to outstanding or good quality receive less intensive Annual Planning Reviews from the LSC • If the review identifies poor quality provision or lack of progress towards the achievement of headline performance measures, the LSC will discuss with providers options for support and intervention • If the LSC considers the self-assessment to lack rigour it will work with the Quality Improvement Agency (QIA) to support the provider in the revision of its self-assessment Whilst monitoring activity may reduce for those centres that are identified as being good, it will increase for struggling centres as the monitor provides them with additional support and contact. Source: (a) Capgemini analysis based on interviews with stakeholder, summer 2006; (b) “Changes in inspection, What they mean for you”, ALI, 04/2005; (c) “Quality improvement and self-assessment”, LSC, 05/2005. Note: ALI – Adult Learning Inspectorate (England), LSC – Learning and Skills Council (England)

Awarding bodiesa Awarding body A using ‘standard’ monitoring to determine monitoring level Inspectoratesb,c ALI using one cycle of standard monitoring and pre-inspection analysis to determine monitoring level Funding and planning bodiesc LSC using a pre-review analysis to determine monitoring level Even once a centre has been identified as being ‘good’ its performance is still often reviewed frequently to check that performance has not dropped The risk associated with risk assessments • Annual monitoring occurs for all centres • ‘Good’ centres self-assess, all others are visited • If standards are suspected of slipping in a ‘good’ centre as a result of the self-assessment, a visit is triggered • Investigations are also triggered if the awarding body identifies any potential malpractice through its analysis of exam outcomes or the exam documentation submitted by the centre • ALI conducts a pre-inspection analysis to determine the extent of an inspection • If the quality of some aspect of provision is found to be unsatisfactory in reduced intensity inspections, ALI may allocate further days to the inspection • The LSC reviews all providers annually • Prior to annual planning reviews, provider submit self assessment reports that are used to determine the extent of the Annual Planning Review Awarding bodies, inspectorates and funding bodies mitigate against the risk of risk-based monitoring by frequently reviewing performance and having triggers in place that prompt monitoring action. Source: (a) Capgemini analysis based on interviews with stakeholder, summer 2006; (b) “Changes in inspection, What they mean for you”, ALI, 04/2005; (c) “Quality improvement and self-assessment”, LSC, 05/2005. Note: ALI – Adult Learning Inspectorate (England), LSC – Learning and Skills Council (England)

Other monitors in and outside of the education sector are illustrating a number of methods that they are using to reduce monitoring burden Simplify monitoring Integrate monitoring Reduce the monitoring activity per monitoring interaction Standardise monitoring • Risk based monitoring approaches • The extent of the monitoring is determined based on the previous review and more recent data and reports • Self-assessment is used to review organisations • Self-assessment reports are often used as part of reviews • SQA centres can be delegated more authority and are monitored only on their quality assurance processes • Conducting desk-based reviews • For example, Edexcel is no longer visiting HEIs but reviewing information submitted electronically • The heath concordat is promoting standardisation through the tools it has developed for implementation, for example: • Baseline assessment, using gap analysis to map existing processes against recommended practice • Evaluating costs and benefits against a best practice report • Developing guidelines for information gathering and sharing • Guidelines for making recommendations to providers, to allow co-ordination and prevent duplication Reduce the frequency of monitoring activity Join up monitoring across awarding bodies • Risk based monitoring approaches • For example, SQA conducts systems audits according to a risk system that means poor centres are monitored more and good centres less • Some bodies inspect or review on behalf of others • QAA assesses quality of teaching on behalf of all higher education funding bodies • Co-ordinated visits • For example, health concordat members are co-ordinating inspections with other reviews Risk based approaches, monitoring via self-assessment and conducting desk-based reviews (rather than visits) are all common.

Centres and awarding bodies Understanding how learners access qualifications Understanding how awarding bodies monitor centres Understanding the impact of awarding body monitoring on centres Steps awarding bodies are taking to reduce monitoring activity

24% of the qualifications in the National Qualifications Framework are General The ‘true’ general qualifications are recognised as AEAs, GCSEs, GCEs and GCE AS In April 2005, there were 768 qualifications in this market GCSEs are at levels 2 and 3 and GCEs at level 3 The General category also includes ‘other’ general qualifications such as graded music and free-standing mathematics qualifications Learners have a wide range of regulated qualifications available to them, ranging from general to vocational to occupational qualifications Categories of regulated qualifications available to learners General qualifications Occupational qualifications • 37% of the qualifications in the NQF are Occupational • Occupational qualifications comprise the largest group of qualifications in the National Qualification Framework • They are primarily NVQs • The majority of qualifications are at Levels 2 and 3 (65%) Vocationally Related qualifications Other qualifications • 27% of the qualifications in the NQF are Vocationally Related • These qualifications can be offered at all levels • 18% of the qualifications in the NQF are in the other category • This category includes Key Skills, Entry Level, Basic Skills and Higher Level qualifications. In April 2005 there were over 4,500 qualifications in the National Qualifications Framework – the list of all regulated qualifications. Source: (a) “The market for qualifications in the UK”, PricewaterhouseCoopers, April 2005. Note: NQF – National Qualifications Framework. Due to their dominance, NVQs are used in this report as a proxy for the occupational qualifications market.

Schools provide learning to children usually up to the age of 16, on behalf of the state Learning and assessment predominantly take place within the school’s own building Principal qualifications offered are general qualifications such as GCSEs Schools may offer a broad range of qualifications Learners access qualifications at centres which can include schools, colleges, training providers and employers Types of centresa Schools Colleges • Colleges include • Further education and sixth form colleges • Higher education colleges (outside project remit unless offering qualifications on the NQF) • Sixth form colleges • Independent schools • Qualifications offered can be the same as at school but often include level 3general qualifications and vocational courses Employers Training providers • Many employers offer qualifications to their employees in house • According to research by PricewaterhouseCoopersb, 40% of employers surveyed provided internal training leading to a qualification • Qualifications offered include both NVQs and VRQs • Employers are most likely to provide job specific training (80%) with 69% providing health and safety training and 53% providing training in new technologyc • Training providers generally provide training in specific skills at designated centres or on an employer’s site • For example, business skills workshops • Qualifications offered often include VRQs • Training providers often offer training in specific areas and therefore have a more focussed range of qualifications than, for example, a college Normally, learners receive training and / or teaching at their centre and are also assessed for their qualification(s) there. Source: (a) “Research into the current centre recognition and centre qualification approval arrangements”, Mori, September 2005, (b) “The market for qualifications in the UK”, PricewaterhouseCoopers, April 2005; (c) “National Employer Skills Survey”, 2003; (d) Capgemini analysis based on stakeholder interviews, Summer 2006.

Awarding bodies develop and award qualifications They also assess and quality assure qualifications and provide customer service to centres and candidates There are over 115 awarding bodies that offer qualifications that are part of the National Qualifications Framework There are only five awarding bodies in England, Wales and NI that can offer General Qualifications These are AQA, Edexcel, OCR, WJEC (Wales) and CCEA (NI) and are known as the unitary bodies The awarding body market is dominated by the large players that make up the Joint Council for Qualifications (JCQ) Members of the JCQ are the unitary bodies listed above plus SQA, the unitary body in Scotland, and City & Guilds, the largest vocational awarding body Of the 115 awarding bodies, the majority are members of the Federation of Awarding Bodies – FABe,1 In order to offer qualifications, centres need a relationship with an awarding body as it is these organisations that develop and award qualifications Awarding body market and relationships with centres Awarding body market overviewb,c,d Awarding body and centre relationshipa • A centre is defined as “any organisation, establishment or individual entity accountable to an awarding body for the assessment arrangements leading to an accredited qualification or unit.”a • A centre must be recognised by an awarding body and approved to offer one of its qualifications before the qualification can be offered to learners • Centre recognition is defined as “a process through which a centre is recorded as having committed itself to maintain the required quality and consistency of assessment and comply with other expectations of the relevant awarding body”a • Centre qualification approval is defined as “a process through which a centre is confirmed as being able to maintain the required quality and consistency of assessment and comply with other expectations of the relevant awarding body”a • Once approved, a centre is subjected to post-approval monitoring by the awarding body • This is defined as “the review of, and reporting on, the centre’s quality assurance arrangements by the awarding bodies”a Centres are accountable to awarding bodies on an ongoing basis for the quality and consistency of assessment and any other expectations that awarding bodies define. Source: (a) “Stakeholder consultation: Centre recognition and centre qualification approval, November 2005 – January 2006”, Centre Recognition Project, 11/2005; (b) “The market for qualifications in the UK”, PricewaterhouseCoopers, April 2005; (c) www.JCQ.org.uk; (d) www.QCA.org.uk; (e) www.awarding.org.uk Note: (1) For a list of FAB members, go to the FAB website and click on the ‘members’ tab: www.awarding.org.uk

Most centres (78%) have between one and five awarding body relationships, but a significant minority (8%) have over sixteen relationships Number of awarding body relationships of centres Average number of awarding body (AB) relationships of centresa Commentarya,b • According to research by MORI, 80% of centres have formed relationships with only 1-5 awarding bodiesa • However, 8% of centres have complex relationship networks, working with over 16 awarding bodies • Colleges tend to have the most relationships with awarding bodies, and schools the least when considering applications for recognition and qualification approval • The total number of awarding body relationships a centre has is determined by factors such as: • The range of qualifications the centre offers • Personal preference of programme leaders • Availability of funding for particular courses or qualifications • Local demand • Legal requirements, for example, health and safety training requirements for employers Centre sample size: 1,000 The number of awarding body relationships a centre has is driven by factors such as the range of qualifications a centre offers, personal preference and local demand. Source: (a) “Research into the current centre recognition and centre qualification approval arrangements”, MORI, September 2005; (b) Capgemini analysis based on stakeholder interviews, Summer 2006.

Centres’ decisions about which awarding bodies they have relationships with is affected by the content of the qualifications and reputation Factors affecting which awarding bodies centres choosea Factors affecting choice of awarding body Commentary • MORI research found that the content of qualification and the reputation of that qualification were seen as important by 96% of centres • 95% of centres felt that the reputation of the awarding body is also important • Fewer centres felt that considerations about the paperwork and processes involved were important when choosing an awarding body (73%) • Fewer still (66%) felt fees were important • Two thirds of centres also cited other factors affecting their choice • This included the overall or ongoing support and guidance offered by an awarding body • 14% of centres cited this factor and it was more important for training providers (19% citing the factor) The content of a particular qualification/ fitness for purpose The reputation of their qualifications The reputation of the awarding body The paperwork and process involved The fees an awarding body charges Centres state that qualification content and reputation are more important when choosing awarding bodies and qualifications than paperwork and processes and fees. Source: (a) “Research into the current centre recognition and centre qualification approval arrangements”, MORI, September 2005.

QCA is the regulatory body in England, DELLS in Wales and CCEA in Northern Ireland1 These organisations work together to regulate awarding bodies and qualifications The purpose of regulation is to ensure that qualifications are fair, standards are secure, and public confidence is maintained They regulate by: Developing and publishing criteria for the accreditation of qualifications There are three types of regulatory criteria – statutory regulations; subject criteria; and codes of practice Accrediting qualifications against those criteria Keeping qualifications under review Publishing and sharing information relating to accredited qualifications In order to award regulated qualifications, awarding bodies must seek recognition and then approval for each qualification from the regulators The regulatory bodies and relationships with awarding bodiesa Regulatory bodies overview Regulatory and awarding body relationships • In order to award qualifications that are part of the NQF, an awarding body must first be recognised by a regulator • An organisation can also enter into partnership with an existing recognised awarding body • Once recognised, awarding bodies can submit their qualifications to the regulators for accreditation • Once accredited, the qualification is placed on the NQF • The recognition process and qualification requirements are outlined in the regulatory document ‘The statutory regulation of external qualifications’ • Awarding bodies with qualifications on the NQF are subjected to ongoingmonitoring from the regulators • Monitoring findings inform decisions about re-accrediting qualifications or, if necessary, withdrawing accreditation Awarding bodies are monitored by regulatory bodies to ensure that they deliver high quality qualifications and provide good customer service in compliance with the regulatory criteria. Source: (a) www.QCA.org.uk. Note: (1) QCA – Qualifications and Curriculum Authority; DELLS – Department for Education, Lifelong Learning and Skills; CCEA – Council for the Curriculum, Examinations and Assessment.

Centres and awarding bodies Understanding how learners access qualifications Summary

Learners access qualifications in centres that are accountable to awarding bodies that are accountable to regulators REGULATORY BODY AWARDING BODY CENTRE LEARNER • Seek recognition from the regulator • Seek accreditation for qualification from the regulator • Recognise centre • Approve centre to offer a qualification • Recognise awarding body • Accredit awarding body’s qualification • Seek recognition from awarding body • Seek approval to offer a qualification from awarding body • Offer qualification to learner • Seek qualifications at centre Before a learner can be offered a qualification, the centre needs to be recognised and approved for that qualification by the awarding body, and the awarding body needs to be recognised and the qualification approved by the regulator. Source: www.QCA.org.ukl; “Research into the current centre recognition and centre qualification approval arrangements”, MORI, September 2005; “The market for qualifications in the UK”, PricewaterhouseCoopers, April 2005.

There are over 115 awarding bodies in the markets and most centres (78%) have between one and five awarding body relationships Number of awarding body relationships of centres Average number of awarding body (AB) relationships of centresa Commentarya,b • Learners are trained and / or taught and usually assessed for their qualification(s) at their centre • Schools, colleges, training providers and employers can all apply to awarding bodies to be recognised as centres • There are over 115 awarding bodies eligible to offer regulated qualifications in the UK • Most centres (78%) have between one and five awarding body relationships, but a significant minority (8%) have over sixteen relationships • The number of awarding body relationships a centre has is driven by factors such as the range of qualifications a centre offers, personal preference and local demand Centre sample size: 1,000 Centres state that qualification content and reputation are more important when choosing awarding bodies and qualifications than paperwork and processes and fees. Source: (a) “Research into the current centre recognition and centre qualification approval arrangements”, MORI, September 2005; (b) Capgemini analysis based on stakeholder interviews, Summer 2006.

Awarding bodies monitor centres for regulatory, quality and commercial purposes Post-approval (or ongoing) monitoring of centres by awarding bodies Post-approval monitoring of centres is defined as: “The review of, and reporting on, the centre’s quality assurance arrangements by the awarding bodies”a DEFINITIONa The purpose of awarding bodies’ monitoring of centres is to: • Maintain quality and standards in their qualifications • Ensure that they are able to satisfy the regulators • Assure their own reputation PURPOSEb Contact with centres that occurs through some monitoring can also result in: • Support for programme leaders • Relationship building • Ensuring opportunities for learners OTHER OUTCOMESb Awarding bodies and others in the environment, including centres, stress that awarding bodies can provide significant support to programme leaders as part of the monitoring process. Source: (a) “Stakeholder consultation: Centre recognition and centre qualification approval, November 2005 – January 2006”, Centre Recognition Project, 11/2005; (b) Capgemini analysis based on stakeholder interviews, Summer 2006.

Awarding bodies tend to define their monitoring by qualification type, but different qualifications are often subject to the same types of monitoring Approaches of awarding bodies monitoring centres by qualification types Main types of qualification Assessment of the qualification Typical associated monitoring • Visits during examinations to check centre’s conduct of exams • Analysis of exam outcomes • Moderation of internally marked work in line with the code of practice1 • A general qualification (GQ) • External assessment of learner achievement through examinations / tests • Internal assessment of written work • External verification of internal assessment and verification conducted in line with the code of practice1 • A national vocational qualifications (NVQ) • Internal assessment of learner competence • Visits during examinations to check centre’s conduct of examination process • Analysis of exam outcomes • A vocationally related qualification (VRQ) (or other qualification)that is examined • External assessment of learner achievement through examinations / tests • External verification of internal assessment and verification • A VRQ (or other qualification) that is based on an assessment of learner competence • Internal assessment of evidence of learner competence Monitoring is driven by the manner in which learner achievement is assessed, meaning different qualification types will be subject to the same monitoring if the assessment methods used are the same. Source: (a) “GCSE, GCE, VCE, GNVQ and AEA Code of Practice, 2006/07”, QCA, ACCAC (as was), CCEA, 2006; (b) “NVQ Code of Practice”, QCA, ACCAC (as was), CCEA, 2001; (c) Capgemini analysis based on stakeholder interviews, Summer 2006. Note: (1) The code of practice is a regulatory document to which awarding bodies must adhere. The regulators in England, Wales and NI have produced two codes of practice, one applying to GCSE, GCE, VCE, GNVQ and AEA qualifications and the other to NVQs.

As monitoring is driven by assessment method, awarding body monitoring can be cut by assessment type as opposed to qualification type Monitoring relating to assessment Assessment types Associated monitoring activity1 Assessment process • Internal assessment of learners’ written work • Internal standardisation of marks • External moderation of internally assessed and standardised work • Work is completed by the learner, usually at the centre • The centre supervises and authenticates the work • Work is marked and standardisedinternally at the centre • The centre submits requested samples of marked work • The awarding body moderates and adjusts marks • External assessment learner achievement of examinations / tests • Visits during examination periods • Analysis of examination outcomes • Learners sit the examinations at their centre • The centre conducts with exam in line with awarding body instructions • The centre submits scripts to the awarding body • All marking is done externally by the awarding body • The awarding body analyses results • Learners provide evidence of competence against standards within the qualification • Internal assessors ensure the validity, authenticity and sufficiency of the evidence produced • Internal verifiers ensure the accuracy and consistency of assessment decisions between assessors • External verifiers monitor the quality and consistency of assessment practices and procedures • Awarding bodies monitor their external verifiers for accuracy and consistency • Internal assessment of learner competence • Internal verification • External verification of internal assessment and verification • Monitoring of external verifiers Further details of the monitoring relating to the assessment of learner achievement are dictated by the codes of practice for NVQs and GCSE, GCE, VCE, GNVQ and AEA qualifications. Source: (a) “GCSE, GCE, VCE, GNVQ and AEA Code of Practice, 2006/07”, QCA, ACCAC (as was), CCEA, 2006; (b) “NVQ Code of Practice”, QCA, ACCAC (as was), CCEA, 2001; (c) Capgemini analysis based on stakeholder interviews, Summer 2006. Note: (1) This monitoring activity is carried out by both the centre (for example, standardisation of marked written work) and by the awarding body (for example, moderation of internally marked and standardised work).

Internally marked work is monitored remotely, with centres submitting samples of marked work to awarding bodies for external moderation Monitoring ‘internal assessment of written work’ Centre responsibilities Monitoring by awarding bodies • The centre is responsible for assessing work internally and standardising assessment across the centre • The centre must ensure that internally marked work is: • Supervised • Carried out by learners under awarding body specified conditions • Authenticated by an internal assessor, and • Assessed in accordance with awarding body guidance • This includes standardising assessments across different assessors and teaching groups • The centre is also responsible for ensuring that the requested samples of marked, internally standardised work are submitted to the awarding body • Awarding bodies moderate the marks submitted by each centre against the specified assessment criteria • This ensures that standards are aligned within and across centres • Awarding bodies draw samples of marked, internally standardised work, to cover the full range of units and to represent adequately the range of attainment in the centre • Where deemed necessary, an awarding body will adjust the marks that have been given by internal assessors • For GQs, any monitoring of internal assessment must adhere to the “GCSE, GCE, VCE, GNVQ and AEA Code of Practice” that is produced by the regulators in England, Wales and NI This type of monitoring often applies to GQs and sometimes elements of VRQs and ‘Other’1 qualifications in the NQF also. Source: (a) “GCSE, GCE, VCE, GNVQ and AEA Code of Practice, 2006/07”, QCA, ACCAC (as was), CCEA, 2006; (b) Capgemini analysis based on stakeholder interviews, Summer 2006. Note: (1) Other qualifications include Key Skills, Entry Level, Basic Skills and Higher Level qualifications. See slide 7 for details of qualifications on the NQF.

Awarding bodies set clear instructions for examination conduct that centres must adhere to These include protocols to be followed When preparing for the examination At the beginning of the examination During the examination At the end of the examination After the examination Examples of procedures specified include: Conditions for storing question papers securely Accommodation requirements, for example, having a clock displayed that all candidates can read clearly What to do if malpractice is suspected For GQs, a centre must adhere to the instructions set out in the “AEA, GCE, VCE, GCSE, GNVQ and ELC: Instructions for conducting examinations” document produced by the JCQ Examination scripts are marked externally, so the monitoring of centres relating to exams involves checking adherence to exam protocols Monitoring of ‘examination conduct’ Centre responsibilities Monitoring by awarding bodies Visits during exam periods • A centre’s conduct of examinations is monitored by visits during the examination period • These checks are in line with the instructions for exam conduct set out by the awarding body • This would include, for example, checking where exam papers are stored, and checking procedures carried out at the beginning, during and at the end of the examination Analysing exam outcomes • Exam scripts are marked externally and awarding bodies will review the outcomes to identify any potential issues This type of monitoring often applies to GQs and some VRQs and ‘Other’1 qualifications in the NQF with examined elements also. Source: (a) “AEA, GCE, VCE, GCSE, GNVQ and ELC: Instructions for conducting examinations, 1 September 2005 to 31 August 2006”, JCQ, 2005; (b) Capgemini analysis based on stakeholder interviews, Summer 2006. Note: (1) Other qualifications include Key Skills, Entry Level, Basic Skills and Higher Level qualifications. See slide 7 for details of qualifications on the NQF.

Centres are responsible for both internally assessing and verifying a learner’s competence Internal assessors are responsible for: Managing the assessment process in a centre This includes planning and maintaining accurate and verifiable learner records Assessing evidence of learner competence Ensuring the validity, authenticity and sufficiency of evidence produced by learners Internal verifiers are responsible for: Sampling evidence of assessment decisions to ensure consistency in the interpretation and application of standards Maintaining records of internal verification and sampling activity Monitoring and supporting assessors, including facilitating staff development and training Providing feedback to the external verifier on the effectiveness of assessment Ensuring adjustments required by the awarding body are implemented For internal assessment of a learner’s competence, internal as well as external verification of the assessment is used Monitoring ‘internal assessment of learner competence’ Centre responsibilities Monitoring by awarding bodies • External verifiers monitor the quality and consistency of assessment practices and procedures by • Ensuring assessments are conducted by appropriately qualified and expert assessors • Checking assessment decisions through sampling and speaking with staff and learners • Confirming assessment decisions are regularly sampled through internal verification • External verifiers provide support and guidance • They also define actions required for centres to bring assessment practices and procedures in line with awarding body expectations • External verifiers visit centres on behalf of awarding bodies • Awarding bodies monitor the performance and judgements of their external verifiers for accuracy and consistency • For NVQs, any monitoring must adhere to the “NVQ Code of Practice” that is produced by the regulators in England, Wales and NI This type of monitoring applies to all NVQs and also some VRQs and ‘Other’1 qualifications in the NQF with internally assessed learner competence. Source: (a) “NVQ Code of Practice”, QCA, ACCAC (as was), CCEA, 2001; (b) Capgemini analysis based on stakeholder interviews, Summer 2006. Note: (1) Other qualifications include Key Skills, Entry Level, Basic Skills and Higher Level qualifications. See slide 7 for details of qualifications on the NQF.

In addition to assessment related monitoring, awarding bodies also monitor centres to ensure ongoing adherence to the centre approval criteria How awarding bodies monitor centres Centre responsibilities Awarding body monitoring activity Assessment process • Post approval to offer a qualifications, the centre ensures that it continues to meet the approval requirements • The awarding body reviews the centre’s arrangements for ensuring that the approval criteria continue to be met • Where necessary, approval is removed by the awarding body • Adherence to centre approval criteria • Systems audit Post approval monitoring is defined as “the review of, and reporting on, the centre’s quality assurance arrangements by the awarding bodies”a Source: (a) “Stakeholder consultation: Centre recognition and centre qualification approval, November 2005 – January 2006”, Centre Recognition Project, 11/2005; (b) “NVQ Code of Practice”, QCA, ACCAC (as was), CCEA, 2001; (c) “Proposed centre recognition requirements”, QCA, ACCAC (as was), CCEA, SQA, 2005; (d) Capgemini analysis based on stakeholder interviews, Summer 2006.

Systems audits can review the management structure of the centre, the implementation of centre policy statements and availability of resources Non-assessment related monitoring Centre responsibilities Monitoring by awarding bodies • Centres are responsible for providing the awarding body with evidence that demonstrates that it is continuing to meet the centre approval criteria • Centre approval criteria may include: • Details of the management structure for the centre • Policy statements, for example, • Health and safety policy • Access to fair assessment statement • Employer and public liability insurance certificate • Disability Discrimination Act 1995 • Complaints procedure • Appeals procedure for candidates • Equal opportunities policy • Malpractice procedure • Commitment to provide resources or systems required for the qualification • Awarding bodies request information from centres to assure themselves (and the regulator) that the centre is continuing to meet the requirements of the approved centre criteria • This information can be requested via centre self-assessment, over the telephone or during visits • If an awarding body collects information during visits, these visits may be part of an external verifier visit • Awarding bodies often speak to staff and learners to gather evidence • In cases where a centre may not be meeting requirements, awarding bodies impose sanctions and/ or penalties on the centre • For NVQs, any sanctions imposed must be in line with the “NVQ Code of Practice” that is produced by the regulators in England, Wales and NI • This type of monitoring is often called a systems audit This type of monitoring is more prominent for NVQs and / or VRQs and ‘Other’1 qualifications in the NQF rather than GQs as centre approval requirements tend be more complex. Source: (a) “NVQ Code of Practice”, QCA, ACCAC (as was), CCEA, 2001; (b) “Proposed centre recognition requirements”, QCA, ACCAC (as was), CCEA, SQA, 2005; (c) Capgemini analysis based on stakeholder interviews, Summer 2006. Note: (1) Other qualifications include Key Skills, Entry Level, Basic Skills and Higher Level qualifications. See slide 7 for details of qualifications on the NQF.

Centres and awarding bodies Understanding how awarding bodies monitor centres Summary

Awarding body monitoring activity relates either to assessment or to a centre’s continued adherence to the centre approval criteria How awarding bodies monitor centres Centre responsibilities Awarding body monitoring activity Rationale for monitoring activity • To ensure that standards are aligned within and across centres • Internal assessment and standardisation of learners’ written work • External moderation of internally assessed work • To ensure a centre’s conduct of examinations adheres to awarding body expectations • Conduct of examinations in line with instructions • Visits during examination periods • Analysis of examination outcomes ASSESSMENT RELATED • To ensure quality and consistency of assessment practices and procedures within and across centres • This ensures standards are aligned • Internal assessment of learner competence • Internal verification of internal assessment • External verification of internal assessment and verification • Monitoring of external verifiers • To ensure that the centre is continuing to meet the requirements of the approved centre criteria • By reviewing the centre’s quality assurance arrangements • Adherence to centre approval criteria • Systems audit CENTRE APPROVAL RELATED The level of monitoring a centre receives will depend on the number, type and subject areas of the qualifications it offers as well as on the number of awarding body relationships it has. Source: (a) “GCSE, GCE, VCE, GNVQ and AEA Code of Practice, 2006/07”, QCA, ACCAC (as was), CCEA, 2006; (b) “NVQ Code of Practice”, QCA, ACCAC (as was), CCEA, 2001; (c) “Proposed centre recognition requirements”, QCA, ACCAC (as was), CCEA, SQA, 2005; (d) Capgemini analysis based on stakeholder interviews, Summer 2006.

    Monitoring impact on a centre is driven by the assessment method of the qualifications it offers and the number of awarding body relationships it has Impact of awarding body monitoring on centres Centre responsibilities Awarding body monitoring activity Monitoring impact relating to: Each qualification Each awarding body • Internal assessment and standardisation of written work • External moderation of internally assessed work Note that JCQ conducts all exam visits for GQs and this is being extended to other exams also • Conduct of examinations in line with instructions • Visits during examination periods • Analysis of examination outcomes 1 • Internal assessment of learner competence • Internal verification of internal assessment • External verification of internal assessment and verification • Monitoring of external verifiers • Adherence to centre approval criteria • Systems audit For internally assessed qualifications, monitoring activity occurs on a per qualification basis meaning the more qualifications a centre offers, the greater the potential monitoring impact. Source: (a) “GCSE, GCE, VCE, GNVQ and AEA Code of Practice, 2006/07”, QCA, ACCAC (as was), CCEA, 2006; (b) “NVQ Code of Practice”, QCA, ACCAC (as was), CCEA, 2001; (c) Capgemini analysis based on stakeholder interviews, Summer 2006; (d) “Eight Pledges: Delivering better service with less bureaucracy, Autumn 2005 - Progress Report”, JCQ, Autumn 2005. Note: (1) JCQ is responsible for conducting all exam visits on behalf of the unitary bodies. If a centre offers GQs from a unitary body and VRQs with examination elements from the same unitary body, then it may receive two exam visits – one from JCQ for GQ exams, and one from the awarding body itself for the VRQ exams.

Reducing the Monitoring Burden on Centres: Understanding the Current Environment