NVAO’s external quality assurance procedures

NVAO’s external quality assurance procedures Ann Van Neygen La Rochelle 11th of May 2012

The position of NVAO ‘Higher education’ Recognition Funding Support Appreciation Value for €Proof of use Quality Access Enhancement NVAO Information Accountability ‘Government’ ‘Society’ (Consumer) protection Support

Aim of NVAO’s system: balancingaccountability and enhancement • Stimulate quality culture • institutional audit • Commit professionals / Increase academic ownership • programme assessment • Reward earned trust • limited programme assessment • Stimulate HE to aim above threshold • accredit as satisfactory, good, excellent

NVAO’s overall approach (1/2) • Institutional audits • Initial accreditation • (New) programmes that want to offer a recognised degree • All bachelor and master’s programmes, including associate degrees • Accreditation • programmes that already offer a recognised degree • Publication of decisions (and officially recognised degrees)

NVAO’s overall approach (2/2) • Audit • Focus on policy & practice re. internal QA system teaching and learning Official register Publication Institution Report • (Initial) accreditation • Concentrate on content (& focus on performance) learning outcomes Programme Report

Institutional audit 5. Does the institution have an effective organisation and decision-making structure regarding the quality of its programmes? 4. How can the institution demonstrate that it systematically improves the quality of its programmes? What is the vision of the institution regarding the quality of the education it provides & of the development of a quality culture? Does the institution have an appropriate policy to realise this vision? 3. How does the institution measure the degree to which this vision is realised? • Positive / Negative / Conditional

Institutional audit: composition audit panel Installed by NVAO At least 4 members, including 1 student The panel commands administrative (Board), educational and audit expertise, is acquainted with developments in the higher education sector at home and abroad, and is authoritative One of the members with Board expertise will act as chair The panel is independent (its members have had no ties with the institution over at least the past 5 years) Not part of panel: NVAO process co-ordinator and secretary (also independent of the institution)

Institutional audit: process • Conversation on management level • Institutional profile: results of previous accreditation • Critical reflection by HEI (max. 50 pages + appendices) • Site visit • 1st visit (1-2 days): exploration of issues, first interviews and feedback, and panel decision on topics for audit trails • 2nd visit (2-3 days; 2 to 4 weeks after 1st visit): issues emerging from the meetings & documents studied during 1st visit; audit trails; at the end feedback to HEI on overall judgement and considerations • Audit trails: panel follows the trail from the institutional level to the implementation of policy and/or the management of problems or vice versa

Institutional audit: decision-making • Judgement on standards: • Meets, does not meet or partially meets the standard • Overall judgement: • Based on its vision of the quality of the education provided, the board of the institution has implemented an effective quality assurance system, which enables it to guarantee the quality of the programmes offered. • Positive, negative or conditionally positive

Institutional audit: decision-making (2) • If positive judgement then NVAO approval valid for 6 years • If conditionally positive then explicit statement of relevant conditions and positive judgement within 1 year: • If conditions are met then NVAO approval for 6 years • If conditions are not met then approval expires • If negative judgement then approval is withheld for at least 3 years: • comprehensive programme assessments • additional assessments for those programmes which have already had limited assessment

Institutional audit: report • For each standard: • Findings • Considerations • Judgment • Improvement suggestions • Summary for wider audience • Report and decision is published on the site of NVAO

Institutional audit Has this institution undergone an institutional audit? Institution Is this institution recognised? Yes No Recognition procedure + decision - decision Yes No Limited programme assessment Extensive programme assessment Programme assessment

Programme assessment Is this programme on the register/ recognised? Limited programme assessment Extensive programme assessment Programme assessment (Lim./Compr.) accreditation procedure (Lim./Compr.) initial accreditation procedure Improvement period - decision + decision + decision - decision • Satisfactory - Good - Excellent Deleted from register Included on register

Programme assessment

Expert panel composition(1/2) • At least four members, one of whom is a student; • at least two authoritative subject-matter experts • At least one with teaching experience at relevant level; • aware of latest international developments in the discipline • expertise in the professional field (where applicable) • educational expertise • assessment or audit expertise;

Expert panel composition(2/2) • The panel is independent • no ties with the institution offering the programme for at least five years; • an independent, external secretary trained and certified by NVAO; • signed declarations of independence and codes of conduct; • panel composition and declarations of independence will be published and made public; • the parties involved in the assessment can report any matters that could affect the independence to NVAO.

Focus on learning outcomes Achieved? Achieved? Achieved? Achieved? … (ECTS) Course catalogue

Programme assessment: process • Who ? • New programmes: NVAO • Recognised programmes: evaluation organisation • Information dossier (max. 50 pages + appendices) • Site visit • Conversations with management, teachers, students, work field • Overall judgment and considerations • Short feedback to HEI

Programme assessment: decision-making • Judgement on standards: • New programme: Satisfactory, unsatisfactory • Recognised programme: Satisfactory, good, excellent, unsatisfactory • Overall judgement: • New programme: Satisfactory, unsatisfactory • Recognised programma: Satisfactory, good, excellent, unsatisfactory

Programme assessment: decision-making (2) • New programmes: • If positive judgement then NVAO approval valid for 6 years • If negative judgement: no approval • Recognisedprogrammes: • If positive judgement then NVAO approval valid for 6 years • If negative judgement: • Limited approval • Conditions for improvement • If conditions are not met approval expires

Programme assessment scale Generic (basic) qualityThe quality that in all reasonableness could be expected of a bachelor’s or master’s programme within higher education, and this from an international perspective. UnsatisfactoryThe programme does not provide generic quality. SatisfactoryThe programme provides generic quality. GoodThe programme is of notably higher quality than generic quality. ExcellentThe programme is of a quality very much above generic quality and fulfils an exemplary role for other relevant programmes.

Programme assessment: report • For each standard: • Findings • Considerations • Judgment • Improvement suggestions • Summary for wider audience • Report and decision is published on the site of NVAO

Internal NVAO workflow (recognised programmes) Application • Letter • Institutional details • Programme details • Expert panel report Allocation • By managing director • First handler=policy advisor • Second handler • Portfolio handler= executive board member Admissibility • Screening application • Panel report • All standards judged • Proof of payment • Completeness Analysis • Template for Analysis • Substantiation judgements • (Objective) findings • (Subjective) considerations • Quality panel report • ≠ quality programme Proposed decision • Decision NVAO • Procedure followed • Panel composition • Essential programme details • Substantiation decision • Status and period of validity • NVAO website • Netherlands: Centraal Register OpleidingenHoger Onderwijs (CROHO) • Flanders: HE Register Final decision Publication

Thank you for your attention a.vanneygen@nvao.netwww.nvao.net

NVAO’s general principles ? • Open and inclusive approach • One framework for all assessments in a given type of procedures • E.g. one assessment framework for all types of initial accreditation • Fit for purpose (~appropriate burden) • Respect for institutional autonomy • Internal quality assurance system • Assessment in line with institutional choices on the basis of the relevant framework • Programme level • Institutional level

NVAO’s framework principles • Generic, descriptive standards ≠ Narrow, prescriptive and highly formulated • Broad acceptance • Focus on what should be done (~fitness of purpose) ≠ How they should be achieved • Do not dictate practice • Focus on teaching & learning (~education) • Incl. academic standards ≠ Research • Incl. professional orientation ≠ Service to society

The self-evaluation report • Central documentation for External QA • Self-contained document – relatively brief • Basic information • Institution / programme details • Presents perspective shared by (internal) stakeholders • Addresses each standard in relevant framework • rationale, practice, ambition • Provide insight into strengths & weaknesses • Maximising strengths? • Converting weaknesses into strengths? • Mandatory annexes

Importance of learning outcomes (& ECTS) • Direct window on what programmes aims to do & what a programme actually does • Less focus on policy, process & procedures • But process & procedures are easier to assess • Facilitates international transparency & comparability • E.g. International benchmarking • Open to all types of teaching & learning • Technology Enhanced Learning / Distance learning • Work-based learning / Assessment of prior learning • Provides a common language for HE & stakeholders • Teaching & Learning

NVAO’s external quality assurance procedures