1 / 38

‘Research into Quality Assurance of Training Provision for the BANKSETA’ - November 9 th 2005 -

‘Research into Quality Assurance of Training Provision for the BANKSETA’ - November 9 th 2005 -. - Index - . The structure of this presentation is as follows …. The research map The challenges of the BANKSETA Quality Assurance from a process perspective

joshua
Télécharger la présentation

‘Research into Quality Assurance of Training Provision for the BANKSETA’ - November 9 th 2005 -

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ‘Research into Quality Assurance of Training Provision for the BANKSETA’ - November 9th 2005 -

  2. - Index - The structure of this presentation is as follows … • The research map • The challenges of the BANKSETA • Quality Assurance from a process perspective • BANKSETA’s view on the definition of Quality Assurance • Selection of findings and conclusions of the research • General conclusions on Quality Assurance Research MapSlide 2 ChallengesSlide 3 Frame work Slide 4-12 Definition Slide 13-17 Findings Slide 18-35 Conclusions? Slide 36

  3. In depth view Fit to challenges Implementation Acceptance - + Definition - Research map - The research followed the following steps, partly in an iterative way … Challenges BANKSETA Relevance (Filter) ‘Practice’ 1 Relevant Practice 1 Practice 2 Literature research Framework ‘Practice’ 2 Practice 3 Legislation / ETQA functions ‘Practice’ .. Interviews Information & Sources ‘Practice’ X Challenges Country Sector Legislation Initial research Best Practice

  4. - Challenges BANKSETA - A summary of the main challenges that the BANKSETA was facing at the start of the research … • Accreditation: What is a user-friendly process for service providers and is it accessible and manageable for small providers? • Programme evaluation/approval: Should this be done and if so, what is the most effective and efficient way to do so? • Overlap: Certain scenarios result in overlaps with other SETAs with different systems in use. How can potential conflict in roles and responsibilities be avoided or resolved? Are MoUs the appropriate mechanisms? • Audits: What is the best combination of internal and external audits and how can the internal audit process be streamlined?

  5. Programme Development- SP - Certification/ Data recording- ETQA - Needs Analysis- SP / Empl - Learning- SP - Assessment- Assessor - - Framework - The Primary Process: The Learning Value Chain     Primary • Characteristics • Volume driven process steps. The more programmes and the more learners, the more often these steps have to be executed • Steps are part of the Learning Value Chain. • If the activities are not (well) executed this has a direct effect on the Quality of Learning.

  6. Certification/ Data recording- ETQA - Programme Development- SP - Assessment- Assessor - Learning- SP - Needs Analysis- SP / Empl - - Framework - Secondary Process: How do we check the Quality of the primary process?     Primary Programme review Learning Evaluation Moderation Internal Secondary Programme review Learning Evaluation Moderation External • Characteristics • Volume driven activities. The more programmes and the more learners, the more often these steps have to be executed • Activities look directly into the steps of the Learning Value Chain. • If the activities are not (well) executed this has an indirect effect on the Quality of Learning.

  7. Programme Development- SP - Certification/ Data recording- ETQA - Needs Analysis- SP / Empl - Learning- SP - Assessment- Assessor - - Framework - Tertiary Process: The capacitating of the entities involved in the primary and secondary processes. Who do we need to capacitate?     Primary Programme review Learning Evaluation Moderation Internal Secondary Programme review Learning Evaluation Moderation External 3 The Moderator The ETQA registers Moderators. 1 Service Provider The ETQA accredits, audits and empowers Service Providers 2 The Assessor The ETQA registers Assessors.

  8. - Framework - Tertiary Process: How is the right capacity of Service Providers, Assessors and Moderators ensured by the ETQA? • Characteristics • Activities are not volume driven. Irrespective of the number of programmes or learners, the activities will take place according to a certain frequency. • Activities do not look directly into the steps of the Learning Value Chain but check the capacity of the different entities involved in executing the steps of the Learning Value Chain and the Secondary (QA) processes connected to it. • If the activities are not (well) executed this has an indirect effect on the Quality of Learning. In take Develop Audit Service Provider Internal Audit External Audit Monitoring visits Workshops Accreditation Tertiary Assessor Forums Workshops ? Registration Moderator Forums Workshops ? Registration

  9. Certification Data recording- ETQA - Programme Development- SP - Assessment- Assessor - Learning- SP - Needs Analysis- SP / Empl - - Framework - The total model is made up of Primary, Secondary and Tertiary Processes all working together to deliver quality for the learner.     Primary Programme review Learning Evaluation Moderation Internal Secondary Programme review Learning Evaluation Moderation External In take Develop Audit Service Provider Internal Audit External Audit Monitoring visits Workshops Accreditation Tertiary Assessor Forums Workshops ? Registration Moderator Forums Workshops ? Registration

  10. - Framework - The first six of the ten ETQA functions can clearly be linked to the framework and fall within the scope of the QA research. ETQA functions linked to framework Accrediting Providers Promoting Quality amongst Providers Monitoring provision Evaluating assessment and facilitating moderation amongst providers Registering assessors Certificating learners ETQA functions outside the framework Co-operation with relevant moderating bodies Recommending new standards or qualifications (or modifications to existing standards and qualifications) to SGBs Maintain a database Submitting reports to SAQA

  11. - Framework - All the ETQA functions are linked to three streams of the Training & Quality Assurance process 6 Certificate learners 4 Evaluating assessment and facilitating moderation amongst providers 3 Monitoring Provision 1 Accrediting Providers 2 Promoting Quality amongst Providers 5 Registering Assessors 4 Evaluating assessment and facilitating moderation amongst providers

  12. - Framework - Legislation indicates the following guidelines for the Quality Assurance processes …

  13. - Framework - Currently the BANKSETA has the following Quality Assurance processes in place …

  14. - Definition - The more we focus on the BANKSETA itself, the more the ‘general knowledge’ needs to be extrapolated to the BANKSETA • Quality Assurance • Quality Assurance in training • Quality Assurance for SAQA • Quality Assurance for the BANKSETA Research Discussion Scope Precision

  15. - Definition - The General Definitions of Quality Assurance …. • ISO 9000:2000 • “Part of quality management focused on providing confidence that quality requirements will be fulfilled.” • Six Sigma • “Work done to ensure that Quality is built into work products, rather than Defects.” This is by (a) identifying what “quality” means in context: (b) specifying methods by which its presence can be ensured; and (c) specifying ways in which it can be measured to ensure conformance. • Q-Africa 2004 Conference Paper • “The sum of activities/elements that assure the quality of products and services.”

  16. - Definition - Quality Assurance for training is considered … • CHE • “The underlying objective of the HEQC with regard to quality assurance is: to ensure that institutions effectively and efficiently deliver education, training, research and community service which are of high quality and which produce socially useful end enriching knowledge…” • Engineering Council • “Quality assurance: in general, a process which examines the quality of a process and its product; when applied to an education and training programme, evaluating the programme and its assessment against standards using defined policies and procedures.” • Higher Education - Australia • “(The) systematic management and assessment procedures adopted to monitor performance and achievement and to ensure achievement of specified quality or improve quality.” (Harman & Meek, 2000, p. 11)

  17. - Definition - The view of SAQA and the SETAs … • SAQA’s definitions • “Quality Assurance means the sum of activities that assure the quality of products and services at the time of production or delivery. Quality assurance procedures are frequently applied only to the activities and products associated directly with the goods and services provided to external customers” • “Quality Assurance means the process of ensuring that the degree of excellence specified is achieved” • “Implicit in SAQA’s implementation of the quality system is the understanding that quality assurance, quality management and accreditation are not things or products; rather, quality is a process.” (Quality is an ongoing event)

  18. Certification Data recording- ETQA - Programme Development- SP - Learning- SP - Assessment- Assessor - Needs Analysis- SP / Empl - - Definition - What does this mean specifically for the BANKSETA? • The extrapolation to a definition of quality assurance for the BANKSETA: • = = = = = = • “Quality assurance refers to the integrated processes that evaluate the provision and outcomes of education, training and development, so that compliance with SAQA requirements is achieved and systems for continuous improvement are in place for the benefit of learners.” • = = = = = =     Primary Outside Scope In Scope

  19. - Findings: Primary Process - The following sources gave input to the findings • Literature • General literature on Quality Assurance • Skills Development Legislation (National & International) • SAQA documentation • ETQA Guidelines and Policies • Interviews • A total of 28 persons were interviewed representing the following groups: Employers, Service Providers, ETQA managers, SAQA officials and independent experts • Practices • 5 ETQAs were researched • 3 International Practices were researched (Aus, New Zealand, Malaysia/Scotland)

  20. Programme Development- SP - Certification / Data recording- ETQA - Needs Analysis- SP / Empl - Learning- SP - Assessment- Assessor - - Findings: Primary Process - Practices of the Primary process     Primary Programme Approval by ETQAs 1) Yes, including SME and review committee 2) Yes, mainly as part of Accreditation 3) Yes, mainly as part of Accreditation 4) Yes, mainly as part of Accreditation 5) Yes, mainly as separate process, no SME needed Programme Approval International Australia: Use of nationally endorsed Training Packages Programme ≠ Material Material ∈ Programme +

  21. Programme Development- SP - Certification / Data recording- ETQA - Needs Analysis- SP / Empl - Learning- SP - Assessment- Assessor - - Findings: Primary Process - The view of experts on the Primary processes     Primary ETQAs: Programme approval is … - Very important - Not so important - SME needed - SME not needed Employers: “Huge problem with time that assessments take” Scope issue ETQAs: “There is a risk in the Employer – Assessor relationship” Scope issue Service Providers: “Too much focus on Assessment and not on Training Needs affects the performance negatively” “SPs tend more toward measuring business impact of training to improve themselves” Employers: “Business Impact will inform the quality better than assessments” Independent Experts: “We don’t know how to measure the impact of the NQF and with that the ETQA function”

  22. - Findings: Primary Process - Programme Approval is a gate keeper for the end result, Subject Matter Expertise can be applied depending on the context … E X A M P L E S • Approval by committee Safety Thoroughness needed in Programme Approval • Material correct according to SME • Aligned with US according to SME Potential Impact of errors Compliance Possibility of relying on self regulatory function of the market Banking Sector • Aligned with US Service • Outcomes based Turn around time from sector in case of incorrect content of programme

  23. Programme Development- SP - Certification / Data recording- ETQA - Needs Analysis- SP / Empl - Learning- SP - Assessment- Assessor - - Findings: Primary Process - Conclusions on the Primary process     Primary 1 “Garbage in is Garbage out” 2 “Performance Management is needed” 3 “Needs analysis falls outside the scope”

  24. Programme Development- SP - Certification / Data recording- ETQA - Needs Analysis- SP / Empl - Learning- SP - Assessment- Assessor - - Findings: Primary Process - Way forward on the Primary process     Primary 2 1 3 • Suggestions • Introduce a gate keeper in the form of Programme Approval:- Minimise the use of Subject Matter Expertise- Take samples based on ‘smart’ selection • Do further research into whole process of assessments. Is it possible to address the challenge of the sector? • Use the Performance Auditing that will be introduced by SAQA to start measuring the performance of the ETQA function from a client perspective (the learner). This will enable continuous improvement of the ETQA processes.

  25. - Findings: secondary process - Practices of the Secondary process Programme review Learning Evaluation Moderation Internal Secondary Programme review Learning Evaluation Moderation External Combined with Approval Does not exist Incidentally considered Moderation by ETQAs Is done incidentally, policy to minimise it. Is not done External moderation is done Moderation International Aus: No secondary processes are performed Internal checks Mostly directly or indirectly covered in Accreditation / QMS criteria

  26. - Findings: secondary process - Experts on the Secondary process Programme review Learning Evaluation Moderation Internal Secondary Programme review Learning Evaluation Moderation External Experts: “Quality Assurance Mechanisms are being set up for the sake of compliance rather than to improve Quality” Independent Experts: “Moderation and Program Evaluation should be built-in systems and part of self evaluation” Independent Experts: “Too much emphasis on assessment and moderation, we should be putting systems in place” SAQA: “The main degrees of freedom that occur where overlap between ETQAs take place, lie in the Secondary Processes” Service Provider: “Let me pass the (accreditation) mark and let me then run my own processes”

  27. - Findings: secondary process - The overlap between different SETAs leaves us with four combinations with many more possible applications (1) … • Service Provider Qualification Sector • Normal Cases • Case 1 I a, b, c A • Case 2 II i, j, k B • Case 3 III x, y, z C • Special Cases • Extra-sectoral I a B • Extra-qualification I i A • Extra-sectoral/qualification I I i B • Extra-sectoral/qualification II I x B

  28. - Findings: secondary process - The overlap between different SETAs leaves us with four combinations with many more possible applications (2) … Service Provider Qualification Sector/ETQA Special Cases 1. Extra-sectoral I a B 2. Extra-qualification I i A 3. Extra-sectoral/qualification I I i B 4. Extra-sectoral/qualification II I x B Are these activities done? Who is doing them? Can they be outsourced to another ETQA?

  29. Programme Development- SP - - Findings: secondary process - Conclusions on the Secondary process Programme review Learning Evaluation Moderation Internal Secondary Programme review Learning Evaluation Moderation External 1 ° Freedom “MoU challenge = secondary processes” Approval “Programme Review = Secondary Process”“Programme Approval = Primary Process” 2 Learning- SP - Programme Review 3 Policing? / Redoing? / Empowering? “Promotion of a Culture of Quality?”

  30. - Findings: secondary process - Way forward on the Secondary process Programme review Learning Evaluation Moderation Internal Secondary Programme review Learning Evaluation Moderation External • Suggestions • Minimise the effort put into Secondary Processes • Rather focus on Tertiary Processes that enable Service Providers to perform internal Secondary Processes • Work in conjunction with SAQA towards clarity concerning the degrees of freedom that occur around the Secondary Processes in case of overlap between ETQAs

  31. - Findings: Tertiary Processes - Practices of the Tertiary process In take Develop Audit Service Provider Internal Audit External Audit Monitoring visits Workshops Accreditation Tertiary Assessor Forums Workshops ? Registration Moderator Forums Workshops ? Registration ETQAs - Accreditation Provisional option often usedAll different categories / Criteria within SAQA framework International AccreditationAus: larger scope, includes advertising and marketing Can, Scotland: incl. e.g. Marketing and Needs Analysis ETQAs - Auditing Internal Audit not always applicable Mostly aligned with Accreditation Mixed with Secondary Processes International AuditingAus: 1-1 alignment with Accreditation

  32. - Findings: Tertiary Processes - Experts on the Tertiary process In take Develop Audit Service Provider Internal Audit External Audit Monitoring visits Workshops Accreditation Tertiary Assessor Forums Workshops ? Registration Moderator Forums Workshops ? Registration ETQAs: “Accreditation is the key-element to ensure performance” Service Provider: “Internal Audits can be a helpful tool to create an internal platform for change and performance improvement” General: “It is possible to get accredited and not perform” Independent Experts: “Accreditation process is often not fast enough, results in bottlenecks” Service Provider: “There is not always a clear link between Accreditation Criteria and how it can positively affect business processes” Service Provider: “Accreditation is still a cumbersome process”

  33.    Certification / Data recording- ETQA - Program Development- SP - Needs Analysis- SP / Empl - Assessment- Assessor - Learning- SP - Taking Corrective Actions Act Planning the Primary Process Checking the results of the Primary Process Plan Check Do Primary Processes - Findings: Tertiary Processes - Although alignment is important, the Demming wheel suggests an accent on the “Act” phase when performing external audits Accreditation Int. Audit Ext. Audit = What are the corrective actions based on Plan, Do, Check? What is the business planning in terms of Programme Focus? What is the procedure for Programme Evaluation? What is the procedure for Programme Development?

  34. Programme Development- SP - Certification/ Data recording- ETQA - Needs Analysis- SP / Empl - Learning- SP - Assessment- Assessor - - Findings: Tertiary Processes - The categories for accreditation/auditing are informed by the Primary and Secondary processes where the SP is involved. Primary     4 1 2 3 Programme review Learning Evaluation Moderation Internal 5 6 7 Secondary 8 e.g. overhead processes SAQA 9 … Sector Specific x

  35. - Findings: Tertiary Processes - Accreditation and Audit Criteria could be summarised and structured as follows … • General QMS requirements, e.g.- Management responsibility- Follow-up systems • Specific Policies, Procedures and Processes, e.g.- Core Programme Development, Programme Review (+ requirements) Provision, Learning Evaluation (+ requirements) Assessment, Moderation - Overhead HR (+ requirements) Financial (+ requirements) … • Static Criteria, e.g.- Admin and Financial Capacity, Legal Entity, …

  36. - Findings: Tertiary Processes - Conclusions on the Tertiary process In take Develop Audit Service Provider Internal Audit External Audit Monitoring visits Workshops Accreditation Tertiary Assessor Forums Workshops ? Registration Moderator Forums Workshops ? Registration 1 “thorough processes are needed” 2 “simple and aligned processes”

  37. - Findings: Tertiary Processes - Way forward on the Tertiary process In take Develop Audit Service Provider Internal Audit External Audit Monitoring visits Workshops Accreditation Tertiary Assessor Forums Workshops ? Registration Moderator Forums Workshops ? Registration • Suggestions • Establish rhythm of Accreditation, Internal Audit and External Audit • Align accreditation 1-on-1 with Audits and site visits • Focus on a Culture of Quality that goes beyond compliance. Look for ‘real’ evidence in the “Act” phase • The strength of the sector suggests that Provisional Accreditation is currently not a necessity. Rather focus on empowerment.

  38. - Conclusions? - The following bullet points remain open for discussion … • Can one size fit all? • Is a sector specific model the route to follow? • Given the degrees of freedom, is it possible to make sensible comparisons? • How close is Best Practice to Common Practice?

More Related