Total quality management based on self-assessment

1. Total quality managementbased on self-assessment Andre Vyt The experience with in Flanders PARTNERS IN QUALITY MANAGEMENT

2. PROZA exists in different versions • Higher education • Secondary education • Vocational training for handicapped persons • Adult education & training centers • Health Care & Social care centers

3. The goal of is • to diagnose strengths and weaknesses of all relevant aspects in the organization • inspired by an all-encompassing and recognized framework (EFQM Excellence model) • by the process-owners (lecturers, researchers, administrators, technicians ...) in quality teams • together with all parties involved: supervisors, colleagues, and clients (students, qualified professionals…) • to set up well-supported specific actions for improvement

4. Inspired by EFQM Excellence Basic conditions Core processes Results

5. The 9 areas of attention Policy and strategy Resources and co-operation Leadership Core processes Student satisfaction Human resources policy Personnel satisfaction Appreciation by society Performance of the organization

6. The content of PROZA • It contains more than 60 focal points over the 9 areas of attention • In a focal point 25 specific items are to be checked/answered • The 25 items are grouped into 5 phases, according to the level of quality management

7. Areas and points of attention

8. Focal points in core processes • Admissions coaching • Communication with students • Curriculum design • Design of course units • Implementation of the curriculum • Teaching and tutoring • Student coaching

9. Focal points in core processes • Practice-based learning • Internationalisation • Testing and assessment • Graduation and alumni-oriented activities • Postgraduate education • Public service provision • Research • Development and practice of arts

10. Quality is variableQuality is linked to the individual Initiation of process-based thinking/actingStarting to work systematically (High level of) ProfessionalismQuality is guaranteed Systematic innovationContinuous improvement Aiming for excellenceRecognition of expertise and high level of quality The 5 phases of development

11. do evaluate - control plan check act improvement assurance time The PDCA-cycle of QM (Deming) quality

12. The higher the level attained,the better the • Formulation and management of processesMeasurements and information gathering • Systematisation and reliability Openness and transparency • Attunement and integration Innovation and improvement • Involvement of different persons Client and external orientation

13. 1: Quality linked to the individual • Quality is the result of individual effort • Short-term thinking • Problems are resolved when they arise • Limited exchange of information • Lack of follow-up commitments • A minimum of quality is present

14. 2: Initiating process-based thinking/acting • Formulation of needs and priorities • Overall commitments enforceable to a limited extent • Increasing attention to information management • Real causes of most problems are known • Performance is evaluated regularly • Improvement is clearly noticeable

15. 3: (High level of) Professionalism • Planned approach • Availability of reliable data • Binding by written commitments • Continuity of service is guaranteed • Level of quality achieved is supported by assessment data • Information flow is effective

16. 4: Systematic innovation • Systematic analysis and assessment of operations • Use of quality indicators as a policy instrument • Structural problem detection & handling • Innovative thinking • Emergence of synergy effects

17. 5: (Aiming for) excellence • Frames of reference, indicators and working methods internationally recognized as best practice • Broad base of support for moving beyond current levels of quality • All quality indicators are moving in a positive direction • Large number of personnel are internationally recognized experts

18. E.g., the focal point “course units” has items such as: • Do lecturers know the function of their course unit in the curriculum? • Do colleagues work together in a team to design course components? • Is the design of nearly all course units geared to the frame of reference of the department? • Are goals of improvement defined on a yearly basis for all course units? • Is the design of specialized course units entrusted to experts?

19. Testing and assessment • Do all students receive a copy of the exam regulations in time? • Is there an attempt to safeguard objective assessment by involving more assessors? • Are assessment methods for each course component clearly geared to the objectives? • Are assessment methods systematically adjusted following evaluation? • Is the quality of the assessment methods regularly tested by external experts?

20. Student admission coaching

21. Answering the questions • Answering categories: yes / no / not applicable / impossible to answer • Lack of measurement data provides relevant information on the need to professionalize the management of information • Concrete questions provide immediate goals of improvement

22. The method used in PROZA • The checklists are filled out individually by the team members • They gather to agree in consensus on the scores of the items and deduce the level of quality management • but also to identify immediately specific goals of improvement • that are selected according priority and written down in a self-evaluation report

23. Score (in consensus) • Actual level: phase within which 3 items are positive • Potential or growth level: phase within which 1 item is positive • … each time on condition that in previous phases at least 3 items are positive

24. Chart of scores within an area

25. Chart of scores over the 9 areas

26. The PROZAself-assessment process • Preparation • Diagnosis • Improvement

27. Preparation • Policy decision to use PROZA and formation of a steering group • Getting people on board by engagements and good communication • Initial screening with quickscan to determine which areas (and foci) to address • Formation of self-assessment groups • Agreement on working method and prior clarification of terminology and gradation words

28. Screening with the quickscan • 200 statements that probe the overall level of quality (20 per area, 40 for core processes) • To be answered with • Adequate (A) • Must be improved / not adequate (NA) • Important (I) • Not so important (NSI) • Preferably by the steering group or key persons

29. The Quickscan

30. The Quickscan

31. Carrying out the diagnosis • Relevant information on the foci can be made/kept available • Gaining a view on hot issues by an inventory of individual scores before the meeting • Value of individual comments on the questions • Short-circuiting endless discussions: • Right/wrong is less essential than identifying improvement • Eventual agreement on skipping items on which only one member answered negatively

32. From diagnosis to improvement • Diagnosis and proposals for improvement are formulated on a report form • Assignment of a priority value • Definitive selection of ultimate priorities by steering group or management • For each selected goal, an action plan must be drawn up on a predefined form by an improvement team

33. Work sheets in PROZA

34. Action plan • Concrete goal • Responsible person • Actions that have to be taken • Persons involved • Time plan • Expected results / deliveries • Budget

35. Client-orientation of PROZA • Evaluation criteria and items were tested on content and applicability in different HE institutions. • Assessment questions have grown out of practical experience and are based on a consensus between TQ managers • The checklists are accompanied by a users guide and a list of concepts • The checklists are delivered electronically, so that institutes can adjust words to match the structure of the organisation

36. Some pro’s of PROZA • Exhaustive and flexible: separate areas and focal points can be analysed for different units in the organisation • It is easy to deduce questionnaires • More than 1000 items result in very specific goals of quality improvement • Based on individual analysis and on consultation with others in teams

37. Links between self-evaluations and external quality assurance Self-evaluation Themes for audits Quality management policy Human resources Means and finances Learning goals Programme content and organization Outside school practice training International policy and activities Training methods and tools Assessment and examination Counseling and guidance Study load and effectiveness Research, arts and societal services Graduates

38. Implementation requirements • Improvement as the final goal: diagnosis and scoring is only the starting point. • Explicit policy and engagement on all levels of the management. • Agreement on aspects of confidentiality and openness of reports • Explicit rules about implementation of selected goals, follow-up and feedback

39. Avoiding pitfalls • If not properly introduced and coached, it may be perceived and labeled as a mass of paperwork which demands more energy than that it should help to improve quality. • If not complemented by a leading and steering management, self-evaluation may bog down in the diagnostic process. • If the instrument is put forward as to be used primarily on teaching processes , it may result in adverse effects of “shooting the piano-player” instead of improving the piano.

40. Guaranteeing the achievementof goals • Diagnoses by self-evaluation teams can exist alongside the management instead of in the heart of the management, leading to quality management on paper instead of effective improvement in reality • Avoid misuse by management to “solve” management problems, or to evaluate employees. • Diagnoses must not only result in clear objectives of improvement but must also be made concrete in realistic project planning according to SMART rules.

41. Relevant Understandable Measurable Behaviorable Attainable Specific Measurable Agreed/acceptable Relevant/realistic Time-related Criteria for project goals

42. For further information… www.prose.be www.prose.be Info@prose.be