QUALITY INDICATOR

1. QUALITY INDICATOR • 22 Mac 2015

2. Quality Indicators Indikator Kualiti dalam Pendidikan Tinggi(Quality Indicators in Higher Education) Learning outcomes: Explain the meaning of quality indicators Describe the characteristics of quality indicators Synthesis the importance of quality indicators in HEI Explain studies related to the application of quality indicators in HEI

3. INDICATORS • Customer satisfaction • Performance

4. Teras PSPTN • Meluaskan akses dan meningkatkan ekuiti; • Menambah baik kualiti pengajaran dan pembelajaran; • Memperteguh penyelidikan dan inovasi; • Memperkasakan institusi pengajian tinggi; • Mempergiatkan pengantarabangsaan; • Membudayakan pembelajaran sepanjang hayat; dan • Mengukuhkan sistem penyampaian KPT.

5. Quality Indicators Aspects of quality in higher education Three aspects: Client quality – what students and employers want from the service. Professional quality – whether the service meets needs as defined by professional providers and whether it carries out techniques and procedures which are believed to be necessary to meet clients needs. Management quality – the most efficient and productive use of resources within limits and directives set by higher authorities. Ovretveit (1992)

6. Quality Indicators Judging quality in higher education Three categories: Academic standards of courses – what the students demands of the students, the extent it meets staff needs. Teaching quality – staff training, appraisal systems, teaching evaluation, reports from external examiners, patterns of employment of graduates, students feedback. Student achievement – completion rates, class of degrees obtained. O’Neil (1994)

7. Quality Indicators Issues to be addressed The need for competent staff Management responsibility Quality policy Quality manuals Quality planning

8. Quality Indicators Performance cultures in higher education HEIs worldwide have undergone reform to improve quality HEIs have implemented systematic and formalized quality assurance processes to achieve greater efficiency and accountability Establishment of quality models and organizations to audit and review university performance Institutional and national quality models and performance indicators are vital components to raise the standard of HEIs Quantitative performance indicators are used to provide international comparisons

9. Quality Indicators Rationale for performance indicators To ensure education provided by HEIs equips students for employment and provide the country with a highly skilled workforce that support economic growth. To contribute to educational, social, and political values.

10. Quality Indicators Purposes of performance indicators in HEIs To monitor own performance for comparative purposes. To facilitate the assessment and evaluation of institutional operations. To provide information for external quality assurance audits. To provide information to the government for accountability and reporting purposes (Rowe, 2004).

11. Quality Indicators The use of performance indicators in HEIs Ensure accountability for public funds Improve the quality of higher education provision Stimulate competition within and between institutions Verify the quality of new institutions Assign institutional status Underwrite transfer of authority between the state and institutions Facilitate international comparisons Ref: Chalmers (2008)

12. Quality Indicators Defining performance indicators Simple indicators – expressed in absolute figures and are intended to provide an unbiased description of a situation or process. Performance indicators – imply a point of reference; for example, a standard, objective, assessment, or comparator, are relative rather than absolute in character. Involve value judgements. General indicators – externally driven and are not indicators in the strict sense; they are frequently opinions, survey findings or general statistics. Ref: Chalmers (2008, quoted from Hanney, Henkel & Kogan, 1997)

13. Quality Indicators Defining performance indicators Currently there is no common definition of performance indicators. PI cannot be considered as facts, but are goal, value and context laden, and utilized in different ways depending on the performance model being used. PI are defined as measures which give information and statistics context; permitting comparisons between fields, over time and with commonly accepted standards. They provide information about the degree to which teaching and learning quality objectives are being met within higher education sector and institutions. Ref: Chalmers (2008, p.3)

14. Quality Indicators Types of performance indicators Input Process Output Outcome The types can be categorized as quantitative indicators and qualitative indicators. Ref: Chalmers (2008, p.3)

15. Quality Indicators Quantitative Indicators define as those associated with the measurement of quantity or amount, and are expressed as numerical values. Input indicators Human, financial and physical resources in supporting institutional programmes, activities and services. Output indicators Output reflects the quantity of outcomes produced, including immediate measurable results, and direct consequences of activities implemented to produce results. Do not demonstrate quality of education, but quantities of outcomes. Ref: Chalmers (2008)

16. Quality Indicators Qualitative Indicators associated with observation based descriptions, rather than an exact numerical measurement or value. Relate to or involve comparisons based on qualities of non-numerical data such as policies and processes for assessing students’ learning, the experience, the content of a mission statement. Outcome Indicators Focus on the quality of educational program, activity and service benefits for all stakeholders. Insightful, meaningful and accurate since they are related to the objectives of higher education. Students are treated as customers. Ref: Chalmers (2008, p.5)

17. Quality Indicators Qualitative Indicators Process Indicators include the means used to deliver educational programmes, activities and services within the institutional environment. qualitative information on teaching and learning such as policies and practices. Considered as most practical, useful and appropriate measures of quality teaching and learning. Ref: Chalmers (2008)

18. Quality Indicators Research conducted in Australia Look at 13 process indicators Mission, Vision and Objectives Teaching and Learning Plans and Policies Teaching and Learning Indicators Internal and External Performance Funds for Teaching and Learning Organizational Unit Review Curriculum Review Assessment and Feedback Policies Graduate Attribute Statement Student experience Ref: Chalmers (2008)

19. Quality Indicators Research conducted in Australia Look at 13 process indicators Professional Development Appointment and Promotion Criteria Review of Academic Staff – performance Recognition of Excellence in Teaching and Enhancing Student Learning Experience Ref: Chalmers (2008)

20. Quality Indicators Research in Australia – Quality Teaching Look at four dimensions of teaching practice Institutional climate and systems – commitment to the enhancement, transformation and innovation of learning. Measure student experience and level of satisfaction. Diversity – relates to ethnic, cultural and socioeconomic diversity as well as students’ abilities, talents and learning approaches. Ref: Chalmers (2008)

21. Quality Indicators Research in Australia – Quality Teaching Look at four dimensions of teaching practice Assessment – the assessment tasks of student enrolled in the program of study. Look at the design, delivery and administration, provision of feedback, moderation, and review of assessment. Engagement and learning community – student engagement, i.e. the student commitment and engagement with their own education. Also includes staff engagement. Ref: Chalmers (2008)

22. Study in Hong Kong and China • Seven factors contributing to the quality of education.They are: • 1 course content; • 2 concern for students; • 3 facilities; • 4 assessment; • 5 medium of instruction; • 6 social activities; and • 7 people.

23. Course content items • The chance that your study fulfils your personal needs • The appropriateness of requirements for your course • The chance to develop your abilities and prepare for your career • The quality of material emphasized in course • The usefulness of the module components offered in your career development • The usefulness of the module components in fulfilling your personal needs

24. Concern for student • The availability of advisers from whom students can seek help • The interest that student advisers take in the progress of their students • The ability of advisers to help students develop their course plan • The willingness of the university management to take the opinions of students • The channels for reflecting students’ ideas to the management

25. Facilities • The availability of quiet places to study in the university • The cleanliness of most facilities used by students • The amount and availability of library facilities • The places provided for students to relax and lounge during the day • The amount and availability of computing facilities • The amount and availability of sports and recreational facilities

26. Assessment • The chance that you do well if you work hard • The appropriateness of the standard of modules offered • The appropriateness of the assessment system • Detailed lecture notes are distributed • The amount of work required in most modules • The amount of time you must spend studying to get a passing grade • The likelihood of getting distinction if you work hard

27. Medium of instruction • Lectures be conducted in Language understood by students • Tutorials be conducted in language understood by students

28. Social activities • The activities and clubs you can join in the university • The social events that are provided for students in the university

29. People • The friendliness of students and the opportunity to make close friends • The chances to meet people with the same interests as you have

30. Conclusion of the study by Kwan & Ng, 1999 • Hong Kong and Chinese students are very practical and only focus on study-related matters rather than social life in campus. • Students in the States are more interested in campus life but the importance of facilities has not been mentioned. • It seems that Hong Kong and Chinese students regard university education as an investment and thus stress course content and facilities.

31. DISCUSSION • Jelaskanelemen yang dipiliholehandasebagaipelajaruntukdinilaidalamkontekskepuasanpelanggan. • Apakriteria yang digunauntukpenilaianindekskepuasanandasebagaipelanggan?

32. Quality Indicators Higher Education in Malaysia– Challenges Peranan universiti dan ahli akademik (University’s and academics’ roles) Perkembangan kurikulum mengikut keperluan pasaran (Curruculum development according to market needs) Penyelidikan, pembangunan dan pengkormesialan dalam sistem inovasi kebangsaan (Research, development and commercialization in the national innovation system) Kaedah pengajaran dan pembelajaran (Teaching and Learning Methods) Perluasan akses dan mobiliti pengetahuan (Accessibility and knowledge mobility)

33. Pekeliling Kemajuan Pentadbiran Awam Bilangan 2 Tahun 2005 • Garis Panduan Bagi Mewujudkan Petunjuk-petunjuk Prestasi Utama Atau Key Performance Indicators (Kpi) Dan Melaksanakan Pengukuran Prestasi Di Agensi Kerajaan

34. Terminologi • Petunjuk Prestasi Utama (KPI) ialah salah satu kaedah bagi mengukur prestasi perkhidmatan agensi-agensi Kerajaan • Perkhidmatan Teras Bidang tanggungjawab agensi sejajar dengan visi (core business): dan misi agensi; • Proses Utama Fungsi-fungsi di bawah perkhidmatan teras yang (core process): perlu dilaksanakan bagi menghasilkan perkhidmatan untuk pelanggan; • Key Performance Petunjuk-petunjuk prestasi utama yang ditentukan Indicators (KPI): sebagai asas mengukur prestasi;

35. NKRA • Education • Crime (Public Safety) • Corruption • Low Income Households • Rural Basic Infrastructure • Urban Public Transportation

36. Quality Indicators Higher Education in Malaysia– Challenges Globalisasi dan piawaian melalui pemeringkatan dan penarafan (Globalization and standardization through development and accreditation) Peluang guna tenaga (Resource utilization opportunities)

37. Quality Indicators Malaysia - Current scenario 20 public universities 21 polytechnics 37 community colleges >400 registered private colleges 21 private universities and university colleges 11 local university branch campuses + 5 foreign university branch campus Areas for indicators: Academic staff Educational programs Student selectivity Educational resources Governance The method used will be peer review.

38. Quality Indicators How are universities ranked? Different ranking approaches: League table – each university is assigned a specific rank. Higher ranks indicate higher quality, lower ranks indicate lower quality. Quality criteria and indicators are used in this ranking methodology to assess universities. Each indicator such as research impact as the number of citations per faculty in the Thompson Scientific Database or teaching quality as in THES are given weight. This approach are applied to all universities assessed.

39. Quality Indicators How are universities ranked? Different ranking approaches: A ranking of individual disciplines or departments instead of whole institutions. A multidimensional concept of university quality instead of a ‘one-size-fits-all’ approach, taking into account the diversity of academic institutions, missions and goals as well as language and cultural specifics. A separate measurement and presentation of single indicators. A presentation of ranking results in rank groups (top, middle, bottom groups) instead of league tables.

40. Quality Indicators Purposes of ranking of HEI Ranking serves several purposes: Responds to demands from consumers for easily interpretable information on the standing of HEIs. Stimulates competition among universities. Provides some rationale for allocation of funds. Helps to differentiate among different types of institutions and different programs and disciplines. Contributes to the definition of ‘quality’ of HEIs within a particular country. (source: Berlin Principles of Ranking of Higher Education Institutions, 2006 http://www.che.de/downloads/Berlin_Principles_IREG_534.pdf)

41. Quality Indicators Berlin Principles of Ranking of Higher Education Institutions Purposes and goals of rankings: Be one of a number of diverse approaches to the assessment of higher education inputs, processes, and outputs. Be clear about their purpose and their target groups. Recognize the diversity of institutions and take the different missions and goals of institutions into account. Provide clarity about the range of information sources for ranking and the messages each source generates. Specify the linguistic, cultural, economic, and historical contexts of the educational systems being ranked – should be aware of possible biases. (source: Berlin Principles of Ranking of Higher Education Institutions, 2006)

42. Quality Indicators Berlin Principles of Ranking of Higher Education Institutions Design and weighting indicators: Be transparent regarding the methodology used for creating the rankings. Choose indicators according to their relevance and validity. Measure outcomes in preference to inputs whenever possible. Make the weights assigned to different indicators (if used) prominent and limit changes to them.

43. Quality Indicators Berlin Principles of Ranking of Higher Education Institutions Collection and processing of data: Pay due attention to ethical standards and the good practice recommendations articulated in these Principles. Use audited and verifiable data whenever possible. Include data that are collected with proper procedures for scientific data collection. Apply measures of quality assurance to ranking processes themselves. Apply organizational measures that enhance the credibility of rankings.

44. Quality Indicators Berlin Principles of Ranking of Higher Education Institutions Presentation of ranking results: Provide consumers with a clear understanding of all of the factors used to develop a ranking, and offer them a choice in how rankings are displayed. Be compiled in a way that eliminates or reduces errors in original data, and be organized and published in a way that errors and faults can be corrected.

45. Quality Indicators Purposes of Quality Indicator System Colorado State, USA: Encouraging continuous improvement by institutions in achieving high levels of performance. Measuring institutional performance and accountability. Determining funding recommendations and the funding distribution for the higher education system. Build public support for increased funding for higher education.

46. Quality Indicators Some Quality Indicators Baccalaureate graduation rates Achievement scores of graduating students on various comprehensive examinations, tests, and /or professional specific licensure or certification examinations Graduates employed or continuing their education Institutional support expenditures – administration expenditure, expenditures per student Undergraduate class size Faculty teaching workload

47. Quality Indicators Current scenario Categories of institutions APEX university Research intensive General The structure of Malaysian Qualifications Agency (MQA) One-stop center for institutions for registration and accreditation of courses

48. Quality Indicators Accountability and Quality • The concept of accountability and quality assessment in higher education is an international phenomenon • In America, many regions are moving toward ‘performance incentive funding’. • In Europe and Australia, the central government is directly involved in establishing quality mechanisms through: Quality control, Quality audit Quality assessment • The agencies involved are like the Higher Education Quality Council and the Higher Education Statistics Agency (HESA)

49. Quality Indicators Accountability and Quality • The focus of attention in performance indicators in the U.S. has been cost efficiency, scientific and technical education, management of intellectual property produced at universities, and undergraduate education. • Less attention is paid to graduate education and research. • Categories of costs for higher education: Education and general: faculty and staff salaries, equipment, libraries, administrative and basic academic computing, and certain capital or such as rent. Cost for sponsored research Costs of student living: room, board, clothing, laundry, entertainment, and etc. Cost of foregone earnings: While disengaged from the productive work force.

50. The Movement in Setting PerformanceIndicators in Higher Education(U.S.A.) • 1980s • Era concerned with growth in enrollments and access was over • Emerging issues include: Public accountability Quality Productivity Undergraduate education • In 1986, all 50 states and the District of Columbia had developed initiatives to improve the undergraduate education