Application of DEA on Teaching Resource Inputs and Learning Performance

1. Application of DEA on Teaching Resource Inputs and Learning Performance Bernard Montoneri Chia-Chi Lee Tyrone T. Lin Shio-Ling Huang Montoneri, Lee, Lin, & Huang

2. Outline • Introduction • Part I. Background • Part II. DEA Model • Part III. Data selecting—Inputs and outputs • Part IV. Empirical results and Suggestions • Conclusion • References Montoneri, Lee, Lin, & Huang

3. Introduction • MOE in Taiwan, system of performance evaluation for academic institutions. • Each institution, self-evaluations concerning teachers’ research results and student’s learning performance. • Avoid low student enrollment, high graduate unemployment, credential inflation, and even closure. • This research focuses on student’s learning performance. • Evaluation method:data envelopment analysis (DEA). • Objective: explore the key indicators contributing to the learning performance of English freshmen conversation courses. Montoneri, Lee, Lin, & Huang

4. Part I. Background • Number of universities in Taiwan: 78 to 147 (1997-2006); ratio of spending by the government in education: 5.37% to 4.39% (1996-2005). • Acceptance rate for universities 90.93% (2006); Taiwan lowest birth rate in the world (0.83) posing the question of the survival of many institutions which are already facing financial plight. • Ministry of Education, to allocate more efficiently limited education resources and to control the quality of schools, undertakes a performance evaluation for every institutions in Taiwan. Montoneri, Lee, Lin, & Huang

5. Part I. Background • Higher Education institutions hope to obtain an excellent evaluation to receive more financial support from the Ministry. • Among the many evaluated items by the Ministry, one of the most fundamental is students’ learning performance. • The quality of students learning not only has an influence on: • The long-term relationship to a country's future growth and competitiveness • The short-term impact, on the employment rate of graduate students, or even whether the school can or cannot attract enough freshmen. Montoneri, Lee, Lin, & Huang

6. Part II. DEA Model • This study aims at exploring the quantitative learning performance of English conversation for freshmen in a university of Taiwan by applying DEA. • Many popular methods for performance evaluation such as Stochastic frontier analysis (SFA) or Structural equation modeling (SEM). • DEA is appealing to scholars since it can assess the technical efficiency of decision making units (DMUs) with multiple inputs and multiple outputs using only information on input and output quantities. Montoneri, Lee, Lin, & Huang

7. Part II. DEA Model • Charnes, Cooper, and Rhodes (1978) expanded Farrell’s (1957) efficiency measurement concept of multiple inputs and single output to the concept of multiple inputs and multiple outputs. • The domain of inquiry of DEA is a set of entities, commonly called decision making units (DMUs), that is the evaluated units, which receive multiple inputs and produce multiple outputs. • The purpose of DEA is to establish the relative efficiency of each DMU within a sample . • The constant returns to scale (CRS) represents the fact that the DMU’s inputs and outputs reach a state of optimal configuration. Montoneri, Lee, Lin, & Huang

8. Part III. Data selecting—Inputs and outputs • The characteristics of the research object are as follows: • Freshmen students of a department of English in a university of Taiwan, entering from the academic year 2004 to 2006. • They follow the same training program of English conversation for one semester to meet the homogeneity of the evaluated object. • A total of 18 students’ classes taught by full-time teachers are selected as the decision making units (DMUs). They are named from D1 to D18. • There are 3 classes per semester. Each class contains around 50 students. Montoneri, Lee, Lin, & Huang

9. Part III. Data selecting—Inputs and outputs • Input dimensions: • I1. The richness of course content: the degree of teachers’ professional knowledge for the preparation of teaching materials. • I2. The diversity of accessed multiple learning channels: whether teachers can increase students’ learning interest and learning motivation. • Output dimensions: • O1. The positive degree of teaching attitude: responding positively to students’ questions; maturity of teachers’ teaching and communication skill. • O2. The students’ learning performance after receiving a period of language training. Montoneri, Lee, Lin, & Huang

10. Part IV. Empirical results and Suggestions • The efficiency value of CCR model: overall technical efficiency of the evaluated unit. If the efficiency value equals to 1, the evaluated unit is efficient; if the efficiency value is less than 1, the evaluated unit is inefficient. • This study analyzes the learning performance by using Frontier Analyst. • The average overall technical efficiency is 0.986. • Five DMUs show the best performance with value of 1 and do not need any improvement in the inputs or in the outputs. • Teachers and students feel at ease, are motivated to work; atmosphere in class good and students feel at ease to speak English. Montoneri, Lee, Lin, & Huang

11. Part IV. Empirical results and Suggestions • 13 inefficient DMUs CCR score smaller than 1: students are more afraid to speak English during the class and are unable to assimilate the entire contents of the course. • Some students think that their teacher has not enough professional knowledge and experience to teach this course. • The teacher needs to improve the learning channels, such as language learning websites, learning softwares, online courses, computer-assisted language learning (CALL). • D3, with the lowest overall technical efficiency, 4.6% of effort needed in the positive degree of teaching attitude and the students’ learning performance. • The teacher should respond to students’ questions more positively and improve the teaching and communication skills. Montoneri, Lee, Lin, & Huang

12. Conclusion • We use DEA to analyze the quantitative learning performance of English conversation for freshmen in a university of Taiwan. • We emphasized on four different dimensions: two inputs (the richness of course content, the diversity of accessed multiple learning channels) and two outputs (the positive degree of teaching attitude and the students’ learning performance). • As universities, departments and teachers are evaluated periodically in Taiwan, it is important to find ways to improve teachers and students’ performance. • In our university, part-time teachers who obtain once a rating lower than 3 on 5 are fired; part-time teachers who obtain once a rating lower than 3.5 cannot teach the same course again. Montoneri, Lee, Lin, & Huang

13. References • Charnes, A., Cooper, W. W., Rhodes, E. (1978). Measuring the Inefficiency of Decision Making Units. European Journal of Operational Research, 2(6), 429-444. • Farrell, M. J. (1957). The Measurement of Productive Efficiency. Journal of the Royal Statistical Society Series A. General, 120 (3), 253-282. • McGowan, W. R., & Graham, C. R. (2009). Factors Contributing to Improved Teaching Performance. Innovative Higher Education, 34 (3), 161-171. • Ministry of Interior (2010). Sixth Weekly Report, Statistical bulletin 2010. Retrieved March 22, 2010, from http://www.moi.gov.tw/stat/news_content.aspx?sn=3829. Montoneri, Lee, Lin, & Huang