Abstract

1. Do Student Attributes Matter in Determining Student Perception of Teaching Effectiveness? Some Australian Experience Mohammad Alauddin School of Economics, The University of Queensland Brisbane, Australia 4072 Email: m.alauddin@economics.uq.edu.au & Clem Tisdell School of Economics, The University of Queensland Brisbane, Australia 4072 Email: c.tisdell@economics.uq.edu.au Presented at the Third Biennial Developments in Business and Economics Education (DEBE) Conference, Cambridge, UK, 1-2 September 2005.

2. Abstract • Student scores of perception of teaching effectiveness are probably used as the most significant indicator of teaching quality. The existing literature assumes (almost as an article of faith) that an average student inter alia puts in the expected number of hours for study, comes well prepared for tutorials, consults the teaching staff on a regular basis, and does not leave all or most of his/her studies until very late in the semester. That these variables determining student attitude and behaviour toward learning can affect student perception of teaching effectiveness has barely been addressed. • Employing student survey data and alternative proxies for perceived teaching quality, this paper argues that perceived teaching quality depends not only on the student perception of instructor attributes, such as presentation and explanation of lecture material, and organisation of the instruction process, but also on the student attributes characterised by student learning attitudes. Deviation from the norm in either situation can significantly affect student assessment of perceived teaching quality. • The findings reveal that instructor’s improvement in organisation, presentation and explanation, emphasis on critical and analytical ability, student’s efforts measured by hours allocated, regularity in consultation make the student better informed and hence, create a positive perception of teaching quality. On the other hand, the lower the number of hours allocated by students to their studies, the higher the propensity to leave everything to the end or just before the final exam, the more prone the students are to record a lower rating for perceived teaching quality. • Key words: Teaching effectiveness, Student attributes and behaviour, Instructor attributes, Student effort. • JEL Classification: A2, I2.

3. Structure & Organization of the Paper • Introduction & Background • Objectives • Data & Methodology • Results & Discussion • Results from SETI (Student Evaluation of Teaching Instruments) data • Results from survey data • Concluding Comments & Implications

4. Introduction and Background • Student scores of perception of teaching effectiveness - probably used as the most significant indicator of ‘teaching quality’. • The existing literature assumes (almost as an article of faith) that an average student inter alia puts in the expected number of hours for study, comes well prepared for tutorials, consults the teaching staff on a regular basis, and does not leave all or most of his/her studies until very late in the semester. • That these variables/attributes determining student attitude and behaviour toward learning can affect student perception of teaching effectiveness has barely been addressed in the existing literature.

5. Introduction and Background continued • It argues that perceived teaching quality depends not only on the student perception of instructor attributes, such as presentation and explanation of lecture material, and organisation of the instruction process, but also on the student attributes characterised by student learning attitudes. • Deviation from the norm in either situation can significantly affect student assessment of perceived teaching quality.

6. Objectives • The paper addresses, amongst others, the following questions: • What are the principal determinants of teaching effectiveness score? • How does an increase or decrease in the perceived score of any determinant affect the probability of getting a higher or lower rating of perceived of teaching effectiveness score? • To what extent do factors representing course content influence their perception of ‘good teaching’? • Two what extent do factors representing student attribute matter in determining student perception of ‘good teaching’?

7. Data & Methodology • This paper commences with the use of SETI data across three courses (two undergraduate and one postgraduate) and over five years involving nearly 1100 students. These are ‘official’ data. Note that they do not include any factors that relate to student attributes. • Subsequently this paper uses student survey data over three years and involving nearly 250 students in two undergraduate courses of different levels. These data contain information on student and course attribute related factors. • Ordered probit analysis is used as a methodology given the ordinal nature of the multiple choice data. • Table 1 presents descriptive statistics using SETI data. • Table 4 provided a description of the variables based on the survey data.

8. Results & Discussion • Results from SETI data (Tables 2 , 3A & 3B) • Results from survey data (Tables 5, 6A, 6B & 6C) • Table 2 – Ordered probit analysis of SETI data. • All Courses – All the instructor attributes except Feedback and communicating Enthusiasm appear significant. This is true for the combined undergraduate group as well. • Organisation, Presentation and Explanation appear to be more important factors given the absolute size of the coefficients. • Feedback is a significant factor at the upper undergraduate course. Respect to students seem important for the lower undergraduate course and the combined undergraduate program.

9. Results & Discussion continued • Encouragement to Thinking rather than Memorising is more important for the upper undergraduate course than for the lower. The opposite seems to be the case for Explanation. • Instructor Knowledge of the course seems important for the lower and combined undergraduate courses but insignificant for the postgraduate course. • Instructor help to develop student Learning Skills is equally important across all the courses.

10. Results & Discussion continued • Table 3A and Table 3B presents results of sensitivity analysis and assess the impact on probability of TEVAL of rating of each instructor attribute of increasing from 4 to 5 or decreasing from 4 to 3. • All variables impact on the probability of getting an Overall rating of 5 given that the attribute ratings increase or decrease –Organisation, Explanation, and Presentation are the most important factors that make perceived teaching effectiveness more sensitive. • Noticeable variation across courses and levels can be identified.

11. Results & Discussion continued • Table 5 sets out results of analysis of survey data for two undergraduate courses. • Student effort measured by number of Hours allocated is highly significant across courses. • Leaving most of the studies until the very end of the instruction period or for the SWOT vac has a negative impact on the perception of Overall teaching effectiveness.

12. Results & Discussion continued • Perceived Relevance and Practical attributes of the topics covered in the course, a perception of a good Blend of theory of application, the extent to which the course overall is perceived to be Practical impact positively on the Overall student rating of the teaching and learning process. • The extent to which student perceive Load to be the same across the courses that the students were completing at the Same time has a significant positive impact on the Overall perception of the teaching quality. On the other hand, Load in the overall degree program or to support oneself financially does not appear to be significant.

13. Results & Discussion continued • The extent to which the process is perceived to have developed students’ Analytical Skills and Critical Abilities and the extent to which the student was able to apply what was learned in the course to non-economics courses have significantly positive effect on the perceived Overall satisfaction. • Students in higher level Course seemed to be more appreciative of the process. • Students who relied more or less exclusively on the lecture Notes i.e. on a narrow range of reading materials displayed a higher propensity to have a negative view of the process.

14. Results & Discussion continued • Sensitivity analysis results reported in Tables 6A-6C suggest that: • Increasing/decreasing student efforts and relevance of contents and topics would have the strongest positive/impact on the probability of getting a rating of 5 in the overall satisfaction rate. • Likewise being less regular entails a reduction in the probability of a higher overall rating.

15. Concluding Comments & Implications • Instructor’s improvement in organisation, presentation and explanation, emphasis on critical and analytical ability, student’s efforts measured by hours allocated, regularity in consultation make the student better informed and hence, create a positive perception of teaching quality. • On the other hand, the lower the number of hours allocated by students to their studies, the higher the propensity to leave everything to the end or just before the final exam, the more prone the students are to record a lower rating for perceived teaching quality.

16. Concluding Comments & Implications continued • There seems to be a strong case for redesigning SETI questionnaires to make it more comprehensive so it can take account of the student efforts. • Additional questions involving student and course attributes are essential to obtain a balanced perspective on measures of teaching quality. This can be a cost effective way of generating a comprehensive set of information that can provide a more definite view of the teaching and learning process. This can also be a very cost effective way of information gathering as a prelude to in-depth analysis. • The present ‘official’ process of collecting data assumes as though student attributes do not matter or at best students behave normally. This is somewhat akin to assuming that the least squares assumptions about the residual term hold and violations are unlikely to occur. This is unrealistic. Ignoring this implies an abstraction from reality.