1. Evaluating programs

2. Problem: You have been given responsibility for the quality of a three year undergraduate program involving some 30 courses. What evidence do you have about the program and its outcomes that informs decisions about quality and quality improvement?

3. Feeling powerless???? GCEQ – but……………… Commission an evaluation by some-one – but what data can be collected, what standards apply, who interprets the data against the standards and how will evaluation be used in decision making?

4. Answer????? • Depends on what you consider to be relevant data and how it might be used either ‘by itself’ or in conjunction with other information

5. The issues for you are: • Getting data that connects to academic work – eg how do students regard my course? • Having tools that allow you to generate data about programs • Being able to connect data and tools with professional judgments to decide about quality improvement? • Getting others to understand the system and its limitations in assisting decisions about quality improvement

6. Model 2 Model 1 All students in all courses in the program Exit students courses Aggregated evaluation items Program evaluation Program characteristic 1 Program characteristic 2 Program characteristic n Program characteristics

7. Program Evaluation at UniSA • Course evaluation aggregated to provide program and sub program data • Policy framework identifies Program Directors to manage evaluation through management of course evaluation • Online delivery that uses data from student information system • Formal Program Evaluation Reports include benchmarking based on data

16. System Components • A workable view about quality • Identified standards • A means of measurement • Judgment against standards • Linkages with quality improvement • Linkages with change agents

17. Program Evaluation at UniSA • A workable view about quality • Resolved through policy directions • Shaped by national ‘tests’ and performance tables • Privileges student data within evaluation process

18. Program Evaluation at UniSA • Identified standards • GCEQ scales related to teaching and generic program outcomes

19. Program Evaluation at UniSA • A means of measurement • Course data that is aggregated rather than specific program instrument • Core items make up standardised survey – items sample scales from the GCEQ and include overall satisfaction • Constructed and delivered online through course home pages

20. Program Evaluation at UniSA • Judgment against standards • Intra program benchmarking according to other high performance courses in relation to items of the core • External benchmarking – overall satisfaction and GCEQ scales

21. Program Evaluation at UniSA • Linkages with quality improvement • Required reportage of evaluation of program • Requirements regarding identification of changes for quality enhancement

22. Program Evaluation at UniSA • Linkages with change agents • Reward systems use evaluation data