Evaluation meets costing: Fight or Flight?

1. Evaluation meets costing:Fight or Flight? Charlotte Ash Sheffield Hallam University http://www.shu.ac.uk/virtual_campus/cnl/team/cea.htm 21st HE Cost-effectiveness framework C

2. The Costs of Networked Learning • Six month study funded by the JISC • Additional support provided by Sheffield Hallam University • Project team - director, manager and additional researchers • For further information, visit the Costs of Networked Learning Web page: http://www.shu.ac.uk/virtual_campus/cnl/

3. Cost problems solutions What has been done already Effectiveness problems solutions C 21st HE Cost-effectiveness framework

4. The Future of Higher Education Universities must: • Change infrastructure • Increase access • Reduce expenditure • Become transparent

5. Assessing Cost-Effectiveness Stage 1: Identify problems with costs Stage 2: Find solutions Stage 3: Testing and refinement Stage 4: Identify effectiveness problems Stage 5: Find solutions Stage 6: Testing and refinement Stage 7: Joint solution to Cost-effectiveness

6. Problems 1 Lack of literature 2 What to cost? 3 How to cost? 4 Whom to cost? Assessing Costs

7. Problem Lack of literature Lack of studies Lack of framework Solution Eclectic and Holistic approach Assessing Costs: Problem 1

8. Problem What to cost? Solution Identify hidden costs Formulate list for each stakeholder: staff, student and institution Structure list into working model Assessing Costs: Problem 2

9. Planning the learning experience/ environment Planning and Development Developing the learning experience/ environment Maintainingthe learning experience/ environment Production and Delivery Academic Staff Support Staff Maintenance and Evaluation Students Managing the learning experience/ environment Providing the learning experience/ environment Lifecycle models Final three stage model Previous five stage model

10. Problem How to cost? Solution Planning framework Planning document Financial schema Assessing Costs: Problem 3 Financial Schema

11. Problem Whom to cost? Whom to include? Who to do? Solution Stakeholders Translator/Negotiator Assessing Costs: Problem 4

12. Solutions 1 Eclectic/Holistic approach 2 Hidden, ListModel 3 Planning framework planning document financial schema 4 Whom to cost: Stakeholders Translator/Negotiator Problems 1 Lack of literature 2 What to cost? 3 How to cost? 4 Whom to cost? Whom to include? Who to do it? Assessing Costs

13. Assessing Effectiveness Reflections on a Model for Evaluating Learning Technologies by Jones, Scanlon and Blake CIAO! evaluation framework developed by the Computers and Learning Research Group at the Open University “Do my students learn better, faster or more using this technology?”

14. Assessing Effectiveness Evaluating Technology-Based Learning by Dr Robin Mason Holistic evaluation Good evaluations demand skill, perception and resources

15. Flashlight 1994 - 1999 • Current Student Inventory (1994) • effectiveness • Costs Analysis Handbook (1999) • costs • Problems: 1 Costs and effectiveness kept separate 2 Individualistic approach prohibits comparisons

16. Assessing Cost-Effectiveness Stage 1: Identify problems with costs Stage 2: Find solutions Stage 3: Testing and refinement Stage 4: Identify effectiveness problems Stage 5: Find solutions Stage 6: Testing and refinement Stage 7: Joint solution to problem of evaluating the cost-effectiveness of HE

17. Evaluation meets costing:Fight or Flight? Charlotte Ash Sheffield Hallam University http://www.shu.ac.uk/virtual_campus/cnl/ Thank you for listening