Costs of e-learning in HE - and the benefits

1. Costs of e-learning in HE - and the benefits Professor Paul Bacsich Sheffield Hallam University, UK

2. Methodological foundation “Suitably interpreted and broadened, and leavened with pedagogy,the rather dry methods of business costing and planningcan provide a sound basis for analysing e-learning developments and their impact.”

3. Summary • CNL Phase 1 [JISC JCALT] • CNL Phase 2 [JISC JCALT] • Effectiveness: impact of other work including analysis of the training sector [EU Telelearn etc] and the NLN Evaluation work [FEDA] • Some impact from related work including RCIT (CNL methods applied to admin IT systems) [JCALT] and soon perhaps from “Managing Change” (formerly TLTR) on change management in HEIs [JCIEL]

4. Main issues What learners want - must be first! Costs Effectiveness

5. What learners want... • From our own work on learners (CNL): • students believe that e-learning increases costs to them • but they behave as though it saves costs • and saves time!

6. What learners want • From our own work on learners (CNL): • students believe that e-learning increases costs to them • but they behave as though it saves costs • and saves time! • From other work (including USA): • they want flexibility (but not too much) • some want overclocking (ie faster throughput)

7. Hidden Costs - examples • Increased home telephone call bills for students due to Internet usage • Entertainment expenses incurred by lecturers but not reimbursed by the HEI • The usage of teaching or research budgets to bolster IT budgets - or vice versa

8. CNL1

9. Conclusions - CNL1 • In order to accurately record the Costs of Networked Learning you must have a universally accepted method of what costs to record and how. • Much of the literature formed interesting background reading but failed to travel far enough towards operational conclusions to be taken (individually) as a basis. • The method must include a lifecycle model, stakeholders, activity based costing and a “cost-aware” approach to planning.

10. Planning and Development Maintenance and Evaluation ProductionandDelivery Course Lifecycle Model Three-phase model of course development

11. Breakdown of three-phase model

12. The main Stakeholders • The HEI • and the staff of that HEI • The learners

13. Stakeholders - two more special cases • The HEI • and the staff of that HEI • The learners • and their organisationand in some cases their parents/partners

14. The Financial Schema- cf Transparency Review • The traditional financial model underpinning work in most HEIs is defective in four ways: No Stakeholders No agreed set of activities Crude Overhead Allocation No time recording

15. Multi-level Three-phase Course Lifecycle Model Activity-Based Costing Flexible Overheads Based on previous works Multi-stakeholder Time Division Time recording Planning Document and Financial Schema

16. Activity-Based Costing

17. Activity-based costing (ABC) ABC was developed as an alternative costing methodology by Robin Cooper and Robert Kaplan of the Harvard Business School (1988) in their research into product costing in the manufacturing industry.

18. Cooper and Kaplan (1988) • They argued that traditional costing distorted product costs (“peanut butter spread approach”) • There is a cost to all activities carried out within an organisation • Activity costs should be distributed to products in relation to their use

19. ACTIVITY DRIVERS Activities method Hold Tutorials Staff time Resources Cost for each Customer/product/channel group FT Business Intelligence Cost driver FT I.T. & Management No. Students DL I.T. & Management

20. Definitions A series of activities required to achieve an outcome Process Invoicing Process A series of related tasks carried out repeatedly Chase a student for late payment Activity Telephone student One action by one person at one point in time Task

21. Advantages of ABC • Gives more than just financial information • Fairer system of overhead allocation • Accountability of central services • Highlights cross-subsidisation • Recognises the changing cost behaviour of different activities as they grow and mature • Can provide data for other initiatives

22. Disadvantages of ABC • Data collection is costly and time consuming • Initially it can be difficult to collect the data you want • Determining appropriate cost drivers • More complex system

23. Implementing ABC takes time “Its important in a service organisation to get started with ABM and not worry about total accuracy from the start. Accuracy will improve with time” Antos (1992)

24. CNL2 • This was a trial of ABC in one faculty of Sheffield Hallam University, using industrial-strength ABC software and consultancy (Armstrong-Laing) • The trial was funded by JISC JCALT • The report of the trial (plus workbook) will be available in August 2001 • The full set of costs are not reported!

25. Tiptoeing into effectiveness issues

26. History • Relatively early in the history of CNL1, effectiveness issues were looked at (Ash, November 1999) • Main conclusion was that a uniformly accepted methodology was needed there also; and some suggestions were given, based to a certain extent on the US Flashlight work. • This work led into the NLN evaluation studies (English FE sector)

27. Effectiveness: a big “border” between E & T • The NSD effect in academia(No Significant Difference) • versus learning gains and large Return on Investment in training world: • 700% ROI at Royal Bank of Scotland (Shepherd) • routine 10-25-40% learning gains • and situations with 60% learning gains

28. NSD in academia... • It is commonly believed, and apparently validated by the seminal work of Thomas L Russell in 1999, that the mode of delivery makes no significant difference to the grades or other performance indicators of students • The Web site teleeducation.nb.ca/nosignificantdifference/ rounds out and updates this earlier work.

29. NSD in academia • It is commonly believed, and apparently validated by the seminal work of Thomas L Russell in 1999, that the mode of delivery makes no significant difference to the grades or other performance indicators of students. • The Web site teleeducation.nb.ca/nosignificantdifference/ rounds out and updates this earlier work. • But we contend that the NSD work has several major flaws - in particular many key variables were not controlled.

30. Self-incrimination? • “There were no significant pre- and post-course differences on the items related to students’ expectations and perceived benefits... • There were no significant differences between students in the [online] compared to students in the [face to face] courses for course retention or course grade... • but there were increases in the amount of time and the purposes for which students used computers." (Green et al 2001)

31. So is a key difference to do with time? • And what is time?

32. What is time? • “Time is the new distance”(Mason, inaugural lecture 2001)

33. Can e-learning save time? • travel time

34. Can e-learning save time? • travel time • reduce “time on task” • by “better” training (25-40%) • by individualised training...

35. Can e-learning save time? • travel time • reduce “time on task” • by “better” training (25-40%) • by individualised training... • by using “time of the 3rd kind” • by consolidating time fragments

36. Time of the 3rd kind • Time1 : on duty - used to be called “at work” • Time2 : off duty (not at work)

37. Time of the 3rd kind • Time1 : on duty - used to be called “at work” • Time2 : off duty (not at work) • Time3 : in-between • on duty but less productive • eg travelling • eg slots in between other tasks • off-duty but somewhat productive • eg in bar with colleagues • at dinner with customers

38. One key conclusion • To understand effectiveness issues we need a much better understanding of time • and its monetary and other value to students • Only when one understands time should one move on to other measures of effectiveness

39. Details about our work at: http://www.shu.ac.uk/cnl/ Paul BacsichProfessor of TelematicsSchool of Computing and Management SciencesSheffield Hallam University