Information Systems and Organisations

1. Information Systems and Organisations Lecture 5: Evaluating IS

2. Scope and Coverage This topic will cover: • Tangible and Intangible benefits • Formal-rational evaluation • Wider evaluation approaches

3. Learning Outcomes By the end of this topic students will be able to: • Understand typical tangible and intangible benefits of IS • Describe a variety of approaches to IS evaluation • Critically analyse evaluation approaches

4. Contents • Tangible and intangible benefits • Relevance of cost benefit measures to evaluating IS • Aspects of evaluation • Interaction of evaluation and organisational contexts • Emphasis – Technology and Organisations

5. Costs of IS (Tangible) • Costs of the technology: • Capital cost of development or purchase • Cost of implementation (project costs) • Costs of the resources to run it: • Hardware • Network • Software • Service and support

6. Benefits of IS (Tangible) • Benefits (related to IS/IT) • Reduce fixed costs of IS/IT service • Reduce management overhead • Benefits to the organisation • Increased efficiency in processes (automation) • Increased quality with reduced costs • Increased sales (activity) per unit of resource • (essentially PRODUCTIVITY benefits)

7. Fixed and Variable Cost In general, managers may prefer VARIABLE costs in many types of operations because they provide a built in method of reducing costs if activity falls and coping with higher activity if it should rise. IS and IT can help reduce FIXED COSTS and in some cases, through appropriate process changes, replace them with VARIABLE COSTS. Meaning that the organisation is viable at lower levels of activity..e.g. as in diagram. Reducing fixed costs will lower the breakeven point

8. IS Driving Efficiency • Some years ago, the following estimates of banking transaction costs were made • Traditional branch based transaction $1.07 • Telephone transaction $0.54 • ATM transaction $0.27 • Internet (online)transaction $0.13 • In fact online transactions have sometimes been estimated as low a $0.01 – the exact figure will vary depending in complexity of the transaction

9. ERP Benefits Firm Infrastructure 55% Human Resource Management Technology Development Margin 13% Procurement Outbound Logistics Inbound Logistics Marketing and Sales Operations Service 32% ERP Benefits % - Source AMR Research April 2004

10. IS Effect is Very Variable • There is no simple relationship between IS/IT investment and productivity • In some cases (e.g. healthcare, social work) productivity is difficult to measure in any event • In some cases (e.g. electronic design, retailing) measuring productivity is easier but outcomes can be difficult to interpret • A late 90’s study of banks in the USA documented a large variability in return on IT investments

11. IT Spending and Profitability 24% Wells Fargo First Tennessee Nations Bank Services Northern Trust First Chicago 12% Returnon Equity Bank of America Fleet Financial 6% $1000 $10,000 $100,000 1995 Information Technology Spending per employee (Banking Sector) Source: Strassman (1997)

12. Costs of IS (Intangible) • Opportunity costs? • Should we be doing something else? • Overhead of running IS/IT • Management • Management time and attention • Governance • Risk • Obsolescence not predictable • ‘Game changing’ innovations

13. Benefits of IS (Intangible) • Benefits to the organisation • Improved communications • Staff morale • Reputation (branding benefits) • Increased customer satisfaction (branding..) • Flexibility and agility • Organisational learning.... • Intangibles – implicitly ‘un-measurable’ • A management judgement is needed to weigh intangible benefits

14. Intangible and Tangible Benefits • Should we ensure that there are always significant tangible benefits when we make an investment decision? • Should we make investment decisions with no way of measuring potential benefits ? • Are intangible benefits simply benefits that we cannot yet measure ?

15. ... Should We Even Worry ? • Historical project experience shows that significant numbers of organisations often never seek to seriously measure benefits following implementation of a new IS

16. Evaluation Perspectives • Formal and rationale all very well – but: • What can be evaluated ? • Who is evaluating ? • What is the purpose of the evaluation ? • Success results from a combination of aspects

17. Main Dimensions of IS Success Development and Process Process Costs Use Process User SUCCESS Information Satisfaction Quality of the IS Product Product Benefits Impact of IS on Organization

18. Who Evaluates? • Business managers • IS/IT managers • IS/IT users • External consultants • External authorities (regulatory bodies perhaps) • Auditors

19. Purpose of Evaluation • Perspectives and Aims • To measure tangible costs and benefits • To endorse success • To investigate failure • To identify good practice • To improve implementation process • Overt and covert motivations may exist

20. IS Success Model - Research • DeLone and Mclean - a large meta study (1992 and 2003) • What factors lead to IS success ? • What is the relationship between these ?

21. DeLone McLean Model System Quality Intention to Use Use Information Quality Net Benefits User satisfaction Service Quality

22. References • Boddy, D., Boonstra ,A., Kennedy, G. (2008) Managing Information Systems : strategy and organisation 3rd ed. FT Pearson. ISBN-13: 978-0273 -71681-5XXX

23. Lecture 5 – Evaluating IS Any Questions?