1 / 23

Our Most Important Challenge Raising the Assessment Bar: A Challenge to our Community

Our Most Important Challenge Raising the Assessment Bar: A Challenge to our Community. Rick Luce, Emory University Library Assessment Conference Seattle – August 4, 2008. Environmental Scan. University mission evolving:

jersey
Télécharger la présentation

Our Most Important Challenge Raising the Assessment Bar: A Challenge to our Community

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Our Most Important ChallengeRaising the Assessment Bar: A Challenge to our Community Rick Luce, Emory University Library Assessment Conference Seattle – August 4, 2008

  2. Environmental Scan • University mission evolving: • Education reform and the Spellings commission  focus on outcomes and accountability • Globalization and competition • New research methods in a networked world: • Rise of eScience / eResearch = new ways to work, new needs and expectations • Data science & data scientists require new organizational environments • Social drivers: • Technology enabled social tools to connect & collaborate

  3. My View after 2 years at Emory Aggressive Strategic Plan completed in first 4 months: • Strategic direction: (1) digital innovations, (2) special collections, (3) delivery of 21C access, resources, and services • $100M requested in new funds over 5 years • Concept approval for ~$36M for 3 new capital projects • Implement annual Business Plan – reviewed quarterly None of which utilize current ARL statistics

  4. We Need a Systems Approach • The human body is a system, our subsystems work together to keep us healthy • Research libraries are systems, requiring a management system to keep the subsystems working together to be healthy Anyone who learns to see the organization as a system can never again feel satisfied with “improvement” initiatives which simply change staffing and the org chart but do not tackle the system itself

  5. Where Assessment Fits Assessment – a method of planning for improvement • Catalyst for organizational change (not a quick fix) • Gain staff understanding for need for improvement & commitment to shared improvement goals Ideally underpinned by a performance measurement matrix balancing: • Quality = customer defined goodness – internal & external • Time = speed, how fast is the response, agility • Cost = resources spent on people, processes, or organizational shifting or rework

  6. Performance Measures: our Vital Signs • Statements without performance measures are wishful thinking -- without data, we don’t know • We all have volume or transaction data - provides no process insight • Move focus on product metrics to process metrics • Process performance (statistics, run charts, variation) • Getting to the ‘right’ metrics • What is the value equation? • How do we compare & differentiate ourselves? • Focus on : Customer, process, sponsorship metrics

  7. Hedgehog View: Constancy of Purpose What are we best at? What drives our economic (or value) engine? What are we passionate about? Adapted from: Jim Collins. Good to Great. 2001

  8. ID Key Success Factors Characteristics of successful organizations:* • Do something others cannot do • Do something well that others do poorly, or • Do something others have great difficulty doing well • ~10-15% (max.) of research libraries content / services are unique What % of the budget resources support that? *Prahalad and Hamel. The Core Competencies of the Corporation. HBR, May/June 1990.

  9. Management Principles of Successful Organizations • Strong customer focus • Effective leadership • Continuous improvement and learning • Management by fact • Fast response • Long-range view of the future • Results orientation • Cooperation, teamwork, partnering

  10. A Value Proposition Example (LANL) • Scientific Productivity & Competitiveness Enhancing research productivity and competitiveness • ROI = $4.50 : 1.00 • Quality and Business Focused • 10 year average: 96% satisfaction rate; 25% delighted • Reduced cost per transaction by a factor of 16 • Quality New Mexico ‘Roadrunner’ awards: 1997 & 2000 • Federal Library of the Year 1999 • “World’s Best Science Library” 2005 Blue Ribbon Review

  11. Follow the Leaders Malcolm Baldrige National Quality Award: recognizes organizations practicing the most effective management methods • High performance is sustainable due to good management practices • Analysis of 600 winners over 10 years: growth = > 2.5 times as fast as peers, more than 2X more profitable • Examines approach, deployment and results

  12. Baldrige Criteria for Performance Excellence • Leadership • Strategic Planning • Student, Stakeholder, and Market Focus • Measurement, Analysis, and Knowledge Management • Workforce Focus • Process Management • Results Requires a system both in approach and deployment

  13. Baldrige Applications: LessonsLearned • Accelerated learning using Baldrige framework • “System” is tough to integrate all at once “Be patient, have discipline” Deming • Importance of supplier partnerships • Difficulty of language translation • Benchmarking data - time series data for competitors – couldn’t be obtained from libraries

  14. Benchmarking • Process for gaining and applying knowledge to improve business process performance from a study of current practices. • A means of using data to identify magnitudes and reasons for variances in performance. • Intent: comparative process data, best practices • Analyze the operation, know the competition & industry leaders • Incorporate the best of the best - become the new benchmark

  15. Missing from Research Library Portfolios • Customer satisfaction index: (delight & loyalty) and perceived value • Product / service quality (defined by the customer) • Process and operational performance - cycle time, productivity • Employee satisfaction – learning, morale, training, alignment of strategy direction and rewards • Measuring supplier performance - quality, process variables, price competitiveness, overall ease of doing business • Financial: cost/value matrix, return on investment, cost avoidance

  16. Customer Satisfaction Metrics Maturing our satisfaction assessment Level 1 – Satisfaction surveys: ‘happiness meters’ Level 2 - What’s important  analysis of customer importance & satisfaction levels Level 3 - How do we rate against best in industry See it from the customers eyes

  17. Measures that matter Align library vital signs with the organization’s drivers • Quality of product and processes • Innovation • Research leadership • Brand identity • Growing market share • Reducing new product development time • Ability to attract and retain employees • Credibility

  18. Avoiding Pitfalls • Measures that don’t focus on strategy • No accountability • Too many initiatives • Forgetting larger organizational drivers • Lack of discipline • Insulating researchers and managers from scholarly communication issues No action without a plan, no plan without data

  19. Business Scorecard - Desired Business Results 1. Customer focus: • Satisfaction, loyalty, value-added 2. Product quality: • E.g., accessibility, usability, accuracy, completeness 3. Operational process performance: • Productivity, competitiveness, cycle time 4.High performance workforce: • Organization performance assessment, formal process changes, employee satisfaction 5. Prestigious reputation -- output results • Strategic performance results, benchmark results, external assessment scores

  20. A Tale of 2 Libraries: Budget Allocations $M

  21. Payoff for Successful Quality Implementation

  22. Any road will do if the destination is unknown The journey to truly superior performance is neither for the faint of heart nor for the impatient. The development of genuine expertise requires struggle, sacrifice, and honest, often painful self-assessment. HBR: Anders Ericsson (FSU), Michael Prietula (Emory), Edward Cokely (Max Planck)

  23. If you wish to do something for the community, build a road. If you wish to do something better for the community, build a bridge – Chinese Proverb Together let’s build the bridge to a new level of assessment practice, supporting continuous improvement and focusing on outcomes and impact.

More Related