1 / 35

”You get what you measure” or not? Challenges for fact-based quality management

”You get what you measure” or not? Challenges for fact-based quality management. Juhani Anttila Venture Knowledgist – Quality Integration www.QualityIntegration.biz , juhani.anttila@telecon.fi. 15.3.2008. These pages are licensed under Creative Commons Nimeä 2.5 lisenssi

norwood
Télécharger la présentation

”You get what you measure” or not? Challenges for fact-based quality management

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ”You get what you measure” or not? Challenges for fact-based quality management Juhani Anttila Venture Knowledgist – Quality Integration www.QualityIntegration.biz , juhani.anttila@telecon.fi 15.3.2008 These pages are licensed underCreative Commons Nimeä 2.5 lisenssi http://creativecommons.org/licenses/by/2.5/deed.fi

  2. ”You get what you measure (*)” or not? Challenges for fact-based quality management • Searching a balanced position to statistical quality management • Factual business performance • Facts, data, information and knowledge for business management • Using measurements, analyses, and knowledge within business management • Example: Using ZEF methodology for business performance evaluations • A topical problem: Quality of networks • Summary and conclusions (*) 32500 Google hits xxxx/2.3.2008/jan

  3. What am I striving for? Statistics Mathematical science pertaining to the collection, analysis, interpretation or explanation, and presentation of data Quality Degree to which a set of inherent characteristics fulfils needs and expectations • Quality management (Quality of management) • Degree to which coordinated activities of management to direct and control the organization fulfill the needs and expectations of organization’s all stakeholders Principles of good management Factual approach to decision making (ISO 9000 - Quality management principles) Management by fact (Malcolm Baldrige - Core values and concepts) Management by processes and facts (EFQM - Fundamental concepts of excellence) Business methodology Business performance management Measuring business process performance 3462/2.3.2008/jan

  4. Positioning to statistics in quality management Constraints: - Theoretical / mathematical difficulties - Problems - Warnings • Quality management traditions: • Statistical process control • SixSigma • Problem solving • Taguchi techniques How to find balance? Darrell Huff 1954 Walter Shewhart 1931 3461/16.2.2008/jan

  5. Information problems and danger of empirism • Data hell – Chaotic data disturbance • Data bulimia or data anorexia • Data mechanization – overly emphasized information technology and impoverished skills • Knowledge splintering into objective information pieces • Data pedantry and data naiveté • Popularization – Complex realities overly simplified and information contents impoverished • Information insecurity • Information capitalism or information proletariat – misuse of information power • Information terror and data crime (Ref.. Jaana Venkula, Huff, etc.) 3465/14.2.2008/jan

  6. Warning about the danger of information (*) Information is not knowledge. Knowledge comes from theory. Without theory, there is no way to use the information that comes to us on the instant. There is no true value of any characteristic, state, or observation. Warning about people rewarding based on quantitative measurements, deadly disease #3 of leadership: “Evaluation of performance, merit rating, or annual review nourish short-term performance, annihilates long-term planning, builds fear, demolishes teamwork, nourishes rivalry and politics. It leaves people bitter, crushed, bruised, battered, desolate, despondent, dejected, feeling inferior, some even depressed, unfit for work for weeks after receipt of rating, unable to comprehend why they are inferior. It is unfair, as it ascribes to the people in a group differences that may be caused totally by the system that they work in.” More important than to create scrupulous and even harmful business measurement systems, is to get tacit knowledge of business leaders and workers effectively interact with each other for creating extensive profound knowledge within an organization. (*) W. Edwards Deming (e.g. in "The new economics") 3466/14.2.2008/jan

  7. Pragmatic (*) genuine knowledge Pragmatism (C.S. Peirce): Action based information / knowledge approach: • Genuine knowledge makes possible to act in a new meaningful way. • Knowledge can be genuine only if it has been tried and justified in operation. • Knowledge is a basis of an organization and a community and their practical operations. • Knowledge cannot be objective. ”The knowledge you take is equal to the knowledge you make.” (**) Charles Sanders Peirce (*) Ref. the other paradigms of knowledge theories: - Heuristic (internal comprehension based knowledge) - Empirical (external observation and experience based knowledge) (Ref.: Jaana Venkula) (**) Lennon, McCartney 2475/4.3.2008/jan

  8. Quantitative approach to driving business performance • Quantitative indication of performance: • Phenomenon to be considered of a business interest and its character – especially in business processes • Measure / indicator, characteristic, quantity • - Metrics, unit of measure • Numerical values and value range of the quantity • Practicalities of performance management: • - Target / required values, observed values • - Meter, gauge, means to measure / observe the quantity • - Measurement, assessment technique • Uncertainty • Conclusions, decisions and actions based on measurements • - Approach and practices for performance improvement 1613/10.3.2008/jan

  9. Performance measurements for business management – Need of business information • Purpose of measurements: • - Research activities for getting new knowledge • - Acquisition of information for planning business or operations • - Controlling operations and processes • Performance monitoring • Determination measurements • Verification measurements, hypothesis testing (AQL / LQ) • Validation testing • - Measurements for problem solving and performance improvement: Kaizen, breakthrough and learning • - Measurements for quality assurance Management areas: - Operational management - Strategic management Scope of measurements: - Performance of business processes - Performance of products (goods and services) Related aspects: - Measurements management - Business performance management 3467/2.2.2008/jan

  10. From business activities to overall performance of an organization and to performance excellence Overall business performance } References: Competitive performance Operational performance - processes - workforce Customer- focused performance Product performance - goods - services Financial and marketplace performance PERFORMANCE EXCELLENCE! Excellence is originated from the business activity not only being superficially outstanding Financial measures (data) Non-financial measures (data) DOMAIN OF ACTION - BUSINESS ACTIVITY (Operational business processes and projects) (Ref.: Malcolm Baldrige Criteria) 0877/22.3.2008/jan

  11. Information available from all key business areas and processes and related requirements: Consistent and balanced quantitative data through measures and indicators of business related facts Level of performance Rate of improvement, or slope of trends in data demonstrating development of business performance ”Three generation reporting”: Past, now, future Pattern of development: Linear, non-linear, spiral Comparisions of performance levels and their development with own targets, competitors, and World-Class benchmarks Relationships of achieved quantitative business results with management and process activities: Cause-effect linkages Interesting information on business performance (Ref.: Malcolm Baldrige Criteria) 3468/2.2.2008/jan

  12. The ‘Now’ of future events and performance The ‘Now’ of past events and performance Temporal dimensions of the present moment “Now” Time } The ‘Now’ of present events and performance Kairos Kronos The present “Now” always includes the impacts of the past events and the challenges of the future events! 2583/5.3.2008/jan

  13. What is information security all about? • Basic concepts characterizing security of the information consist of: • Integrity – information that one is using for his/her actions is accurate • Availability – one has access to the relevant information when and so long it is needed • Confidentiality – information one is using is not manipulated by anybody else. • Additionally authenticity (information is authentic, trustworthy, or genuine) and authority (one has right to use the information) aspects are significant especially when using the means of electrical communication. • The most essential information security aspect is privacy! (Ref.: ISO/IEC 27001 / 27002) 2476/11.2.2008/jan

  14. You don’t get necessarily what you measure • Measurements in all kinds of business activities are important for management but in practice: • Measurements are often excessive, unfocused or wrong (*) • Business results you get don’t necessarily relate to what you measure • What you get through measurements is not necessarily meaningful or good for organization or people being managed • Business leaders may not know what they don’t know (“Tacit ignorance”) • Measurements results represent explicit knowledge of the business performance but business intentions and real business results are principally of tacit knowledge of business leaders and workers in the organization. • No measurements can be objective but they are always affected by the intension and awareness of somebody – even in physics objectivity is impossible. What is being measured, by what kinds of means or methodology, what is obtained through measurements, and how the measurements results are understood – they all depend on somebody’s intention and awareness. (*) Dr. Juran: “Vital few” / “Trivial many” 3469/5.3.2008/jan

  15. Facts & knowledge for managing business activities Wisdom - myths - values Knowledge - explicit records - tacit knowledge (know-how, competence) Intervention Plan / Act Reflecting and deciding Information ”Ba” Analysing You get what you measure A P Data C D Environments ... Measuring Effects Facts The performance reality of the company business processes 0609/25.3.2006/jan

  16. PDCA model (*) for a good management: Coordinated activities to direct and control an organization (**) • P = Planning • D = Doing • C = Checking • A = Acting • Applying PDCA model: • Rational control (operational) • Continual rational small step improvement (operational), “Kaizen” approach • Innovative breakthrough changes (strategic) (***) • PLANNING: • Business and management models • Business plan • Approaches and methodology • ACTING: • Preventing actions • Improving actions • Re-engineering • Communicating • Recognizing and rewarding A P C D • CHECKING: • Assessing the performance • Reviewing the performance • DOING: • Deploying the approach and achieving the results • Controlling operational performance • Corrective actions All three PDCA loops modes and four PDCA phases are strongly information / knowledge dependent (*) Deming / Shewhart Cycle (**) ISO 9000 (***) Dr. Shiba 2343x/15.8.2007/jan

  17. Control of the business activities- feedback Business requirement input Business output Operations (Business processes) Top • Notes: • Requirements for control : • Throughput times Tfb<<Top • ”A decision must be made • at lowest possible level in • the organization and as close • as possible to where its • outcome will be executed.” • (Peter Drucker) Business control input Feedback Tfb Target 1847/26.2.2008/jan (Ref.: Bode, Asby)

  18. Traditional quality tools Traditional ”quality tools” have a long history and are very strongly related with data prosessing and management of business operations. Assortment of the methods may seem confusing, despite that they have been arranged into different groups such as Seven basic quality tools, Seven new management and planning tools, 127 tools, Kaizen tools, SixSigma tools, etc. tools. These tools include e.g.: • Process management tools: • Control charts • IDEF0 • Process capability methods • Flow charts • Simple basic tools: • Data collection tools • Idea creation, brainstorming tools, nominal group technique • Check lists / sheets • Survey methods • Affinity diagram • Matrix diagram • Histogram / bar graph • Scatter plots • Relations diagram • Cause-effect, fishbone or Ishikawa diagram • Pareto chart • Evaluation and decision-making tools, decision three, force-field diagram • Voting methods • Inspection, testing or acceptance methods: • Attribute testing • Variable testing • Data analysis methods: • Statistical tests • Variance analysis • Regression analysis • Design of experiments • Z-score analysis • Value analysis • Project management tools: • Pert / Gantt chart • Reliability (dependability) tools: • Failure mode effects and criticality analysis (FMECA) • Fault tree analysis (FTA • Reliability prediction • Reliability testing Recommendation: Organizations should establish and maintain their tool sets for quality management, “Business excellence tool kit”, as well as the measurement and information management at large. 3470/26.2.2008/jan

  19. Using measurements, analyses, and knowledge within business management (*) • Enablers criteria • Category 4: Measurement, analysis, and knowledge management: • Measurement, analysis, and improvement of organizational performance (Assessment area4.1) • 4.2 Management of information, information technology, and knowledge (Assessment area4.1) • Results criteria: • Category 7: Results • Product outcomes (Assessment area 7.1) • Customer-focused outcomes (Assessment area7.2) • Financial and market outcomes (Assessment area7.3) • Workforce-focused outcomes (Assessment area7.4) • Process effectiveness outcomes (Assessment area7.5) • Leadership outcomes (Assessment area7.6) (*) Malcolm Baldrige Criteria 2008 3472/.20.3.2008/jan

  20. Using measurements, analyses, and knowledge within business management(*) • Enablers criteria: • Criterion 2: Policy and strategy - Policy and strategy are based on information from performance measurement, research, learning and external related activities (Criterion part 2b) • Criterion 4: Partnership and resources - Information and knowledge are managed (Criterion part 4c) • Results criteria: • Customer results (Criterion 6) • People results (Criterion 7) • Society results (Criterion 8) • Key performance results (Criterion 9) (*) EFQM Criteria 3473/.20.3.2008/jan

  21. Information technology serves our knowing and learning as an extension of the body Impact of technology: The form of a medium imbeds itself in the message, creating a symbiotic relationship by which the medium influences how the message is perceived, creating subtle change over time. We shape our tools and afterwards our tools shape us. Use of technology: • Collecting data • Processing data and information • Storing data and information • Sharing / communicating information and knowledge (IT = Interactive technology) Marshall McLuhan 1911-1980 3474/2.1.2008 /jan

  22. Collaborative knowledge working / learning environment with Web 2.0 tools, “Enterprise 2.0” Collaborative knowledge working environment for activity XXX at organization XYZ: Group- members Common shared knowledge-base accumulated, discussed, and updated by the group-members (Wiki) Knowledge feeds from group-members' findings, observations, and reflection, and comments (Blog) Discussion forums Automatic information feeds from external sources, e.g. Internet (Aggregators) Documents files •  Links to • Other tools (calendar, to-do list, diary, Skype, virtual meeting room, etc.) • Administration of the environment 3372/15.9.2007/jan

  23. Web 2.0 challenges Web 2.0: Second-generation of Internet-based services - such as social networking sites, wikis, communication tools, and folksonomies - that let people collaborate and share information online in ways previously unavailable RSS: (Rich Site Summary) Really Simple Syndication Mashup: A website or application that combines content from more than one source into an integrated experience Tag cloud: A visual depiction of content tags used on a website Facebook: a social utility that connects people with friends and others who work, study and live around them RTE – Real Time Economy: Speeding up information flow between companies, each having capabilities to monitor their businesses continuously and quickly react to changes and exceptions (Ref.: http://www.oreilly.com/pub/a/oreilly/tim/news/2005/09/30/what-is-web-20.html ) 3460/2.3.2008/jan

  24. Example: ZEF evaluation methodology • ZEF is an advanced interactive evaluation methodology and tool that is originated from strong core competences of evaluation technique and human interface technology. • Origin: • ZEF methodology was developed in Finland at Oulu University as Z-scored Electronic Feedback (= ZEF) tool. Effectiveness of its two dimensional evaluation structure was validated by University of Lapland. • Commercial ZEF solution is a product of Oulu based company ZEF Solutions Inc. • Usage: • ZEF is an all-purpose tool that is suitable for all kinds of organizations. • ZEF evaluation services are web-based solutions operating as SaaS (Software as a Service). With ZEF on-line tool one may collect, compare and evaluate data effectively and efficiently from even a big group of people. • ZEF tool has been used by a lot of organizations for business information and hundreds thousands of ordinary citizens in several survey-cases of general societal interest. 3336/.20.3.2008/jan (Ref.: http://zefsolutions.com/www/english , http://qiblog.blogspot.com/2007/06/carry-out-appropriate-surveys.html )

  25. ZEF evaluations • ZEF tool is applicable for: • All kinds of quality-related performance evaluation, feedback and survey applications • E.g. 3-In-1 assessment scheme that combines Malcolm Baldrige model, EFQM model and ISO 9000 and is easy to tailor to specific business aspects or organizational strategic priorities. Examples include modifications for • Health services quality management (CEN/TS 15224) • Food safety management (ISO 22000) • Information technology service management (ISO/IEC 20000) • Process performance evaluations • Customer / workforce satisfaction • Other business related assessments needs including • Innovations evaluation • Risk, vulnerability and threat analyses • Strategy analyses • Customer lead surveys • Other general e.g. social, entertainment, etc. applications including • National innovation policy surveys • Parliamentary / presidential election comparisons • Education and school evaluations • Idols rating 3476/20.3.2008/jan

  26. Improving effectiveness / efficiency of self-assessments with an advanced tool (ZEF) 3288/20.3.2008/jan (*) ZEF Solutions Inc., Finland: www.zef.fi

  27. Performance evaluation for excellence through a multi-faceted and consistent approach Overall business performance (e.g. 3-In-1 criteria) 5. Human resource focus 2. Strategic management 7. Business results 1. Leadership 3. Customer and market focus 6. Process management 4. Measurement, analysis,and knowledge management Leadership performance Social responsibility Customer satisfaction Market communication Information security Innovations IT Governance Product performance Board operations Strategic performance Corporate security Risk management Networking capability Process performance Project performance Supplier performance Quality Assurance: ISO 9001, AQAP Environmental management People satisfaction 360 feedback OHS assessment 3325/26.3.2008/jan

  28. Results of ZEF evaluations (“3-in-1”) Assessesment scheme “3in1”: EFQM, Malcom Baldrige and ISO9000 combined in one scheme Category 4: Measurement, analysis, and knowledge management 1. We follow up fulfilment of targets and development of performance on company and process levels with adequate measures and indicators. 2. We follow up effectiveness and efficiency of our business processes. 3. We have easily available analyzed and current information on performance of our company and its development against targets and other references. 4. Our real-time internal accounting operates in advance. 5. Our employees measure, analyze and evaluate quality of their own work in order to make decisions and initiate improvement actions for their work in line with our company-wide plans. … … Etc. other assessment items 3475/20.3.2008/jan

  29. Results of ZEF evaluations 3334/.20.8.2007/jan

  30. Statistical inferences from ZEF evaluations • Detailed business information is obtained from ZEF raw data through statistical treatments. All individual respondent answers to the query items in the ZEF evaluations are recorded as separate data pieces that are then used for statistical analyses as needed: • The basic ZEF report shows the means and standard deviations of all respondents’ answers per query items in one or two dimensions as needed. • The result data may be normalized according Z-scored transformation that is useful to emphasize relative differences among query items. • Answers from respondents may be grouped in arbitrary way or presented as individual answers. • Results of different ZEF evaluations, e.g. of different points in time, may be compared by using ZEF’s “comparison engine” function. • Raw data of evaluations may be presented in the formats of Excel (.xls) or "Concurrent Versioning System" (.cvs) in order to facilitate the use of more sophisticated statistical tools, e.g. SPSS, for data analyses. ZEF tool’s interactive user-interface enhances user’s deliberation when he/she is responding to the query items and also enhances understanding the relative differences among query items. That makes answering more motivated and justified. 3471/.20.3.2008/jan

  31. Z-score transformation Z-score for an item indicates how far and in what direction that item deviates from its distribution's mean expressed in units of its distribution's standard deviation. The z-score transformation is such that if every item in a distribution is converted to its z score, the transformed scores will have a mean of zero and a standard deviation of one. Z-score transformation is especially useful when seeking to compare the relative standings of items from distributions with different means or different standard deviations. (Ref.: http://www.sysurvey.com/tips/statistics/zscore.htm ) 3478/20.3.2008/jan

  32. Z-scored normalized evaluation results (ZEF) 3477/.20.3.2008/jan

  33. Summary and conclusions • Competitiveness and success of all organizations are based on right business related information and knowledge on time. • Impressive business phenomena and their nature including stochastic features should be taken into account in planning business performance measurements. • Business related measurements and information may have many different purposes. • One should recognize vital few valid subjects for business performance measurements. • Measurements should be based on sound methodologies and effective measuring tools. • Observed data should be understood in the business context and also data’s statistical viewpoints should be noted. • Tacit knowledge and explicit information should be brought together in making conclusions and activating managerial measures. • Interactive information technology gives remarkable advantage and serves as an extension of our body and senses both in making observations and in using information within business communities. • Cooperation of quality and statistics experts is obviously useful. 3479/.20.3.2008/jan

  34. j i Overall quality of a network Win / Win in network Overall quality of a network Qvm: Degree to which the network fulfils needs and expectations of all network members: - m members in the network - Each member gets benefits (Si) but also loses something (Ai). S S A A Network member i Network member j Knots with few links Power law of a scale-free network: Knots with many links Number of knots with k links A topical problem: How to measure in practice the overall quality of a network? Number of links (k) 3464/15.2.2008/jan

  35. Please, evaluate my presentation and material and give your feedback with ZEF: http://www.zef.fi/service/user/?q=597 Thank you! xx/15.2.2008/jan

More Related