1 / 29

To Evaluate Benefits Realization to SPI Using CBD Architecture

To Evaluate Benefits Realization to SPI Using CBD Architecture. Nov. 5. 2003. Kyung whan Lee, professor Computer Science and Engineering Division, Chung-Ang University (Kwlee@object.cau.ac.kr). Table of Contents. Introduction Performance Measures and GQM Approach

callie-flores
Télécharger la présentation

To Evaluate Benefits Realization to SPI Using CBD Architecture

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. To Evaluate Benefits Realization to SPI Using CBD Architecture Nov. 5. 2003 Kyung whan Lee, professor Computer Science and Engineering Division, Chung-Ang University (Kwlee@object.cau.ac.kr)

  2. Table of Contents • Introduction • Performance Measures and GQM Approach • Data Collection and calibration for the process performance • GQM-based Questionnaire for the process capability • Process Capability Model to the Earned Value • Benefits Realization Model and Conclusions • References

  3. Introduction (1) Lynch and Cross(1) suggested that customer satisfaction, flexibility, and productivity are the driving process upon which company objectives are based. Business goals and strategies, together with factual data about attributes of product quality and process performance, are the key drivers that lead to actions that improve a software process. We must also look at the benefit that justify the costs associated with changing the process.

  4. Goals and Strategies Measure Vision Bussiness Unit Financial Market Bussiness Operating Systems Customer Flexibility Productivity satisfaction Cycle time Departments and Work Centers Quality Delivery Waste External Effectiveness Internal Efficiency Introduction (2) Figure1. Performance Pyramid

  5. 2. Performance Measures and GQM Approach (1) The new measurement is formed under a simple assumption: fast cycle time and elimination of waste improve productivity and enable lower prices and increased sales, which in turn improve financial performance. On the figure 1, Performance Pyramid, we can make the relationships between the local operating measures and the financial results. For example, companies that achieve fast cycle times are reporting triple the revenue growth and double the profits over industry-average competitors. Fast cycle times and the elimination of waste will equal better financial performance. The GQM approach (3) is a systematic way to tailor and integrate an organization’s objectives into measurement goals and refine them into measurable values. The approach assumes that software development is still an immature engineering discipline and thus much is unknown about its characteristics and performance.

  6. 2. Performance Measures and GQM Approach (2) The GQM-based measurement approach has been implemented as part of an organization-wide process improvement program. As a direct result of the measurement program, interest in software reuse has increased and several other projects have renewed their efforts to develop reusable models. The reuse package described earlier is still frequently used within the organization to show the benefits of reuse.

  7. Reduced price Increased revenues Improved benefit realization Increased earned value Improved return on investment (ROI) Increased sales More customers Lower General and Administrative expense Lower operating expense Improved Quality Faster cycle time Less waste (e.g. carrying reuse) On-time delivery 2. Performance Measures and GQM Approach (3) • The new measurement and the bottom line

  8. 3. Data Collection and calibration for the process performance (1) The operational activities of measurement begin collecting data. The procedures that we define for collecting and refining data need to be integrated into our software processes and made operation. Collecting data is more than just making the measurements. It consists of the implementing our plans, ensuring the work, and sustaining for the measurement activities of the results. The documentations of our procedure include follows: • Identifying the responsible persons and organizations • Specifying where, when, and how measurements will be done • Defining the procedure to be used for recording and reporting the results

  9. 3. Data Collection and calibration for the process performance (2) We collected the data by performing the assessments with SPICE and marked GQM-based process maturity questionnaire. The requirements help to ensure that the assessment output is self-consistent and provide the evidence for the substantiation of markings. The assessment has to be conducted in accordant to the documented process that in capable of meeting the assessment process. The assessment process shall contain minimally the following activities such as planning, data collection, data validation, process rating, and reporting. The qualified assessors are educated and certified with ISO Guide62 and TR15504, and I identify the responsible persons to be rating on the maturity questionnaire. For the calibration of the isolate data, need to verify the calibration model. These problems are caused by two sources. First, the capability of respondent to the questionnaire is low. Second, it is the case of confusion for the respondent by the item of questionnaire itself.

  10. 3. Data Collection and calibration for the process performance (3) Now, we make a decision to exclude the first source because the reviewer is the certified assessor. If, however, there is a wild deviation between assessors and is too large, there will be some problem in the product of work on the process. And there are some ambiguities in the process definition and its questionnaire. Therefore, we want to verify the confidence of the calibrated model with the hypothesis and assumptions as shown in <Table 1>

  11. 3. Data Collection and calibration for the process performance (4) <Table 1> Assumption and hypothesis to verify the calibrated model Assumption 1 : Current maturing questionnaire is not trusted Assumption 2 : If population follows the normal distribution (Z) by the science of statistics theory, it will be in the administration section (believability) within more than 98%. Assumption 3 : The same (n) of population, which follows the normal distribution by the science of statistics theory, forms T-distribution. Assumption 4 : In case of the distribution sample group (N) comes to population and is to follow the normal distribution (Z), if the sample is followed to the T-distribution by the science of statistics theory. Null hypothesis (H0) : if the sample group (N) is the significant leve (P) = 0.2, that is in reliable if it showed within the confidence interval 99.8%. Alternative hypothesis (H1) : Cannot be trusted sample group (N) is not existed within the confidence interval 99.8%.

  12. Population Data Sample space=>(N) (Inference) n1 n2 n3 n4 Defect Data 3. Data Collection and calibration for the process performance (5) Let us <Table 1> put into a diagram as follows; <Figure 3> Process data verification

  13. 3. Data Collection and calibration for the process performance (7) Therefore, the calibration model established as presented in <Table 2> <Table 2> calibration model of process data

  14. 4. GQM-based Questionnaire for the process capability (1) • Project practical experiences • Size of work product • Defect opportunity • Capability of defect operation • Defect introduction by Software quality assurance activity • Development maturity capability using CBD Architecture • Capability metric subscript with the GQM-based approach I can get the meta data using by the Goal-Question-Metrics approach as follows: • Quality: • Defect density • Impact of requirement change rate • Effectiveness using CBD architecture

  15. 4. GQM-based Questionnaire for the process capability (2) • Delivery: • Delivery time • Reduced time-to-market using CBD architecture • Cycle time: • Classification of process, activity, and task time • Man-Months used(effort distribution) • Size estimation • Efficiency using CBD architecture • Waste: • Rework • Productivity • Reuse • Efficiency using CBD architecture

  16. 5. Process Capability Model to the Earned Value (1) Earned value is measure of budgetary performance that relates actual expenditure to technical achievements as determined by milestone completions. Trend analysis use in conjunction with earned value. In business case, trends are heavily used in forecasting. For examples, managers can consider future raises when estimating labor costs by inflating salaries based on trends in economic index data. They can plot buying trends as a function of their sales forecast. In software organizations during testing, software engineers can examine defect find-and-fix rates to determine whether they are removing defects more quickly than they are finding them. Software managers can plot their rate of progress in achieving milestones as a means to assess their programmatic performance to supplement concept like earned value.

  17. 5. Process Capability Model to the Earned Value (2) 6 Sigma typically monitors the process using control charts, which compare control limits with specification limits to determine process capability. We measure process capability in terms of capability indices which are simplified measures that quickly describe the relationship between process variability and the specification limits’ spread. Process capability is the number of standard deviations that fit between the mean and specification limit; this unit corresponds to defect probability. The essential value of process capability analysis is its ability to predict the percentage of process characteristics or products that the process will produce within specification limits. When analyzing process performance data, we must be constantly concerned that we have identified all sources of variation in the process. When data are aggregated, our process will be particularly susceptible to overlooked or hidden sources of variation.

  18. 5. Process Capability Model to the Earned Value (3) The example that follows shows some of the advantages to calculate the process capability. 1) On the quality of performance pyramid; - work product defect density - technical documentation defect density - code defect density - defect management density - requirement change rates - typical costs : appraisal costs

  19. 5. Process Capability Model to the Earned Value (4) • On the delivery of performance pyramid; • differences between the budget and the actual progress • differences between the plan and the actual progress of planning and analysis activity • differences between the plan and the actual progress of design activity • differences between the plan and the actual progress of construction activity • differences between the plan and the actual progress of testing activity • typical costs : document, reports, records, memos, indoctrination costs of custom process

  20. 5. Process Capability Model to the Earned Value (5) • On the cycle time of performance pyramid; • resources allocation for the planning and analysis activity • resources allocation for the design activity • resources allocation for the construction activity • resources allocation for the testing activity • differences between the budget and the actual human resources • typical costs : policies, procedures, processes, activity • On the waste of performance pyramid; • productivity • documentation size to SLOC • work product size to a man-month • reuse rates of code • accuracy rates of size estimation • typical costs : training, labor hours, travel costs, project costs, administration

  21. yes yes BCWP >= BCWS? BCWP >= Cost? Develop/update Plans, BCWS Performance to plans yes no no Determine Corrective Action 5. Process Capability Model to the Earned Value (6) Now, we can design the earned value system monitors benefis realized; - financial benefits realized vs. cost (ROI) - benefits realized vs. schedule - actual cost vs. schedule and propose classical earned value process as follows (4); BCWS : Budgeted Cost of Work Scheduled, BCWP : Budgeted Cost of Work Performed, Cost : Actual Cost of Work Performed

  22. Assumption INTER MEDIATE OUTCOME •reduce defect • time to market • cycle efficiency • reduce rework ∂F OUTCOME = 0 • • ∂Rq * ∂Rd • productivity • increased customer satisfaction • flexibility ∂F = 0 ∂Rc * ∂Rw 5. Process Capability Model to the Earned Value (7) • Benefit Realization Results Chain to Calculate Earned Value : ① elaboration of results chain assumption - base of estimation ② value management -assumption monitoring - progress/risk monitoring : earned value extension - change management contribution contribution : reduce time to deliver product-delivery effectiveness : reduce time to process

  23. 6. Benefits Realization Model and Conclusions (1) On GQM-Based questionnaire we defined quality and delivery for the external effectiveness, and cycle time and waste for the internal efficiency to performance pyramid as follows; quality ::= Rq (defect management, process capability, SPI cost, component asset), delivery ::= Rd (schedule, progress, marketing, time-to-market) and cycle ::= Rc (process capability, process improvement, resource management) waste ::= Rw (productivity, reduce rework, reuse capability, expenditure rate)

  24. 6. Benefits Realization Model and Conclusions (2) Now we propose the benefit maximize function as follows; F (Rq, Rd, Rc, Rw) = 0. To maximize benefit function for the external effectiveness, ∂F = 0, ∂Rq* ∂Rd and to maximize benefit function for the internal efficiency, ∂F = 0, ∂Rc* ∂Rw

  25. 6. Benefits Realization Model and Conclusions (3) These condition links to assumptions, which condition the realization of outcomes on the benefit-realization approach chain. For the assumptions we can propose the procedures for benefit realization as follows;

  26. n aijA1j i,j =1 6. Benefits Realization Model and Conclusions (4) I. To evaluate process capability and estimate project size, II. To calculate investment cost A0 : the existence project cost A1 : cost for SPI using CBD Architecture A1j : j = 1, n (SPI items), aij : i, j = 1, n (weight of each item) And then A1 = Σ • To make cause-effect graph for the weighted cost • To make cause-effect graph for the weighted external effectiveness and internal efficiency • Now calculate the following benefits;

  27. External Effectiveness + Internal efficiency A0+A1 project benefits project benefits – project cost project costs project cost 6. Benefits Realization Model and Conclusions (5) benefit realization = (%) Benefit / cost ratio = ROI = * 100(%) GQM-Based questionnaire gives us potential indicators of customer satisfaction, process flexibility, and productivity. Using them we can evaluate the benefit realization to software process improvement with CBD architecture which identify process measure to the business indicators and relate the costs and benefits of SPI also to the business indicators.

  28. 요약문 GQM 기반 설문서는 고객에 대한 만족도, 프로세스의 유연성, 및 제품의 생산성을 측정하고 이를 위해서 투자된 비용의 효율을 평가 할 수 있는 데이터를 수집해 준다. CBD 아키텍쳐에 의해서 GQM기반 설문서에 의해서 수집할 데이터를 조정하여 신뢰성 구간을 설정하고 SPI에 대한 현실적인 이익을 계산하는 것은 CBD 아키텍쳐를 사용한 소프트웨어 시스템의 효율을 평가하는 기초가 된다 비즈니스 버전에 맞추어서 CBD 아키텍쳐를 사용하여 개발한 소프트웨어 시스템의 Benefit Realization, Benefit/Cost Ratio, 및 ROI를 계산할 수 있는 모델은 CBD 아키텍쳐의 재사용을 권장할 수 있는 기준과 지침이 될수 있다.

  29. References • William A. Florac, Anita D. Carleton, Measuring the software process, 1999. SEI Series, Addison Wesley. • Richard L. Lynch, Kelvin F. Cross, Measure up!, 1995, Blackwell. • Frank Van Latum, Rini Van Soligen, Adopting GQM-Based Measurement in an Industrial Environment, 1998, IEEE software. • Value-Based software Engineering : A Case Study, Barry Boehm, Computer, March 2003, IEEE

More Related