1 / 42

Strategic Engineering: Designing Systems for an Uncertain Future

Change Propagation Analysis in Complex Technical Systems. Strategic Engineering: Designing Systems for an Uncertain Future. Pratt & Whitney Fellows Lecture UTRC, East Hartford, Connecticut. Olivier L. de Weck, Ph.D. deweck@mit.edu

emlyn
Télécharger la présentation

Strategic Engineering: Designing Systems for an Uncertain Future

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Change Propagation Analysis in Complex Technical Systems Strategic Engineering: Designing Systems for an Uncertain Future Pratt & Whitney Fellows Lecture UTRC, East Hartford, Connecticut Olivier L. de Weck, Ph.D. deweck@mit.edu Associate Professor of Aeronautics & Astronautics and Engineering Systems Associate Director – Engineering Systems Division June 3, 2009

  2. Olivier de WeckAssociate Professor of Aeronautics and Astronautics and Engineering Systems, Associate Director of ESDMIT School of Engineeringdeweck@mit.edu • Background: dipl. Ing. in Industrial Engineering, ETH Zurich 1993 Engineering Program Manager, Swiss F/A-18, 1994-1997 SM and PhD in Aerospace Systems Engineering, MIT 1999-2001 • Research Focus: Strategic Engineering of Complex Systems: How can they be designed with flexibility to evolve over time, while exploiting commonality across projects? • Work In: Aerospace Systems (NASA space exploration, communications satellites), Automotive (GM, ArvinMeritor), Oil & Gas Industry (BP), Complex Electro-Mechanical Products (Xerox,…)

  3. Outline • Motivation • F/A-18 Experience • Sources of Engineering Change • Change Propagation Analysis • Analysis of Change Requests at Raytheon IDS • Generalized Method and Insights • Technology Infusion Analysis • Infusion of New Technologies in Digital Printing at Xerox • Technology Infusion Analysis Framework • Strategic Engineering Framework • Discussion

  4. F/A-18 Experience Multidisciplinary systems like military aircraft are very complex and highly coupled Typically optimized for mission performance Difficult to change design when requirements change during lifecycle Issue of Engineering Changes is critical: - some were anticipated (avionics, software) - others were not (structural airframe) - how can designing for changeability be made more deliberate and systematic? Change Propagation Example: Material substitution from aluminum to titanium (to increase fatigue life) in fuselage caused changes to mass properties, eigenmodes, manufacturing, flight control software etc…

  5. F/A-18 Center Barrel Section (cont.) Y488 Y470.5 Wing Attachment Y453 74A324001

  6. Manufacturing Processes Changed Original Change Fuselage Stiffened Flight Control Software Changed Center of Gravity Shifted Gross Takeoff Weight Increased F/A-18 Complex System Change F/A-18 System Level Drawing

  7. de Weck’s focus How to design products and systems for uncertainty? • Robust Design: Design systems so that they will perform “adequately” over a large range of future operating conditions (passive approach) • Flexible Design: Design systems so that they can “easily” be changed to adapt to uncertain future conditions (active approach) • When is this important? (10-10-10 Rule) • Long lifecycles (> 10 years) • Large irreversible investments (> $10 million) • Large requirements/usage uncertainty (>10% volatility per annum) • Why is it important? • Systems and products not designed for changeability oftentimes: • Are locked into a suboptimal configuration operational inefficiencies • Require expensive a-posteriori modifications cost of changes/retrofits • Need to be prematurely replaced with new systems replacement cost and time • This raises important research questions: • How and why are engineering changes initiated? • How do changes propagate through a complex, multidisciplinary and coupled system and can such propagation be quantified a-posteriori and predicted a-priori? • How can systems be designed to be more changeable? • Are there underlying principles and methods of changeability across multiple domains?

  8. How and why are changes Initiated? Drivers for change exist in all lifecycle phases! Eckert C., de Weck O., Clarkson J., et al., “ENGINEERING CHANGE: DRIVERS, SOURCES, AND APPROACHES IN INDUSTRY” , INTERNATIONAL CONFERENCE ON ENGINEERING DESIGN, ICED'09, 24 - 27 AUGUST 2009, STANFORD UNIVERSITY, STANFORD, CA, USA (accepted)

  9. Approaches for Managing Engineering Change Quality Function Deployment (QFD) for Change Minimization Most firms viewed change as something to be avoided but recognized that the ability to change is critical at the same time Formal Change Management

  10. Change Propagation AnalysisSystems Engineering for Changeability • Change Propagation Analysis • Giffin M, de Weck O.L., et al., Change Propagation Analysis in Complex Technical Systems, DETC2007-34652, Proceedings of ASME IDETC/CIE 2007, in press ASME Journal of Mechanical Design Network plot of largest change network of 2579 associated change requests in radar system design at Raytheon (tracking mode).

  11. Complex Sensor System Complex hardware, software, human operators Derivative of earlier system 9 Year development 46 Areas Hardware Software Program Documentation System Map (graph) Interconnections between areas System Description

  12. Change Request Database technical, managerial, procedural track parent, child, siblings by areas with unique ID number chronologically numbered IDs Data Mining Procedure Export from DBMS to text file Written into MySQL database with Perl scripts Equivalent to a MS Word document with 120,000 pages Sorting, Filtering, Anonymizing Write simplified change request format (see right side) Data Set Typical Change Request

  13. There is an inverse relationship between expected change magnitude and its frequency of occurrence. Very large changes are rather infrequent while small changes on the other hand are commonplace. Nearly half the small (magnitude 0) changes were either withdrawn, superseded or disapproved (46.6%), whereas nearly all the large changes (magnitude 5) were approved and carried out to completion (96.7%). Initial Analysis

  14. Apply Graph Theory to extract networks of connected changes parent-child changes sibling changes Most changes are only loosely connected 2-10 related changes Some large networks emerged Question: do these networks emerge from a single initial change? Change Networks Legend Change proposed parent child Change rejected sibling sibling Change implemented

  15. Change Propagation Network Network plot of largest change network in the dataset, with 2579 associated change requests.

  16. 1: Y5 new CRs ID 1-22000 Analysis of 87CR Network 8000 12156 13320

  17. 2: Y6 new CRs ID 22000-26000 23942 23945 23992 24980 23729 23922 23024 23821 24781 23925 23831 25481 8000 22850 24659 25053 25476 24927 25515 12156 24926 22946 25463 13320

  18. 3: Y7 new CRs ID 26000-29000 27585 28213 28187 28007 28166 28122 28153 27027 28695 28790 28567 28788 23942 28846 27627 28878 23945 28531 26333 28528 27656 28428 28009 26331 23992 28186 27169 24980 23729 28067 27023 23922 23024 28529 28821 23821 24781 23925 28601 28162 28696 23831 25481 8000 22850 24659 25053 25476 27952 24927 25515 12156 24926 22946 25463 27592 26117 13320

  19. 4: Y7 new CRs ID 29,000-31,000 27585 30143 28213 28187 28007 30344 28166 28122 28153 27027 28695 28790 28567 28788 29538 30614 29547 23942 29399 28846 27627 30465 28878 23945 28531 26333 30148 28528 27656 28428 28009 26331 29711 23992 28186 27169 24980 23729 28067 27023 23922 30771 23024 28529 28821 30126 23821 30548 29353 29731 29826 29226 30466 30501 24781 29227 23925 30503 28601 28162 29744 28696 23831 25481 8000 22850 24659 25053 25476 27952 24927 25515 12156 24926 22946 25463 27592 26117 13320

  20. 5: Y7 new CRs ID 31000-32645 27585 30143 28213 28187 28007 30344 28166 28122 28153 27027 28695 28790 28567 28788 29538 30614 29547 23942 29399 28846 27627 30465 28878 23945 28531 26333 30148 28528 27656 28428 28009 26331 29711 23992 28186 27169 24980 23729 28067 27023 23922 30771 23024 28529 28821 32289 30126 23821 30548 29353 29731 29826 29226 30466 30501 31471 24781 29227 23925 30503 28601 28162 29744 28696 23831 25481 8000 22850 31973 24659 25053 31972 25476 27952 24927 25515 32645 12156 31966 24926 22946 31235 25463 27592 26117 13320 31967

  21. 6: Y8/9 update final status of all CRs 27585 30143 28213 28187 28007 30344 28166 28122 28153 27027 28695 28790 28567 28788 29538 30614 29547 23942 29399 28846 27627 30465 28878 23945 28531 26333 30148 28528 27656 28428 28009 26331 29711 23992 28186 27169 24980 23729 28067 27023 23922 30771 23024 28529 28821 32289 30126 23821 30548 29353 29731 29826 29226 30466 30501 31471 24781 29227 23925 30503 28601 28162 29744 28696 23831 25481 8000 22850 31973 24659 25053 31972 25476 27952 24927 25515 32645 12156 31966 24926 22946 31235 25463 27592 26117 13320 31967

  22. Observations • The 87CR network did not initiate with a single CR and then grow gradually by change propagation • Several initially unrelated changes grew together to form a larger network over time • A few changes are highly connected • Examples: 24781(7), 29226 (7), 28009 (7) • highly connected changes are not necessarily parent changes • Most changes only connect to one or two other changes

  23. Bi-Partite Graph Analysis 30143 27585 28213 28187 28007 30344 28166 28122 27027 28153 28695 28567 28788 28790 29538 30614 29547 23942 29399 28846 27627 30465 28878 23945 28531 26333 30148 28528 27656 28428 26331 23992 29711 28009 28186 27169 24980 23729 28067 23922 27023 30771 23024 32289 28821 28529 30126 29826 23821 29226 29353 30548 29731 System Network Map 30466 30501 31471 23925 24781 30503 29227 28601 28162 29744 28696 23831 8000 22850 25481 31973 25053 24659 31972 27952 24927 25476 25515 32645 31966 12156 22946 24926 31235 27592 25463 31967 26117 13320 Change Propagation Network

  24. 87CR-area classification perfect reflectors Area 5 Reflectors Area 19 Area 3 Area 10 Acceptors Areas 4, 6, 14, 20 Area 11, 23, 35 Area 1 no CRs issued or all CR’s unresolved perfect acceptors

  25. Change Propagation Index (CPI) change propagation probability • Classify each area • Absorber, Carrier, Multiplier total completed changes in Area j instigating area DDSM Change Propagation Frequency receiving area A change in Area 1 caused changes in Area 6 with a frequency of 4.17%. -1 <= CPI <= +1

  26. System Area Classification CPI Spectrum • Areas found to be strong multipliers • 16: hardware performance evaluation • 25: hardware functional evaluation • 5: core data processing logic • 32: system evaluation tools • 19: common software services • 3: graphical user interface (GUI) • Areas found to be perfect reflectors • 27, 41: look like perfect absorbers • but actually zero changes implemented • despite numerous changes proposed • = perfect reflectors

  27. Discovered new change pattern: “inverted ripple” system integration and test bug fixes [Eckert, Clarkson 2004] subsystem design major milestones or management changes component design Change Request Generation Change Requests Written per Month 1500 1200 900 Number Written 600 300 0 1 5 9 77 81 85 89 93 73 13 17 21 25 29 33 37 41 45 49 53 57 61 65 69 Month

  28. Recent result (last week) • Change pattern on a successful oil & gas project What factors lead to front vs. backloading of change activity?

  29. Future Work on Change Propagation • Change Prediction: evaluate change components (networks) and paths in detail versus those predicted by current tools. • How good are our predictions regarding propagation paths versus actual changes? • How good are our predictions regarding actual versus planned effort? • How can change propagation patterns observed on past projects be leveraged for design decisions (e.g. drawing module boundaries, actively embedding change absorbers, decomposing multipliers) in new projects? • Data Processing: standardize methods for refining data, tracing large change networks in greater depth- attempt to reconstruct logic, particularly where proposed changes were rejected or were superseded by other changes. • Staffing: analyze effects of staffing on changes and components • What differences occur in change propagation patterns based on which personnel work on the changes? • Can change-related personnel information be used for performance and status assessments in technical organizations? • Contractual: can change propagation information be used to write better prime and sub-contracts such that contractual changes and amendments can be incorporated more easily in those projects involving multiple firms. • Statistical: are there critical numbers for change propagation? Limits on the number of propagation steps? If a change network grows, are additional changes more or less likely? All of these will require close industry-academia collaboration !

  30. Technology Infusion Analysis [Smaling R., de Weck O., Assessing Risks and Opportunities of Technology Infusion in System Design, Systems Engineering, 10(1), 1-25, 2007 Technology Infusion AnalysisSystems Engineering for Changeability How to quantify costs and benefits of technology infusion under uncertainty? (INCOSE 2007 Outstanding Paper Award)

  31. Technology Infusion (System Level) Technology Development DDSM Model Capital Investment (NRE) uncertainty Technology Infusion (Subsystem Level) uncertainty Test Vehicle Economy Vehicle add-on Cost ($) CAD Model uncertainty Engine Integration Vehicle Fleet Technology Societal Impact (Super-System Level) Competition Regulations uncertainty Improved Emissions Fuel Economy Environment [2.14] Smaling R., de Weck O., “Assessing Risks and Opportunities of Technology Infusion in System Design”, Systems Engineering, 10(1), 1-25, Spring 2007

  32. Industry Case StudyXerox iGen3 Technology Infusion iGen3 Digital Printing Press Auto Density Correction Technology • High-end digital printing market is very competitive market space, where demand for lower operating cost and superior image quality is key to product survival. • To improve performance metrics mentioned above, an Auto Density Correction Technology has been infused into current Xerox iGen3 Digital Production System. Suh. E.S., Furst M.R., Mihalyov K.J, de Weck O., “Technology Infusion for Complex Systems: A Framework and Case Study”, SE-080701, Systems Engineering, 13 (3), 2010 (in press)

  33. ASME 2008 International Design Engineering Technical Conference August 3-6, 2008, New York, New York, USA GUI Feeder Software Print Engine Image Path Stacker Print Engine Media Path Print Engine Marking Path Print Engine Control Path Print Engine Frame iGen3 Baseline Design Structure Matrix (DSM) “Characterize the system”

  34. ASME 2008 International Design Engineering Technical Conference August 3-6, 2008, New York, New York, USA Infused Technology – ΔDSM captures Changes Impact of Technology Infusion on Current System Technology ΔDSM TI is the unweighted ratio of actual changes over possible changes TI (Technology) ~= 8.5% • Complete ΔDSM for Auto Density Correction Technology • captures all changes made to basic system to infuse the technology • count number of cells in baseline DSM affected by technology • compute technology invasiveness index (between 0 and 100%) • also estimate non-recurring effort (engineering hours)

  35. ASME 2008 International Design Engineering Technical Conference August 3-6, 2008, New York, New York, USA Technology Infusion Framework D DDSM Step 3 TI Effort Technology Infusion Identification 1 4 Baseline Product Value V(g) Baseline System DSM 2 6 Modified Product Value V(Dg) 5 Performance and Cost Models D 7 E[DNPV] Revenue Impact Risk-Return Curve for Technology Probabilistic NPV Analysis 8 Cost Impact 10 9 Technology Infusion Evaluation s[DNPV]

  36. Cost Benefit Analysis: ΔNPV • Nominal ΔNPV estimated using upfront development cost and recurring variable cost and savings. • Monte Carlo simulation performed to estimate range of ΔNPV, given uncertainty in future demand of the technology infused product and post-sale cost savings

  37. flexibility real options commonality platforms Design for Changeability Design for Commonality Spatial Dimension Temporal Dimension changes standardization uncertainty variety at t=to+Dt requirements change and x* is no longer optimal more than one variant of the system is needed: x1*, x2, … xn Strategic (Systems) Engineering – “Lung Chart” Design of complex products and systems (10-10-10 Rule) technology markets regulations System Architecture concept Integrated Modeling and Simulation performance, cost, risk Multidisciplinary Design Optimization “optimal” design x* at t=to

  38. Principles of Strategic Engineering • A rigid design will be optimal (max NPV) if future events unfold exactly as forecasted • A robust design can minimize the standard deviation of outcomes (reduce risk), but will usually also lower the expected NPV and max achievable NPV (limit opportunity) • The larger the degree of uncertainty, the more valuable flexibility (=ease of change) will be. Flexible designs can increase the E[NPV], while limiting downside and maximizing upside • The larger the change costsfrom one configuration to another the more likely that the current system will be continued due to “architectural lock-in”, despite operational sub-optimality

  39. “we are betting the farm” Strategically Redesign Flexible Design “we can adapt” Optimize for Expected Requirement Robust Design “we will be ok no matter what” “we know what’s coming” Strategic Engineering Map Degree of Uncertainty s Relative Change Costs DC/LCCr

  40. Where do various systems at UTC fall ? Degree of Uncertainty Otis s communication satellites Sikorsky Elevators Commercial Aircraft ? wireless sensor networks automotive platforms Aircraft Engines P&W consumer products Refrigeration Systems ? highway infrastructure ? Carrier water supply system Relative Switching Costs DC/LCC

  41. Strategic Engineering • Strategic Engineering is the process of designing systems and products in a way that deliberately accounts for future uncertainties and change such that their lifecycle value is maximized. • Review of research questions: • How and why are engineering changes initiated? • How do changes propagate through a complex, multidisciplinary and coupled system and can such propagation be quantified a-posteriori and predicted a-priori? • How can systems be designed to be more changeable? • Are there underlying principles and methods of changeability across multiple domains? • For more information visit: • http://strategic.mit.edu

  42. Questions?

More Related