applying systems engineering

1. Applying Systems Engineering Module 6: Architecture Selection - Trade Studies Welcome !Welcome !

2. Identifying what decisions need making at this level of the System Design is not always easy. System Design decisions should always be based on the System Requirements underlying the topic of the decision. There is never enough time or money to consider all the possible myriad of alternatives to any question.Identifying what decisions need making at this level of the System Design is not always easy. System Design decisions should always be based on the System Requirements underlying the topic of the decision. There is never enough time or money to consider all the possible myriad of alternatives to any question.

3. Where are We in theSE Process? Determine critical sub-set of performance, functional, and/or constraining requirements to develop detailed criteria from. Establish attainable decision criteria and measures for each element of the critical set. Identify relative priorities between the elements of the set. Establish a weighting system based on distribution of priorities. Define weight factors for each decision criteria. Document resulting information in preparation for analyzing alternatives.Determine critical sub-set of performance, functional, and/or constraining requirements to develop detailed criteria from. Establish attainable decision criteria and measures for each element of the critical set. Identify relative priorities between the elements of the set. Establish a weighting system based on distribution of priorities. Define weight factors for each decision criteria. Document resulting information in preparation for analyzing alternatives.

4. The Pitfall of the Point Solution What was the mission of the men marooned on the island? What is the likely outcome of the �Point Solution?� Would a trade analysis have helped?What was the mission of the men marooned on the island? What is the likely outcome of the �Point Solution?� Would a trade analysis have helped?

5. Key Terminology Architecture diverse Consequences Analysis Decision Criteria Functional Alternatives Killer Trades Sensitivity Analysis Trade Study Trade Tree

6. Architecture What is Architecture? Architecture is the solution, or the arrangement of the unit operations (subsystems or components or configuration items) that form the system that solves the problem. Solve the Problem? Yes, performs the functions to meet the requirements (or to the performance level defined by the requirements) that we developed in response to the problem statement, i.e., the Mission Statement or Mission Analysis.

7. Architecture � The Problem And, just how do we do this? First step in Architecture Definition: Identify potential solutions. Hard work, creativity, inventiveness, research and development, thoroughness, intelligence, historical successes, methodical approaches, analysis, modeling, experimentation, muddling, brute force, and so on provides alternatives. Selection of the �Best of Show� Trade studies or decision analysis is then used to select the best available architecture (solution). And what is Best? Therein lies the rub.

8. Identification of Possible Architecture Pick last year�s solution It worked before, it can work again. Classic approach: �Do it the way we�ve always done it!� Chief Engineer Refer to the company expert Use what is available Take what you have and force it to work. Muddling Try everything you can think of or afford. Work without a plan, don�t keep records, you won�t know what went wrong before. Try everything See solution of typical mind problem, Excel file �Houses and Colors�. Read Problem first, on sheet marked �Problem.� Then see Solution sheet. Systems Engineering Systematically review functions and requirements; convene integrated product teams, hold braining storming sessions, hold Value Engineering sessions, generate a multitude of ideas in open work sessions. Repeat if necessary Model Based Design Convert Functional Flow Block Diagram into a model (computer, mathematical, or mechanical). Identify a unit operation to perform each function. Continually test, update and improve model. Example � Model of SBW treatment.

9. Identification of Possible Architecture Using prior slide, have class discussion of approaches they have used, observed, participated in. Which were successful? Why? Under what conditions?

10. Selection of the �Best of Show� We now have a number of candidate solutions, ranging from 0 to infinity Next step is to select the �preferred choice� The preferred choice should be the �best choice� But what is the Best? Cheapest, fastest, highest quality, optimum, CEO�s choice, default if only one is available, Decision analysis is then used to select the best available architecture (solution) If there is 0 (no solution) or 1 (point solution) candidate, the decision is trivial If there are more than 1 candidate, then the analysis performed to support the decision is a Trade Study

11. About the Term Trade Study Often referred to as Trade-off Study Incorrectly written Tradeoff Study Google search shows that Trade Study is preferred over Trade-off Study 5 to 1 ISO 15288 uses Trade-off Study, INCOSE Handbook uses Trade Study. Often used interchangeably with alternatives study INEEL Engineering just does a trade and publishes it in an EDF (Engineering Design File) without a name. Value Engineering is a serious contender, often confused with a Trade Study

12. Who does What? One final word before we begin to learn about Trade Studies There will be a decision maker, who represents the stakeholders There will be the engineering or research team that develops the alternatives There is the systems engineer who does the Trade Study Each has a separate role, albeit in small projects these roles may all be assigned to one person.

13. Why do Trade Studies? Management cannot make key programmatic decisions without detailed, rigorous, verifiable data. Customer expects/demands trade studies be performed on all major system elements. Need something to provide technical defensibility for the program Trade studies provide an accepted method to document choices for management decisions. We do trade studies automatically in the course of good engineering practice. Even if a trade study reveals that the point solution (single candidate) was the preferred solution, it now has documented justification as the preferred solution. Trades(trade studies, trade-off studies) provide the needed information for decision making. Trades provide documentation to support the decision and should become subject to document control as part of the design documentation. Trades, their documentation and presentation, eliminate second guessing of the design. Customer requirements for trade studies as deliverables ensure that the customer will have visibility into the contractor�s design development. Trades(trade studies, trade-off studies) provide the needed information for decision making. Trades provide documentation to support the decision and should become subject to document control as part of the design documentation. Trades, their documentation and presentation, eliminate second guessing of the design. Customer requirements for trade studies as deliverables ensure that the customer will have visibility into the contractor�s design development.

14. Trade Studies Consist of Many Facets

15. Trade Study Approach Note the iterations possible you can revisit the Decision Criteria based on the Sensitivity Analysis you can revisit the Alternatives based on the Adverse Consequences Analysis there is a hidden iteration coming in Synthesis trade studies methodologies are used throughout the SE process. This course focuses on only two of the many used of trade studies: Functional trades: to eliminate unacceptable sub-system alternatives System trades: to identify preferred system solution(s).Note the iterations possible you can revisit the Decision Criteria based on the Sensitivity Analysis you can revisit the Alternatives based on the Adverse Consequences Analysis there is a hidden iteration coming in Synthesis trade studies methodologies are used throughout the SE process. This course focuses on only two of the many used of trade studies: Functional trades: to eliminate unacceptable sub-system alternatives System trades: to identify preferred system solution(s).

16. Trade Study ApproachSteps 1-5 (This is at program start!) 1. Determine what System Design Decisions need to be made 2. Develop a Trade Tree of System Decisions based on current knowledge Establish System Design Decision Criteria. Stakeholders and decision makers should concur with these criteria. SE�s structure identification. Often times, Stakeholders don�t care until they see potential results. However, CERCLA has formally established criteria a) Based on System Requirements b) Nmerical if possible c) Threshold (must have) and Balancing Criteria (desirable attributes) d) Independent 4. Assign weights to the Decision Criteria 5. Develop all suitable Alternatives All these steps were covered in the last moduleAll these steps were covered in the last module

17. Trade Study ApproachSteps 6-11 6. Perform a Killer Trade if needed (Killer is a jargon term used by some to mean trade criteria that eliminate the alternative on a single criterion, also known as Musts vs Wants in KT.) 7. Have discipline experts score the alternatives 8. Capture trade study results on a Decision Analysis Worksheet or other form 9. Perform a Sensitivity Analysis. Does any one variable or combination of variables drive the decision? Show on the Analysis Worksheet. 10. Perform an Adverse Consequences Analysis. Revise the alternatives if needed. 11. Final documentation of trade. (Should be placed under configuration control.) Killer Trades use Threshold Criteria to reduce the number of alternatives to a reasonable number. Too many alternatives usually means a lack of independence between them. Alternatives should be scored by the experts without knowledge of the weights. The experts will have point solutions as well. Proper presentation of the trade is key to acquiring customer buy in. Sensitivity Analyses may refine the decision criteria to sharpen the trade. Adverse Consequences Analyses may eliminate an alternative or add a criterion Document, document, document! Killer Trades use Threshold Criteria to reduce the number of alternatives to a reasonable number. Too many alternatives usually means a lack of independence between them. Alternatives should be scored by the experts without knowledge of the weights. The experts will have point solutions as well. Proper presentation of the trade is key to acquiring customer buy in. Sensitivity Analyses may refine the decision criteria to sharpen the trade. Adverse Consequences Analyses may eliminate an alternative or add a criterion Document, document, document!

18. Step 1: Determine Key Decision Points Review Requirements Did a majority of customers have the same requirement? Are there known constraints (i.e., cost, risk, schedule)? Review Functions Are any functions new or questionable? What functions exist in the top tiers of the hierarchy?

19. Step 2: Develop aTrade Tree A trade tree can be functionally oriented (see example) based on key decisions (see EMI WIPP/NO WIPP Trade Tree) or a combination (though these can be unruly).A trade tree can be functionally oriented (see example) based on key decisions (see EMI WIPP/NO WIPP Trade Tree) or a combination (though these can be unruly).

20. Step 3: Establish Decision Criteria How do we select between alternatives? Look at: System Requirements TPMs compliance measures other factors (i.e., political issues, public concerns) Determine how to measure them absolute vs. comparison units or scales Focus on the discriminators � you won�t have the resources to study everything Always Consider Cost and Schedule Impacts Document Them If a requirement is anything that is needed, then it is very likely that it will be impossible for any one solution to meet all of someone�s needs. Trade-offs will have to be made, compromises reached. Priorities need to be set. Page 18 foot note 11.If a requirement is anything that is needed, then it is very likely that it will be impossible for any one solution to meet all of someone�s needs. Trade-offs will have to be made, compromises reached. Priorities need to be set. Page 18 foot note 11.

21. Qualitative Criteria Guidelines Are composed of comparitors and figures of merit; The comparitor provides a basis for a comparison to be made The figure of merit defines how close the criteria are to the basis Subjective figures (i.e., good, better, best) or ROM figures (i.e., twice as expensive, five times safer) are typically used Should always be tied to a known (or suspected) customer issue or constraint Can show the possibility/probability of meeting the constraint, resolving the issue, avoiding the issue Should be expressed in terms or graphics meaningful to those making decisions based on data collected (think Consumer Reports) Provide numerous examples Tailor examples to students background and activities Mention the KISS principle Provide numerous examples Tailor examples to students background and activities Mention the KISS principle

22. Quantitative Criteria Guidelines Composed of a number and unit of measure; The number provides a magnitude (how much) The unit of measure gives the number meaning (what) Should be expressed in units of measure meaningful to those making decisions based on data collected Should be tied to performance or functional requirements Can be represented by single dimensional units (e.g., hours, meters, dollars) ... but ... Multi-dimensional units of measure are typically used (e.g., miles/gallon, dollars/unit quantity) Provide numerous examples Tailor examples to students background and activities Mention the KISS principle Provide numerous examples Tailor examples to students background and activities Mention the KISS principle

23. Keys to Good Decision Criteria Based on Requirements Hierarchical Basis Good Discriminators Independence Quantitative Documented Criteria not based on requirements are usually invalid. If the systems design decision was based on the functional analysis, most of the criteria will come directly from the requirements allocated to the function in question. Organize your criteria Each criterion should clearly point out differences between any alternatives. Each criterion should be independent of the others, i.e., do not use both �Simplicity� and �Complexity� as criteria. Numbers are better than loose terms such as worst, bad, even, better, best. You must always document your criteria and how you got them so neither you nor your customer will second guess your results.Criteria not based on requirements are usually invalid. If the systems design decision was based on the functional analysis, most of the criteria will come directly from the requirements allocated to the function in question. Organize your criteria Each criterion should clearly point out differences between any alternatives. Each criterion should be independent of the others, i.e., do not use both �Simplicity� and �Complexity� as criteria. Numbers are better than loose terms such as worst, bad, even, better, best. You must always document your criteria and how you got them so neither you nor your customer will second guess your results.

24. Step 4: DefineWeighting Schemes Start with weight = 1 Rarely is there a better weight than 1 Consider pair-wise comparisons (qualitative judgments converted to quantitative weights) Employ Analytical Hierarchy Processes (probabilities) Should reflect the allocation of requirements to functions

25. Step 5: Develop Alternatives In this Functional trade application, consider ALL possible ways to perform each system function.

26. Step 6: �Killer Trades� Too many alternatives indicate poorly understood problem interdependent alternatives invalid alternatives included hopeless situation Killer Trade is a simple trade to eliminate unsuitable alternatives specifically use threshold criteria (must have) eliminate alternatives that fail threshold criteria or score particularly low Document the Killer Trade Use the Keep It Simple Stupid (KISS) principle Unsuccessful Killer Trade may indicate more work needs to be done on the Trade Tree Killer Trades are short, quick, and easy Killer Trades may provide the preferred solution The desired result of the entire SE process is to be �approximately right, not exactly wrong! There is never enough time or money to be exactly right.Use the Keep It Simple Stupid (KISS) principle Unsuccessful Killer Trade may indicate more work needs to be done on the Trade Tree Killer Trades are short, quick, and easy Killer Trades may provide the preferred solution The desired result of the entire SE process is to be �approximately right, not exactly wrong! There is never enough time or money to be exactly right.

27. Step 7: Scoring the Alternatives Collect Data from subject matter experts affected customer group(s) Typical Tools Written / Oral Surveys Prototypes Vender Specifications Modeling / Simulation Develop written rules for scoring!

28. Step 8: Capture Trade Study Data Trade Study Description The Trade Table is a good means for communicating trade results to management. However, have slides of all the trade documentation in your back pocket. Generally, Trade Tables show pros and cons. Developed after the trade is complete.The Trade Table is a good means for communicating trade results to management. However, have slides of all the trade documentation in your back pocket. Generally, Trade Tables show pros and cons. Developed after the trade is complete.

29. Trade Study Results This an example for steps #8 and #11 (see Slide8). This an example for steps #8 and #11 (see Slide8).

30. Step 9: Sensitivity Analyses Always required if some alternatives score closely Provides trade validation Sensitivity Analysis Process vary criteria weights one at a time (increase or decrease and re-normalize) and recompute total weighted scores identify insensitive criteria - very little change in scores and rankings of alternatives consider eliminating insensitive criteria and if so, recompute total scores Document the analysis Sensitivity analyses provide good trade study validation. Documentation of the sensitivity analysis eliminates second guessing on decisions Sensitivity analyses provide good insight into system design decision criteria developmentSensitivity analyses provide good trade study validation. Documentation of the sensitivity analysis eliminates second guessing on decisions Sensitivity analyses provide good insight into system design decision criteria development

31. Step 10: Adverse Consequences Analysis Consider for each alternative that is not a clear loser Brainstorm all potential drawbacks for selecting each considered alternative look at lower scoring criteria, expand on why score is so low consider interface impacts consider risk factors Reconsider alternatives based on adverse consequences - reconfirm rankings Document analysis as part of trade Adverse consequences analyses are another good trade validation process. They must be documented as part of the trade. They may modify the alternatives being considered.Adverse consequences analyses are another good trade validation process. They must be documented as part of the trade. They may modify the alternatives being considered.

32. Step 11: Documentation

33. Trade Studies In Summary Killer Trades help eliminate unsuitable alternatives (typical of large �brainstorming sessions� Scoring alternatives should be performed by discipline experts without knowledge of the weights Good documentation of trades eliminates second guessing on decisions Even if the point solution wins, it�s justification is documented

34. Objectives Review Identify Key Terminology Describe a generic Trade Study Methodology Apply Trade Study Methodology to conduct Functional Trades. Identifying what decisions need making at this level of the System Design is not always easy. System Design decisions should always be based on the System Requirements underlying the topic of the decision. There is never enough time or money to consider all the possible myriad of alternatives to any question.Identifying what decisions need making at this level of the System Design is not always easy. System Design decisions should always be based on the System Requirements underlying the topic of the decision. There is never enough time or money to consider all the possible myriad of alternatives to any question.