The authors’ affiliation with The MITRE Corporation is provided for identification purposes only, and is not intended to

1. Requirements engineering for dependable systemsTutorialJoint Meeting ofDC/Baltimore SIGAda Chapter DC Chapter of ACM11 October 2005Dr. William Bailwbail@mitre.orgThe MITRE Corporation The authors’ affiliation with The MITRE Corporation is provided for identification purposes only, and is not intended to convey or imply MITRE's concurrence with, or support for, the positions, opinions or view points expressed by these authors.

2. Agenda • Introduction to tutorial content • Dependability and requirements • Nature of requirements • Developing requirements • Quality factors for requirements • Common pitfalls • Summary

3. What we will cover • The nature of dependability • The nature and role of requirements • The various types of requirements, their role in development, their impact on system success • Important characteristics of requirements (quality attributes) • The different ways that requirements need to be handled • A set of common challenges and strategies for how to manage them • What we will not cover • Particular approaches to requirements definition (such as specific tools and techniques) – this is beyond the scope (and time)

4. Agenda • Introduction to tutorial content • Dependability and requirements • Nature of requirements • Developing requirements • Quality factors for requirements • Common pitfalls • Summary

5. Focus is on dependability • As defined by IFIP WG-10.4 "The trustworthiness of a computing system which allows reliance to be justifiably placed on the service it delivers“ http://www.dependability.org/ • Dependability is partially a concern of the systems’ specification which defines the services to be delivered • The specification defines (or should define) what aspects of the system need to be dependable (what services) • and what dependable means in the context of that specific system • The nature of reliance must be defined • Total conformance to spec, or • Degraded levels allowed, or • Optional feature, or….

6. Some aspects of reliance • The system does not fail when a service is requested • Software users, selfishly so, expect the systems they use to be completely dependable • But may be unwilling to pay the cost of developing such systems • In practice, experienced users of specific systems learn the undependable features of the system and work around them • Sometimes, this is unacceptable • When developing the requirements for a system, the nature of the reliance must be clearly defined • What do the users expect? • What do they need? • What will they tolerate? • Where are the risks and dangers?

7. Dependability • Characterizing dependability has been an important focus for years • e.g., NASA’s HDCP – High Dependability Computing Project • http://hdcp.org/ • HDCP “investigates and evaluates new approaches for improving NASA's ability to create dependable software for mission applications” • Participants: • Carnegie Mellon University > University of Washington • Massachusetts Institute of Technology > NASA Ames Research Center • University of Maryland > University of Wisconsin-Milwaukee • University of Southern California • 1970 – IEEE-CS TC on Fault-Tolerant Computing • 1980 – IFIP WG 10.4 - Dependable Computing and Fault Tolerance • 1985 - J.-C. Laprie dependability synthesis • 1989 – IFIP Working Conference on Dependable Computing for Critical Applications

8. Laprie Dependability Tree Avižienis, Algirdas, Jean-Claude Laprie, and Brian Randell. “Fundamental Concepts of Computer System Dependability”. IARP/IEEE-RAS Workshop on Robot Dependability: Technological Challenge of Dependable Robots on Human Environments. May 21-22, 2001

9. Dependability attributes (page 1 of 2) • Availability – the ability of the system to be ready for operation at any point in time • Usually defined as the proportion of time that the system is ready for use, to the total time period of interest • e.g. if the goal is for the system to be available for use for all but 4 seconds per year, then the availability is 0.9999999 over the year. • Reliability – the ability of the system to operate for a period of time without failure • Defined as either the failure rate (failures per time period) or as the average operational time between failures (MTBF). A typical reliability requirement would be 100 hours, meaning that the system is required to operate continuously for 100 hours on the average before failing. • Safety – the ability of the system to avoid actions that result in catastrophic actions that result in human injury, large financial loss, or damage to important assets • Includes actions that are required as well as actions that must be avoided.

10. Dependability attributes (page 2 of 2) • Confidentiality – the ability of the system to avoid disclosure of information to unauthorized recipients • Integrity – the ability of the system to avoid being corrupted, including: • the correctness and consistency of the software code and the data structures that contain the information being processed • the system’s ability to avoid corruption while it is executing, including protection against unauthorized modification or destruction of the information and the code. • Maintainability – the ability of the system to be easily repaired, enhanced, or modernized

11. Information Assurance • A newer thrust regarding dependability – IA – expands on the Dependability Tree • Availability • Integrity • Confidentiality • Authentication – A security measure designed to establish the validity of a transmission, message, or originator, or a means of verifying an individual's authorization to receive specific categories of information. • Non-repudiation – An assurance that the sender of data is provided with proof of delivery and the recipient is provided with proof of the sender's identity, so neither can later deny having processed the data. as previously discussed Department of Defense. Information Assurance - DoD Directive.8500.1. Oct 2002.

12. Acceptability • Overall, key focus is on what the stakeholders want and need • Defined through the requirements • Generally willing to make tradeoffs among various system attributes • Higher cost in exchange for more functionality • Late delivery just to get the system operational while it is still useful • Tradeoffs can be expressed in an Acceptability Framework • A classification of requirements and system attributes organized to permit making trade-offs to determine whether system is acceptable for use

13. Acceptability Framework

14. Agenda • Introduction to tutorial content • Dependability and requirements • Nature of requirements • Developing requirements • Quality factors for requirements • Common pitfalls • Summary

15. What are requirements? • IEEE Std. 610.12-1990, IEEE Standard Glossary of Software Engineering Terminology Requirement: (1) A condition or capability needed by a user to solve a problem or achieve an objective. (2) A condition or capability that must be met or possessed by a system or system component to satisfy a contract, standard, specification, or other formally imposed documents. (3) A documented representation of a condition or capability as in (1) or (2). See also: design requirement; functional requirement; implementation requirement; interface requirement; performance requirement; physical requirement.

16. Analysis of definition • IEEE definition is broad – term often used carelessly • Popular use often ignores this definition • Anything that is “required” • Ranges from hopes to dreams to budgets to schedules... • “This schedule is required”…”This budget is required”…“These software (SW) components must be used”…”These algorithms must be used”… • Different types of requirements need to be handled differently • because they affect system/SW development in different ways • This section of the tutorial focuses on differentiating different types of requirements • and on recommending how to handle them appropriately • Remember that all types of requirements are important in some way Just because something is a requirement, does not mean that it is a requirement Huh?

17. Could you repeat that? • I’m sorry – I must have missed something • Just because someone says ”I require xxx” does not mean that this is a software requirement • That is, it might not be a behavioral specification • “I require this product to be developed within 6 months” • A constraint on development period but says nothing directly about product behavior • Important, but plays different role • Our focus is on keeping different types of requirements separate • and managed appropriately Just because something is a requirement, does not mean that it is a requirement “You get what you spec, not what you expect”

18. IEEE Std 830-1998 • IEEE Std 830-1998 – IEEE Recommended Practice for Software Requirements Specifications: “A requirement specifies an externally visible function or attribute of a system” • We can see inputs and the outputs, but not what happens inside • For any product (SW, HW, total system), the behavioral requirements for that product specify its externally visible behavior • as seen by other systems outside System

19. Levels of systems • But each such system could be part of a larger system • Which has its own requirements (externally visible behavior) System

20. Context of requirements • All requirements are defined in context of a specific component (e.g., black box) • Which may consist of additional constituent components (e.g., subsystem, modules,...) • Hence there are multiple levels of requirements based on level of component • System level, subsystem level, software configuration item (SCI) level, component level, software unit level,... • Component design (its architecture) consists of: • The requirements for behavior of each constituent component • The interrelationships between the components • Interaction of components produces the behavior of parent component Input Output Component A Requirements

21. Major categories of requirements • Behavioral - externally visible behaviors of an item (aka functional specifications, functional requirements) • Quality-of-construction - qualitative attributes of an item, such as maintainability and portability • Often not directly observable • Usually deal with how product can be handled • Programmatic - terms and conditions imposed as a part of a contract exclusive of behavioral requirements (e.g., costs, schedules, organizational structures) aka contractual • Addresses development of product • Implementation - aka implementation constraints, design constraints – restrictions placed on developers that limit design space • e.g., Use of specific software components (DII COE) • e.g., Imposition of specific algorithms • e.g., Customer-mandated architectures (e.g., Joint Technical Architecture (JTA))

22. Types of behavioral requirements(1 of 3) • Functional - input-output behavior in terms of responses to stimuli • Output = fn (input), e.g., x  x2 • Interface - characteristics of component’s interfaces • e.g., interfaces with other components (peer-to-peer) • e.g., appearance of operator screens (user) • e.g., computing infrastructure attributes / APIs (infrastructure) • Temporal - speed, latency, and throughput of functional behaviors • e.g., display refreshed screen every 0.5 sec, e.g., x  x2 in 0.5 sec • e.g., transmit filtered data within 2 sec of receiving unfiltered data • e.g., process 10,000 database requests per hour • Capacity - amount of information that can be handled • e.g., 25 simultaneous users • e.g., 20,000 employee records

23. Types of behavioral requirements (2 of 3) • Resource utilization- limitations on computer resources that can be used • Memory and processor usage • e.g., limit of 5 gb disk memory for scratch storage • e.g., limit of 20% of total processor cycles (loading) • Trustworthiness - degree of confidence in product’s delivery of functions • Reliability - MTTF = 30 hrs • Availability – 99% over 30 days • Safety – e.g., actions to avoid • Confidentiality – “the system shall not allow access to system features unless operator first enters correct ID and password.” • Integrity (partial) – “the system shall not be vulnerable to spyware”

24. Types of behavioral requirements (3 of 3) • Usability - how easy it is for an operator/user to make use of the system • For system-to-system interfaces – deals with the complexity of the interfaces, their ease of implementation, and their efficiency of operation • “Use of system resources by other systems shall be facilitated by minimum messaging and handshaking” • For human operators – deals with the complexity of the interfaces relative to the how operators can operate with them, the ease of learning, and the efficiencies with which operators can exploit the system services • “Operators shall be able to use system services with minimum number of actions.”

25. Characteristics of behavioral requirements • Often, behavioral requirements can be directly tested • Functional, interface, temporal, capacity • Apply input, observe input, compare with oracle • For some requirements, analysis of test results required • “Every 10 minutes the average of the room temperatures over the previous 10 minutes will be displayed” • For others, direct testing might not be feasible (or possible) • “System will have 0.9999999 availability (down 4 sec/year)” • “System shall run for 100 years without failure” • or be practical • “System shall be able to handle 1000 users simultaneously” • Some are qualitative – cannot be tested directly • Trustworthiness and usability

26. Quality-of-construction requirements • Qualitative attributes of an item • Deal with how the product can be handled • Not usually directly measurable or observable • We have measures that can give us insight into these qualities • to help us to infer level of quality • Based on related quantitative attributes of systems • Evaluation of these requirements tend to be based on subjective and heuristic criteria. • Examples: • Portability – ease with which component can be ported from one platform to another • Maintainability – ease with which product can be fixed when defects are discovered • Extensibility – ease with which product can be enhanced with new functionality • Integrity(shared with behavioral)– resistance to unauthorized change

27. Programmatic (contractual) requirements • Terms and conditions imposed as a part of a contract exclusive of behavioral requirements • Address development aspects of product • Examples • Costs • Schedules • Organizational structures • Key people • Locations • While these are required characteristics of development effort, they are not characteristics of the product • Such requirements may directly affect ability to achieve desired levels of dependability • e.g., not enough money or time

28. Implementation requirements (1 of 2) • Restrictions placed on developers that limit design space • Two important types: • Product design and implementation constraints– restrictions on design styles and coding • Process and development approach constraints – restrictions on processes and techniques • Examples • Use of specific software components (can also be viewed as interface requirements depending on context) • Imposition of specific algorithms • But sometimes algorithms can be used to define functionality • Required use of specific designs • Imposition of specific coding styles • Requirement to use specific fault tolerance mechanisms • Requirement to employ an object-oriented development approach

29. Implementation requirements(2 of 2) • Important note - an implementation constraint to a system may be a requirement to a SW component within that system • Paul Simon: “One man’s ceiling is another man’s floor.” • Implementation constraints are (in one sense) independent of the required system behavior • Generally, there are many different ways that a specific set of requirements could be implemented • Implementation constraints narrow this design freedom • Reduce the degrees of freedom available to the developers • Sometimes, for dependable systems, constraints are essential • Constrain design to certain patterns known for enhancing dependability (e.g., fault tolerance, exception handling) • Enforce use of certain algorithms which have been verified to provide effective solutions

30. Agenda • Introduction to tutorial content • Dependability and requirements • Nature of requirements • Developing requirements • Quality factors for requirements • Common pitfalls • Summary

31. Developing requirements • Now we now what requirements are • But where do they come from • and how do we define them? • Requirements grow out of user needs • These needs are analyzed, and approaches to satisfying them are defined • Approaches are refined into specific requirements for a system that will provide needed services • Note that a system can consist of • Software – that is our job • Hardware – we have yet to figure out how to get rid of it • People – the hardest piece to the puzzle – they are soooo unpredictable

32. System Concept Set A System Reqs Capabilities Set A System Dsn System Reqs System Concept Set B Mission Needs Capabilities Set B System Dsn System Reqs Capabilities Set C System Reqs System Concept Set C Decision Process Decision Process Decision Process Decision Process Decision points for requirements definition Each step in progression involves deciding between alternative approaches SW Reqs SW Reqs SW Reqs System Dsn Decision Process SW Reqs System design Software design

33. SW requirements analysis • Requirements allocation to SW components is not the same as defining SW requirements • Once requirements allocated to components, SW requirements analysis (SRA) needed to: • Derive SW requirements • Place into form suitable for implementation • Involves trade-off analyses Systems Engineering Skipping SRA is very risky Sys Reqs Anl Sys Dsn “Final” SRSs Final SCIs Draft SRSs SW Reqs Anl SW Dsn SW Impl SW Test Software Engineering

34. SW requirements analysis techniques • Different techniques/processes are used • ad hoc, functional, object-oriented, formal • New processes arrive every day • Agile Unified Process, Extreme Programming, Cleanroom Software Engineering • Many tools exist • DOORS, Analyst Pro, Rational Rose, … • Many notations exist with which requirements may be expressed • UML, Z, State charts, English, Latin, … • Whatever technique is used, goal is to develop a set of well-defined requirements that clearly describe what the system is to do • or the component, or the unit,… • All should produce the same result – a description of behavior of the system • Important to select technique to be appropriate to system

35. When requirements are defined • Often, requirements are not complete until product is complete • During systems definition phase • Usually at a general level • During system design phase • Detail added, capabilities become functions • During SW requirements analysis • More detail added, more specific behaviors and formats • During SW design/implementation • Refinement / clarification / more detail • e.g., GUI details often deferred until later • Changes / adaptations • Additions (e.g., evolutionary development)

36. Agenda • Introduction to tutorial content • Dependability and requirements • Nature of requirements • Developing requirements • Quality factors for requirements • Common pitfalls • Summary

37. Requirements quality factors • So, how do I know that my system requirements are good enough? • IEEE Std 830-1993 (IEEE Recommended Practice for Software Requirements Specifications) provides a set of quality factors for requirements specifications for systems • Contains nine criteria against which a requirements set can be assessed • Absence of these qualities is strongly correlated to subsequent problems in software development, i.e., ignore at your own risk • These factors are particularly crucial for developing dependable systems: • Complete • Unambiguous • Correct • Consistent • Verifiable • Modifiable • Traceable • Ranked for importance • Ranked for stability

38. Description of quality factors (page 1 of 2) • A requirements specification for a system is complete if all external behaviors are defined • A requirement is unambiguous if it has one and only one interpretation • A requirements specification is correct if every requirement stated in the specification is one that the software shall meet • A requirements specification is consistent if • It agrees with its baseline, foundation specification (e.g., system specification) • No subset of requirements within the specification conflict with each other • A requirements specification is verifiable if every requirement contained in the specification can be verified • A requirements specification is modifiable if changes can be made to the requirements without major disruption of the structure of the specification

39. Description of quality factors (page 2 of 2) • A requirements specification is traceable if • the origin of each requirement is clear, and • the structure of the specification facilitates the referencing each requirement within lower-level documentation • A requirement is ranked for importance if it is assigned a rating of its criticality to the system, based on negative impact should requirement not be implemented or fail during execution • A requirement is ranked for stability if its likelihood to change is identified, based on changing expectations or level of uncertainty in its description

40. Agenda • Introduction to tutorial content • Dependability and requirements • Nature of requirements • Developing requirements • Quality factors for requirements • Common pitfalls • Summary

41. Common pitfalls • Lack of prioritization of requirements • Over-specified / over-constrained / unbounded • Volatile and late-defined requirements • Unknown “physics” • Mixing requirements and design

42. Lack of prioritization of requirements • Not all requirements are equal • Some are more firm than others • Some are more important than others • Others may be guesses and have flexibility about final behavior • Need to provide info to developers about where • Requirements may change • Less firm requirements exist • Areas of flexibility • Areas of firmness • Recommendations • Define rigidity of requirements and ranges of acceptance • Thresholds and objectives • “Must haves” versus “wanna haves” versus “wouldn’t it be nice”

43. Over-specified / over-constrained / unbounded • Sometimes requirements are too ambitious, too restrictive, or too general • Too ambitious – results in gold-plating • Unneeded capabilities created, unattainable functions defined • Too restrictive – results in narrow, point solutions • System rapidly becomes outdated when mission changes • Too general – results in inefficient system that does everything but not well • Result is wasted resources • Recommendations (these are very general but important) • Focus on prioritization of requirements • Ensure what is needed is emphasized • Build system in a series of increments • With most critical functions completed early

44. Volatile and late-defined requirements (1 of 4) • Requirements always change • Some don’t change, but are defined late • Not necessarily bad but careful management necessary to avoid • expensive rework (and cost and schedule impact) • compromises to functionality • Crucial to associate levels of risk to levels of change • Some changes are low-risk • Other may be high risk • Related to amount of rework require • Developers better able to design defensively if they know • Which requirements are likely to change • Degree of change that could be expected

45. Volatile and late-defined requirements (2 of 4) • Depends on attributes of the requirement and its linkage to design • Some can be defined early or late • Some must be defined early • Some should be defined later • Important attributes (i.e., how to decide…) • If level of understanding of desired behavior is low (exact behaviors not well understood or unknown) – delay in definition may reduce risk • If defined and frozen early, later changes may impact design and cause rework • If high likelihood that requirement will change – delay in definition may reduce risk • Avoids rework due to late changes

46. Volatile and late-defined requirements (3 of 4) • Important attributes (cont’d) • If a requirement has high or complex external component dependencies – early resolution may reduce risk • Late changes likely to affect external systems/components • If a requirement has strong internal design dependencies – early resolution may reduce risk • Late changes may force extensive rework due to design dependencies Early definition Late definition high low Level of understanding of desired behavior Likelihood that requirement will change External component dependencies Internal design dependencies high low complex simple strong weak

47. Volatile and late-defined requirements (4 of 4) • The following are recommendations to address requirements volatility: • Define requirements with priorities and likelihood to change • Allows designers to insulate themselves from unexpected change • Ensure design accommodates expected changes • Where possible, allow run-time reconfiguration to allow changing behavior without changing requirements • e.g., screen color options • Correlate with assessment of late definitions • Assess dependencies between requirements and design • Some requirements deeply affect design globally • Others have limited design impact (GUI formats) • Ensure requirements dependencies are well understood • Define and monitor requirements stability with metrics • Track immature requirements, undefined requirements, and changing requirements

48. Unknown “physics” (1 of 2) • i.e., the outside world • Sometimes we don’t know what the behavior of the software should be • Because of our lack of knowledge of real world • Typical for embedded systems • Results in changing or late-defined requirements • Also may need to change internals of systems (algorithms, design structures) • Sometimes need to use the software system to explore its operational environment • and determine its correct behavior • and update SW based on discoveries • Frequently, discoveries are at fine grain level • Adjustments to constants

49. Unknown “physics” (2 of 2) • Sometimes, changes are to algorithms • In such a case, requirements statements must explicitly recognize situation • Design (and process) must allow for experimentation / exploration • Product is also a prototype used for requirements elicitation • Mitigation strategies include: • Use of executable models and prototypes • Use of simulations to depict external environment • Use of data logging functions to collect relevant data • Iterative development of system, and use of iterations to probe and explore the environment

50. Mixing requirements and design(1 of 2) • Because of complex interrelationships among types of requirements, it is crucial to identify them and keep them separate • Why? • Behavioral specs deal with external entities (mutual dependencies) • Other systems • Hardware • People • Changing them affects external entities • Not always possible to make changes • May involve negotiation • Implementation requirements deal with internal design • No direct external dependencies • Typical implementation constraints • Algorithms • Flow charts