Session 2 - 12:50 to 1:45 SRD artifacts (4) Functional domain requirements ( 7 )

1. Session 2 - 12:50 to 1:45 SRD artifacts (4) Functional domain requirements (7) Quality requirements (21)

2. Artifacts of requirements development Include: • Unambiguous and accurate goal statement(s) and, if necessary (based on complexity and experience): • glossary of precise definitions for domain concepts, verifiable quality measures, and terminology in the quality achievement and verification strategies • Sufficient set of unambiguous capabilities and qualitiesto achieve the goal(s) • Sufficient set of unambiguous and effective strategies and tacticsto achieve the capabilities and qualities

3. Goals and capabilities The software must control the driving behavior of an autonomous vehicle within a specified ODD (operational design domain) at SAE level 5 (human driving eliminated)by reacting to sensor input and behavior rules. • This entails: • Communicating with passengers about initial destination and route restrictions and changes • Determining current location • Developing and adjusting a safe driving plan • Communicating with other roadway users • Entering a roadway from a parking place • Exiting a roadway to a safe parking place • Normal stopping and starting • Executing a safe stopping command • Lane keeping and changing • Vehicle following • Achieving, keeping, and adjusting safe speeds • Navigating intersections, turns, roundabouts, parking lots, and accesses • Following human traffic control guidance • Reporting vehicle or passenger emergencies

4. More capabilities • As well as continuously: • Executing a driving plan • Identifying, predicting the behavior, and responding to proximate objects • Scanning for hazards e.g., driver door opening • Obeying traffic signs, signals, advisories, and laws • Monitoring secure broadcasts of weather, traffic, and roadway conditions • Monitoring cabin temperature and passenger restraints • Detecting and responding to weather, traffic, roadway, vehicle, and passenger conditions • Displaying defensive and safe driving behavior • Displaying natural driving behavior • Validating sensor input • Monitoring software and system behavior • Detecting and mitigating software and system failures • Logging to a black box

5. This may expose misunderstandings or inadequate understanding. It may also improve the effectiveness of verification tactics. Requirement specifications and glossary result from a risk management tactic (define, then design) forbringing precise restrictions to light (and enabling their confirmation)

6. Session 2 - 12:45 to 1:45 SRD artifacts (4) Functional domain requirements (7) Quality requirements (21)

7. Some autonomous vehicle hazards What about hazard combinations?

8. What is risk mitigation?

9. Developing mitigation method requirements If <hazard condition(s)>, then <mitigating action(s)> If the motor dies on a roadway and the vehicle is moving and a “safe and traversable glide path” is accessible, then turn on hazard lights, call for help, signal appropriately, steer onto the glide path, decrease speed as needed, and brake when “stopping is safe”.

10. Who decides what should be done when a motor dies? • Will “dead motor” be detected? • What should happen when motor dies and the vehicle is: • Stopped in a • safe place? • dangerous place e.g., railroad crossing? • Moving, but • can “stop safely”? • can’t “stop safely”?

11. How should this hazard be mitigated?

12. Monitoring Behavior Monitor the behavior of complex and/or safety-critical software bydetectingdangerous and erratic behavior and by state logging This mitigates some risk, because software can fail inmanyimagined and unimaged ways

13. 'My gas pedal is stuck' BMW driver tells 911 as he speeds down a Florida interstate at nearly 100 mph Feb 14, 2018 BMW spokesperson called the scenario "implausible“ "All BMW vehicles, .., employ an electronic accelerator pedal which uses software logic to override the accelerator whenever the brake pedal is pressed while driving. This fail-safe software means that if the vehicle detects that both pedals are depressed, the on-board electronics will reduce engine power so that the driver may stop safely.” Unintended Acceleration Again Disrespecting complexityis dangerously naive No requirement for global monitoring of vehicle behavior

14. Session 2 - 12:45 to 1:45 • SRD artifacts (4) • Functional domain requirements (7) • Quality requirements (21)

15. Quality requirements A quality goal (requirement) is a specified achievement level of a quality attribute e.g., security Since quality attributes can conflict,e.g., security and usability, tradeoffs may be necessary Quality models are embedded in requirements models Quality goals (requirements) should be derived from useful quality models

16. What makes a quality model useful? Significant support for4quality requirement development tasks: • Selectrelevant qualities. • Identify their supporting qualities and potential conflicts. • Specifygoals for each selected quality along with levels and priorities. • For the selected architecture, designan achievement, monitoring, and verification strategy for each quality and estimate their effort.

17. BABOK quality model • Availability • Compatibility • Functionality • Maintainability • Performance Efficiency • Portability • Reliability • Scalability • Security • Usability • Certification • Compliance • Localization • Service Level Agreements • Extensibility (Almost) alphabetical list of 15 qualities

18. ISO 25010 2-Dquality model • Internal qualities (which support all others) are tucked under Maintainability • understandability – missing • reviewability – missing • verifiability - missing

19. Quality models for certification IIBA – CBAP based on list 9000 CBAP certified QAI– CSQA based on list 22000 to 25000 CSQA certified world-wide IREB – CPRE based on list, but references ISO model 56700 in 84 countries have passed the CPRE Foundation Level ASQ – CSQE based on list, but references ISO model 1500 CSQE certified world-wide 90,000 professionals certified based on simplistic quality models

20. SEI 2-D quality model PART TWO QUALITY ATTRIBUTES CHAPTER 4Understanding Quality Attributes (19) CHAPTER 5Availability (25) CHAPTER 6Interoperability (15) CHAPTER 7Modifiability (15) CHAPTER 8Performance (17) CHAPTER 9Security (13) CHAPTER 10Testability (17) CHAPTER 11Usability (11) CHAPTER 12Other Quality Attributes (19)

21. Wiegers & Beatty 3-D quality model

22. LiteRM 3-D quality model a internal qualities directly visible only to developers external qualities fully visible to users mixed qualities partially visible to users [think icebergs] a a

23. Basic internal qualities support all others Runtime self-checking Mainly verified by reviews, analysis, and measurement 3 pillars of internal quality

24. Some external & mixed qualities Primarily verified during system test Needs full range of verification tactics

25. Support relationships between types Basic internalssupportall other qualities including themselves Some externalssupport some mixed

26. Common characteristics and values [1 of 4] • DefinitionSafety: A system’s ability to do little or no harm to valuable assets • Software subfield Safety engineering • Assumptions/Rationale • Safety is a fragile quality because it depends on many other qualities and the accurate identification of safety hazards. • Leading indicators– provide preoperational evidence of quality goal achievement • Ratio of hazards added during HA technical review to hazard count after HA technical review • Ratio of safeguard defects found during a technical review to number of safeguards • Ratio of safeguard defects found during testing to number of safeguards • Operational measures • Number of “dangerous” failures or defects detected per time interval • Greatest harm from a harmful event • Ratio of actual loss to acceptable loss in a duration

27. Common characteristics and values [2 of 4] • Supported bydependability, ease of learning • Conflicts withefficiency, interoperability, adaptability qualities • Threats [identify using hazard analysis] • Mitigations [identify after identifying hazards] • Other achievement tactics • identify safety-critical users • identify valuable assets and hazards • identify safety-critical and safety-related functions and constraints • isolate and protect safety-critical functions • guard safety-critical functions with explicit conditions • alert users to dangerous actions with rotating warning messages • precede each dangerous action with a delay so users can change their minds and cancel • design interfaces that prevent and detect user errors

28. Common characteristics and values [3 of 4] • Verification tactics • verify all supporting qualities • conduct a safety audit to review hazards and mitigations for completeness and effectiveness • thoroughly test each safeguard • track time since last “dangerous” failure or defect • track number of “dangerous” failures or defects • monitor for misbehavior e.g., unanticipated acceleration • monitor system state to make sure safety-critical and safety-related functions are active • Elicitation questions • Associated tools • Resources[pointers] • Risk factors • a. Developer understanding = [superficial, limited, deep] • b. Cost of implementation, verification, & maintenance = [high, medium, low] • c. Feasibility (technical, cost, understanding) = [low, medium, high]

29. Resourcesfor qualities with large “bodies of knowledge”

30. Common characteristics and values [4 of 4] Other features a. Sources/Enterprise goals: b. Type = mixed behavior quality c. Associated scope = [system, <specific partitions>] d. Design scope = het cc [het cc, hom cc, universal, local] e. Consensus priority = [critical, important, desirable] f. Architecture-relevant = yes [yes, maybe, no] Past quality goal specs Past achievement, verification, and monitoring strategies Current quality goal spec Current achievement, verification, and monitoring strategies

31. Features of the LiteRM quality model • 3-dimensional • dynamic • tailorable • improvable • embedded quality goals & strategies • essentially complex • mildly defective • incomplete,but provides a rich template • and many examples of characteristics • supporting the quality achievement, • monitoring, & verification tasks.

32. Fun is big business Video game industry earns over $100B USD per year A target market ? $90B USD lifetime 65% of American adults play video games

33. Example of Planguage Quality Measure

34. Comparing usefulness of quality models

35. Resources http://www.quality-aware.com/software-quality-KB.php 1. Downloadable LiteRM quality model (pdf) with > 60 qualities > 30 characteristics per quality 2. Downloadable chapter on quality goals (pdf)