8 . Fault Tolerance in Software

1. 8. Fault Tolerance in Software 8.5 Construction of Acceptance Tests Goal: describe the types and selection criteria for acceptance tests Two levels where acceptance tests can be performed: The higher-level: tests if the outputs of the program are consistent with the functional requirements (functional acceptance test). The lower-level: tests sections of the code to ensure that key variable and functions have been properly executed. High-level tests are always required!

2. 8. Fault Tolerance in Software 8.5 Construction of Acceptance Tests Acceptance tests may be devised such that they test for what a program should do or for what a program should not do: e.g., acceptance tests in a SW controlling an automated rail transit system can be written to test for conformance to specified velocities at each location on the route or for violation of safety rules (not exceed a certain velocity around curves or in the vicinity of a station).

3. 8. Fault Tolerance in Software 8.5 Construction of Acceptance Tests Acceptance tests fall into four categories , which serve as a useful starting point for the designer confronting a section of critical code: • Satisfaction Requirements • Accounting Tests • Reasonableness Tests • Computer Run-Time Tests

4. 8. Fault Tolerance in Software 8.5 Construction of Acceptance Tests Satisfaction Requirements An example: one the the simplest forms is the use in theinversion of mathematical operations.

5. 8. Fault Tolerance in Software 8.5 Construction of Acceptance Tests Satisfaction Requirements Devising satisfaction of requirement tests may be a difficult and subtle problem: Another example: at the completion of a sortprogram, the acceptance test checks if the elements are in uniformlydescending order and if the # of elements in the sorted set is equal to the # of elements in the original set. This test is not complete. Changes in an element during execution would not be detected. An additional test ensuring that every element in the sorted set appeared in the unsorted set should be rejected because of run-time requirements.

6. 8. Fault Tolerance in Software 8.5 Construction of Acceptance Tests Satisfaction Requirements Text editing systems, compilers, and similar programs, tests for satisfaction of requirements constitute the most promising approach.

7. 8. Fault Tolerance in Software 8.5 Construction of Acceptance Tests Accounting Checks Suitable for transaction-oriented applications with simple mathematical operations, such as airline reservation systems, library records, and bank transactions. The simplest form is the Checksum, where all records of a particular program can be compared between source and destination.

8. 8. Fault Tolerance in Software 8.5 Construction of Acceptance Tests Reasonableness Tests Detect SW failures by use of precomputed ranges, expected sequences of program states, or other relationships that are expected to prevail. The difference between Satisfaction of Requirements Tests and Reasonableness Tests is that the latter are based on physical constraints while the former type are based on logical or mathematical relationships.

9. 8. Fault Tolerance in Software 8.5 Construction of Acceptance Tests Reasonableness Tests E.g. (1), determination of the true airspeed (a computed quantity) from the indicated airspeed (a sensed quantity) in a flight control system. The first level of acceptance test is based on a precomputed range is that the speed must be within the structural capabilities of the airframe (140 to 1,100 km/h) for a commercial subsonic aircraft. Thus, if the true airspeed is outside this range, there is something wrong with either the sensor or the computer (an additional possiblity is that tha aircraft is out of control).

10. 8. Fault Tolerance in Software 8.5 Construction of Acceptance Tests Reasonableness Tests Further refinements on this test would use a reasonable range of changes to true airspeed: if changes between the current airspeed and the previous value indicate accelerations beyond the aircraft limit, then an abnormal condition exists: E.g., if the previous true airspeed is 1,000 km/h and the subsequent calculation (next tenth of a second) results in an airspeed of 1,020 km/h, the accetance test will detect an error condition because the implied acceleration is almost 6g!

11. 8. Fault Tolerance in Software 8.5 Construction of Acceptance Tests Reasonableness Tests A second example of Reasonableness Test is based on progression between subsequent states: e.g., in a telephone switching system, it is not reasonable to proceed from a connected state to a ringing state or line-busy state. However, such a test is not exhaustive, it would not detect the premature termination of a connection.

12. 8. Fault Tolerance in Software 8.5 Construction of Acceptance Tests Run-Time Checks Consist if those provided by most current computers as continuous execution sequences which are often HW implemented and/or embedded in special SW or OS. E.g., tests that detect anomalous states such as divide-by-zero, overflow, underflow, undefined operation code, unauthorized entries to subroutines, and write-protection violations. Upon detection of a run-time condition error, a status register is set and transfer to an alternate routine can be effected.

13. 8. Fault Tolerance in Software 8.5 Construction of Acceptance Tests • 8.5.1 Placement of Acceptance Tests within the Program • The placement of Acceptance Tests is based on the development of “fault trees” below the functional requirements for the application program. • The development of fault trees involves a description of the structural aspects of subroutines, procedures, or data structures contained within the program.

14. 8. Fault Tolerance in Software 8.5 Construction of Acceptance Tests • 8.5.1 Placement of Acceptance Tests within the Program • Development of fault trees for the placement of acceptance tests.

15. 8. Fault Tolerance in Software 8.5 Construction of Acceptance Tests • 8.5.1 Placement of Acceptance Tests within the Program • To illustrate the approach, take the fault-tolerant multiprocessor computer (FTMP) developed by the C. S. Draper Labs.: consists of 10 processors configured as 3 triads and a single spare. • OR gates mean that all potential failures must be tested by the Acceptance Test. • AND gates mean that only one event must be tested. E.g., “Stuck-in Applications Routine” failure can occur only when two conditions prevail simultaneously: (1) Interval time is not set, and (2) a design flaw in the applications routine causes a failure to terminate.

16. 8. Fault Tolerance in Software 8.5 Construction of Acceptance Tests • 8.5.1 Placement of Acceptance Tests within the Program • “Clock Failure” is considered a HW failure. Thus, it is not checked by Acceptance Tests. • Failures indicated by “—” are out of the scope of SW reliability.

17. 8. Fault Tolerance in Software 8.5 Construction of Acceptance Tests • 8.5.1 Placement of Acceptance Tests within the Program • Top-level fault tree for the FTMP dispacher.

18. 8. Fault Tolerance in Software 8.5 Construction of Acceptance Tests • 8.5.1 Placement of Acceptance Tests within the Program • Timing failures.

19. 8. Fault Tolerance in Software 8.5 Construction of Acceptance Tests • 8.5.1 Placement of Acceptance Tests within the Program • As conclusion: It is evident from this example that considerable insight into the operation of the SW system is required for drawing of fault trees. So, special attention must be driven to the system under development in terms of specification, coding, and documentation.