1 / 28

A Scalable Approach to Architectural-Level Reliability Prediction

This presentation discusses the SHARP Framework, a scalable approach to predict reliability at the architectural level in concurrent software systems. Early design decisions significantly impact software quality, making early assessment essential. The MIDAS case study demonstrates how architectural models representing system structure can effectively analyze reliability, addressing scalability issues by generating submodels from use-case scenarios. SHARP outperforms traditional flat models in scalability and accuracy, providing insights into system reliability even amidst dynamic conditions, paving the way for future work in contention modeling.

lamis
Télécharger la présentation

A Scalable Approach to Architectural-Level Reliability Prediction

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Scalable Approach to Architectural-Level Reliability Prediction Leslie Cheung Joint work with Leana Golubchik and Nenad Medvidovic

  2. Motivation • Many design decisions are made early in the software development process • These decisions affect software quality • Need to assess software quality early • If problems are discovered later (e.g., after implementation), they may be costly to address

  3. Motivation • We focus on assessing software reliability using architectural models in this talk • Reliability: the fraction of time that the system operates correctly • Architectural models: describes system structure, behavior, and interactions

  4. Case Study: MIDAS Measure room temperature and adjust the temperature according to a user-specified threshold by turning on/off the AC Sensor: measures temperature and sends the measured data to a Gateway Gateway: aggregates and translates the data and sends it to a Hub Hub: determines whether it should turn the AC on or off AC: Control the AC GUI: View current temperature, and change thresholds

  5. Motivations • Existing approaches for concurrent systems: keeps track of the states of all components • MIDAS Example • State: (Sensor1, Sensor2, Gateway, Hub, GUI, AC)

  6. Motivations (Taking Measurements, idle, idle, idle, Processing User Request, idle) • Existing approaches for concurrent systems: keeps track of the states of all components • MIDAS Example • State: (Sensor1, Sensor2, Gateway, Hub, GUI, AC)

  7. Motivations (Failed!, idle, idle, idle, Processing User Request, idle) • Existing approaches for concurrent systems: keeps track of the states of all components • MIDAS Example • State: (Sensor1, Sensor2, Gateway, Hub, GUI, AC)

  8. Motivations (Taking Measurements, idle, idle, idle, Processing User Request, idle) • Existing approaches for concurrent systems: keeps track of the states of all components • MIDAS Example • State: (Sensor1, Sensor2, Gateway, Hub, GUI, AC) • Problem: Scalability • e.g., 2 Gateways,10 Sensors each • >5000 states • How about real-world applications, which may have 100s of Sensors and Gateways? •  The models are too big to solve

  9. The SHARP Framework • SHARP: Scalable, Hierarchical, Architectural-Level Reliability Prediction Framework • Idea: generate part of the system model at a time by leveraging use-case scenarios • Solving many smaller models is more efficient than solving one huge model

  10. MIDAS Use-Case Scenarios • MIDAS example • Sensor Measurement • GUI Request • Control AC

  11. Modeling concurrency: instances of scenarios may run simultaneously MIDAS Example Processing a GUI request while processing sensor measurements  Sensor Measurement and GUI request scenarios run simultaneously Multiple sensors  Multiple instances of the Sensor Measurement scenario The SHARP Framework

  12. The SHARP Framework • Generate and solve submodels according to the system’s use-case scenarios • Generate and solve a coarser-level model for system reliability • Describe what happens when multiple instances of scenarios are running • Make use of results from the submodels

  13. The SHARP Framework

  14. The SHARP Framework R1 m1

  15. The SHARP Framework R2 m2 R3 m3

  16. The SHARP Framework • Generate and solve submodels according to the system’s use-case scenarios • Generate and solve a coarser-level model for system reliability • Describe the number of active instances of each scenarios • Make use of results from the submodels

  17. The SHARP Framework

  18. The SHARP Framework

  19. The SHARP Framework

  20. The SHARP Framework m1 R1 m2 R2 m3 R3

  21. The SHARP Framework m1 R1 m2 R2 R m3 R3

  22. Evaluation • To show… • SHARP has better scalability than a flat model that can be derived from existing approaches, and • SHARP is accurate, using results from the flat model as “ground truth” • Experiments • Computational cost in practice • Sensitivity analysis

  23. Computational cost in practice • Example: MIDAS system, varying the number of Sensor component (x-axis) • Y-axis: number of operations needed to solve the model

  24. Sensitivity Analysis We are primarily interested in what-if analysis Is Architecture A “better” than Architecture B? but not Will my system’s reliability greater than 90%? What is the probability that I can run my system for 100 hours without any failure? Focusing on trendsis meaningful at the architectural level

  25. Sensitivity Analysis “Ground truth”: results from the flat model Vary Sensor failure rate

  26. Conclusions • Assessing software quality early is desirable • Scalability is a major challenge in reliability prediction of concurrent systems using architectural models • We tackle address this challenge by leveraging a system’s use-case scenarios in SHARP • Future Work: Contention modeling • Work thus far: assume no contention • However, concurrency  contention

  27. The End

  28. Defects • Architectural: Mismatches between architectural models • e.g., An interaction protocol mismatch between 2 comps • System: Limitations of components • e.g., Sensor has limited power • Allow system designers to evaluate how much reliability will improve if defects are addressed • Cost

More Related