Architecture Analysis to Predict Project Risks Using Containers to aid Risk Assessment

1. Architecture Analysis to Predict Project RisksUsing Containers to aid Risk Assessment CRC PhD Student Conference 2019 Part Time: Andrew Leigh Start Date: October 2015 Supervisors: Dr. M Wermelinger, Prof. A Zisman

2. Introduction • Cheaper to fix problems earlier in SDLC • Avoid big risks using MCDA, AHP, RCDA etc • But still, design flaws can pose implementation risks Architecture and Design Review UML Risk Container Analysis Code Source Code Review • Design risks need to be managed to help projects be successful in terms of quality, cost and schedule Test • https://www.callforparticipants.com/study/8GYWX/analysis-of-uml-software-architectures

3. Developed the Design Rule Hierarchy (DRH) algorithm to separate modules of related elements in UML designs to maximise developer parallelism Related Work Wong et al., 2009 Xiao et al., 2014 Mo et al., 2015 Leigh et al., 2016 Leigh et al., 2017 • Used DRH to extract DRSpaces from source code to find architectural roots of error-proneness • Used DRSpaces to locate and classify common causes of error-proneness in a range of projects • Applied DRSpaces to UML designs to isolate areas of implementation error-proneness (relative) • Compared effectiveness to Resource and Use Case Containers • https://www.callforparticipants.com/study/8GYWX/analysis-of-uml-software-architectures

4. Risk Containers Design Rule Containers Use Case Containers Resource Containers Based on design rules (key interfaces) that split an architecture into independent modules Populated with classes dependent upon an external resource (database table, service, file, etc.) Populated with classes that fulfil a use case • https://www.callforparticipants.com/study/8GYWX/analysis-of-uml-software-architectures

5. Overall Method • Determine how element isolating container types are: • Calculate distribution of architecture elements between containers • Calculate percentage of internal coupling • Determine how risk predicting container types are: • R1: Correlate container level design coupling to implementation error-proneness • R2: Correlate container change propagation probability to observed co-change Element Isolating ∧ Risk Predicting ⟶ Risk Isolating • Determine how meaningful container types are to practitioners: • Test to see if presenting designs as containers makes it easier to find errors and change impacts Risk Isolating ∧ Meaningful ⟶ Useful for risk management

6. Classes tend to share fewer DRCs than RCs and UCCs Element Isolating • % of internal ”isolated” coupling much higher in DRCs than RCs and UCCs • Not replicated in controls (random assignment to containers)

7. Co-Change Predicting Correlation between internal container change propagation probability calculated from the design and code co-change is strong an significant for all container types

8. Probability of Co-change Only DRCs contain classes with a greater probability of co-change than classes that do not share containers for all four software projects studied

9. Developer’s View (API) Top 5 match = 4 true positives, 1 false positive and 1 false negative Precision = 0.8 Recall = 0.8

10. % of cyclic dependencies identified correctly Participant Experience Meaningful & Useful? Time taken per review question (minutes) % of change impacts identified correctly • Container groups take 30% longer. • DR Containers take 15% longer. • More single diagram cyclic dependencies identified correctly in container groups. • Hard to separate container type performance with current participation levels. • DR Containers slightly better for multi diagram cyclic dependencies than control. • Not much difference between containers and control for change impacts. • https://www.callforparticipants.com/study/8GYWX/analysis-of-uml-software-architectures

11. Key observations to date: • DRCs most element isolating and isolate highest % of internal coupling • CPP metrics calculated for design containers correlate to code co-change • Classes more likely to change with other classes they share DRCs with • Agreement between dev.’s view of isolated co-change areas and DRCs • Participants detect more cyclic dependencies isolated within smaller container diagrams than control. Not much difference for change impacts • Validity: • Other container types • 4 software systems studied • Emergence and popularity of agile methods (no design) • Meaningful study: • Relatively small toy architecture • Only one error inducing flaw – cyclic dependency • Only 13 participants to date Summary • https://www.callforparticipants.com/study/8GYWX/analysis-of-uml-software-architectures

12. Questions?