Software Quality & QA Management

1. Software Quality &QA Management Jatin Parekh Sr Dir of QA, Visa jparekh@visa.com

2. What is Quality • Fitness of use: Quality means the product or service does what it is expected to do • Customer expectation: Quality is exceeding the customer expectations. Quality is delighting the customer • Competition: How company’s products or services compared to competitors and compared to those offered by company in the past.

3. Benefits of good quality • Customer loyalty • Strong brand • Command premium price • Reduced cost • Cost of maintaining • Customer support • Attracting and retaining employees

4. Quality Characteristics • Ease of use • Simplicity of design • Available • Reliable • Safe - secure • Performance • Responsive • Maintainable - ability to make changes quickly

5. Software Quality Attributes • Software performs as defined by accepted requirements (external quality) • Correctness (behaves per spec) • Integrity (data integrity) • Usability (ease of use) • Efficiency (system resources) • Reliability (mean time between failure

6. Software Quality Attributes (cont) • System structure is open to change over time (internal quality – what users don’t see) • Maintainability (cost of correcting errors) • Testability (cost / ease of testing) • Flexibility (cost of product modification) • Reusability (cost of transferring a module to another application) • Efficiency (coding efficiency, system resources) • Capacity / Scalability (ability to serve at peak load) • Security (encryption / malicious injection; OWASP) C

7. Quality control vs Quality Assurance • Quality Control: The observation techniques and activities used to fulfill requirements for quality • Design / code review, Change mgmt / release process,Measurements and metrics • Quality Assurance: The planned & systematic activities implemented in a quality system so that quality requirements will fulfilled • Test strategy, test execution, defect mgmt, etc.

8. Software Testing • Unit testing • Functional testing (use cases, API) • Black box testing (end user) • Boundary testing • White / grey box testing • Testing with mock components • Internationalization (I18N) / Localization L10N • Integration / end to end testing • Usability / UI testing • Installation & upgrade • Regression testing

9. Software Testing (contd) • Rollback / recovery & Failover • Security -Open Web App Security Project (OWASP) • Reliability & robustness (e.g. error handling, protocol conformance) • Performance (bottlenecks / memory leaks / cpu usage responsiveness) • Scalability / stress / volume / capacity • Compatibility (e.g. DB changes with old code) • Migration • Documentation (Help / User’s manual

10. Software Test Category • Build verification testing (BVT) • User acceptance testing (UAT) • Regression testing • Smoke / sanity testing • Ad hoc testing • Alpha (internal users) / bug bash • Beta testing

11. Testing tools • Test case management tools • Testing tools • Commercial / open source tools • Customer tools / utilities • Automation tools (e.g. Selenium for UI automation) • Test framework • Execute scripts calling multiple tools • Ability to configure env • Ability manage test data • Schedule tests • Reporting • Manage results • Measurement tools (code coverage, static analysis tools)

12. QA Activities • Planning • Overall test strategy in test plan document • Review use cases & spec • Test case generation • Test env setup • Tools development • Automation script development • Test execution • Defect logging and verifying • Test results review • Release checklist

13. Software Industry / Technology Trends • Shift from Installed software to software delivered as a service (SaaS) • Vendor can upgrade software frequently • Shift from physical infrastructure to cloud infrastructure • Improved scalability • Desktop/PC computing to mobile / wearable devices • Need for software to provide consistent look/feel • Security • Data in transit / at rest • Govt regulations

14. Water Fall vs Agile Method • Water fall • Final product is well defined • Design, development, & testing is done in sequence • Clients won’t have the ability to make change • Agile • Allows for making changes after planning • Ability to react to competition • Development & testing is done in parallel and completed in each sprint, typically 2 to 4 weeks. • Product can be launched at the end of every sprint • Ideal for SAS / web products.

15. Agile Methodology • Scrum is software development process • Daily standup of a small self contained team • 4-6 dev / 2-3 QA / Product owner / Scrum Master • Sprint retrospective and planning Core Agile Process

16. Agile Methodology & QA • Frequent releases (continuous delivery) • Upgrade features piecemeal • Different behavior for diff users (A/B testing) • Reliance on communication rather than documentation • Development and testing hand in hand • Think about testing before code development • Continuous integration testing • Regression testing in <2 days • ~ 100% Automation is Key to Success

17. Managing Quality • Manage Project Quality: Quality Control and Quality Assurance • Full time job for large, complex systems • Establish organization’s quality procedures & standards • Select appropriate procedures and standards based on project scope • Ensure that standards are followed

18. Managing Quality (contd) • Project planning • QA project plan encompasses test strategy plan, resource plan, execution plan, checklists • Project tracking, Status, Reports • Tools and methods • Build vs buy • Cost of quality • Tradeoff in the midst of time / resource constraints • What to test, when to test, and how much to test

19. Managing Quality (contd) • Metrics & Measurements • Test execution monitoring • Defect trends • Code coverage • Performance measurements • Defect management • Defect Triage • When to push for fix and when to push back

20. Role of QA Lead • Manage relationship with stakeholders • Product managers, Project managers, Development, IT, Ops, Customer Care, Clients, other QA teams • Negotiations (timeline, resource, expectations) • Establish win-win situation • Organization of the team • Project based • Matrix organization • Geographical based • Anticipate & manage change • Metrics driven approach for decision making

21. Role of QA Lead (contd) • Key responsibilities • Owns the quality of a release • Recruiting • Goals setting • Coaching, motivating, & conflict resolution • Challenge high performers / manage low performers • Communication • Manage remote employees • Feedback & Reviews • Honesty & fairness

22. Recap • Roles & responsibilities of QA in Org • Agile / rapid deployment cycle (1 to 4 wks) • Automation & Tools • Software development / programming experience • Offshore and onshore delivery • Startup vs Established companies • Documentation, schedule, level of quality • Perception of QA in Org

23. Career in QA • Key traits: • Software development & programming knowledge • Ability to troubleshoot • Wanting to know how things work end to end • Curiosity: ask questions & be proactive • Communication • Wanting to break things • Managerial vs Technical track

24. Q & A