The quality-Centric software development process

1. The quality-Centric software development process Michael Burnside Twitter: @SQAEvangelist Blog: http://sqaevangelist.blogspot.com/ Website: http://sqaevangelist.com/ Software Quality Assurance, Quality Engineering, and Web and Mobile Test Automation Expert 31 May 2013

2. The Quality-Centric Software Development Process Table of Contents: • Goals • Definition • Statement • About Me/My Qualifications – Why should I take your advice or listen to this presentation? • Myths of Traditionally Practiced Software Development • Overview of traditional Development-Centric tasks and responsibilities for team members. • Advantages over using Development-Centric Approach • The Process, Players, and Activities • References

3. The Quality-Centric Software Development Process Goals: • Develop the highest quality of software with the minimum amount of effort and expense to the company. • Create the best and most useful User Experience to the customer (some would refer to as the “end-user”).

4. The Quality-Centric Software Development Process Definition: The Quality-Centric software development process focuses on the execution of a team of very highly technically capable, narrowly-focused, and dedicated software related professionals with the goal of building and delivering the very highest quality of product to the customer and document software and processes (product requirements, software development, and testing) such that it reduces risk to the company in the event that team members leave the team so that further progress and support of the product continues with the least possible risk to the company and ultimately, to the customer and User Experience (UX).

5. The Quality-Centric Software Development Process Statement: You cannot test quality into software. What does this mean to you?

6. The Quality-Centric Software Development Process About Me: Before I start, let me introduce myself and thank you for allowing me the opportunity to present to you my ideas. Your open-mindedness and ability to be a listener, and being a proponent for change is appreciated and encouraged. I have worked for almost 20 years as a software developer (and later software architect) in Basic, assembly (Motorola 68010), Pascal, C, C++, Perl, and later more focused on Java, J2EE (server-side and client-side using Swing) and then, after much career reflection and disappointment with the extremely low focus on developer unit testing and many software organization’s low respect for QA activities; I decided to transition to the Quality Engineering role to organizations in the software development process with goal of raising the bar of quality in all software products I was part of testing and becoming what I feel is a complete “software engineer” and VP Engineering at some point in the future using the best practices that I have learned.

7. The Quality-Centric Software Development Process Software Myths (Part 1): • The QA team can ensure product quality. Reality: The QA team is really a testing team whose main role is to help determine product functional requirements, steps to test, and acceptance criteria, and then execute testing and report those results to the engineering and product management teams. In the Development-Centric process, product quality and release times are usually determined by engineering managers and, sometimes, executive management on a rushed (and usually unrealistic) schedule. The usual result is that a decision is made to release on the unrealistic schedule date, a product of very low quality, and compromise the User Experience and then to release the product anyways, regardless of the opinion of QA team (including QA managers and testing team members that recommend that the product not be released). There are a group of individuals that test the product and deliver accurate results of the tests to the other stakeholders. The people who test the product are a part of a larger team that determines what is acceptable product functionality and user experience to release for customer use.

8. The Quality-Centric Software Development Process Software Myths (Part 2): 2) The Agile process makes developing software easier and using it results in higher-quality software and better User Experience. Reality:Agile is really about improving work team communication and identifying risk and changing direction quickly because of business needs. It is a process that is a very good (and better) process when applied for developing software than using earlier methods (waterfall, etc.) but Software Testing, and the activity milestones in regards to the testing activities, are not directly addressed. In fact, testing activities and goals are actually more difficult to accomplish within the compressed timeframes of a sprint. This is especially true when it comes to the actual time to test new features and do regression during the sprints. Testing activities are more difficult using the Agile process.

9. The Quality-Centric Software Development Process Software Myths (Part 3): 3) It’s mainly the fault of the Quality Assurance team when a showstopper/critical bug is found in the production environment. Reality: The problem is very rarely due to the incompetence of a person on the QA team or the QA team and QA processes.It has much more to do with the dysfunction of the team itself, or, in most cases, the lack of communication, and in some cases: schedule pressure; identifying test cases, or unawareness of testing environment differences between local, QA, staging and production environments. Schedule pressure within short Agile sprints reduce the time to identify any potential problems.

10. The Quality-Centric Software Development Process Software Myths (Part 4): 4) Product Development teams need to have much more development resources than QA resources. Typical VP of Engineering will recommend 5:1 Dev:QA ratio . Reality: The ratio of onsite (not counting remote from India, China, etc.) of Dev:QA resources need to be at least 1:1 and even better 1:1.5 or 1:2.

11. The Quality-Centric Software Development Process Software Myths (Part 5): 5) If conducting main business and software development operations in the USA, remote QA/Testing resources and teams from India, China, Russia, etc. are as effective and from a resources dev:QA standpoint count the same as a local/onsite resources from a dev:QA ratio standpoint. Reality:Remote QA resources from half-way around the world as the US (or wherever the main team is located) are not nearly as effective as on-site and local QA in the USA. Technical ability and skill of remote and local resources are, in most cases, the same. The main issue is delay of communication. In general, remote QA teams can test the product but any problems found in the product have a communication delay of 8-12 hours. On-site resources that can communicate in real-time with the development team without time delays are critical in effect of testing of the software development process.

12. The Quality-Centric Software Development Process Software Myths (Part 6): 6) The main goal of automated testing is to find bugs. Reality:The first goal of automated testing is to assist the manual QA person in testing more of the product, ie; code coverage without the physical use of their time. The secondary goal is to find bugs that may be applicable to user experience. Example: for mobile applications, this usually consists of finding memory management and crash problems. Test Automation effort is a “marathon” and not a “100 yard dash”. Have a long term plan with the automated tests. Do it right. Don’t rush it. Remember that software is being written for the singular purpose of not using human effort and time to effectively test the product and that it must work correctly. Rushing the completion of automated test code results in the same mistakes as the customer-facing product. People who write test automation code have one customer: The manual Testing person. The manual testing person is the end-user of automated test code.

13. The Quality-Centric Software Development Process Software Myths (Part 7): 7) For mobile applications, Testing teams can switch easily from testing applications on different platforms, ie; iPhone, iPad, Android tablets and phones, Kindle. Reality: This is an extremely inefficient and unproductive use of testing teams. Allow the testing person to focus on testing a product on one platform (iOS, android, etc) and become an expert. In general, your test person(s) should use the product much more than the end user (customer) and become familiar with every aspect of the OS and the app running on that OS. Each test team members focuses on intense testing of the product on one product and platform only.

14. The Quality-Centric Software Development Process Software Myths (Part 8): 8) Quality can be Tested into Software. Reality: You cannot test quality into software. Quality Assurance team cannot guarantee quality. The reason is that quality is an opinion that varies. If you want good quality, it has to be very clearly defined; ie; what features and user experience will make this product to be determined that has good quality. Acceptance Criteria for defined test cases required to be tested are the barometer for quality. Acceptable product quality must be pre-determined before software development and testing activities occur.

15. The Quality-Centric Software Development Process Overview of traditional Development-Centric tasks and responsibilities for team members • The VP/Director of Engineering only have software development backgrounds; they have not worked in a QA/testing role. Thus, they have no real understanding of QA/QE related issues or day-to-day challenges. • VP Engineering/Product Management determines product release dates without determining crucial product requirements or necessary implemented test cases or test results. • QA resources and management are overruled when vehemently rejecting product release to the customer due to concerns of user experience or number of critical bugs found during testing. • Leaders of software development teams have only a development background (never worked as a QA person/role) and Product Managers compromise acceptance criteria in order to release software based on non-attainable product release schedules. • Quality focus is lip-service only; meeting release dates, and compromising acceptance criteria or input from QA are the only barometer of product success.

16. The Quality-Centric Software Development Process Advantages over using Development-Centric Approach (Part 1) • Software releases can be done after each Sprint in the Agile process. Therefore, there are no pre-determined and unrealistic and unattainable product release dates from Product Management or Engineering teams. • Software development team does not write any automated test code (even low-level/white box testing code). They focus on what they do best (using latest technologies, best practices, and design patterns to develop elegant, high-quality software). • The QA team becomes a Test organization and guarantees only correct testing of the software and deliver results to stakeholders. The Test team does not ensure quality. The testing organization only ensures the product was tested and results are accurate based on that testing. • Acceptance Criteria of test cases determines, which are created collaboratively with Product Management, Development, and Testing team, determine acceptable product quality. There is no gray area or opinion. If the software is tested and passes all identified required tests, then the product may be a candidate for release to the customer. • Product Management and Testing organization determine if a product is released. The Software development team has no input or influence on product releases or dates to release.

17. The Quality-Centric Software Development Process Advantages over using Development-Centric Approach (Part 2) • The VP of Engineering must have experience as a QA or Test person at some point in their career. • Knowledge of software, release history, testing processes, (ie; everything possible) goes into the wiki or inline documented code (javadoc, etc) such that it reduces the risk of depending on individuals when they leave the company or organization. When employees leave companies, they take enormous valuable, and critical information with them. Use wiki and whatever other resources to reduce product maintainability risks. The goal is easiest possible product maintainability. • If possible, testing activities should be done by an outside company, a disinterested 3rd party – a separate company that has no personal connections to development, product management or executive teams. If not, then the testing organization should have a much closer relationship with Product Management than the Development team. • Test Cases, Steps to Test, and Acceptance Criteria determine acceptable product quality for release to the end user (customer). There are no gray areas.

18. The Quality-Centric Software Development Process The Process, Players, and Activities (Part 1) • The process uses Agile. It is the best process to develop software and it is recommended to use 2 week sprints. 3 week sprints can be used but you need to be aware of how 1 working week could impact the product releases in regards to competitors or business climate changes. • The Players are: VP Engineering who has experience in software development and QA/Testing; Product Manager(s); Testing team; Software Development team; User Experience team to represent the user. • It is recommended to have a disinterested 3rd party (not employees of the company) to test the product and deliver testing results. • The software and product management team focus on test cases, steps to test, acceptance criteria, and thoroughly and completely document these requirements and prioritize for each sprint. • The software development team does not write any test code and only does minor unit testing and local tests (what some would refer to as “happy-path”). • The Testing team tests the product manually (a human being/person) and with automated code at every level (black box/white box, API level, mock testing, performance and memory management, crash, etc) and for every purpose (smoke, sanity, performance, exception cases, etc., - all tests). Manual testers use the automated code to expand their product testing coverage. • The manual Test team is responsible for executing and verifying that the automated testing team’s code is working and tests the product correctly. • The automated Testing team’s customer is the manual Testing team. The manual Testing team uses the automated code to expand their testing of the product. • The manual test person is the most important resource. They not only validate the customer-facing product, but also the automated testing code is working and testing the product correctly. This creates a very close relationship with the manual/automated Test resource.

19. The Quality-Centric Software Development Process The Process, Players, and Activities (Part 2) • The testing team reviews with the Product Management team the results of tests and the product is not released unless testing team validates all required test cases have passed – through manual or automated testing. • The focus is on User Experience of the product. The goal is 100% happiness from all users of the product. • Ideally, for every single developer, there should be one manual tester, and one automated tester that interact real-time (not remote or 8-hour delays). That shifts the focus on meeting Acceptance Criteria and testing as many test cases possible. • Although test automation is crucial, manual testing is very important also. The human element in testing must be there to have all confidence of how a person will use the product. • Initially, Test automation focuses on code coverage, not finding bugs. Eventually, test automation can cover exception cases and, in mobile applications, client memory management problems. Code coverage at every level is the main goal. White box, black box testing for code coverage is the focus. Plan for long-term testing of the product and how the maintenance costs could affect the finances of the company. • Test automation is a marathon, and not a 100-yard dash. Test automation is still software, but for testing purposes, not customer facing product purposes. DO NOT put deadlines on test automation code. It takes time to build good quality testing code also.

20. The Quality-Centric Software Development Process The Process, Players, and Activities (Part 3) • A group called Product Council meets once per sprint to identify new stories, prioritize them, and assign implementation of them into Agile sprints. The team consists of Product Management, and management from the software development team and Test team. If there are Testing tasks needed for a story implementation, these tasks get discussed and effort number assigned to them the same as a development task would have. • As mentioned before, Agile Sprints are 2 weeks long (10 working days) and product releases happen after each Sprint is complete. Get the improved product into the customer’s hands asap. • If a story has QA tasks, the 3 tasks are: Create Test Cases, Approve Test Cases, Execute Test Cases. • The timeframe for development and testing activities during the 10 day Sprint can be found here: Here Achieve and complete the milestones on the timelines for each task extremely close. A ½ or 1 day delay can severely affect the testing activities and jeopardize the story being completed for the sprint. • A person involved in story implementations (dev, test) can work on multiple stories during a sprint, but no more than 3, and, the effort point total cannot be greater than the maximum for all stories. In fact, it is recommended to reduce multi-tasking to the minimum possible. The goal is to allow focus on the story implementation and test to the maximum extent possible. • Shared responsibility for success and failure. If bugs are found in production environment, it is a failure of the team, not the Test resource. • Before a product is approved for release at the end of the sprint, the whole team (product management, developer, test teams and management) come together for ½ day and all test and use the product for a minimum of 4 hours and have lunch. This is called the “wrecking crew”. The company buys lunch for the team. Everyone hangs out and relaxes, have fun using the product and trying to find bugs and develop an opinion of user experience. All employees must be on-site for this activity. Benefits: all stakeholders get experience and familiarity with all parts of the product, maybe find bugs, form an opinion on user experience (UX). All team members must be involved in this activity WITH NO EXCEPTIONS. • Works hours for each team member should not exceed 45 hours per week. • Working during the weekend (Saturday and Sunday) is not allowed.

21. The Quality-Centric Software Development Process Thanks! Special Thanks to Matt Au, who is an amazing software engineer and a smart person that I have known and respected for 10 years, for reviewing several versions of this document and providing great feedback and recommendations for improvement which I have enthusiastically implemented in the presentation that you have just read. I sincerely hope this presentation was useful to you and that your company or organization implement the information and process in this presentation to the fullest extent. At the very least, I hope it inspires discussion and action for software and user experience improvement. If you have any feedback, please feel free to get in touch with me (via Twitter) at @SQAEvangelist. I would love to hear very constructive comments on how to improve this presentation and make it more effective.