Getting out of the Testing Game

1. Getting out of the Testing Game By Bill Matthews Test Architect • Manager • Technical Tester @Bill_Matthews Bill.Matthews@TargetTesting.co.uk

2. What is the Testing Game? • Testing is seen as an end in itself rather than a means to an end • Testing has become Nominalised (verb treated as a noun) • Testing is seen as a barrier that must be passed • Testing driven by process not outcomes • Testing is disconnected from its context • Those outside testing dictate test activities

3. The challenge • “Testing is taking too long” • “Testing doesn’t find all the bugs” • “It all goes fine until we hit testing…then it just falls apart” • “The PMs struggle to get good testing done” • “We use industry best practices but I’m sure they’ve changed since we last had a consultant in” • “Can you help us improve our testing process practices?”

4. The Challenge • Company was in a competitive market and so always had various offers in place. • Most offers took about 4 weeks to implement from idea to launch. • Frequently had to roll back changes because of errors. • Releases were slow, very early morning affairs • Rollbacks resulted in system outages => possible loss of revenues • Difficulties in getting a clear picture of what was happening during testing • Phased testing…for small changes

5. How the management viewed testing

6. How would you improve the situation? • Traditional approaches to test process improvement take an analytical approach. • Assumes current practices approximate some “best practice” • Measures against an idealised “best practice” • Suggests a prescription to bring current practices closer to the “best practice” • What if your processes are fundamentally broken and don’t fit your current needs? Design Thinking (Exploring What If)

7. 2 minute introduction to the Business Canvas Model

8. The Value of Testing • The Customers and Value Proposition segment is where the value of our testing activities can be realised. Who are your customers and what problem do we solve for them? • Do we solve the “testing problem” for them? • If you are in the Testing Game…yes. • And our customers also think that’s the problem we solve for them • If not the “testing problem” – what problem do we solve?

9. How do our customers want to be engaged? • The Customer Relationship and Channels segments is where we understand how our customers want to interact with us. • So how do we deliver the information our customers value? • Those in the Testing Game think it’s all about Test Plans and Metrics • But have never really explored how their customers (really) want to work with them? • If you were a customer how would you want to be engaged? • Think about the user experience/journey when they engage with you – does it feel right and appropriate?

10. How do deliver something our customers value • Key Activities, Key Partner & Key Resource Segments • This is where the activities associated with Testing sit. • What are the key activities that we need to do (and do well) to deliver value? • Those in the Testing Game have a testing centric view so know all about these segments. • But customers just get what they are given – take it or leave it. • Wouldn’t it be better to have a customer centric view and adapt the How in line with the Need?

11. What we came up with

12. The catalyst for change • What would be different if this is how we approached testing? • A flood of energised ideas • Not all of them practical but at least we were Exploring What If • What’s stopping us? • Only the Permission to change

13. Organic change • Some changes that happened: • The test team started to communicate…and it was infectious! • Collaboration - No longer a long chain of sequential processes • Abandoned the “one size fits all” processes and naturally adopted a more context driven approach • “Does that mean we don’t need to write test scripts?” • Test Plans were replaced with Kanbans • the PMs were wary of this at first but agreed to let us try and manage our own work. • Soon Kanbans were popping up all over the office (outside of the test team) • Focus was on the flow of information • PMs finally stopped telling the testers how to test…and we stopped having test phases based around testing levels. • The team seemed more energised • Overall time for small changes reduced to about 2 week • still not great but an improvement

14. So the Business Canvas Model is Magic then? • So all the change happened because of a single sheet of paper? • No really, the magic is with the people • It helped the team see a wider context and focus on what adds value • In most cases they already had ideas on how to improve their practices. • Gave them a framework to discuss what they do without really talking about testing • They were given permission to make changes • A little sad that they needed to be given permission