Paul Gerrard

First published 23/12/2013

*** See our 'Test Strategy in a Day' workshop ***

Test Strategy is a Thought Process, Not a Document

Test strategy is one of those nebulous terms for an activity that everyone has to undertake for their development projects. The problem with test strategy is that there isn't a single agreed definition of what it is. Some people treat Test Strategy as a document: "A high-level description of the test levels to be performed and the testing within those levels for an organization or programme". So - a strategy could range from one to hundreds of pages of text. All we need is a document template!

Not quite.

Over the last twenty years, almost all of the Test Strategy documents Paul has reviewed have been "copy and edits" of documents for previous projects. All that changed was the names and the dates. And all of these document were read by … no one.

We suggest that rather than being a document, test strategy is a thought process. The outcome of the thinking might be a short or a long document, but really, the strategy must address the needs of the participants inside the project as well as the customers of the product to be built. It needs to be appropriate to a short agile project or to a 1000 man-year development. It has to have the buy-in of stakeholders but most importantly, it must have value and be communicated. That's quite an ambitious goal. but we think it is achievable.

A Strategy Answers Questions

Before your organisation, your team or you as an individual test a system, you need some questions answered. These questions are pretty fundamental and yet, we have met many teams who could not answer them - even when they were in the thick of the testing. Here is a sample of the type of questions to be answered?

• Who are your testing stakeholders? What do they want from testing? Why do they want it? In what format? How often?
• What is the scope of testing? What is out of scope? How will you manage scope?
• How much testing is enough? What models should be used to guide the testing? How will you assess coverage?
• and so on...

One way of looking at strategy is to regard it as a way of answering these questions. But not every question can be answered up-front. Some questions cannot be answered now, but perhaps later when information come to light. So the strategy must provide guidance for answering these questions and making decisions. It does this in three ways:

1. Some decisions can be made now, directly and with confidence.
2. Some decisions cannot be made now, but the strategy will provide a process, method or mechanism for reaching a decision.
3. Some decisions cannot be made now, and no method could be formulated, but the strategy can provide some guiding principles to be followed in reaching a decision.

In this way, a strategy written early in a project, before all the information required is available, can flex and support a project that is heading into unknown territory. 

Is Agile Test Strategy an oxymoron? We don't think so. If you look at the three levels of decision making above, structured or waterfall projects aim to answer every questin ahead of time. So, in these projects, most of the quesitons can be directly answered so the emphasis is on 'type 1' decisions. In Agile or less certain projects, the lack of knowledge, or the need to remain flexible demands that the emphasis will shift more towards 'type 2' or 'type 3' decisions.

The Trick is to Ask the Right Questions

The challenge in test strategy is not providing perfect answers to these questions. The secret to success is in asking the right questions in the first place. Over twenty years, we have found oruselves asking the same questions again and again. The Test Axioms are an attempt to arrange these questions into a more usable structure. The Axioms represent context-neutral aspects of testing to think about. The 16 axioms are organised into three Axiom groups: Stakeholder, Design and Delivery. Each Axiom has 5-10 questions associated with the, the Axioms therefore provide a checklist of questions to ask, a a structure for your strategy document, should you coose to write one.

Our Test Strategy workshop

We have created a workshop titled 'Test Strategy in a Day' that you might find useful. In this workshop, we present a practical definition of a Test Strategy, provide a simple template for creating one and describe a systematic approach to thinking the right way. It's an interactive session where we invite you to bring your test strategy problems with you - we'll try and address them during the day.

• Test strategy is a thought process, not a document
• The goal is to acquire and communicate knowledge of achievement
• How a valuable test strategy can be formulated and communicated

 

To view the workshop brochure click here.

Tags: #testaxioms #TestStrategy #AgileTestStrategy

Paul Gerrard My linkedin profile is here My Mastodon Account