My experiences in the Test Engineering business; opinions, definitions and occasional polemics. Many have been rewritten to restore their original content.
First published 22/06/2021
It's a common question. I need to lead, but do I have the ability to do that?
There's lots of talk about 'born leaders' but what if I'm not a born leader? Can I still succeed?
In this video, I suggest leadership can be learned and enhanced with practice and support.
Paul Gerrard My linkedin profile is here My Mastodon Account
First published 10/05/2016
There's been a thoughtful debate on the notion of 'test automation' being a rather bad, lazy, misrepresentation as a concept or entity or thing. James Bach and Michael Bolton wrote a paper here. Alan Richardson, in Testing Circus Magazine invites you to join the Anti Test-Automation Brigade. Pretty much, I agree with the sentiments expressed. The term is bad, it exhibits and probably promotes muddled thinking. I will do my best not to use it. Unless I need to reference the range of tools that dominate the testng tools market.
I'll return to this a bit later.
My Dad called himself an engineer (or rather, his company did). In his national service, he joined the Royal Engineers and was very proud of his time in the post-war, peacetime Army. He told me that he spent much of his time going on training courses. When offered the choice of a course on "4.5 ton truck engine maintenance" or two weeks tramping around muddy fields in Germany, unlike many of his comrades who preferred the mud to a classroom or workshop, he chose the courses.
He also did real some engineering - building and dismantling bridges, towing tanks out of ditches and other hearty activities. (Once a tank was bogged down near a churchyard. Dad's idea was to pass a cable around the church to improve the towing angle on the cable. When the recovery truck pulled, the tank was unmoved, but the church started to disintigrate. They stopped the truck and figured out another way to extract the tank, leaving the church intact to great relief). And so on.
So when the time came to selecting a course at university - an engineering degree was the natural choice for me. I thought I would be part of a select, engineering elite. Yes, there were civil, electrical and mechanical engineers, but they were all a happy and respected band of professionals. Then I worked at a large civil engineering consultancy. Everyone was an engineer. And then I noticed, half the male population called themselves engineers of one sort or another. When I got into the software business, I discovered some software people also called themselves engineers. This was a step too far.
Now, my main criteria for the definition of an engineer is someone who works with concepts, designs or materials whose behaviour are underpinned by the laws of physics. Software, in my humble opinion, does not obey any laws of Physics that I know - so it is not an engineering discipline. Discuss. At any rate, it seems nowadays that anyone can call themselves an engineer. Raging against this calumny seems not to be a good use of my time.
The testing tool vendors are not about to change their marketing and rename all their tools on the basis of complaints from a few vociferous individuals. So I suspect we are stuck with the Test automation term for the foreseeable future. I support the view that test automation is a lousy term, but I don't thnk I'm going to get excited about removing or replacing it.
However.
The view that test automation does not encompass support for all of testing is obvious and if test automation does have a scope, it covers the application of tests (or checks). Some tools can perform logging and reporting, possibly, but let me ignore that for the moment. I am using the teminology of the New Model below.
Let me suggest test automation refers to the 'Applying' activity only. If you add in other utilities that come under the banner of "testing with tool support" - these utilities mostly fall into the Applying aspect of testing. Building environments, preparing test data, generating combinations, analysing results, editors, comparators, analysers, harnesses, frameworks, loggers, cleaner-uppers and tear-downers and so on are all pretty much covered by the Applying activity. So if test automation represents the Applying activity only, it may not be well named, but it could at least be well-scoped.
Perhaps we should rename Test Automation ... Test Application? Or just Application?
More importantly, the nine other activities in the New Model plus the judgements of model adequacy or inadequacy hardly get a mention in the argument which ignores the opportunities to support thinking, collaboration, modelling, test design, record-keeping and so on.
It seems to me that the testing industry - both testers and vendors - are obsessed with test automation to the degree that (with some exceptions) testers are not demanding product to support exploration and modelling activities on the left hand side of the New Model and vendors are not investing in R&D. There are signs this is changing but it has been a slow process so far. Supporting these activities could bring huge benefits to testers. The market for these tools (required by all testers possibly) is also huge. Why are test design tools not being demanded and delivered?
If we label all test tools test automation, it take our eyes of the real prize - tools to support all of testing. Testers and vendors need a broader perspective.
I've been banging on about wanting Test Design tools rather than Test Execution tools for some time. I dug out a talk of 1999 which I gave several times including a variation at STAR East 1999. I am quite pleased - maybe 75% could be repeated today without embarrassment. But I am also rather depressed that I am making almost identical recommendaitons nearly sixteen years later. Click on the image or here to see the slides.
Firstly, you may know I am working to create a comprehensive tools directory at https://tkbase.com - it covers DevOps, Collaboration and Testing and there are not ten or twenty tool types - there are over 180 tool types with 2414 tools listed.
Second, I am writing a paper on the applicaiton of automation AI and Deep Learning to support the other (exploration, thinking, collaboration and record keeping) activities of testing.
Finally, for the last few months, I've been writing and experimenting with a robot that supports exploration and testing - exploratory testing - if you like. It will support team exploration, pair testing and can also be voice controlled. I'll have something to demonstrate in the next month or so. I'll be talking about experiences with this more fully later in the year. If you are interested in being a pre-alpha tester - do get in touch :O)
What are you doing to further the case for tools that support all of testing?
Paul Gerrard My linkedin profile is here My Mastodon Account
First published 17/03/2016
For as long as the web has existed, there have been websites that provide lists of references to tools that support, for example, test automation. These web pages and sites have been set up by individuals, wishing to share their knowledge of software tools for their own communities.
It's a burdensome task to create and maintain these lists. Vendors move webpages around, they rename tools, they merge and split tool functionality, they add new tools and new vendors and tools are popping up all the time. It's really hard to maintain the accuracy of lists like these. If you look around the various websites that provide such lists, this is what you tend to find:
The Tools Knowledge Base is a free-to-use service providing information on tools, vendors and the consultants and service companies that support them.
If you are tools user: we'd like you to register, and identify the tools you use. your tools chain will appear on your profile. If a tool you use is not in TKB, then we invite you to create it. (If you want to embed the tools you use as a list in your website - we have an API for that).
If you are a tools expert or tools service provider: please see above, plus... We are looking for people who are knowledgeable enough to review or possibly edit the features listings for the tools you know well. The features hierarchy will grow and evolve over time - help us to perfect it. We will list you as a service provider on the tools you know best. It's the least we can do.
If you are a tool vendor: we ask you to search for the tools you offer and check they are in the system and properly described. If your tools exist in the system and you want to manage the information we hold, that's fine - let us know and we can make you the administrator (after a couple of quick checks). If you want, we can maintain the data on your behalf, for a small fee. Do get in touch. Alternatively, nominate a tools expert (see above) and we'll invite them to keep your details correct.
If You Own/Contribute to an Open Source Project: We make exactly the same offer as we make to the vendors. you are free to edit the information for your tools in the same way. you might already be represented on a site like GitHub - we're just offering an extra publicity channel - it might help you reach a broader audience. People looking for tools often start their search with proprietary products but rarely see free tools listed side by side. Now is your chance.
If you are a blogger: Search for one of your recent blog posts and if you find it - your blog is already indexed in the system. If not, you can suggest the blog and register it yourself. Please note we are looking for blogs that cover our scope (DevOps, Testing, Collaboration). Company blogs are acceptable if they focus on these topic areas. Some general technology blogs might be acceptable, at our discretion. Offensive or blatantly commercial posts are not acceptable.
If your blog is indexed, let us know, we will give you credit for it on your profile.
If you maintain your own online tools listing: please get in touch. We believe we already have a more comprehensive list (in our scope) than anyone else. We offer to provide you with ad free, embeddable tool type listings for your existing site. Join us as a partner and tools expert and help us to improve the data in our system to improve the listings on your site. We currently have 183 tool type listings. All are available for free. We'll give you your own lists and track the usage data and share that with you too. Drop me a line and I can explain what we can do for you.
If you want a tools listing on your own site: Get in touch. We make the same offer as above.
Can I advertise my tool or service on your site? Of course you can. We offer competitive CPM (cost per thousand impressions) rates. Our users are tools-focused so your ads will be seen by the right people. We will of course provide monthly statistics for your perusal.
Please take a look and let me know what you think.
Paul Gerrard My linkedin profile is here My Mastodon Account
First published 22/05/2014
Some weeks back I presented a webinar about the Internet of Everything, with special focus on what the concept is and how it is going to affect us.
This was the first of the webinars based on the article series that I am writing for Tea-time with Testers further articles and webinars will be appearing soon.
Paul Gerrard My linkedin profile is here My Mastodon Account
First published 10/02/2017
The Test Management Forum was set up in 2004 and the first meeting took place on 28 January of that year. We have run events every quarter since then, and the 53rd and latest meeting took place at the end of January. Over 2,600 people have attended the London events over the years. Although most Forum attendees come from the UK, the uktmf.com website has approaching 10,000 registered users worldwide and the LinkedIn group 11,800 members.
The TMF continues to be a popular Forum with a global (and not just a UK) influence.
The original aim of the Forum was to bring together more senior practitioners, to network and share knowledge. The core of the Forum meetings are discussion sessions. They are run by an expert in their field and comprise an introductory presentation of 15-30 minutes or so, followed by a facilitated discussion of the issues raised in the talk. Sessions are 75 minutes long and often stir up vigorous debate. The format and ethos of the Forum are unchanged since 2004.
But over the last thirteen years, there have been many changes in the industry and in the Test Management community. I’d like to talk around some of these changes and explain why we decided to bring the TMF up to date with an industry that is far different from that in the early days of the Forum.
In 2004, the Agile movement was still in the early stages. For the next ten years or so, we ran many sessions that explored the changing (sometimes disappearing) role of test managers. I recall having many conversations with people who had been ‘Agiled’ and were concerned their role and contribution to Agile projects was uncertain.
The Test Managers I knew personally reacted in very different ways. Some managers and leaders became testers again, some specialised in test automation or security testing or usability. Others moved closer to their business, became business analysts or consultants. Quite a few were promoted out of testing altogether to become project managers, development leads or heads of solution delivery. Some senior managers retired from IT altogether.
Agile had a big impact, but approaches such as continuous delivery, DevOps, shift- left, shift-right, shift-wherever and analytics are changing the roles of testers in fundamental ways, and I should say, usually for the better. Some testers are falling by the wayside; getting out of testing. The pressures on Test Managers continue, but new doors are opening and opportunities emerging. Some test managers become Scrum Masters, more are finding new roles which I have labelled as Test Masters and in general, these people are performing an Assurance role.
Assurance (as distinct from Quality Assurance) is very much focused on delivery. The assurance role supports the delivers team by acting, at various times, as a process consultant, as a testing expert, as a reviewer, as a project-board level advisor and sometimes as an auditor. This variety of roles is more senior, more comprehensive and more influential than the traditional test manager role. The range of skills and authority required are beyond many test managers – it is not for everybody. But Assurance is a more senior role with a broader influence and is a natural advancement for aspiring test professionals.
With this background, we have decided to re-brand the Test Management Forum to be the Assurance Leadership Forum (or ALF) from April 2017. The ALF has very much the same goals as the TMF, but with a broader organisational, managerial and delivery-focused remit. Test leadership and technical testing issues will figure prominently in the ALF, but the range of topic areas will increase, with an alignment to the aspirations and career progression of senior practitioners.
Mike Jarred, of the Financial Conduct Authority has helped behind the scenes with the Forum for some time. Mike is also one of those managers with a testing background who has advanced beyond testing and into broader software delivery management. He is well placed to take over the Programme Management of the Forum. In December, I moved house to Macclesfield in Cheshire, making it harder for me to host and programme the London-focused Forum events. Mike offered to take the programme role and I gladly took up his offer.
I will continue to be the ‘host’ of the Forum and of course support Mike in putting the events together. My more focused role will allow me to take a more active part in the discussions themselves, for a change! I’ll be asking for expressions of interest in setting up a Northern Forum in the Summer before too long, I'm sure.
In the meantime, I hope you will support the Assurance Leadership Forum, and Mike in leading it.
The existing uktmf.com domain will continue to point to the Forum website, but a new domain ukalf.com points to the same content and will be our preferred url from now on. The LinkedIn group will be also renamed shortly.
Over the next few weeks, we’ll be rewriting some of the content of the website to better define the purpose of the ALF. I expect we’ll have a debate on possible directions in the next Summit. The First ALF Summit will take place on 25 April in London and if you want to contribute to the day, do let us know.
Paul Gerrard My linkedin profile is here My Mastodon Account
First published 01/07/2015
At the end of May I spent a very pleasant few days in Minsk, Belarus at the SQA Days conference. Many thanks to Vladislav Orlikov for inviting me and giving me the opportunity to visit a new country and meet a lot of very nice and smart people.
Vlad sent me a link to the video of my Keynote and you can see it below. The background noise is mostly the sound of the simultaneous translation, which diminishes a few minutes into the video.
How to Test the Internet of Everything (in English) from Vlad Orlikov on Vimeo.
Vlad also used a PDF I sent him to print 500 copies of The Tester's Pocketbook to give to every attendee. Books are highly regarded in Belarus and for 24 hours I was extremely popular and had to sign (I estimate) 150-200 of them!
Tags: #IOE #IOT #InternetofEverything #InternetofThings
Paul Gerrard My linkedin profile is here My Mastodon Account
First published 28/11/2012
Last Friday, I was happy to present to the Skillsmatter Agile Testing and BDD Exchange at 'The Crypt'. Thanks to Gojko Adzic for inviting me and chairing the day.
You can see the video of my talk here. Afraid Skillsmatter don't allow embedding the video.
The significance of the talk is that, although I've been writing about my frustration with testing (Testing is in a Mess) and the Redistribution of Testing for 18 months or so, last week's talk is the first that makes a coherent argument for redistribution of testing in non-Agile environments.
I positioned the talk as a sales pitch to corporates but being presented to Agile devotees, so please excuse the (I'm Agile, honestly guvnor) caveats at the beginning.
Tags: #agile #businessstorymethod #BusinessStoryManager #BDD #redistributionoftesting
Paul Gerrard My linkedin profile is here My Mastodon Account
First published 20/12/2012
After the Next Generation Testing event in December 2012, Unicom asked me to summarise the thinking behind the proposal for the redistribution of testing. Four minutes 11 seconds. :O)
Tags: #BDD #redistribution #tdd
Paul Gerrard My linkedin profile is here My Mastodon Account
First published 23/12/2013
*** See our 'Test Strategy in a Day' workshop ***
Test strategy is one of those nebulous terms for an activity that everyone has to undertake for their development projects. The problem with test strategy is that there isn't a single agreed definition of what it is. Some people treat Test Strategy as a document: "A high-level description of the test levels to be performed and the testing within those levels for an organization or programme". So - a strategy could range from one to hundreds of pages of text. All we need is a document template!
Not quite.
Over the last twenty years, almost all of the Test Strategy documents Paul has reviewed have been "copy and edits" of documents for previous projects. All that changed was the names and the dates. And all of these document were read by … no one.
We suggest that rather than being a document, test strategy is a thought process. The outcome of the thinking might be a short or a long document, but really, the strategy must address the needs of the participants inside the project as well as the customers of the product to be built. It needs to be appropriate to a short agile project or to a 1000 man-year development. It has to have the buy-in of stakeholders but most importantly, it must have value and be communicated. That's quite an ambitious goal. but we think it is achievable.
Before your organisation, your team or you as an individual test a system, you need some questions answered. These questions are pretty fundamental and yet, we have met many teams who could not answer them - even when they were in the thick of the testing. Here is a sample of the type of questions to be answered?
One way of looking at strategy is to regard it as a way of answering these questions. But not every question can be answered up-front. Some questions cannot be answered now, but perhaps later when information come to light. So the strategy must provide guidance for answering these questions and making decisions. It does this in three ways:
In this way, a strategy written early in a project, before all the information required is available, can flex and support a project that is heading into unknown territory.
Is Agile Test Strategy an oxymoron? We don't think so. If you look at the three levels of decision making above, structured or waterfall projects aim to answer every questin ahead of time. So, in these projects, most of the quesitons can be directly answered so the emphasis is on 'type 1' decisions. In Agile or less certain projects, the lack of knowledge, or the need to remain flexible demands that the emphasis will shift more towards 'type 2' or 'type 3' decisions.
The challenge in test strategy is not providing perfect answers to these questions. The secret to success is in asking the right questions in the first place. Over twenty years, we have found oruselves asking the same questions again and again. The Test Axioms are an attempt to arrange these questions into a more usable structure. The Axioms represent context-neutral aspects of testing to think about. The 16 axioms are organised into three Axiom groups: Stakeholder, Design and Delivery. Each Axiom has 5-10 questions associated with the, the Axioms therefore provide a checklist of questions to ask, a a structure for your strategy document, should you coose to write one.
We have created a workshop titled 'Test Strategy in a Day' that you might find useful. In this workshop, we present a practical definition of a Test Strategy, provide a simple template for creating one and describe a systematic approach to thinking the right way. It's an interactive session where we invite you to bring your test strategy problems with you - we'll try and address them during the day.
To view the workshop brochure click here.
Paul Gerrard My linkedin profile is here My Mastodon Account
First published 15/07/2013
See below the four presentations given by Paul at the World Conference on Next Generation Testing held in Bangalore, India between 8th and 12th July 2013.
Tags: #nextgentesting
Paul Gerrard My linkedin profile is here My Mastodon Account