Paul Gerrard

First published 30/09/2009

In the most unlikely place, I recently came across a simple explanation of an idea which helps resolve a long-standing tension between the need for IT process maturity and the tendency of developers to work informally. Testers are often caught in the centre of this tension, particularly because of their need to write tests early with accurately predicted test results.

For some years, I’ve struggled to strike a balance between formalised development methods and the practical, but rather informal, adaptations almost always adopted by projects under pressure to deliver. I firmly believe in the value of "process" in software engineering; I don’t believe in the immature "back-of-fag-packet" practices prevalent in the early days of computing. But I have also noticed that a formalised process doesn’t guarantee a successful project, and absence of a method does not automatically doom a project to failure. Above all, I’ve mused long and hard over why, if a repeatable, mature process is such a good thing, don’t IS practitioners more readily accept methods, like SSADM, and quality standards which emphasise process improvement, like TickIT?

In my experience, formalised approaches are millstones to the average practitioner. Excessive paperwork, unnecessary overhead and cost are the usual cries of pain from the software developer. The quality manager, in turn, accuses the software developer of artisan tendencies and general intellectual disregard for the necessity of engineering discipline.

On balance, I tend to side with the developer that the formalities of quality management systems and hindrance than a help. In fact, in their emphasis I believe they cause a backlash. Practitioners are turned off, and miss the essential message that there are better ways to develop software than in an undisciplined free-for-all.

In the last five years, I’ve explored Rapid Application Development (RAD) as the middle ground. RAD flexible and doesn’t rely on a conventional forest of paper deliverables. What is striking to me is that RAD embodies so many practices which successful project teams often seem to adopt naturally.

As good as I think it is, RAD raises serious questions. Structured development methods rely on the outputs of a stage against a (notionally complete) definition prepared at the outset. By contrast, RAD dispenses with many intermediate products, focuses on validation of the end product, and generally conducts business a lot more interactively, in a climate which allows for a level of uncertainty.

Despite extensive definition of the products and process of a RAD development, proponents of conventional methods find it extremely difficult to imagine how it is possible to operate without the elements they believe are essential to good software engineering. These include complete, unambiguous statements of requirements, fully decomposed analyses and designs, and fully specified programs. These deliverables must be presented in strictly defined formats, subjected to formalised reviews and/or tested, and finally signed off. Moreover, it is not sufficient that happen strictly according to pre-defined procedures.

When the full formality of a quality-managed development processes is applied to the full range of software projects, it creates ludicrous anomalies. The worst such excess I encountered was that of a project leader spending a week preparing a project plan for a two-day project. The need for a lightweight, fleet a’ foot approach is self-evident to me, but how can it be explained convincingly where the line between the essential and the superfluous gets drawn?

With all this as background, I happened to be reading a book on object oriented modelling with the scintillating title, "UML Distilled", by Martin Fowler. The book is an abbreviated description of The Universal Modelling Language, being developed by three gurus of object oriented methods, Booch, Jacobson and Rumbaugh. The second chapter is an overview of a development process. Mr Fowler has "changed my life" with the following paragraph:

"Projects vary in how much ceremony they have. High-ceremony projects have a lot of formal paper deliverables, formal meetings, formal sign-offs. Low-ceremony projects might have an inception phase that consists of an hour’s chat with the project’s sponsor and a plan that sits on a spreadsheet. Naturally, the bigger the project, the more ceremony you need. The fundamentals of the phases still occur, but in very different ways."

This is practically all that Fowler says on the subject. But it is enough. In one simple word, ceremony, he encapsulates a clear distinction between formality and the trappings of formality. To amplify a bit, imagine the difference between a registry office and a church wedding. Both achieve the same central aim, legally marrying the couple, both are formal, both have defined processes, but one has a lot more ceremony than the other. It is not formality which distinguishes them, but rather the level of ceremony.

The same applies to development processes whether RAD or other. All software development activity should be formal, but it is not necessary to burden all projects with high ceremony. Quality management systems, quality standards and development methods need to factor in the concept of an appropriate level of ceremony.

So then, how much ceremony is enough? Fowler relates high-ceremony to large projects, but I think there are other factors. The two which we see as key are:

• the organisational distance between the producers and the acceptors of the software; the greater the distance, the more ceremony required.
• the length of time between the inception and delivery of software product; the greater the time, the more ceremony needed.

RAD projects can be low-ceremony because, by design, they have a short organisational distance between software producers and acceptors – they are part of the same team – and the time from inception to delivery is always short. By the same token, it is easy to define projects which legitimately require high ceremony, and would not be suitable for RAD.

The concept of high and low ceremony is not only useful for putting quality management and formal processes in practical perspective, I believe it is equally powerful in attracting reluctant developers to better practice. It has got to help communicate why and how a well-defined process doesn’t have to equate to meaningless paperwork and all the other negative associations developers have about formalised processes.

I wholeheartedly recommend, "UML Distilled, Applying the Standard Object Modelling Language" by Martin Fowler with Kendall Scott, published by Addison Wesley Longman, 1997. If you find the idea of ceremony intriguing, you will find many other useful ideas there. Try, for example, Fowler’s description of "skeleton" which complements a low ceremony process.

© 1997 Paul Herzlich.

Tags: #ceremony #paulherzlich

Paul Gerrard Please connect and contact me using my linkedin profile. My Mastodon Account

First published 05/11/2009

This year, I was asked to present two talks on 'Past, Present and Future of Testing' at IBC Euroforum in Stockholm and 'Future of Testing' at SQC London. I thought it would be a good idea to write some notes on the 'predictions' as I'm joining some esteemed colleagues at a retreat this weekend, and we talk about futures much of the time. These notes are notes. This isn't a formal paper or article. Please regard them as PowerPoint speaker notes, not more. They don't read particularly well and don't tell a story, but they do summarise the ideas presented at the two talks.

Registered users can download the paper from the link below. If you aren't registered, you can register here.

Tags: #futures

Paul Gerrard Please connect and contact me using my linkedin profile. My Mastodon Account

First published 05/11/2009

we're stood in a boat ankle deep in water. the cannibals are coming to kill and eat us. The testers are looking for the holes in the boat saying – we can't push off yet the river is full of hungry crocodiles.

The testers are saying – if we push off now, we'll all die.

The skipper is saying – if we don't push off soon, we'll all die.

I'ts the same with software.

Tags: #ALF

Paul Gerrard Please connect and contact me using my linkedin profile. My Mastodon Account

First published 04/11/2009

Hi,

With regards to the ATM accreditation -see attached. The cost of getting accredited in the UK is quite low – UKP 300 I believe. ISTQB UK will reuse the accreditation above.

Fran O'Hara is presenting the course this week. Next week I hope to get feedback from him and I'll update the materials to address the mandatory points in the review and add changes as suggested by Fran.

I've had no word from ISTQB on availability of sample papers as yet. I'll ask again.

I have taken the ATA exam and I thought that around one third of the questions were suspicious. That is, I thought the question did not have an answer or the provided answers were ambiguous or wrong. Interestingly, there are no comments from the client on the exam are there?

If their objective is to pass the exam only, then their objective is not the same as the ISTQB scheme. The training course has been reviewed against the ATA Syllabus which explicitly states a set of learning objectives (in fact they are really training objectives, but that's another debate). The exam is currently a poor exam and does not examine the syllabus content well. It certainly is not focused on the same 'objectives' as the syllabus and training material. If the candidates joined the course thinking the only objective was to pass the exam, then they will not pay attention to the content that is the basis of the exam. I would argue that the best way to pass the exam is to attend to the syllabus. The ‘exam technique’ is very simple – and the same as the Foundation exam. A shortage of test questions should not impair their ability to pass the exam. The exam is based on the SYLLABUS. The course is based on the SYLLABUS.

Here's my comments on their points – in RED.

• The sessions were not oriented to pass the exam. They were general testing lessons� the main objective of the training should be to prepare the assistants for the examination. That is not the intention of the ISTQB scheme. If we offered a course that focused only on passing the exam we would certainly lose our accreditation. Agree that a sample paper is required ((ISTQB to provide). It is extremely hard to prepare course material for the exam without having a sample paper. Although I have taken the exam (and found serious fault with it) I have not got a copy and was not allowed to give feedback. Most of the dedicated time in the training was not usable to pass the exam: The training was more oriented to test management than test analyst, which was the objective. I don’t know if this is true of the material, or the way you presented it. Since the course is meant to be advanced and not basic, the material will be more focused on the tester making choices rather than doing basic exercises. The syllabus dedicates three whole days to test technques – not management specific material. For example: a lot of time dedicated to risk management theory and practice and the specific weigh in the exam for that section was not so high. True. The section on risk based testing is too long and needs cutting down.
• More exercises needed: the training included some exercises but were similar to the foundation level ones. The training provider must be responsible to find and include advanced exercises. The exercises are similar to the Foundation course exercises because the Foundation syllabus is reused. The difficulty of the ATA exercises is slightly higher. However, because the exam presents multiple choice answers, the best technique for obtaining the correct answer may not be how one tests. This is a failure of the exam not the training material. (Until we get a sample paper, how can we give examples of exam questions?) o Examples of exercises: ? For an specific situation: How many test conditions.. using this test technique??? Not sure I understand. Is the comment, “can we have exercises that ask, how many conditions would be derived using a certain technique?” Easily done – just count the conditions in the answer. ? From our experience the exercises included in the exam were similar to the basic one but more complex. Are they saying the ATA exam was like the Foundation exam – but more difficult? That is to be expected. Perhaps we provide some exercises from Foundation materials but make them a little more involved. There are a small number, but I agree we need to provide a lot more.
• The training would include more reference to the foundation level Er not sure what this means. Could or should? Are they asking for more content lifted from Foundation scheme to be included in the course? In fact, much of the reusable material is already in the course (it’s much easier to reuse rather than write new!) Not sure what they are asking here.
• Sample exams needed Agreed!
• A lot of time dedicated in the sessions to theory than can be just self studied by assistants: i.e. Quality attributes This is possible. Perhaps we could extract content from the syllabus and publish this as a pre-read for the course? There are some Q&A in thehandouts already, but more could be added. However, a LOT of the syllabus could be treated this way.
• More practical needed for the following modules: o Defect management Isn’t this covered in the Advanced Test Management syllabus? (They want LESS management don’t they?) o Reviews: in the training we covered theory (types, roles�) but not practical questions like the exam�s We don’t know what the review questions in the exam look like. They are unlikely to be ‘practical’.

The general conclusion is that the training should be pass exam oriented. See my comment above. If this is REALLY what they want – they do not need a training course. They should just memorise the syllabus, since that is what the exam is based on. Some of the comments above, I think are legitimate and we need to add/remove/change content in the course. Some of the ATM material could be reused as it is possibly more compact. (Risk, incidents, reviews). Yes we need more sample questions – agreed! But I think some of the comments above betray a false objective. If we taught an exam-oriented course they would pass the exam but not learn much about testing. This is definitely NOT what the ISTQB scheme is about. However, people like Rex Black are cashing in on this. See here: https://store.rbcs-us.com/index.php?option=com_ixxocart&Itemid=6&p=product&id=16&parent=6 What will you suggest to the client re: getting their people through the exams? I hope some of the text above will help. If you do have specific points (other than above) let me know. I will spend time in the next 2-3 weeks updating the materials.

Tags: #ALF

Paul Gerrard Please connect and contact me using my linkedin profile. My Mastodon Account

First published 03/12/2009

You may or may not find this response useful. :–)

“It depends”.

The “it depends” response is an old joke. I think I was advised by David Gelperin in the early 90s that if someone says “it depends” your response should be “ahh, you must be a consultant!”

But it does depend. It always has and will do. The context-driven guys provide a little more information – “it depends on context”. But this doesn't answer the question of course – we still get asked by people who really do need an answer – i.e. project managers who need to plan and to resource teams.

As an aside, there’s an interesting discussion of “stupid questions” here. This question isn't stupid, but the blog post is interesting.

In what follows – let me assume you’ve been asked the question by a project manager.

The 'best' dev/tester ratio is possibly the most context-specific question in testing. What are the influences on the answer?

• What is the capability/competence of the developers and testers respectively and absolutely?
• What do dev and test WANT to do versus what you (as a manager) want them to do?
• To what degree are the testers involved in early testing (they just system test? Or are involved from concept thru to acceptance etc.)
• What is the risk-profile of the project?
• Do stakeholders care if the system works or not?
• What is the scale of the development?
• What is the ratio of new/custom code versus reused (and trusted) code/infrastructure?
• How trustworthy is the to-be-reused code anyway?
• How testable will the delivered system be?
• Do resources come in integer whole numbers or fractions?
• And so on, and so on…

Even if you had the answers to these questions to six significant digits – you still aren’t much wiser because some other pieces of information are missing. These are possibly known to the project manager who is asking the question:

• How much budget is available? (knowing this – he has an answer already)
• Does the project manager trust your estimates and recommendations or does he want references to industry ‘standards’? i.e. he wants a crutch, not an answer.
• Is the project manager competent and honest?

So we’re left with this awkward situation. Are you being asked the question to make the project manager feel better; to give him reassurance he has the right answer already? Does he know his budget is low and needs to articulate a case for justifying more? Does he think the budget is too high and wants a case for spending less?

Does he regard you as competent and trust what you say anyway? This final point could depend on his competence as much as yours! References to ‘higher authorities’ satisfy some people (if all they want is back-covering), but other folk want personal, direct, relevant experience and data.

I think a bit of von Neumann game theory may be required to analyse the situation!

Here’s a suggestion. Suppose the PM says he has 4 developers and needs to know how many testers are required. I’d suggest he has a choice:

• 4 dev – 1 tester: onus is on the devs to do good testing, the tester will advise, cherry pick areas to test and focus on high impact problems. PM needs to micro manage the devs, and the tester is a free-agent.
• 4 dev – 2 testers: testers partner with dev to ‘keep them honest’. Testers pair up to help with dev testing (whether TDD or not). Testers keep track of the coverage and focus on covering gaps and doing system-level testing. PM manages dev based on tester output.
• 4 dev – 3 testers: testers accountable for testing. Testers shadow developers in all dev test activities. System testing is thorough. Testers set targets for achievement and provide evidence of it to PM. PM manages on the basis of test reports.
• 4 dev – 4 testers: testers take ownership of all testing. But is this still Agile??? ;–)

Perhaps it’s worth asking the PM for dev and tester job specs and working out what proportion of their activities are actually dev and test? Don’t hire testers at all – just hire good developers (i.e. those who can test). If he has poor developers (who can’t/won’t test) then the ratio of testers goes up because someone has to do their job for them.

Tags: #estimation #testerdeveloperratio

Paul Gerrard Please connect and contact me using my linkedin profile. My Mastodon Account

Background

Michael Bolton recently posted a message on LinkedIn as follows (https://www.linkedin.com/posts/michael-bolton-08847_low-code-testing-tools-are-really-low-testing-activity-6921751556463685633-pcQS:

“Low-code testing tools” are really “low-testing code tools”

In what follows I infer no criticism of Michael or the people who responded to the post, whether positive or negative.

I want to use Michael’s proposition to explain why we need to be much more careful about:

• How we interpret what other people say or write
• How we assess its clarity, credibility, logical consistency or truthfulness
• Whether we understand the assumptions, experience or agenda of the author, or of ourselves as the audience
• And so on…

This is a critical thinking exploration of a phrase of a ten-word, five-term sentence.

I’ve been reading a useful book recently, “The Socratic Way of Questioning” – written by Michael Britton and published by Thinknetic. Thinknetic publish quite a few books in the critical thinking domain and you can see these here: (https://www.amazon.co.uk/Thinknetic/e/B091TWBHXN/)

I’ll use a few ideas I gleaned from the book, describe their relevance in a semi-serious analysis of Michaels’ proposition and hopefully, illustrate some aspects of Critical Thinking – which is the real topic of this article. I don’t claim to be an expert in the topic. But I’m a decent reviewer and can posit a reasonable argument. I’ll try to share my thought process concerning the proposition and take a few detours on the overall journey to expand on a few principles.

I’ve no doubt that some of you could do a better job than I have.

The importance of agreed definitions

The first consideration is that of definition. How can you have a meaningful discussion without having agreed meanings of the words used in that discussion? Socrates himself is said to spend at least half of his time in argument trying to get consensus on the meanings of words. Plato’s Republic, the most famous Socratic dialogue spends most of its time defining just one term – justice

Turning towards the proposition, what do the terms used actually mean? Do they mean the same thing to you as they mean to Michael? Is there an agreed, perhaps universally agreed definition of these terms? So, what do the following terms mean?

• Code
• Testing
• Tools
• Low-Code
• Low-Testing

Now, in exploring the potential or actual meanings of these terms, I’ll have to use some, perhaps many terms that I won’t define. Defining them precisely in a short article is inevitably going to get very cumbersome – so I won’t do that. Of course, we need an agreed glossary of terms and their definition as the basis of the definitions at hand. But it’s almost a never-ending circle of definitions. The Oxford English Dictionary, when I last looked, has compiled over 600,000 definitions of words and phrases covering “1000 years of English” (https://public.oed.com/history/">https://public.oed.com/history/).

Recently, I have been doing some research with ‘merged’ regular English dictionaries and some of the testing-related glossaries available on the internet. I now have some Python utilities using open-source libraries to scrape web pages and PDF documents and scan them for terms defined in these sources as well as detection of phrases deemed ‘meaningful’. (What meaningful means is not necessarily obvious). I'll let you know what I'm up to in a future post.

We have a real problem in the language used in our product and service marketing, and sadly our exchanges on testing topics in discussions/debates

Definitions

Here goes with my definitions and sources, where I could identify and/or select a preferred source. I have tried to use what I think might be Michael’s intended definition. (I am surely wrong, but here goes anyway).

Code: generally defined as a programming or script language used to create functional software to: control devices; capture, store, manipulate and present data; provide services to other systems or to humans. In this context, we’re thinking of code that controls the execution of some software under test.

Testing: …is the process of evaluating a product by learning about it through experiencing, exploring, and experimenting, which includes to some degree: questioning, study, modelling, observation, inference, etc. (Source: (https://www.satisfice.com/blog/archives/856) – there’s a security vulnerability on the page by the way – so use an incognito window if your browser blocks it).

Tools: Oxford English Dictionary): a device or implement, especially one held in the hand, used to carry out a particular function. Merriam Webster: something (such as an instrument or apparatus) used in performing an operation or necessary in the practice of a vocation or profession. In our context – software test execution automation – we refer to software programs that can: stimulate a system under test (SUT); send inputs to and receive outputs from the SUT; record specific behaviours of the SUT; compare received outputs or recorded behaviours with prepared results. And so on.

Low-Code: There is no agreed definition. Here is an example relating to software-building which is reasonable (https://www.mendix.com/low-code-guide/">https://www.mendix.com/low-code-guide/): “Low-code is a visual approach to software development that optimizes the entire development process to accelerate delivery. With low-code, you can abstract and automate every step of the application lifecycle to streamline the deployment of a variety of solutions.” I’d substitute software test development for software development. In the context of testing, the terms ‘codeless’ is more often used. This definition appears on a tools review website ((https://briananderson2209.medium.com/top-10-codeless-testing-tools-in-2020-every-tester-should-know-2cb4bd119313)): “Simply put, codeless test automation means creating automated tests without the need to physically write codes. It provides testers lacking sophisticated programming skills with project templates for workflow, element libraries, and interface customization.”

Low-testing: I’m pretty certain Michael has invented this term without defining it. But we can guess, I think. There are two alternatives – either (a) the quantity of testing is lower (than what?). We don’t know. But let’s assume it refers to ‘enough’... or (b) the quality of testing is lower than it should be (by Michael’s standard, or someone else’s – we don’t know). I suspect (b) is the intended interpretation.

Interpreting the Proposition

Since we now have some working definitions of the individual terms, we can expand the proposition into a sentence that might be more explicit. We’ll have to document our assumptions. This carries a risk: have we ‘guessed’ and misunderstood Michael’s purpose, thinking, assumptions and intended meaning?

Anyway. Here goes:

“Using test execution tools that require less (or no) coding means the quality (or quantity, possibly) of your testing will be reduced.”

Here are our assumptions in performing this expansion:

• The ‘low-testing’ phrase implies the lower quality or quantity testing is achieved. (Does it matter which? It would probably help to know, of course).
• Only using such tools causes this loss of quality or quantity. Does having such tools, but not using them, not?
• Having knowledge of some of Michael’s previous writings about tools and testing justifies our (still ambiguous) expanded phrase, I believe.

Ambiguities

We have several ambiguities in our reformulation of the original proposition:

• Do low-code tools have the effect of lowering the quality/quantity of testing or do ‘high-code’ – that is, all tools – do this?
• Do tools lower the quantity, or do they lower the quality of testing? (Or both?)
• Lowered compared to what? Testing without tools? (Or see below… tools with ‘higher-code’)
• What is meant by ‘the quality of testing’? How is that defined? How could it be measured? Evaluated?
• The quantity of testing might be reduced – but might this be a good thing, if the quality is the same or improved?
• There’s no mention of the benefits of using (low-code) tools – but is the overall effect of using tools (low or high code) detrimental or beneficial?
• There is no mention of context here. Is the proposition true for all contexts or just some? Could using such tools possibly be a universally best practice? Or a universally bad one?

Why This Proposition?

Is this proposition making a serious point? Does it inform a wider audience? I don’t think so. It’s clickbait. Some people will (perhaps angrily) agree and some (angrily) disagree. These people will probably be driven by preconceptions and existing beliefs. Do ‘thinking people’ respond to such invitations?

It’s a common tactic on internet forums and I’ve used the tactic myself from time to time. Although I think it’s a lazy way of starting a debate because the ‘thoughts from abroad/musings/worldview/the toilet’ (strike out what doesn’t apply) – tend not to trigger debate, they just cause some folk to extol/confirm their existing beliefs.

So, don’t ask me, ask Michael.

Summary

I’ve written over a thousand words so far, analysing a ten-word, five-term proposition. I’m sure there is much more I could have written if I thought more deeply about the topic. A word-ratio of 100 to 1 isn’t surprising. Many books have been written discussing less complicated but more significant propositions. Love, justice, quality, democracy… you know what I mean.

The lack of agreed definitions of the terms we use can be a real stumbling block, leading to misinterpretations and misconceptions hardly likely to improve our communications, consensus and understanding. Even with documented definitions there are problems if people dispute them.

Before you engage in a troublesome conversation of a proposition, ask for definitions of the terms used and try, with patience, to appreciate what the other person is actually saying. You may find gold or you may demolish their proposition. Or both.

Finally

Am I suggesting you apply critical thinking to all conversations, all communications? No of course not. Life is too short. And surely, if you analyse all your partner’s words you are bound to land in scalding hot water. So, like testing and all criticism – it has its place. A weapon of choice, in its place.

Critical thinking, like testing, should be reserved for special occasions. When you really want to get to the bottom of what an author, speaker or presenter is saying or a supplier is supplying. How can you agree or disagree without understanding exactly what is being proffered? Critical thinking works.

What’s my Response to the Proposition?

“After all this circumlocution, what do you think of the proposition, Paul?”

I think there are more interesting propositions to debate.

Tags: #thinkingtools #testing #low-codetesttool #codelesstesttool #criticalthinking #clickbait #testingterminology #CriticalThinking

Paul Gerrard Please connect and contact me using my linkedin profile. My Mastodon Account