First published 30/09/2009
In the most unlikely place, I recently came across a simple explanation of an idea which helps resolve a long-standing tension between the need for IT process maturity and the tendency of developers to work informally. Testers are often caught in the centre of this tension, particularly because of their need to write tests early with accurately predicted test results.
For some years, I’ve struggled to strike a balance between formalised development methods and the practical, but rather informal, adaptations almost always adopted by projects under pressure to deliver. I firmly believe in the value of "process" in software engineering; I don’t believe in the immature "back-of-fag-packet" practices prevalent in the early days of computing. But I have also noticed that a formalised process doesn’t guarantee a successful project, and absence of a method does not automatically doom a project to failure. Above all, I’ve mused long and hard over why, if a repeatable, mature process is such a good thing, don’t IS practitioners more readily accept methods, like SSADM, and quality standards which emphasise process improvement, like TickIT?
In my experience, formalised approaches are millstones to the average practitioner. Excessive paperwork, unnecessary overhead and cost are the usual cries of pain from the software developer. The quality manager, in turn, accuses the software developer of artisan tendencies and general intellectual disregard for the necessity of engineering discipline.
On balance, I tend to side with the developer that the formalities of quality management systems and hindrance than a help. In fact, in their emphasis I believe they cause a backlash. Practitioners are turned off, and miss the essential message that there are better ways to develop software than in an undisciplined free-for-all.
In the last five years, I’ve explored Rapid Application Development (RAD) as the middle ground. RAD flexible and doesn’t rely on a conventional forest of paper deliverables. What is striking to me is that RAD embodies so many practices which successful project teams often seem to adopt naturally.
As good as I think it is, RAD raises serious questions. Structured development methods rely on the outputs of a stage against a (notionally complete) definition prepared at the outset. By contrast, RAD dispenses with
many intermediate products, focuses on validation of the end product, and generally conducts business a lot more interactively, in a climate which allows for a level of uncertainty.
Despite extensive definition of the products and process of a RAD development, proponents of conventional methods find it extremely difficult to imagine how it is possible to operate without the elements they believe are essential to good software engineering. These include complete, unambiguous statements of requirements, fully decomposed analyses and designs, and fully specified programs. These deliverables must be presented in strictly defined formats, subjected to formalised reviews and/or tested, and finally signed off. Moreover, it is not sufficient that happen strictly according to pre-defined procedures.
When the full formality of a quality-managed development processes is applied to the full range of software projects, it creates ludicrous anomalies. The worst such excess I encountered was that of a project leader spending a week preparing a project plan for a two-day project. The need for a lightweight, fleet a’ foot approach is self-evident to me, but how can it be explained convincingly where the line between the essential and the superfluous gets drawn?
With all this as background, I happened to be reading a book on object oriented modelling with the scintillating title, "UML Distilled", by Martin Fowler. The book is an abbreviated description of The Universal Modelling Language, being developed by three gurus of object oriented methods, Booch, Jacobson and Rumbaugh. The second chapter is an overview of a development process. Mr Fowler has "changed my life" with the following paragraph:
"Projects vary in how much ceremony they have. High-ceremony projects have a lot of formal paper deliverables, formal meetings, formal sign-offs. Low-ceremony projects might have an inception phase that consists of an hour’s chat with the project’s sponsor and a plan that sits on a spreadsheet. Naturally, the bigger the project, the more ceremony you need.
The fundamentals of the phases still occur, but in very different ways."
This is practically all that Fowler says on the subject. But it is enough. In one simple word, ceremony, he encapsulates a clear distinction between formality and the trappings of formality. To amplify a bit, imagine the difference between a registry office and a church wedding. Both achieve the same central aim, legally marrying the couple, both are formal, both have defined processes, but one has a lot more ceremony than the other. It is not formality which distinguishes them, but rather the level of ceremony.
The same applies to development processes whether RAD or other. All software development activity should be formal, but it is not necessary to burden all projects with high ceremony. Quality management systems, quality standards and development methods need to factor in the concept of an appropriate level of ceremony.
So then, how much ceremony is enough? Fowler relates high-ceremony to large projects, but I think there are other factors. The two which we see as key are:
- the organisational distance between the producers and the acceptors of the software; the greater the distance, the more ceremony required.
- the length of time between the inception and delivery of software product; the greater the time, the more ceremony needed.
RAD projects can be low-ceremony because, by design, they have a short organisational distance between software producers and acceptors – they are part of the same team – and the time from inception to delivery is always short. By the same token, it is easy to define projects which legitimately require high ceremony, and would not be suitable for RAD.
The concept of high and low ceremony is not only useful for putting quality management and formal processes in practical perspective, I believe it is equally powerful in attracting reluctant developers to better practice. It has got to help communicate why and how a well-defined process doesn’t have to equate to meaningless paperwork and all the other negative associations developers have about formalised processes.
I wholeheartedly recommend, "UML Distilled, Applying the Standard Object Modelling Language" by Martin Fowler with Kendall Scott, published by Addison Wesley Longman, 1997. If you find the idea of ceremony intriguing, you will find many other useful ideas there. Try, for example, Fowler’s description of "skeleton" which complements a low ceremony process.
© 1997 Paul Herzlich.
Tags: #ceremony #paulherzlich
Paul Gerrard
Please connect and contact me using my linkedin profile.
My Mastodon Account
£10 (free p&p)
