Babel, Software, Work

Here’s a bit I rather liked in Fred Brooks’ classic essay on the management of software engineering projects, The Mythical Man-Month:

According to the Genesis account, the tower of Babel was man’s second major engineering undertaking, after Noah’s ark. Babel was the first engineering fiasco.

The story is deep and instructive on several levels. Let us, however, examine it purely as an engineering project, and see what management lessons can be learned. How well was their project equipped with the prerequisites for success? Did they have:

A clear mission? Yes although naively impossible. The project failed long before it ran into this fundamental limitation.

Manpower? Plenty of it.

Materials? Clay and asphalt are abundant in Mesopotamia.

Enough time? Yes, there is no hint of any time constraint.

Adequate technology? Yes, the pyramidal or conical structure is inherently stable and spreads the compressive load well. Clearly masonry was well understood. The project failed before it hit technological limitations.

Well, if they had all of these things, why did the project fail? Where did they lack? In two respects – communication, and its consequent, organization. They were unable to talk to each other; hence they could not coordinate. When coordination failed, work ground to a halt. Reading between the lines we gather that lack of communication led to disputes, bad feelings, and group jealousies. Shortly the clans began to move apart, preferring isolation to wrangling.

TMMM, as the essay has come to be known, was first published in 1975 so some of what Brooks had to say has dated a bit, but there’s lots of stuff in there that’s wise and fresh to a reader in the present day. I’d add that the prejudices of those who, like me, were educated in the humanities might have about a work of this kind are pretty well confounded, since Brooks’ writing manages to be extremely clear and precise whilst achieving a nicely relaxed, off-duty-with-tumbler-in-hand tone, and that’s not an easy trick to pull off.

I’ve a sense that portions of the audience are beginning to wriggle uncomfortably in their seats: doesn’t this blog generally cover current affairs from a moderately egg-headed philosophical/social-scientific perspective? Why is this guy wittering on about elderly texts about the management of software projects?

Don’t start chucking popcorn yet, folks; I may have some problems with the egg-headed part of the brief, but I aim eventually to pull this around to some stuff that seems a bit closer to the central topics Crooked Timber usually covers.

The main thing that TMMM is famous for pointing out is that you can’t get a project which is behind its schedule back on track by adding more code-monkeys to it: Brooks offers as a oversimplification the slogan that ‘adding manpower to a late software project makes it later’. Why would that be?

Well first up, some tasks are inherently not susceptible to being partitioned into subtasks, so adding extra people to do them avails you precisely nothing: as Brooks memorably puts it, ‘the bearing of a child takes nine months, no matter how many women are assigned’.

More fundamentally, where a task can be partitioned, one has to take into account the communications overhead which comes from adding people. In general, the people working on a given subtask will have to communicate with oneanother and also with those working on the other subtasks. This means that as the number of developers grows linearly, the number of communication paths grows exponentially.

And systems of any size cannot be absorbed by new monkeys straight away: they need to be inducted into the code’s strangenesses and mysteries (which it undoubtedly has, since the project is presumably late for a reason) by someone who knows about them. Assigning such a guru to training the new chimps has an obvious opportunity cost.

Brooks puts it like this: ‘since software construction is inherently a systems effort – an exercise in complex interrelationships – communication effort is great, and it quickly dominates the decrease in individual task times brought about by partitioning’.

By emphasizing the importance of communication as a necessary feature and overhead cost of software development, Brooks nudged people towards recognising that software gets built by people, too. When we manage to do away with the picture of systems being built by interchangeable typing monkeys, we might begin to think about running projects in a fashion that reflects a different picture.

Of course, anyone who has done software-related stuff professionally (or who, indeed, has worked in any other project-based environment) knows that Brooks may as well have never bothered writing his famous essay. Managers still try to save projects by adding people, and seem perpetually surprised when it doesn’t work. More centrally, they continue to pray for a future in which employees really are entirely fungible ‘plug’n’play’ components who can be heaved about the place according to whim. And then sometimes they behave as if that future is already here.

So by the late ‘eighties, we have the opportunity to read books like Tom de Marco and Timothy Lister’s PeopleWare, which tries to explain why some different approaches need to be tried. The really wonderful thing about PeopleWare is that it hammers home the point that projects are staffed by human beings, who (shock) aren’t interchangeable, (horror) have lives, and (shame) would really prefer to hang on to them. It does so by drawing on stacks of empirical evidence about projects that succeed, projects that fail, the circumstances under which they do each, and so on. (There’s also a fascinating chapter on the lessons that the work of Christoper Alexander might have for the physical configuration of the modern work place en route.)

De Marco and Lister spend a lot of time explaining why teams which are treated decently outperform those which aren’t (and hence why the hardest-headed bean-counter ought to pay attention to their findings), but there’s also a real sense of anger at the sheer human waste which is generated by the many shoddily run projects which end up eating up the lives of those who are unfortunate enough to find themselves assigned to them. (There’s a good blast of that anger here, in a letter DeMarco wrote about so-called ‘Death-March’ projects.)

To pull things back towards Crooked Timber territory, what I most admire about PeopleWare is the way that DeMarco and Lister are reacting against a sort of Rand Corporation-inspired, Econ 101, and above all dehumanised conception of the working world. He understands amongst other things that (most) people care about their work, and want to do it well; that people are demotivated by foolish processes dictated from above whose only possible rationale is that those who are required to follow them cannot be trusted; and that work cannot replace the many other things that make for a decent human life.

I certainly have no idea about the state of the academic literature on this kind of thing, but I find it encouraging that it’s not too hard to find serious software methodologists beginning to look into the seriously into the contribution that sociological and ethnographic research strategies might make to their efforts to work out how to design projects. This paper, for instance, makes me feel optimistic that one day, the human beings may win out in the software industry.

Now, I’ve been drawing on some stuff that aims to solve a very particular problem: how the hell to run programming projects on time, to budget, with the right feature-set, and without driving anybody to a nervous breakdown. But the broader lessons about the work-place seem pretty obvious: businesses are run by people, who can only do their jobs well if they have some kind of internal motivation to do them at all, and who will do them poorly if they are not trusted to think for themselves. The pleasures of craft, and the autonomy allowed to professionals, are things that most of us need in order to be productive workers. We also need them to be happy human beings.

Perhaps someone should send a copy of PeopleWare to the senior managers at British Airways, who clearly don’t understand any of the above.