Babel, Software, Work

by Tom on July 29, 2003

Here’s a bit I rather liked in Fred Brooks’ classic essay on the management of software engineering projects, The Mythical Man-Month:


According to the Genesis account, the tower of Babel was man’s second major engineering undertaking, after Noah’s ark. Babel was the first engineering fiasco.

The story is deep and instructive on several levels. Let us, however, examine it purely as an engineering project, and see what management lessons can be learned. How well was their project equipped with the prerequisites for success? Did they have:

A clear mission? Yes although naively impossible. The project failed long before it ran into this fundamental limitation.

Manpower? Plenty of it.

Materials? Clay and asphalt are abundant in Mesopotamia.

Enough time? Yes, there is no hint of any time constraint.

Adequate technology? Yes, the pyramidal or conical structure is inherently stable and spreads the compressive load well. Clearly masonry was well understood. The project failed before it hit technological limitations.

Well, if they had all of these things, why did the project fail? Where did they lack? In two respects – communication, and its consequent, organization. They were unable to talk to each other; hence they could not coordinate. When coordination failed, work ground to a halt. Reading between the lines we gather that lack of communication led to disputes, bad feelings, and group jealousies. Shortly the clans began to move apart, preferring isolation to wrangling.

TMMM, as the essay has come to be known, was first published in 1975 so some of what Brooks had to say has dated a bit, but there’s lots of stuff in there that’s wise and fresh to a reader in the present day. I’d add that the prejudices of those who, like me, were educated in the humanities might have about a work of this kind are pretty well confounded, since Brooks’ writing manages to be extremely clear and precise whilst achieving a nicely relaxed, off-duty-with-tumbler-in-hand tone, and that’s not an easy trick to pull off.

I’ve a sense that portions of the audience are beginning to wriggle uncomfortably in their seats: doesn’t this blog generally cover current affairs from a moderately egg-headed philosophical/social-scientific perspective? Why is this guy wittering on about elderly texts about the management of software projects?

Don’t start chucking popcorn yet, folks; I may have some problems with the egg-headed part of the brief, but I aim eventually to pull this around to some stuff that seems a bit closer to the central topics Crooked Timber usually covers.

The main thing that TMMM is famous for pointing out is that you can’t get a project which is behind its schedule back on track by adding more code-monkeys to it: Brooks offers as a oversimplification the slogan that ‘adding manpower to a late software project makes it later’. Why would that be?

Well first up, some tasks are inherently not susceptible to being partitioned into subtasks, so adding extra people to do them avails you precisely nothing: as Brooks memorably puts it, ‘the bearing of a child takes nine months, no matter how many women are assigned’.

More fundamentally, where a task can be partitioned, one has to take into account the communications overhead which comes from adding people. In general, the people working on a given subtask will have to communicate with oneanother and also with those working on the other subtasks. This means that as the number of developers grows linearly, the number of communication paths grows exponentially.

And systems of any size cannot be absorbed by new monkeys straight away: they need to be inducted into the code’s strangenesses and mysteries (which it undoubtedly has, since the project is presumably late for a reason) by someone who knows about them. Assigning such a guru to training the new chimps has an obvious opportunity cost.

Brooks puts it like this: ‘since software construction is inherently a systems effort – an exercise in complex interrelationships – communication effort is great, and it quickly dominates the decrease in individual task times brought about by partitioning’.

By emphasizing the importance of communication as a necessary feature and overhead cost of software development, Brooks nudged people towards recognising that software gets built by people, too. When we manage to do away with the picture of systems being built by interchangeable typing monkeys, we might begin to think about running projects in a fashion that reflects a different picture.

Of course, anyone who has done software-related stuff professionally (or who, indeed, has worked in any other project-based environment) knows that Brooks may as well have never bothered writing his famous essay. Managers still try to save projects by adding people, and seem perpetually surprised when it doesn’t work. More centrally, they continue to pray for a future in which employees really are entirely fungible ‘plug’n’play’ components who can be heaved about the place according to whim. And then sometimes they behave as if that future is already here.

So by the late ‘eighties, we have the opportunity to read books like Tom de Marco and Timothy Lister’s PeopleWare, which tries to explain why some different approaches need to be tried. The really wonderful thing about PeopleWare is that it hammers home the point that projects are staffed by human beings, who (shock) aren’t interchangeable, (horror) have lives, and (shame) would really prefer to hang on to them. It does so by drawing on stacks of empirical evidence about projects that succeed, projects that fail, the circumstances under which they do each, and so on. (There’s also a fascinating chapter on the lessons that the work of Christoper Alexander might have for the physical configuration of the modern work place en route.)

De Marco and Lister spend a lot of time explaining why teams which are treated decently outperform those which aren’t (and hence why the hardest-headed bean-counter ought to pay attention to their findings), but there’s also a real sense of anger at the sheer human waste which is generated by the many shoddily run projects which end up eating up the lives of those who are unfortunate enough to find themselves assigned to them. (There’s a good blast of that anger here, in a letter DeMarco wrote about so-called ‘Death-March’ projects.)

To pull things back towards Crooked Timber territory, what I most admire about PeopleWare is the way that DeMarco and Lister are reacting against a sort of Rand Corporation-inspired, Econ 101, and above all dehumanised conception of the working world. He understands amongst other things that (most) people care about their work, and want to do it well; that people are demotivated by foolish processes dictated from above whose only possible rationale is that those who are required to follow them cannot be trusted; and that work cannot replace the many other things that make for a decent human life.

I certainly have no idea about the state of the academic literature on this kind of thing, but I find it encouraging that it’s not too hard to find serious software methodologists beginning to look into the seriously into the contribution that sociological and ethnographic research strategies might make to their efforts to work out how to design projects. This paper, for instance, makes me feel optimistic that one day, the human beings may win out in the software industry.

Now, I’ve been drawing on some stuff that aims to solve a very particular problem: how the hell to run programming projects on time, to budget, with the right feature-set, and without driving anybody to a nervous breakdown. But the broader lessons about the work-place seem pretty obvious: businesses are run by people, who can only do their jobs well if they have some kind of internal motivation to do them at all, and who will do them poorly if they are not trusted to think for themselves. The pleasures of craft, and the autonomy allowed to professionals, are things that most of us need in order to be productive workers. We also need them to be happy human beings.

Perhaps someone should send a copy of PeopleWare to the senior managers at British Airways, who clearly don’t understand any of the above.

{ 9 comments }

1

tew 07.30.03 at 12:14 am

Sounds like you’re about two hours of websurfing from coming across Extreme Programming, the latest incarnation of this phenomenon.

2

Tom Runnacles 07.30.03 at 12:36 am

XP? Got me bang to rights, guv. Actually I’m reading Kent Beck’s original book on the subject right now.

I’m in favour: test first, pair programming, refactor lots, all fine with me.

In fact, I’ve spent quite a bit of time working on projects which half-heartedly follow XP practices, but I must admit I’ve never experienced the whole shebang implemented at once.

I doubt it’s a silver bullet, but some well-intentioned and extremely smart people have good things to say about it. If XP helps developers stay in control of their code (and their lives), all the better.

3

Russell L. Carter 07.30.03 at 1:52 am

“I’m in favour: test first, pair programming, refactor lots, all fine with me.”

I’ve always thought that XP (well just the pair programming) is just another way of keeping the worker-bees on the job. Continuous peer-review, and all that. But the insidious thing is that it is voluntary, and that one just ‘slings code’; there’s no way to build in the fact that I don’t want to sit there and watch you think. So you won’t. I’m watching you… tapping my fingers…

Collaboration is at the essence of creativity to be sure, but almost all successful collaborations are self-selected.

I recognize that there is a certain irony to posting this complaint to a blog entry.

4

Shai 07.30.03 at 2:13 am

Extreme Programming has some nice ideas (pair programming), but it also has some faddish elements that usually appeal to people who have never taken a course in software engineering or managing software development. XP is only one process model among many.

see for example,
CMU Software Engineering Institute

They used to have the lectures of their graduate course in managing software development available for free in video format, but unfortunately the page seems to be gone now. There are lecture notes everwhere at other universities for other courses though (google).

Fred Brooks 2000 Turing lecture titled “The Design of Design” is here. If anyone wants to see the full video in a larger resolution I do have a link, but I won’t post it here because I’m afraid it will be flooded off the internet (email me if interested).

5

Shai 07.30.03 at 3:24 am

Actually, the video on the navy.mil site is 647MB mpeg1 in addition to being a lower resolution to the other link which is a 135MB mpeg4 wmv. So again, email if interested. I’m watching it again right now… pretty interesting.

6

Tripp 07.30.03 at 5:02 pm

I’ve been thinking of “The Mythical Man Month” as it relates to the latest trend to send Software project overseas to China and India. Apparently, programmers are paid 1/12th what they are in the US. I can just imagine some manager thinking “Wow, I can get the job done in 1/12th the time,” or any other combination of cost/time.

I make the following predictions:

1. The trend is inevitable, and will continue.
2. Savings will be less than expected.
3. US programmers will be relegated to the software services industry.
4. You’ll have extremely cheap software, but you’ll have to hire a service person to install and configure it.

7

Scott Martens 07.30.03 at 5:44 pm

Tripp, I’ve been expecting a software model of that type for some time too. Shrink-wrap software is being replaced by a more modular vision of software, where both code and computer services come as a package. This sort of deal is more labour intensive, but the services side is more and more the work of less espensive computer people instead of elite CS grads. It also means that software is more responsive to actual customer needs. Even though it’s more labour intensive, it seems that this sort of model can lower total cost of use and increase value added enough to justify it.

8

Lurker 07.31.03 at 10:21 pm

XP practices have greater applicability to in-house software projects and to a lesser degree, works for hire. How many customers do you know that want to be caged with software developers for months on in? Watching them program? Not many me thinks.

As for as embedded systems, e.g. cell phones, TV’s, video games, appliances, and all those other invisible computers, it has little or no practicality. The design for test piece is useful, but that discipline exists independent of XP.

For your further software adventures, I recommend http://www.joelonsoftware.com. He has an archive of really interesting articles, and he writes in a way that, though simplified for we human language impaired engineers, you liberal arts folks can appreciate. Be sure to check out the philosophy of how he hires and ranks his employees. It’s an interesting rubber-meets-the road study of someone who recognizes the absurdities of the whole software development enterprise, and his attempts to make it more humane.

9

JD 08.05.03 at 3:06 am

This means that as the number of developers grows linearly, the number of communication paths grows exponentially.

Please, please do not abuse the word “exponentially” in this fashion. “Exponentially” may be the most pseudoscientifically abused word in the English language—people just use it as a puffed-up synonym for “really really fast” when they want to blind their listeners with “science”.

The number of communication paths in a software project grows quadratically, i.e. as the square of the number of developers (unless you assume that communication paths are between the elements of the power set of all developers—which neither Brooks, nor anybody else I’m aware of, has proposed.).

Comments on this entry are closed.