Actually they finally gave up after 10 years, ripped out the automatic system, and replaced it with a standard manual system augmented with modern labeling/sorting technology (which of course had improved over those 10 years). But your point is still a good one.

Cranky

The reason is that the world is just not a static place, even (especially) in the civil service. Not only are there lots of stakeholders, their interests keep changing. Plus none of them can really know what they want until they see it.

Degree of difficulty rises disproportionately with project size, so I’ve learned the hard way that IT projects should be as small and modest as possible; never try to do too much. Also treat all tenderers as filthy liars who’ll promise the world until they get you locked into them, upon which they will proceed to merrily screw you for all the Treasury is worth.

Part of it is due to impedence mismatching between the “specs.”
The paperwork says that work is done one way, (mis)managers think that work is done some other way, and the workerbees have a thirdway where work actually gets done.

And private industry isn’t any better at getting IT to work. Just better at hiding the dead bodies and spinning what can’t be shoved under the carpet. For a wonderful example, I refer one to the Denver Airport baggage handling system. The engineering management said “projects of this size take 4 years to complete,” while the sales mismanagement said “airport is finished in 2 years, so we sold it to them, you have 2 years to make it work.” As everyone knows, the baggage handling system was functional (meaning: it stopped losing and shredding bags) 2 years after the airport opened. Was it late or on-time? Does it matter when the business ends up going out of business? I’m currently involved in a similar fiasco. An internal project to replace a currently selling software product (that cannot possibly run on Vista) is 3 years behind schedule. The mismanagers are now claiming that this project is only 2 years old and therefore can’t possibly be 3 years late. My estimate is that there is about 6-9 more calendar months of work left before it is sellable, but the PHBs all say it must ship next month.

I’m coming to the conclusion that the reason that outsourcing/consulting is becoming (or is already?) a form of political patronage and graft. I’m running for elected office this November, and the more I see of the contracting/outsourcing of governmental functions, the more I smell graft.

A more reasoned answer is that large IT projects usually fail for sociological and design related reasons, rather than engineering ones. If you’re designing a computer system that will be used by an organisation, then you need to understand how work is done within that organisation (which rarely has much to do with org charts, or the fantasies of the business managers commissioning the project), and have a strong sense of how the IT project is going to augment/improve/change the functioning of that organisation. The skills required are a mixture of design and sociology ones, with a smattering of psychology. Not the kind of skills that business, or IT, grads typically have, even though in the UK they are the ones typically doing this kind of work.

Somebody mentioned Denmark as an example of somewhere that manages this kind of stuff. Their health systems were designed by HCI specialists, and seem to work quite well. In contrast for the NHS, doctors and nurses were excluded from the design process and we wonder why it didn’t meet their needs very effectively…

Abb1, the thing about ITIL is that it was invented by the government’s IT specialisation when the government had an IT specialisation. Since then, they sacked all the geeks and outsourced to EDS.

Secondly, and more generally, Redwood doesn’t seem to realise that there is a way of having government IT requirements met in a more incremental and human-scale way whilst not using as many consultants – but it’s called “doing it in house”, and it requires that all the organisational entities who might want their own small problems solving have their own engineers. This means re-establishing a huge government-wide IT career path, a public sector IBM. Is this really what he wants? (Mind, I’m quite keen on it myself.)

My guess with CREST is that there was a degree of atomisation in the abstract conceptualisation of the system—getting stuff from A to B without fuckups—that dovetailed pretty well with the basic skillset of the people assigned to code it all up. There’s an algorithmic purity to that kind of thing. I’d agree with those upthread that taking a UNIXy approach—dividing projects into small, self-contained, discrete functions—is optimal, though that still raises the problem of interoperability at the most basic level.

(Still, that also reminds me of a conversation ten years ago with a friend at a Large Consultancy Firm who’d been flown out to San Francisco and wanted a primer on the whole dot-com thing.)

My job was divided into getting the project completed and producing convincing but entirely fictional MS Project charts to send upstairs in order to keep the MBAs happy.

I wasn’t ‘experienced in project management’ for that particular gig: I was a) sufficiently experienced to know how projects came together in practice; b) a good enough bullshitter to keep the suits happy. I also had a passing knowledge of The Mythical Man-Month, which always helps.

Some years later, I had an informal interview for another IT-related gig with A Large Financial Institution, and demurred because after an hour of abstract explanation of the position from the guy who would have been my boss, I didn’t have a fucking clue what they actually wanted doing.

I have no idea whether Crapita represents primarily the suits upstairs or not, but I do know that Michael Gove—to pick on him, not just because of the CiF piece, but because he’s a weaselly little fucker—has lots of friends who got milkroaded into consultancy, and will be only too pleased to assist him in proposing projects that will, at the best of times, be managed at the ground level by people who a) may not have a fucking clue what they’re meant to be delivering; b) will cook the books to keep the suits upstairs happy. Let me repeat: that’s at the best of times.

The fundamental basis of this analysis is that the waterfall model of systems development, revolving around the fully-defined, correct, and complete “spec”, is valid. I am not fully convinced by the agile/extreme camps yet but it is well known that the waterfall model hasn’t done very well over the last 20 years. The response so far has been the PMI, CMMI, and attempts to ratchet the process and the spec down tighter and tighter until nirvana is reached and I have to say that doesn’t seem to be going very well either.

Cranky

Further, this is the point; presumably Redwood imagines that Government IT requirements will be matched by the decentralised efforts of local techies. Unfortunately, this would require rebuilding a major IT specialisation all over the entire civil service and beyond.

Or does he just want to smash things?