This is a book with some important, even profound, ideas about politics,institutions, the virtues of democracy and what it takes to realize them, but it is written so so very, very diffusely that it will will have next to no impact, which is a shame. Let me try to lay out the main path of argument, which is rather lost amid the authors’ digressions and verbiage.

We live in big, complex societies, which means we are thoroughly interdependent on each other, and that we will naturally have different ideas about how our life in common should go, and will have divergent interests. This means that politics we shall always have with us. It also means that political problems are largely ones about designing and reforming the institutions which shape how we interact with each other. But because political problems are so hard, even if we could agree on what we wanted our institutions to achieve (which we don’t), we can basically never know in advance what the best institution for a given problem is. (That markets should always and everywhere be the default institution is a claim Knight and Johnson carefully examine before rejecting, whereas I would simply mock.) We also can basically never be sure when changed conditions will make existing institutions unsatisfactory. Put this together and what we need is, as they say, experimentation, with meta-institutions for monitoring how the experiments are going, and deciding when they should be changed or stopped.

This is where democracy comes in. It has priority, not as a first-order institution for getting everything done, but as a second-order institution for checking on and revising other institutions. No other organizational form is as well-suited to checking whether an institution is actively working; some (e.g., markets and courts) are positively pessimized for monitoring their own performance. Democracy has, importantly, two crucial parts: voting, or some similar means of aggregating choices, and debate, the arguments which come before and continue after every vote. Democratic voting is a way of making choices. Democratic debate is a tool for cognition, for harnessing the dispersed knowledge of the citizens and their diversity of perspectives and insights. (There are appropriate citations here
to Ober and Page, among others, though not, oddly, of Lindblom[1].) The two together are a centralized competitive mechanism for discovering what to do, and for revisiting those decisions in the light of experience. Democracy does not produce agreement, and doesn’t need to. Instead it gives us, for the most part, a shared understanding of what we disagree about.

This sort of competition is going to work better as we expand the pool of people who can contribute to it. This means not just having a vote, but being able to cast it as their reason, conscience and interests tell them, not according to the threats of their boss, preacher or family. But because debate is crucial to democracy, people also need to be able to participate in that debate effectively: they need to be able to read (or hear or see…) the debate, and write their own contributions. Others must not ignore or, worse, silence them just because of the kind of people they are (men, women, blacks, Jews, Muslims, Baptist, janitors, engineers, artists…), though they might ignore them if they don’t make sense. So Knight and Johnson recover the usual civil liberties — freedom of speech and of the press, universal suffrage through secret ballots, etc. — as conditions for making democracy work.

But remember that democracy is going to work better the more people can and do really (“effectively”) contribute, especially to the debate. To use an analogy Knight and Johnson don’t: in a sense I could run a marathon, since it is not legally or even physically impossible for me to go through those motions. But there are a lot of obstacles in my way: I have a sedentary job which I let consume a lot of my time, and I have a 38-year-long history of not being very athletic and all the habits and physical drawbacks which go along with that. If I really, really wanted to run a marathon, I am young enough that years of dedication would probably get me there, but it would be very hard, and would get easier if I had a very different environment; it would be easier still if I could have had that more favorable environment for a long time already.

To participate in the democratic debate, people need a lot of skills and cognitive tools: literacy; numeracy; knowing what other people are going on about and why it matters to them; the cultural knowledge and rhetorical skill to argue effectively with fellow citizens[2]; knowledge of the world in general. Gaining all these skills and tools takes teachers and time. (Some people learn such things under extraordinarily bad conditions; expecting everyone to do so is like expecting every middle-aged office worker to become a marathon-runner.) Gaining these skills also takes a brain which is not too damaged by malnutrition, lead poisoning, chronic stress, etc., or simply too inflexible with age. Even citizens who have these skills need free time, not taken up by getting a living, if they are to use them. Economic barriers matter too: if the cost of making oneself heard is owning a TV station, effectively we’ve limited debate to the friends and servants of TV-station-owners. But since democracy works better the more minds it can draw on, and the more diverse they are, that is not apt to be a good situation even for station-owners.

Making sure that everyone has an equal opportunity to participate in democracy would be very demanding, and we are very far from doing so. We are even far from making sure everyone has some non-farcical minimum of opportunity. We can and should move towards spreading those opportunities, and make democracy more of a reality and less of a mere promise.

This, then, is the main path of thought in The Priority of Democracy; I find it extremely attractive. I have left out a lot of detail, and a lot of side-roads, such as the discussion of institutional economics (drawing on Knight’s earlier book), and everything about philosophical Pragmatism. It obviously matters a great deal to Knight and Johnson to see themselves as heirs to Dewey, and what they have to say about Pragmatism is interesting to me, but I don’t see how it makes any difference. Someone who thought the whole Pragmatist tradition was rubbish from beginning to end could still accept all of Knight and Johnson’s substantive arguments about politics, institutions, and experimental democracy. (They are after all very close to Karl Popper’s ideas in The Open Society and Its Enemies; naturally, Popper is not mentioned.) On top of this, they spill a lot of ink in disputes with fellow political theorists and/or capital-P Pragmatists, and generally using five words instead of two.

None of this takes away from the value of what they’ve done. But all of this makes it harder to get to that value, and reduces the impact the book will have. (If I pressed it on friends or students, almost none of them would actually read it.) There is a good short, pointed book in here; someone should write it.

[1]: I mean not Lindblom’s later books on the market system, which they do cite, but his earlier ones on democracy as decision-making through “partisan mutual adjustment”, especially A Strategy of Decision (with David Braybrooke, 1963), and The Intelligence of Democracy (1965). These emphasized the impossibility of designing “optimal” solutions from scratch for actual social or political problems, which are instead dealt with by making multiple steps, in many senses “partial”, and often correcting the results of earlier steps taken under the direction of other aims or other understandings. In doing this, persistent disagreement about values and priorities can be not just inevitable but a positive boon. I will venture to quote a somewhat long passage from The Intelligence of Democracy:

Looking at complex problem solving as a strategy, one can wonder whether serial and remedial methods are enough to provide reasonable assurance that adverse consequences and policy failures will be straightened out. One can easily imagine a decision maker who can return in later policy steps to no more than a few of a variety of neglected adverse consequences resulting from an earlier policy step. This possibility points directly to the need for a multiplicity of decision makers and, more than that, to a multiplicity marked by great variety of attitudes and interests, so that no one line of adverse consequence fails to come the attention of some decision maker.

The great multiplicity of decision makers in, say, American public policy making can be seen, therefore, as a great strength where problem solving cannot be synoptically accomplished but must be strategically pursued. Multiplicity copes with the inevitability of omission and other errors in complex problem solving. Were there no decision makers with a stake in international trade, we might wonder whether farm policy might not put strains on international trade to which farm-policy decision makers might themselves be inadequately sensitive; but we know that, if the strains appear, those decision makers who have a stake in international trade will attack them as their own problem. Were it not for decision makers with an interest in parks and recreation, we might wonder whether an urban redevelopment board could be trusted to make decisions on the relocation of commercial houses within a city.

If, through multiplicity, decision makers mop up the adverse consequences of each other’s inevitably imperfect decisions, multiple decision makers will, in addition, compelling call to others’ attention aspects of the problem they cannot themselves analyze. Moreover, just as the single decision maker will sometimes anticipate adverse consequences that he must nevertheless treat as separate problems for fear of making his existing problems unmanageable, so also other decision makers can anticipate what they either cannot anticipate or cannot attend to. They can then treat the anticipated adverse consequences as their problems, even attacking them simultaneously with their attack on the initial problem. [pp. 151—152]

Lindblom was, to be fair, rather complacent about how well America was doing at making sure diverse values had a say in the political process in the mid-1960s, but that’s another story. ^

[2]: It’s easy, on this basis, to argue for compulsory education strongly focused on assimilating all children to a single common culture. In other words, put this aspect of Knight and Johnson together with Ernest Gellner, and you easily get not multiculturalism
but E. D. Hirsch. There’s even a formally parallel argument for eugenics, but it’d be much weaker. (It’d be far more intrusive than melting-pot schooling, there’s no diversity-preserving equivalent of bilingualism or code-switching, and, most of all, we have no idea of how to do it.) ^

Attention conservation notice: Over 7800 words about optimal planning for a socialist economy and its intersection with computational complexity theory. This is about as relevant to the world around us as debating whether a devotee of the Olympian gods should approve of transgenic organisms. (Or: centaurs, yes or no?) Contains mathematical symbols (uglified and rendered slightly inexact by HTML) but no actual math, and uses Red Plenty mostly as a launching point for a tangent.

There’s lots to say about Red Plenty as a work of literature; I won’t do so. It’s basically a work of speculative fiction, where one of the primary pleasures is having a strange world unfold in the reader’s mind. More than that, it’s a work of science fiction, where the strangeness of the world comes from its being reshaped by technology and scientific ideas—- here, mathematical and economic ideas.

Red Plenty is also (what is a rather different thing) a work of scientist fiction, about the creative travails of scientists. The early chapter, where linear programming breaks in upon the Kantorovich character, is one of the most true-to-life depictions I’ve encountered of the experiences of mathematical inspiration and mathematical work. (Nothing I will ever do will be remotely as important or beautiful as what the real Kantorovich did, of course.) An essential part of that chapter, though, is the way the thoughts of the Kantorovich character split between his profound idea, his idealistic political musings, and his scheming about how to cadge some shoes, all blind to the incongruities and ironies.

It should be clear by this point that I loved Red Plenty as a book, but I am so much in its target demographic¹ that it’s not even funny. My enthusing about it further would not therefore help others, so I will, to make better use of our limited time, talk instead about the central idea, the dream of the optimal planned economy.

That dream did not come true, but it never even came close to being implemented; strong forces blocked that, forces which Red Plenty describes vividly. But could it even have been tried? Should it have been?

“The Basic Problem of Industrial Planning”

Let’s think about what would have to have gone in to planning in the manner of Kantorovich.

I. We need a quantity to maximize. This objective function has to be a function of the quantities of all the different goods (and services) produced by our economic system.

Here “objective” is used in the sense of “goal”, not in the sense of “factual”. In Kantorovich’s world, the objective function is linear, just a weighted sum of the output levels. Those weights tell us about trade-offs: we will accept getting one less bed-sheet (queen-size, cotton, light blue, thin, fine-weave) if it lets us make so many more diapers (cloth, unbleached, re-usable), or this many more lab coats (men’s, size XL, non-flame-retardant),or for that matter such-and-such an extra quantity of toothpaste. In other words, we need to begin our planning exercise with relative weights. If you don’t want to call these “values” or “prices”, I won’t insist, but the planning exercise has to begin with them, because they’re what the function being optimized is built from.

It’s worth remarking that in Best Use of Economic Resources, Kantorovich side-stepped this problem by a device which has “all the advantages of theft over honest toil”. Namely, he posed only the problem of maximizing the production of a “given assortment” of goods—- the planners have fixed on a ratio of sheets to diapers (and everything else) to be produced, and want the most that can be coaxed out of the inputs while keeping those ratios. This doesn’t really remove the difficulty: either the planners have to decide on relative values, or they have to decide on the ratios in the “given assortment”.

Equivalently, the planners could fix the desired output, and try to minimize the resources required. Then, again, they must fix relative weights for resources (cotton fiber, blue dye #1, blue dye #2, bleach, water [potable],water [distilled], time on machine #1, time on machine #2, labor time [unskilled], labor time [skilled, sewing], electric power…). In some contexts these might be physically comparable units. (The first linear programming problem I was ever posed was to work out a diet which will give astronauts all the nutrients they need from a minimum mass of food.) In a market system these would be relative prices of factors of production. Maintaining a “given assortment” (fixed proportions) of resources used seems even less reasonable than maintaining a “given assortment” of outputs, but I suppose we could do it.

For now (I’ll come back to this), assume the objective function is given somehow, and is not to be argued with.

IIA. We need complete and accurate knowledge of all the physical constraints on the economy, the resources available to it.

IIB. We need complete and accurate knowledge of the productive capacities of the economy, the ways in which it can convert inputs to outputs.

(IIA) and (IIB) require us to disaggregate all the goods (and services) of the economy to the point where everything inside each category is substitutable. Moreover, if different parts of our physical or organizational “plant” have different technical capacities, that needs to be taken into account, or the results can be decidedly sub-optimal. (Kantorovich actually emphasizes this need to disaggregate in Best Use, by way of scoring points against Leontief. The numbers in the latter’s input-output matrices, Kantorovich says, are aggregated over huge swathes of the economy, and so far too crude to be actually useful for planning.) This is, to belabor the obvious, a huge amount of information to gather.

(It’s worth remarking at this point that “inputs” and “constraints” can be understood very broadly. For instance, there is nothing in the formalism which keeps it from including constraints on how much the production process is allowed to pollute the environment. The shadow prices enforcing those constraints would indicate how much production could be increased if marginally more pollution were allowed. This wasn’t, so far as I know, a concern of the Soviet economists, but it’s the logic behind cap-and-trade institutions for controlling pollution.)

Subsequent work in optimization theory lets us get away, a bit, from requiring complete and perfectly accurate knowledge in stage (II). If our knowledge is distorted by merely unbiased statistical error, we could settle for stochastic optimization, which runs some risk of being badly wrong (if the noise is large), but at least does well on average. We still need this unbiased knowledge about everything, however, and aggregation is still a recipe for distortions.

More serious is the problem that people will straight-up lie to the planners about resources and technical capacities, for reasons which Spufford dramatizes nicely. There is no good mathematical way of dealing with this.

III. For Kantorovich, the objective function from (I) and the constraints and production technology from (II) must be linear.

Nonlinear optimization is possible, and I will come back to it, but it rarely makes things easier.

IV. Computing time must be not just too cheap to meter, but genuinely immense.

It is this point which I want to elaborate on, because it is a mathematical rather than a practical difficulty.

“Numerical Methods for the Solution of Problems of Optimal Planning”

It was no accident that mathematical optimization went hand-in-hand with automated computing. There’s little point to reasoning abstractly about optima if you can’t actually find them, and finding an optimum is a computational task. We pose a problem (find the plan which maximizes this objective function subject to these constraints), and want not just a solution, but a method which will continue to deliver solutions even as the problem posed is varied. We need an algorithm.

Computer science, which is not really so much a science as a branch of mathematical engineering, studies questions like this. A huge and profoundly important division of computer science, the theory of computational complexity, concerns itself with understanding what resources algorithms require to work. Those resources may take many forms: memory to store intermediate results, samples for statistical problems, communication between cooperative problem-solvers. The most basic resource is time, measured not in seconds but in operations of the computer. This is something Spufford dances around, in II.2: “Here’s the power of the machine: that having broken arithmetic down into tiny idiot steps, it can then execute those steps at inhuman speed, forever.” But how many steps? If it needs enough steps, then even inhuman speed is useless for human purposes…

The way computational complexity theory works is that it establishes some reasonable measure of the size of an instance of a problem, and then asks how much time is absolutely required to produce a solution. There can be several aspects of “size”; there are three natural ones for linear programming problems. One is the number of variables being optimized over, say n . The second is the number of constraints on the optimization, say m . The third is the amount of approximation we are willing to tolerate in a solution—- we demand that it come within h of the optimum, and that if any constraints are violated it is also by no more than h . Presumably optimizing many variables ( n >> 1), subject to many constraints ( m >> 1 ), to a high degree of approximation ( h ~ 0), is going to take more time than optimizing a few variables ( n ~1), with a handful of constraints ( m ~ 1 ), and accepting a lot of slop ( h ~ 1). How much, exactly?

The fastest known algorithms for solving linear programming problems are what are called “interior point” methods. These are extremely ingenious pieces of engineering, useful not just for linear programming but a wider class of problems called “convex programming”. Since the 1980s they have revolutionized numerical optimization, and are, not so coincidentally, among the intellectual children of Kantorovich (and Dantzig). The best guarantees about the number of “idiot steps” (arithmetic operations) they need to solve a linear programming problem with such algorithms is that it’s proportional to

(I am simplifying just a bit; see sec. 4.6.1 of Ben-Tal and Nemirovski’s Lectures on Modern Convex Optimization [PDF].)

Truly intractable optimization problems — of which there are many — are ones where the number of steps needed grow exponentially². If linear programming was in this “complexity class”, it would be truly dire news, but it’s not. The complexity of the calculation grows only polynomially with n , so it falls in the class theorists are accustomed to regarding as “tractable”. But the complexity still grows super-linearly, like n^3.5. Where does this leave us?

A good modern commercial linear programming package can handle a problem with 12 or 13 million variables in a few minutes on a desktop machine. Let’s be generous and push this down to 1 second. (Or let’s hope that Moore’s Law rule-of-thumb has six or eight iterations left,and wait a decade.) To handle a problem with 12 or 13 billion variables then would take about 30 billion seconds, or roughly a thousand years.

Naturally, I have a reason for mentioning 12 million variables:

In the USSR at this time [1983] there are 12 million identifiably different products (disaggregated down to specific types of ball-bearings, designs of cloth, size of brown shoes, and so on). There are close to 50,000 industrial establishments, plus, of course, thousands of construction enterprises, transport undertakings, collective and state forms, wholesaling organs and retail outlets.
—Alec Nove, The Economics of Feasible Socialism (p. 36 of the revised [1991] edition; Nove’s italics)

This 12 million figure will conceal variations in quality; and it is not clear to me, even after tracking down Nove’s sources, whether it included the provision of services, which are a necessary part of any economy.

Let’s say it’s just twelve million. Even if the USSR could never have invented a modern computer running a good LP solver, if someone had given it one, couldn’t Gosplan have done its work in a matter of minutes? Maybe an hour, to look at some alternative plans?

No. The difficulty is that there aren’t merely 12 million variables to optimize over, but rather many more. We need to distinguish between a “coat, winter, men’s, part-silk lining, wool worsted tricot, clothgroup 29—32” in Smolensk from one in Moscow. If we don’t “index” physical goods by location this way, our plan won’t account for the need for transport properly, and things simply won’t be where they’re needed; Kantorovich said as much under the heading of “the problem of a production complex”. (Goods which can spoil, or are needed at particular occasions and neither earlier nor later, should also be indexed by time; Kantorovich’s “dynamic problem”) A thousand locations would be very conservative, but even that factor would get us into the regime where it would take us a thousand years to work through a single plan. With 12 million kinds of goods and only a thousand locations, to have the plan ready in less than a year would need computers a thousand times faster.

This is not altogether unanticipated by Red Plenty:

A beautiful paper at the end of last year had skewered Academician Glushkov’s hypercentralized rival scheme for an all-seeing, all-knowing computer which would rule the physical economy directly, with no need for money. The author had simply calculated how long it would take the best machine presently available to execute the needful program, if the Soviet economy were taken tobe a system of equations with fifty million variables and five million constraints. Round about a hundred million years, was the answer. Beautiful.So the only game in town, now, was their own civilised, decentralized idea for optimal pricing, in which shadow prices calculated from opportunity costs would harmonise the plan without anyone needing to possess impossibly complete information. [V.2]

This alternative vision, the one which Spufford depicts those around Kantorovich as pushing, was to find the shadow prices needed to optimize, fix the monetary prices to track the shadow prices, and then let individuals or firms buy and sell as they wish, so long as they are within their budgets and adhere to those prices. The planners needn’t govern men, nor even administer things, but only set prices. Does this, however, actually set the planners a more tractable, a less computationally-complex, problem?

So far as our current knowledge goes, no. Computing optimal prices turns out to have the same complexity as computing the optimal plan itself ³. It is(so far as I know) conceivable that there is some short-cut to computing prices alone, but we have no tractable way of doing that yet. Anyone who wants to advocate this needs to show that it is possible, not just hope piously.

How then might we escape?

It will not do to say that it’s enough for the planners to approximate the optimal plan, with some dark asides about the imperfections of actually-existing capitalism thrown into the mix. The computational complexity formula I quoted above already allows for only needing to come close to the optimum. Worse, the complexity depends only very slowly, logarithmically, on the approximation to the optimum, so accepting a bit more slop buys us only a very slight savings in computation time. (The optimistic spin is that if we can do the calculations at all, we can come quite close to the optimum.) This route is blocked.

Another route would use the idea that the formula I’ve quoted is only an upper bound, the time required to solve an arbitrary linear programming problem. The problems set by economic planning might, however, have some special structure which could be exploited to find solutions faster. What might that structure be?

The most plausible candidate is to look for problems which are “separable”, where the constraints create very few connections among the variables. If we could divide the variables into two sets which had nothing at all to do with each other, then we could solve each sub-problem separately, at tremendous savings in time. The supra-linear, n^3.5 scaling would apply only within each sub-problem. We could get the optimal prices (or optimal plans) just by concatenating the solutions to sub-problems, with no extra work on our part.

Unfortunately, as Lenin is supposed to have said, “everything is connected to everything else”. If nothing else, labor is both required for all production, and is in finite supply, creating coupling between all spheres of the economy. (Labor is not actually extra special here, but it is traditional⁴.) A national economy simply does not break up into so many separate, non-communicating spheres which could be optimized independently.

So long as we are thinking like computer programmers, however, we might try a desperately crude hack, and just ignore all kinds of interdependencies between variables. If we did that, if we pretended that the over-all high-dimensional economic planning problem could be split into many separate low-dimensional problems, then we could speed things up immensely, by exploiting parallelism or distributed processing. An actually-existing algorithm, on actually-existing hardware, could solve each problem on its own, ignoring the effect on the others, in a reasonable amount of time. As computing power grows, the supra-linear complexity of each planning sub-problem becomes less of an issue, and so we could be less aggressive in ignoring couplings.

At this point, each processor is something very much like a firm, with a scope dictated by information-processing power, and the mis-matches introduced by their ignoring each other in their own optimization is something very much like “the anarchy of the market”. I qualify with “very much like”, because there are probably lots of institutional forms these could take, some of which will not look much like actually existing capitalism. (At the very least the firm-ish entities could be publicly owned, by the state, Roemeresque stock-market socialism, workers’cooperatives, or indeed other forms.)

Forcing each processor to take some account of what the others are doing, through prices and quantities in markets, removes some of the grosser pathologies. (If you’re a physicist, you think of this as weak coupling; ifyou’re a computer programmer, it’s a restricted interface.) But it won’t, in general, provide enough of a communication channel to actually compute the prices swiftly — at least not if we want one set of prices, available to all. Rob Axtell, in a really remarkable paper, shows that bilateral exchange can come within h of an equilibrium set of prices in a time proportional to n²log(1/h), which is much faster than any known centralized scheme.

Now, we might hope that yet faster algorithms will be found, ones which would, say, push the complexity down from cubic in n to merely linear. There are lower bounds on the complexity of optimization problems which suggest we could never hope to push it below that. No such algorithms are known to exist, and we don’t have any good reason to think that they do. We also have no reason to think that alternative computing methods would lead to such a speed-up⁵.

I said before that increasing the number of variables by a factor of 1000 increases the time needed by a factor of about 30 billion. To cancel this out would need a computer about 30 billion times faster, which would need about 35 doublings of computing speed, taking, if Moore’s rule-of-thumb continues to hold, another half century. But my factor of 1000 for prices was quite arbitrary; if it’s really more like a million, then we’re talking about increasing the computation by a factor of 10²¹ (a more-than-astronomical, rather a chemical, increase), which is just under 70 doublings, or just over a century of Moore’s Law.

If someone like Iain Banks or Ken MacLeod wants to write a novel where they say that the optimal planned economy will become technically tractable sometime around the early 22nd century, then I will read it eagerly. As a serious piece of prognostication, however, this is the kind of thinking which leads to”where’s my jet-pack?” ranting on the part of geeks of a certain age.

Nonlinearity and Nonconvexity

In linear programming, all the constraints facing the planner, including those representing the available technologies of production, are linear. Economically, this means constant returns to scale: the factory need put no more, and no less, resources into its 10,000th pair of diapers as into its 20,000th, or its first.

Mathematically, the linear constraints on production are a special case of convex constraints. If a constraint is convex, then if we have two plans which satisfy it, so would any intermediate plan in between those extremes. (If plan A calls for 10,000 diapers and 2,000 towels, and plan B calls for 2,000 diapers and 10,000 towels, we could do half of plan A and half of plan B, make 6,000 diapers and 6,000 towels, and not run up against the constraints.) Not all convex constraints are linear; in convex programming, we relax linear programming to just require convex constraints. Economically, this corresponds to allowing decreasing returns to scale, where the 10,000 pair of diapers is indeed more expensive than the 9,999th, or the first.

Computationally, it turns out that the same “interior-point” algorithms which bring large linear-programming problems within reach also work on general convex programming problems. Convex programming is more computationally complex than linear programming, but not radically so.

Unfortunately for the planners, increasing returns to scale in production mean non-convex constraints; and increasing returns are very common, if only from fixed costs. If the plan calls for regular flights from Moscow to Novosibirsk, each flight has a fixed minimum cost, no matter how much or how little the plane carries. (Fuel; the labor of pilots, mechanics, and air-traffic controllers; wear and tear on the plane; wear and tear on runways; the lost opportunity of using the plane for something else.) Similarly for optimization software (you can’t make any copies of the program without first expending the programmers’ labor, and the computer time they need to write and debug the code). Or academic papers, or for that matter running an assembly line or a steel mill. In all of these cases, you just can’t smoothly interpolate between plans which have these outputs and ones which don’t. You must pay at least the fixed cost to get any output at all, which is non-convexity. And there are other sources of increasing returns, beyond fixed costs.

This is bad news for the planners, because there are no general-purpose algorithms for optimizing under non-convex constraints. Non-convex programming isn’t roughly as tractable as linear programming, it’s generally quite intractable. Again, the kinds of non-convexity which economic planners would confront might, conceivably, universally turn out to be especially benign, soeverything becomes tractable again, but why should we think that?

If it’s any consolation, allowing non-convexity messes up the markets-are-always-optimal theorems of neo-classical/bourgeois economics, too. (This illustrates Stiglitz’s contention that if the neo-classicals were right about how capitalism works, Kantorovich-style socialism would have been perfectly viable.) Markets with non-convex production are apt to see things like monopolies, or at least monopolistic competition, path dependence, and, actual profits and power. (My university owes its existence to Mr. Carnegie’s luck, skill, and ruthlessness in exploiting the non-convexities of making steel.) Somehow, I do not think that this will be much consolation).

The Given Assortment, and Planner’s Preferences

So far I have been assuming, for the sake of argument, that the planners can take their objective function as given. There does need to be some such function, because otherwise it becomes hard to impossible to chose between competing plans which are all technically feasible. It’s easy to say “more stuff is better than less stuff”, but at some point more towels means fewer diapers, and then the planners have to decide how to trade off among different goods. If we take desired output as fixed and try to minimize inputs, the same difficulty arises (is it better to use so less cotton fiber if it requires this much more plastic?), so I will just stick with the maximization version.

For the capitalist or even market-socialist firm, there is in principle a simple objective function: profit, measured in dollars, or whatever else the local unit of account is. (I say “in principle” because a firm isn’t a unified actor with coherent goals like “maximize profits”; to the extent it acts like one, that’s an achievement of organizational social engineering.) The firm can say how many extra diapers it would have to sell to be worth selling one less towel, because it can look at how much money it would make. To the extent that it can take its sales prices as fixed, and can sell as much as it can make, it’s even reasonable for it to treat its objective function as linear.

But what about the planners? Even if they wanted to just look at the profit (value added) of the whole economy, they get to set the prices of consumption goods, which in turn set the (shadow) prices of inputs to production. (The rule “maximize the objective function” does not help pick an objective function.) In any case, profits are money, i.e., claims, through exchange, on goods and services produced by others. It makes no sense for the goal of the economy, as a whole, to be to maximize its claims on itself.

As I mentioned, Kantorovich had a way of evading this, which was clever if not ultimately satisfactory. He imagined the goal of the planners to be to maximize the production of a “given assortment” of goods. This means that the desired ratio of goods to be produced is fixed (three diapers for every towel), and the planners just need to maximize production at this ratio. This only pushes back the problem by one step, to deciding on the “given assortment”.

We are pushed back, inevitably, to the planners having to make choices which express preferences or (in a different sense of the word) values. Or, said another way, there are values or preferences — what Nove called “planners’ preferences” — implicit in any choice of objective function. This raises both a cognitive or computational problem, and at least two different political problems.

The cognitive or computational problem is that of simply coming up with relative preferences or weights over all the goods in the economy, indexed by space and time. (Remember we need such indexing to handle transport and sequencing.) Any one human planner would simply have to make up most of these, or generate them according to some arbitrary rule. To do otherwise is simply beyond the bounds of humanity. A group of planners might do better, but it would still be an immense amount of work, with knotty problems of how to divide the labor of assigning values, and a large measure of arbitrariness.

Which brings us to the first of the two political problems. The objective function in the plan is an expression of values or preferences, and people have different preferences. How are these to be reconciled?

There are many institutions which try to reconcile or adjust divergent values. This is a problem of social choice, and subject to all the usual pathologies and paradoxes of social choice. There is no universally satisfactory mechanism for making such choices. One could imagine democratic debate and voting over plans, but the sheer complexity of plans, once again, makes it very hard for members of the demos to make up their minds about competing plans, or how plans might be changed. Every citizen is put in the position of the solitary planner, except that they must listen to each other.

Citizens (or their representatives) might debate about, and vote over, highly aggregated summaries of various plans. But then the planning apparatus has to dis-aggregate, has to fill in the details left unfixed by the democratic process. (What gets voted on is a compressed encoding of the actual plan, for which the apparatus is the decoder.) I am not worried so much that citizens are not therefore debating about exactly what the plan is. Under uncertainty, especially uncertainty from complexity, no decision-maker understands the full consequences of their actions. What disturbs me about this is that filling in those details in the plan is just as much driven by values and preferences as making choices about the aggregated aspects. We have not actually given the planning apparatus a tractable technical problem(cf.).

Dictatorship might seem to resolve the difficulty, but doesn’t. The dictator is, after all, just a single human being. He (and I use the pronoun deliberately) has no more ability to come up with real preferences over everything in the economy than any other person. (Thus, Ashby’s “law of requisite variety” strikes again.) He can, and must, delegate details to the planning apparatus, but that doesn’t help the planners figure out what to do. I would even contend that he is in a worse situation than the demos when it comes to designing the planning apparatus, or figuring out what he wants to decide directly, and what he wants to delegate, but that’s a separate argument. The collective dictatorship of the party, assuming anyone wanted to revive that nonsense, would only seem to give the worst of both worlds.

I do not have a knock-down proof that there is no good way of evading the problem of planners’ preferences. Maybe there is some way to improve democratic procedures or bureaucratic organization to turn the trick. But any such escape is, now, entirely conjectural. In its absence, if decisions must be made, they will get made, but through the sort of internal negotiation, arbitrariness and favoritism which Spufford depicts in the Soviet planning apparatus.

This brings us to the second political problem. Even if everyone agrees on the plan, and the plan is actually perfectly implemented, there is every reason to think that people will not be happy with the outcome. They’re making guesses about what they actually want and need, and they are making guesses about the implications of fulfilling those desires. We don’t have to go into “Monkey’s Paw” territory to realize that getting what you think you want can prove thoroughly unacceptable; it’s a fact of life, which doesn’t disappear in economics. And not everyone is going to agree on the plan, which will not be perfectly implemented. (Nothing is ever perfectly implemented.) These are all signs of how even the “optimal” plan can be improved, and ignoring them is idiotic.

We need then some systematic way for the citizens to provide feedback on the plan, as it is realized. There are many, many things to be said against the market system, but it is a mechanism for providing feedback from users to producers, and for propagating that feedback through the whole economy, without anyone having to explicitly track that information. This is a point which both Hayek, and Lange (before the war) got very much right. The feedback needn’t be just or even mainly through prices; quantities (especially inventories) can sometimes work just as well. But what sells and what doesn’t is the essential feedback.

It’s worth mentioning that this is a point which Trotsky got right.(I should perhaps write that “even Trotsky sometimes got right”.)To repeat a quotation:

The innumerable living participants in the economy, state and private, collective and individual, must serve notice of their needs and of their relative strength not only through the statistical determinations of plan commissions but by the direct pressure of supply and demand. The plan is checked and, to a considerable degree, realized through the market.

It is conceivable that there is some alternative feedback mechanism which is as rich, adaptive, and easy to use as the market but is not the market, not even in a disguised form. Nobody has proposed such a thing.

Errors of the Bourgeois Economists

Both neo-classical and Austrian economists make a fetish (in several senses) of markets and market prices. That this is crazy is reflected in the fact that even under capitalism, immense areas of the economy are not coordinated through the market. There is a great passage from Herbert Simon in 1991 which is relevant here:

Suppose that [“a mythical visitor from Mars”] approaches the Earth from space, equipped with a telescope that revels social structures. The firms reveal themselves, say, as solid green areas with faint interior contours marking out divisions and departments. Market transactions show as red lines connecting firms, forming a network in the spaces between them. Within firms (and perhaps even between them) the approaching visitor also sees pale blue lines, the lines of authority connecting bosses with various levels of workers. As our visitors looked more carefully at the scene beneath, it might see one of the green masses divide, as a firm divested itself of one of its divisions. Or it might see one green object gobble up another. At this distance, the departing golden parachutes would probably not be visible.

No matter whether our visitor approached the United States or the Soviet Union, urban China or the European Community, the greater part of the space below it would be within green areas, for almost all of the inhabitants would be employees, hence inside the firm boundaries. Organizations would be the dominant feature of the landscape. A message sent back home, describing the scene, would speak of “large green areas interconnected by red lines.” It would not likely speak of “a network of red lines connecting green spots.”⁶

This is not just because the market revolution has not been pushed far enough. (“One effort more, shareholders, if you would be libertarians!”) The conditions under which equilibrium prices really are all a decision-maker needsto know, and really are sufficient for coordination, are so extreme as to be absurd.(Stiglitz is good on some of the failure modes.) Even if they hold, the market only lets people “serve notice of their needs and of their relative strength” up to a limit set by how much money they have. This is why careful economists talk about balancing supply and “effective” demand, demand backed by money.

This is just as much an implicit choice of values as handing the planners an objective function and letting them fire up their optimization algorithm. Those values are not pretty. They are that the whims of the rich matter more than the needs of the poor; that it is more important to keep bond traders in strippers and cocaine than feed hungry children. At the extreme, the market literally starves people to death, because feeding them is a less”efficient” use of food than helping rich people eat more.

I don’t think this sort of pathology is intrinsic to market exchange; it comes from market exchange plus gross inequality. If we want markets to signal supply and demand (not just tautological “effective demand”), then we want to ensure not just that everyone has access to the market, but also that they have (roughly) comparable amounts of money to spend. There is, in other words, a strong case to be made for egalitarian distributions of resources being a complement to market allocation. Politically, however, good luck getting those to go together.

We are left in an uncomfortable position. Turning everything over to the market is not really an option. Beyond the repulsiveness of the values it embodies, markets in areas like healthcare or information goods are always inefficient (over and above the usual impossibility of informationally-efficient prices). Moreover, working through the market imposes its own costs (time and effort in searching out information about prices and qualities, negotiating deals, etc.), and these costs can be very large. This is one reason (among others) why Simon’s Martian sees such large green regions in the capitalist countries — why actually-existing capitalism is at least as much an organizational as a market economy.

Planning is certainly possible within limited domains — at least if we can get good data to the planners — and those limits will expand as computing power grows. But planning is only possible within those domains because making money gives firms (or firm-like entities) an objective function which is both unambiguous and blinkered. Planning for the whole economy would, under the most favorable possible assumptions, be intractable for the foreseeable future, and deciding on a plan runs into difficulties we have no idea how to solve. The sort of efficient planned economy dreamed of by the characters in Red Plenty is something we have no clue of how to bring about, even if we were willing to accept dictatorship to do so.

That planning is not a viable alternative to capitalism (as opposed to a tool within it) should disturb even capitalism’s most ardent partisans. It means that their system faces no competition, nor even any plausible threat of competition. Those partisans themselves should be able to say what will happen then: the masters of the system, will be tempted, and more than tempted, to claim more and more of what it produces as monopoly rents. This does not end happily.

Calling the Tune for the Dance of Commodities

There is a passage in Red Plenty which is central to describing both the nightmare from which we are trying to awake, and vision we are trying to awake into. Henry has quoted it already, but it bears repeating.

Marx had drawn a nightmare picture of what happened to human life under capitalism, when everything was produced only in order to be exchanged; when true qualities and uses dropped away, and the human power of making and doing itself became only an object to be traded. Then the makers and the things made turned alike into commodities, and the motion of society turned into a kind of zombie dance, a grim cavorting whirl in which objects and people blurred together till the objects were half alive and the people were half dead. Stock-market prices acted back upon the world as if they were independent powers, requiring factories to be opened or closed, real human beings to work or rest, hurry or dawdle; and they, having given the transfusion that made the stock prices come alive, felt their flesh go cold and impersonal on them, mere mechanisms for chunking out the man-hours. Living money and dying humans, metal as tender as skin and skin as hard as metal, taking hands, and dancing round, and round, and round, with no way ever of stopping; the quickened and the deadened, whirling on. … And what would be the alternative? The consciously arranged alternative? A dance of another nature, Emil presumed. A dance to the music of use, where every step fulfilled some real need, did some tangible good, and no matter how fast the dancers spun, they moved easily, because they moved to a human measure, intelligible to all, chosen by all.

There is a fundamental level at which Marx’s nightmare vision is right: capitalism, the market system, whatever you want to call it, is a product of humanity, but each and every one of us confronts it as an autonomous and deeply alien force. Its ends, to the limited and debatable extent that it can even be understood as having them, are simply inhuman. The ideology of the market tell us that we face not something inhuman but superhuman, tells us to embrace our inner zombie cyborg and loose ourselves in the dance. One doesn’t know whether to laugh or cry or running screaming.

But, and this is I think something Marx did not sufficiently appreciate, human beings confront all the structures which emerge from our massed interactions in this way. A bureaucracy, or even a thoroughly democratic polity of which one is a citizen, can feel, can be, just as much of a cold monster as the market. We have no choice but to live among these alien powers which we create, and to try to direct them to human ends. It is beyond us, it is even beyond all of us, to find “a human measure, intelligible to all, chosen by all”, which says how everyone should go. What we can do is try to find the specific ways in which these powers we have conjured up are hurting us, and use them to check each other, or deflect them into better paths. Sometimes this will mean more use of market mechanisms, sometimes it will mean removing some goods and services from market allocation, either through public provision⁷ or through other institutional arrangements⁸. Sometimes it will mean expanding the scope of democratic decision-making (for instance, into the insides of firms), and sometimes it will mean narrowing its scope (for instance, not allowing the demos to censor speech it finds objectionable). Sometimes it will mean leaving some tasks to experts, deferring to the internal norms of their professions, and sometimes it will mean recognizing claims of expertise to be mere assertions of authority, to be resisted or countered.

These are all going to be complex problems, full of messy compromises. Attaining even second best solutions is going to demand “bold, persistent experimentation”, coupled with a frank recognition that many experiments will just fail, and that even long-settled compromises can, with the passage of time, become confining obstacles. We will not be able to turn everything over to the wise academicians, or even to their computers, but we may, if we are lucky and smart, be able, bit by bit, make a world fit for human beings to live in.

[1] Vaguely lefty? Check. Science fiction reader?Check. Interested in economics? Check. In fact: family tradition of socialism extending to having a relative whose middle name was “Karl Marx”? Check. Gushing Ken MacLeod fan? Check. Learned linear programming at my father’s knee as a boy? Check. ^

[2] More exactly, many optimization problems have the property that we can check a proposed solution in polynomial time (these are the class “NP”), but no one has a polynomial-timeway to work out a solution from the problem statement (which would put them in the class “P”). If a problem is in NP but not in P, we cannot do drastically better than just systematically go through candidate solutions and check them all. (We can often do a bit better, especially on particular cases, but not drastically better.) Whether there are any such problems, that is whether NP=P, is not known, but it sure seems like it. So while most common optimization problems are in NP, linear and even convex programming are in P.^

[3]: Most of the relevant work has been done under a slightly different cover—- not determining shadow prices in an optimal plan, but equilibrium prices in Arrow-Debreu model economies. But this is fully applicable to determining shadow prices in the planning system.(Bowles and Gintis: “The basic problem with the Walrasian model in this respect is that it is essentially about allocations and only tangentially about markets — as one of us (Bowles) learned when he noticed that the graduate microeconomics course that he taught at Harvard was easily repackaged as ‘The Theory of Economic Planning’ at the University of Havana in 1969.”) Useful references here are Deng, Papadimitriou and Safra’s “On the Complexity of Price Equilibria” [STOC’02. preprint],Condenotti and Varadarajan’s “Efficient Computation of Equilibrium Prices for Markets with Leontief Utilities”, and Ye’s “A path to the Arrow-Debreu competitive market equilibrium”. ^

[4]: In the mathematical appendix to Best Use, Kantorovich goes to some length to argue that his objectively determined values are compatible with the labor theory of value, by showing that the o.d. values are proportional to the required labor in the optimal plan. (He begins by assuming away the famous problem of equating different kinds of labor.) A natural question is how seriously this was meant. I have no positive evidence that it wasn’t sincere. But, carefully examined, all that he proves is proportionality between o.d. values and the required consumption of the first component of the vector of inputs — and the ordering of inputs is arbitrary. Thus the first component could be any input to the production process, and the same argument would go through, leading to many parallel “theories of value”. (There is a certain pre-Socratic charm to imagining proponents of the labor theory of value arguing it out with the water-theorists or electricity-theorists.) It is hard for me to believe that a mathematician of Kantorovich’s skill did not see this, suggesting that the discussion was mere ideological cover. It would be interesting to know at what stage in the book’s “adventures” this part of the appendix was written.^

[5]: In particular, there’s no reason to think that building a quantum computer would help. This is because, as some people have to keep pointing out, quantum computers don’t provide a generalexponential speed-up over classical ones. ^

[6]: I strongly recommend reading the whole of this paper, if these matters are at all interesting. One of the most curious features of this little parable was that Simon was red-green color-blind.^

[7]: Let me be clear about the limits of this. Already, in developed capitalism, such public or near-public goods as the protection of the police and access to elementary schooling are provided universally and at no charge to the user. (Or they are supposed to be, anyway.) Access to these is not regulated by the market. But the inputs needed to provide them are all bought on the market, the labor of teachers and cops very much included. I cannot improve on this point on the discussion in Lindblom’s The Market System, so I will just direct you to that(i, ii).^

[8]: To give a concrete example, neither scientific research nor free software are produced for sale on the market. (This disappoints some aficionados of both.) Again, the inputs are obtained from markets, including labor markets, but the outputs are not sold on them. How far this is a generally-viable strategy for producing informational goods is a very interesting question, which it is quite beyond me to answer.^

Kieran’s post about his book on organ donation gives me a hook to write something about the other end of the system, about organ recipients and the institutions which are supposed to match them up with donated organs. More specifically, how one such institution, the Kaiser HMO of Northern California, quite spectacularly failed several thousand people who were depending on them, by not matching them up. The story has been around since early May, when it was broken by Charles Ornstein and Tracy Weber in the Los Angeles Times (cached here), since confirmed by an investigation by Medicare/Medicaid. It doesn’t seem to have gotten all that much attention among the blogs, but it’s outrageous, and deserves, for that reason alone, to be better known.

(I was hoping to end my guest-blogging here by kvetching about econophysics, which is merely trivial; but that will have to wait until next week, back at my own blog.)

Kaiser is a very large and old HMO, with a huge presence in the Bay Area and northern California. (In fact it was one of the very first HMOs, started by Kaiser Steel during WWII as a way of attracting workers to its foundries and shipyards when wages were frozen.) It actually consists of non-profit health plans and hospitals, and for-profit physicians’ partnerships, the Permanente Medical Groups; normally, patients in the health plans have to go to physicians in the groups to have their medical expenses covered. One notable exception, up until 2004, was the 1500—2000 Kaiser patients in northern California who needed kidney transplants because of renal failure: those procedures were sub-contracted to the hospitals at UCSF and UC Davis, which also managed the patients’ places on the waiting list.

That last bit requires some explanation. Once organs are donated, they need to be matched up to recipients. This is done by an organization called the United Network for Organ Sharing, which tries to trade off urgency, seniority (i.e., time spent waiting for an organ), proximity and compatibility (since the closer the match between the donor’s immune system and the recipient’s, the less problems from rejection). There is a heavy weight put on seniority, though especially close immunological matches can over-ride it. Each transplant center is responsible for keeping the network up-to-date about their patients who need organs, their immunological profiles, and their time spent on the waiting list.

What seems to have happened is that in 2002, a transplant surgeon named Arturo Martinez proposed to Kaiser that it could save money, and increase the utilization of its hospitals’ surgical capacity, by bringing the kidney transplant program in-house, and Kaiser agreed, with Martinez becoming head transplant surgeon. (It would be unfair, at this point, to say that Kaiser did this because it meant more business for the for-profit Permanente Medical Group, but it’s hard to imagine that counted against the proposal.) As of mid-2004, Kaiser patients on the waiting list were informed that they would no longer be covered for transplants at UCSF or UC Davis, though they were free to go ahead and have them if they could come up with the money (roughly $100,000).

So far, all this is maybe a little self-serving on Kaiser’s part, but not, in itself, appalling. (It’s certainly more than legitimate for health-care organizations to try to save money.) What happened, though, was that Kaiser completely screwed up the program. Remember that organs are allocated (basically) through the UNOS system. The patients were being removed from the listings under the university hospitals, and being added to the listings under the new Kaiser transplant program. Unless this was done correctly, this would mean that they’d look like new names on the list, and so all of their accumulated waiting time, one of the main determinants of priority, would vanish. This happened to a huge number of people on the list, basically reducing the chance that they’d get a kidney to next to nothing. This becomes less surprising when one learns that Kaiser never consulted UNOS about the massive transfer of patients it was planning, and “placed responsibility for submitting patient data … in the hands of a single clerk who had one hour of telephone training on UNOS’s database”, though not any more excusable. Needless to say, patients were not told that by staying with Kaiser, they were losing their place on the lists, and thereby reducing their odds of survival. Some of them, at least, seem to have been assured that they were keeping their places, when that wasn’t true, though this is less clear to me.

Losing seniority on the transplant lists wasn’t the only problem. Kaiser did very few transplants, compared to the number of organs which were available. This happened in part because they just didn’t have the capacity to keep up with their many patients (at one point they were down to a single nephrologist for the whole program, who was also supposed to be its medical overseer), and in part because of what seems to have been mis-placed perfectionism or caution. These combined to the point of repeatedly turning down “zero mismatch” kidneys, ones where the likely compatibility over-rode considerations of seniority. This happened several dozen times at least — twice for one patient alone. Again, needless to say, patients weren’t told about this. In a “it’s not a bug, it’s a feature” moment, Kaiser initially attempted to defend its program by pointing out how few patients had died after transplants — since they’d done so few.

What strikes me as especially outrageous about all this is that the people being screwed over were people who needed new kidneys. To state the obvious, anyone who needs an organ transplant is very ill. It’s maybe less obvious that being that ill is a full-time job. One of the vital parts of the body is no longer working; to substitute for it requires extraordinarily complicated, time-consuming and generally unpleasant procedures. People who need new kidneys are people who are kept alive by dialysis, which is, indeed, complicated, time-consuming, often painful, almost always exhausting, and carries a non-trivial risk of infections, possibly fatal. People who need new kidneys are also often people who are very ill in other ways, since it’s not that common for both your kidneys to just stop working if nothing else is going on. (Kidney problems are, for instance, a not-uncommon complication of diabetes, and of high blood pressure. Dialysis, naturally, messes with blood pressure, adding yet another variable to monitor and regulate.) Simply staying alive, when you are multiply-sick person with organ failure, can pretty much demand all the time and attention you have to give, and a fair chunk of your loved ones’ as well. (There are good reasons why the families of people in situations like this tend to fall ill themselves.) You are certainly not in a position to check up on whether your medical organization has, through incompetence, messed up your position on the transplant lists. (Some Kaiser patients actually tried to keep up with their place on the lists, but were given the run-around.) And as for switching to another medical organization, do please show me the company which will extend coverage to someone who needs a new kidney, at a price which can be afforded by someone who needs a new kidney.

Since the Times broke the story, there’s been some improvement. The doctor who was medical head of the program, and apparently at least partly responsible for snafu of not transferring patients’ time, Sharon Inokuchi, has been relieved of her administrative duties. (In fact, if memory serves, she left Kaiser, but now I can’t find where I think i read that.) The program has been investigated by the Center for Medicare and Medicaid Services, which basically confirmed the newspaper reports, and forced it to promise major changes; it could still lose its eligibility for funding under those programs. The California state agency which regulates managed care is still, I believe, investigating. There’s talk of large fines, and there will certainly be lawsuits. All of this is to the good; it’s certainly better than nothing. But still, thinking about this makes me angry: Kaiser had a duty towards many very sick people, who were in a very poor position to look after themselves. It failed in that duty quite dramatically. In any organ transplant program, patients will die while waiting for a match. In most kidney transplant programs, though, about twice as many patients receive transplants as die while waiting; Kaiser managed to reverse that ratio. While it’s hard, in the nature of things, to identify any one patient who’s died and say “They would have lived, if only Kaiser hadn’t done this”, it’s almost certain that more of these people have died than would have otherwise. I don’t have a better remedy to propose than fines or lawsuits or institutional tinkering, but they all seem horribly inadequate.

(Via David R. in e-mail.)

Attention Conservation Notice: Over 1500 words on a wacky quasi-socialist economic scheme, from someone utterly lacking in credentials in economics. The scheme does not respect the sanctity of private enterprise, but at the same time would not reduce the alienation of labor one iota. Includes a lengthy quotation of a game-theoretic impossibility result.

In the previous installment in this series of modest proposals for the reform of corporate governance, I looked at ways of making the incentives of the managers of large, publicly-held corporations align more closely with those of their long-term shareholders. This left alone the question of the beneficiaries of corporate value; assuming that the managers are busily working to maximizing their revenue streams, who benefits from their industry and diligence? Having just read Mark Greif’s great essay on redistribution in n+1, I would like to make a suggestion. (Issue 4; long excerpt here, as pointed out by Matt in the comments.)

The text for today is Gary Miller’s Managerial Dilemmas: The Political Economy of Hierarchy, an excellent book which I learned about from Henry Farrell. Ambitiously, Miller tries to explain why hierarchical corporations exist at all, why they take some of the forms they do, and how, in part, their form relates to their performance. Much of the book, especially the first part, is a partially-successful attempt to find good economic reasons for their features, i.e., efficiency-enhancing ones. (He does not seriously consider the option that enterprises are hierarchical for non-economic reasons, say that some people like bossing others around, which hierarchies let them do, and those people are able to select hierarchies over other, more efficient, forms. After all, it’s hardly historically unprecedented for powerful people to prefer institutions which lower aggregate output but give them a bigger share of the product. See, e.g., here.) He also tries to explain why theories of corporate organization that rely solely on economic “mechanism design”, i.e., structuring information and material incentives, will actually lead to sub-optimal results, for pretty basic game-theoretic reasons; getting beyond these impasses is fundamentally a political problem. This is potentially quite subversive in its own way, but it’s really the first part of the work, about the economic justification of the hierarchical enterprise, that I’m going to twist and abuse.

One of the features of the modern corporation that Miller attempts to rationalize is the existence of shareholders who are passive and, in the overwhelming majority, utterly disconnected from the day-to-day or even year-to-year operations of the company. He does so by means of the following impossibility theorem, attributed to Bengt Holmstrom. Having tried to summarize Holmstrom’s theorem better than Miller, and failed, I’ll just quote Miller.

Holmstrom assumes that there are n agents whose actions determine a level of revenue x. The actions taken are unobservable and are costly to each of the agents. In particular, we assume the production function is a team production in which the productivity of each individual’s action is determined by other individuals’ levels of effort.

Holmstrom points out the desirability of three characteristics of an incentive system — and then shows that they are logically inconsistent. First, Holmstrom examines the Nash equilibrium outcome of an incentive system. At such an equilibrium, each individual will find that he or she could not do better by choosing a different effort level, as long as all others do not change their effort levels. Simple marginal analysis tells us that, in such an equilibrium, each person will find that his or her marginal cost of effort is exactly equal to the marginal gain; otherwise, the individual could be better off by working harder or not as hard. Second, Holmstrom stipulates that the outcome be budget balancing — that is, the incentive system should exactly distribute the revenues generated by the actors among the actors. Third, Holmstrom examines Pareto efficiency. This means that the outcome should be such that the individuals in the organization could not find a different outcome that would make them all better off.

Holmstrom shows that no budget-balancing system can create a Nash equilibrium that is also Pareto efficient. In other words, every budget-balancing incentive system will induce a social dilemma among its participants. The reason is that individuals will bring their own marginal costs of effort into equality with their own marginal gain. This means that each individual will not undertake an additional unit of effort that will produce less individual gain than individual cost — even if that extra unit of effort produces more gain for other individuals on the team.

As an example, suppose there is some individual who has a marginal revenue productivity of $12: Each unit of her own effort generates an extra $12 for the team. According to Pareto optimality, she should exert additional effort as long as the cost to her of that effort is less than or equal to $12; each such unit of effort generates more revenue for the team that it costs her as an individual. The only way to motivate her is to make sure that she gets all of the marginal revenue of her last unit of effort. In a team, it is impossible for this to be the case for every individual, as long as the incentive system is budget balancing. If everyone gets all of the last dollar produced, the team will have to pay out more in incentives than it generates. But if the individual gets only one-third of the marginal revenue from her actions, she will work only as long as her effort costs her less than $4 per unit. [pp. 129—130]

This suggests a rather unusual role for shareholders: they provide a money-sink, someplace money can go other than those actually involved in production. This means that the economic mechanism no longer has to be budget-balancing, which actually makes efficiency possible. Miller suggests that this is one reason why the modern public corporation, with its separation between legal ownership (by stockholders) and day-to-day control (by managers) can work, to the extent that it does. It is precisely because the shareholders are passive, with very limited influence over the actual running of the corporation!

Today’s modest proposal — and I should make it very clear that Miller suggests nothing of the kind — is to take this separation of functions even further. Shareholders can use their legal ownership to intervene in the running of the company, though it is hard (and managers try to make it harder). By doing so, however, they become players in the team-production game, and lose their useful role as a money-sink. To limit this danger, while retaining the advantages of competitive markets for capital allocation and corporate control, I suggest the following. A substantial fraction — say three-quarters — of all profits of publicly-held corporations are to be paid to a new institution, which we might call the National Mutual Fund. (Close corporations and partnerships are exempt.) Once a year, the Fund would pay out its accumulated profits as dividend checks, giving an equal amount to every adult citizen. And that’s it.

Substantially reducing the flow of dividends associated with stock ownership should cause a large one-time shock to the level of the stock market. (Roughly speaking, shares should drop by about 3/4.) However, because the Fund collects uniformly, it should not distort relative prices, which are what matter for purposes of capital allocation. The net worth of stock-holders, likewise, will suffer a one-time drop, but this will be partially compensated for by their receiving payments from the Fund in the future. Anyway, lots of things affect the value of stock holdings; it’s not like someone purchased their labor with a promise of future benefits, and then tried to back out of a freely-entered contract when it came time to pay up.

A further wrinkle would be to curb the practice of retained earnings. These account for a huge fraction of corporate capital formation, but they are also one of the ways in which managements escape market discipline. (For some figures on this, see Henwood’s Wall Street, pp. 72—76.) I suppose one could make a Hayekian argument in favor of the practice, but, really, if management can make a good case that a pet project will earn at least a normal rate of return, it shouldn’t be hard for them to raise funds on the open capital market, and if they can’t make such a case, it’s hard to see how they’d be discharging their fiduciary duties to shareholders by pursuing it anyway. This reform, I should add, is logically separate from that of instituting the National Mutual Fund. However, since corporations would pay more out in dividends, it would tend to increase the value of shares, reducing the shock to the level of the stock market.

It is hard to see why the actions of the National Mutual Fund could not be at least as rule-bound and de-politicized as those of a central bank run by skilled technocrats. Indeed, it would seem easier to reduce the discretion of the Fund’s officials to the vanishing point, and to strictly keep it from meddling with the affairs of any corporation, which would be deeply counter-productive. For their part, the citizens receiving the dividends would get the benefits of “portfolio diversification in their income“, but their incentives to meddle politically with individual firms, even quite large firms, would be quite muted. Moreover, they would have a direct and tangible incentive in the health of the corporate sector as a whole, making them less likely to support market-distorting measures to benefit particular firms, geographical regions or industrial sectors. We would move, in short, towards a true ownership society.

*: Actually, there’s a very substantial excerpt ; thanks to Matt in the comments for pointing it out.

It’s a hot, lazy Sunday, which seems like a good time for browsing through livejournal communities dedicated to photos of peacefully rusting machines, quietly crumbling buildings, and similar modern ruins:

Abandoned Places [via David Chess]

Decayed Machinery [via I forget who, years ago]

The photographers are all amateurs, so the quality (to the slight extent I can judge) is quite variable, but many manage to capture the suggestion of sunset and sadness, of unhappy stories brought to a close, which fascinates me about such scenes. Some of these photos, in fact, seem as good as, say, those in Terry Evans’s book on the former Joliet Arsenal, Disarming the Prairie, bringing to mind the words of the poet:

These are the halls of the dead, where the spiders spin and the great circuits fall quiet, one by one.

— But I see I’m getting melodramatic, and it’s just too hot and sticky and still to sustain that.

Would making Berkeley’s first-year economics Ph.D. graduate students this fall read short biographies of William Gates and William Marshall as a way of getting at the idea that there are non-market societies that work very differently from our own today—would that be a teaching idea of extraordinary brilliance or of total insane lunacy?

And: those of us who teach things other than economics, what books do or should we hand out as ice-axes for our students’ frozen seas? ( This one is mine.)

You, the author of the paper, have a responsibility to make your ideas comprehensible. If the first method you choose to explain them fails, then you listen to what people say about where they lost all track of understanding and write a new paper—- with NEW explanations, not the same explanations that failed last time only renumbered. … [This is] not something that is drilled into young scientists; that it is YOUR responsibility to make your ideas clear to others, not their responsibility to try to figure out what you are talking about. As science grows ever larger and ever more complex, I think it is time for all scientists to be much more explicit and much more ruthless on this point.

Science is a social, collaborative process, so part of being a good scientist is effective communication. Scientific communication is overwhelmingly written communication (scientific disciplines are, in a sense, literary communities), so part of being a good scientist is being a good writer. Unfortunately, scientists get little training in writing, and much of that consists of being advised to follow the rules found in horrid little compendia. Fortunately, there is some actual research on effective written communication, that is, on how to arrange your words so that their readers tend to acquire clear notions of your ideas. The best practical guide here, I’ve found, is Joseph William’s Style: Towards Clarity and Grace. However, I have just discovered (via Paradise Blogged) a fine essay by George Gopen and Judith Swan, “The Science of Scientific Writing“, which gives a clear yet concise presentation of the work. (Gopen and Williams are collaborators.) Here is their own summary of how to be clear:

1. Follow a grammatical subject as soon as possible with its verb.
   
2. Place in the stress position the “new information” you want the reader to emphasize.
   
3. Place the person or thing whose “story” a sentence is telling at the beginning of the sentence, in the topic position.
   
4. Place appropriate “old information” (material already stated in the discourse) in the topic position for linkage backward and contextualization forward.
   
5. Articulate the action of every clause or sentence in its verb.
   
6. In general, provide context for your reader before asking that reader to consider anything new.
   
7. In general, try to ensure that the relative emphases of the substance coincide with the relative expectations for emphasis raised by the structure.
   

If these rules, and the notions behind them, are valid, they should apply to more than just
scientific writing; in particular, they should hold for other kinds of academic prose. Looking at their examples of revising scientific writing, expressing the same complicated and precise ideas in more easily grasped ways, I couldn’t help but be reminded of the humanistic “bad writing” controversy, where the case for the defense often seems to rest on complexity, and the example of scientific jargon. The examples of Gopen and Swan, Williams, etc., show that those defenses do not hold. Scholars of the humanities may have reasons for being unclear which don’t apply to scientists, but I can’t think of any good ones.

A staple of bad movies and trashy novels about scientists (i.e., the kind I read) is the neglected genius whose ideas are rejected with incomprehension by the scientific establishment during his life, because they are simply Too Far Ahead Of His Time to be grasped by lesser mortals, only for the scientific community to rediscover these insights decades later. This sort of thing can make for entertaining fiction (if dreary self-mythologization), but it simply doesn’t happen all that often in real life, especially not when the hero is a part of the establishment, and indeed a much-honored one. It certainly doesn’t show up, with documentary evidence, in the staid, reliable pages of Reviews of Modern Physics. Nonetheless:

Gregory L. Eyink and Katepalli R. Sreenivasan, “Onsager and the theory of hydrodynamic turbulence”, Reviews of Modern Physics 78 (2006): 87—135; no free copy

Abstract: Lars Onsager, a giant of twentieth-century science and the 1968 Nobel Laureate in Chemistry, made deep contributions to several areas of physics and chemistry. Perhaps less well known is his ground-breaking work and lifelong interest in the subject of hydrodynamic turbulence. He wrote two papers on the subject in the 1940s, one of them just a short abstract. Unbeknownst to Onsager, one of his major results was derived a few years earlier by A. N. Kolmogorov, but Onsager’s work contains many gems and shows characteristic originality and deep understanding. His only full-length article on the subject in 1949 introduced two novel ideas — negative-temperature equilibria for two-dimensional ideal fluids and an energy-dissipation anomaly for singular Euler solutions — that stimulated much later work. However, a study of Onsager’s letters to his peers around that time, as well as his private papers of that period and the early 1970s, shows that he had much more to say about the problem than he published. Remarkably, his private notes of the 1940s contain the essential elements of at least four major results that appeared decades later in the literature: (1) a mean-field Poisson-Boltzmann equation and other thermodynamic relations for point vortices; (2) a relation similar to Kolmogorov’s 4/5 law connecting singularities and dissipation; (3) the modern physical picture of spatial intermittency of velocity increments, explaining anomalous scaling of the spectrum; and (4) a spectral turbulence closure quite similar to the modern eddy-damped quasinormal Markovian equations. This paper is a summary of Onsager’s published and unpublished contributions to hydrodynamic turbulence and an account of their place in the field as the subject has evolved through the years. A discussion is also given of the historical context of the work, especially of Onsager’s interactions with his contemporaries who were acknowledged experts in the subject at the time. Finally, a brief speculation is offered as to why Onsager may have chosen not to publish several of his significant results. [My links.]

Nobody outside of statistical physics (and maybe physical chemistry) has heard of Onsager, but he was indeed a great physicist, albeit in a very technical, non-flashy way. By the time he did this work on turbulence, he was already well-known in statistical mechanics for the analytical solution of the Ising model, his theory of phase transitions in liquid crystals, and, perhaps most importantly, a pair of classic papers from 1931 which basically founded modern irreversible thermodynamics, for which he would eventually win the Nobel Prize. (Eyink and Sreenivasan give a fuller discussion of his accomplishments, including the Onsager-Machlup theory of non-equilibrium processes, on which Eyink himself has done important work.) We’re definitely not talking about some marginal figure cut off from the scientific community.

Nonetheless, his attempts to get people to pay attention to these ideas on turbulence were singularly unsuccessful. The reaction of Theodore von Kármán, a deservedly great name in fluid mechanics, was to describe it (in a letter to his student C. C. Lin) as “somewhat ‘screwy’ “; Onsager also corresponded with Lin, who replied in the classic manner of someone wanting to put an end to a conversation (quoted on p. 117): “I am sorry to say that I have not made much progress, except that I desire still more to see something done in this line to bring your ideas down to my level of understanding.” As for the statistical physicists, Eyink and Sreenivasan describe their reaction as one of “polite incomprehension” (except on the part of von Neumann — in an unpublished report). The fact that one of Onsager’s letters describing his ideas (reproduced as Appendix A in this paper) is headed “The little vortices who wanted to play”, and begins “Once upon a time there were n vortices of strengths K1, … , Kn in a two-dimensional frictionless incompressible fluid” probably didn’t help, either. Most of all, a combination of discouragement over this reception, a tendency to be a slow and perfectionist author, and having scads of major research projects going simultaneously kept Onsager from even trying to publish any of this material.

The moral, I hope, is clear: statistical physicists who wander into other areas of science, and find their ideas dismissed by the best experts on those subjects, should nonetheless publish in Physical Review, in a “Fools! I’ll show them all!” spirit, provided they are Lars Onsager.

(It’s interesting that this paper was written by two physicists active in this area, rather than by a historian of science. It seems doubtful to me that a historian, reading the relevant materials in the Onsager archives, would have realized that there was a story here, unless they were familiar with modern work on turbulence at a deeply technical level — unless they had “contributory” as well as “interactional” expertise. And if anyone had gone over those archives around 1990, before these ideas were re-discovered, what would they have made of it?)

Two of my more public-spirited fellow citizens have recently identified looming threats to our own Commonwealth of Pennsylvania.

1. Our beloved junior senator, Rick Santorum (via Pharyngula):
   

   Most scientists unfortunately, those that certainly are advocating for this [embryonic stem cell research], and many others feel very little moral compulsion. It’s a utilitarian, materialistic view of doing whatever they can do to pursue their desired goals.

   
   I, for one, will be happier voting on Mr. Santorum’s re-election in November, knowing that my ballot will play a part in the age-old struggle between utilitarian materialism and deontological idealism, as well as the sagas of human-canine relations and Old Corruption.


2. Our beloved linguistics professor, Mark Liberman:
   

   More than a third of all Pennsylvanians are native speakers of a language other than English — and many of them have not even tried to learn English since immigrating, or at least prefer to carry out their daily lives in another language, living together in neighborhoods where their native language dominates. Some people worry that the majority status of English is critically endangered. 25 years ago, a major political figure warned that these “aliens … will never adopt our language or customs, any more than they can acquire our complexion”, and so far, his prediction seems to be right on the money.

   
   And let’s not forget what they’ve done to our cooking!

Under the rubric “Can Blogging Derail Your Career?”, the Chronicle of Higher Education has seven bloggers discussing Yale’s decision to not hire Juan Cole as a professor of history, and the role, if any, played by his blog in that decision: Siva Vaidhyanathan, Dan Drezner, Brad DeLong, Michael Bérubé (all: yay!), Glenn Reynolds and Ann Althouse (both: hiss), and Erin O’Connor (null result), with a “response” by Cole, which doesn’t actually address the others’ posts specifically, and reads like a separate essay on the same subject as the others. (Via DeLong.)

(Some of the things which were written about Cole as part of the controversy (e.g.,) give the impression of a professor who attains incomprehensibility not through obscurity but through foaming at the mouth. As it happens, though, I sat in on his seminar on millenarian movements when I was a post-doc at Michigan, and nothing could be further from the truth. I suppose I could have missed all the sessions which degenerated into hours-long rants about Zionist Entities… Of course, I don’t know why Yale didn’t give him the job, but if it was because they thought he was too spittle-flecked to be presentable to parents and alumni, they were misinformed.)

The fact that this post is not filed under “Middle East Politics” isn’t going to stop anyone in the comments, is it?

First off, I should thank Henry and the rest of the Timberites for the kind invitation to guest-post, and that very warm introduction. In exchange, I’m going to blog more or less as I usually would, only here. This means some big bricks of posts about “complex systems”, so called, which is or was my scientific field, more or less; and also any miscellaneous outrages which catch my eye this week. Mounting my usual hobby-horses on this stage is a poor exchange for their generosity, but mounting hobby-horses is why I started blogging in the first place, and anyway I’m big on conscienceless nomothetic exploitation of cooperators.

Today I want to talk (below the fold) about some recent work in the statistical mechanics of disordered systems, which might help explain how our sense organs work, and actually involves some good uses of the self-organized criticality and power laws; tomorrow or the day after I’ll get to the smoldering question of “Why Oh Why Can’t We Have Better Econophysics?”

Folklore says that the dark-adapted human eye can detect a single photon; this isn’t quite true, but we can consciously detect a few tens of photons, and some species are that sensitive. Of course, we can see not only in the dark but also during broad daylight, but then the number of photons falling on every part of the retina is huge; the eye isn’t overwhelmed and saturated, though now one or ten photons more or less makes no discernible difference. In the jargon, the eye, and the other sensory organs, have both a large “dynamic range” (we can see in the dark and in the daylight), and “nonlinear response” (changes which are noticeable in the dark aren’t against a high-intensity background). Some version of these facts, including the basic (power-law) form of the relationship between physical stimulus intensity and perceived sensory magnitude, have been known since the nineteenth century. This makes it all the more puzzling that sensory neurons show a linear response over a narrow dynamic range, beyond which they saturate.

You could evade this difficulty by having lots of neurons with different operating ranges, so that raising stimulus intensity saturated some but activated others. The problem is that there don’t seem be that wide a spectrum of operating ranges for individual neurons. In a recent paper, Osame Kinouchi and Mauro Copelli (who blog together at Semciência) offer another way, which has to do with the way sensory neurons interact with each other in a network.

Osame Kinouchi and Mauro Copelli, “Optimal dynamical range of excitable networks at
criticality”, Nature Physics 2 (2006): 348—351; free preprint version at q-bio.NC/0601037 *

Abstract: A recurrent idea in the study of complex systems is that optimal information processing is to be found near phase transitions. However, this heuristic hypothesis has few (if any) concrete realizations where a standard and biologically relevant quantity is optimized at criticality. Here we give a clear example of such a phenomenon: a network of excitable elements has its sensitivity and dynamic range maximized at the critical point of a non-equilibrium phase transition. Our results are compatible with the essential role of gap junctions in olfactory glomeruli and retinal ganglionar cell output. Synchronization and global oscillations also emerge from the network dynamics. We propose that the main functional role of electrical coupling is to provide an enhancement of dynamic range, therefore allowing the coding of information spanning several orders of magnitude. The mechanism could provide a microscopic neural basis for psychophysical laws.

Neurons, like muscle cells, are “excitable”, in that the right stimulus will get them to suddenly expend a lot of energy in a characteristic way — muscle cells twitch, and neurons produce an electrical current called an action potential or spike. Kinouchi and Copelli use a standard sort of model of an excitable medium of such cells, which distinguish between the excited state, a sequence of “refractory” states where the neuron can’t spike again after it’s been excited, and a resting or quiescent state when the right input could get it to fire. (These models have a long history in neurodynamics, the study of heart failure, cellular slime molds, etc.) Normally, in these models the cells are arrayed in some regular grid, and the probability that a resting cell becomes excited goes up as it has more excited neighbors. This is still true in Kinouchi and Copelli’s model, only the arrangement of cells is now a simple random graph. Resting cells also get excited at a steady random rate, representing the physical stimulus.

Kinouchi and Copelli argue that the key quantity in their model is how many cells are stimulated into firing, on average, by a single excited cell. If this “branching ratio” is less than one, an external stimulus will tend to produce a small, short-lived burst of excitation, and there will be no spontaneous activity; the system is sub-critical. If the branching ratio is greater than one, outside stimuli produce very large, saturating waves of excitation, and there’s a lot of self-sustained activity, making it hard to use a super-critical network as a detector. At the critical point, however, where each excited cell produces, on average, exactly one more excited cell, waves of excitation eventually die out, but they tend to be very long-lived, and in fact their distribution follows a power law.

(People who teach courses on random processes are very fond of branching processes, because the basic model can be solved exactly with hundred-year-old math, but there are endless ramifications, and some of the applications are very technically sweet. Like most mathematical scientists, Kinouchi has certain tools he tends to return to, and critical branching processes are one of them.)

As Kinouchi and Copelli say in their abstract, the idea that the critical point, where things are just about to go unstable, is a useful place for processing or transmitting information is a persistent theme of complex systems. (You could, arguably, even trace a version of the idea back to William James’s Principles of Psychology.) It has also, before this, been one of the weakest of our ideas. The original work from the 1980s on “evolving to the edge of chaos” has proved impossible to replicate, I would even say experimentally refuted. (Why the phrase and idea continue to propagate is another question for another time.) Stu Kauffman’s studies of models of gene regulatory networks certainly suggests that information moved through these most easily near their critical point, but I don’t think anyone has done a careful information-theoretic analysis of that. In any case, E. coli doesn’t care about the bandwidth of its regulatory network: it cares about reliably making lactase when it only has lactose to eat, i.e., specific adaptive functions. Prior to this, I can only think of one situation where the idea has been made precise and has strong evidence to back it up (namely, this paper), but that’s a purely mathematical exercise of no biological relevance.

What Kinouchi and Copelli have done is very different: they’ve actually identified something biologically important which is maximized at the critical branching ratio, namely the dynamic range. The network as a whole responds to the stimulus, and its dynamic range can be many orders of magnitude wider than that of its component cells. It is this enhancement which is maximized at the critical branching ratio, and falls off sharply for networks which are even a little sub- or super- critical. As a bonus, the shape of the response function is of the correct power-law form, though, in their model, the exact exponent isn’t right. Modifying the network structure, or some model details, changes the exponent, but the dynamic range is still sharply peaked at the critical branching ratio.

There are a lot of other nice things about this paper, which I won’t get in to least I repeat it all, but I will point out one thing: while their central qualitative results are pretty robust to small tweaks, there are some details of their model which make it a fair caricature of some excitable media, but not all. These are quite deliberately matched to properties of the olfactory system and the retina, but wouldn’t work in, say, the cortex, where the dynamics of excitation are different. So this isn’t an “over-universal” model, but one of particular phenomena produced by particular mechanisms. In fact, looking at olfaction, they are able to make a prediction about the effects of knocking out specific genes which are involved in the fast, symmetrical electric couplings they assume. Nobody seems to have done those experiments yet, but, at least to this non-biologist, it seems feasible, and, now, very interesting.

*: Here’s an anecdote illustrating how broken academic publishing is. Kinouchi and Copelli work at the University of Saõ Paulo, which doesn’t, for reasons of economy, subscribe to Nature Physics. To get an electronic copy of their own published paper, they were forced to write correspondents at other universities. I couldn’t help them, because my school doesn’t feel like it can afford to subscribe to Nature Physics either.