Monochrome swans

by John Q on September 15, 2014

I have a request[^1] for help from scientifically literate readers. A lot of my research work is focused on the problem of unforeseen contingencies, popularly, if ethnocentrically, described as “black swans”. In particular, I’m interested in the question of how you can prepare for such contingencies given that, by definition, you can’t foresee exactly what they will be. One example, with which I’m very pleased, is that of the precautionary principle. It seems reasonable to say that we can distinguish well-understood choices involving hazards from those that are poorly understood, and avoid the latter, precisely because the loss from hazard cannot be bounded in advance.

Anyway, I was thinking about this in relation to the actual case of black swans (or, from my own perspective, white swans). The question is: what principles would help you to avoid making, and acting on, the assumption “all swans are white (or, in my own case, black)”. It seems to me that the crucial fact here is that the shift from black to white, or vice versa, is, in evolutionary terms, a small one. So, if you used something like cladistics, you would avoid choosing feather color as a defining feature of swans, and birds in general. As I understand it, a phylogenetic approach starts with features that are very strongly conserved (body plans) and proceeds from there. But, rather than assume that my own understanding is correct, it seemed simpler to ask.

[^1]: There’s a blog-specific word for this, but I refuse to use it



P.M.Lawrence 09.15.14 at 10:01 am

Ah, the old unknown unknowns pleuy, as Inspector Clouseau might say.

I can tell you of an approach that is sometimes used to estimate how many proof reading errors or computer bugs have not been found, but only rarely as management is usually very frustrated to be told that there are errors out there that nobody has found and so can’t be corrected; they prefer the confirmation bias of thinking that they have paid people to find and fix the errors, so when they can no longer find any there can no longer be any, right? Also, it’s expensive if done right and worthless if not.

The method is to set some independent teams to work finding errors (but not fixing them, as that introduces a bias into what the other teams are working with). Then the teams identify what they have found, and a table is set up to indicate which errors have been found by each of 1, 2, 3 … of the teams. Then the table’s distribution is analysed to get an estimate of how many errors have been found by 0 teams, i.e. not found at all. A simple situation where each team was told to find and report just one error would nearly follow a Poisson distribution, but more errors detected per team yield more data to analyse and it is likely that the teams are actually not as independent as they might be, even if they are carefully kept at working separately, as they may well share various preconceptions or habits (e.g. being more alert at one end of the material, or more predisposed to notice errors like the ones they noticed early on); regardless, enough different enough work should still show a pattern that can yield a useful quantitative estimate of known confidence, since that would bring out and adapt to team correlations as well, much as when a pub dartboard is taken down its outline can be discerned from the pattern of misses it leaves (something like this was used to work out which parts of Second World War bombers needed more armour, from the damage patterns of those that made it back). Clearly, the estimates are easier to obtain from just two teams told to find ten (say) errors each; if they only have one duplicate, that’s a 10% overlap, implying a total of 100 errors and so 81 undetected errors. But that could easily fail by undershoot, as it uses too few teams to bring out and adapt to team correlation effects. Another defect of the approach: in its simple form it only brings out pointwise errors, not system based ones that rest on incompatibilities between various points that might each be sound individually. Although techniques analogous to multi-layer neural net training can bring those within reach, even the pointwise approach is already far too expensive in time and skilled labour for most budgets.

I have somewhere heard that a certain British police force once analysed its murder detection rates broken down by the number of different techniques that had worked or would have worked on known cases, so as to estimate the number of cases that had gone completely undetected; the story goes that that study was quickly halted when it began to show an embarrassingly large number.

Anyhow, for your purposes I imagine putting different speculative teams to work brainstorming on a “Delphic Oracle” basis would yield a number of “Bleached [because identified] Swan” scenarios that could be tabulated in this way and used to generate an estimate and confidence for the number of Black [because unidentified] Swan situations within some universe of discourse, perhaps even with estimates of their severity obtained in a similar way.


Joshua W. Burton 09.15.14 at 12:34 pm

Well, clearly the chance that Aristotle’s readers would have suddenly discovered a black swan in southern Europe just by waiting around for a few more millennia is just as small as it seemed the day the Prior Analytics was written. What actually happened in the literal black swan case was the discovery of Australia and Chile.

So, probably the guiding principle is to footnote every universal with a domain of observed applicability. Then, when you’re Dampier or Cook and the Blue Hills are just visible on the horizon, the domain of “temperate lands on the planet where swans evolved” is visibly noncoincident with the domain of “lands where all swans have long been observed to be white,” and you land with a reasonably open mind.

The real lesson here is that encountering kangaroos is enormously more interesting, less predictable, and more typical of new horizons than encountering peculiar swans. It wouldn’t be much of an improvement over naive inductive reasoning to come up with a model that would let you anticipate old things in funny hats, but never new things.


BenK 09.15.14 at 12:39 pm

You are asking a strange sort of question.

Here’s the basic problem. ‘Black swan’ is simply a stand-in for the concept of the unimagined. If you spend some time imagining additional possibilities – opportunities or risks – then you simply shift a number of things out of the category of imagined.

There is a cost to imagination. If, as a driver, you look for cars to the front, left, right and behind, but end up getting crushed by a falling helicopter – does that mean it was a sensible thing to add looking up to your routine during a commute? The cost of awareness is greater than the risk of occurrence. ‘But,’ you might say, ‘it could have kept you alive!’ – or been a distraction that helped you get blindsided by a dump truck.

So real black swans are not determined by the assumption that ‘all swans are white.’ Certainly not a stated assumption, because that already brought the concept of different color swans into the picture explicitly. Rather, they exist outside that space of conception and discussion.

Preparing against black swans… is, as far as I can tell, an exercise in suboptimizing; a kind of extreme hedge. If you can imagine that the preparation you are making could be useful, then it isn’t preparation for a black swan – its just preparation for some tail occurrence that exists in your recognized universe of possibility.


Joshua W. Burton 09.15.14 at 12:42 pm

When I was little, I had a half-clever idea about getting around Gödel’s first incompleteness theorem: whenever you encounter an undecidable statement, look at its outermost quantifier. If it reads “There exists . . .” then add it as a new postulate; if it reads “For all . . .” then add its negation. The metareflective principle here is, of course, motivated by black swans: just about any damn thing could exist, and hardly anything is always so. Needless to say, this doesn’t work in set theory (it’s pretty easy to write equivalent Gödel sentences in both forms and force the metareasoner into contradiction), but as a skeptical principle about unknown domains in the physical world it’s not too bad.


Mike Huben 09.15.14 at 1:05 pm

The basic problem is that of induction: we have created a universal without much care. The first thing to do is to be humble, and admit the limits of your claim “all swans are white”. Cygnets are swans that are immature, and they are grayish brown. Adult mute swans (the species originally observed) have a black patch between the eye and the bill, and the bills and feet are orange. Internally, swans are nowhere really white: most organs and tissues have various queasy colors. That immediately gets you to a more accurate “adult swans in the area observed have white plumage excepting a few bits.” That’s a much more reasonable premise to start with. (Similar exercise: find the numerous faults with “the sky is blue.”)

A biological idea such as observed strong conservation of some characters during evolution is not really any better than “all swans are white” because it has the same sorts of limitations. But with some understanding of development, it gets worse.

Adult coloration is a developmental characteristic, controlled by the switching on or off of genes that produce pigments such as melanin. Mute swans produce melanin (I’m presuming that’s the pigment) in the feathers of juveniles and in the patch between the eye and the beak. The melanin production is switched on in those tissues. Evolutionarily, it could be a fairly simple thing to switch melanin production in adult plumage on or off: switching it off is likely how white plumage in swans evolved, since the ancestors of swans probably were not white. Reversals of characters are not uncommon in evolution. And the switching can be very localized: there are also black-necked swans in South America.

Indeed, if the pigment that gives the orange color to the bill was switched on in the production of adult plumage, we could have orange swans too.

The idea of using evolutionary principles to help analyze unknowns is interesting, but probably won’t work. It’s very hard to rule out things because of Orgel’s Second Rule: “Evolution is cleverer than you are.”


Glen Tomkins 09.15.14 at 1:17 pm

Well, the good news is that we deal with this sort of problem all the time in medicine, so there’s an entire continent of practice to explore to help understand how to deal with black swans in ways that minimize error. Every time a patient walks into your exam room, there is a non-zero possibility that their complaints are caused by some hitherto undescribed illness.

The bad news is that we don’t do a very good job of handling black swans, so it’s more like a continent of malpractice to explore for negative lessons. A really big and growing continent…


Z 09.15.14 at 1:33 pm

So, if you used something like cladistics, you would avoid choosing feather color as a defining feature […] [A] phylogenetic approach starts with features that are very strongly conserved (body plans) and proceeds from there.

In fact, and since you ask, I think this formulation could easily be misguided (the “proceeds from there” is slightly ambiguous). The logic of cladistic classification is strictly based on evolution, so it grants secondary importance to external features and insofar that it does concern itself with them, it strives to treat all features as equal, even negative ones (such as being tailless) or ephemeral ones (such as exhibiting pharyngeal clefts during the first weeks of embryonic development).

However, a taxonomic classification such as the epoch-making one of Linneaus, which does rely on external stable features shared by all species in a given class and does give preeminence to certain features over others, would have not only not posited monochrome swans but would have directly lead to the hypothesis of black swans, since swans are part of the family Anatidae, which includes many grey, black and multicolored species in Northern Europe alone. Putting it differently, hypothesizing on the basis of Cygnus cygni that swans are white would have entailed that despite the general and obvious similarities, Cygnus cygni was unrelated to geese and ducks.

Returning to evolution, the variation and heritability of feather color among birds is such a paradigmatic and easily demonstrable example of the general phenomenon that Darwin chose it as its first example to open the Origin of Species, so sure he was that no reasonable person could dispute that feather colors in pigeons were driven both by heritability and selection. So from any biological point of view, monochrome swans was an exceptionally poor hypothesis .


bill benzon 09.15.14 at 1:37 pm

I’m not sure what you’re looking for, John, but you might want to take a look at Robert de Vany, Hollywood Economics (my review HERE), which comes higly recommended by Nassim “Black Swan” Talib. The “black swans” de Vany is interested in are those very few blockbuster movies that earn megaprofits. Since he’s investigating past data of box office returns (a 10-year run in the 1990s) there’s no mystery about what those movies are. What interests de Vany is whether or not those films, or even the merely profitable ones, could be predicted on the basis of information in hand no later than the end of the first weekend of showing. Are there marquee names, for example, that will guarantee profitability or even blockbusterhood? No there are not.

I don’t know whether or not there’s anything in the book that would help you. But, those Hollywood businessmen are doing everything they can to predict which movies will succeed and trying to hedge against failure. There’s an interesting section of the anti-trust break-up of the old studio system back in the 1950s. De Vany argues, plausibly in my view, that the cozy arrangements between studios and theatres that existed in the “Golden Age” were not, in fact, conspiracies against the public interest. Rather, they made it possible for the studios to pump out product on a routine and profitable basis (by reducing transcation costs, which went up when the studios were forced to disband).


Nickp 09.15.14 at 1:39 pm

Not sure I understand your question, but I think you are asking “When classifying organisms like swans, how do you set up a classification scheme that is least likely to be overturned by a new discovery?”

I think, the short answer is that you don’t worry about it all that much. Instead of trying to predict unknowns, you have a mechanism for changing the classification as necessary.

So, you set up your classification scheme to unambiguously distinguish between the entities that you know. Ideally, you use traits that don’t independently evolve to similar states very frequently, but you work with what you’ve got. If some new entity is discovered that doesn’t fit any group in your classification scheme, you either place it in a new category or revise the criteria and reorganize the classification scheme to fit it in with one of the existing groups. Among orchids, there has recently been wholesale reorganization of a lot of genera based on DNA sequencing, because it turns out that the criteria used by Victorian botanists to define the genera (e.g. number of pollen clusters, flower morphology, etc) evolve rapidly and unrelated plants can end up with the same features.

So, if one of the defining characteristics of genus Cygnus were “white feathers,” then a newly discovered black bird from Australia would be not-Cygnus. If it were a sister-group to all your white Cygnus species, you might raise new genus and place the black bird in that. If you show via other criteria that the black birds are nested phylogenetically within your genus Cygnus, you’d either split Cygnus or redefine it to include birds of different colors.


Joshua W. Burton 09.15.14 at 1:42 pm

Similar exercise: find the numerous faults with “the sky is blue.”

When our kids were of a certain age, the parenting best-practice of giving bad answers to discoverable questions turned this one into a family running joke.

“Abba, why is the sky blue?”

“Because it reflects the sea.”
“Because there is less oxygen up there, like with arterial and venous blood. When the sun gets low, you see the sky turn red.”
“Because if it were green, you wouldn’t know where to stop mowing,”
“It’s not actually blue, it’s clear. The blue you see is called cyan; it’s actually the exact opposite color of red. What you’re seeing is the negative image of the blood vessels in your own retina.”
“That’s the average color of the whole planet; light scatters around in the upper atmosphere. You can see on Mars that the sky is pale orange for the same reason.”


J Thomas 09.15.14 at 1:43 pm

My advice would be to look carefully at what you are depending on.

If you have the assumption “Our crops will always produce more than we need to feed our population”, some black swan event might make that false. You can store food enough to prepare for that. You can cheaply store enough food to prepare for a 2% shortfall. At more bother and expense you can prepare for a 10% shortfall. With more effort you can prepare for a 100% shortfall, and with more still you can prepare for two years of 100% shortfall. At this point you might figure that anything that wipes out our crops for two years is likely to kill a lot of people so you won’t need much more food stored than that.

It’s unforseen events you can’t predict. But if you know what you care about, you can predict whatever you care about getting disrupted, and you can look for ways to palliate that disruption.

To some extent it pays to keep alternatives available. If you have multiple kinds of crops in use then something which damages one might not damage the others. But complexity has its cost too. When you find yourself switching among alternatives, there’s room for more incompatibilities than with a simpler system, there are more things to go wrong.

It’s tempting to figure you shouldn’t use anything new until after you have a long track record using it. That results in people trying to be loyal to their cultural traditions, and it’s pretty stultifying.


sPh 09.15.14 at 2:03 pm

It is interesting that the world of corporate projects (and to a certain extent operations as well) is currently dominated by the thinking of the Project Management Institute, ISO 9xxx/10xxx, CMMI, etc to which the concept of an “unknown unknown” is unacceptable, and under which any management actions designed to prepare in general terms for handling classes of situations – as opposed to specific step-by-step “process” procedures – is heresy and shall be punished as such. Under this line of thinking human beings are as demigods and every action and consequence is foreseeable, even under a situation of exponential choice.


Mr Punch 09.15.14 at 2:44 pm

We have a way of being absolutely certain about things you can’t really know about – it’s called religion. (This is why, several steps of reasonong down the road, the precautionary principle takes us back to the Dark Ages.)


MDH 09.15.14 at 2:52 pm

I don’t have good terminology for the following, but I think about this problem thus:

The list of factors which theoretically bear on some phenomenon of interest is essentially innumerable. What we all do all the time is attempt some rank ordering where the top X number sufficiently explain the phenomenon (in most occurrences). Black swans are “unexpected” not because some other thing happened, but because the thing you thought would (even if it’s “no event”) did not. That is, of course, saying the same thing, but it puts the emphasis where it should be: on your original model of the event.

Back to the list. Rare events don’t seem to arise just from the sudden appearance of a previously unknown factor. Rather, it seems like there are future states of the world that effectively reorganize and re-weight all some set of the factors you’re already sort of thinking about. It’s not much help to just say, “well, think about factors which may be particularly sensitive to futures that look different than the one I’m expecting” because, you know, different how? Unsatisfactorily, I think the solution most people give to this problem is a sort of dodge. They invoke “resilience” which is just to say, “we will necessarily fail to articulate all the ways in which the future may be different in X domain, but we think we’ll be able to say all the ways that a failure in X domain can manifest itself in Y domain, and we’ll go about making sure all those ways can be mopped up.” So, in planning, it’s fantasy documents all the way down.


Socal Rhino 09.15.14 at 4:00 pm

I think the key point is the distinction between risk (can be modeled) and uncertainty (can not) and the need to build in resiliance to better prepare you to deal with events, positive or negative, that cannot be predicted. As in, keep a rainy day fun even in Southern California even during a drought.


mud man 09.15.14 at 4:05 pm

I think the problem is in constructing universals, which Russell as well as Godel showed to be futile, even for aspiring foundationalists/positivists. We can have useful observational regularities with varying degrees of trustworthiness, and indeed we make a lot of hay with such.

Thinking of crows rather than swans for some reason, someone wishes to demonstrate that all crows are black. It being a rainy day unsuitable for looking for crows, we can stay indoors and begin a program of looking at all non-black objects to see whether any of them are crows. Logically sound procedure although silly, it depends on knowing what we mean by (and having access to) “all objects”.

Thomas Nagel’s recent “Mind and Cosmos” makes a good start against this kind of reductionism, but then he got bogged down in how the evolution of “rational thought” enables us to make absolutely veridical statements about reality-in-itself. Or I got bogged down. Anyway, Analytics is by definition not Synthetics.


T. 09.15.14 at 4:55 pm

(I don’t know how much of this you already know, so I apologize if I’m talking down to you. Then again, I only have a Master’s, so you might want a second opinion anyway.)

You’ve sort of got it backwards: cladistics doesn’t start with a conserved body plan and work up, it starts with observed characters and works down.

The idea that taxonomic groups should be monophyletic, i.e., descended from a single common ancestor, was already well established by the time cladistics came around. The defining attribute of cladistics was the rejection of any criterion of overall similarity as taxonomically meaningful. Cladists asserted that taxonomic groups should be based only on shared derived characters, or synapomorphies(*), attributes which arose in the most recent common ancestor of a group and are shared by its descendants. No other types of characters could have any kind of taxonomic meaning, as they could provide no evidence of monophyly.

Of course the immediate objection here is that it’s difficult to know which characters are synapomorphies without some prior idea of how evolution proceeded, so the first step in erecting a cladistic classification scheme is to construct a phylogenetic tree. The cladistic approach to tree-building, in turn, runs as follows: first, one creates a huge table of all the character states (black feathers, white feathers) for all of the taxa under consideration, and second, one generates successively larger groupings by looking for
shared character states, ending with a group described by character states uniting all the taxa. Each set of groups generated in this way constitutes a tree, and the tree that features the fewest transitions in character states— the most parsimonious path that evolution could take— is considered to be the best one. This still leaves us with the problem of determining which direction evolution proceeded in, as a change in character state doesn’t intrinsically convey any information about which states are ancestral and which states are derived. (Did black feathers come before white ones, or did white feathers come before black ones?) For that, one has to supply some outside information in the form of an outgroup, a taxon which you know is distantly related to all the taxa you’re interested in creating a tree for and thus probably possesses ancestral character states(**).

(In practice, systematists don’t use this method, known as “maximum parsimony,” very often anymore. These days, it’s more commonly a matter of using maximum likelihood or Bayesian methods to find the most likely or most credible tree given a set of gene sequences and a statistical model of sequence evolution. I suspect that this might eventually be a problem, as “turtles are archosaurs because my Bayesian analysis of a concatenated dataset of 234 genes partitioned by codon position says so” is great phylogenetics but largely incomprehensible taxonomy; it doesn’t provide a good guide as to whether something is a turtle or not.)

The point of all this is that for a strict cladist there is no such thing as a “body plan” per se. In principle, there is no sense in which you start with conserved features at all— rather, you start with your massive table of characters and find out what’s conserved and what isn’t. Cladistics strives to make no assumptions about whether feather coloration, or anything else, is a good taxonomic indicator for birds. This notion is closely related to the argument that “phyla aren’t real”— yes, the major groups of the Metazoa do appear to have clusters of seemingly basic morphological characters, but so what? Who are you to say what’s basic and what isn’t? Aren’t assertions like “it’s very important whether your skeleton is on your inside or your outside” just human prejudice, as far as evolution is concerned?

So, to answer your question, a cladistic approach operates by not making any assumptions about the color of unobserved swans. It merely provides a reliable guide for redrawing your taxonomic categories after you’ve found swans that are different from the swans you’ve seen before. Cladistic classifications do comprise hypotheses about the history of life on Earth which can be falsified by observation, but those aren’t hypotheses of the form “all swans are white,” they’re hypotheses about what swans are most closely related to. Although it may not be strictly epistemologically correct, you can gain confidence that your tree (and thus your classification) is a good one by observing how stable it is as you add more characters and more taxa; if it stays more or less the same as you gather more and more information, then it’s probably right.

I think that the kind of taxonomy you describe is closer to the world that predates cladistics, the “evolutionary systematics” of biologists like Ernst Mayr. (I have to add the caveat that evolutionary systematics was always rather vague, so far as I can tell, so it’s hard to say that anything in particular resembles it.) Mayr argued that taxonomists should consider similarity as well as ancestry when defining taxonomic groups. Thus evolutionary systematists would consider Reptilia a valid grouping because the reptiles are clearly much more similar to each other than they are to birds. If you use this kind of criteria in your taxonomic system, then judgements about the whiteness of swans, or the degree to which some feature is strongly conserved, greatly matter.

*Cladistics has a lot of specialized terms, like “zygotaxon” and “tokogenetic.” I’m trying to elide them, on the grounds that it’s not necessary to know them if you don’t plan on doing systematics.

**I never really got this until I encountered Joe Felsenstein’s description of outgroups in Inferring Phylogenies, in which he says that using an outgroup to root a tree “amounts to knowing the answer in advance.” You can also use a molecular clock to root a tree, but that’s another story altogether.


Robespierre 09.15.14 at 6:03 pm

I think J Thomas pretty much has it right. We may not be able to foresee all possible problems, but we can design our systems so that they will not fall apart when things turn out different from our expectations.


Bruce Wilder 09.15.14 at 6:36 pm

It seems to me that Nickp @ 9 has the right of it, if we’re taking the example seriously. You actually want to commit to “All swans are white” because the goal is to notice when you are wrong, to notice that a black swan event has occurred when it occurs. The fundamental precept is that all information derives from differences; the point of a plan or a budget is not to prevent deviations from plan or budget, but notice the deviations. And, the secondary precept is that all learning is from error.

For this reason, I would question whether sPh @ 12 is entirely fair in attributing the superhuman as a premise to the schemes of ISO multiple sigma quality control and the like. If I’m trying to prevent, say, airplanes from crashing, I may engage in seemingly odd prescriptions, like calculating how far apart the prescribed routes taken by planes crossing the Atlantic should be, such that eastbound planes “never” collide headon with westbound planes. The distribution of actual routes taken around the prescribed route will follow some distribution, because following the prescribed route is a more-of-less controlled process. Controlling that process of following a prescribed route is very, very complex, combining seemingly incommensurate elements, ranging from the technical limits of various instruments for navigation and control of the flight path to the effects of weather to the psychiatric health of pilots, solar storms or acts of war.

Simply measuring the deviations of actual flight paths from planned flight path and guestimating the distribution is not an entirely idle exercise, I would suppose. Looking at the variation in a sample of actual flights gives some idea of how large a margin of safety is embedded in a certain routine prescribed distance between eastbound and westbound. It might be considered a fairly grave error to assume for the purposes of estimation that the distribution was gaussian rather than cauchy, and it would also be a grave error, certainly, to not monitor the system from a meta-level, looking for drift in the complex interactions, which produce seemingly stable distributions around prescribed flight paths.

The epistemological question has a clear answer: we learn from error and, in a sense, only from error. The moral question is whether we also have to have disaster, an error with a negative value. Does the feedback loop have to entail a catastrophic loss to motivate learning?

With some given number of transatlantic flights, any prescribed distance between eastbound and westbound comes with an estimable probability of mid-air collision. Whether it is better to say, “given that the system works within normal parameters” or not, as a qualification, is difficult to ponder.

If you are not looking, and looking with some structured understanding including prescribed values and expectations, you are not going to notice the emergence of a previously unknown unknown, transforming it into a known unknown (and eventually, perhaps, a known variable controlled within certain parameters). In aviation — I don’t know the story well enough to repeat in detail, but some other commenter might — I beleive it was the development of instrument landing systems that led to noticing previously unimagined wind shear events, some of which merely scared pilots and some of which killed pilots and their passengers.

The moral — or if you prefer, the economic — question is whether people had to die to motivate learning.

Errors have to occur. But, errors occur because you have ventured a structured falsifiable hypothesis, so to speak. You’ve made a plan, that a god can laugh at, and when you get the joke, when the punchline arrives, you learn.

What JQ proposes — whether it would make sense to adopt a meta-imperative governing the design of hypotheses with the aim of reducing the number of dis-confirmations — seems to me to be about the economic costs, where “learning” is motivated by disaster.

Discovering black swans, presumably was rewarding. It wasn’t a disaster, which any one should have wished to avoid. In that sense, Taleb’s choice of metaphor was ill-advised. Applied to financial catastrophe, it distracts us from the clever ways the guilty avoid just retribution by frightening the horses and hiring corrupt pissants to play Chicken Little, but that’s another topic.


Bruce Wilder 09.15.14 at 6:44 pm

In the comment above, I meant “the development of instrument landing systems” to signify the development of a highly structured, mechanical model of landing an aircraft encompassing some finite number of circumstantial variables. In practice, it was found necessary to develop ways to detect the possibility of wind shear as an additional possible circumstance, after discovering a way in which the initial model failed.

I don’t know if I needed to explain all that, but there it is.


David of Yreka 09.15.14 at 7:19 pm

If asked such a question my first comment would be that this usage of “black swan” is a metaphor and not a particularly good one. Does it encode the idea of a series of somehow similar events occasionally punctuated by dissimilar events? Or does it encode the idea of something truly unknowable in advance? Or is its purpose to obscure and confuse, as one might imagine a device intended to enable its user to evade responsibility for failing to take proper precautions in the face of reasonably predictable disasters? Bad metaphors have their uses in political speech, which I guess is a just a special case of all of Orwell’s art being propaganda, but that does not make them good.

Second, and more to the point, you might want to think of the coloration of swans in terms of causally significant networks. I suppose that from a swan’s point of view, being white would be something like a badge of tribal identity. Suppose for the sake of argument that something useful is gotten, anyway. As I would put it, the causal network associated with whiteness is (1) caused by genetics, and (2) whiteness enables some kind of IFF-recognition. That is important in a social species: I won’t argue. But if I were a swan I might think it much more important that I have properties like being warm-blooded, and having wings, feathers, a heart, digestive system, eyes, and so on. These features are also consequences of genetics, but they participate in many more causal relationships (most poignantly: being alive and a swan), and so disrupting them would bring you great misfortune indeed.

In more abstract terms, then, we might imagine a directed graph where nodes represent features and causal relationships are represented by arcs. A feature that causes no other feature, i.e. has no out-arcs, is a feature whose modification has no consequence. On the other hand, a feature with many links to and from other features is one that you’d hesitate to disrupt: changes would propagate in possibly unexpected and deleterious ways. And the denser the network, and the more poorly characterized and nonlinear the relationships, the less you would be able to predict the outcome.

In passing, I note that the case of a cyclic graph would make me feel cautious. Such graphs can indicate feedback loops, which can be self-stabilizing, explosively unstable, metastable in some number of discrete states, and possibly all of the above, if you will stipulate sufficiently large, nonlinear, time-varying, and densely interconnected graphs.

So how to avoid being blindsided? Understand the causal relationships and try to imagine the ripple-on effects of changes at the nodes. Which nodes should one focus on? I would say, start by looking at nodes that are in dense, cyclic subgraphs with strong causal relationships to other nodes that you care about. That’s one heuristic. Another might be, look at nodes whose causal relationships are difficult to quantify, but which have (possibly) strong relationships to other nodes that you (transitively) care about. How many genes cause a swan to be either white or black? I have no idea. How many aspects of swan lifestyle are facilitated by their color scheme? I have no idea. Is the color of one’s feathers as important as having a functioning heart? I suspect not.

Anyway I still think it’s a bad metaphor.


fivegreeleafs 09.15.14 at 8:19 pm

I’m interested in the question of how you can prepare for such contingencies given that, by definition, you can’t foresee exactly what they will be

The question is: what principles would help you to avoid making, and acting on, the assumption

Not an expert by any standard, (and not familiar with your area of expertise), but very interested in the subject…

Since you already are using (if I understand you correctly), biological metaphors and analogies as scaffolds, may I suggest another one: the (adaptive) human immune system?

If there ever were a system “designed” to handle “unforeseen contingencies” (within its domain), this must surely be one of the better candidates. The complexity is truly daunting, and the learning curve steep indeed (depending on your background of course), but the richness of this canvas (when mastered), is awe inspiring.

It seems to me that the crucial fact here is that the shift from black to white, or vice versa, is, in evolutionary terms, a small one

It might be, but then again, it might not… here there be dragons me thinks :)


John Quiggin 09.15.14 at 8:41 pm

This is all very helpful thanks.


john c. halasz 09.15.14 at 9:11 pm

Can the Ebola virus genetically mutate into an air-borne form? ISTM that is a currently live debate, with experts lined up on each side, which exemplifies JQ’s puzzle, and might be worth attending too. (With a nod to @22).


shah8 09.15.14 at 9:47 pm



shah8 09.15.14 at 10:10 pm

Aside from the joke…

Black swan always was a very poor metaphor, geared for the scientifically and rhetorically innumerate. Applying cladistics to a poor metaphor does not help in a situation where you don’t know whether you’re overdetermined or underdetermined in thinking about any one topic.

It’s better to think in terms of , where sometimes, the rules don’t work the way they’re supposed to–in a way more more cogent to N.N. Taleb’s point. You applied local expertise, it always worked well, and then you move somewhere else, and that Volvariella volvacea turns out to be Amanita phalloides and you wind up having to have your stomach pumped, even though you can tell Amanita caesarea from the far more beautiful Amanita muscaria.

The moral of the story alway was that damn…good to have a well taxed society with ambulance and hospitals when you need ’em, hey? You’re supposed to think about residual risk when you read that book!


Omega Centauri 09.15.14 at 11:05 pm

There are things we didn’t imagine, and there are things we have no problem imaging but dismiss as impossible or simply highly improbable. We can imagine things like the universe going through some sort of big-rip transformation which suddenly makes life (or even atoms) impossible, but that doesn’t mean there is anything we can do about them. There are also low probabilty events we know about, but dismiss from our planning. We know a super-volcanoe eruption could upend our economic plans, but we don’t plan for it for instance. I’m not sure the class of completely unknown unknowns is of greater concern/menace, than that class of known -but only in a statisical sense events that could be catastrophes should they occur.


Mike Furlan 09.16.14 at 12:24 am

Also consider the idea of things that are robust, and things that are anti-fragile. Taleb has as much as you would want on that topic.


nnyhav 09.16.14 at 1:36 am


Guano 09.16.14 at 4:05 pm

“In particular, I’m interested in the question of how you can prepare for such contingencies given that, by definition, you can’t foresee exactly what they will be.”

You can ask yourself occasionally: What assumptions am I making here? Is there a possibility that these assumptions are wrong? If these assumptions are incorrect, how big is the risk?

Taleb talks a lot about financial models that have been derived from statistical relationships. If we don’t know why X is correlated with Y we have difficulty in knowing much about those occasions when Y does not follow X. And we don’t think about the magnitude of the impact when Y does not follow X.


TM 09.16.14 at 6:01 pm

All I can contribute is an old joke. An engineer, a physicist, and a mathematician take a train journey in a foreign country. While they look out the window, they see a herd of brown cows standing on a meadow. Says the engineer: “Aha, in this country, all cows are brown.” Says the Physicist, rolling eyes: “Silly engineer, you cannot conclude all cows in this country are brown, only all cows on this meadow are brown.” Says the mathematician: “Silly physicist, you cannot conclude that these cows are all brown, only that they are brown on one side.”


Peter T 09.16.14 at 11:12 pm

Is it fair to say that probability likes to look at single events, or at aggregations of events, assigned to classes. Certainty (or uncertainty) tends to look at structures – how things fit together, what the connections are and how much room there is for movement. It’s closely tied to what degree of understanding one has of the empirical situation. Staying with swans, it’s fairly obvious one thought about that plumage colour in birds is easily varied (even in swans, as cygnets are grey), so a black swan is not a big leap. A four-legged swan is a very big leap, one we would now hypothesise as most likely a genetic defect unlikely to be viable. A flying swan above a certain size we now know to be an impossibility. In old Ireland, people must have listened to the tale of the Children of Lir with a delicious sense that people could actually be turned into swans – after all, magic was attested as real in the past and in far-off places (Icelandic sagas are for the most part firmly grounded, but take the odd bit of magic in as factual a way as they deal with meals and fights).

Absent a good understanding of the facts and the structure, our judgements of the possible are unmoored. Green Lanternism in politics is one example. I don’t know if such an understanding is amenable to general rules and calculation. My sense is that it’s not: it differs from domain to domain, and each has its own issues.


js. 09.17.14 at 1:20 am

I’m not really sure what question you’re asking, and so also what kind of answer you’re looking for, but there’s been a _ton_ of work done on natural kinds. A lot of this work is inspired by Kripke and Putnam (who I should note mostly focused on examples from chemistry, like “gold” and “water/H20”—largely because they’re way easier one would think), but there has been plenty of stuff done on biological kinds as well—the first link has a section on biological kinds in particular. (This kind of thing was never really my focus, so my knowledge about it is at grad-seminar level vs. actual research level.)


Dr. Hilarius 09.17.14 at 6:45 am

I have no idea how to formalize Mr. Quiggin’s question (or a solution) but note that unforeseen contingencies are sometimes unforeseen only because those doing the planning have a limited viewpoint.

Consider an earthen dam. It’s been engineered to withstand floods of great magnitude, wind, ice, vehicular traffic across it, and even terrorist acts involving explosives. But no one considered burrowing rodents. A biologist might have asked “what about those nutria digging into the dam” but the engineer’s world did not contain rodents.

In the medical world, black swans are called zebras. As in “when you hear hoofbeats, think horses, not zebras.” Usually a sound position but if unthinkingly adhered to it can mislead. I had co-workers frustrated in dealing with physicians, who refused to deviate from the usual differential diagnosis pathways, even when told that they had been in a tropical country and had been in contact with unusual species (including blood and excreta). No zebras here, horseman pass by.


Guano 09.18.14 at 10:22 am

Dr Hilarius – ” ….unforeseen contingencies are sometimes unforeseen only because those doing the planning have a limited viewpoint.”

Precisely. In fact, I thought that this was Taleb’s main point: that many practitioners (especially economists) apply correlations and approximations without considering their boundary conditions (and with a great deal of blind optimism and wishful thinking). They are protected by many of their peers who say “no-one could have predicted that” while those who really understand the field have indicated precisely those risks.


Billikin 09.18.14 at 8:03 pm

There are two related questions (at least) regarding black swans. One is the question of whether they exist, given that they have never been observed. The other is the question fat tails of probability distributions. I have not studied the latter.

As for the former, a lot of people make the mistake of looking at whether blackness (or whiteness) is associated with swanness. That is a different question. From a scientific point of view, we are interested in observations. Once we have decided what an observation is, we can ask this question:

Given that this is an observation of a swan, what is the probability that the swan is black (or perhaps, not white)?

I was originally inclined to that approach, but eventually came around to the logical question, which is also Hempel’s (as in Hempel’s Raven):

Given that this is an observation, what is the probability that it is of a black (non-white) swan?

It is the job of the scientist to approach the first question by setting things up so that the probability of observing a swan is greater than usual. Answering the second question by sitting in your armchair is not science. In fact, to answer the second question properly you have to set things up so that he probability of observing a non-white swan is as great as you can make it.

In any event, whichever question you attempt to answer, the main point is the same: confirmatory evidence is very, very weak.

A few years ago I was thinking about Hempel’s Raven again, from an evolutionary perspective instead of just from the perspective of logic and probability. I realized that from that perspective non-black ravens almost certainly exist or have existed. A brief web search revealed that to be the case. :)


Blissex 09.18.14 at 10:40 pm

Curiously I posted the following yersterday (comments #29) and it has duisappeared.

«unforeseen contingencies, popularly, if ethnocentrically, described as “black swans”»

If I understand Nassim Taleb correctly that’s not how he characterizes “black swan” events, and is it indeed very far from his conception of “black swans”.

«In particular, I’m interested in the question of how you can prepare for such contingencies»

I dearly hope that our poster has read the somewhat fuzzily written “Antifragile” book by Nassim Taleb. That his is reply both to “black swans” and what you are asking, except that is now ambiguous:

«such contingencies given that, by definition, you can’t foresee exactly what they will be.»»

OOPS, major bait-and-switch here: you have talked of “black swan”, “unforeseen contingencies” and “can’t foresee”. As to the latter two it seems to me a confusion between “unforeseen” and “unforeseeable”, where the difference between the two is pretty gigantic.

«what principles would help you to avoid making, and acting on, the assumption “all swans are white (or, in my own case, black)”.»

That’s a different topic again. That’s the ancient problem of extrapolation, which is based on the alternative between making sharp predictions and accepting that they may be wrong, or making vague predictions that are never wrong.

«One example, with which I’m very pleased, is that of the precautionary principle. It seems reasonable to say that we can distinguish well-understood choices involving hazards from those that are poorly understood, and avoid the latter, precisely because the loss from hazard cannot be bounded in advance.»

“Antifragile” by Nassim Taleb has extensive discussions of this and much more.

Comments on this entry are closed.