Against the Copernicans

If I know something special about each of the three cases I mention, then I know something special in the salient sense about pretty much everything. So the formula is useless.

But I have no idea how I am supposed to know anything special in the generic case at the end. All I know is that the event E3 has been running for as long as the event E2, which follows from their definitions. Is knowing the definition of an event ‘special information’? In that case the formula really can’t be applied. I can’t apply it to show running times if I know what it is for a show to be running. This makes the formula utterly trivial. In that case it might not be false, but it is even more useless than I thought.

The interesting applications of this type of reasoning come with the expected survival time of the human species, no? For instance, a natural extension is to suppose that you are a randomly chosen individual from the set of all human beings who will ever live, from which you can (if you like) draw some fairly strong conclusions about the expected number of human beings in the future.

There’s some interesting overlap with Fermi’s paradox here as well.

Actually there is a connection between Bostrom and Gott’s reasoning. In both cases they rely pretty heavily on a principle of random selection, and draw very strong conclusions from it. Arguably some forms of Bostrom’s arguments rely on something like a Copernican principle as well. But all these arguments seem completely absurd to me – you can’t get important information, like the probability of extinction, from as little information as they provide.

the slip is clearly from “randomly” to “a random draw from a uniform distribution”, because this is the point at which Monton appears to have circumvented the normal licensing procedure for taking expectations.

In fact actually I don’t think that you can take a random draw from a uniform distribution without “knowing something special” in the sense Brian means in 2, because this would be equivalent to saying that you could pick out a specific point on a continuous line without any identifying characteristics at all. I can feel myself shading into my general war against the Axiom of Choice here though.

Isn’t the issue with the Australia example that it is not a stable process, but a process showing exponential growth over time?

I always understood the “when will the world end” questions as based on a stable process which had a constant P(end of the world) on any given day. Or the Berlin wall classic example as having P(Berlin Wall coming down) that was the same on each given day (or week, or whatever observation period seems sensible).

It strikes me that the question “will the population grow or shrink in the next decade?”, which does not describe a stable process (rather it describes a roughly exponential growth) is a very different question, proabalistically, than, say, “will the weather be cloudy a month from now?”, which describes a random but stable process (i.e. the chances of cloud on a given day are roughly the same for any given day).

Does that make any sense to the people who understand stats better than I?

That’s it, dsquared. You’ve gone to far this time. Against the Axiom of Choice? You’re going down.

Probability is a guide to life. My state of ignorance about the future of the climate shouldn’t be taken by anyone (including me) as a precise guide to much of anything. Hence it shouldn’t determine a precise probability.

The whole non-ergodicity/fundamental uncertainty thing, right. I’m sure this is correct … but I’d like a better explanation exactly why.

This fellow’s thesis was the subject of a New Yorker article several years ago, which took it very seriously. It was ridiculous then and it was ridiculous now. As Brian shows, it gives absurd results. But more importantly, it NEVER has any useful predictive force.

Imagine a phenonemon whose life span was unknown: the Berlin Wall, say. We begin making monthly observations of the Wall from the time it was erected in August 1961. We make our calculation as to its life expectancy, which each month becomes longer as the Wall’s lifespan lengthens. On November 1, 1989, we make our observation and predict with 95% confidence that the Wall’s likely lifespan is between 8.1 months and 1053 years. On December 1 we return and – the Wall is down! Where did we go wrong?

Well, it is true that if we look at our data, we find that, except for the first three and the last seven months, our prediction did accurately bracket the life of the Wall. This is by definition. So Gott would say that his method worked. But it tells us nothing that we need to know. The Wall could have come down in any given month, or be standing still, and the calculations would be proven accurate to within 95% overall while always being incorrect as of the month prior to the fall of the Wall. So the method will always mislead us about changes in future events. If we rely on it, we will always be surprised when an event ends.

Note, by the way, that Gott games the system with his use of examples. Broadway shows, restaurants, and heads of government in parliamentary systems share a common characteristic: they are very unstable early in their existences, but gain stability for a long middle period. So if you apply his method to them, they appear to prove it. But if you apply his method to things that have a different lifetime trajectory – human beings, say, or American presidencies – they will not fit his model nearly as well.

“Note, by the way, that Gott games the system with his use of examples. Broadway shows, restaurants, and heads of government in parliamentary systems share a common characteristic: they are very unstable early in their existences, but gain stability for a long middle period. So if you apply his method to them, they appear to prove it. But if you apply his method to things that have a different lifetime trajectory – human beings, say, or American presidencies – they will not fit his model nearly as well.”

Posted by Bloix

This sounds like an exponential distribution for the survival time (‘gain stability for a long middle period’). For the exponential distribution, an interesting characteristic is memorylessness – increasing observed survival times don’t affect the expected future survival time.

“So using the Copernican formula, we can deduce that the conditional probability of A still running at z given that B is still running at z is 1… the conditional probability of A still running at z given that B is still running at z is 1. So I’m sure there is something deeply mistaken with the Copernican formula.”

No there isn’t. The formula assumes that your position in time is a random sample during the lifespan of the object you observe.

The *only* situation in your counter example where this is true is when E1, E2 & E3 all start and end at the same time. In which case the conditional probability of A still running at z given that B is still running at z is 1 for obvious reasons. It’s a tautology – your application assumes they all start and end at the same time so it’s not really a surprise that A’s always running given B is.

It’s a valid, though not particularly useful, technique for deriving some kind of estimate in cases where you know next to nothing about the situation. The problem is that people keep insisting on applying it to situations where the real problem is not a lack of information, but a lack of any good idea how to apply it.

For instance, we have an absolutely enormous collection of information which is, presumably, relevant to the lifetime of the human race. Biology, geology, stellar mechanics, rates of technological progress, practically everything we know has some relevance.

We just don’t know how to do the calculation. That doesn’t mean the Copernican priciple is applicable.

I haven’t thought too much about it, but if the running of A and B are independent events, then Prob(E3) = Prob(E1) Prob(E2). I’m not sure what the mistake is in your calculation, maybe you cannot assume that the time of observation is a random variable and then use the same time for both A and B.

Suppose you’ve been imprisoned by the king and told that you will be executed some time in the future, and that the day of your execution will come as a surprise to you. The king knows you’re a good statistician, so you conclude that you’ll be executed on a day when your likelihood of dying that day is less than 5 percent. On what day will you be executed?

Whenever you make statistical calculations without knowing the underlying mechanics of thing you’re studying, you’ll get screwy results like these. I could sample random wavelenghts of light coming from the sun, assume they must follow a bell curve and conclude that our sun will run out of energy in 10 years. As it turns out, assuming Guassian distributions in black-body radiation lead to catastrophic (and wrong) conclusions.

Likewise, the Copernican principle applies to measurements of the event, not to the fact of its existence. To assume that the existence of everything is a stochastic process that flips from true to false with some fixed probability is, as you demonstrate ridiculous. It also throws out a lot of information about the event in question.

“It’s true that you should only be able to derive something like my result in cases where we take it as given that E1 and E2 end at the same time.”

The problem more fundamental. If we observe Plays A and B are both running – and we know Play A started before Play B – then we know our observation isn’t a random sample from within the life span of Play A. Because we can’t have observed the duration of time when Play A was running but Play B wasn’t. So we’ve no business using the formula to calculate Pr(E1 at z). The formula does not apply to E1.

“The principle is meant to give us probabilities of the duration of things of unknown length. If we only apply it when we know how long the event is, well then it will give plausible results, but won’t tell us anything we don’t already know.”

I don’t want to sound like I’m being too picky about something you took three seconds to type into a comment box, but I think that phrasing misses an important distinction. Your claim is different to Gott’s. There’s a difference between: (A) knowing how long an event is, and (B) knowing that you’ve a random sample from within the lifespan of the event.

B’s is significantly different to A. You can know you’ve chosen a random raffle ticket without knowing how many tickets are in the raffle. In this different context you can use Gott’s maths to infer the number of tickets. All we need to know is that we’ve randomly sampled them.

Obviously, that kicks the debate back to arguing for or against the validity of the random sampling assumption. I think John’s wrong that Gott’s suggestion is just the principle of insufficient reason. I think Gott would argue that in some cases you can have positive reason to believe that your observation is a random sample – from knowledge of the nature of what your observing and the circumstance of the observation – and this justifies applying the formula.

leederick :

So we’ve no business using the formula to calculate Pr(E1 at z). The formula does not apply to E1.

I also thought at first that the time of observation could not be random for both A and B, but this is not so as long as the time was not chosen to be in the lifetime of A and B, it just happened to be so. Otherwise, whenever I observed an event E, someone else could argue that the time was not random, simply by observing another event that started after E. Imagine that the observations of E1 and E2 were done by two different persons, each one not knowing anything about the other play. Then the first person, observing E1, would apply the Copernican formula; whether he can apply it or not cannot depend on whether some other person is observing another event.

Besides, even if you leave out E1, you are still left with the problem that Pr(E2 at z)=Pr(E3 at z).

If it’s wrong, for reasons of excessive knowledge, to apply the principle to E3, then I can’t see why it would be OK to apply the principle to E1 and E2. After all, we know something about E1 and E2 as well – namely that they are entailed by E3.

In any case, I certainly wasn’t *assuming* that the events were independent, or that either distribution was uniform. I was just applying the principle given; a principle I think is utterly crazy. The principle says that in the absence of information to the contrary, we can assume we are dealing with uniform distributions here.

Now either knowing the logical relations between E1, E2 and E3 counts as information to the contrary, or it doesn’t. If it does, then we can never apply the principle, because we always know that two events collectively entail, and individually are entailed by, their conjunction. So the principle is vacuous. If it doesn’t, then the principle leads to an absurd result. So we get the conclusion that it seems to me we always get with these principles saying we can derive something from ignorance, either vacuity or absurdity.

The mistake is so simple I don’t know how we’ve managed to made such a meal of it.

Gott’s method is to make a frequentist statement about tfuture based on the long run properties of a random sample.

What was the first thing any of us were taught about frequentist statistics? You determine what hypothesis you’re going to test, then you look at your data. The example supposes we’ve looked at the data, seen that both plays are still running, then decided to do the calculations for E1, E2 and E3.

This means the results for E1 & E2 are wrong. We wouldn’t be worrying about this example had Play B not yet started, or the play’s not overlapped, or had Play A ended. So the probability statements about E1 & E2 can’t be justified based on the long run properties of random samples that Gott uses to contruct his formula. Our running these tests have been influenced by our looking at the data.

I think E3 is the only circumstance in your example where it’s sound to apply the principle. You can’t apply it to E1 or E2.

I didn’t like the counterexamples, but I don’t like the original reasoning either. I’ll repeat it first in case I misunderstood.

Choose that has a distinct beginning and a distinct end. Note the beginning. Then at some random time between the beginning and end, note that it has not ended yet. Since we assume the time was random, we can suppose that it’s a sample from a uniform distribution. There’s 1/2 chance it is in the first half and 1/2 chance it’s in the back half, so that’s a 50% chance the end will come in less time than has already passed, and a 50% chance it will last longer than it’s already lasted. By the same reasoning there’s only a 5% chance that it will end in less than 1/19 the time that’s already passed, and a 5% chance its total life will be 20 times or more what it’s already survived.

This bothers me somehow. If it’s right I should be able to get the same result with a different method. So — If the interval is some unknown N, and I assume nothing about N except that it’s larger than my observation A, what is the maximum entropy distribution? N is in the interval (A, infinity) and I’ve assumed nothing else about it, so I must suppose that every value in that interval is equally likely. So any value I choose for the mean is too small. I get a different result.

Let me try again. In a finite universe there can be a lot more material things that last a short time than that last a long time. The longer something lasts the longer it takes up mass and space that something else could use. There’s only room for so many long-lived stars. There’s room for a lot of short-lived tritium atoms. So maybe I shouldn’t assume that every lifetime is just as likely.

If we make the assumption that there is some unknown mean duration M, and nothing else, then the maximum entropy distribution is an exponential. And if we remove all the cases where x

j thomas: it is worked out in the paper of Monton-Kierland cited in the original post: Gott’s conclusion follows if the assumption on the distribution is that Pr(N) is proportional to 1/N .
You are correct in saying that without this assumption, you can’t conclude anything.