The Logic of Yogic Discovery

by Daniel on July 27, 2006

As I posted over on one of my other blogs, one’s first reaction to this paper is horrified amusement that it got printed in a reasonably respectable journal. The authors are mainly from the faculty of “Maharishi University” and it’s a study of the efficacy in reducing the frequency of terrorist incidents in Israel and Lebanon of installing a group of people practising Transcendental Meditation. It is, to be honest, pretty whacky stuff, although my personal opinion is that the meditators get the best of the methodological debate which followed (really, the yogis were not pulling any statistical funny business and they did find a significant effect; it’s discussed in this rather good article on statistical methodology generally)

But really, who is in the wrong here?

Second thoughts on this topic and a brief discussion with Henry have left me thinking that it is rather shameful that my first instinct was to think that the Maharishis ought to have been banned a priori from publishing articles in a reputable journal simply because they are Maharishis. This goes back to a lot of the issues that we discussed with respect to Steve Fuller’s article in the Chris Mooney seminar, which annoyed so many of our readers. My reaction to the Maharishi paper is exactly the sort of prejudice that shouldn’t exist in a scientific journal – it is clear from the paper (and I am afraid you will have to take my word from this unless you have a subscription to JSTOR, but I have read it and I am not intentionally bullshitting) that the Maharishis have gone not just some of the way to meet science in the middle, but all the way to meet science on its own ground. There really is nothing in the methodology of the paper that would give any grounds for rejecting it. The only grounds on which one would not want to publish this paper are that it is claiming that you can bring about world peace by sitting in a room going om.

I don’t know as much about “intelligent design” as Steve or some of the people who disagreed with him in that debate, so I don’t know whether the ID community has any members who are prepared to get involved in the scientific debate in the same way that the community of interest in the Maharishi field effect were in 1990 [1] (I do of course know all about the ID practitioners who are utterly dishonest and not prepared to use anything resembling the scientific method, so perhaps we could not bother to go over that ground in the comments?). However, I think that the Maharishi issue shows what they are up against; as far as I can tell from a few minutes with Google Scholar, the JCR article is basically the only article ever published in a proper journal on the Maharishi field effect (there is a report of an experiment in Washington DC in Social Indicators Research 1999 but then that is it). I think that during a lot of the discussion of the ID crowd, we are probably underestimating the extent to which the “non-core” science types are being unfairly treated by the mainstream of science, for social rather than scientific reasons. This doesn’t make me a believer in creation science or transcendental meditation, but if we are believers in the scientific method, we ought to take it seriously.

By the way, because I am far more of an obsessive single issue nut than any follower of the Maharishi Mahesh Yogi, I will point out that the fact that the “natural experiment” methodology in social sciences can deliver the result that transcendental meditation stops wars, ought to give a hell of a lot more trouble to the fans of Steven Levitt and “Freakonomics” generally than it in fact does.

[1] I think that the Maharishi movement has gone downhill a bit since the 1990s; they don’t stand in UK elections as the “Natural Law Party” any more, and their main big field experiment in Skelmersdale didn’t achieve critical mass (btw, I am not sure that I agree with Dr Geoffrey Clement’s assessment of the data in that interview). They do still maintain the Maharishi Institute of Technology in Fairfield, Iowa, and the Sidhaland community is still going in Lancashire – I couldn’t quite be sure whether it still has the MMY’s endorsement though.

{ 120 comments }

1

Ray 07.27.06 at 5:59 am

The problem is that the Maharishi crowd make this prediction every time they set up a centre, or want some publicity. If TM has these effects, they should be able to say that in July/August they will meditate in Jerusalem, in September in New York, in January in London, etc etc, and each time these indices will go down. If they make all those predictions, and then submit for publication the one city where the indices responded as predicted… it’s like Uri Geller claiming he can fix watches because look, this one is now telling the right time.

2

taion 07.27.06 at 6:53 am

Judging by Schrodt’s comment of “I am not, repeat, not accusing the authors of fraud”, I’d guess that his sentiment was much the same as yours.

Still, perhaps my imagination is limited, but to what extent can ID proponents possibly meaningfully enter a scientific debate? If would be difficult at best to demonstrate that it is necessarily impossible for some structure to have arisen as a consequence of evolution; merely to say that something is improbable at first glance due to something like complexity, specified complexity, or what have you is to say very little at all.

3

Daniel 07.27.06 at 6:53 am

this is basically the thread of the methodological critique in the followup JCR article; that the causation runs from the effect to the treatment rather than the other way round. I think the yogis do a reasonably good job of defending themselves against it in the specific case from the literature. I think the real problem is that statistical methods in social sciences are nowhere near as powerful as people want them to be.

(there is also the related critique that the Maharishis would just never have submitted the study for publication if it had not found the right effect, but if we are going to start hanging people for that then it is basically game over for more or less all of economics and an inconveniently large amount of bioscience, as Deirdre McCloskey keeps reminding us).

4

Steve LaBonne 07.27.06 at 7:50 am

Scientific method != statistical hypothesis testing. (Life-science areas that rely most heavily on it, like epidemidiology, are the parts of biology with the least solid results.) Science is not a matter of applying statistics to any random asinine hypothesis that happens to come along. Discussions of statistical methodology have essentialy zero to do with the reasons why ID is crap.

If all you have is a hamemr…

5

Steve LaBonne 07.27.06 at 7:51 am

(Or a hammer!)

6

Seth Finkelstein 07.27.06 at 7:52 am

Sigh. “Extraordinary claims require extraordinary evidence”.

That’s the missing piece in focusing purely on methodology. Let pass somebody’s trivial experimental paper which is probably a statistical artifact, because hey, nobody’s reading it anyway, and they need the publication record, and suddenly, other people want to know why the morphogenic field of tranquility produced by chanting “Om” isn’t the same as, e.g., drug effectiveness testing, because they both have statistics.

Look, why is it so hard to deal with the fact that if someone sounds like a snake-oil peddler after you’ve already seen a dozen snake-oil peddlers, people don’t want to waste their time? It’s not orthodoxy, it’s lack of patience for slogging through yet another crank or religious fanatic’s fantasy, where you know they aren’t going to believe the refutation anyway.

7

Daniel 07.27.06 at 8:09 am

I don’t agree that “extraordinary claims require extraordinary evidence” in order to get published. This seems like a very restrictive standard to me, and one that is clearly open to gerrymandering by whoever gets to decide what is extraordinary.

8

SamChevre 07.27.06 at 8:16 am

taion,

As one of the few regular commenters who has some sympathy for ID, I will try to answer your question, “to what extent can ID proponents possibly meaningfully enter a scientific debate?”

It isn’t easy. The basic issue is defining which questions can be answered scientifically, what a scientific answer involves, and what a scientific answer excludes.

The big question is, “Can a scientific explanation rely on an intelligent-but-unknown actor?” Secondarily, how can you figure out when that explanation is needed?

If I asked you to prove that your car was (or was not) designed by people, how would you do it–assuming you had a junkyard full of cars, but lived in a Stone-Age society?

9

SamChevre 07.27.06 at 8:19 am

I think the comment in 3 is key–and any field (including my own, economics) that can’t survive it is not really properly scientific. If you select the studies published, you can get the results you want. A 95% confidence interval means that 5 studies out of 100 will be wrong–will disproveprove a true null. If you publish all 100, it will be clear–if only the 5 “interesting results” are published, the statistical methodology can be perfect and the results published still be wrong.

10

A-ro 07.27.06 at 8:21 am

You say: “…I will point out that the fact that the “natural experiment” methodology in social sciences can deliver the result that transcendental meditation stops wars, ought to give a hell of a lot more trouble to the fans of Steven Levitt and “Freakonomics” generally than it in fact does.

This strikes me as a cheap shot at something you were already skeptical about. I’m sure there is a literature debating whether natural experiments of various sorts provide believable evidence. Why not cite some of it? (Or contribute to it!)

Why not just be honest and say: “even though this paper provides evidence of a quality level that I would usually accept, I am so skeptical of their hypothesis that I will require more evidence to be convinced on this issue.” It’s not really an unreasonable thing to say. After all, you can “prove” a false hypothesis (to the 95% level) 1 out of 20 times you try. If that reasonably approximates your view (and I am interested to see if it does), why hide behind attacking the entire category of methodolgy?

11

Daniel 07.27.06 at 8:30 am

#4 seems to be simply stating that a particular materialist view is definitive of science and no alternatives can be entertained, which looks too much like destroying the village in order to save it for my liking.

12

Steve LaBonne 07.27.06 at 8:41 am

Uh, no, #4 is about the limitations of statistical methods. (It’s a bit better if you can use a Bayesian rather than a frequentist approach, in which case you can properly incorporate the implausibilty of silly hypotheses in your priors.)

By the way, I detest that word “materialism”, it’s really completely meaningless. If something has an observable effect on the world, what does it matter whether you call it “material”? And if it doesn’t have an observable effect on the world, what does it mean to say it “exists”? But a frequentist hypothesis test supposedly showing that some utterly implausible effect, for which there is no remotely credible mechanism, meets some leve of “statistical significance”, is merely a parody of science and belongs in no journal except perhaps Annals of Improbable Research. There’s a hell of a lot more to scientific methodology than that.

13

Ray 07.27.06 at 8:47 am

The problem with relying on an “intelligent-but-unknown actor” to explain things is that it’s an explanation that doesn’t explain.
Why is that there?
Some bloke put it there.
When?
Dunno.
Why there?
Dunno.
Why is it red?
Dunno.
What’s it supposed to do?
Dunno.

If your only alternative explanation is “Dunno”, then “some bloke” is a bit of an improvement. But not much. Figuring out when that explanation is needed is pretty simple though.

14

Steve LaBonne 07.27.06 at 8:48 am

Also, who put the bloke there?

15

Ray 07.27.06 at 8:53 am

Dunno

16

SamChevre 07.27.06 at 8:54 am

Ray,

It may not explain, but it can still be accurate–that’s the problem.

If I try to explain the remains of Inca temples in Peru as the result of geology, I’ll have a lousy explanation. If I explain them as something people built, for not-well-understood purposes using not-well-understood technology, I would consider that a better explanation–even if it still leaves most of my questions unanswered.

17

Doormat 07.27.06 at 9:01 am

If your only alternative explanation is “Dunno”, then “some bloke” is a bit of an improvement.

This is the key point though: when it comes to “The origin of species” (and, I suspect, varying levels of terrorism in the Middle East) we do have rather good alternative explainations. I strongly suspect Ray was hinting at this when he wrote the above quote. If it wasn’t for strongly held religious beliefs, there would be no reason to even wonder about ID– that’s the real problem I have with it– it’s not something that is suggested to us because of flaws in standard evolutionary theory.

As for statistical reasoning, I’m reminded of the alledged link between Stalk migration and human baby arrival: is this an urban myth? If not, then it shows that stats basically prove nothing unless you already have some mechanism you are testing. Of course, the Maharishis (is that already plural?) have a mechanism in mind, but personally, I’m going to remain skeptical!

18

Ray 07.27.06 at 9:04 am

If you explain Incan remains as something that people built, you can make some predictions based on that. You would expect to find pottery in there, for example. There might be distinctive patterns found on the pottery and also on the walls. You would expect to find similarities between those man-made things and other man-made things in the same period and area. You’d expect to find middens. You’d expect to find lots of human bones nearby.
If your explanation for the Incan temple is “God made it pop into existence”, you don’t have a research project.

19

Steve LaBonne 07.27.06 at 9:05 am

samchevre, the problem is there is just enough content in ID to sink it. The content has essentially been provided by Debmski- the “eaplanatory filter”- nope, doesn’t work, there is a voluminous literature refuting it six ways from Sunday and it’s really just an example of petitio principi anyway; and the infamous “no free lunch theorem” which turns out only to prove that selection averaged over all possible fitness landscapes gets you nowhere, as if anybody doubted that!

20

Daniel 07.27.06 at 9:06 am

If something has an observable effect on the world, what does it matter whether you call it “material”?

It apparently matters a lot to you, Steve, because you (unless I am misinterpreting you) don’t want articles showing that transcendental meditation at a distance has an observable effect on the world to be published in journals.

A decline in violence which was contemporaneous with and (conditional on a reasonable set of potential confounding factors) Granger-caused by an intervention in the form of transcendental meditation, is an observation in good standing and just the sort of thing that ought to be allowed into the scientific literature as far as I can see. “Frequentist statistics” are a red herring here; the statistical methodology is simply being used to confirm that the outcome was large relative to the normal variance and that the Granger-causation occurred – to confirm that this was in fact an observation.

As far as I can see, your idea appears to be that some kinds of theories are such that they should not be allowed to publish supporting evidence. Pushing the criterion into the first word of the phrase “credible hypothesis” doesn’t really make any progress from insisting on your particular materialist idea in the first place.

I take a scientific version of the Webb’s view on founding the LSE on this one. They were left a sum of money on condition that they used it to futher the cause of socialism, and instead used it to found the LSE, using the justification to the executors that a neutral inquiry into social conditions would be the best way of establishing the truth of socialism. Well that didn’t work quite as planned, but it seems to me that science is far better defended against theories like those of the Maharishi by giving them the fairest chance possible than by trying to gerrymander.

21

Daniel 07.27.06 at 9:08 am

(an example of what I mean is given by #18 above; Steve makes a sensible argument against intelligent design, but its plausibility is undermined by the fact that he has admitted in #4 that he actually wants to gerrymander scientific arguments. As far as I can see, this is exactly Steve Fuller’s point about science in a democratic society).

22

Steve LaBonne 07.27.06 at 9:17 am

No, daniel, you’re reading through your peculiar cognitive lenses and not getting what I wrote. 1) Mere statitistics without a reasonable supporting mechanism is in no sense “evidence” in science. Many epidemidiologists have discovered this to their cost; all too many supposed causal links, that had nothing but statistics on their side. have been proclaimed that later had to be abandoned, often at the cost of fueling public cynicism about medical science. 2) Science is not about statistically testing any old observation statement whether or not it connects in any way with the existing body of knowledge. In epistemology I’m a fan of Haack’s “crossword puzzle” model of knowledge- candidates for new knowledge must both be supported by observation and cohere in significant ways with existing knowledge. 3) Repeatedly attempting to pin the useless word “materialisim” (are quarks “material”? Are strings and branes, if they turn out to exist?) on anyone who points out your outsider’s defective understanding of how science is done, even after your target has disavowed that tired and shopworn word, is just a cheap rhetorical move not an argument, and is unworthy of you.

23

Ray 07.27.06 at 9:20 am

re 19 – aren’t you begging the question?
You want this single study to do the work of both demonstrating that TM has an observable effect on the world, and of proving what that effect is.
I don’t think it’s closed-minded and unreasonable to look for two different proofs. First prove that there is something happening, then examine what its effects are.

24

SamChevre 07.27.06 at 9:27 am

Steve and Ray,

To avoid derailing the thread totally, this is my last post on ID unless daniel requests otherwise.

Here’s my basic criteria for proving/disproving ID as an explanatory theory for the universe or some of its features.

The same criteria and technique has to give the right answer in three other cases:

Stonehenge
A society with only Stone-Age technology studying a junkyard
The Giant’s Causeway (in County Antrim)

25

Ray 07.27.06 at 9:39 am

As far as I know, we’ve already come up with the right answers for 1 and 3, and it wasn’t by shrugging our shoulders and saying ‘some bloke must have done it’.
And frankly, I don’t think much of the stone-age-tribe-meets-junkyard analogy. A junkyard in the middle of a jungle or desert, with no other signs of modern humans? A junk yard that never has new junk added? And the correct answer – from a stone age tribe – has to be ‘a more advanced civilisation built it’, not ‘the gods did it’ and not ‘dunno’?

26

C.J.Colucci 07.27.06 at 9:43 am

I don’t know if it is still true, but for a long time there was a striking statistical correlation between the behavior of the U.S. stock market and the winner of the Super Bowl (for you World Cup fans, the national championship of the sport known in the colonies as “football”). The stock market moved up when the winner was either a member of the National Fottball Conference or an original member of the old National Football league (before the merger between the old NFL and the old American Football League) and moved down when the winner was an American Football Conference team not originally part of the old NFL (some teams, Pittsburgh significantly, were old NFL teams that were placed in the AFC).
The statistical correlation was quite robust, but there was no conceivable causal mechanism. From 1967 on, the US stock market rose most years. Until the recent run by New England (AFC and not old NFL), most of the dominant teams in the different eras were either NFC (Cowboys, 5 wins, 49ers, 4 wins, Packers, 3 wins, Giants and Redskins, 2 wins each) or old NFL (Pittsburgh, 5 wins) teams. And in some years, the NFC team played an old NFL team (Minnesota-Pittsburgh, Baltimore-Dallas), so the winner was guaranteed to meet the criterion for a stock market rise. Presumably, this dominance was based on football-related trends, not the state of the economy or investor expectations.
The point is that it is always possible, after the fact, to find a robust statistical correlation between X and some chosen Y. Without a reason to believe before the fact in a causal relationship between X and Y, the correlation is just a freak stat. Do we really need to investigate the hypothesis that cancer causes smoking?

27

Daniel 07.27.06 at 9:45 am

sam: quite the reverse, I’d love to see more. Is there anything you’d think of as being closer to the classic idea of the scientific method.

Steve, Ray: This was an experiment. The yogis decided to pick a location where there was a lot of violence, move a bunch of meditators in and measure the response. It’s not “statistical” at all (except in as much as it is a measurement of the rate of violence, which is a statistic). This isn’t epidemiology or anything like it; it was an experiment.

Steve: pick your own word if you don’t like “materalism” but I am going to reserve the word “science” for a neutral inquiry that does not presuppose any particular statements about the world, unless convinced I should do otherwise.

28

Daniel 07.27.06 at 9:48 am

re 26, this isn’t analogous to the Maharishi article. What would have been analogous would have been if, as a test of the hypothesis, they had waited for a bull market and then fixed the Superbowl so that an AFC team won and then measured the market falling by more than you would have expected given the normal variance, earnings, interest rates etc. I don’t see how you’re going to rule out experiments like the one the Maharishis carried out on any grounds other than not liking the theory they’re trying to test.

29

Daniel 07.27.06 at 9:53 am

regarding 25, say that we all get wiped out by bird flu and all that are left are our dogs. If a bunch of Martian scientists lands on earth and see a load of pointers, greyhounds, pitbulls and toy poodles, then they go home explaining to themselves that the animals they have just seen got to the shapes they were because of evolutionary selective pressure in some fitness landscape or other, have they not missed an important part of the story?

30

Steve LaBonne 07.27.06 at 9:54 am

Daniel, “neutral” does not mean exemplifying the old wheeze about having a mind so open that your brain falls out your ear. And in experimental science one anyway does not publish results from a single experiment that hasn’t been repeated, even when the effect under consideration actually has a plausible mechanism, and even when the definition of the outcome is a lot less subject to fiddling than is the case here. Finally, the criticisms found just in that abstract by Schrodt should be more than sufficient to get a paper rejected by any respectable journal.

31

Steve LaBonne 07.27.06 at 9:57 am

If those Martian scientists arrive a fairly modest doggie generations after the catastrophe that wisde spectrum of variation will no longer exist. Furthermore, even if they arrive before that point but are competent evolutionary / developmental geneticsits, thorough investigation will alert them to the fact that something is fishy.

32

Steve LaBonne 07.27.06 at 9:58 am

There should have been “number of” before “doggie generations”, sorry.

33

loren king 07.27.06 at 9:59 am

daniel: “It apparently matters a lot to you, Steve, because you (unless I am misinterpreting you) don’t want articles showing that transcendental meditation at a distance has an observable effect on the world to be published in journals.”

I like Steve’s earlier point that hypothesis testing doesn’t exhaust a scientific approach (it matters, for instance, how well your hypothesis actually captures the postulated mechanisms at work, and how the hypothesis flows from a coherent theoretical framework). But I agree with Daniel that this sort of stuff passes the hurdle for publishable research — which, of course, can be controversial and wrong (so long as it is controversial and wrong in interesting and testable ways). My problem with the Maharishi intervention studies has always been how controversial, untested, and generally fuzzy their stated mechanisms are. Their study designs always look for broad outcomes that (sort of, kinda, roughly) follow from the working of a (fuzzy, controversial) mechanism, instead of actually unpacking, explaining, and testing the specific elements of the purported mechanism.

34

Daniel 07.27.06 at 10:02 am

Daniel, “neutral” does not mean exemplifying the old wheeze about having a mind so open that your brain falls out your ear

Steve, you’re making a joke here where what you need is an argument. Neutral does in fact mean what it says; that you don’t judge the process by your own beliefs about the outcome. I would suggest that “neutral” does in fact mean “neutral” and I have yet to see a convincing reason why it should really mean “gerrymandered”.

in experimental science one anyway does not publish results from a single experiment that hasn’t been repeated

Really? I put it to you that “yes one does”. This is just factually not true.

even when the effect under consideration actually has a plausible mechanism,

As I’ve said above, this is pure and simple gerrymandering. If you are going to adjust your definition of the scientific method depending on whether you like the hypothesis being tested or not, that is unscientific.

Finally, the criticisms found just in that abstract by Schrodt should be more than sufficient to get a paper rejected by any respectable journal.

Are you Milloy in disguise? You seem to share his uncanny ability to assess scientific papers from their abstracts, and his spiritual insight into scientific respectability.

35

Daniel 07.27.06 at 10:05 am

Loren:

My problem with the Maharishi intervention studies has always been how controversial, untested, and generally fuzzy their stated mechanisms are.

I don’t entirely disagree with you, with the caveat that this is what everyone thinks about the experimental design of themotheruns. But I agree that this is definitely a debate within science, not one between science and non-science. It is made a lot more difficult to have that debate if we ensure that the Maharishis never tighten up their act by bullying them out of the Journal of Conflict Resolution and leaving them with the (entirely justified) sense that they didn’t get a fair go.

36

Daniel 07.27.06 at 10:07 am

If those Martian scientists arrive a fairly modest doggie generations after the catastrophe that wisde spectrum of variation will no longer exist.

Nothing important depends on the specific breeds; it just makes the example clearer. Take as many generations as you like and the dogs will still not have developed back into wolves and someone who has a theory of dogs which does not include a designer is still missing something.

37

Steve LaBonne 07.27.06 at 10:07 am

No, loren just explained why asking for unpacking and explanation of a proose mechinism is not “gerrymandering”. It’s garden-variety scientific quality control, without which the literature could be overrun with garbage in short order. It’s not whether one “likes” a hypothesis, it’s whether it can be unpacked and (hypotehtically)explained in ways that make any connection with existing knowledge, and also about whetehr the authors of a hypothesis have made any real attempt to do so.

You try to talk the talk, but in some fundamental way I really don’t think you “get” the natural sciences.

38

Steve LaBonne 07.27.06 at 10:09 am

They may not be wolves, but they will be “generic” dogs of a fairly uniform type, who as such are not likely even to attract any particular attention from the martians.

39

Steve LaBonne 07.27.06 at 10:10 am

“Proposed” not “proose”, sorry. Trying to type at high speed in between trying to get actual work done has its problems…

40

C.J.Colucci 07.27.06 at 10:10 am

Daniel: No reason to “rule them out,” just no reason to take them as particularly useful. After all, it might actually be true that some genetic factor that predisposes one to cancer also makes one more likely to smoke, but I wouldn’t recommend that funding agencies put money into it until there’s a plausible story beyond the numbers.

41

bi 07.27.06 at 10:10 am

It’s also conceivable that the Maharishis (happened to) pick a time around the retirement of Menachem Begin to do their “om” experiments. Ray raises a valid objection: even if the drop in terrorism was statistically insignificant, the study does precious little to rule out other possible causes for this drop.

42

bi 07.27.06 at 10:11 am

s/insignificant/significant/

43

SamChevre 07.27.06 at 10:13 am

Daniel,

It’s somewhat disputable what “the classic idea of the scientific method” would include/exclude, so I’ll define it as “observable, predictable, repeatable”.

Those criteria do not work well for any questions about specific artifacts. They rely on multiple observations of the same phenomenon. Thus, they are always problematic when dealing with unfamiliar things of which only one is observed. (This problem is a sci-fi classic–how do you recognize another intelligent society that manifests very differently than yours.) It is thus hard to come up with a “scientific” origin theory–some things are repeatable and observable (mutation and survival of the fittest) and some things aren’t (the Big Bang).

Many scientists follow a “materialist” theory (Steve is in this category) that excludes the supernatural from consideration, since it doesn’t meet the “predictable, repeatable, observable” test. I think that this excludes too much–understanding a junkyard would not be helped by ruling out “someone built it” as an explanation. (Ray–the people discovering the junkyard have our level of scientific knowledge, but no metal or glass and thus no knowledge of metal-working).

I work in the pattern-recognition math area (I’m an actuary). Pattern-recognition is not easy; sometimes things are strongly correlated but there’s no causal mechanism that is not a just so story (credit-scoring as a component of underwriting). I don’t think that materialistic evolution vs ID (theistic evolution is an older term for the same idea) has yet gotten beyond two sets of just-so stories.

44

Steve LaBonne 07.27.06 at 10:16 am

Who designed the designer?

45

bi 07.27.06 at 10:19 am

SamChevre: There’s a valid way to rule out “supernatural” explanations:

_Falsifiability._

That’s the main reason people consider creationism to be bogus. If anything and everything can be interpreted to be evidence for creationism (or, not against creationism), then what exactly are we testing again?

In a way, the Maharishis should be credited for putting forward a falsifiable hypothesis, even if the results still show some ambiguities at this point.

46

SamChevre 07.27.06 at 10:23 am

bi,

OK, I’ll add falsifiable to my list. Now, explain how you use it in the junkyard case. (Or, alternatively, explain how you’d apply it to evolution at the life/non-life boundary.)

47

Steve LaBonne 07.27.06 at 10:29 am

Who designed the designer, samchevre? Organized complexity needs to be explained, not just postulated. Apart from the feebleness of the actual proposals by people like Dembski, that’s the core of the problem with ID. It reminds me of panspermia as an “explanation” of the origin of life on earth- it’s just a way of trying to sweep the whole problem under the carpet.

Anway Daniel’s post is much more interesting than ID, so let’s please not continue to hijack the thread.

48

SamChevre 07.27.06 at 10:34 am

Steve,

“Who designed the designer” is pre-mature unless you know more than “there was a designer.” One can, for example, identify early stone tools as designed and made by intelligent creatures without knowing exactly which of the early humanoids made them–much less how those humanoids got to that location.

49

Daniel 07.27.06 at 10:41 am

I maintain that sensible scientific discussion of intelligent design is entirely ontopic for this thread.

It’s not whether one “likes” a hypothesis, it’s whether it can be unpacked and (hypotehtically)explained in ways that make any connection with existing knowledge

Ahhh, I now know the word that describes this position better than materialism; “Whiggery”. It just doesn’t fit with the history of science. There are all sorts of hypotheses which couldn’t be explained in ways that made any connection with existing knowledge – the galvanic response of frogs’ legs is the one that comes to my mind, but I could proliferate them. Thomas Kuhn made an entire theory out of the observation that science is full of things that don’t connect to the existing knowledge.

50

Doormat 07.27.06 at 10:42 am

Samchevre: Again, this is my point!! Most scientists think that modern evolutionary theory (that is, include genetics, modern theories about geology etc. as supporting evidence) IS AN EXCELLENT description of how the life we see on Earth came to be so diverse and complex, even if it doesn’t have a huge amount to say about life’s ultimate beginnings. So there is no need to bring in ID to explain anything. I will, however, credit you with being up-front about you belief that evolution is a “just-so story”.

To take your stone tool anology: this is bad, because we already know that primitive humans existed, and as others said above, where we find stone tools, we often find other evidence of early humans (or at least intelligent animals). If we found stone tools with absolutely no supporting evidence of life, then it would be bad science (not to mention logic) to immediately suggest a “designed origin”: the onus should be to see if there exist other explainations, or other supporting evidence. A good example might be crop circles: some people see evidence of ET-intelligence behing them; I prefer to more prosaic explaination of bored artists.

51

Steve LaBonne 07.27.06 at 10:42 am

But biologists need to go one step further and account for the existence of those humanoids. Pulling a designer out of a hat does precisely zero work for us. If everything we see really reflects the arbitrtary interventions of a super-intelligent being then the real bottom line is that science is impossible and we’re fooling oruselves to think we can do it. I can’t disprove that possibility any more than I can disprove the proposition that I am all that exists and I’m imagining this whole interaction with you- but that doesn’t mean I need to take either possiblity seriously.

52

Donald Johnson 07.27.06 at 10:44 am

I don’t have any sympathy for actually-existing ID theory and while I am a Christian, I don’t think there’s any good evidence for supernatural intervention in evolution. If there were, fine, but so far there isn’t. Okay, disclaimers out of the way–

It often does sound like some of the anti-ID crowd want to win the argument by definition. If you don’t like samchevre’s junkyard in the jungle analogy, it’s easy to change. Let a plane crash in a remote jungle, or have a 16th century galleon be smashed in a storm at sea, with all hands washed overboard, and all their belongings, and let it run aground on the coast of Australia. It’s made of wood and it is like no piece of technology the locals have ever seen, so are they supposed to conclude it’s a weird kind of tree? Would it be a rule of science that you don’t postulate ridiculously advanced technology that is indistinguishable from magic, when there’s an example that has just washed up on your beach?

The reason for dismissing ID is simply because none of their examples of things conventional evolutionary theory can explain are all that impressive. Darwin spoke of complex organs which exist in one species solely to benefit another species (with no reciprocal benefit provided as a result) as an example of something his theory couldn’t explain. So far nothing like that has popped up.

53

Steve LaBonne 07.27.06 at 10:46 am

Kuhn’s theory, based on a few cherrypicked examples, is an extremely poor fit with the actual history of science. Kuhnian mysticism and “whiggery” are not the only alternatives on offer.

54

matt m 07.27.06 at 10:51 am

Suppose they had run the experiment the same way, but instead of meditating had drunk six coca-colas a day, or avoided stepping on sidewalk cracks, and the same change in terrorist activity had been observed (as, I believe, it would)? Would it still merit publication? Is there any hypothesis so lacking in a plausible causal mechanism that statistical significance shouldn’t be a sufficient hurdle?

55

loren king 07.27.06 at 11:02 am

A feature of actual scientific practice that we haven’t considered yet, but which is clearly relevant to the Maharishi stuff, is stuff like active gatekeeping at the journals being debated by Steve and Daniel, but whether anyone bothers to critique and replicate a published finding. If the mechanism seems too outlandish and underdeveloped, my sense is that most people simply won’t bother addressing it, even if such work would be pretty interesting.

In fact, back in grad school, I asked Orme-Johnson to send me his dataset for the SIR study of DC. He gratefully obliged. I was (and am) sceptical of the findings, and wanted to pair their data with other series in the greater DC area, longer time series, a wide range of alternative model specifications and estimators, etc etc.

But what the study really needs is a bunch of other intervention studies, varying the factors under experimental control (type of meditation, for instance, intenstity of concentration, etc) and far more attention to spatial properties of the data not controlled by the experimenter.

Who has the time and money for those sorts of replications?

(And maybe, somewhere in the back of my sleep-deprived and caffeine-addled head, was the thought that, hey, maybe we really can control distant features of the universe merely with consciousness — and maybe when I figure it out in the data, I’ll start getting weird calls from people in hip latex and leather ensembles, talking in riddles (“you have to let it all go: fear, doubt, disbelief”) … yeah, like no one else was in dissertation-writing hell when the Matrix came out?)

…anyway, my point being, that even when unconvential findings get published, the reource and time constraints on taking them seriously as scientific debates are considerable (even ignoring for the moment plausible reputational worries).

Another example: a recent paper in a newish political science journal (Perspectives on Politics) argues for a genetic basis to political opinion. The paper of course doesn’t get precise about specific neurological mechanisms and evolutionary trajectories and selection pressures (much as the Maharishi stuff avoids such details about consciousness fields, etc). The paper cries out for reanalysis and critique, but who has the time?

56

loren king 07.27.06 at 11:06 am

“In fact, back in grad school, I asked Orme-Johnson to send me his dataset for the SIR study of DC. He gratefully obliged. “

I meant to say “graciously” but actually, I remember him seeming quite happy that someone, even some anonymous grad student, was taking the study seriously enough to request the data. I doubt he was grateful, however!

57

SamChevre 07.27.06 at 11:06 am

Donald Johnson,

I think that you put my key point better than I could: It often does sound like some of the anti-ID crowd want to win the argument by definition. That’s why I say that if you want to disprove ID to my satisfaction, your methodology needs to work in the other cases I mentioned.

Doormat,

Don’t misunderstand me–evolution (broadly defined, as you mentioned) explains some things well enough not to be a just-so story. However, there are a lot of things that the explanations are still in the just-so story stage (evolutionary psychology is the worst IMO).

Also, I go by Sam usually, not SamChevre–it’s written without a space as an artifact of the time before spaces could be put in text strings.

58

Daniel 07.27.06 at 11:17 am

Who has the time and money for those sorts of replications?

Unfortunately apparently not so much the Maharishi Mahesh Yogi these days.

I always find these guys rather interesting.

59

Ray 07.27.06 at 11:19 am

(Wow, that’ll teach me to go to a meeting.)
If we all got wiped out by bird flu tomorrow and martians landed on Saturday, they’d find a lot of odd-looking dogs and a lot of buildings full of corpses. By the time the remains of human civilisation have disappeared completely, dogs won’t look very odd.
52 – that would be a good argument if ID could come up with anything that was not otherwise explainable. But they don’t. They’re not 16th century aboriginals marvelling at washed-up galleons, they’re yokels gasping at a crisp that looks like Elvis.
and all the way back to 20 – all the TMers had in their study was a coincidence. There was no observable mechanism to explain how the two facts could be correlated. If you have a lot of coincidences, then yeah, maybe you should look to see if you’re missing a causal connection. One coincidence is not enough.

60

bi 07.27.06 at 11:20 am

Donald Johnson, SamChevre: I for one am willing to leave open the possibility that it’s a weird kind of tree. But I don’t see any need to quickly jump to any conclusion — after all, would it be wise to demand that Egyptian hieroglyphics be completely deciphered before the discovery of the Rosetta stone?

As evidence for the “galleon is a weird kind of tree” hypothesis, an aborigine might exhibit a way to grow galleons from galleon seeds. As evidence against it, one might see an actual bunch of people constructing a galleon. (And if seeds can grow galleons, anyone who suggests that the seeds were acted upon by an undetectable intelligent force to produce a galleon should get whacked.)

loren king: in a way, that’s Ray’s point.

61

Sebastian Holsclaw 07.27.06 at 11:23 am

“My problem with the Maharishi intervention studies has always been how controversial, untested, and generally fuzzy their stated mechanisms are. Their study designs always look for broad outcomes that (sort of, kinda, roughly) follow from the working of a (fuzzy, controversial) mechanism, instead of actually unpacking, explaining, and testing the specific elements of the purported mechanism.”

I’m not convinced that this is a particularly good objection. Much of the first step is observation of what happens–long before you can provide a good expanation of why it happens. It is almost trivial at this point to describe the actual interactions of gravity–but unless I’ve missed something lately there isn’t a great explanation of the mechanism by which two masses interact (they bend space-time? that’s nice but shouldn’t Loren and Steve ask for an explanation of how they do that?). And of course people should investigate the mechanisms. But that doesn’t make the observation non-science.

Medical science and observation led to the use of anti-biotics well before the mechanism that antibiotics use was understood.

62

Sebastian Holsclaw 07.27.06 at 11:26 am

“after all, would it be wise to demand that Egyptian hieroglyphics be completely deciphered before the discovery of the Rosetta stone?”

I don’t think that is a good analogy. I think you could plausibly have argued that hieroglyphics were intelligently designed and had meaning long before we understood what that meaning was.

63

Ray 07.27.06 at 11:28 am

Number of times meditation and a fall in terrorism have gone together: 1
Number of times apples have been observed to fall from trees: ?

64

bi 07.27.06 at 11:31 am

Sebastian Holsclaw: actually I was referring to the need to jump to conclusions based on incomplete information.

65

Sebastian Holsclaw 07.27.06 at 11:41 am

Just to be clear, I strongly suspect that TM™ does not cause reductions in terrorism. But if you want to scientifically dismiss a paper on the subject the objection that we don’t understand the mechanism by which meditation could work is not an objection at all. It is perfectly valid science to say “I have seen ‘W’ seemingly cause ‘X’ and I have no idea how it might do so”.

The fact that the current scientific understanding of the world doesn’t allow for the interaction isn’t an objection either–unless you believe we are already at the highest height of all possible scientific understanding. The right skeptical question is “can we repeat it”. Who is going to fund the next study?

66

SamChevre 07.27.06 at 11:46 am

OK, so I’m going to try tying this discussion back to the original post.

The problem is that observations of phenomena with no known, material causal mechanism are problematic for scientists. They could be statistical artifacts (as many of us suspect the Maharishi study is). They could be observations of a not-yet-studied causal mechanism (like the frog legs). They could have a perfectly good explanation, that we have only partly worked out and that there are still alternatives to. (DNA as a trait-producing mechanism, given that we are now learning that the same DNA can be differently expressed). And they could be observations of something that is real, but not in our calculations. (God, aliens, another civilization)

The problem that some of us have is that some scientists want to say that only some of the explanations for these phenomena are worth studying; that is, they want to define “science” to exclude what others see as likely explanations. In the short term, that makes their arguments unconvincing. In the long term, whether their arguments are right or not, it won’t much matter; regardless of what the establishment thought 1000 years ago, that the Earth orbits the Sun is clearly established today.

67

Daniel 07.27.06 at 11:47 am

Number of times meditation and a fall in terrorism have gone together: 1

But we can’t tell whether this is because it is genuinely a coincidence or because of interference in the journal publication process.

68

Steve LaBonne 07.27.06 at 11:50 am

“Medical science and observation led to the use of anti-biotics well before the mechanism that antibiotics use was understood.” Uh, no. Antobiotics were discovered as bacteriocidal agents. Long after the germ theory of disease was discovered. Not knowing HOW the compounds killed bacteria does not change the fact that there were sound scientific reasons for believing that a bacteria-killing compound would be useful in treating bacterial infections. In this case it’s the mechanism of disease-fighting that’s relevant. Bad example.

69

Steve LaBonne 07.27.06 at 11:52 am

samchevre, I again point you to my #51. “Get out of jail free” cards simply can’t do any useful work in science.

70

loren king 07.27.06 at 11:52 am

Sebastian: “I’m not convinced that this is a particularly good objection. Much of the first step is observation of what happens—long before you can provide a good expanation of why it happens.”

Sure, but I think it’s a bit weird if you’re characterizing scientific observation as somehow unburdened by theoretical and philosophical commitments (theories and commitments that good scientists are up front about, of course, and willing to reconsider).

And even if science did proceed from theoretically and metaphysically untarnished observation (a point I’d dispute as a general characterization, but it has some plausibility for initial theorizing, and lets grant it more generally for a moment), in the Maharishi case your point probably isn’t especially applicable, because they do come to their observations with an elaborate metaphysical account of the mechanisms they claim to be testing (reality is fundamentally various manifestations of consciousness in a substratum of pure consciousness, etc). I’m all for observation leading to interesting stuff to be explained later, but if you’re going to claim that reality is fundamentally consciousness and that TM gives the tools to hack that reality, then I’d prefer to see experiments and quasi-experiments that test specific elements of the TM hackers toolkit, not treat the mechanism as a black box.

“It is almost trivial at this point to describe the actual interactions of gravity—but unless I’ve missed something lately there isn’t a great explanation of the mechanism by which two masses interact (they bend space-time? that’s nice but shouldn’t Loren and Steve ask for an explanation of how they do that?).”

Actually, I often ask my physicist friends exactly these sorts of philosophical questions, and they are surprisingly eager to speculate based on the latest work going on (but then, several of them are mathematicians rather than experimentalists, so perhaps they are really metaphysicians at heart?)

“Medical science and observation led to the use of anti-biotics well before the mechanism that antibiotics use was understood.”

Certainly, but a good scientific approach then keeps hammering away until we really understand the mechanisms at work. Otherwise, we might rest content with explaining antibiotic effects as balancing the woods and vital energies of the body. The initial observations are helpful, but alone they don’t help us figure out whether competing theories (traditional Chinese versus western molecular-organic models, for instance) are better or worse as explanations of how things work in the body. To do that, we need more observations tailored to distinguishing between competing hypotheses about mechanisms and processes.

71

dthurston 07.27.06 at 12:04 pm

It’s interesting to compare the non-response to the Maharishi study to the response to the original Bible Codes paper. There was a lot of hype and nonsese later, but the original paper was published in a good journal and had careful statistics. In that case other academics did take time to investigate the claims and show other possible causes.

72

SamChevre 07.27.06 at 12:10 pm

Steve,

You’re missing something. I agree, “Get out of jail free” cards simply can’t do any useful work in science. But partial explanations CAN do very useful work for science–they can help us figure out what not to look at. To use Donald’s galleon example, knowing that “it was built” will keep us focused on the question “how do we build one?”, instead of pursuing fruitless experiments in how to grow galleons.

73

Steve LaBonne 07.27.06 at 12:13 pm

I don’t think the origin of galleons is a problem in current biology, samchevre. Come up with a real biological example where you think ID could help biologists out of a current impasse, and we can discuss it. These artificial thought experiments get us nowhere.

74

SamChevre 07.27.06 at 12:22 pm

Steve,

I’ll have to pass on “current examples in biology”. I’m a mathematician/accountant with an economics background, not a biologist; the last biology class I took involved looking at an earthworm through a microscope. I think the thought experiments are helpful, because we know what the right answer is. (They are like test cases in a computer program–you wan tto make sure your program gives the right answer when you know what the right answer is, before asking it for answers when you don’t.)

75

loren king 07.27.06 at 12:26 pm

Sebastian, on the TM findings: “if you want to scientifically dismiss a paper on the subject the objection that we don’t understand the mechanism by which meditation could work is not an objection at all. It is perfectly valid science to say “I have seen ‘W’ seemingly cause ‘X’ and I have no idea how it might do so”.”

Personally, I don’t dispute your concluding point in this passage (which is why I agree with Daniel that these papers passed the hurdle for publication as valid empirical research in the social and behavioural sciences). But I see no problem criticizing a paper that postulates speculative and controversial mechanisms, yet provides an empirical test that doesn’t really give us much (if any) explanatory resolution with respect to those mechanisms.

To be clear, it isn’t that the TM folks are finding this correlation between meditation and bad societal stuff (crime, terrorism) and then saying they have no idea what causes it. They are making some very controversial claims about the nature of reality and consciousness, and then suggesting that the findings support their controversial causal model. I’m not convinced (not least because I think they need to do a vast amount of theoretical and experimental heavylifting in physics and neuroscience before I’d take the intervention studies as plausible tests of their causal hypothesis).

76

Steve LaBonne 07.27.06 at 12:34 pm

You see, that’s just it- ID proponents and sympathizers have nothing of use to offer working biologists, and very few of them even have much actual knowledge of biology. Their two intellectual leading lights are 1) a biochemist who hasn’t done any real research in years, who has repeatedly lied through his teeth about the current state of the scientific literature by claiming there is no published research where there is actually piles of it, who likes to talk about mousetraps (his version of galleons) more than about living organisms, and who made a complete ass of himself in Dover; and 2) a mathematician (who, remarkably, is extremely prone to making embarrassingly trivial mathematical errors.) The ID movement simply has nothing at all to offer to biologists. It is a propaganda movement pure and simple, cooked up to advance Johnson’s “wedge strategy”, with the hope (which turned out to be delusional)of escaping the First Amendment demise that previously overtook “creation science”. You’re wasting your time paying attention to this crap.

77

Steve LaBonne 07.27.06 at 1:02 pm

It is almost trivial at this point to describe the actual interactions of gravity—but unless I’ve missed something lately there isn’t a great explanation of the mechanism by which two masses interact (they bend space-time? that’s nice but shouldn’t Loren and Steve ask for an explanation of how they do that?)

Evangelical Scientists Refute Gravity With New ‘Intelligent Falling’ Theory

78

loren king 07.27.06 at 1:14 pm

Steve: “Evangelical Scientists Refute Gravity With New ‘Intelligent Falling’ Theory”

Not to drag this even further afield, but who can forget the compelling evolutionary theory of gravity? (once fruit fell in all directions, but only those fruits that fell earthwards could pass along the “direction of fall” trait …).

79

Steve LaBonne 07.27.06 at 1:18 pm

That’s a good one. I’m sure if we try we can work the Strong Anthropic Principle into this discussion somewhere as well. Might as well go for the whole enchilada ;)

80

Sebastian Holsclaw 07.27.06 at 1:21 pm

“in the Maharishi case your point probably isn’t especially applicable, because they do come to their observations with an elaborate metaphysical account of the mechanisms they claim to be testing (reality is fundamentally various manifestations of consciousness in a substratum of pure consciousness, etc). I’m all for observation leading to interesting stuff to be explained later, but if you’re going to claim that reality is fundamentally consciousness and that TM gives the tools to hack that reality, then I’d prefer to see experiments and quasi-experiments that test specific elements of the TM hackers toolkit, not treat the mechanism as a black box.”

Ahh, but now we get to the crux of the issue from a scientific perspective. You don’t like their explanation (which is fine) so you want to deny their observations. It is perfectly fine to say that their explanations don’t adequately explain their observations. That can lead to great science. But that provides absolutely no reason to deny their observations.

Once again, I would be shocked if their observations turn out to be other than non-reproducible statistical artifact. But that is just my personal prejudice about how I think the world works–not a valid scientific judgment.

81

loren king 07.27.06 at 1:23 pm

dthurston: “It’s interesting to compare the non-response to the Maharishi study to the response to the original Bible Codes paper. … the original paper was published in a good journal and had careful statistics. In that case other academics did take time to investigate the claims and show other possible causes.”

The differing reactions might be traceable to the sort of work needed to test the respective claims. I know when I was messing around with the TM stuff, my first inclination was to find a meta-level criticism and leave it at that (i.e. poor estimation strategy, missing plausible variables, etc), because the work required to evaluate their findings properly would be extraordinarily time-consuming and expensive. Indeed, even taking their experiments at face value and wading through the statistical work would take a fair bit of time (digging up other datasets, getting GIS data and matching with the social indicators they use, testing alternative specifications and estimators, although it is just the sort of thing you could farm off on a graduate social statistics methods course as a replication assignment … hmmm …).

82

loren king 07.27.06 at 1:52 pm

sebastian: “now we get to the crux of the issue from a scientific perspective. You don’t like their explanation (which is fine) so you want to deny their observations.”

But I’m not denying their observations (I’m not exactly sure what you mean here, but I’ll take you to mean ‘deny the existence of’). I’m denying that their observations support their explanation in the way they suggest.

And it’s not that I don’t like their explanation — this isn’t a matter of personal preference. Indeed, I personally think it’d be really cool if consciousness were the ultimate stuff of reality (whatever that might mean) and that we could hack it with our minds (that’s the part that sounds cool to me). I simply don’t think the TM folks have done the conceptual and experimental gruntwork needed to make scientific sense of that causal model, and to test it with the tools they use (the urban intervention studies).

83

albert 07.27.06 at 1:55 pm

“now we get to the crux of the issue from a scientific perspective. You don’t like their explanation (which is fine) so you want to deny their observations. It is perfectly fine to say that their explanations don’t adequately explain their observations. That can lead to great science. But that provides absolutely no reason to deny their observations.”

But it *does* provide good reason to doubt their observation. Their method is equivalent to saying “I’ve got x-ray glasses and can tell what color underwear you have on” and then getting published if I turn up significant results on an occassion. It’s not just that the explanation of how my x-ray glasses work is insufficient, it’s that I’m the only person in the world who knows how to wear the right way to get the necessary result. No amount of non-yogic experiment could appropriate refute this finding, because the instrument they use is, as was said above, a black box. You have to be able to both open that box and show what’s inside AND explain how it works in order for an account to be sufficiently scientific.

For a slightly different explanation, see “experimenter’s regress.”

84

mc 07.27.06 at 2:10 pm

But it does provide good reason to doubt their observation.

Absolutely. But I don’t agree that statement their observation is doubtable precludes it from being publishable. This is their interpretation of an observation. If it’s so clearly wrong, it can be refuted easily in another study: via replication, unpacking the black box, etc.

Where I think the disagreement here lies is that I think that it’s perfectly acceptable to publish an opening gambit with spectulative interpretations that others can work on. Whereas others here believe that a published article must be the authoritative and final word.

85

Sebastian Holsclaw 07.27.06 at 2:32 pm

“It’s not just that the explanation of how my x-ray glasses work is insufficient, it’s that I’m the only person in the world who knows how to wear the right way to get the necessary result. No amount of non-yogic experiment could appropriate refute this finding, because the instrument they use is, as was said above, a black box. You have to be able to both open that box and show what’s inside AND explain how it works in order for an account to be sufficiently scientific.”

I can’t agree. If you have glasses that accurately can see the color of people’s underwear, that is clearly a scientificly interesting observation even if your explanation for how it works is clearly wrong. If that is reproducible it would be great to figure out how it really works.

So if TM really did reduce terrorism, I wouldn’t care that the explanation isn’t likely, it would be worth investigating anyway.

86

Steve LaBonne 07.27.06 at 2:34 pm

Where I think the disagreement here lies is that I think that it’s perfectly acceptable to publish an opening gambit with spectulative interpretations that others can work on.

Well, as I think about this more, it seems to me that the cross-purposes this discussion has at times gotten into arise in part from the fact that standards on this point quite frankly differ greatly among fields, certainly in the natural sciences (I note that the specific topic of this discussion more properly pertains to the social sciences where I’m really not in any position to judge.) When a field is relatively undeveloped and/or inherently has a very high speculation quotient- let’s pick evolutionary psychology as an example that periodically gets bounced around in this forum- stuff not much better than “TM prevents wars” actually does get published a lot. In more mature fields there’s so much really solid stuff to keep up with that there’s just not much room for far-out speculation either in the journals or in practitioners’ heads, a phenomenon to which Loren King has repeatedly and rightly alluded above.

87

Ray 07.27.06 at 2:43 pm

67 But we can’t tell whether this is because it is genuinely a coincidence or because of interference in the journal publication process.

It’s hard to argue that the existence of one published TM article is evidence for the existence of other TM articles that meet the same standards, but can’t get published.

Sebastian – 80 You don’t like their explanation (which is fine) so you want to deny their observations.

I don’t think I’ve seen anyone arguing that crime/terrorism did not go down in the period studied (or that the meditation did not take place as they said it did, and that is something you’d want to check). So no questioning of the observation. People are questioning the idea that the two events are causally related.

88

Ray 07.27.06 at 2:51 pm

If you have glasses that accurately can see the color of people’s underwear, that is clearly a scientificly interesting observation even if your explanation for how it works is clearly wrong. If that is reproducible it would be great to figure out how it really works.

Quick thought experiment for you, Sebastian. On the tube into work tomorrow, this guy comes up to you, and says Peter Pan flew into his bedroom last night and sprinkled pixie dust on his glasses. Now their magic powers allow him to see that your underwear is white. Do you
a) think ‘Lucky guess’
b) run away very quickly, or
c) think “Fascinating! I must bring this man to my lab, so I can study his glasses further!”

89

SamChevre 07.27.06 at 2:55 pm

Steve,

Re your #73 and #76–Come up with a real biological example where you think ID could help biologists out of a current impasse–can you do the inverse? (This is a real question–I will listen to the answer.) In other words, is there any current subject of study in biology where narrowly evolutionary theory (defined to exclude the parts of evolutionary theory that, say, Dembski and Donald Johnson would agree with, like genetics) provides the answer?

90

Donald Johnson 07.27.06 at 3:00 pm

If the argument here is just that Behe and Dembski haven’t presented anything convincing, I’ve got no problem with that. My objection was to claims that even in principle ID couldn’t be science. Well, if intelligent aliens (or God) really had tinkered with the evolution of life, it’d be a little silly to define the nature of science in such a way that we couldn’t acknowledge what they’d done.

Now for a slight threadjack–on the subject of the anthropic principle, a couple of Bayesians named Bill Jefferys and Michael Ikeda claim that the fine-tuning argument actually tends to refute supernaturalism. I suspect this is wrong, but wasn’t able to convince Jefferys (or myself) in a usenet discussion I’d be too embarrassed to link to even if I went and found it. Briefly, if I’m not butchering it, their argument is that if the physical constants allow life to exist, it’s what you expect in a naturalistic world, but God could have created an arbitrary one instead and still allowed life. So the fact that the physical laws allow for life supports naturalism, not supernaturalism. It’s given a Bayesian gloss. I don’t think they correctly state the logic of the fine-tuning argument. I think the fine-tuning argument is that the meta-laws, so to speak, that gave us the physical constants most likely would have produced non-life friendly constants and the fact that the constants turned out to be life-friendly means Someone stacked the deck. (Or alternatively, there’s a huge number of parallel universes and we pop up in the life-friendly ones.) Jefferys didn’t agree and I could be wrong–for that matter, it’s been a few years since the argument I had with him.

The address is here–

http://quasar.as.utexas.edu/anthropic.html

I’d try linking it, but I messed up last time–gotta check and see what I’ve been doing wrong.

91

Donald Johnson 07.27.06 at 3:02 pm

Oh, it links automatically. I didn’t know that, or had forgotten.

92

Steve LaBonne 07.27.06 at 3:07 pm

I don’t understand your question, samchevre. If you’re asking, is there something in biology that biologists are scratching their heads about saying, “wow, we really don’t see how eveolutionary biology could account for this”, then the answer is no (understanding that the amount of detailed explanation currently available will differ greatly for different questions. Thank goodness, since it would be very sad if everything was solved and science was over!)

Wnat to understand why the hooha over ridiculous ID crap is so frustrating to a biologist? Incredible progress, almost unimaginable even 30 years ago, is being made in fields like the evolutionary genetics of animal development, yet gets an infinitesimal fraction of the public attention that the IDiots command. If you want to see how infinitely more fascinating that stuff is than any thought that will ever occur to William Dembski in the next million years, try a book I’ve plugged before, Endless Forms Most Beautiful by Sean Carroll.

93

albert 07.27.06 at 3:07 pm

“if TM really did reduce terrorism” -Sebastian

Well of course, if TM did reduce terrorism, or if my special glasses actually told me the color of your underwear, then those would be worth more investigation. The whole point is, though, that those claims don’t in themselves tell you “if”. They are comprised solely of a statistically significant relationship that is not (as it stands) causal and is based on a black-boxed observational technique.

and re #84, I’m not saying it shouldn’t be published, I’m just surprised the study, at its present state, was published.

94

Jon H 07.27.06 at 3:07 pm

sebastian writes: “It is perfectly valid science to say “I have seen ‘W’ seemingly cause ‘X’ and I have no idea how it might do so”.”

Only if the observation has been made under controlled conditions that remove other possible causes.

If a bunch of TM people are meditating, and they observe a large metal cylinder lift into the air, they could say that their meditation seemingly caused the cylinder’s levitation. But it might be because the cylinder was a helicopter.

95

Ray 07.27.06 at 3:14 pm

I don’t think the argument is that we can never raise the possibility of aliens, or even God, doing something.
It’s more that it makes no sense to reach for the invisible superbeings every time you encounter a problem that you can’t explain immediately – especially if you don’t have any reason other than “it’s hard” to look for God as your explanation, and if this explanation has no possibility of leading to new research, and if there are no discoveries you could make that would rule out your mysterious intelligent designer having some part to play.
In those circumstances, calling on a mysterious intelligent designer looks less like an attempt at an explanation, than an attempt to prevent explanation.

96

Jon H 07.27.06 at 3:26 pm

sebastian writes: “If you have glasses that accurately can see the color of people’s underwear, that is clearly a scientificly interesting observation even if your explanation for how it works is clearly wrong. “

It isn’t interesting if the “experiment” were conducted by guessing the color of the underwear of elementary school boys, where a 95% accuracy level would be easy to obtain by just saying “white” all the time.

Do the glasses work? No – the “experiment” was poorly designed by using participants who generally wear white underwear, making it easy to guess.

97

Jon H 07.27.06 at 3:32 pm

daniel writes: “But we can’t tell whether this is because it is genuinely a coincidence or because of interference in the journal publication process.”

Oh please. If the Maharishis had reams of studies showing that TM stops conflict, they’re free to publish in any venue they can find. I’m sure they have enough sucker’s cash to print their own journal of TM studies.

If they have the data, they can get it published in journals. If they have anecdotes and coincidence, they won’t get it published in journals.

It was sunny the last time I visisted Seattle. That does not mean I can publish a scientific article that I caused the sunny day.

98

Steve LaBonne 07.27.06 at 3:47 pm

Samchvre, I tried again to make sense of your question, and what you don’t seem to get is that “narrowly evolutionary theory” is not some well-defined separable part of biology. This is like asking a chemist “can you explain the reactivity of this molecule precisely using ‘narrowly’ organic chemistry but no quantum mechanics?” Evolution and the rest of biology are no more separable subjects than organic and quantum chemistry are nowadays. Evolution is as much involved in making sense of everything in biology as QM is in chemistry. Conversely, nobody aimagines that you can get the answer to that chemistry question by asking a physicist who has never studied organic chemistry.

99

C.J.Colucci 07.27.06 at 4:11 pm

On further review, it seems I cheated the Washington Redskins out of a Super Bowl title. They won 3, not 2.
And to be clear, I don’t have a problem with the study having been published. It just doesn’t seem worth taking seriously or doing the work to test.

100

SamChevre 07.27.06 at 4:12 pm

Sorry, Steve–I’m not trying to be confusing here.

My question is–what does the contested portion of evolutionary theory tell you that is valuable in the real world of biology? ID proponents don’t contest genetics, so far as I know–so telling me that genetics is very valuable isn’t an answer.

I’m not asking for an answer that doesn’t involve genetics (or, in your example, organic chemistry). I’m just asking for an example where genetics isn’t the entire answer (since no one is contesting genetics).

101

SamChevre 07.27.06 at 4:23 pm

Followup–as an example, let’s say there is an ongoing debate between quantum mechanics and “sam’s mechanics”. If both give the same results for organic chemistry, then saying “organic chemistry works–here’s proof” doesn’t help prove either quantum mechanics or sam’s mechanics at all.

102

dsquared 07.27.06 at 4:31 pm

I simply don’t think the TM folks have done the conceptual and experimental gruntwork needed to make scientific sense of that causal model

But (and I think this serves as an answer to everyone else who made the same point), how can they be expected to, when the game is so obviously rigged against them? Of course it makes no sense for the yogis to throw good money after bad. They’ve been shut out of the scientific process, and so a question that could have been settled through science has been settled through non-scientific means. That’s Steve Fuller’s point I think.

I’d also add that a very great deal of evolutionary psychology (and behavioural economics, and a few other worthless subfields I could name) appears to have benefited from the opposite effect – they’ve found it a lot easier than they should have done to get published, meaning that a number of theories which actually wouldn’t necessarily do all that better than TM or intelligent design, have acquired an altogether spurious plausibility.

103

Ray 07.27.06 at 4:43 pm

samchevre, PZ Myers from Pharyngula is the best person to ask that question. I’m sure he’ll answer you at great length. (And if we’re lucky, he might even let us watch)

104

Steve LaBonne 07.27.06 at 4:45 pm

Samchevere, the answer to #100 is “pretty much everything”, evolution being the indispensible central explanatory principle of biology just as QM is of chemistry.

The answers to #101 are many and various but just to pick two examples there is no reason in “Bill’s biology” why organisms should fit into a consistent hierarchical classification system of exactly the sort that would result from descent with modification, nor any explanation in “Bill’s biology” for certain quite arbitrary biochemical similarities among all organisms (like the genetic code) without invoking common descent. (So much so that Behe, the only actual biologist trying to play this game, has pretty much conceded not only common descent but when pressed, even much else in evolutionary theory- only where there is what seems to him a hard problem, i.e. an opportunity to sneak some theology in, does he selectively invoke his get out of jail free card. That’s not science. He is really just a theistic evolutionist of a particularly exaggerated and incautious kind, because that’s as far as any biologist can possibly go, even one trying to make a name for himself as a proponent of “ID”.)

For much much more info I invite you to visit talkorigins.org where you will find a goldmine of information on ID and other forms of anti-evolution, and equally on real evolutionary biology. There’s just no substitute for knowing a considerable amount of biology and I can’t really teach you a freshman biology course here!

105

Steve LaBonne 07.27.06 at 4:51 pm

I’d also add that a very great deal of evolutionary psychology (and behavioural economics, and a few other worthless subfields I could name) appears to have benefited from the opposite effect – they’ve found it a lot easier than they should have done to get published, meaning that a number of theories which actually wouldn’t necessarily do all that better than TM or intelligent design, have acquired an altogether spurious plausibility.

I quite agree with this which is exactly why I’m so puzzled that in the previous breath you advocate relaxing publication standards.

The rigging the game argument is crap. Both the Maharishi’s crew and the IDiots have ample resources to pursue any research program they want. They might have to publish initially in their own journals but if they could produce anything of real substance to publish there, they’d have something solid to point to and get the dsicussion rolling. Nobody’s stopping them- they don’t need NSF funding.

106

albert 07.27.06 at 6:45 pm

D2: They’ve been shut out of the scientific process, and so a question that could have been settled through science has been settled through non-scientific means. That’s Steve Fuller’s point I think.

That’s exactly Fuller’s point, but it’s also exactly what’s wrong with Fuller’s point.

There may be science that’s feasible but never gets done because of prevailing social forces or prejudices in science. Perhaps if we saw some better examples than ID & TM it wouldn’t seem like Fuller is just letting in the crazy.

107

loren king 07.27.06 at 9:11 pm

D^2: “But (and I think this serves as an answer to everyone else who made the same point), how can they be expected to, when the game is so obviously rigged against them?”

I share Steve’s and Albert’s scepticism with the Fuller critique you’re endorsing here (at least stronger variants of it), but on my specific complaint I’ll just note that some of the gruntwork I was referring to doesn’t require huge research budgets.

For instance, the TM folks make a big deal about their metaphysics of consciousness, and Orme-Johnson himself is a physicist, so I’d expect to see a lot more theoretical physics going on at the very least, just to make some sort of rigorous sense of the idea that consciousness is ultimately the stuff of reality (and that meditation, through certain mechanisms, lets us alter features of that reality in ways that don’t make sense under existing models of physical and particularly neurological processes). It isn’t as if they don’t have the money and institutions to do that sort of gruntwork (they have their own university, after all).

I think I might buy a weak version of Fuller’s concern: prevalent attitudes and existing institutional incentives may make some interesting scientific work difficult merely because it doesn’t fit neatly into popular (and thus lucrative) research agendas.

For example, I find plausible Evelyn Keller’s argument about how gendered attitudes and political associations slowed down research in cell biology, by devaluing some german research in development, and by focusing researchers so intently on the nucleus at the expense of studying carefully the rest of the cell.

But I’m not convinced that the exclusions are legion or especially durable: I suspect that good and useful work eventually makes it (as it has in cell biology and developmental genetics), and that ultimately unsupportable biases and omissions eventually get corrected.

Now, if we want to refute this sort of ‘Horatio Algiers’ optimism about the self-correcting property of scientific practice, I’m not sure TM and ID are the best candidates.

Why? Because it’s not at all clear that TM or ID are suffering primarily because they’re being shut out of the scientific club, or because they’re victims of Reindeer games once they’ve fought their way into the clubhouse (“they wouldn’t let poor Rudolf …”). Rather, I suspect they suffer in credibility because they seem to avoid the really interesting (and controversial) questions that their overarching approach raises, and those are in large measure conceptual questions about the precise workings of causal mechanisms.

Perhaps science needs persistent empirical anomalies to inspire just this sort of theoretical work? Thus (the argument might go) if no one can fund research to find those anomalies and get their work published in reputable journals, then no one in the broader scientific community has an incentive to tackle the controversial conceptual innovations when they are occasionally proposed by lone voices in esoteric journals.

Maybe that’s the problem here, but my gut sense is that the relationship between theory and empirical anomaly is complicated and reciprocal; that the messiness of this relationship is actually more forgiving to unconventional thinking than many charge (at least in the longer term of a few generations); that TM simply doesn’t have the sort of theory yet that could result in a fruitful interplay between empirical challenges and conceptual innovation; and finally, that this failing is not obviously attributable to the culture and institutions of mainstream science (although of course that cannot help, I’ll grant).

108

Chris 07.28.06 at 12:31 am

OK, just to generalise the problem slightly more here’s an example from the human, um, ‘sciences’. Rosemary Crossley (reasonably well known as the author of Facilitated Communication Training)had an article accepted for the European Journal of Communication Disorders dealing with the case study of someone whose expressive language problems improved markedly with a touch on the shoulder and who could, having speech, discuss this.

Crossley concluded that “The case does, for example, seem to demonstrate that language difficulties remediable by touch do exist.” but that “One of the problems with many critics is that they seem to want the explanations to come before the observations.” As the editors had eventually realised that the piece was controversial they added a series of commentaries (I’m mildly surprised that JSTOR didn’t adopt this generally recognised method of washing your hands of the controversy) in which one said that “Crossley offers no justification as to why a touch of a finger can have the effect… … we would like at least some of the explanations to come with the observations.”

To which Crossley retorted that
“I have not attempted to provide explanations for phenomena for which I have no explanations. However, theories are not essential accompaniments to observations. Apples fell before Newton provided us with an explanation. In the affairs of this world, and in our science, we must take into account both facts for which we have explanatory theories and facts for which we have, as yet, no such theories. The absence of a justificatory theory (or, indeed, the possible incorrectness of a justificatory theory) should be no barrier to the acceptance of a fact. Language disorders that are susceptible to remedy through touch exist. I appreciate any and all attempts to explain these phenomena, but their existence does not depend on my ability to account for them.”
I hope that counts as a new hare.

109

derrida derider 07.28.06 at 1:25 am

Clearly those powers meeting in Rome to discuss Lebanon need to agree to fund more meditation.

I wonder if the the Maharishi’s results would be the same if the study period included the last month or so?

110

john m. 07.28.06 at 3:14 am

That’s why I say that if you want to disprove ID to my satisfaction, your methodology needs to work in the other cases I mentioned.

Excellent approach and one I shall cherish in using to drive people crazy when discussing ID. So, prove to me that God/Aliens/Whatever don’t exist…

Also, I can’t resist mentioning this (though it is shameful for me to know it), you may benefit from reading the introduction to Battlefield Earth by L. Ron Hubbard who clearly knew a thing or to about science and religion – though to be fair, should anyone have to read such a thing? Anyway, in it, he discusses his aim to write a science fiction book rather than a fantasy. His distinction, drawn from hazy memory, was to take an example of a character needing a sword. Fantasy: poof! the sword appears by magic, no further explanation necessary. Science Fiction: some extrapolated mechanism based on science. Either way the character gets a sword.

111

SamChevre 07.28.06 at 8:20 am

Steve,

Thank you! That’s the kind of information I needed. My problem is that there is so much out there that I don’t always know where to start looking; that’s the disadvantage of being largely self-taught–I know lots of information, but not how it all fits together.

I still seems to me, though, that “evolution provides an explanation of phenomena X” is not quite on point. Think of some of the other theoretical models that have been supplanted–Newtonian physics and Ptolemy’s astronomy. They provided explanations of most common phenomena that were good enough to provide accurate predictions of those phenomena; it is by looking at the (rare) case where they did not work that the need for a different paradigm was realized. Their replacements provide explanations that predict exactly the same phenomena 95% of the time; it is only in a few cases that the (very different) replacement theories predict (accurately) a different result.

112

Steve LaBonne 07.28.06 at 8:33 am

samchevre, you simply have to know a lot more biology than you do know in order to see why this ain’t gonna happen. If interested, again I recommend you read some of the excellent articles at talk.origins (and talk.design), and also hang around the Pharyngula blog (P.Z.Myers’s place)- PZ is very good at providing clear lay-level summaries of recent work in evolutionary developmental biology (evo-devo, as it’s known.)

Now, I’ve been polite, but this business of people who by their own description know next to nothing about science “sympathizing” with one or the other anti-science line of bullshit really does get my goat. Has it occurred to you that it’s pretty damn arrogant to express such “sympathies” in the absence of any knowledge and understanding of what you’re talking about? Do you really think you can come up with questions that thouands of scientists working across many years haven’t thought of? Frankly, it’s just anti-intellectualism of the rankest variety.

113

Ray 07.28.06 at 8:41 am

But the ID argument is such an obvious fudge, it has absolutely no credibility.
“This feature could not possibly have evolved, therefore it must have been designed!…..what’s that you say? you’ve discovered how it evolved?… Okay then, this feature can’t have evolved, it must have been designed!”
Relativity doesn’t just explain a little bit of the universe, it explains the whole thing, and it makes predictions about things that haven’t been discovered. The paradign changing detail may have been small, but everything changes once that is explained. ID ‘explains’ a tiny little thing, shifts ground as soon as it’s falsified, and makes no predictions. It is only ever an appeal to ignorance.

114

Ray 07.28.06 at 8:42 am

113 a response to 111, obviously.

115

Matt McIrvin 07.28.06 at 8:58 am

“I simply don’t think the TM folks have done the conceptual and experimental gruntwork needed to make scientific sense of that causal model.”

“But (and I think this serves as an answer to everyone else who made the same point), how can they be expected to, when the game is so obviously rigged against them?”

Have you ever read their writings about that causal model? They often appear in two-page newspaper ads. The problem with the model is that, in its fundamentals, it doesn’t make any sense. It’s not coherent enough that an outsider could actually get predictions out of it. They’ll write down something like a supergravity Lagrangian and draw arrows pointing to the terms in it labeled things like “bliss”, “immortality”, etc. How you get from that to “the square root of 1 percent of the world’s population” being a critical meditation mass to create world peace, I have no idea, and I strongly suspect that neither do they, because the hypothesis doesn’t have the internal coherence to make it clear. This experiment could be successfully replicated a hundred times, it it would tell us something surprising and important, but that business about the bliss term in the Lagrangian still wouldn’t make any sense.

The interesting thing is that physicists at Maharishi University actually do publish a lot of field theory work in mainstream physics journals (or at least they did in the 1990s when I was a grad student). The publication of these is completely noncontroversial, because they have to do with stuff that makes internal sense.

So I guess I do think some hypotheses deserve publication more than others. If I were to write a paper that said “zombies from Neptune made 2+2=5, and then a miracle occurs, and therefore I can predict the following about the physical properties of ammonium perchlorate”, and there followed a methodologically impeccable description of an experiment involving ammonium perchlorate, I would expect the referees to object to the theoretical background section, and so, I think, would any reasonable reader. Even if my result is right it doesn’t sanely tell you anything about the zombies from Neptune.

116

loren king 07.28.06 at 9:38 am

On the Crossley case study, linked to by Chris, I find the ‘observation before explanation’ rejoinder, while not unreasonable, to be a bit simplistic.

Sure, apples fell before Newton, but it isn’t as if there haven’t been widely held superstitions and metaphysical beliefs floating about for centuries, about why objects fall toward the earth. No doubt somewhere in the mists of evolutionary time, some early homo sapiens saw fruit fall, and maybe had the glimmerings of curiosity about it, absent anything we’d recognize as a causal story, but by the time anyone was systematically cataloguing the apparent facts of the natural world, I suspect there was already a significant background of familiar conjectures and suppositions that guided observations (with a desire either to confirm or challenge received wisdom). Many of those suppositions turned out to be bad explanatory models (cough – Aristotle), and were replaced by better ones once people did start to think more carefully about how things work.

That isn’t to deny the value of a scientific division of labour: no one should have to do all the work. But it’s a bit annoying when someone who publishes a case study (that just happens to support a clinical technique they endorse) cries foul when people ask for some sort of plausible causal story about what might be going on. (On the other hand, it seems as if at least some of her respondents may have been just a wee bit condescending, so perhaps this is something specific to this field, and to this exchange in particular.)

117

Steve LaBonne 07.28.06 at 9:47 am

Ray- you forgot one more possible dodge, really the “best” of all: if you’re Michael Behe, when you’re testifying in Dover and are confronted with a stack of papers which are all about the evolutionary origins of things you claim couldn’t have evolved, you just lie under oath, claiming you’ve read papers you’ve never actually read, and dismiss them with a wave of the hand as irrelevant. It goes without saying that you also don’t update / correct the lies in your published book to the effect that literature searches you claim to have done, in fields where there are quite a few papers, turned up nothing. Really, what we have here is nothing more than the time-dishonored creationist habit of lying for Jesus.

118

bi 07.28.06 at 9:57 am

Just one more remark: In a way, repeatability a.k.a. reproducibility also applies to singular things such as Stonehenge. While there’s only one Stonehenge, there are lots of people who can observe Stonehenge, so the “observability” of Stonehenge is itself repeatable. Can one say the same for Claude Vorilhon’s aliens?

119

loren king 07.28.06 at 10:05 am

Matt: “They’ll write down something like a supergravity Lagrangian and draw arrows pointing to the terms in it labeled things like “bliss”, “immortality”, etc.”

Wow, I’m posting way too much on this thread, but it’s a topic close to my heart (or at least, beer-tinged memories of college dorm life): back in my final year of undergrad, a bunch of us had the Maharishi Lagrangian ad plastered on our residence room doors, with different labels circled (I think I had “world peace”). We had to buy five or six copies of The Weekly World News to build our Maharishi “Wall of Wisdom” (we also included a related story about a meditating levitator who accidently materialized in a wall — bummer). When we got sick of doing grad school applications, we hatched a plan to rent a van and drive down for a week, to discuss this particular piece of mathematical artwork with whomever we found on their campus (“really? that‘s the “eternal bliss” element? ’cause we thought maybe, like, it was a distinct expression altogether”). Never happened, alas.

120

Christine 07.28.06 at 3:07 pm

Loren: fun idea on statistical methods course. We could do it!

Leaving aside the mechanisms issue, there’s a question of whether this is good empirical work, and if so whether it has implications for the policy experiment literature like that of Levitt (among very many other economists). It doesn’t seem to me to be particularly close to the natural experiment literature, really.

A quick critique of the paper, taking it very seriously. First: the time series model seems to include only a few weeks, which is quite frankly not long enough to develop a decent model. Second, their policy intervention variable (which actually seems to have been number of practitioners, rather than on or off?) is likely to be affected by violence on any given day itself (so not exogenous, though they do discuss this possibility). Third, there’s no control by comparison to another area or areas (eg: does meditation in Jerusalem affect violence in Korea?). They don’t even do a proper statistical comparison of Jerusalem v Lebanon, although their theory suggests no effect on Lebanon until numbers of practitioners are high enough.

There are a few recent papers in economics about issues re policy experiment studies (Bertrand/Duflo/Mullainathan on correlations in series across time; and Conley/Taber on small numbers of policy changes). But in most cases they are way more considered than this one. Most of the economics studies, including all of Levitt’s work, at least use other states as controls, and typically have more than one policy change (in the abortion case, different states changed policy at different times, and identification is from those timing differences).

Also, as Loren points out, it’s probably not particularly costly to run (true) random experiments of TM, which is certainly not the case for, say, abortion laws. So lets have more of them, I say!

Comments on this entry are closed.