Oh, man, that had to sting. Sometimes I’m glad I’m not a pure academic.
I have enough problems dealing with the persistent idiot in the room AND the realization that there’s at least three other people understanding the material about 10x better than I do as is.
Doing it for a living, instead of just for personal enlightment….
Also, the pays sucks. At least in the CS field. :)
I am indebted to John Rogers for the image, which seemed irresistibly appropriate in this case.
David Kane is more interesting than most Lancet denialists, because he at least lets the better angels of his nature peek out, as when he sent the draft of his (atrocious) statistical critique to Tim Lambert … then he promptly slams the door on them, as when he sent the wretched thing on to Michelle Malkin without waiting for criticism from the competent.
Kane seems to want to be accepted as a good-faith critic of the two Johns Hopkins studies, but he destroys any chance of this when he offers one contrived objection after another, accuses the authors of fraud, etc., etc. It is hard to tell what reality he is living in nowadays.
One would not think it possible, but it gets worse. On his own blog, Kane writes:
“One of the more annoying tendencies of Lancet defenders is that they refuse to recognize the breadth of academic criticism against the Lancet papers. Over time, I hope to gather some of that criticism together…”
Actually, that thread makes me feel terribly melancholy. I mean, give it a rhetorical push in a different direction, and it would be a great example of intelligent dialogue between scholars and consumers of scholars about methodology, etcetera. All it would take in this case is David Kane being a lot humbler, more curious and more generous in his questions about the Lancet study, more accepting of the good faith of researchers.
All it would take in this case is David Kane being a lot humbler, more curious and more generous in his questions about the Lancet study, more accepting of the good faith of researchers.
Well, this is the trouble with online dialogs of all kinds in this area: all it would take for them to be fantastic would be for people to be intellectually honest, responsive to evidence, and willing to accept that experts really can be making good-faith and correct arguments even when they disagree with you … The trouble is that achieving that is not a matter of the discussion getting “a rhetorical push” in one direction rather than another. It’s a matter of people willing to be intellectually honest, responsive to argument and evidence, etc, etc.
I rather agree with Burke: it’s depressing as hell. At first I was laughing–that “your boy took a hell of a beating” bit was great, though I had to look it up on Wikipedia–but I started to realize *this is how it happens*. This is how truth fails to win out; falsehood just refuses to *stay down*. And why should it? There is just *no cost* to doubling down at every refutation; his amplifiers honestly don’t care, and the true audience (those who just *want* to believe, for understandable psychological reasons) will eagerly seize at “disagreement” and “engagement” as evidence of reasonable doubt. This is all most likely for the sake of putting a “thanks to all the statisticians at Deltoid who helped with the latest version!” in a footnote to the next, equally absurd, “debunking” “paper”; and you bet it’ll be linked by Malkin et al.
In some ways this is a lot like the Beauchamp thing; there it was journalism, here it’s science.
Re 7, it also seemed to me that it might have helped if the people pointing out Kane’s mistakes had been more civil. There were a number of “gotcha! ha ha” remarks that he was actually able to refute while at the same time he did seem to be susceptible to reason, so the tone of the conversation made me cringe.
“It’s a matter of people willing to be intellectually honest, responsive to argument and evidence, etc”
Wasn’t Kane doing this by posting his work, engaging his critics civilly (for the most part – certainly more civilly than v.v.), and admitting when he’d been shown to be wrong?
“Wasn’t Kane doing this by posting his work, engaging his critics civilly (for the most part – certainly more civilly than v.v.), and admitting when he’d been shown to be wrong?”
*No*, actually. Not at all. And this is really quite important. The bottom line is that there were, and are, *no good reasons* for his skepticism: he’s not qualified to do the stats, he has no knowledge of the surveying strategies, &c. It’s not just that he’s had some errors pointed out, it’s that it has become clear that the only reason for his continuing to hold his beliefs is that he wants them to be true.
Being responsive to argument and evidence in this situation *just means* backing down entirely: admitting that the only appropriate posture for someone in his epistemic position is to believe the demographers of the world when they tell him he’s wrong and that it’s not their job to tell him why when he can’t seem to understand it.
It’s quite true he was being civil, but that’s not the point: the point is to pursue reliable procedures for getting at the truth. DK isn’t doing that, period.
Plus, of course, the absurdity of making the L1 study your white whale in the light of the last few years.
“The bottom line is that there were, and are, no good reasons for his skepticism”
I think it’s always reasonable to start from a position of skepticism. And on points such as, can one reproduce the CI with the available data there appeared to be some legitimate space for argument or at least confusion by those not true experts. The conclusion seems to be that parts of the data were available (to some people) (but are no longer) and the results on them check out (which I think he eventually admits) and that other parts of the data aren’t available (reasonably but perhaps annoyingly so if one is skeptical) so the basic result can’t be independently checked. If that’s a correct assessment you can see why Kane might reasonably be unsatisfied with the situation.
I readily admit that if some guy had for political reasons challenged a result of mine and wanted to rerun the analysis starting with the raw data but didn’t know enough particle physics to do the job, I would have been deeply annoyed and probably less than cooperative, but then again the invisible width of the Z isn’t of urgent policy interest.
rilkefan – there is a bit of a track record here – e.g. he has accused the authors of the study of fraud at least once to my knowledge (I think someone on the thread points out another instance), and otherwise behaved pretty badly (the whole Malkin thing). Direct accusations of fraud, however prettily expressed, do not for civil conversation make. If he isn’t getting the benefit of the doubt, it’s because he really doesn’t deserve it. Me, I’m just “high-fivin”:http://scienceblogs.com/deltoid/2007/08/robert_chung_on_david_kane.php#comment-553469 that he has denied being a political scientist.
Well, then fair enough, I only glancingly followed this story. I had thought the Malkin thing had just been him acting stupidly, not badly, and if he’s actually claimed fraud then I won’t defend him.
“I think it’s always reasonable to start from a position of skepticism.”
What could you possibly mean by this?
When a particle physicist like yourself tells me something about particle physics, I don’t start from a position of skepticism, nor should I. When a demographer tells me something about demographics, I don’t start from a position of skepticism, nor should I. When an astrologer tells me my future, I *do* start from a position of skepticism, as I should. Given the radical specialization of knowledge in the contemporary world, the most important tool for properly calibrating your beliefs is a good sense of knowing who to rely on and why.
Not only does DK not seem to have any sense of his limits, or any realistic set of priors (again, let’s be serious: does ANYONE think that the basic result of L1 or L2, that the death rate went up, is false?)–not only is he a bad seeker of truth, personally–but his entire project can only serve to distort other, less culpable, parties’ truth-detection abilities.
As Robert Chung puts it so well in his Comment, “Yes, in many cases that does mean answering questions; however, in your particular case, the situation is far more complex. You actually destroy knowledge, and your paper creates ignorance. In this situation, the way to be true to my professional responsibilities is to not be your enabler and to assure you that I ain’t your monkey.”
The difference between that thread and the Black Knight scene from Monty Python is that everyone can see that the Black Knight has, in fact, lost; nothing turns on his refusal to admit it. Whereas quite a lot turns on the refusal of unreliable epistemic authorities to “just stay down.”
Hence my first impulse to laugh, and my later impulse to cry.
In fact, Rilkefan, your very reaction on this thread exemplifies the danger: you, an intelligent and good-faith consumer of news, were misled by his polite tone into thinking that “there appeared to be some legitimate space for argument or at least confusion by those not true experts.” But this “legitimate … confusion” is entirely the *result* of efforts like DK’s.
If someone says X=a, it’s sensible to ask what the uncertainty on a is and how the systematics were controlled, to do a back-of-the-envelope check, to try to get some sense of the chi-squared for the old results in the field, … I learned recently that an error-estimation method I used in my thesis isn’t well-founded and can be plain wrong in parts of phase space. And Mrs. Rilkefan is a social scientist and tells me harrowing stuff about the methods of statistical analysis prevalent in her field.
“there appeared to be some legitimate space for argument or at least confusion by those not true experts”
Here I was in particular referring to the question of CI estimation, where SG was contradicted by Robert. Also, for the record, when I read the second paper I thought the backup of the analysis was really thin, and I’m unconvinced (without forming an opinion, since it’s not at all my field, I haven’t followed the debate closely, and the data’s not available) that the interview process and site selection were bias-free – it’s just really hard to measure complicated things.
Good faith skepticism starts with, “Ok, I want to look at why you’re making this claim, and your evidence for it. I don’t want to take anything on faith.” Not, “I’m sure you’re wrong, because the results you’re reporting aren’t convenient for my political views and for my common sense understanding of things.”–e.g., refusing to take seriously someone else’s findings *because* of a particular conviction or faith in an opposite finding.
‘Not “I’m sure you’re wrong, because the results you’re reporting aren’t convenient for my political views”‘
Sure – in fact one has to be especially careful about approaching results one dislikes. Of course this mirrors the importance of doing blinded analyses – I don’t recall if this result was produced that way.
“and for my common sense understanding of things.”
Here I’m not so sure it’s bad to say “X is unlikely because of my prior so I’ll give it extra scrutiny”. If the result in question had been a factor of ten higher it would have been reasonably rejected out of hand, no?
Rilkefan, the only part of that thread where there has been any evidence of disagreement amongst critics of David Kane is where I contradicted Robert (not the other way around I think) about whether or not one can or cannot construct the Confidence Intervals with only the data DK has. I hadn’t read some of Robert’s previous comments clearly and I immediately suggested to David that he pay heed to Robert rather than me.
That moment of “legitimate space for argument” arose after multiple threads in excess of 300 comments. During these comments it has become obvious that David Kane doesn’t even know how to calculate a crude mortality rate (CMR), its confidence intervals or a relative risk CI. So how could he have had “legitimate space for argument” in his original formulation of the argument on any basis but that of blind faith?
Also the “gotcha” moment you refer to is the point where everyone finally points out to DK that the reason he has been “unable to replicate” the CMRs is that he was using the wrong formula. Given that his “suspicions” are based on his not being able to replicate things, it hardly seems fair to describe this as a “gotcha” moment.
Also the reason DK has been so “civil” is that part of his tactic is to provoke incvility in his critics. He presents a blatantly wrong (oh, sooo wrong!) argument in a civil tone, and then provokes uncivil responses by ignoring all criticism and continuously insinuating fraud. Then he responds to the rudeness with typical civility, and his allies in the right-wing-brainfuck-osphere claim all leftist scientists are biassed frothing crazies. It’s a standard tactic. NOte that he isn’t always so polite though – in the original thread he deliberately singled out those who were “serious” (at that time dsquared and Robert) from those who were not (at that time me). Now I am in the “serious” club. This doesn’t strike me as very nice at all.
I have not been able to follow the statistical arguments very well, but it’s been quite clear what his strategy has been: Smiling and acting politely, so that any non-statistician looking in will see no understandable arguments, only one calm, dignified, polite academic and a bunch of incensed critics (which he has patiently wound up).
sg: “That moment of “legitimate space for argument†arose after multiple threads in excess of 300 comments.”
Which I haven’t read, fair enough, and if he started from a “this paper is a fraud” standpoint my initial response was dumb.
I just saw a guy trying to reproduce a result without the skills to do it but earnestly trying to learn from a bunch of rude experts who didn’t agree on a critical point and getting a bad-faith reception (‘Why did you post the data then’/’I only posted the public subset’; ‘Your spreadsheet omits the critical piece of info which refutes you – you’re cheating”/”No, I rechecked, it’s there”/”[Crickets]” [this is mostly what I meant by “gotcha”]) and, untrollishly, actually learning – and finally the situation boiling down to, the result can’t be replicated because of security concerns. Which, depending on one’s philosophy, may be somewhat unsatisfactory.
bi, you have to start somewhere (from a Bayesian POV anyway), and if you start out in an unlucky spot it takes a long time to get where you should be. It’s not always trivial to distinguish that situation from the data being bogus because you left some piece of equipment unplugged or whatever. It’s sensible to check the plugs when you get a surprising result.
and finally the situation boiling down to, the result can’t be replicated because of security concerns.
I can’t follow the stats either, so I’m not capable of evaluating the substance of the argument. But this is a misunderstanding of what was said, and assuming the people arguing with Kane are honest albeit rude, it looks like the sort of misunderstanding Kane was actively attempting to create.
Kane first (at least in this latest set of arguments) attacked the paper on the ground that some of the calculations were done wrongly in a way that he could demonstrate from the data released (that is, that a proper calculation of the confidence interval for total deaths, based on the information in the paper, would include zero.) This attack appears to have been not only wrong, but laughably wrong, indicative of his total misunderstanding of the statistics involved. (I can’t judge this for myself — I’m just summarizing the reactions of the people carrying on the argument.) And he fought it for hundreds of comments, never actually admitting his basic confusion.
His complaints didn’t “boil down to’ the result that he was complaining about can’t be replicated because of security concerns. The result he was complaining about can be replicated just fine — Robert Chung in the thread linked did it. Not all the data can be released because of security concerns, but that’s absolutely conventional for this sort of study, and obviously necessary when you consider the situation of the subjects. The complaint about unreleased data doesn’t salvage some fragment of Kane’s original complaint — it’s pure changing of the subject.
I just saw a guy trying to reproduce a result without the skills to do it but earnestly trying to learn from a bunch of rude experts who didn’t agree on a critical point and getting a bad-faith reception
First, sg directly above denied that there was any disagreement on a critical point; as one of the people involved in the disagreement, his statement that it wasn’t central should be worth something. Second, the maddening thing is that this guy is “a guy trying to reproduce a result without the skills to do it”, but rather than “earnestly trying to learn”, he’s putting up a paper on Michelle Malkin’s website in which he claims that his failure to reproduce the result is indicative of a flaw in the result rather than in his skills. If he weren’t publicizing himself as an expert in the relevant statistics, capable of debunking the paper, no one would be angry with him.
me: “and finally the situation boiling down to, the result can’t be replicated because of security concerns.”
lb: “I can’t follow the stats either, so I’m not capable of evaluating the substance of the argument. But this is a misunderstanding of what was said”
With all due respect, if you don’t have a background in measuring stuff, you can’t make that evaluation. The result _is_ a central value and a CI.
“Not all the data can be released because of security concerns, but that’s absolutely conventional for this sort of study, and obviously necessary when you consider the situation of the subjects.”
Possibly I am confused — I was relying on sg’s post as follows:
Rilkefan, the only part of that thread where there has been any evidence of disagreement amongst critics of David Kane is where I contradicted Robert (not the other way around I think) about whether or not one can or cannot construct the Confidence Intervals with only the data DK has. I hadn’t read some of Robert’s previous comments clearly and I immediately suggested to David that he pay heed to Robert rather than me.
This appears to say that there was no significant disagreement among the people criticizing Kane on that thread.
With all due respect, if you don’t have a background in measuring stuff, you can’t make that evaluation. The result is a central value and a CI.
It’s my understanding from the thread (and even without a background in measuring stuff, I can read with enough understanding to determine what claims are being made — just not whether they’re well supported) that the central value and CI Kane was criticizing can be replicated from the data released, at least closely enough to render his criticisms nonsensical.
Do you follow the point that Kane’s positioning himself as a hapless but honest and civil seeker of knowledge, being shamefully abused by those who know more than he does when all he wants is to be educated, is inconsistent with his publication of a purportedly expert ‘debunking’ of the Lancet studies, and that that inconsistency is what’s made people annoyed with him?
“you have to start somewhere (from a Bayesian POV anyway)”
One doesn’t just start _anywhere_. There’s only one correct place to start: our current state of actual knowledge. If I’m given a coin to toss, and I have no other actual knowledge of the coin, the only prior I can acceptably use is that which assigns equal probabilities to heads and tails. That’s the only correct starting point.
“finally the situation boiling down to, the result can’t be replicated because of security concerns.”
Hey folks, should we still remain civil when rilkefan starts making up bogus facts such as this?
And, come on… it’s good enough to mouth big sciencey-sounding words like “prior”, “Bayesian POV”, never mind if they’re being used wrongly and the mistakes are totally _elementary?_ Is this how rilkefan views science?
Civility of discourse is, of course, always a worthy goal—and, had rilkefan read the nearly two years of posts preceding this last round he would have found that David Kane has received more civility than perhaps he had deserved after having floated the fraud balloon. Nonetheless, incompetence should never trump truth simply because it speaks well and softly. You can speak with a civil tongue to a mule, but sometimes it pays more attention to a big stick. In this particular instance what works on mules also worked on a horse’s ass.
The CI-on-the-CMR argument is a red herring. It has no bearing on the conclusion of the 2004 Roberts article, though many have been taken in far enough to think it does. What has been the main consequence of Kane’s paper? Google up “David Kane Lancet” and you’ll find how it was interpreted: commentators nearly uniformly deferred on Kane’s statistical arguments (which are impenetrable not because the level of statistics is so rarefied but because they have been so poorly argued) and instead seized on Kane’s charge that the Roberts team had refused to supply the data or code needed in order to test the veracity of their conclusions. Malkin wrote that Kane himself emphasized this to her, and everyone else echoed it. Go ahead, google it up and check. Even among the intelligent (hi there, lizardbreath), not many understand statistical arguments, but everyone can understand when one side refuses to provide the data and code needed to test a claim.
However, David’s claim is false. David Kane was unable to replicate the results not because the Roberts team had refused to share data or code, but because he didn’t know how to do it. The information supplied by Roberts twenty months ago has always been sufficient to allow verification as long as you know what you’re doing.
This was the focus of the bet that David Kane refused, and still refuses, to honor. Bells cannot be unrung, but in this case David claims that there is nothing that needs correction or retraction. Over the last twenty months, we had shown him several different ways in which his claim was false—yet he continued to maintain and promulgate it. Not that I would recommend it to anyone who actually has anything resembling a life, but had rilkefan traced through those twenty months of posts, he’d see that more than a year ago, and then again before the JSM I showed David the exact code and exact data I used to replicate and verify the findings. Both times, this was in direct response to a request made by David; both times, David ignored the proof in front of him and continued to charge that the results were not replicable because Roberts had refused to share the data and code. In fact, David has charged fraud a second time: he has charged Roberts knowingly suppressed a finding from the original article because it would have undercut the conclusion. This was the statistical argument that no one could understand—and also has been shown to be flawed because David did not know how to calculate a proper CI.
But perhaps there is a case for fraud after all. Consider the prima facie evidence:
1. David Kane has a Harvard PhD. Could someone with a Harvard PhD truly be that incompetent?
2. David Kane is a fellow at the Institute for Quantitative Social Science at Harvard. Why did he not vet his paper with IQSS colleagues rather than send it to Malkin, telling her that the results in the Roberts article were not replicable?
3. There are clearly people affiliated with the IQSS who are competent to have judged the merits of the paper and to point out that the results were replicable: Gary King, the director of the IQSS and a person frequently cited by Kane, is the co-author of a forthcoming book on mortality forecasting. How could someone with a Harvard PhD, and affiliated with the IQSS, not avail himself of this resource unless he were acting in bad faith?
4. David Kane promoted his own R package on Iraq over the original data spreadsheet supplied by Roberts to him. That package included every line in the spreadsheet except the one line that showed how to calculate the CMR—the key statistic that Kane has claimed no one could replicate. Could a Harvard PhD affiliated with the IQSS truly have made such a careless oversight over the course of last 20 months when that charge is so central to his critique?
This is, of course, not proof. It is merely a chain of events that can point either to stunning incompetence, or willful fraud. In either event, one can, and should, conclude that David Kane is not credible. There are legitimate critiques, and legitimate critics, of the Roberts and Burnham papers. David Kane isn’t one of them.
I just saw a guy trying to reproduce a result without the skills to do it but earnestly trying to learn from a bunch of rude experts who didn’t agree on a critical point and getting a bad-faith reception (‘Why did you post the data then’/’I only posted the public subset’; ‘Your spreadsheet omits the critical piece of info which refutes you – you’re cheatingâ€/â€No, I rechecked, it’s thereâ€/â€[Crickets]†[this is mostly what I meant by “gotchaâ€]) and, untrollishly, actually learning – and finally the situation boiling down to, the result can’t be replicated because of security concerns. Which, depending on one’s philosophy, may be somewhat unsatisfactory.
That’s what you saw? Hmmm.
1. Had David been an earnest learner, he would have learned these things from the previous times when we corrected him.
2. I don’t think SG and I disagree on a critical point.
3. There is, however, a critical point: David’s claim was that nobody could replicate either the estimated CMRs or excess mortalities. Why didn’t he resolve it before he posted the paper?
4. The spreadsheet wasn’t David’s; Roberts supplied the spreadsheet to David, who re-copied every relevant bit of it except the part that he then claimed no one could replicate. You missed this. He re-checked that the original spreadsheet included the formula, and admitted that his package did not. Gotcha.
5. It’s unclear that David is actually learning: his current position is that the reason why he couldn’t replicate the CMR is because the CMR shouldn’t be defined the way it is. That was his “penance” post.
6. The results can be replicated.
Other than these errors, the rest of your paragraph is fine (viz., David is a guy trying to reproduce a result without the skills to do it, and I’m rude).
that “your boy took a hell of a beating†bit was great, though I had to look it up on Wikipedia
Ah, I think that’s a national touchstone: you’re either British or Norwegian if you know that one, or you’re unusually well-informed.
it might have helped if the people pointing out Kane’s mistakes had been more civil.
Fuck that for a game of soldiers. It might had helped if David! “Yes!” Kane! hadn’t gone running to Michelle Malkin and Michael Fumento for peer review.
“had rilkefan read the nearly two years of posts preceding this last round he would have found that David Kane has received more civility than perhaps he had deserved after having floated the fraud balloon.”
As I acknowledged above.
“4. The spreadsheet wasn’t David’s; Roberts supplied the spreadsheet to David, who re-copied every relevant bit of it except the part that he then claimed no one could replicate. You missed this. He re-checked that the original spreadsheet included the formula, and admitted that his package did not. Gotcha.”
I think he took you on faith in his first comment but then said: ‘I have just checked the R package linked above and, indeed, I did distribute the entire Excel spreadsheet including the cells you reference. I “suppressed” nothing.’
Did I miss something subsequent to that?
“6. The results can be replicated.”
Hmm, I see I got the sign wrong on who was withdrawing what – I’ll shut up now.
rilkefan asked in reference to David Kane’s R package:
Did I miss something subsequent to that?
No, you missed the precursor. David Kane included in his R package what he called “a cleaned up” version of the original spreadsheet. Go look at his R package. His cleaned up version included everything except the line that showed the CMR which he then claimed, both loudly and widely, could not be replicated from the data supplied, charging bad faith on the part of Roberts. David Kane now admits that his cleaned up data frame omits that single line, but doesn’t consider that to be suppressing anything. This is actually a more severe case than taking a single quoted sentence from some paper out of context; in that case, one can misunderstand or not read the previous or following sentences. In this case, he went to the trouble to reconstruct, clean up, re-format, and re-package all of the data except for a single line. David refers to that as being “careless.” Man, that’s carelessness. How odd that errors due to his carelessness always end up on one side of the ledger.
Speaking of which, lizardbreath was slightly off in #31: David Kane wanted to criticize the central estimate of excess mortality but had been getting a higher estimate than the one given by Roberts (cf. David’s post #63 in that thread). Funny how he never mentioned that earlier. Also, in that post he claimed that was the “first time” he’d ever seen anyone replicate the reported central estimate. That’s more carelessness: I’d shown it to him at least twice before. The difference is, on those previous occasions I’d used a more civil tone. Think he’ll overlook it this time?
Sorry to have come late to this party! There is much to respond to, but I’ll separate out the topics. I am especially hopeful to get some replies from Tim Burke, whom I respect very much and who I think has misunderstood some things.
Henry writes:
[Kane has] behaved pretty badly (the whole Malkin thing).
How should I have handled the “Malkin thing?” The sequence of events, undisputed by anyone, is:
1) I ask Tim Lambert to post a copy of a draft of my article on Deltoid so that I can get some comments. Tim kindly does so. Much fun discussion and learning occurs. Note, for example, how dsquared was confused in thinking that the bootstrap was used to calculate the confidence intervals for the CMR estimates. I was pleased to be able to teach dsquared something about the Lancet papers given how much I have learned from him.
2) Malkin reads my paper and the discussion at Deltoid. I did not contact her.
3) Malkin e-mails me for permission to post my paper. I gave it to her. She posted it. Now, I confess to having misunderstood her request. I thought that she was just going to quote it, link to the Deltoid discussion and, perhaps, host a copy at her site. I did not understand that she meant to post a copy of the entire thing. But even if I had understood her request, I would have said Yes.
How would Henry (and Tim Burke) suggest I behave differently? If some other blogger sent you a similar request, would you refuse it, deny them the right to post your paper? (It isn’t even clear what the legalities are here.) You would give them permission to link to it (you can’t deny them that) and quote from it (you can’t deny them that either), I would think. Is there any author at Crooked Timber who would (try to) deny another blogger link/quote permission.
So, one might say that that is what I should have done. But even if I had done that, I have no doubt that the attacks from people who don’t like Malkin would have been every bit as vociferous.
So, how about some advice from CT authors (and Tim Burke)? When I have a new draft of my paper and it is made public (either at Deltoid or elsewhere), what should I do when someone asks to link/quote/excerpt it?
This is not a direct reply but just a bit of background on why I and many others are Lancet skeptics. The L2 authors have made some of the data available. A dozen or so teams have asked to see it. All requests (including mine) have been granted except for one or two. The most prominent team to be turned down is also (surprisingly?) the Lancet’s most sophisticated and serious critic. These folks (Sean Gourley, Jukka-Pekka Onnela, Neil Johnson, Gesine Reinert and Michael Spagat) are all real academics who think that the Lancet results are very, very wrong.
Are Crooked Timber readers really comfortable defending a study which shares its data with some people but not with its most accomplished critics? I am not. In fact, I am unaware of a single similar situation, of the data for published paper being shared with some outsiders but not others. Can anyone point to a similar case? I know plenty of academics who never share any data but I have never seen a case of selective sharing, especially when the selection goes against the most informed critics.
Note that this is quote annoying for me! Instead of doing my own work on the project, I get to spend time answering questions from these folks because they don’t have access to the data themselves. (I am not allowed to share the data with them but I am allowed to answer their questions.) Quite annoying! Here is their latest work. How can anyone defend refusing to share data with these folks while simultaneously sharing the same data with almost everyone else who asks?
How can anyone defend refusing to share data with these folks while simultaneously sharing the same data with almost everyone else who asks?
Are you complaining about the poor manners of the L2 authors, or are you suggesting that sharing data with “almost everyone else who asks” is a method of concealing fraud or error? If the first, I’m not sure why it’s worthy of discussion. If the second, your point seems under-argued.
David Kane tells us in the Deltoid thread he links to: “Elizabeth Johnson (who did the statistical work on the paper) has confirmed that I am correct and dsquared is wrong. Crude mortality rates were not estimated via bootstrap.”
David, can you quote precisely what she wrote and what question she was responding to? It’s not a big deal but since you are the only source for this particular claim I’d like to know just what evidence you have.
OK, I have to admit I skipped over all the trite pleasantries, and all the technical discussion on right and wrong formulas and stuff, and all the accusations of suppression of data.
You see, I was having trouble figuring out why Malkin would want a draft of Kane’s paper. And why Kane would willingly oblige.
_Why would Malkin want a copy of Kane’s paper?_ So that she can read it and understand it? So that she can give informed, technical comments on it? (But given that Kane actually got a formula totally wrong) But she’s not a scientist or a statistician. Can Kane expect her to want to do anything else, other than trumpet his paper draft in a fit of Mindless Link Propagation(tm)?
_So why would Kane willingly send his paper to Malkin?_ Either he was thinking something silly at that time, or he truly wanted her to mindlessly propagate it. Either way, it doesn’t reflect well on him.
Which leads me to the next two questions: _Why did Kane not vet his paper with his Harvard IQSS colleagues._ And, _why’s he not doing that now, despite Robert’s advice above?_
Yes, here’s the “advice from CT authors” that Kane is asking for, and that’s already been given: vet the paper with IQSS colleagues who actually know their stuff.
Alternatively, Kane can send the next draft to Bill O’Reilly, to widen his Mindless Link Propagation(tm) network.
Just to clarify my question to David Kane, I’m obviously not suggesting that CMRs were “estimated via bootstrap”, whatever the hell that might mean. I just can’t see how she could have used that bizarre phrase and it would be nice to know what she was actually trying to convey.
1) Lizardbreath, I do not think that the L2 authors are guilty of “poor manners.” I believe it is ridiculous for them to refuse to share data with Gourley et al. I think it is unprecedented (counter-examples welcome!) to share data from a published paper with some critics but not others.
2) Kevin asks for a precise quote from the e-mail I received from Elizabeth Johnson. Can’t do that without her permission! The Lancet authors have asked me not to quote anything from our correspondence. (They have provided me with a couple of statements on specific topics which I have been able to quote.) I am not going to waste one of my favors from the authors just to satisfy your curiosity. Feel free to e-mail them yourself. What I have told you is the truth (as I deduced from circumstantial evidence in the Deltoid thread). The bootstrap was not used for any of the CMR calculations. Neither she nor I can be much more clear than that.
Good faith skepticism starts with, “Ok, I want to look at why you’re making this claim, and your evidence for it. I don’t want to take anything on faith.†Not, “I’m sure you’re wrong, because the results you’re reporting aren’t convenient for my political views and for my common sense understanding of things.â€â€”e.g., refusing to take seriously someone else’s findings because of a particular conviction or faith in an opposite finding.
I think that this is quite unfair as a description of what I have done. I came to this story in 2004 as an interested bystander. I had no priors except a generalized skepticism of applied work which just barely rejects the null hypothesis. I sought to replicate L1 and was stymied by the authors. They refused to share their code. They refused to share the data at a detailed level (as they eventually did with L2). Les Roberts was, to put it simply, quite rude.
But that was it. I didn’t do anything beyond make a few comments (all, I think reasonable) at Deltoid. I never wrote a paper, never contacted Malkin, never sought to make a big stink. I was certainly a skeptic, but a quite one. I came to the debate with exactly Tim’s recommended attitude: ““Ok, I want to look at why you’re making this claim, and your evidence for it. I don’t want to take anything on faith.†No one can provide any evidence to the contrary.
It was only when the 2006 paper came out, with its ‘not credible‘ response rate, that I really dove into the topic, really started to spend time on it, really started to write about. Now, it is hard for any one else to know if, deep within my own heart, I came to the debate, in 2006, with an attitude that would make Tim Burke proud, but that was two years into the debate, two years of being given the brush off by the L1 authors.
And, for the record, I am not certain that I am right! There is a good chance (5%, 10%?) that the L1 authors are correct. If they shared their data and code with everyone, we could all update our priors.
But let me turn the tables on Tim (and others). Jon Pedersen is someone respected by all sides in this debate. After reading :
Pedersen said he thinks the Lancet numbers are “high, and probably way too high. I would accept something in the vicinity of 100,000 but 600,000 is too much.”
Why are you so sure that the Lancet authors are correct and Pedersen is wrong?
If you weren’t an egregious hack, you’d have replied that the paper was still a work in progress, needed peer review etc, and politely asked her to abstain from linking and posting until you had gotten proper feedback on it. I don’t for a moment think that you weren’t entirely aware of the likely consequences – you have, after all, been at pains to tell us that _Little Green Footballs_ is one of your favourite blogs, so you can’t pretend that you don’t have some idea of what happens when this kind of thing gets politicized by the more enthusiastic folks on the right wing. As Robert said over to Deltoid, your net contribution to public knowledge has been heavily in the negative – and I don’t believe that you are or were unaware of this for a moment.
Nor is your reply to Tim Burke at #47 at all believable to anyone who has followed your track record. You came into this with an axe to grind. You’ve demonstrated a repeated lack of good faith and have more than once accused your targets of fraud based on your own quite tendentious misunderstandings. To put this another way – if the Lancet authors had done something as egregious as failing to include the one important formula in a package that they claimed represented the work of the other side in the debate (to name only the most obviously awful of your errors), you’d have been shouting fraud from the rooftops. What happened over at Deltoid isn’t intellectual debate – it’s garbage pick-up as art form (as a couple of commenters have noted, it’s depressing that it has to be done, but if it is to be done, at least it’s being done funnily and well).
by the way, I reckon people should leave off slagging off rilkefan – there’s no reason why everyone should have been aware of the sorry sequence of events, and those who only saw the recent Deltoid thread might very reasonably have failed to understand the context from which it sprang.
and final comment before I get back to my real life – the issue under debate in this thread isn’t whether people should be convinced that “the Lancet authors are correct” – they may indeed be wrong. The issue is whether people should or should not be convinced that _one particular critic_ of the Lancet authors – David Kane – is full of crap. And if you want to convince people that you aren’t – well after the various shenanigans of the last many months, you’re going to have your work cut out for you mate.
If you weren’t an egregious hack, you’d have replied that the paper was still a work in progress, needed peer review etc, and politely asked her to abstain from linking and posting until you had gotten proper feedback on it.
The purpose of posting it to Deltoid was to get feedback on it prior to submitting it for publication. Don’t you know academics who seek comments, who post drafts of their papers before submitting them for peer-review? I wanted and got feedback on it, but from Deltoid readers and from Malkin readers (although it is true that the former were more knowledgeable and helpful).
As Robert said over to Deltoid, your net contribution to public knowledge has been heavily in the negative – and I don’t believe that you are or were unaware of this for a moment.
The only reason that the data from L1 is publicly available is because of my efforts (and those of Tim Lambert). It is amazing how much you can know about me given the fact that we have never met. Again, if someone from, say, DailyKos asked you from permission to link/quote/excerpt one of your papers, would you turn them down?
Nor is your reply to Tim Burke at #47 at all believable to anyone who has followed your track record. You came into this with an axe to grind.
And you know this how? I was involved with this for two years before I published anything which anyone found objectionable. See my (few) previous comments at the old Deltoid. I even convinced Tim to correct a (very minor) mistake he had made with regard to the authors’s politics. If I am just in this to grind axes, then why wait two years to start the process?
To put this another way – if the Lancet authors had done something as egregious as failing to include the one important formula in a package that they claimed represented the work of the other side in the debate (to name only the most obviously awful of your errors), you’d have been shouting fraud from the rooftops.
This never happened. I included the entire spreadsheet from Deltoid in the R package which I distributed. I created a data frame from the raw data in exactly the way that any competent R programmer would have. Don’t believe me? Check for yourself. Just because Robert Chung says something does not make it true. Please try to educate yourself on these details before engaging in baseless slander.
and final comment before I get back to my real life – the issue under debate in this thread isn’t whether people should be convinced that “the Lancet authors are correct†– they may indeed be wrong. The issue is whether people should or should not be convinced that one particular critic of the Lancet authors – David Kane – is full of crap.
Charming. And you wonder why I think of Crooked Timber as the Little Green Footballs of the academic left? Comments like this are very LGFesque.
I have no problem with CT authors who don’t care about this dispute. Life is short. But your drive-by slander is annoying. Better to spend time at Deltoid where at least the critics take the time to read what they are criticizing.
David Kane, what kind of useful feedback were you expecting from Malkin and her readers again? And why do you still refuse to vet your paper with people at the IQSS?
And:
“What I have told you is the truth (as I deduced from circumstantial evidence in the Deltoid thread).”
“Please try to educate yourself on these details before engaging in baseless slander.”
From the circumstantial evidence of these two statements, I conclude that David Kane is a hack. This is the truth.
I was involved with this for two years before I published anything which anyone found objectionable.
when exactly did your fraud allegation occur, David?
I believe it is ridiculous for them to refuse to share data with Gourley et al.
except that you admitted on the Deltoid thread that these authors can’t use any publicly available data to check their main street bias thesis, due to confidentiality concerns.
I think it is unprecedented (counter-examples welcome!) to share data from a published paper with some critics but not others.
This claim is really starting to give me the shits. Why don’t you share with us details of any cases of data sharing that you know of, the circumstances of the sharing, and what happened? I think this is “unprecedented” only because you have never previously heard anything about how any authors share data, good or bad.
I suggest we write a paper which starts out like this:
Abstract
A recent study by Burnham et al. claimed that Iraq mortality rates were higher after the US intervention into Iraq than before. In this paper, we attempt to show that Burnham et al.’s results cannot be replicated; and even if they can be replicated, the definition of crude mortality rate (CMR) they used is a wrong one; and even if it is not a wrong one, the very fact that Burnham et al. refuse to release their data to Gourley et al. proves that their results are bogus; and even if they do send their data to Gourley et al., it does not invalidate any of the rest of the breadth of academic criticism against the Lancet papers; and even if it does invalidate the rest of the breadth of academic criticism against the Lancet papers, the truth remains that the Lancet papers are bad for freedom and capitalism and the American Way.
= = =
I invite Crooked Timber readers to fill in the rest of the paper while I sit back and offer virtual US$10,000 bills.
If we’re still on speaking terms with Steve Fuller I’d be interested in his views on this one because it seems to me to be an excellent test case of where something like his views on inclusiveness in science break down.
Methods
Just as Einstein proved his theory of relativity entirely through the use of sophisticated thought experiments, we construct proofs of all these assertions using an elaborate series of thought experiments based on
1) use of hazily defined prior distributions
2) symmetric normal distributions and their ratios
3) italicization of the words think and seems
It certainly seems that any proof based on these three components is logically perfect.
All data analysis was conducted in R, using code supplied by deltoid readers. Statistical significance is determined by the method of overlapping confidence intervals for sample statistics with unequal variances, which seem to work in all cases no matter the complexity of the problem. Any errors are the fault of the fundamental theory of epidemiology, which often seems to be wrong. Insinuations of fraud and incompetence were entirely the responsibility of the lead author, and we think they are self evident in every case.
I think the main long term consequence of this frankly excruciating episode will be that I (and I hope, others) will forever use “Excellent points!” as a means of subtly implying that I think my interlocutor might be completely at sea.
The central problem with the Lancet study was that it was conducted by people who, before the war started, were against the war, people who felt that the war was likely to increase civilian casualities and who, therefore, had a expectation/desire (unconscious or otherwise) to find the result that they found.
That sort of thing is a big problem with cancer research as well.
Damn! Tim said it better, and briefer, than I could.
Soldiering on, anyway …
You came into this with an axe to grind.
And you know this how? I was involved with this for two years before I published anything which anyone found objectionable.
[sound of gong]
By David Kane’s own account, the very first comment he made on this subject was the one just quoted. He entered this discussion by saying that the study could not be trusted because its authors’ motives were suspect, and that this was a valid objection to the study’s methodology and / or its results. And he has not deviated from that pattern of argument since then.
Look — I know this thread is titled “Hilarity”, so I hesitate to harsh anyone’s buzz. But I need to point one more time to the depressing “fraud balloon” incident. When Amy Perfors of Harvard’s IQSS removed David Kane’s post accusing Roberts et al. of fraud, she justified the act by saying that
The tone is unacceptable, the facts are shoddy, and the ideas are not endorsed by myself, the other authors on the sidebar, or the Harvard IQSS.
Now, some people might have interpreted that to mean that the “Case for Fraud” post was a warm turd in the punchbowl of reasoned discussion. But not David Kane. Lest anyone accuse him of being able to take a hint, he reproduced the offending post on his own blog. His privilege, of course. But what I see has an axe head and an axe handle, is recently sharpened and is in David Kane’s hands as he bends over a grindstone. Of course, maybe that’s what I wanted or expected to see, so who knows if it’s really an axe to be ground after all?
{ 1 trackback }
{ 65 comments }
dsquared 09.07.07 at 4:57 pm
Hilarious and simultaneously awful. I honestly couldn’t bear it after a while. Tim’s latest post is pretty hilarious too.
Morat20 09.07.07 at 5:47 pm
Oh, man, that had to sting. Sometimes I’m glad I’m not a pure academic.
I have enough problems dealing with the persistent idiot in the room AND the realization that there’s at least three other people understanding the material about 10x better than I do as is.
Doing it for a living, instead of just for personal enlightment….
Also, the pays sucks. At least in the CS field. :)
quicksand 09.07.07 at 5:56 pm
I just about lost it at the comment appealing for Kane to just “stay down!”
What a perfect image — a badly outclassed fighter, way too punch-drunk to realize when to quit.
jre 09.07.07 at 6:21 pm
I am indebted to John Rogers for the image, which seemed irresistibly appropriate in this case.
David Kane is more interesting than most Lancet denialists, because he at least lets the better angels of his nature peek out, as when he sent the draft of his (atrocious) statistical critique to Tim Lambert … then he promptly slams the door on them, as when he sent the wretched thing on to Michelle Malkin without waiting for criticism from the competent.
Kane seems to want to be accepted as a good-faith critic of the two Johns Hopkins studies, but he destroys any chance of this when he offers one contrived objection after another, accuses the authors of fraud, etc., etc. It is hard to tell what reality he is living in nowadays.
trane 09.07.07 at 7:03 pm
One would not think it possible, but it gets worse. On his own blog, Kane writes:
“One of the more annoying tendencies of Lancet defenders is that they refuse to recognize the breadth of academic criticism against the Lancet papers. Over time, I hope to gather some of that criticism together…”
http://lancetiraq.blogspot.com/2007/09/amer-and-scheuren.html
Stuart 09.07.07 at 7:18 pm
refuse to recognize the breadth of academic criticism against the Lancet papers.
Even if there was broad academic criticism of the Lancet papers, wouldn’t accurate criticism be slightly more useful?
Timothy Burke 09.07.07 at 8:00 pm
Actually, that thread makes me feel terribly melancholy. I mean, give it a rhetorical push in a different direction, and it would be a great example of intelligent dialogue between scholars and consumers of scholars about methodology, etcetera. All it would take in this case is David Kane being a lot humbler, more curious and more generous in his questions about the Lancet study, more accepting of the good faith of researchers.
Grand Moff Texan 09.07.07 at 8:44 pm
As I once said about a similar “debate” over on Pharyngula, it’s like watching a truck run over a chicken, again and again.
.
Kieran Healy 09.07.07 at 9:58 pm
All it would take in this case is David Kane being a lot humbler, more curious and more generous in his questions about the Lancet study, more accepting of the good faith of researchers.
Well, this is the trouble with online dialogs of all kinds in this area: all it would take for them to be fantastic would be for people to be intellectually honest, responsive to evidence, and willing to accept that experts really can be making good-faith and correct arguments even when they disagree with you … The trouble is that achieving that is not a matter of the discussion getting “a rhetorical push” in one direction rather than another. It’s a matter of people willing to be intellectually honest, responsive to argument and evidence, etc, etc.
stuart 09.07.07 at 10:21 pm
It’s a matter of people willing to be intellectually honest, responsive to argument and evidence, etc, etc.
Which no doubt everyone agrees everyone else should be, but they are right so all that stuff is unnecessary for them, of course.
X. Trapnel 09.07.07 at 10:32 pm
I rather agree with Burke: it’s depressing as hell. At first I was laughing–that “your boy took a hell of a beating” bit was great, though I had to look it up on Wikipedia–but I started to realize *this is how it happens*. This is how truth fails to win out; falsehood just refuses to *stay down*. And why should it? There is just *no cost* to doubling down at every refutation; his amplifiers honestly don’t care, and the true audience (those who just *want* to believe, for understandable psychological reasons) will eagerly seize at “disagreement” and “engagement” as evidence of reasonable doubt. This is all most likely for the sake of putting a “thanks to all the statisticians at Deltoid who helped with the latest version!” in a footnote to the next, equally absurd, “debunking” “paper”; and you bet it’ll be linked by Malkin et al.
In some ways this is a lot like the Beauchamp thing; there it was journalism, here it’s science.
I think I’ll go home and cry now.
rilkefan 09.07.07 at 10:47 pm
Re 7, it also seemed to me that it might have helped if the people pointing out Kane’s mistakes had been more civil. There were a number of “gotcha! ha ha” remarks that he was actually able to refute while at the same time he did seem to be susceptible to reason, so the tone of the conversation made me cringe.
rilkefan 09.07.07 at 10:49 pm
“It’s a matter of people willing to be intellectually honest, responsive to argument and evidence, etc”
Wasn’t Kane doing this by posting his work, engaging his critics civilly (for the most part – certainly more civilly than v.v.), and admitting when he’d been shown to be wrong?
X. Trapnel 09.07.07 at 10:59 pm
“Wasn’t Kane doing this by posting his work, engaging his critics civilly (for the most part – certainly more civilly than v.v.), and admitting when he’d been shown to be wrong?”
*No*, actually. Not at all. And this is really quite important. The bottom line is that there were, and are, *no good reasons* for his skepticism: he’s not qualified to do the stats, he has no knowledge of the surveying strategies, &c. It’s not just that he’s had some errors pointed out, it’s that it has become clear that the only reason for his continuing to hold his beliefs is that he wants them to be true.
Being responsive to argument and evidence in this situation *just means* backing down entirely: admitting that the only appropriate posture for someone in his epistemic position is to believe the demographers of the world when they tell him he’s wrong and that it’s not their job to tell him why when he can’t seem to understand it.
It’s quite true he was being civil, but that’s not the point: the point is to pursue reliable procedures for getting at the truth. DK isn’t doing that, period.
Plus, of course, the absurdity of making the L1 study your white whale in the light of the last few years.
rilkefan 09.07.07 at 11:29 pm
“The bottom line is that there were, and are, no good reasons for his skepticism”
I think it’s always reasonable to start from a position of skepticism. And on points such as, can one reproduce the CI with the available data there appeared to be some legitimate space for argument or at least confusion by those not true experts. The conclusion seems to be that parts of the data were available (to some people) (but are no longer) and the results on them check out (which I think he eventually admits) and that other parts of the data aren’t available (reasonably but perhaps annoyingly so if one is skeptical) so the basic result can’t be independently checked. If that’s a correct assessment you can see why Kane might reasonably be unsatisfied with the situation.
I readily admit that if some guy had for political reasons challenged a result of mine and wanted to rerun the analysis starting with the raw data but didn’t know enough particle physics to do the job, I would have been deeply annoyed and probably less than cooperative, but then again the invisible width of the Z isn’t of urgent policy interest.
Henry 09.08.07 at 12:02 am
rilkefan – there is a bit of a track record here – e.g. he has accused the authors of the study of fraud at least once to my knowledge (I think someone on the thread points out another instance), and otherwise behaved pretty badly (the whole Malkin thing). Direct accusations of fraud, however prettily expressed, do not for civil conversation make. If he isn’t getting the benefit of the doubt, it’s because he really doesn’t deserve it. Me, I’m just “high-fivin”:http://scienceblogs.com/deltoid/2007/08/robert_chung_on_david_kane.php#comment-553469 that he has denied being a political scientist.
rilkefan 09.08.07 at 12:12 am
“there is a bit of a track record here”
Well, then fair enough, I only glancingly followed this story. I had thought the Malkin thing had just been him acting stupidly, not badly, and if he’s actually claimed fraud then I won’t defend him.
X. Trapnel 09.08.07 at 12:19 am
“I think it’s always reasonable to start from a position of skepticism.”
What could you possibly mean by this?
When a particle physicist like yourself tells me something about particle physics, I don’t start from a position of skepticism, nor should I. When a demographer tells me something about demographics, I don’t start from a position of skepticism, nor should I. When an astrologer tells me my future, I *do* start from a position of skepticism, as I should. Given the radical specialization of knowledge in the contemporary world, the most important tool for properly calibrating your beliefs is a good sense of knowing who to rely on and why.
Not only does DK not seem to have any sense of his limits, or any realistic set of priors (again, let’s be serious: does ANYONE think that the basic result of L1 or L2, that the death rate went up, is false?)–not only is he a bad seeker of truth, personally–but his entire project can only serve to distort other, less culpable, parties’ truth-detection abilities.
As Robert Chung puts it so well in his Comment, “Yes, in many cases that does mean answering questions; however, in your particular case, the situation is far more complex. You actually destroy knowledge, and your paper creates ignorance. In this situation, the way to be true to my professional responsibilities is to not be your enabler and to assure you that I ain’t your monkey.”
The difference between that thread and the Black Knight scene from Monty Python is that everyone can see that the Black Knight has, in fact, lost; nothing turns on his refusal to admit it. Whereas quite a lot turns on the refusal of unreliable epistemic authorities to “just stay down.”
Hence my first impulse to laugh, and my later impulse to cry.
X. Trapnel 09.08.07 at 12:23 am
In fact, Rilkefan, your very reaction on this thread exemplifies the danger: you, an intelligent and good-faith consumer of news, were misled by his polite tone into thinking that “there appeared to be some legitimate space for argument or at least confusion by those not true experts.” But this “legitimate … confusion” is entirely the *result* of efforts like DK’s.
rilkefan 09.08.07 at 1:11 am
“What could you possibly mean by this?”
If someone says X=a, it’s sensible to ask what the uncertainty on a is and how the systematics were controlled, to do a back-of-the-envelope check, to try to get some sense of the chi-squared for the old results in the field, … I learned recently that an error-estimation method I used in my thesis isn’t well-founded and can be plain wrong in parts of phase space. And Mrs. Rilkefan is a social scientist and tells me harrowing stuff about the methods of statistical analysis prevalent in her field.
“there appeared to be some legitimate space for argument or at least confusion by those not true experts”
Here I was in particular referring to the question of CI estimation, where SG was contradicted by Robert. Also, for the record, when I read the second paper I thought the backup of the analysis was really thin, and I’m unconvinced (without forming an opinion, since it’s not at all my field, I haven’t followed the debate closely, and the data’s not available) that the interview process and site selection were bias-free – it’s just really hard to measure complicated things.
Timothy Burke 09.08.07 at 2:17 am
Good faith skepticism starts with, “Ok, I want to look at why you’re making this claim, and your evidence for it. I don’t want to take anything on faith.” Not, “I’m sure you’re wrong, because the results you’re reporting aren’t convenient for my political views and for my common sense understanding of things.”–e.g., refusing to take seriously someone else’s findings *because* of a particular conviction or faith in an opposite finding.
rilkefan 09.08.07 at 3:29 am
‘Not “I’m sure you’re wrong, because the results you’re reporting aren’t convenient for my political views”‘
Sure – in fact one has to be especially careful about approaching results one dislikes. Of course this mirrors the importance of doing blinded analyses – I don’t recall if this result was produced that way.
“and for my common sense understanding of things.”
Here I’m not so sure it’s bad to say “X is unlikely because of my prior so I’ll give it extra scrutiny”. If the result in question had been a factor of ten higher it would have been reasonably rejected out of hand, no?
bi 09.08.07 at 5:15 am
rilkefan:
“Here I’m not so sure it’s bad to say ‘X is unlikely because of my prior so I’ll give it extra scrutiny’.”
Prior based on _what?_ When this “prior” has no evidence backing it up, the word “prior” is just a pseudo-scientific euphemism for “ideology”.
SG 09.08.07 at 7:07 am
Rilkefan, the only part of that thread where there has been any evidence of disagreement amongst critics of David Kane is where I contradicted Robert (not the other way around I think) about whether or not one can or cannot construct the Confidence Intervals with only the data DK has. I hadn’t read some of Robert’s previous comments clearly and I immediately suggested to David that he pay heed to Robert rather than me.
That moment of “legitimate space for argument” arose after multiple threads in excess of 300 comments. During these comments it has become obvious that David Kane doesn’t even know how to calculate a crude mortality rate (CMR), its confidence intervals or a relative risk CI. So how could he have had “legitimate space for argument” in his original formulation of the argument on any basis but that of blind faith?
Also the “gotcha” moment you refer to is the point where everyone finally points out to DK that the reason he has been “unable to replicate” the CMRs is that he was using the wrong formula. Given that his “suspicions” are based on his not being able to replicate things, it hardly seems fair to describe this as a “gotcha” moment.
Also the reason DK has been so “civil” is that part of his tactic is to provoke incvility in his critics. He presents a blatantly wrong (oh, sooo wrong!) argument in a civil tone, and then provokes uncivil responses by ignoring all criticism and continuously insinuating fraud. Then he responds to the rudeness with typical civility, and his allies in the right-wing-brainfuck-osphere claim all leftist scientists are biassed frothing crazies. It’s a standard tactic. NOte that he isn’t always so polite though – in the original thread he deliberately singled out those who were “serious” (at that time dsquared and Robert) from those who were not (at that time me). Now I am in the “serious” club. This doesn’t strike me as very nice at all.
Harald Korneliussen 09.08.07 at 12:28 pm
“Excellent suggestions!”
I have not been able to follow the statistical arguments very well, but it’s been quite clear what his strategy has been: Smiling and acting politely, so that any non-statistician looking in will see no understandable arguments, only one calm, dignified, polite academic and a bunch of incensed critics (which he has patiently wound up).
rilkefan 09.08.07 at 5:00 pm
sg: “That moment of “legitimate space for argument†arose after multiple threads in excess of 300 comments.”
Which I haven’t read, fair enough, and if he started from a “this paper is a fraud” standpoint my initial response was dumb.
I just saw a guy trying to reproduce a result without the skills to do it but earnestly trying to learn from a bunch of rude experts who didn’t agree on a critical point and getting a bad-faith reception (‘Why did you post the data then’/’I only posted the public subset’; ‘Your spreadsheet omits the critical piece of info which refutes you – you’re cheating”/”No, I rechecked, it’s there”/”[Crickets]” [this is mostly what I meant by “gotcha”]) and, untrollishly, actually learning – and finally the situation boiling down to, the result can’t be replicated because of security concerns. Which, depending on one’s philosophy, may be somewhat unsatisfactory.
bi, you have to start somewhere (from a Bayesian POV anyway), and if you start out in an unlucky spot it takes a long time to get where you should be. It’s not always trivial to distinguish that situation from the data being bogus because you left some piece of equipment unplugged or whatever. It’s sensible to check the plugs when you get a surprising result.
LizardBreath 09.08.07 at 5:14 pm
and finally the situation boiling down to, the result can’t be replicated because of security concerns.
I can’t follow the stats either, so I’m not capable of evaluating the substance of the argument. But this is a misunderstanding of what was said, and assuming the people arguing with Kane are honest albeit rude, it looks like the sort of misunderstanding Kane was actively attempting to create.
Kane first (at least in this latest set of arguments) attacked the paper on the ground that some of the calculations were done wrongly in a way that he could demonstrate from the data released (that is, that a proper calculation of the confidence interval for total deaths, based on the information in the paper, would include zero.) This attack appears to have been not only wrong, but laughably wrong, indicative of his total misunderstanding of the statistics involved. (I can’t judge this for myself — I’m just summarizing the reactions of the people carrying on the argument.) And he fought it for hundreds of comments, never actually admitting his basic confusion.
His complaints didn’t “boil down to’ the result that he was complaining about can’t be replicated because of security concerns. The result he was complaining about can be replicated just fine — Robert Chung in the thread linked did it. Not all the data can be released because of security concerns, but that’s absolutely conventional for this sort of study, and obviously necessary when you consider the situation of the subjects. The complaint about unreleased data doesn’t salvage some fragment of Kane’s original complaint — it’s pure changing of the subject.
I just saw a guy trying to reproduce a result without the skills to do it but earnestly trying to learn from a bunch of rude experts who didn’t agree on a critical point and getting a bad-faith reception
First, sg directly above denied that there was any disagreement on a critical point; as one of the people involved in the disagreement, his statement that it wasn’t central should be worth something. Second, the maddening thing is that this guy is “a guy trying to reproduce a result without the skills to do it”, but rather than “earnestly trying to learn”, he’s putting up a paper on Michelle Malkin’s website in which he claims that his failure to reproduce the result is indicative of a flaw in the result rather than in his skills. If he weren’t publicizing himself as an expert in the relevant statistics, capable of debunking the paper, no one would be angry with him.
rilkefan 09.08.07 at 6:54 pm
me: “and finally the situation boiling down to, the result can’t be replicated because of security concerns.”
lb: “I can’t follow the stats either, so I’m not capable of evaluating the substance of the argument. But this is a misunderstanding of what was said”
With all due respect, if you don’t have a background in measuring stuff, you can’t make that evaluation. The result _is_ a central value and a CI.
“Not all the data can be released because of security concerns, but that’s absolutely conventional for this sort of study, and obviously necessary when you consider the situation of the subjects.”
True, but irrelevant to my conclusion.
rilkefan 09.08.07 at 7:01 pm
lb: “First, sg directly above denied that there was any disagreement on a critical point”
I think you’re confused here.
LizardBreath 09.08.07 at 7:17 pm
Possibly I am confused — I was relying on sg’s post as follows:
This appears to say that there was no significant disagreement among the people criticizing Kane on that thread.
With all due respect, if you don’t have a background in measuring stuff, you can’t make that evaluation. The result is a central value and a CI.
It’s my understanding from the thread (and even without a background in measuring stuff, I can read with enough understanding to determine what claims are being made — just not whether they’re well supported) that the central value and CI Kane was criticizing can be replicated from the data released, at least closely enough to render his criticisms nonsensical.
Do you follow the point that Kane’s positioning himself as a hapless but honest and civil seeker of knowledge, being shamefully abused by those who know more than he does when all he wants is to be educated, is inconsistent with his publication of a purportedly expert ‘debunking’ of the Lancet studies, and that that inconsistency is what’s made people annoyed with him?
LizardBreath 09.08.07 at 7:18 pm
Sorry, I don’t know why I said Kane was criticizing the central value — I don’t think he did.
bi 09.08.07 at 7:41 pm
rilkefan:
“you have to start somewhere (from a Bayesian POV anyway)”
One doesn’t just start _anywhere_. There’s only one correct place to start: our current state of actual knowledge. If I’m given a coin to toss, and I have no other actual knowledge of the coin, the only prior I can acceptably use is that which assigns equal probabilities to heads and tails. That’s the only correct starting point.
“finally the situation boiling down to, the result can’t be replicated because of security concerns.”
Hey folks, should we still remain civil when rilkefan starts making up bogus facts such as this?
bi 09.08.07 at 7:51 pm
And, come on… it’s good enough to mouth big sciencey-sounding words like “prior”, “Bayesian POV”, never mind if they’re being used wrongly and the mistakes are totally _elementary?_ Is this how rilkefan views science?
It certainly is how Republicans view science.
Robert 09.08.07 at 7:57 pm
Civility of discourse is, of course, always a worthy goal—and, had rilkefan read the nearly two years of posts preceding this last round he would have found that David Kane has received more civility than perhaps he had deserved after having floated the fraud balloon. Nonetheless, incompetence should never trump truth simply because it speaks well and softly. You can speak with a civil tongue to a mule, but sometimes it pays more attention to a big stick. In this particular instance what works on mules also worked on a horse’s ass.
The CI-on-the-CMR argument is a red herring. It has no bearing on the conclusion of the 2004 Roberts article, though many have been taken in far enough to think it does. What has been the main consequence of Kane’s paper? Google up “David Kane Lancet” and you’ll find how it was interpreted: commentators nearly uniformly deferred on Kane’s statistical arguments (which are impenetrable not because the level of statistics is so rarefied but because they have been so poorly argued) and instead seized on Kane’s charge that the Roberts team had refused to supply the data or code needed in order to test the veracity of their conclusions. Malkin wrote that Kane himself emphasized this to her, and everyone else echoed it. Go ahead, google it up and check. Even among the intelligent (hi there, lizardbreath), not many understand statistical arguments, but everyone can understand when one side refuses to provide the data and code needed to test a claim.
However, David’s claim is false. David Kane was unable to replicate the results not because the Roberts team had refused to share data or code, but because he didn’t know how to do it. The information supplied by Roberts twenty months ago has always been sufficient to allow verification as long as you know what you’re doing.
This was the focus of the bet that David Kane refused, and still refuses, to honor. Bells cannot be unrung, but in this case David claims that there is nothing that needs correction or retraction. Over the last twenty months, we had shown him several different ways in which his claim was false—yet he continued to maintain and promulgate it. Not that I would recommend it to anyone who actually has anything resembling a life, but had rilkefan traced through those twenty months of posts, he’d see that more than a year ago, and then again before the JSM I showed David the exact code and exact data I used to replicate and verify the findings. Both times, this was in direct response to a request made by David; both times, David ignored the proof in front of him and continued to charge that the results were not replicable because Roberts had refused to share the data and code. In fact, David has charged fraud a second time: he has charged Roberts knowingly suppressed a finding from the original article because it would have undercut the conclusion. This was the statistical argument that no one could understand—and also has been shown to be flawed because David did not know how to calculate a proper CI.
But perhaps there is a case for fraud after all. Consider the prima facie evidence:
1. David Kane has a Harvard PhD. Could someone with a Harvard PhD truly be that incompetent?
2. David Kane is a fellow at the Institute for Quantitative Social Science at Harvard. Why did he not vet his paper with IQSS colleagues rather than send it to Malkin, telling her that the results in the Roberts article were not replicable?
3. There are clearly people affiliated with the IQSS who are competent to have judged the merits of the paper and to point out that the results were replicable: Gary King, the director of the IQSS and a person frequently cited by Kane, is the co-author of a forthcoming book on mortality forecasting. How could someone with a Harvard PhD, and affiliated with the IQSS, not avail himself of this resource unless he were acting in bad faith?
4. David Kane promoted his own R package on Iraq over the original data spreadsheet supplied by Roberts to him. That package included every line in the spreadsheet except the one line that showed how to calculate the CMR—the key statistic that Kane has claimed no one could replicate. Could a Harvard PhD affiliated with the IQSS truly have made such a careless oversight over the course of last 20 months when that charge is so central to his critique?
This is, of course, not proof. It is merely a chain of events that can point either to stunning incompetence, or willful fraud. In either event, one can, and should, conclude that David Kane is not credible. There are legitimate critiques, and legitimate critics, of the Roberts and Burnham papers. David Kane isn’t one of them.
Robert 09.09.07 at 12:16 am
rilkefan wrote:
That’s what you saw? Hmmm.
1. Had David been an earnest learner, he would have learned these things from the previous times when we corrected him.
2. I don’t think SG and I disagree on a critical point.
3. There is, however, a critical point: David’s claim was that nobody could replicate either the estimated CMRs or excess mortalities. Why didn’t he resolve it before he posted the paper?
4. The spreadsheet wasn’t David’s; Roberts supplied the spreadsheet to David, who re-copied every relevant bit of it except the part that he then claimed no one could replicate. You missed this. He re-checked that the original spreadsheet included the formula, and admitted that his package did not. Gotcha.
5. It’s unclear that David is actually learning: his current position is that the reason why he couldn’t replicate the CMR is because the CMR shouldn’t be defined the way it is. That was his “penance” post.
6. The results can be replicated.
Other than these errors, the rest of your paragraph is fine (viz., David is a guy trying to reproduce a result without the skills to do it, and I’m rude).
nick s 09.09.07 at 7:13 am
that “your boy took a hell of a beating†bit was great, though I had to look it up on Wikipedia
Ah, I think that’s a national touchstone: you’re either British or Norwegian if you know that one, or you’re unusually well-informed.
it might have helped if the people pointing out Kane’s mistakes had been more civil.
Fuck that for a game of soldiers. It might had helped if David! “Yes!” Kane! hadn’t gone running to Michelle Malkin and Michael Fumento for peer review.
rilkefan 09.09.07 at 4:36 pm
“had rilkefan read the nearly two years of posts preceding this last round he would have found that David Kane has received more civility than perhaps he had deserved after having floated the fraud balloon.”
As I acknowledged above.
“4. The spreadsheet wasn’t David’s; Roberts supplied the spreadsheet to David, who re-copied every relevant bit of it except the part that he then claimed no one could replicate. You missed this. He re-checked that the original spreadsheet included the formula, and admitted that his package did not. Gotcha.”
I think he took you on faith in his first comment but then said: ‘I have just checked the R package linked above and, indeed, I did distribute the entire Excel spreadsheet including the cells you reference. I “suppressed” nothing.’
Did I miss something subsequent to that?
“6. The results can be replicated.”
Hmm, I see I got the sign wrong on who was withdrawing what – I’ll shut up now.
Robert 09.09.07 at 7:10 pm
rilkefan asked in reference to David Kane’s R package:
No, you missed the precursor. David Kane included in his R package what he called “a cleaned up” version of the original spreadsheet. Go look at his R package. His cleaned up version included everything except the line that showed the CMR which he then claimed, both loudly and widely, could not be replicated from the data supplied, charging bad faith on the part of Roberts. David Kane now admits that his cleaned up data frame omits that single line, but doesn’t consider that to be suppressing anything. This is actually a more severe case than taking a single quoted sentence from some paper out of context; in that case, one can misunderstand or not read the previous or following sentences. In this case, he went to the trouble to reconstruct, clean up, re-format, and re-package all of the data except for a single line. David refers to that as being “careless.” Man, that’s carelessness. How odd that errors due to his carelessness always end up on one side of the ledger.
Speaking of which, lizardbreath was slightly off in #31: David Kane wanted to criticize the central estimate of excess mortality but had been getting a higher estimate than the one given by Roberts (cf. David’s post #63 in that thread). Funny how he never mentioned that earlier. Also, in that post he claimed that was the “first time” he’d ever seen anyone replicate the reported central estimate. That’s more carelessness: I’d shown it to him at least twice before. The difference is, on those previous occasions I’d used a more civil tone. Think he’ll overlook it this time?
David Kane 09.09.07 at 7:48 pm
Sorry to have come late to this party! There is much to respond to, but I’ll separate out the topics. I am especially hopeful to get some replies from Tim Burke, whom I respect very much and who I think has misunderstood some things.
Henry writes:
How should I have handled the “Malkin thing?” The sequence of events, undisputed by anyone, is:
1) I ask Tim Lambert to post a copy of a draft of my article on Deltoid so that I can get some comments. Tim kindly does so. Much fun discussion and learning occurs. Note, for example, how dsquared was confused in thinking that the bootstrap was used to calculate the confidence intervals for the CMR estimates. I was pleased to be able to teach dsquared something about the Lancet papers given how much I have learned from him.
2) Malkin reads my paper and the discussion at Deltoid. I did not contact her.
3) Malkin e-mails me for permission to post my paper. I gave it to her. She posted it. Now, I confess to having misunderstood her request. I thought that she was just going to quote it, link to the Deltoid discussion and, perhaps, host a copy at her site. I did not understand that she meant to post a copy of the entire thing. But even if I had understood her request, I would have said Yes.
How would Henry (and Tim Burke) suggest I behave differently? If some other blogger sent you a similar request, would you refuse it, deny them the right to post your paper? (It isn’t even clear what the legalities are here.) You would give them permission to link to it (you can’t deny them that) and quote from it (you can’t deny them that either), I would think. Is there any author at Crooked Timber who would (try to) deny another blogger link/quote permission.
So, one might say that that is what I should have done. But even if I had done that, I have no doubt that the attacks from people who don’t like Malkin would have been every bit as vociferous.
So, how about some advice from CT authors (and Tim Burke)? When I have a new draft of my paper and it is made public (either at Deltoid or elsewhere), what should I do when someone asks to link/quote/excerpt it?
David Kane 09.09.07 at 8:04 pm
This is not a direct reply but just a bit of background on why I and many others are Lancet skeptics. The L2 authors have made some of the data available. A dozen or so teams have asked to see it. All requests (including mine) have been granted except for one or two. The most prominent team to be turned down is also (surprisingly?) the Lancet’s most sophisticated and serious critic. These folks (Sean Gourley, Jukka-Pekka Onnela, Neil Johnson, Gesine Reinert and Michael Spagat) are all real academics who think that the Lancet results are very, very wrong.
Are Crooked Timber readers really comfortable defending a study which shares its data with some people but not with its most accomplished critics? I am not. In fact, I am unaware of a single similar situation, of the data for published paper being shared with some outsiders but not others. Can anyone point to a similar case? I know plenty of academics who never share any data but I have never seen a case of selective sharing, especially when the selection goes against the most informed critics.
Note that this is quote annoying for me! Instead of doing my own work on the project, I get to spend time answering questions from these folks because they don’t have access to the data themselves. (I am not allowed to share the data with them but I am allowed to answer their questions.) Quite annoying! Here is their latest work. How can anyone defend refusing to share data with these folks while simultaneously sharing the same data with almost everyone else who asks?
Comments welcome, especially from Tim Burke.
LizardBreath 09.09.07 at 8:31 pm
How can anyone defend refusing to share data with these folks while simultaneously sharing the same data with almost everyone else who asks?
Are you complaining about the poor manners of the L2 authors, or are you suggesting that sharing data with “almost everyone else who asks” is a method of concealing fraud or error? If the first, I’m not sure why it’s worthy of discussion. If the second, your point seems under-argued.
Kevin Donoghue 09.09.07 at 8:47 pm
David Kane tells us in the Deltoid thread he links to: “Elizabeth Johnson (who did the statistical work on the paper) has confirmed that I am correct and dsquared is wrong. Crude mortality rates were not estimated via bootstrap.”
David, can you quote precisely what she wrote and what question she was responding to? It’s not a big deal but since you are the only source for this particular claim I’d like to know just what evidence you have.
bi 09.09.07 at 8:49 pm
OK, I have to admit I skipped over all the trite pleasantries, and all the technical discussion on right and wrong formulas and stuff, and all the accusations of suppression of data.
You see, I was having trouble figuring out why Malkin would want a draft of Kane’s paper. And why Kane would willingly oblige.
_Why would Malkin want a copy of Kane’s paper?_ So that she can read it and understand it? So that she can give informed, technical comments on it? (But given that Kane actually got a formula totally wrong) But she’s not a scientist or a statistician. Can Kane expect her to want to do anything else, other than trumpet his paper draft in a fit of Mindless Link Propagation(tm)?
_So why would Kane willingly send his paper to Malkin?_ Either he was thinking something silly at that time, or he truly wanted her to mindlessly propagate it. Either way, it doesn’t reflect well on him.
Which leads me to the next two questions: _Why did Kane not vet his paper with his Harvard IQSS colleagues._ And, _why’s he not doing that now, despite Robert’s advice above?_
bi 09.09.07 at 8:52 pm
Yes, here’s the “advice from CT authors” that Kane is asking for, and that’s already been given: vet the paper with IQSS colleagues who actually know their stuff.
Alternatively, Kane can send the next draft to Bill O’Reilly, to widen his Mindless Link Propagation(tm) network.
Kevin Donoghue 09.09.07 at 8:53 pm
Just to clarify my question to David Kane, I’m obviously not suggesting that CMRs were “estimated via bootstrap”, whatever the hell that might mean. I just can’t see how she could have used that bizarre phrase and it would be nice to know what she was actually trying to convey.
David Kane 09.10.07 at 2:09 am
1) Lizardbreath, I do not think that the L2 authors are guilty of “poor manners.” I believe it is ridiculous for them to refuse to share data with Gourley et al. I think it is unprecedented (counter-examples welcome!) to share data from a published paper with some critics but not others.
2) Kevin asks for a precise quote from the e-mail I received from Elizabeth Johnson. Can’t do that without her permission! The Lancet authors have asked me not to quote anything from our correspondence. (They have provided me with a couple of statements on specific topics which I have been able to quote.) I am not going to waste one of my favors from the authors just to satisfy your curiosity. Feel free to e-mail them yourself. What I have told you is the truth (as I deduced from circumstantial evidence in the Deltoid thread). The bootstrap was not used for any of the CMR calculations. Neither she nor I can be much more clear than that.
David Kane 09.10.07 at 2:33 am
Tim Burke writes:
I think that this is quite unfair as a description of what I have done. I came to this story in 2004 as an interested bystander. I had no priors except a generalized skepticism of applied work which just barely rejects the null hypothesis. I sought to replicate L1 and was stymied by the authors. They refused to share their code. They refused to share the data at a detailed level (as they eventually did with L2). Les Roberts was, to put it simply, quite rude.
But that was it. I didn’t do anything beyond make a few comments (all, I think reasonable) at Deltoid. I never wrote a paper, never contacted Malkin, never sought to make a big stink. I was certainly a skeptic, but a quite one. I came to the debate with exactly Tim’s recommended attitude: ““Ok, I want to look at why you’re making this claim, and your evidence for it. I don’t want to take anything on faith.†No one can provide any evidence to the contrary.
It was only when the 2006 paper came out, with its ‘not credible‘ response rate, that I really dove into the topic, really started to spend time on it, really started to write about. Now, it is hard for any one else to know if, deep within my own heart, I came to the debate, in 2006, with an attitude that would make Tim Burke proud, but that was two years into the debate, two years of being given the brush off by the L1 authors.
And, for the record, I am not certain that I am right! There is a good chance (5%, 10%?) that the L1 authors are correct. If they shared their data and code with everyone, we could all update our priors.
But let me turn the tables on Tim (and others). Jon Pedersen is someone respected by all sides in this debate. After reading :
Why are you so sure that the Lancet authors are correct and Pedersen is wrong?
Henry 09.10.07 at 2:55 am
If you weren’t an egregious hack, you’d have replied that the paper was still a work in progress, needed peer review etc, and politely asked her to abstain from linking and posting until you had gotten proper feedback on it. I don’t for a moment think that you weren’t entirely aware of the likely consequences – you have, after all, been at pains to tell us that _Little Green Footballs_ is one of your favourite blogs, so you can’t pretend that you don’t have some idea of what happens when this kind of thing gets politicized by the more enthusiastic folks on the right wing. As Robert said over to Deltoid, your net contribution to public knowledge has been heavily in the negative – and I don’t believe that you are or were unaware of this for a moment.
Nor is your reply to Tim Burke at #47 at all believable to anyone who has followed your track record. You came into this with an axe to grind. You’ve demonstrated a repeated lack of good faith and have more than once accused your targets of fraud based on your own quite tendentious misunderstandings. To put this another way – if the Lancet authors had done something as egregious as failing to include the one important formula in a package that they claimed represented the work of the other side in the debate (to name only the most obviously awful of your errors), you’d have been shouting fraud from the rooftops. What happened over at Deltoid isn’t intellectual debate – it’s garbage pick-up as art form (as a couple of commenters have noted, it’s depressing that it has to be done, but if it is to be done, at least it’s being done funnily and well).
Henry 09.10.07 at 2:57 am
by the way, I reckon people should leave off slagging off rilkefan – there’s no reason why everyone should have been aware of the sorry sequence of events, and those who only saw the recent Deltoid thread might very reasonably have failed to understand the context from which it sprang.
Henry 09.10.07 at 3:01 am
and final comment before I get back to my real life – the issue under debate in this thread isn’t whether people should be convinced that “the Lancet authors are correct” – they may indeed be wrong. The issue is whether people should or should not be convinced that _one particular critic_ of the Lancet authors – David Kane – is full of crap. And if you want to convince people that you aren’t – well after the various shenanigans of the last many months, you’re going to have your work cut out for you mate.
David Kane 09.10.07 at 3:14 am
Henry writes:
The purpose of posting it to Deltoid was to get feedback on it prior to submitting it for publication. Don’t you know academics who seek comments, who post drafts of their papers before submitting them for peer-review? I wanted and got feedback on it, but from Deltoid readers and from Malkin readers (although it is true that the former were more knowledgeable and helpful).
The only reason that the data from L1 is publicly available is because of my efforts (and those of Tim Lambert). It is amazing how much you can know about me given the fact that we have never met. Again, if someone from, say, DailyKos asked you from permission to link/quote/excerpt one of your papers, would you turn them down?
And you know this how? I was involved with this for two years before I published anything which anyone found objectionable. See my (few) previous comments at the old Deltoid. I even convinced Tim to correct a (very minor) mistake he had made with regard to the authors’s politics. If I am just in this to grind axes, then why wait two years to start the process?
This never happened. I included the entire spreadsheet from Deltoid in the R package which I distributed. I created a data frame from the raw data in exactly the way that any competent R programmer would have. Don’t believe me? Check for yourself. Just because Robert Chung says something does not make it true. Please try to educate yourself on these details before engaging in baseless slander.
David Kane 09.10.07 at 4:10 am
Henry writes:
Charming. And you wonder why I think of Crooked Timber as the Little Green Footballs of the academic left? Comments like this are very LGFesque.
I have no problem with CT authors who don’t care about this dispute. Life is short. But your drive-by slander is annoying. Better to spend time at Deltoid where at least the critics take the time to read what they are criticizing.
bi 09.10.07 at 4:16 am
David Kane, what kind of useful feedback were you expecting from Malkin and her readers again? And why do you still refuse to vet your paper with people at the IQSS?
And:
“What I have told you is the truth (as I deduced from circumstantial evidence in the Deltoid thread).”
“Please try to educate yourself on these details before engaging in baseless slander.”
From the circumstantial evidence of these two statements, I conclude that David Kane is a hack. This is the truth.
bi 09.10.07 at 4:19 am
(To rephrase that: David Kane, when’ll you stop assuming the worst of everyone else while insisting that everyone else assume the best of you?)
SG 09.10.07 at 5:55 am
when exactly did your fraud allegation occur, David?
except that you admitted on the Deltoid thread that these authors can’t use any publicly available data to check their main street bias thesis, due to confidentiality concerns.
This claim is really starting to give me the shits. Why don’t you share with us details of any cases of data sharing that you know of, the circumstances of the sharing, and what happened? I think this is “unprecedented” only because you have never previously heard anything about how any authors share data, good or bad.
bi 09.10.07 at 6:08 am
Hmm… is this why David Kane’s refusing to vet his paper with his IQSS colleagues?
His anti-Lancet work has no credibility with actual academics, so he’s hoping to find a mindless propagation network in Michelle Malkin?
Back in the day, this practice was called “forum shopping”. I wonder what the wingnuts call it now.
bi 09.10.07 at 8:38 am
I suggest we write a paper which starts out like this:
Abstract
A recent study by Burnham et al. claimed that Iraq mortality rates were higher after the US intervention into Iraq than before. In this paper, we attempt to show that Burnham et al.’s results cannot be replicated; and even if they can be replicated, the definition of crude mortality rate (CMR) they used is a wrong one; and even if it is not a wrong one, the very fact that Burnham et al. refuse to release their data to Gourley et al. proves that their results are bogus; and even if they do send their data to Gourley et al., it does not invalidate any of the rest of the breadth of academic criticism against the Lancet papers; and even if it does invalidate the rest of the breadth of academic criticism against the Lancet papers, the truth remains that the Lancet papers are bad for freedom and capitalism and the American Way.
= = =
I invite Crooked Timber readers to fill in the rest of the paper while I sit back and offer virtual US$10,000 bills.
dsquared 09.10.07 at 8:40 am
If we’re still on speaking terms with Steve Fuller I’d be interested in his views on this one because it seems to me to be an excellent test case of where something like his views on inclusiveness in science break down.
SG 09.10.07 at 8:57 am
Methods
Just as Einstein proved his theory of relativity entirely through the use of sophisticated thought experiments, we construct proofs of all these assertions using an elaborate series of thought experiments based on
1) use of hazily defined prior distributions
2) symmetric normal distributions and their ratios
3) italicization of the words think and seems
It certainly seems that any proof based on these three components is logically perfect.
All data analysis was conducted in R, using code supplied by deltoid readers. Statistical significance is determined by the method of overlapping confidence intervals for sample statistics with unequal variances, which seem to work in all cases no matter the complexity of the problem. Any errors are the fault of the fundamental theory of epidemiology, which often seems to be wrong. Insinuations of fraud and incompetence were entirely the responsibility of the lead author, and we think they are self evident in every case.
nick s 09.10.07 at 1:03 pm
And you wonder why I think of Crooked Timber as the Little Green Footballs of the academic left?
No, we’ve already established why you think such things; now we’re just speculating on the price.
When Atrios talks of ‘zombie lies’, it’s sometimes unclear whose bodies those lies feast upon. Not so here.
sg: you forgot “4) liberal! use! of exclamation! marks! to assert! understanding!”
dsquared 09.10.07 at 1:25 pm
I think the main long term consequence of this frankly excruciating episode will be that I (and I hope, others) will forever use “Excellent points!” as a means of subtly implying that I think my interlocutor might be completely at sea.
Tim Lambert 09.10.07 at 6:35 pm
David Kane, in 2005
That sort of thing is a big problem with cancer research as well.
jre 09.10.07 at 8:26 pm
Damn! Tim said it better, and briefer, than I could.
Soldiering on, anyway …
[sound of gong]
By David Kane’s own account, the very first comment he made on this subject was the one just quoted. He entered this discussion by saying that the study could not be trusted because its authors’ motives were suspect, and that this was a valid objection to the study’s methodology and / or its results. And he has not deviated from that pattern of argument since then.
Look — I know this thread is titled “Hilarity”, so I hesitate to harsh anyone’s buzz. But I need to point one more time to the depressing “fraud balloon” incident. When Amy Perfors of Harvard’s IQSS removed David Kane’s post accusing Roberts et al. of fraud, she justified the act by saying that
Now, some people might have interpreted that to mean that the “Case for Fraud” post was a warm turd in the punchbowl of reasoned discussion. But not David Kane. Lest anyone accuse him of being able to take a hint, he reproduced the offending post on his own blog. His privilege, of course. But what I see has an axe head and an axe handle, is recently sharpened and is in David Kane’s hands as he bends over a grindstone. Of course, maybe that’s what I wanted or expected to see, so who knows if it’s really an axe to be ground after all?
John Protevi 09.11.07 at 12:10 pm
And you wonder why I think of Crooked Timber as the Little Green Footballs of the academic left?
Whatever his faults, let no one say David Kane is not a comedian of the highest order.
John Protevi 09.11.07 at 12:12 pm
OR,
Let no one say that David Kane, whatever his faults, is not a comedian of the highest order.
“Leave no modifier undangled.”
Comments on this entry are closed.