Citing blogs

by Henry on April 7, 2008

Andrew Gelman politely suggests that he deserves some credit.

Matthew Atkinson, Ryan Enos, and Seth Hill sent along this paper:

Recent research finds that inferences from candidate faces predict aggregate vote margins. Many have concluded this to mean that voters choose the candidate with the better face. We implement a survey with participant evaluations of over 167,000 candidate face pairings. Through regression analysis using individual- and district-level vote data we find that the face-vote correlation is explained by a relationship between candidate faces, incumbency, and district partisanship. We argue that the face-vote correlation is not just the product of simple voter reactions to faces, but also of party and candidate behavior that affects which candidates compete in which contests.

This is great stuff. They’re talking about a 2005 article by Alexander Todorov, Anesu Mandisodza, Amir Goren, and Crystal Hall which found that people thought the faces of winning congressional candidates looked more “competent” than faces of losing candidates. I wrote about this about a year ago and expressed skepticism about the interpretation of those findings .

It seemed likely that the more competent-looking candidates were more likely to be the ones that were more credible candidates for political reasons that were not directly related to looks (incumbency, ability to raise money, etc). In short, I suspected that, even if the voters had no idea what the candidates looked like, the Todorov et al. findings could still occur. Well, Atkinson et al. didn’t just speculate, they went and did a bunch of analyses.

… I feel a little awkward saying this, but I think they should refer to my blog entry from last year, since it’s the first publication that I know of that questioned the “faces decide elections” reasoning. Even though Atkinson, Enos, and Hill probably came up with their ideas on their own and only encountered my blog entry later (as noted above, they went far beyond my speculations and did actual research), it would still be appropriate to cite it as relevant early work.

It seems to me that this raises some interesting issues of credentialling. Andrew seems to me to be quite right in suggesting that he deserves some degree of acknowledgement for having first raised the possibility of the alternative line of causation in a public forum. But how best to acknowledge this? It seems to me that this kind of contribution falls midway between corridor talk and published research. If the three authors of the paper had been in Andrew’s department, Andrew might easily have raised it in conversation with them rather than in a blogpost, in which case you would hope that they would acknowledge it, perhaps through an informal-ish footnote (“we are grateful to Andrew Gelman for having pointed this out to us as a possible alternative explanation” or somesuch). If Andrew had raised it, perhaps as an aside, in a research paper of his, you would hope that they would have provided a cite to the relevant paper. But a blogpost isn’t exactly either of these. If we take as given that some credit ought to be given to Andrew’s post, what precise form should this credit take? This is in one way a somewhat pedantic and persnickety question, but since attribution norms play an important role in determining who does well, and who doesn’t do well in academia, it’s not entirely an inconsequential one.

{ 29 comments }

1

Sean Carroll 04.07.08 at 4:04 pm

I’ve certainly seen frequent formal citations to “unpublished,” or “personal communication,” or to a talk rather than a journal article. I don’t see why blogs should be any different, especially since there is a record that the reader can actually go and retrieve. I would vote for an ordinary citation, if the content of the blog post was really germane.

2

J.W. Hamner 04.07.08 at 4:47 pm

Seems sort of unfair to me to expect people to scour the internet to make sure some random guy didn’t publish a blog entry putting forth your hypothesis as a vague musing. If you didn’t do anything other than “wonder out loud” and haven’t submitted to a peer review process than your “work” isn’t worth citing in my eyes.

Though I work in medical science, so you can’t get published without actual data… so theorizing is not really interesting to me.

3

Sortition 04.07.08 at 4:51 pm

Why is giving credit and getting credit such an important part of academic activity anyway? Science is supposed to be about understanding things, not about who said it first.

4

Henry 04.07.08 at 4:56 pm

Why is giving credit and getting credit such an important part of academic activity anyway? Science is supposed to be about understanding things, not about who said it first.

Because people’s careers, job prospects, salaries etc depend upon their ability to claim to do original research that is well regarded by others in their field. Basic incentives, as in much else.

5

Brian Weatherson 04.07.08 at 5:01 pm

Credit is important because it’s the currency in which discovery is rewarded. We’re currently working (as we have been more or less for centuries) under the assumption that giving people this kind of reward is a cost-effective way of encouraging them to produce more knowledge and understanding. Michael Strevens (NYU) has some papers arguing that this is an efficient mechanism – even though it is hard to see why priority should be such a big deal in the abstract.

6

Rich B. 04.07.08 at 5:04 pm

It’s unclear to me what evidence Mr. Gelman has that the authors of the study even was aware of his blog post, if they didn’t cite to him in their paper.

The first page of the article thanks “our survey participants from February, March, and May 2007.” Mr. Gelman’s blog post was from April 27, 2007.

Is there now a requirement during a literature review to analyze all potentially relevant blog posts that are made both before and during (if not after?) the conducting of the experiment itself?

“We are grateful to Andrew Gelman for having pointed this out to us as a possible alternative explanation” is simply not accurate. Mr. Gelman didn’t ‘point this out’ until 2/3 of the way through the interviews, after they had already come up with the possible alternative.

I can imagine the “gray area” blog posts that you are thinking about, but I’d be really concerned about pushing the gray this far.

7

MSS 04.07.08 at 5:19 pm

Not only about people’s careers, but also about the basic accumulation of knowledge. It may be useful to later researchers to know the intellectual history of a line of argument.

And to the second comment above, I do not think the burden is to “scour the Internet” to make sure one’s idea has not been mentioned somewhere. I think the burden is that if someone actually did see your argument, and then build on it, he or she is obligated to reference the source. The same logic applies to publications. Sometimes a previous published item has already raised a similar point, and the researcher is honestly unaware of it. That is sure to happen at times, given the fragmentation of the literature, not all of which is indexed centrally even in today’s on-line world. But if you know of a prior relevant argument, cite it, regardless of the source.

8

J.W. Hamner 04.07.08 at 5:31 pm

#7:

When I’ve cited papers, 99% of the time I’ve cited their results, not their discussions. Discussions can certainly be helpful, but I don’t think that somebody saying “well maybe our results our explained by mechanism X” is cite-worthy. If you show that it’s mechanism X, then that’s another story.

I have trouble seeing how a blog entry could be anything other than hypothesis with no testing, which strikes me as mostly worthless.

9

leederick 04.07.08 at 6:11 pm

I really don’t think Gelman’s deserves credit for his ‘insight’. It’s not really a difficult leap to make. If you see any study saying that X is associated with Y and considering the hypothesis that X may cause Y, anyone with a basic understanding of statistical reasoning will automatically wonder whether this due to confounding rather than a causal effect.

Contributions need to reach a certain standard before they’re creditable. I don’t think you can reasonably claim credit for ideas that are obvious to anyone with a moderate understanding of the topic area. Maybe you deserve credit if you suggest a specific confounder which accounts for a large proportion of the association, and a specific mechanism which explains the reason for the relationship between the three variables. But saying it could be counfounding, and then giving a list of possible confounders and tagging on an etc doesn’t cut it.

10

Dennis 04.07.08 at 6:14 pm

I guess this should be cited as “impersonal communication.”

11

MSS 04.07.08 at 6:19 pm

Well, I should add that the one citation to Fruits & Votes actually demonstrated the mechanism in question.

Blog entries can do more than “hypothesis without testing.” However, I am not prepared to concede that a hypothesis (even without testing) is unworthy of citation just because it appeared on a blog, or in a hallway conversation.

It seems to me that there are fair numbers of publications that propose hypotheses but do not test (perhaps a conceptual piece, or the data simply are not yet available). Are such publications therefore not cite-worthy?

I suppose worthlessness is in the eyes of the beholder.

12

Righteous Bubba 04.07.08 at 6:24 pm

(The band has just watched a TV special about Dr. Rockso, in which Murderface stated he was the lead song-writer for Dethklok.)
Nathan: Yeah, uh…you know, you also said that you’re the Dethklok song writer.
Murderface: (clearly pleased with himself) Did I? I, eh, I don’t remember.
Nathan: Yeah…you did. And you’ve never written anything, ever.
Murderface: What about Planet Piss?
Nathan: First of all, that’s not Dethklok.
Pickles: You never even completed one song, neither.
Murderface: That’s not to say I’m not capable of writing a song!
Nathan: (getting angry) In this case, it is to say you’re not capable!
Murderface: I could have written any Dethklok song, I coulda written any of ‘em!
Nathan: But you didn’t write any– but you didn’t, though!
Murderface: But I could have!
Nathan: But you didn’t!

(pause)

Murderface: BUT I COULD HAVE!
Nathan: Well, I could’ve invented the, uh, the floor, you know, but I didn’t!
Murderface: But that doesn’t mean you shouldn’t get credit for inventing the floor!
Nathan: (babbling) That should mean that I don’t get credit for it!

13

J.W. Hamner 04.07.08 at 6:33 pm

Blog entries can do more than “hypothesis without testing.” However, I am not prepared to concede that a hypothesis (even without testing) is unworthy of citation just because it appeared on a blog, or in a hallway conversation.

To me, at most it’s “acknowledgment” worthy in most cases I can envision. If I’m standing by my poster at a meeting, and somebody suggests something that turns out to be true with further examination then I would certainly note their contribution in the ackowledgments… but it falls short of authorship and a citation of a personal communication in scientific literature seems unorthodox at the very least.

However, one example of “more than a hypothesis” that I can think of would be a science blogger saying “I wonder if it’s this” and then laying out a complete or close to complete experimental protocol to answer said question. I could see how that’s worthy of a cite, though I would probably contact the blogger and see if they wanted a larger conceptual role as a coauthor if I planned to take up their proposal.

14

richard 04.07.08 at 6:53 pm

Perhaps it would be polite, but I’d be extremely wary of extending the requirement to cite to blogs, simply because of the search issues. I also think the blog as a debating arena is pleasingly and usefully free from accountability: things said in blogs are unofficial, uncertified and at-your-own-risk. It’s a credit-free and penalty-free environment, where you can kick an idea around without it having the authority or professional exposure of a journal article. Why would we want this to change?

My understanding is also that anything you write in a blog is public domain. Am I wrong?

15

Sortition 04.07.08 at 6:57 pm

Credit is important because it’s the currency in which discovery is rewarded.

This seems like an anti-scientific setup. Scientific discovery is supposed to be its own reward, rather than a tool for self-advancement. The expected problems follow: prevalence of academic self-advertising, ubiquitous academic politics, and ultimately, outright fraud.

16

Righteous Bubba 04.07.08 at 7:04 pm

Shanghai Jiao Tong University ranking of universities.

It seems natural but not necessarily fair to me to judge the importance – not quality – of an individual by some sort of cite-ranking system. And people do that.

17

laura 04.07.08 at 7:34 pm

I think that blog posts, if really an original idea or theory or whatever, should absolutely be given proper academic citation. If someone jacked an idea that I brought up on a blog and used it for an academic paper without citing me, I would be really pissed off. I would be really pissed if someone stole one of my ideas from a hallway discussion and used it for one of their papers. This problem is one of the reason that I’ve held back from discussing my research area on my blog.

18

Stentor 04.07.08 at 8:00 pm

I had assumed the reason you use an “informal-ish footnote” for a hallway conversation is because there’s no record of that conversation that someone else could go back and look at for themselves (such backtracking being one of the main reasons for citing). So if you get an idea from a blog post, there’s a record someone else could look at, and thus a full citation is in order.

19

Bruce Baugh 04.07.08 at 8:23 pm

Richard: “My understanding is also that anything you write in a blog is public domain. Am I wrong?” Yeah, you are. Standard copyright law applies. When it’s set down in a lasting medium, which has been tested and found to include HTML, it’s yours with all the usual rights unless you explicitly waive them. For an example of such a waiver, see the bottom of the page at Boing Boing with the Creative Commons declaration and link to the specific terms Cory et al use. If there’s nothing like that, the contents of a web page are precisely as protected as they would be in other media.

20

Bruce Baugh 04.07.08 at 8:25 pm

Er, I didn’t mean that to sound scoldy, but left out the part about “You are wrong, but in a way that’s very common, thanks both to honest misunderstanding and some folks deliberately spreading claims known to them to be false, for various reasons”.

21

John Quiggin 04.07.08 at 8:36 pm

Copyright only applies to the words in which ideas are expressed. Ideas are in the public domain.

In any case, the question here is acknowledgement. The most common CC license requires attribution.

22

Laura 04.07.08 at 11:31 pm

I’ve just written a paper which quotes from an academic’s blog – with full and standard attribution of course – but the issue that arose for me in doing so was wondering whether the author would actually be entirely happy to have his or her blog quoted in that way.

I wondered because the material I quoted was written in a spontaneous and unrevised way quite unlike this academic’s reviewed and formal work. The spontaneity was partly why I wanted to use it, although of course the substance was also important to my argument.

I actually did wonder if I should perhaps ask the person first, but in the end decided against doing so, because I figured that as a professional writer he or she did understand what publication invites and entails.

23

vivian 04.08.08 at 12:51 am

If a blog post was relevant to a paper of mine, and I was aware of it, I would cite it happily. The point is to make it easy for the reader to find the material, in case they want to check my honesty or respond in print to me. The fact that lots of people cite a blog post will not (by itself) make tenure committees give them weight in tenure battles. It is not as though my citing a blog post cheapens or dilutes the citations to peer-reviewed work, any more than citing newspapers or hallway conversations does. The reader can easily figure out which sources are really fundamental to the paper, and which are lit review/acknowledgements of similar projects. That’s like saying “good morning” to someone you pass in the hallway “here we both are, in the same issue-space.”

It isn’t clear to me whether the authors knew of Gelman’s post in advance. If they didn’t, guess he’s not as famous as he thinks. But if they did, well, I would have noted that in print.

Is some of the reluctance to cite blog posts a fear that the audience will think one is immature, like citing pop song lyrics or something? Because that really has nothing to do with the quality of one’s research or argument. The test is relevance, with a bias towards more citations when in doubt.

24

J.W. Hamner 04.08.08 at 1:41 am

Is some of the reluctance to cite blog posts a fear that the audience will think one is immature, like citing pop song lyrics or something? Because that really has nothing to do with the quality of one’s research or argument. The test is relevance, with a bias towards more citations when in doubt.

For me, I have trouble imagining a scenario that would warrant it. Stating a hypothesis publicly, or in private, is not impressive to me and does not deserve a citation in my view. Hypotheses are really a rather minor part of the scientific method… it’s ridiculously easy to come up with confounds/questions. We could spitball probably a dozen (at the very least) of those for any paper you wanted to pick. What matters is not the question per se, but the way you go about trying to address it. Raising the question may or may not be acknowledgment worthy… but we definitely don’t get into citation or authorship range until we start talking about experimental design or actual evidence; not just theories.

I don’t have any problem citing blogs if they do some actual work… a mini meta analysis, perhaps? However I have yet to see an example of blog work that deserves such honors.

25

Laura 04.08.08 at 2:18 am

j.w. hamner your remarks probably apply better to scientific disciplines than to the humanities where ‘actual work’ is done on blogs with pleasing regularity.

26

J.W. Hamner 04.08.08 at 3:19 am

j.w. hamner your remarks probably apply better to scientific disciplines than to the humanities where ‘actual work’ is done on blogs with pleasing regularity.

Quite right. I had no intention of commenting on the humanities, as I know nothing about how that system works, though I am happy to learn.

In that sense of discovery, I wonder what sort of credentialing you might need before citing a blog (if any). Would you only cite a blog of a learned colleague that you know well? Would you cite someone you don’t, personally, know, but who has a PhD and a good reputation? Would a grad student in a reputable program be worthy? What about a really smart undergrad? Or a gifted high school student?

I don’t have a PhD (thus you can disregard what I say for that reason alone, if you choose), so I have no love for The System and how respect is doled out… but I think there is a lower limit on credibility when making a reasonable citation… it probably isn’t level of education, but I have trouble believing something that hasn’t been peer reviewed is ever going to crest that threshold.

To try and rephrase more simply… if someone presented an argument you disagreed with, what kind of blog post cite would convince you that you had it wrong? Can someone provide me with examples?

27

Laura 04.08.08 at 4:05 am

Well, my field is English Literature. When I cite for historical facts I guess I depend on the established authority of the person I’m citing to some extent. But even there the existence of a PhD or not will not have much influence on whether I will accept one interpretation of historical events over another.
But good literary criticism almost by definition speaks for itself and no further credentialling or underwriting is required.

28

novakant 04.08.08 at 9:30 am

the first publication that I know of that questioned the “faces decide elections” reasoning.

Hmm, looking at the actual actual blog post in question, I wonder if this guy isn’t taking himself a little too seriously.

29

conchis 04.09.08 at 3:58 pm

“Is some of the reluctance to cite blog posts a fear that the audience will think one is immature, like citing pop song lyrics or something?”

In my case, the answer is definitely yes. There are a couple of arguments relevant to issues I’m writing on, that as far as I can tell, have only been made on blogs. I actually really want to cite them (mostly as a foil to eventually disagree with, but that’s somewhat beside the point). But I’m not really sure how exactly to do so, and am definitely a little concerned that it may seem unprofessional to be citing blog posts, and that others won’t take them seriously because they’re not “proper” publications.

Comments on this entry are closed.