Response by Malhotra and Margalit to Their Critics

by John Holbo on May 30, 2009

This is a follow-up to the distinctly non-sober but not wholly unuseful thread attached to my post on the Boston Review piece on Malhotra and Margalit’s survey research on anti-semitism and the financial crisis. The authors have asked for a chance to explain themselves, and their methodology, which has come in for a lot of criticism of an unavoidably speculative sort in comments to my post. Let’s hope this clears a few things up. Let’s try to be civil, shall we? The following is, obviously, not by me but by Malhotra and Margalit. And not edited by me in any way. – John Holbo

We are glad that our article generated thoughtful discussion, and we would be happy to address some of the questions people raised in the comments section. If our responses do not specifically address your particular comment, apologies in advance. Our goal here is to touch on some of the main issues.

First off, our objective in this article was scholarly, and was not to promote any particular public policy. Our main recommendation is that media outlets be careful in introducing religion and ethnicity into their reporting of the financial crisis, as this may have unintended consequences on various attitudes. Additionally, we show that the number of people who explicitly provided a response blaming “the Jews” was extremely high. We are not claiming that the point estimate of 25% is the exact percentage of prejudiced people in the population. Indeed, statistical uncertainty by itself bounds the estimate between 21-29%. Additionally, variants in question wording will of course lead to somewhat different figures. Our main point is that this figure is surprisingly high, and we were alarmed that such a high percentage of respondents would be willing to reveal this. Moreover, the survey experiment is truly the main finding in our article, and we should have ensured that this was more prominently featured. We find that associating Madoff with his ethnicity has downstream policy consequences on topics having nothing explicitly to do with anti-Semitism. Lastly, our advisors in graduate school once told us that conducting survey research is like performing surgery in a sewer. It is a very complex, messy undertaking because measurement is so difficult. Hence, we want to assure people that we have leveraged our extensive and rigorous training in public opinion research and survey methodology to do our best to study the question at hand. Reasonable people can have methodological disagreements, but keep in mind that most alternative approaches will introduce their own set of biases. For example, if we had used “Jews” instead of “the Jews,” it could easily be said that many Jews were involved in the financial crisis and that our question was not truly tapping ethnic prejudice.

We also address a few more technical details below:

1. Some people asked whether the difference between Republicans and Democrats is robust to the inclusion of various controls (education, region, gender, race, income, etc.). We find that the difference between Democrats and Republicans is highly robust to these controls when: (1) estimating a multivariate regression model; and (2) conditioning on these variables. That being said, the gap between partisans is definitely a correlation, and it not causal. But it is a highly robust correlation.

2. Some people raised some questions about how the overall blame attribution question was asked. No survey item is perfect, and you have to choose the one that is going to bias your findings in the reverse direction from your alternative hypothesis. The question we asked has been used previously, and generally underestimates the level of racial prejudice. We were very surprised that this ended up not being the case, most likely due to the anonymous Internet survey mode (as opposed to phone or face-to-face interviews). Second, any biases will likely only change baseline levels of blame, and not differences in blame between various population subgroups. And it is these subgroup differences that we are most interested in.

3. Some people wondered if people would be so disgusted with the question that they would not answer. Non-response is not an issue in this dataset. Only 0.7% of respondents refused to answer this question.

4. Some people wondered if the fact that the survey was anonymous made it less valid. Quite the opposite. Anonymity increases response validity because people do not feel a need to provide socially-desirable responses to a live interviewer. For example, in face-to-face and phone surveys, reported voter turnout rates are about 80-90%, far exceeding the 50-60% of people who actually turn out to vote. In Internet surveys, the reported turnout rates are closer to 60-70%. I think this comment had to do with some confusion over how the survey was conducted.

5. Some people asked about the representativeness of Internet surveys. The sampling design was not a random probability sample. Rather, the respondents were part of SSI’s online panel. Several researchers have argued that Web surveys can approximate the representativeness of those conducted via random digit dialing. See Clarke et al.’s paper in Political Analysis in 2007. This was not an anonymous Internet survey like the ones you see on news websites, however. Rather, panelists were invited such that the overall distribution would be representative on a host of demographic variables. As one reader pointed out, this is somewhat similar to the approach used by YouGov/Polimetrix, a highly-respected firm used by both academic researchers and major corporations. The representativeness of this survey on variables such as gender, age, and party identification approximate distributions from the 2008 American National Election Study, which used a face-to-face probability sample. Also, it is important to note that all surveys are opt-in, and many of the RDD telephone polls you read about in the media have response rates of 5-10%, meaning that we don’t observe the responses of the 90-95% of people who chose not to participate. Hence, random sampling is not a panacea for representativeness. Finally, the internal validity of the survey experiment is not compromised by a non-random sample, since the treatment itself was randomized. This is no different from medical or psychology studies that use convenience samples to assess the effects of various treatments.

6. Some people asked whether we asked people to assess blame for other groups in addition to “the Jews.” We asked them about some other groups: loose government regulations, Wall Street financial institutions, people who took out mortgages they could not afford. Unsurprisingly, people blamed these groups much more than “the Jews.” But the difference between Republicans and Democrats flipped for borrowers, which is the crucial point. That suggests that it is not simply that Democrats are blaming everyone more. Regrettably, we did not ask them to evaluate any other ethnic groups. That would be a very useful test to assess the robustness of the results, and we hope someone will do that.

{ 97 comments }

1

Righteous Bubba 05.30.09 at 3:45 am

Thanks to the authors for the follow-up and to John for the allowance of a follow-up post.

2

bob mcmanus 05.30.09 at 4:36 am

Our main recommendation is that media outlets be careful in introducing religion and ethnicity into their reporting of the financial crisis, as this may have unintended consequences on various attitudes.

Certainly good advice.

There has been a lot of coverage, in print, blogs, and the financial tv networks. I would be interested to know how often religion and ethnicity have actually been mentioned in the coverage of the financial crisis, and whether it has been rising. I may have missed the times Maria Bartimoro started her standup with “Goldman-Sachs, the International Jewish Conspiracy, announced today…” I don’t watch CNBC that much.

3

bob mcmanus 05.30.09 at 4:51 am

Ethnicity was actually relevant to the Bernie Madoff scandal because much of Madoff’s clientele was Jewish, and Madoff abused his connections within the Jewish charitable community.

I cannot think of another example, in hundreds or thousands of articles and reports, where ethnicity was mentioned. Zero. Not one. I may be underinformed, and would welcome a correction. More than one or a few, please.

4

Tim Wilkinson 05.30.09 at 5:29 am

A few other questions which have arisen in the course of the discussion:

Can you confirm that both parts of the study formed part of a single survey?

What was the overall sample size for this study?

How were subjects for the study selected from the 2,768 participating in the larger survey?

Did all participants in the larger survey answer the questions featured in this study?

What were the proportion (or number) of total subjects of this study falling into each of the two partisan groups, and to neither?

Could you provide the complete text of the questionnaire? If not:
a) Could you provide the text of (one version of) the article about Madoff shown to respondents?
b)At what point in the process were respondents shown that article?
c)Were there any other questions or information involving jewishness in the survey as a whole? If so what were they and at what stage did they occur?

Could you provide a breakdown for each of the 5 permitted responses to each question in the first (non-experiment) part of the study, by partisan group and overall? If not could you provide figures for the ‘moderate’ responses? Do you also have a breakdown by Jewish and non-Jewish ethnicity, as in the survey experiment?

Could you provide a similar breakdown for the second part, including by partisan group?

Is there a tendency for questionnaire respondents to be biased in favour of middle or neutral options? If so is this more pronounced than among interview respondents?

Do questionnaire responses tend to differ from those in interview responses (perhaps for example because respondents are less likely to comprehend the question or think about the answer)?

You state that merely using the term ‘Jews’ might not provide a measure of prejudice. How clearly do you think that those answering the questions would have consciously distinguished between this term and ‘the Jews’?

How likely is it that respondents would interpret ‘how much to blame’ as ‘how well-represented among those to blame’ when the subject is a segment of the population with no clear connection as a whole to the events in question?

How would you assess the likelihood that presenting the options as ‘a list of things people claim were responsible’ might have a bandwagon effect, thus increasing the tendency to ascribe blame? Might this be more pronounced in cases to which the prior tendency was low?

You state that ‘it is not simply that Democrats are blaming everyone more’. How plausible is it that Democrats are blaming everyone more except for groups they specifically regard as victims?

What assessment would you make of the possibility that biases in the first question might have changed baseline levels of blame carried forward into the experimental part of the study?

How likely is it that a person assenting to a proposition involving ‘the Jews’, even if not intending to refer to Jews en masse, might thereby acquire (consciously or not) some increased hostility to that group?

How likely is it that a person for whom the issue of Jews being to blame for the crisis has been rendered more salient or plausible might then assign increased blame to a particular jewish businessman?

How likely is it that a person assigning increased blame to a particular jewish businessman might then be more inclined to blame business in general?

What influenced you in the decision to group ‘a moderate amount’, ‘a lot’ and ‘a great deal’ together in the results? Why was ‘a little’ not regarded as indicating prejudice?

What led you to the conclusion that current events provide an ‘instance of an economic downturn sparking anti-Semitic sentiments’?

You state that Berinsky and Mendelberg’s finding ‘is consistent with the finding that information about Madoff being Jewish can have an indirect, and perhaps even unconscious, effect on people’s thinking about the response to economic crisis’ does this express a confirmatory relationship?

5

bob mcmanus 05.30.09 at 5:43 am

2008 The Year in Anti-Semitism …Anti-Semitism and the Financial Crisis. Anti-Defamation League.

The usual suspects and blog commenters. There might be more overseas.

I do understand that academic scholarship has no need to show relevance or utility.

6

magistra 05.30.09 at 5:44 am

This clears up a fair number of concerns about the survey, but the response doesn’t respond to comments about the range of possible answers available and how they are grouped together in the tabulations.

Regrettably, we did not ask them to evaluate any other ethnic groups.

One of the most revealing tests would have been to ask whether blacks were responsible for the crisis, given a widespread claim in the rightwing media that the problem is all the fault of bankers being encouraged/forced to give mortgage to unsuitable people (i.e. those who are poor and black)

Another interesting test would be to try the question about Jews in the UK. There are surveys which would give you baseline views about different rates of anti-Semitism in UK/US. As far as media coverage goes, Madoff was a one day wonder and I don’t think there have been any Jewish people prominent in the implosion at all (Scots have been the ethnic group coming out of the coverage worst). So that might give some kind of handle on media effects.

7

Neil Malhotra 05.30.09 at 6:41 am

Sorry, here are the full distributions:

FULL SAMPLE:
A great deal: 3.5%
A lot: 4.3%
A moderate amount: 16.8%
A little: 13.8%
Not at all: 61.6%

REPUBLICANS:
A great deal: 2.6%
A lot: 1.7%
A moderate amount: 14.0%
A little: 16.7%
Not at all: 64.9%

DEMOCRATS:
A great deal: 4.0%
A lot: 6.0%
A moderate amount: 22.0%
A little: 12.0%
Not at all: 56.0%

Obviously, one of the problems with Likert scales is that the true meaning of each particular response option is somewhat ambiguous. However, the verbal labels on these response options are based on extensive psychometric research showing that people numerically space these labels in equal intervals.

8

Henri Vieuxtemps 05.30.09 at 7:14 am

…like performing surgery in a sewer […] For example, if we had used “Jews” instead of “the Jews,” it could easily be said that many Jews were involved in the financial crisis and that our question was not truly tapping ethnic prejudice.

Right, or like pumping sewage into an operating room. Clearly, this is entrapment. Ask a bunch of random people: “are the Italians responsible for organized crime in NY city? – yes, somewhat, or absolutely not” and see what happens. Garbage in, garbage out.

9

Neil 05.30.09 at 7:27 am

Henri, you might be right about what we would get if we asked that question, but surely it would reveal something disturbing about the survey respondents? What would *you* have said in response to the questions? I assume you would have said ‘no’ to both; if you hadn’t, I would think less of you. If all that is right, then I fail to see how this is garbage.

10

Doctor Slack 05.30.09 at 7:49 am

Tim Wilkinson pretty much covers it. Especially: You state that merely using the term ‘Jews’ might not provide a measure of prejudice. How clearly do you think that those answering the questions would have consciously distinguished between this term and ‘the Jews’?

Leveraging my extensive and rigorous training as an Internet asshole, the bright-line distinction apparently being drawn between “Jews” and “Da Jews” did particularly puzzle me.

11

Doctor Slack 05.30.09 at 7:49 am

Or, what Henri said.

12

Martin Bento 05.30.09 at 8:36 am

Now, now, let’s be nice.

For clarity: are the responses in this thread from “Neil” coming from Dr. Malhotra?

On the “Jews” vs. “The Jews” question, I want to amplify what Tom said. Dr. Malhotra seems to believe that the other phrasing would have been likely to garner a different response showing less seeming prejudice. But would the survey respondents understand the significance of this difference in phrasing? It requires both a sensitivity to subtle implications of grammar and a knowledge of the history of antisemitic rhetoric, since I think fewer people would attribute the same significance to a difference between “Serbs” and “The Serbs”. The possibility that what Dr. Malhotra has captured is a difference in understanding of the question, not in attitude, is bolstered by the fact that he finds a significant inverse correlation between supposed antisemitism and educational level. So my first question is:

1) Dr. Malhotra, how do you correct for this possibility?

With a preliminary result showing an increased Democratic tendency to blame Jews for the crisis, you decided to test this against the hypothesis that Democrats just tend to blame everyone more. This seems to me an odd baseline. Is there research showing some people tend to either blame all groups or none for events, rather than differing in who they blame? This seems particularly relevant as you used the assertion that borrowers are to blame as a neutral control. But, as Magistra suggests, this assertion is not neutral with regard to racial or ethnic prejudice, at least not as it has been presented in the media, especially by those commentators and in those venues that get the most attention and respect from Republicans. Rather, efforts to increase minority homeownership have been directly and loudly blamed.

2) How do you determine that the difference in the answer to this question reflects, greater antisemitic prejudice among democrats, and not greater prejudice among Republicans against blacks and latinos, a prejudice for which there is much evidence beyond this study?

3) Why did you decide to publish a partial result in the popular media prior to peer review?

4) Will you make the survey and responses available for scrutiny?

I want to thank you for coming here, as I suggested in the other thread, and John Holbo for welcoming you.

13

Henri Vieuxtemps 05.30.09 at 8:54 am

Neil, I hope I would’ve answered that the question is absurd, but I can imagine feeling compelled to give one of the multiple choice answers; you know, because I’m being paid for participation, or I don’t want to disappoint the questioner, or whatever.

If I decided to answer, I would’ve probably tried to figure out what the questioners meant by it. I suppose I could give them the benefit of the doubt and conclude that they are asking if a significant number of the organized crime figures are people of Italian ancestry and the right answer is “somewhat”.

Now, I am a person interested in politics who reads blogs and all that. Not everybody is. For example, if my wife is asked, I’m pretty sure she won’t even notice the difference between “Italians” and “the Italians” and just say “yes”.

14

e julius drivingstorm 05.30.09 at 8:54 am

Neil, thanks for stopping by, seriously.

There were these two guys who tried to rent a car in a foreign land. Not knowing the rules of the road they kept wrecking as soon as they left the lot. Finally, the agent rented them a camel, saying that the animal will stop at the red light and go on green. Soon after that they walked back to the car lot sans camel and the agent wanted to know why. They explained they had no problem until they were stopped on red, and someone yelled, “Look at those two assholes on that camel!” Well, they got off to look and the light turned green…

I haven’t read the Vanity Fair and Christian Science Monitor articles, nor the others alluded. Is there a real groundswell for this research? Or is it research for the sake of research because that is what you do well?

15

br 05.30.09 at 11:11 am

Malhotra mentions another interesting and robust difference between democrats and republicans, about the culpability of borrowers. So democrats are significantly more likely than republicans to blame “jews’ , while being significantly less likely to blame borrowers. This might be suggestive of a difference in basic mindset, “blame others” versus “take personal responsibility”. It would be really interesting to see the full survey results, especially broken out by political affiliation. Are they available on the web somewhere? If not, perhaps they could be.

16

Neil 05.30.09 at 11:32 am

Henri, note that one of the options was “not at all”.Note that most people actually chose it. So you its false that asking questions like that is unable to get at differences between people. You are right that we need pay attention to the prgmgatics of survey design (that’s what you’re getting at, though I’m not sure you know that). That’s what social scientists do. That’s why they choose locutions like “the jews”: because they understand the effect that framing questions like that gets. You see, that’s what gathering data is about.

17

novakant 05.30.09 at 11:54 am

That’s what social scientists do. That’s why they choose locutions like “the jews”: because they understand the effect that framing questions like that gets. You see, that’s what gathering data is about.

If that’s what social scientists do and what they think gathering data is about, then I’ll never put faith in a social science survey again.

18

Ben Alpers 05.30.09 at 12:00 pm

So democrats are significantly more likely than republicans to blame “jews’ , while being significantly less likely to blame borrowers. This might be suggestive of a difference in basic mindset, “blame others” versus “take personal responsibility”.

Unless the Republicans in question were borrowers who defaulted on their loans, I don’t see how their blaming borrowers is an example of “taking personal responsibility.”

19

kid bitzer 05.30.09 at 12:13 pm

may be worth noting that phrases of the form “the phi-s” can play other roles.
it can be used to pick out one of a number of options that has been made salient.

for instance, suppose i have an urn full of marbles, 75% blue, 25% red.
i start drawing marbles at random, and showing them to respondents.
then i ask them: which ones are showing up more often, blues or reds?

they say, “the blues”.

they are not taking any stance on the existence of an essentialistic species or kind of blue marbles as a quasi-racial type. they’re just distinguishing one option from small menu of options.

i’ve got buds, miller lites, and lone-stars in a tub of ice; what do you want?
“i’ll have the bud.”
you want to rent a full-size car? all i’ve got left are infinitis and camris. “i’ll take the camri.”

my point is that any general semantic difference between “jews” and “the jews” may be significantly reduced or over-ridden by a choice-situation in which one group has been made salient.

20

kid bitzer 05.30.09 at 12:16 pm

my #19 roughly in support of h.v.’s #13

21

Henri Vieuxtemps 05.30.09 at 12:32 pm

Yeah, it certainly does count as “gathering data”, anything can be ‘data’.

I don’t like “not at all” either, the problem is that by answering “not at all” you’re confirming the premise that such a thing as “the Jews” exists and can be responsible for something or other. To some of us, this is outside rational thinking; it’s already anti- (or, possibly, philo-) semitic.

And if you do have to accept the premise, then how do you know it’s “not at all”?

22

Martin Bento 05.30.09 at 3:44 pm

Neil (not Malhotra I guess), the question may have gotten at differences between people, but the difference could be in how to interpret the question. After all, we seem to have differences among ourselves here in how to read that question. The fact that the apparent antisemitism correlates inversely with educational level would be consistent with the difference being in understanding of the question. Of course, it would also be consistent with less educated people being more prejudiced against Jews or with some combination of these factors. But how does the survey differentiate between these explanations?

23

Tim Wilkinson 05.30.09 at 4:42 pm

A couple more (rhetorical?) questions:

1. Did you examine, and what is your opinion on, the possibility that blame is considered a limited commodity, especially as between groups of people rather than abstract entities, so that those who had already assigned a larger (smaller) quantity of blame to another group would thereby tend to assign less (more) to a subsequently considered group?

2. Was the refusal rate atypical for any of the questions, and in particular for the ‘The Jews’ subquestion? Is it usual to supply no ‘don’t know’ option, and what are the expected effects of that omission?

24

Tim Wilkinson 05.30.09 at 4:57 pm

Incidentally, I’ve had a look at Berinsky and Mendelberg and they appear to claim that (not-too-roughly:) perceptions of a Jewish subject as shady can increase the degree to which that subject is perceived as liberal (opposed to conservative).

This is explained as the ‘shadiness’ cue activating an otherwise non-operative social (thus unacceptable) stereotype of Jews as shady, which then amplifies a political (thus acceptable) stereotype of Jews as liberal. Unfortunately, the authors do not appear to rule out the possibility that both perceived Jewishness and perceived shadiness can increase the perception of liberality (liberalism?).

This is perhaps due to a framing of the matter in terms of stereotypes rather than some more neutral term for describing a (not necessarily symmetric) association between traits. The significant study dealt only with 3 of the 4 duples: non-Jewish non-shady, Jewish non-shady and Jewish shady; thus the authors do not test for the ‘stereotyping’ of non-Jewish shady, or simply shady, people as liberal.

Instead, they tested in a separate study for a tendency to stereotype liberals as shady (i.e. they got it back to front as well as hiving the question off into a different study).

Unfortunately, this observation seems to put me more or less in Devastating Critique territory, and thus renders my conclusion rather implausible for general reasons. But as a good foundherentist and individualist, I accord more weight to the immediate, tractable and particular ‘evidence’ of my understanding of the matter, and do not therefore reject the conclusion on those general grounds.

If I am against all the prior odds right, then it might be an example of what general ‘philosophical skills’ can do for (or in this case to) those in other disciplines – relevant to a recent thread. This seems unbearably arrogant, so perhaps someone can help me to resolve the matter by pointing out my error. Otherwise, all this might tend to undermine the force of the ‘Devastating Critique’ heuristic somewhat, which would at least make this remark kinda on-topic.

To make it so in any case, I’d add that it is, as they say, far from clear to me that the Berinsky/Mendelberg thesis has more than tenuous relevance to the Malhotra/Margalit experiment, which is AFAICT concerned with the question whether (unavoidably roughly:) the perception of Jewishness in a prominent member tends to increase hostility to a class of wealthy persons.

But what do I know?

25

Sam C 05.30.09 at 6:00 pm

‘This might be suggestive of a difference in basic mindset, “blame others” versus “take personal responsibility”.’ – br at 15.

Um, you might want to google ‘fundamental attribution error’ there.

26

Tim Wilkinson 05.30.09 at 6:49 pm

re #24 had another incredulous look, and there is an additional complication which doesn’t seem to mitigate the problem. I won’t go into it; the relevant section is pp 851-2 for anyone who is remotely interested.

27

roy belmont 05.30.09 at 9:20 pm

McManus#3:
Ethnicity was actually relevant to the Bernie Madoff scandal because much of Madoff’s clientele was Jewish, and Madoff abused his connections within the Jewish charitable community.
This was the only ethnic specificity I caught on the MSM and their internet venues. Madoff virtually alone as the face of the causative agency for whatever it was that happened. It had the effect of creating sympathy for, and confirming the image of, Jews as victims of the “crisis”. Balancing Madoff’s culpability with the pain of his victims.
That this is early days of something much larger than mere economic “crisis” is pretty obvious, and I share your discomfort as to the likelihood of military something, or somethings, or worse still to come.
People in breadlines, people who have seen their children’s futures vaporized, don’t have much time or energy for subtle distinctions and nuanced readings of complex situations. They want blameworthy visible individuals. They want scapegoats. They want villains. And they want them to suffer.
My guess is few if any readers here are getting food from the Food Bank, but rather most are if anything looking to avoid that by any means, as well as looking to preserve their children’s futures from the vaporizing effect of any coming turmoil. Kowtowing discreetly to what looks to be the dominant power of the near future makes a lot of pragmatic sense in that regard.

28

christian h. 05.30.09 at 10:07 pm

This is absurd. From my point of view, the only possible “answer” to a question about “the Jews” is f– off. Even answering “the Jews are not at all responsible” constitutes an acceptance of the validity of the category “the Jews” and is therefore racist. So do the study authors suggest that 99% of Americans are antisemites? I guess not. The truth is that the racism was inherent to the question, not the answers.

29

Martin Bento 05.30.09 at 10:52 pm

Given that the survey was opt-in, I would like to know how it was characterized to those invited to participate in it. Selection bias could come in here as well.

30

Martin Bento 05.30.09 at 11:19 pm

With regards to the survey, are there any demographic differences between respondents opposing tax breaks in the three groups?

31

Martin Bento 05.30.09 at 11:20 pm

When I said “the survey” was opt-in, I meant the initial one, though I suppose the question applies to both if they were both opt-in.

32

Martin Bento 05.30.09 at 11:37 pm

Christian, yes, if one recognizes that as the premise of the question, there is no legitimate answer. But if one reads “the Jews”, simply as “some Jewish people”, then Not at All is simply false, unless you have an extreme theory of the crisis (neither the financial industry nor the Fed can have any culpability). Many Republicans do have such a theory, blaming the crisis on government encouragement of black and latino homeownership. Therefore, the difficulties in this question could produce a partisan skew not attributable to antisemitism.

33

TS 05.30.09 at 11:42 pm

I have no reason to doubt the overall conclusions of the study, but isn’t there likely to be some bias due to the skew in the options provided — four assessing various degrees of blame and only one that doesn’t? I suspect that at least some have a tendency to go for the options in the middle, so if you give people a broader range of options for one side (e.g., 2+2 is (1) 4, (2) slightly greater than 4, (3) a lot greater than 4, (4) much greater than 4, and so on) you get different results, especially from the clueless and the “moderate”.

Neil Malhotra:
“A great deal: 3.5%
A lot: 4.3%
A moderate amount: 16.8%
A little: 13.8%
Not at all: 61.6%
However, the verbal labels on these response options are based on extensive psychometric research showing that people numerically space these labels in equal intervals.”

34

Martin Bento 05.30.09 at 11:59 pm

Dr. Malhotra has posted some of the survey questions in the comments at the Boston Review article. Reproducing those here:

1. Many people have speculated about the reasons why the global economy has collapsed in the last few months. Below is a list of things people claim were responsible. How much to blame were each of these things for the financial crisis? (a great deal; a lot; a moderate amount; a little; not at all)

Wall Street financial institutions
Individuals who took out loans and mortgages they could not afford
The Jews

Do you favor or oppose each of the following as part of a new economic stimulus package, if that meant in the short term an increase in the national debt?

-Tax cuts for business in order to help save and create jobs

Strongly favor
Somewhat favor
Somewhat oppose
Strongly oppose

-Creating new jobs in government agencies

Strongly favor
Somewhat favor
Somewhat oppose
Strongly oppose

/cut and paste

The first question that comes to my mind is whether there is a correlation between those who assigned blame to financial institutions and to The Jews, and an inverse one between those who blamed homeowners. The initial description of this made it sound, to me at least, like blaming borrowers and blaming Jews were two separate questions, but it appears they were part of the same question. Is it possible that those inclined to blame borrowers would be less likely to blame Jews simply as an artifact: most of the poor borrowers are popularly perceived to be non-Jewish, and putting both responses in the same question would seem to encourage internal consistency in the responses.

35

Neil Malhotra 05.31.09 at 12:14 am

TS: That is a very astute observation. I definitely think that may have contribute to some, but not all, people selecting “a moderate amount.”

The other thing that concerns me (but I don’t have a solution for yet) is that respondents tend to trust the interviewer/researcher, so perhaps the question signaled that it is OK to blame Jews, since a high-status person is asking a question like this.

This is the main reason for replication, using different scales/wordings/etc. Our main objective was to introduce this topic, and show a first round of evidence. Obviously, I’ve published many peer-reviewed papers in which I usually conduct four or five studies to get it right (not one). However, in this case, I wanted to be topical and essentially write a journalistic two-page piece based on some data (not pure speculation). Hopefully, it will encourage exploration now, which it could not have done if and when it appears in print in 2 years.

-Neil Malhotra

36

Martin Bento 05.31.09 at 12:46 am

Dr. Malhotra. If this study is not up to your usual academic standards for the sake of quick publication, do you think it would have been appropriate to indicate the tentative nature of the study in the article? The article presents some dramatic and potentially explosive conclusions after all, and it presents them without caveat. If your intent is to influence the political discourse, as you seem to state, do you think influencing it with substandard material, not clearly labeled as such, has destructive potential?

37

Neil Malhotra 05.31.09 at 12:58 am

Martin:

Maybe an additional sentence of caveat given the “dramatic and potentially explosive conclusion” would have been wise. =) But I probably would not change any words of the given article. I suggest you re-read the article; the tone is basically a scientific reporting of a single study. Nothing is inflammatory; it is a straightforward discussion of data analysis. I am highly confident in the data analysis, so I do not think it is “substandard material.” Nothing is “tentative.” As I said, people can have reasonable methodological disagreements, and conduct follow-ups to add to the body of evidence.

Neil Malhotra

38

vivian 05.31.09 at 1:20 am

Neil, thanks for responding in this much detail – in some ways the Review article felt like a teaser, and this is more satisfying. You say that #35 “The other thing that concerns me (but I don’t have a solution for yet) is that respondents tend to trust the interviewer/researcher, so perhaps the question signaled that it is OK to blame Jews, since a high-status person is asking a question like this.”
One way is, after this question, ask a similar question (or question sequence) with a different framing. “Were the financial crises caused by a few people or institutions, or by widespread actions of most investors?” “Do you think it helps to generalize about the actions of broad groups of people, like the Jews, or the Wasps or the immigrants?” The point of those questions would not be about eliciting information, though you could probably word it to both signal and learn. There is probably a relevant question sequence in the Frankfurt school and similar research, no?

39

Martin Bento 05.31.09 at 1:33 am

Dr. Malhotra. I apologize if my tone was little off in my previous comment. However, you have said that your general approach – your standard – is to issue several versions of a survey to eliminate framing issues and the like and to submit to peer review. You said that you did not do that in this case because you wanted to be topical. If this study does not have the quality controls that are a normal part of your standard, is it not defensible to call it “substandard”? That’s what I meant: you did not apply same quality control as you normally do to your academic work.

40

Neil Malhotra 05.31.09 at 1:41 am

Hi Martin: No offense taken. I applied the same quality control in designing and analyzing this particular study. However, to get into the top journals, you sometimes have to run 2 more studies, to address a lot of the follow-up questions that arise. This is a highly-inductive process, where you learn based on each round of data collection. Basically, my writeup of “Study #1” in a full-fledged 40-page journal article might look very similar to the BR piece (with some more technical detail, of course). The Monkey Cage has a good discussion of why academics have a hard time being relevant, and a lot has to do with the slowness of research. I should have framed the BR piece better, but the intent was supposed to be an opening salvo and introduction to a research agenda. Think of it as the first step in my inductive reasoning process, which I am very comfortable sharing with the world to elicit feedback (whether it be at a conference, a media interview, or as an op-ed).

vivian: Those are very good ideas. Thank you!

41

Martin Bento 05.31.09 at 1:52 am

Dr. Malhotra. Is there any chance, as part of this inductive process, that you could make available the complete survey and results? I am particularly interested, for example, in the responses to the first question above: did blaming borrowers correlate inversely with blaming Jews? Also, in the second survey, did the demographics of the group that opposed tax cuts after prompting on Madoff’s ethnicity differ from the group that opposed them without such prompting? Really, though, the best thing would be to make it all public, as I’m sure have different questions.

42

novakant 05.31.09 at 1:57 am

the only possible “answer” to a question about “the Jews” is f—off.

That’s exactly what I thought. How about adding an option saying “I don’t like loaded questions designed to make me look like an anti-semite” in the next survey.

43

Neil Malhotra 05.31.09 at 2:11 am

Martin:

On your first question, the two items were uncorrelated (Pearson: r=-.04; polychoric: r=-.07).

Not sure I totally understand your second question, but randomization was successful. There was no significant difference between treatment groups across observables. Also, the test you are envisioning (as I understand it) is inappropriate because you can’t condition on post-treatment variables.

We are not going to make the data available prior to publication (these things are expensive), but may have a working paper (as part of a larger set of studies) up soon.

44

Martin Bento 05.31.09 at 2:20 am

See that’s the problem, Neil. This has already been published, but in a way that prevents full independent evaluation.

45

Martin Bento 05.31.09 at 2:43 am

Tim, in comment 24 you seem to be arguing with yourself and I can’t tell who won.

46

Neil Malhotra 05.31.09 at 2:45 am

The BR piece is not a peer-reviewed academic publication. Researchers generally do not provide proprietary survey data prior to peer review. That is the norm among academic professionals. I think your opinion on this norm is fair and sensible from the perspective of an outside observer.

47

Martin Bento 05.31.09 at 3:04 am

Yes, that’s how it works. But “prior to peer review’ usually also means “prior to publication”, but it does not mean that if you publish partial findings in the mass media. As it stands, you’ve made a set of claims based on a study no one else has seen or evaluated. These have been presented to the public as empirically sound in the ways that one would expect such claims from an academic of your standing to be sound. But such claims normally follow peer review. You may feel that your own analysis and confidence should be sufficient for the public to grant this information the credence they would to peer-reviewed findings (after all, you surely don’t expect the lay public, most of whom have never heard of peer review, to make the distinction), but if this were valid, there would be no need for peer review, and I take it you are not arguing that.

48

Neil Malhotra 05.31.09 at 3:14 am

With the Internet (and the ability to disseminate information easily), it is very possible that the concept of peer review will be defunct at some point since the world at large serves as a marketplace for ideas. Moreover, nobody thinks the mechanism of peer review is perfect for eliminating Type I and II errors.

My understanding is that people talk about their working papers on blogs/the mass media/etc., and it stimulates timely discussion. Media outlets tend to do a good job saying whether the research has been published in a peer-reviewed journal or not. Nowhere is our BR piece did we say that this paper was published or vetted by peer-review. I trust the public to process and update based on those cues. Personally, when I read people’s working papers on blogs (this is common practice, by the way), my evaluation is based upon: (1) my reading of the work; and (2) the reputation of the scholar. Not that the paper is currently sitting on a reviewer’s desk.

I apologize if my understanding of the process is not accurate, but this does appear to be the norm in the Internet age.

My last word on this is as follows. Given that the topic is so charged, I think making all of this abundantly and obviously clear would have been optimal. However, I still believe that exciting new research should be shared and explored.

49

Martin Bento 05.31.09 at 3:30 am

I hope that is merely your last word on the subject of peer review, as you have not yet addressed the bulk of the questions raised in this thread on the substance of the paper.

50

Martin Bento 05.31.09 at 3:37 am

You also did not say that the paper was not peer-reviewed, or give any indication what that means. Since the public does not likely understand peer review, its default assumption would be that your findings here should be given the same credence as any other findings from you, which is highly questionable. Publishing research in progress on blogs such as this is less of a problem as the audience tends to know the score, and, even if not, there is enough interactivity that things can be made clear to those who do not. The mass media is not like this and should not be grouped with blogs for this purpose.

51

magistra 05.31.09 at 7:21 am

I’d also like to ask about why you decided to highlight party affiliation as the most important corrolation with anti-Semitism. (You also mentioned educational level, but that was far less prominent in the story). Were political leanings actually the most statistically significant factor? Were there other factors that were statistically significant?

I’d also ask if ‘the intent was supposed to be an opening salvo and introduction to a research agenda’, why there wasn’t one mention of the fact that this wasn’t a fully completed study? It would be easy to put in ‘our initial results were’ or ‘preliminary’. Many casual readers of a newspaper article will not spot what’s not mentioned (i.e. no mention of peer review).

In other words, to what extent were you positioning your article in a particular frame to get newspaper interest: a definite result that was politically controversial?

52

Henri Vieuxtemps 05.31.09 at 7:36 am

Ah, (thanks, 34) so these are the things people claim were responsible for the crisis: financial institutions, borrowers, and the Jews. I see. It’s a good thing nobody calls it “a failure of capitalism” or anything.

53

Tim Wilkinson 05.31.09 at 11:25 am

@45. The Devastating Critique won. Berinsky and Mendelberg is demonstrably and fatally flawed, though this thread was not the place for a two-page note to that effect. All the agonising was slightly parodic but also illustrative of a common enough epistemic quandary that’s very sensitive to deferential considerations.

Well done for your leading role in wheedling out the (predictable) answers to many of the questions on the list @ 4, 23 and vindicating the Devastating Critique of the BR piece (notwithstanding the frantic backtracking of Dr Malhendra – tell him I said hello by the way!). Good to have a written acknowledgement that all those quoting the piece as a serious finding by guys from Stanford shouldn’t be doing so. And similarly thanks to TS.

The man outstanding line of inquiry is this one on the ostensibly more interesting Madoff/Jewish/big business finding (now advertised as preliminary/exploratory):

What assessment would you make of the possibility that biases in the first question might have changed baseline levels of blame carried forward into the experimental part of the study?

more specifically:

How likely is it that a person assenting to a proposition involving ‘the Jews’, even if not intending to refer to Jews en masse, might thereby acquire (consciously or not) some increased hostility to that group?

How likely is it that a person for whom the issue of Jews being to blame for the crisis has been rendered more salient or plausible might then assign increased blame to a particular jewish businessman?

How likely is it that a person assigning increased blame to a particular jewish businessman might then be more inclined to blame business in general?

54

Tim Wilkinson 05.31.09 at 11:37 am

-You know the Bad Thing?
-Er, yeah.
-People are saying the Jew Gang did it. Think they’re right?
-What?
-Are they right to say the Jew Gang did it?
-Er, well, no.
-You don’t sound very sure. Either the Jew Gang did it or they didn’t do it – which one was it?
-They didn’t do it.
-Who didn’t do it?
-Er, the Jew Gang. Can I go now?
-So you’re saying the Jew Gang didn’t do it? Say it.
-OK, the Jew Gang didn’t do it. Can I go now?
-Not if you want your TV treat.
-OK
-So did you see the game?
-No.
-Anyway, you know the Bad Thing we were talking about?
-Er, yeah.
-This was the main guy who did it. He’s like a big business man.
-Oh.
-Yeah and he’s a Jew.
-Oh, really? I didn’t know that…
-Yeah and you know all the other big business guys like him, like the other ones who did the Bad Thing?
-Yeah, big business guys, yeah…hmmm…
-They’re saying we should give them a stack of cash.
-Yeah?
-Yeah. They’re saying they might use it to help out with fixing things up, you know, fix the Bad Thing.
-I’m not giving a goddam dime to that goddam gang of goddam Jews!
-So I can put you down as a no then?

55

Henri Vieuxtemps 05.31.09 at 12:04 pm

Lol, 54. It is almost exactly like that; pushing emotional buttons to produce a dramatic result. It’s also funny that they use cheap and uncontroversial bait like Madoff, who, of course, has very little to do with the crisis.

56

MarkUp 05.31.09 at 2:09 pm

”to produce a dramatic result.”

That’s topical and exciting to you sir.

This post sponsored by AT&T text massaging messaging and idol worshiping services.

57

josh 05.31.09 at 5:49 pm

It is interesting to compare the response here to that to another co-authored article by two political scientists, which appeared in a non-peer reviewed, general interest review (and was, some critics charged, presented so as to court publicity), made some controversial claims relating to Jews — and was far less methodologically rigourous than the Malhotra-Margalit study seems to have been. This isn’t to say that anyone’s wrong to be sceptical of the M-M paper (though, if so, I think for the sake of consistency one would have to apply the same scepticism to an awful lot of survey research; but that’s certainly a defensible position); nor that the criticisms of Mearsheimer and Waltz were fair or well-informed (some were, some weren’t). But those who have been quick to dismiss the one and accept the other might ask themselves why this is so.
(Again, to be clear, I’m distinguishing here between defending M&W against [often ideologically motivated] charges of anti-semitism, and actually defending their thesis as conclusive, and their judgment in publishing the original article in the manner that they did as beyond reproach. In other words, the question is about applying different standards of rigour to the two studies, and not one’s feelings about the subsequent, politically-motivated, public debates).

58

Tim Wilkinson 05.31.09 at 6:00 pm

Google hits FWIW:

jews malhotra margalit – 1130 hits

jews malhotra margalit stanford – 628 hits

jews malhotra margalit found OR shows – 629 hits (overlap with previous: 94)

.

jews malhotra margalit "if true, it's wonderful news" – 4 hits

jews malhotra margalit "doubt the accuracy" – same 4 hits (overlap with previous: 4)

59

Tim Wilkinson 05.31.09 at 6:15 pm

The dot was because the html preview was doing something weird.

@57 – pull the other one, it’s got bells on.

60

Martin Bento 05.31.09 at 6:48 pm

For the record, I want to make clear some of the things that I don’t believe Malhotra addressed or addressed adequately:

1) Whether all survey respondents understood “The Jews” option the same way or would distinguish between this and a simple “Jews” formulation that would not necessarily carry an implication that an ethnic group was acting as a monolith. Malhotra has responded that even if his formulation overstated the level of bias, it should be expected to do so evenly, not to create divides within demographic groups but: a) one of his claims was a higher than expected level of antisemitism generally, not just the democratic skew, b) given that we’re dealing with subtleties of language and history, it is not clear that responses to this question could be expected to be uniform across demographics of education, and his own results do show a significant inverse correlation between education and supposed antisemitism, consistent with the hypothesis that the difference in answers reflects at least partly a difference in understanding of the question, c) it seems likely from the general discussion in the media and on the Internet that prevailing views among Republicans place greater blame on minority (but not Jewish) borrowers and the government, and prevailing views among Democrats place greater blame on the financial industry (the complete answers to the first question with demographic analysis would provide some insight into this, but Malhotra has not provided that). Given the greater presence of Jews in the financial industry, the blame of “Jews” could be an artifact of blame of the financial industry, for the minority for whom “Jews” is understood non-monolithically (Malhotra’s data might partially speak to this question, but it is not available). Malhotra does not address this, d) Much discourse from media venues and figures respected by Republicans has emphasized the racial or ethnic component to government encouragement of homeownership, which in turn is blamed for the crisis. This could easily have a racist component, but Malhotra does not address the question.

2) The survey was opt-in, which could introduce selection bias depending on how the survey was presented. “Would like you to take a survey about how much The Jews are to blame for our current troubles?” could obviously produce a skewed response. I don’t mean to suggest that Malhotra would use so crude a description, but there is a vector for bias here. Malhotra was asked about this and did not respond.

3) Malhotra’s strongest finding by his own account is that opposition to tax cuts to big business increases following the reading of an article about Madoff that mentioned his Jewishness, relative to one that did not. In this area, though, he does not tell us the demographics of the change, beyond the fact that it was limited to non-Jews. I asked him about this, but he apparently did not understand the question. Label the three survey groups a, b, and c, where a received no and c the strongest cuing about Madoff’s ethnicity. Call the fraction of each that opposed tax cuts x(a), x(b), and x(c). The finding is that x(a) < x(b) < x(c) to the point of statistical significance. The question is, beyond Jewishness, are the demographic characteristics of x(a) different than x(b) or x(c)? In other words, which demographic groups are responding to the ethnic cues? Does the claimed greater antisemitism of Democrats show up also in this part of the survey?

4) In terms of the credibility of the survey, in response to my concerns about peer review and the opaque process, Malhotra wrote: “Nowhere is our BR piece did we say that this paper was published or vetted by peer-review. I trust the public to process and update based on those cues.”. I think it unjustifiable to assume either that the public understands peer review and how important a component it is of academic credibility or that the public or even academics would assume that results presented to them without caveat and with no mention of peer review have in fact not been reviewed and would discount credibility accordingly. I question whether academic conclusions presented in the popular press that have been peer-reviewed are normally explicitly so labeled: that is not what I have observed. A Google search for “peer review” produces in the first 3 pages of results wikipedia and a host of relatively obscure mostly academic sites. No mass media, though media sites normally come towards the top of Google results, reinforcing the notion that “peer review” is seldom explicitly mentioned in the mass media, but remains implicit.

61

Martin Bento 05.31.09 at 10:08 pm

Josh, as far as I recall, and please correct me if I’m wrong, Mearshimer and Walt’s piece was based entirely on public information. If you wanted to dispute their conclusions, as many did, the relevant information was as available to you as it had been to them. Malhotra has drawn conclusions from a survey to which only he has access. He pushed these conclusions into the popular press circumventing peer review, but invokes lack of peer review to justify keeping the data hidden. Hence the information necessary to evaluate his claims is not available. None of this would apply to survey research generally unless the researchers publicly draw conclusions without peer review and still refuse to publish the data, making a full independent assessment impossible

62

Tim Wilkinson 05.31.09 at 10:39 pm

MB I hear jingling

63

Martin Bento 05.31.09 at 10:54 pm

Tim, obviously that’s some kind of joke, but I don’t get it. Maybe I’m being obtuse today.

64

Martin Bento 05.31.09 at 11:05 pm

I had forgotten that Malhotra found a greater tendency to blame borrowers among Republicans. Though he treats this as a control position, it is highly dubious that it is racially neutral. In fact, it is quite consistent with the view that what is actually being measured is, in large part, hostility towards blacks, latinos, and the poor among Republicans. Of course, one can blame borrowers without specifically blaming blacks or the poor, but the reverse is not true, as that is clearly the role the blacks and the poor played. Those who blame blacks must also blame borrowers, though the survey, at least what we have seen of it, unfortunately makes no attempt to measure any prejudice but antisemitism. So some subset of the “blame borrowers” contingent could be expected to be those whose opinions have a racial or class prejudice other than antisemitism (unless you think that latter group the empty set, an unlikely position, I think), and this group, to the extent they were being consistent, would be less likely to fall into the “blame Jews” (understood non-monolithically) group because of, not despite, their racial or class judgments.

To be fair, I do think it is possible to hold that government encouragement of minority homeownership contributed to the crisis without being racist (though holding it the primary cause without such a prejudice is an extremely strained argument; if not prejudiced, you are probably deeply misinformed). But if one is racist (against blacks or latinos), that is the position one is most likely to hold. One could also say that blaming Wall Street is the position an antisemite would be likely to hold. But we have plenty of evidence of continuing anti-black and latino racism independent of this study; it should be a given. High levels of antisemtism are something the study claims to establish.

65

Tim Wilkinson 06.01.09 at 12:56 am

@63: Not obtuse, just not psychic. And not a joke, just a (jocular) reference to my #58, and @61 your response (or rather your responding at any length) to #57. Not that you could be expected to trace my thought process back down that track, but in any case I guess the expression ‘pulling my leg’, still less its extensions, isn’t current in the states? Anyway, ‘pull the other one’: ‘you must be [[provocatively]] kidding’. Commonly elaborated with ‘it’s got bells on’. End of earnest explanation eliminating any trace of jocosity.

@64: Not sure I follow this – maybe getting a bit too tenuously speculative? Obviously speculation is unavoidable when the information is so scarce, and it is interesting in its own right to look into the possible undermining defeaters of a study like this, but we need to be clear about what is discussion and what counts as substantial criticism.

The whole survey/stats/empirical science thing involves taking into account all salient possibilities and then assuming that other things are equal. I reckon the obviously very relevant factors speak for themselves – and getting into more remote possibilities may tend to dilute the stringer points that have established that the article is scientifically worthless as published. Sorry that all sounds a bit overcritical and inappropriately didactic…

66

Martin Bento 06.01.09 at 1:36 am

Tom, I assume you mean 63, not 64, since 64 was the comment you were in. Perhaps I’m not on as strong a ground with that argument, but it does seem some attempt to account for other sorts of racism is called for, especially when they can be expected to correlate so strongly with this option (as I said, if you blame blacks, how can you not blame borrowers?). But it’s true that sometimes piling on arguments is counter-productive.

67

Martin Bento 06.01.09 at 3:43 am

Er, Tim, not Tom. Sorry.

68

Alarob 06.01.09 at 5:35 am

Neil Malhotra’s description of the article has undergone an interesting change over the course of this thread. He begins: “our objective in this article was scholarly, and was not to promote any particular public policy.” But @35 Malhotra tells us “I wanted to be topical and essentially write a journalistic two-page piece based on some data.” He recasts the piece @40 as “the first step in my inductive reasoning process,” which he chose to share with “the world” to elicit feedback — not, however, through a blog, listserv, or other medium of “the Internet age” (@48), but by publication in a (non-peer-reviewed) magazine.

Peer review, he suggests, is yesterday. Anyway it takes too long. Yet he declines to engage with the very significant concerns raised in this thread, and well summarized by Martin Bento.

By @48 the article seems to have become a “working paper,” and Malhotra pretends that there is no real difference between publishing a working paper on a blog (“common practice, by the way”) and publishing it in the “mass media,” as he has done. This is nonsense. I respect Malhotra’s intelligence enough to believe that he knows it’s nonsense.

Enough has been said about the flawed design of the survey, especially question 1. I’ll just point to the odd choice of the word “things” to include both “institutions” and people, including “The Jews.” It suggests far less attention to the wording of this survey than Malhotra has claimed for himself. (“…Below is a list of things people claim were responsible. How much to blame were each of these things for the financial crisis? (a great deal; a lot; a moderate amount; a little; not at all)” [emphasis added])

This has been a very disappointing showing from Malhotra and the absent Margalit. It’s my opinion that the Boston Review should publish their retraction.

69

Tim Wilkinson 06.01.09 at 9:57 am

MB 66 (65?)- yes, I was being obtuse there. I suppose I just think the more fundamental problem is the token choice of sole alternative hypothesis ‘Democrats (tend to) blame everyone’. Thus amended it would also be uneliminated if Dems were (are) less inclined to blame the poor/victims, or if Republicans were inclined (as someone suggested) to be more judgmental about personal financial responsibility.

It strikes me that a veru basic factor relating to the experiment (Madoff) part is that real-world examples were used so that this kind of complicating factors weren’t controlled for (as well as using the same subjects as for the first part – which it appears they did though M won’t confirm it).

I was wondering what would happen to the numbering, as I’ve got a comment awaiting moderation which I can see and you presumably can’t yet. Not that it’s worth the moderator’s effort really.

70

Tim Wilkinson 06.01.09 at 10:44 am

Alarob, MB: A couple more highlights from NM’s remarks:

“academics have a hard time being relevant, and a lot has to do with the slowness of research” – in other words, in order to be ‘relevant’, you can cut corners. (Implicit was the idea that if it’s not in an academic journal it doesn’t matter if it’s substandard – never mind if it involves propagating falsehood to the public and various polemicists – as long as it doesn’t spoil your CV. )

I also liked “nothing was inflammatory – it was a straightforward discsussion of research” – apart from, for example, the entirely unfounded assertion that current events provide an ‘instance of an economic downturn sparking anti-Semitic sentiments’…

And then the staggering stuff about peer review/transparency! Reminds me of Blair et al’s attitude to government – I know I’m basically right; all this other stuff is just pointless technicality. Or ‘noble cause’ corruption in the police. In fact the academic canons involving peer review etc are just as integral to the validity of a scientific (political science/social psychology) paper as getting the experimental design and methodology right – they are even part of that methodology you might say. Justice – not done unless seen to be done. Conflict of interest – exists if it might exist. Academic validity – not achieved if not independently verified to have been achieved.

Or something along those lines.

71

Tim Wilkinson 06.01.09 at 12:18 pm

Anyway this party is clearly winding down. Before we go our separate ways, leaving our napkin-jottings to be tidied away, ‘What’, as the question goes, ‘Is to Be Done’?

Alarob @68 (67?) – there won’t be a retraction, and there is possibly a bit of a tendency for a closing of academic ranks, and certainly little mileage for professionals in criticising a ‘pop’ piece, so I don’t expect any intervetion from that quarter. If anything is indeed to be done, I/someone should get round to a clear and concise but comprehensive statement of all those things that have been clearly established as wrong with the piece (as supplemented by NM’s comments). Something dull and worthy just for the record, and preferably published in some salient, stable and enduring (and if poss reasonably reputable) location.

This may seem excessive, but for my part I do get affronted by the arrognance and irresponsibility (in Malhotra’s case – in others, deliberate mischief-making) of this kind of thing, and the ease with which people get away with it. (No #57, not that kind of thing – though especially since the rise of the neocons and their Likudnik/pro-torture etc agenda, I do I suppose take an increased interest in the abuse of antisemitism accusations/insinuations.)

And we must remember – Dr Malhotra (and possibly Dr Margalit) is relying on us to provide an inferior counterpart of peer review (and peer response) in this brave new, uh, marketplace of ideas.

As an example/case study: I did an insanely plodding and almost unreadable (it had sentences like this one in it) internet article directed at A. Dershowitz (#57’s ears prick up again here – but I won’t/wouldn’t rebut those insinuations even if I could/can), refuting his irritatingly frequent, self-satisfied and unquestioned, though rather obviously false, claims to have opposed the Iraq war.

Dersh’s output (at least since he co-authored a couple of actual articles in actual law journals, way back in the civil-rights days) is a bit like this piece: it’s influental and benefits from apparent academic credentials (I can’t imagine how many rpm Felix Frankfurter must be going at), but isn’t actually (if you look carefully enough or pin the author down) put forward as serious academic output, with the odd exception like his painful foray into legal philosophy Rights from Wrongs. Partly as a consequence (though only partly, as this does apply to, e.g., R from W) criticising the stuff is considered unnecessary, unprofitable and infra dig by (most of) yer actual academic experts.

Anyway, Dersh and the war: even though (or because) the arguments and meta-arguments are rather long-winded, tedious and often obvious, they had (I thought) to be put together and made tangible somewhere. No-one would remember or recite them after reading them on a blog even like this one – but if there’s some salient and single convenient locus where they reside (albeit in slightly embarrassingly badly-written and over-cautious form) to be referred to, people will do so. The Dersh piece has actually been cited in print a couple of times so far, so is at least definitely entering the human ‘memome’ – and a url is a lot more transmissable than a long-winded and somewhat intricate argument. Someone has to do these things IMHO.

Right, that’s enough. The hour grows late, our number dwindles and I am pontificating more loudly and at greater length, as our host helpfully calls us a taxi…

Quick PS – on the commonplace allusion to a ‘marketplace of ideas’ – a possible CT blog post on the applicability of such an analogy, and its desirability as a model? It’s no doubt been discussed before, though not as much nor I should think as profitlessly as has the analogy with biological evolution by natural selection, that other Invisible Hand working to make sure whatever happens happens…

72

Martin Bento 06.02.09 at 3:39 am

Well, one thing I think we should do is visit the comment places of some of the more prominent of the google hits, and point out some of the problems with taking this study seriously. I plan on hitting Volock and the Boston Globe. Don’t overdo it; we don’t want to be seen as spamming, just getting the full story out. After all, we wouldn’t want the megaphone that accrues to us blog commenters to drown out the Stanford professors who get published in the Boston Review (/snark). I realize some people think we may just be drawing more attention to a study that hasn’t gotten that much so far, but I think this is a sleeper and will eventually attract more attention if it remains unchallenged.

73

Martin Bento 06.02.09 at 6:37 am

Tim, oh, by the way, the reason I didn’t get the “jingling” reference is that I hadn’t spotted the “bells on” comment upthread. We do say “pulling my leg” in the states, though the “bells on” variant is something I only encounter online.

74

rich 06.02.09 at 2:51 pm

Seems to me that reference to the actual results do not support the assertion of full-on and broad anti-Semitism. The original post made the all-out claim that “32% of Democrats and 18% of Republicans ‘blame the Jews’ at least moderately for the financial crisis. ” Even a cursory look at the actual responses to the question directly contradicts the (implied and explicit) statement that 32% & 18% “blame The Jews” for the financial crisis, which totally undermines the supposition that anti-Semitism lurks everywhere and rears its ugly head in economic downturns. You cannot lump varying numerical results like that to make an overarching claim about a highly qualitative subject.

Respectfully, inserting the words “at least moderately” into the above sentence is a fig leaf at best for an unwarranted conclusion. That sentence clearly advertises, explosively, that “32% of Democrats and 18% … blame the Jews” — at best an irresponsible construction that does not accurately reflect the actual survey results.

The ACTUAL survey results reveal that Malhotra & Margalit could only 3.5% of the full sample (2.6 & 4.0%) felt the Jews were “a great deal” to blame. That’s indicative of an entirely different story than the one the authors present. Prejudice is necessarily absolutist in nature, and responsible analysts might be justified in suggesting that up to 7.8% of respondents in the full sample (‘a great deal’ + ‘a lot’) pointed to an irrational and perhaps anti-Semitic sentiment — but no more.

But the phrase “a moderate amount” clearly steps out of the realm of virulent hatred and these results cannot be attributed to anti-Semitism. How many Aryan Youth or neo-Nazis have you ever heard say ‘I attribute a moderate amount of responsibility to The Jews, because although they control the global financial system, there’s enough blame to go around and after all, plenty of Gentiles work at CitiBank too. None. You never hear that. Anti-Semitism is an all or nothing proposition.

And once you realize that, one understands that the overall representation and summary ‘results’ are totally compromised.

The phrase “a moderate amount” by definition excludes the survey results from being interpreted (and I use that word loosely) as anti-Semitic, for at least two reasons. Moderation and balance are not part of the anti-Semitic equation, nor of hatred in general, because anyone engaged in bigotry must label human beings with absolutist descriptors (necessarily untrue) to degrade others and justify their own self-worth. Anyone who ‘blames the Jews’ in part is not driven by hatred, but may well believe that people who happen to be Jewish—among other non-Jewish people—participated in creating systemic problems that led to the financial collapse. Second, the survey options do not necessarily measure strength of anti-Semitism, but perceptions about the level of participation in the financial industry. It’s entirely possible to interpret that 16.8% as a wholly positive response: asked if they ‘blame the Jews’, these respondents may well be thinking ‘Hell No, I don’t blame the Jews, they were only involved “a moderate amount” because plenty of other groups participated too.’

At root, there’s an internal contradiction in “[] blame the Jews …at least moderately” that precludes any assumptions of anti-Semitism. So it is no surprise that that self-defeating construction is preceded by “[a surprisingly vast proportion of respondents (i.e., 32% & 18%)]” blame the Jews … ” You don’t want to lead with language exposing the absence of bigotry, especially when it’ll lead people to figure out that the solid numbers were in fact 2.6% & 4.0%.

Again, a very strong disjunction here between what’s being measured and the conclusion. So much so, that Malhotra & Margalit cross over into the absurd: this is a scene out of Monty Python’s Life of Brian in which Brian is asked “How much do you hate the Romans?” And of course the question is absurd and childish so Brian is left to protest “A lot!”

How do you prove such a thing? You either do or you don’t hate The Romans/The Jews, and like Brian though from the flip side, M&M’s survey respondents are left trying to gauge how much they hate ‘The Jews’.

Of course, their question is bait. Use of the term “the Jews” traps anyone who responds to the question. It is a loaded phrase; one designed to elicit specific conclusions; and cannot be relied upon to deliver reproducible results. So much so that any (non-anti-Semitic) respondent who honestly & objectively thought that people who happened to be Jewish were “a great deal” responsible — in the most factual & non-emotional sense — had to answer that they “blamed the Jews” in order to participate in this survey.

Malhotra & Margalit clearly suffer from a garbage in-garbage out problem, one that colors their reading of the results. Having constructed questions that included widely-known loaded phrases with strong potential to skew responses in either direction, it’s no surprise they drew the conclusion they did. It’s what they were looking to find.

I want to say I’m surprised Malhotra & Margalit got this far. But frankly, their errors are precisely what make this such a fine case study in mis-translating statistical data into qualitatively accurate sentences in the English language. And mis-translations of topic and process are at issue on the back-end as well: it helps to understand what anti-Semitism (bigotry) is, and whether you are measuring it.

Neil Malhotra @ 7 provided the full distributions, Dem & Repub; below is the distribution from the full sample.

FULL SAMPLE:
A great deal: 3.5%
A lot: 4.3%
A moderate amount: 16.8%
A little: 13.8%
Not at all: 61.6%

75

rich 06.02.09 at 3:16 pm

christian h. raises the obvious concern:
_____
christian h. 05.30.09 at 10:07 pm

This is absurd. From my point of view, the only possible “answer” to a question about “the Jews” is f—off. Even answering “the Jews are not at all responsible” constitutes an acceptance of the validity of the category “the Jews” and is therefore racist. So do the study authors suggest that 99% of Americans are antisemites? I guess not. The truth is that the racism was inherent to the question, not the answers.
_____

christian h. is right on. The validity of the survey itself is in question.

But beyond that, we cannot assume that respondents actually accepted that premise or reacted predictably. There are a whole range of possible interpretations people make before proceeding to answer:

a) The Presumed Anti-Semite — “Ah-ha! Finally a chance to answer a poll conducted by researchers who really understand that The Jews are to blame for everything! At long last!
b) The American Non-Bigot — “I can’t believe these yokels actually used the term “the Jews” in an actual survey! Don’t they know it’ll muck up their results? Better read carefully.”
c) The Fair-minded Naif — “Blame ‘the Jews’? Boy that’s careless—musta meant Jewish people. I guess I’ll just answer the question I thought they meant to ask rather than the one they actually did, even if it could be read the wrong way… (because who would test anti-Semitism this way?)”

Several observations:
— The thought process of respondents is not predictable.
— 59% of Democrats and 37% of Republicans, pencils poised, may’ve thought: “this question confirms my view that 95% of idiot sociologists will think this proves anything.”
— Nothing about this survey accounted for the lack of predictability, nor for the highly variable interpretations and responses likely to arise from such a poorly constructed survey instrument.
— The question does not measure anti-Semitism.

76

Henri Vieuxtemps 06.02.09 at 3:32 pm

Right. Also his explanations of the absence of a peer review (it’s necessary, but not for this publication, actually it’s a thing of the past) remind me of the broken kettle joke: I returned it whole, and I never borrowed it, and it was already broken when I got it.

77

rich 06.02.09 at 4:27 pm

kid bitzer @ 05.30.09 at 12:13 pm

kid blitzer gets at yet another interesting point. Surveys set up a set of implicit rules. To answer successfully and accurately, you have to understand and respond to those rules.

it may be worth noting that phrases of the form “the phi-s” can play other roles.
it can be used to pick out one of a number of options that has been made salient.

for instance, suppose i have an urn full of marbles … then i ask them: which ones are showing up more often, blues or reds?

they say, “the blues”.

they are not taking any stance on the existence of an essentialistic species or kind of blue marbles as a quasi-racial type. they’re just distinguishing one option from small menu of options.

In short, Malhotra & Margalit identify the category of people about whom they desire respondents to state their opinion. That demand overrides any essentializing framing associated with Jewish people, by the rules of the survey game. The sample is asked who do you blame, not whether you agree that the loaded term is appropriate or not. Unless you object, and actually walk out, you’ll be counted as accepting the premise. Neil has already misconstrued the low drop-out/nonresponse rate erroneously. Though use of the loaded term ‘the Jews’ can pump responses up or down the available scale, there’s no way to know in which direction, or how far, that actually happened.

Malhotra & Margalit also play a disingenuous semantic game, often used in the discipline of Philosophy, that sucks in respondents based on one advertised premise, but then betrays those respondents by switching assumptions and measures midway through the ‘game’.

So there are two sets of rules for two games in play here. The survey game, and the philosophy game. No disrespect to philosophers, but often enough a statement is made indicating one agreed-upon set of semantic rules, upon which one combatant ‘wins’ by switching definitions in midstream to claim the concluding ‘victory’. (Ex–Environmental Ethics: ‘It’s ethical if it’s good for the cow.’ Philosopher, disingenuously: ‘If you change the oil in a tractor, it’s good for the tractor, so that can’t be the basis for ethical standing.’)

Malhotra & Margalit ask who is to blame, studiously identifying which group is at issue. When they draw conclusions, the rules of the survey have been changed.

The question is no longer ‘How many people think this group is to blame a little or a lot?’ Now the question is ‘How many respondents are anti-Semitic, and How much do they hate The Jews? A little? “A moderate amount” of hatred reserved for The Jews? Or “a great deal” of anti-Semitism with my survey and my apple pie?

That switcheroo or semantic game-playing or disingenousness or sleight of hand — whatever you feel is accurate — exposes another fatal methodological flaw.

If the survey rules really purported to assess prejudice, it’d be one thing. But that’s not what they present themselves as measuring. Additionally, we don’t know whether respondents fled from responding at the high end (‘a great deal’) or were attracted to it when they encountered the loaded terminology. It can’t be assumed that people answer honestly to polls that are obvious traps.

78

Tim Wilkinson 06.02.09 at 6:21 pm

New (net) hits in last 2 days:
jews malhotra margalit – WAS 1130 hits, NOW 1140

No disrespect to philosophers, but often enough a statement is made indicating one agreed-upon set of semantic rules, upon which one combatant ‘wins’ by switching definitions in midstream to claim the concluding ‘victory’. (Ex—Environmental Ethics: ‘It’s ethical if it’s good for the cow.’ Philosopher, disingenuously: ‘If you change the oil in a tractor, it’s good for the tractor, so that can’t be the basis for ethical standing.’)

Well, I’m biased, but don’t think that’s very fair, at least the example mentioned. Non-philosophers tend to get impatient with philosophers because they think they’re being deliberately obtuse or underhand, when in fact they’re just doing their job. This example sounds like a bit of dialectic to me – formulation, counterexample, reformulation…etc etc. It’s not switching meanings – it’s trying to establish the meaning (or the nature of the principle, or something anyway). Certainly not a ‘victory’ any philosopher would be interested in gaining.

On the other hand in the context it could have been a bit of snide point-scoring, but philosophers are always providing counterexamples to each others’ suggestions – it’s helpful. Even though it can be annoying, what is then annoying is that you hadn’t got it right, rather than that someone was rude enough to point the fact out – unless they keep quiet until after you’ve published I suppose…

Anyway that’s only a side-issue so no need to get into it really. Just sticking up for philosophers.

79

rich 06.02.09 at 7:53 pm

Tim,
Don’t mean to overgeneralize here, and like you will not be exploring that phenomenon today.

If two discussants are listening to each other in good faith and moving together towards a mutually-produced insight or truth, that’s great. All well and good.

When both parties know what the semantic sense of a premise is, but one party takes issue not with that intended meaning but rather pretends to mis-state that semantic premise or pretends the 1st party has misunderstood it, in order to ‘win’ a point, I think the process is not what ‘you’/philosophers often want it to be. Or like to present it as. In such cases, it seems to me the emphasis is on taking ownership of Truth at the expense of your fellow philosopher, or scoring a ‘win’ by betraying a mutually constructive dialog with a fellow equal; in which case the purpose is to own the discourse and dominate the other person. Not to reach new heights of enlightened understanding.

I’ve seen too many examples of this behavior — approach — to misunderstand the dynamics in play here. This has nothing to do with rudeness. I do not cast all philosphers in this light, nor claim that all are just mean, combative jerks. I merely point out that some folks are vested in pointedly mis-understanding what someone else is saying in order to take ownership of what passes for truth, by dominating a social interaction, both at the expense of the other party. This is not ‘teaching’.

Language has performative aspects, which are routinely ignored in these circumstances. As much as outsiders are obligated to understand the mechanics/dynamics of philosophy, so too are philosophers obligated to understand the reality of domains they seek to address (in my example, ethics, environment, language). If no attempt is made to do that, we again are left playing by the rules of philosophy without reference to the reality of other fields or customs of other disciplines.

You wrote: “It’s not switching meanings – it’s trying to establish the meaning.” That’s not the case–the leader of the discussion could have asked ‘what precisely do you mean by ‘it’s good for’? Let’s examine that turn of phrase.’ Instead, the counter-example was couched as an ‘Ah-ha’, despite having to take a stance that a) was non-responsive to the premise, b) disingenuously posited that a mistake had been made by switching semantic meanings of the same phrase in mid-discussion, and c) incorrectly implying that introduction of a counterexample had any utility, was somehow appropriate, or was used to teach rather than show someone up. (Try to think of this as a composite example/case.)

Please note three observations:

–It’s been said here that “other disciplines don’t understand what we do.” That’s not borne out by experience: many/most know full-well ‘what philosophers do’, read the game expertly, but have more respect for their peers and choose not to participate.

–You wrote that “Even though it can be annoying, what is then annoying is that you hadn’t got it right . ..” That is precisely the antithesis of the case-in-point, and wholly misrepresents the pedagogical dynamic. Time and again, the purpose was not to tease out a more precise meaning or more accurate insight, but to converse in bad faith and to behave unresponsively. Be aware that my example is a stand-in for that continual pattern. In short, I hadn’t gotten it wrong or been shown the light of understanding: rather, an entire classroom was left wondering why prof & TA felt themselves an authoritative arbiter when they didn’t engage substantively or process-wise in a manner that helped them as teachers understand what it was they presumed to cut down, either interpersonally in class or across the issues & disicplines at hand.

–You write, “when in fact they’re just doing their job.” What is ‘their job’? To teach? I think everyone is clear what the job of a philosopher is. Obviously I understand your statement about establishing meaning, working through a dialectic process, and providing counterexamples. When ‘finding counterexamples’ is actually an exercise in semantic bad faith, it’s another kind of game entirely. The language becomes performative, and it’s not about helping each other discover useful, accurate insights. When the finding of fault takes precedence over making minimal effort to understand what the other person’s saying, it’s no longer about teaching. When semantic bad-faith becomes the operant method, the process has gone far, far beyond “a bit of snide point-scoring” as you put it. That’s small potatoes. Learning from anyone and from your own students, is pretty much the only sure sign of a good teacher (what did I mean by ‘good’?!? Not sinful??). At the other end of the spectrum, a radical refusal to learn from those around you marks a teacher-led reversal of what everyone believes to be in place and in order: ‘school’, ‘classroom’, ‘teacher’.

I did find, in this specific class, a clear misunderstanding in philosophy of the relationship of teacher and student, of process and of teaching teaching itself, and of the general social and pedagogical rules governing these generally. Equally damaging was the refusal to understand the substance, insights and processes of other disciplines: if one fails to understand language, ecology and ethics, and refuses to grasp how their contributions bear upon a given discussion, then philosophy will have little of consequence to contribute to cross-disciplinary debate or to any of those peer disciplines.

The point is simple. If it’s not a two-way street, then it’s not a street you’re on. The teacher who doesn’t understand this, regardless of discipline, isn’t teaching or contributing.

Again, I don’t assume anything about philosophers in general, nor do I presume to critique ‘All of Philosophy’; I merely point out that method can be madness in one very specific sense of that term. Can it be rational to proceed down that path?

80

Martin Bento 06.02.09 at 8:01 pm

Rich, I don’t agree that there is no such thing as moderate prejudice, although by grouping the moderates with a much smaller group of stronger respondents and invoking the history of 19th and 20th century European antisemitism, with its grim denouement, as a parallel, Malhotra may be overstating the antisemitism he claims to have found. But I think the stronger point is that it is not clear that he has found the antisemitism he claims to have.

81

Martin Bento 06.02.09 at 8:13 pm

OTOH, your point that anyone who thought Jewish people as individuals had some culpability had to either “blame the Jews” categorically to some degree, or, recognizing the antisemitic trap, answer “not at all”, even though that does not accurately capture their opinion, is well-taken and in line with what I and others have said.

82

Martin Bento 06.02.09 at 8:28 pm

Something else: Malhotra said he wasn’t sure he understood my question about the demographics of the survey groups cued (or not) about Madoff’s ethnicity. He said “There was no significant difference between treatment groups across observables. ” He did not say whether there were such differences among the subsets who favored tax cuts (save on the parameter of Jewishness). However, if the demographic skew he claims to have identified is correct – if there is more antisemitism among Democrats than Republicans – we should expect to see that here as well. Democrats should, being more antisemitic, respond more strongly to the ethnic cues about Madoff and therefore being more strongly represented in the groups that received the cuing and favored the cuts than in the group that did not (it stands to reason that Democrats will likely be more strongly represented in all groups that favor such policies, as such policies are in keeping with Democratic positions that have nothing to do with antisemitism. The question is whether ethnic cuing makes Democrats even more likely to support such policies, relative to other groups). It seems likely that Malhotra would realize this, and would mention it if it supported his conclusion, especially as he regards this as the strongest part of his study. Yet he has not said this and said he didn’t understand the question when asked (in more general terms). Also, the fact that he factored out Jews means that he did do some demographic analysis of the groups opposing tax cuts, but apparently not to test one of his main theses, or, if so, he has not shared the results with us.

83

Tim Wilkinson 06.02.09 at 10:22 pm

rich @79, I can be a bit naive – and not very observant – about how thoroughly unpleasant people can be for motives I don’t really get. I also probably wasn’t really thinking in terms of classroom situations. I take your point about not overgeneralising. Hard to say much more about the case you describe, as your remarks are (of necessity I imagine) in quite general terms. But you pretty clearly describe something fairly monstrous. You should re-post the comment – or a link – to the recent thread on a similar topic – it might be of interest as a case study?

84

Martin Bento 06.02.09 at 10:40 pm

Oops, at least two points in the foregoing I misstated “opposed” the cuts as “favored” the cuts. I think my meaning was clear, but apologies if not.

“Democrats should, being more antisemitic, respond more strongly to the ethnic cues about Madoff and therefore being more strongly represented in the groups that received the cuing and favored the cuts than in the group that did not.”

should read:

“Democrats should, being more antisemitic, respond more strongly to the ethnic cues about Madoff and therefore be more strongly represented in the groups that received the cuing and opposed the cuts than in the group that did not.”

and

“The question is whether ethnic cuing makes Democrats even more likely to support such policies, relative to other groups”

should read:

“The question is whether ethnic cuing makes Democrats even more likely to oppose such policies, relative to other groups”

85

Tim Wilkinson 06.02.09 at 10:58 pm

MB @82 Yes – given that the experiment was the bit that was supposed to establish actually operative prejudice, rather than something less ‘meaningful’, among the same subset (‘To assess more deeply whether the tendency among a subset of Americans to blame the Jews is meaningful, we conducted a controlled experiment’) you certainly would have thought that a similar gap between partisan groups would have been reported if found. A breakdown by partisan group for the 2nd study was one of the disregarded requests in the litany @4.

I suppose putting them all together was a mistake insofar as the aim was to get answers, as it made it more difficult unobstrusively to cherry-pick acceptable questions, as well as looking a bit daunting/excessively demanding.

86

roy belmont 06.03.09 at 8:06 pm

christianh#28:This is absurd. From my point of view, the only possible “answer” to a question about “the Jews” is f—off.
Besides that being the optimal answer to almost any poll question, and ignoring the residual bizarrity of “obscenities” having to be strategically disemvowelled and deconsonated to ensure the comfort levels of who exactly I’m not sure but yes okay no “the Jews”. No such thing. False construct. Canard.
And but yet also then as well no “the Americans”, as in “The Americans” didn’t invade Iraq, because there is no “the Americans”.
“The Germans” didn’t start WW2 because no.
Etc.
But then on the other hand Rahm Emmanuel isn’t exactly what was it, mopping floors at the White House, is he?
And whoever invaded Iraq pretty clearly did so at the urging of a semi-covert relatively small group of mostly all men many of whom the majority of whom were and are Jews. Not that Richard Perle carries Allen Ginsberg’s social consciousness vote by inherited proxy but still…
Remember Lynndie England? Abu Ghraib poster girl?
A hillbilly. Perfectly acceptable term of bigoted generalization with derogatory overtones, having no more basis in actual reality than any other “the Blanks…” formation. And isn’t that a ponder now how that makes it so smoothly into the mainstream media and popular internet discussions. Because its target is an economically powerless social minority, of course.
It’s kind of amusing to see how far people will run away from glaringly obvious facts to preserve their sense of their own fairness. But also kind of scary, as the world tilts toward dystopic delamination and rabid self-interest and blind ethnic chauvinism become survival characteristics, for those that have them. Enabling is the downside of over-compensating for perceived historical injustice.
It will be a comfort for the unbigoted to know that no matter how badly they’ve been rooked, reamed, or rode hard and put up wet, they were nice people all the way, very concerned about the treatment of others, sort of, though universal anti-bigotry does still seem a ways off, considering how much anti-Arab prejudice is prevalent in our noisy little world.
Also the moral one-sidedness of the Sri Lankan/LTTE Tamil conflict.
Not much concern voiced about that that I’ve heard or seen. No polls, no poll criticisms. And the victims there have suffered tremendously, unjustly, and “fate” bids fair promise to continue that suffering, with no serious objections raised by otherwise “nice” folks in the mostly complacent but increasingly somewhat anxious bastions of privilege from which emanate these concerns about superficial propriety and semantic political correctness.

87

Martin Bento 06.03.09 at 8:22 pm

Tim, yes, I think all of your questions were spot-on, but there were so many that Malhotra had an excuse to ignore you, which he basically did. Better to pick a few of the strongest and push for answers than to get baroque. Ah, well. In any case, I’m working on a summary that I hope to have posted here before comments close.

88

Martin Bento 06.04.09 at 2:21 am

Here’s what I have for a summary. Who do you all think?

There are a number of problems with Malhotra and Margalit (hereafter “Malhotra” for simplicity) article that anyone tempted to take it seriously should consider. These have come up in two discussion threads on the academic blog Crooked Timber (here and here), the second of which Malhotra himself participated in.

1) Though the article does not mention it, the work has not been peer reviewed and Malhotra has declined to share his data. By going to the popular press, Malhotra circumvents peer review, but to justify not sharing his data, he invokes the fact that the study has not yet been peer reviewed. Peer review is a vital part of academic quality control where prominent authorities in a field check each other’s work, normally prior to publication. Work that is not peer reviewed is not normally considered fit for academic publication. Though the popular press has no such standards, academic work cited there is usually work that has been published first in academic journals and therefore has been peer reviewed. In response to questioning about the lack of peer review, Malhotra wrote: “Media outlets tend to do a good job saying whether the research has been published in a peer-reviewed journal or not. Nowhere is our BR piece did we say that this paper was published or vetted by peer-review. I trust the public to process and update based on those cues.” Although he claims media outlets tend to say whether work has been peer reviewed, he himself, writing in a “media outlet”, did not do this. He apparently assumed that if he did not mention peer review explicitly that the public would assume the work was in fact not peer reviewed and “process and update” (i.e., discount credibility) accordingly. In fact, the popular press normally cites published academic work without mentioning peer review, which the public does not understand. A Google search for “peer review”, as of this writing, shows no mass media sources in the first three pages of results, reinforcing the observation that peer review is seldom mentioned there. There is nothing in the article to indicate that the study should not be given the same credence as any other academic work, but for anyone who accepts the legitimacy of peer review, it is hard to see how it could be. After some insisting, Malhotra acknowledges that he should have made this clear “I think making all of this abundantly and obviously clear would have been optimal. However, I still believe that exciting new research should be shared and explored.” However, he is not really sharing his research – his study – he is keeping it private, and he is certainly not leaving us free to explore it. He is only giving us some conclusions with selected supporting data points.

2) In response to discussion of the preceding concern among others, Malhotra speaks of this study in terms that seem much less conclusive than the impression the article gives. The article draws dramatic conclusions presented without caveat. In discussion, Malhotra describes the study thusly: “the intent was supposed to be an opening salvo and introduction to a research agenda. Think of it as the first step in my inductive reasoning process, which I am very comfortable sharing with the world to elicit feedback.”

3) The question about “The Jews” has as its premise that it makes sense to place blame on an ethnic group, and acceptance of this premise is what is taken to denote antisemitism. But no option was provided that specifically rejects the premise; even holding “The Jews” blameless does not reject “The Jews” as a category of people that might be blamed.

4) Malhotra himself has admitted that simply specifying “Jews” instead of “The Jews” may have gotten a different result or a result that would be interpreted differently. It is not clear that all respondents will understand the intended distinction between “Jews” and “The Jews”, however. Grammatically, the distinction is arguably spurious: people often use the definite article when selecting one from a limited menu of choices. The difference is chiefly an artifact of the use of “The Jews” as a monolithic historical agent in the history of antisemitic rhetoric. It is unlikely that all respondents are familiar with this history. The fact that blame of “The Jews” varies inversely with education is consistent with the notion that the differences between respondents are in their understanding of the question, rather than their substantive views. Malhotra has provided no way to separate this effect from the antisemitism he claims to have found.

5) Malhotra has said that the strongest finding of his study was the second survey, which was intended to verify that the conclusions drawn from the first were meaningful. As Malhotra put it in the article: “To assess more deeply whether the tendency among a subset of Americans to blame the Jews is meaningful, we conducted a controlled experiment.” This survey measured opposition to tax cuts for big business among groups that had (or had not) received cuing about the ethnic identity of Bernie Madoff (who is Jewish). This second survey provided an opportunity to confirm or refute the position that Democrats were more antisemitic than Republicans. Although one would expect Democrats to tend to oppose such tax cuts more than other groups as part of their general political philosophy, if Democrats were antisemitic, they should take this position by an even stronger margin relative to other groups when Madoff’s ethnicity was mentioned or suggested. Did the data show this? Malhotra was asked this question in two different ways by two different people and declined to answer, ignoring the question in one case and claiming not to understand it in the other.

6) Malhotra tests the thesis that Democrats are antisemitic against the thesis that they just blame everyone more, which is an odd alternative, since blame is usually at least somewhat selective. To test this he looks at the tendency to blame borrowers who took loans they could not afford, which he finds stronger among Republicans. By using this position as a control for antisemitism, he is positioning it as a neutral position in terms of racial and other prejudices. Otherwise, the antisemitism he claims to be measuring could be an artifact of another kind of prejudice, and, indeed, there is reason to believe this is the case. There has been much arguing in the mass media, particularly from those figures and in those venues most respected by Republicans, that the cause of the financial crisis has been government efforts to encourage minority homeownership, generally understood as Black and Latino, not Jewish, homeownership. While blaming borrowers does not necessarily imply blaming racial minorities, the reverse is not true, because it is as borrowers that racial minorities have played a role in the crisis. Those who blame blacks must blame borrowers; no other option makes sense. While one could blame minority housing initiatives without being racist, those who are racist would obviously have a bias in this direction. Therefore, the group that blames minorities because of racial prejudice constitutes some portion of the group that blames borrowers. Is it a significant portion? There is much other evidence outside this survey both that racism against these groups persists and that some believe that minority housing initiatives are responsible for the crisis, so this group is likely to be significant.

People who understood the question as simply one of whether some Jewish people had culpability, rather than Jews as a category, would be unlikely to blame Jews if they blame borrowers, simply because that is not the role Jews largely held. On the other hand, if they blame Wall Street, they may blame Jews as an artifact, because of the perception that many Jews are prominent on Wall Steet. Therefore people who blamed borrowers because of anti-black or anti-latino racism would be less likely to blame Jews in this particular framing because of, not despite, their racial prejudices. Of course, this implies that they are being at least somewhat consistent in their responses. One factor that encourages such consistency, and that was obscured in the article, is that the question about the Jews and the question about borrowers were both part of the same question. This matter might be clearer if Malhotra had made any attempt to measure other sorts of racial or ethnic prejudice than antisemitism, but he did not, at least not in those portions of the study that he has chosen to tell us about.

89

Martin Bento 06.04.09 at 7:13 am

One more point I forgot to make: he refers to the antisemitism he believes he has found as “not the first instance of an economic downturn sparking anti-Semitic sentiments”, making a causal connection between the economic downturn antisemitism he claims to have found. But unless he has done this survey both before and after the downturn or has other data he can cite (neither of which he has indicated, though again, he’s only telling us as much about his research as he wants ), he doesn’t even have a correlation, much less the basis for a causal inference. This doesn’t in itself undermine his other points, but it does show how far astray of ordinary academic cautions he has allowed himself to go in his argument.

90

Martin Bento 06.04.09 at 7:16 am

“between the economic downturn antisemitism he claims to have found”

should be

“between the economic downturn and the antisemitism he claims to have found”

91

Tim Wilkinson 06.04.09 at 8:44 am

MB: Will have a look yet but typed this out last night and didn’t post it (very pleasing/uinfuriating that CT doesn’t discard entered content when refreshing the page!)

MB @87: Yes, though Malhotra also ignored single questions too, and I wanted to get them all there at the top where they would definitely be seen – by NM and anyone else. Only the first 8 or so were requests for their secret info – the others more talking points – or at most nice-to-haves.

In fact NM only provided two other bits of hard info in the comments that hadn’t already gone up on the BR comments section – and both of those were deadbatting negatives (@43). The stuff in the main post was all either opinion or technical stuff that would obviously have been done by the book – like a phrenologist providing details of how accurate his calipers were.

I suppose the useful bits were some of the quotable gems about peer review, relevance v rigour, etc.

The only one I really regret not getting an answer to was the first one, i.e. can you confirm this was a single survey? I’m pretty sure there were other things said which implied that it was. If so, the whole ‘controlled experiment’ was highly suspect – methodologically I mean, of course. When you’ve just been led into claiming the Jews are moderately to blame for the financial crisis, then the subtle effect M&M were looking to tease out is that much more likely to emerge.

Also, the experiment used a real and well-known example instead of a constructed profile, which must have introduced a lot of noise. It might be worth looking at the paper they cite (Berinsky & Mendelberg – The Indirect Effects of Discredited Stereotypes in Judgments of Jewish Leaders) for an example of how a similar experiment is done when it is for publication – also for some background on the theories of associative networks they were working with, and indeed fixating on to the exclusion of other explanations.

It’s informative to see the lack of clarity and rigour in use of terms and the construction & testing of hypotheses, as well as the monocular approach to interpretation. Given that these are basically psychology experiments it was a bit of an eye-opener to me.

I should say I might be writing something about this myself elsewhere – though almost certainly not for profit and if so only nominal – but just thought I’d say before you post any summary, as I can hardly avoid incorporating some/all of it into whatever I do produce. I’ll almost certainly write up a short critical note on B&M too as I find it quite shocking really – and because they’ve actually written it up no detective work or conjecture is needed, which is nice.

92

Tim Wilkinson 06.04.09 at 9:43 am

Roy @86: I tend to agree – when you consider everyone on this thread is already primed by the context to see THE JEWS jumping out at them, it becomes a bit difficult to judge how someone who’s hurriedly completing question 47 in this week’s box-ticking exercise (if in doubt, tick the middle one) is going to view it.

On the subject of the weird world of arbitrary taboos, I’m sure I’ve heard, as well as ‘trailer trash’ which is grim enough, the term ‘white trash’ (i.e. not the standard kind of trash?) still in apparently normal use (I suppose it must have been on US TV).

Was the Burroughsian(?) imagery included to make the point that it’s the content that is the bigger challenge to comfort levels than the explicit spelling out of familiar ‘foul’ words (in any case largely divorced from their other literal meanings)? I agree, as my discarded lox-and-cream-cheese bagel bears witness.

OTOH, transgression of these conventions about vocab, if sufficiently prevalent, may tend just to cause inflation, e.g. the oedipal US variant which I haven’t quite got used to yet. And those who swear (cuss?) a lot for effect, rather than as punctuation per UK all-male demotic usage, are parasitically relying on those conventions, so rather less clever/iconoclastic than they think. Fucking cunts.

(Jamaican swearing uses bodily excreta rather than sex as the source of vocab, which seems healthier to me – though I personally don’t agree with shunning menstrual fluid as unclean.)

93

Martin Bento 06.04.09 at 3:41 pm

It occurs to me, re: #90, that NM probably does have “other data he could cite”, as there no doubt have been previous attempts to measure antisemitism in the US, and they very likely did not find it as high as Malhotra does. If one accepts Malhotra’s findings, then, one does have a temporal correlation, though still not nearly enough for a causal inference. Of course, contradiction of previous studies is also consistent with Molhotra simply being wrong.

94

Tim Wilkinson 06.04.09 at 10:45 pm

MB @93 – in fact whoever put that remark in (it was yet another on the ignore list) was pretty obviously not even trying (very hard) to make it sound like a scientific finding. For it to be one, there would need to be evidence from previous downturns as well. And it’s practically unthinkable that there would be a previous study that would be meaningfully comparable with this one.

Another thing I noticed a while ago and forgot to record – in one of the comments (I think), NM says that he was ‘shocked’ that so many were ‘willing to admit to’ (or something like that) their putatively antisemitic sentiment – suggesting among other things that he actually suspected that the result was to do with the way the questionnaire elicited the answer rather than the underlying level of such sentiment. Still, no point speculating about personal opinions I suppose.

95

Tim Wilkinson 06.04.09 at 11:54 pm

BTW I did indeed read your summary, which is useful and none of which I think I would disagree with. The few points I have will be negative I’m afraid, since there’s no point in reiterating point-by-point the fact that we are clearly largely in agreement.

On a relatively trivial point, I’d say put more paragraph breaks in! Other than that, I’d add the startlingly unhelpful remarks that I’d probably differ on some points of emphasis, and I think there are other things I’d want to put in, and some bits e.g. the Republican racism stuff that I wouldn’t go into in so much detail. I suppose the general point would be to avoid discussing particular alternative interpretations of the data that haven’t been ruled out, as that could get pretty open-ended, and stick to a more generalised method-focussed approach – the standard things such as tendency to choose middle options, priming effects of the first part on the second…

Again kind of thinking aloud about what my own approach might be, one possibility might be to start with what you might call the presentational side, taking a narrative approach – starting with the shock finding and then relating the process of gradual backpedalling in response to criticisms about the generally unscientific approach taken.

(Continues musing…) perhaps then looking at the survey from the point of view of the respondent to illustrate among other things what I consider probably the main points relating to the actual survey design; first the effect of a questionnaire format with a middle option, especially one described as ‘moderate’; and second the subliminal framing effect of forcing such an ‘I blame the Jews to degree (0,1,2,3,4)’ answer when it comes to the experimental part of the survey. I’m vaguely thinking along the lines of the rather clumsy illustrative dialogue above, but…not that, obviously, but

But to continue with half-formed ideas couched in over-general terms: I suppose in terms of structure, it’s possible to distinguish 1) the article itself, tone, lack of caveats, unbased assertions etc as well as lack of transparency and scrutiny, 2) the survey design and why the results can’t be assumed unbiased and 3) perhaps not really distinguishable from and straddling the other two, deficiencies in the interpretation of the results.

Well that was a load of rambling really. Sorry, it’s late. The link is very apt.

96

Martin Bento 06.05.09 at 1:07 am

The link is hilarious. Never seen those guys before. I forgot about the middle option bit; Malhotra conceded that point, so I should put it in. Will think over the other ideas.

97

Martin Bento 06.05.09 at 1:23 am

I see your point that alternative hypotheses could be a long list, though I don’t see the harm in putting in one. The problem I wanted to get at is that of positioning “blame borrowers” as a neutral position for those with racial prejudice, which it is not, given the media discourse that supports that conclusion. Also, it shows the significance of having “blame borrowers” and “blame The Jews” as options in the same question and treating one as a control on the other. To me, the wording of the article suggests two separate questions, just as it suggests two separate surveys, which you question.

Comments on this entry are closed.