Psephological Concussion

by Henry Farrell on March 31, 2004

One of the better indicators of statistical significance is the so-called “interocular trauma test.” It’s only satisfied when you have results that are so glaringly obvious that they hit you between the eyes. “Nasi Lemak,” a barely anonymous political scientist, uses pollingreport.com data on Bush’s approval ratings to come up with “two graphs”:http://nasilemak.blogspot.com/2004_03_28_nasilemak_archive.html#108066053879869096 that pass this test with flying colours. Of course, trends can change over time, but there sure looks to be something important happening here …

{ 47 comments }

1

Albert Law 03.31.04 at 1:29 am

Is Kerry’s as obvious?

2

StatBoi 03.31.04 at 1:29 am

You mean “glaringly obvious” in the sense that it’s glaringly obvious he’s using the old trick of starting the Y axis at a number well above zero to distort the scale of the graph? I’d suggest that “Lemak” might find this link useful, or maybe this one. Something tells me, however, that he’s already well acquainted with the material. He just doesn’t think anybody else is.

3

Matt McIrvin 03.31.04 at 1:53 am

Radio Free Monkey is way ahead of him, and also tries to have graphs with the axes in sensible places, though I think they could still use some refinement with regard to choices of colors and marker styles. Prof. Pollkatz had a similar set of charts, but they haven’t been updated in a while.

4

Henry 03.31.04 at 1:58 am

Statboi – you’re talking complete and utter smack. What Nasi Lemak has done is widely accepted as best practice – he’s focusing on a segment of the Y axis to show the range of variation. No distortion involved at all – it’s exactly the right way to present the trend that he’s interested in.

Don’t believe me? Let’s ask “Edward Tufte”:http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=00003q&topic_id=1, whose work you seem to think would be “useful” for Nasi Lemak.

bq. In general, in a time-series, use a baseline that shows the data not the zero point. If the zero point reasonably occurs in plotting the data, fine. But don’t spend a lot of empty vertical space trying to reach down to the zero point at the cost of hiding what is going on in the data line itself. (The book, How to Lie With Statistics, is wrong on this point.)

My advice to you: if you want to cast aspersions on someone else’s professionalism and honesty in future, it would probably be a good idea actually to read the authorities who you cite to. Otherwise, you’re sometimes going to find that they agree with him, rather than you. And that’s embarrassing.

5

Rv. Agnos 03.31.04 at 2:10 am

Is the moral we are supposed to learn that no matter how bad a president screws up, at least 45% of the people will still approve of him?

6

belaborer 03.31.04 at 2:18 am

The scale doesn’t mislead much does it? I don’t look at the graphs and think that Bush’s support is approaching zero – the numbers are right there. It’s the consistent trend down, interrupted by a few rally-round-the-chump moments, that draw the eye.

Anyway, I see why a terrorist attack might heighten one’s identification w/ their state, and sympathy with the victims in a kind of “but for the grace of God go I,” but why should a national tragedy necessarily boost support for the incumbent? If Bush’s post 9/11 performance was outstanding in some way, that’s one thing, but the swing in support seems to come before that could be known.

Are we children hiding under the covers wanting daddy to kill the monster under the bed? What if daddy’s an idiot whose monster-abatement strategy proposes setting the house on fire? IOW, could the bounce have gone to any warm body who currently occupied the White House? Clinton?

Or was there some kind of firm hand and steely resolve that comforted, hit the sweet spot of revenge and grace? I must’ve missed that.

7

derPlau 03.31.04 at 2:49 am

statboi: Yes, not including zero on your Y-axis is a classic way of presenting data in a misleading way. However, it’s not always misleading — and this is one of the cases where it isn’t.

In general, the apppropriate range for the Y-axis depends on what comparison is being made. Bar graphs comparing the effectiveness of one product to another, for example, can mislead customers into thinking Toothpaste A is twice as effective as Toothpaste B, if A is 60% effective and B is 55% effective, and the Y-axis starts at 50%. But note that this is misleading because people are drawing conclusions about a propoprtional difference between the two products — in this case, zero has a special meaning.

In the case of Bush’s approval ratings, zero doesn’t really mean anything special. What we’re interested in is the trend over time, in essence making the approval rating at time zero the reference point — the real zero. The question is, “is there a consistent downward trend in Bush’s numbers” — not, “are Bush’s numbers 90% lower than they were on 12 September 01”. The real question is one of absolute difference over time, not proportional difference.

8

JK 03.31.04 at 4:17 am

How come nobody has thought to ask what the normal pattern is over a presidential first term?

Duh.

Bush’s performance would ‘hit you between the eyes’ rather differently in that (appropriate) context. Look it up.

9

DonBoy 03.31.04 at 4:45 am

C’mon, jk — you’ve made the insinuation, why don’t you look it up and show us? Don’t forget to list separately the data for presidents who were re-elected vs. those who weren’t.

10

Statboi 03.31.04 at 4:53 am

Tufte says:

But don’t spend a lot of empty vertical space trying to reach down to the zero point at the cost of hiding what is going on in the data line itself.

So? That doesn’t mean that removing vertical space in order to hide what’s going on is perfectly okay.

11

Matt McIrvin 03.31.04 at 5:16 am

jk, here’s a comparison of all presidents since Carter, plus Nixon. In absolute approval rating Bush is not doing so badly right now, but the trend is not so much stronger than those of his dad (coming down off the first war with Iraq) and Jimmy Carter (coming down off a long-forgotten spike at the very beginning of the Iran hostage crisis). You can see the comparison.

Bear in mind that, despite the beating he took through most of the campaign, George H. W. Bush didn’t do so badly, had a late surge, and could have won had it come a little earlier. These things aren’t set in stone, and Bush could win by default if Kerry isn’t a strong enough candidate. But it sure doesn’t look like a blowout. At this point, the winners on the chart were all trending steadily up.

12

ArchPundit 03.31.04 at 5:20 am

P 93 The Elements of Graphing Data by Cleveland

“Huff’s presumption is that viewers will not look at tick mark labels and apply the most trivial of quantitative reasonsing.”

“For graphical communication in science and technology assume the viewer will look at the tick mark labels and understand them. Were we not able to make this assumption, graphical communication would be far less useful. If zero can be included on a scale without wasting undue space, then it is reasonable to include it, but never at the expense of resolution.”

Or, ,more simply, get a clue statboi. Wrong audience with which to whine about your poor understanding of graphing data.

13

Matt McIrvin 03.31.04 at 5:22 am

By the way, that chart also enables one to state with authority that one Watergate is equivalent to four and a half blowjobs.

14

neil 03.31.04 at 5:30 am

To be fair to statboi, the zero point -is- relevant in this graph. Consider if you took the subset of the data between mid-September ’01 and the middle of ’02, and drew that graph with 90% and 70% as the boundaries. What you’d get is a picture approximately like the one linked to, showing a precipitous drop. And yet as this graph correctly shows, mid-2002 was actually a pretty strong point for Bush.

The thing that statboi misses, I think, is that there IS a zero point on this graph, even if it’s not marked with a zero. At 50% approval, Bush has a 0% net positive approval rating. Above that point, he is popular with a majority of the population; below that point, he is unpopular with a majority. It would’ve been misleading to put 70% at the bottom of the graph, because (as we can see) you can go a lot lower than 70% and still be popular. But you can’t go lower than 50% and still be popular. Easy.

If you took this graph and rescaled it all the way down to zero, you’d actually be making it -more- misleading, by making it appear that Bush had that much farther to drop before reaching a crossover point. Not so: the crossover point has been reached already, as the graph shows.

15

derPlau 03.31.04 at 5:54 am

Come to think of it, how come nobody ever complains that figures like this don’t go all the way up to 100%? If viewers are stupid enough not to look at the actual numbers on the Y-axis, aren’t they going to be just as misled by an apparent proximity at the top as they are at the bottom? Obviously in many cases there isn’t a finite boundary at the top – but in the case of percentage data, the Huff/statboi argument logically requires that all figures go all the way from 0 to 100.

16

JK 03.31.04 at 5:57 am

Thanks for the chart, Matt. I read this data as follows:

(i) Every president was in the low to mid fifties at the three year point. Even Carter managed to bounce there for a few months.

(ii) If you were higher than the low to mid fifties 18 months into your first term, you got to the level described in (i) by declining. If you were lower than that 18 months into your first term you got there by improving. No surprises here.

(iii) After the three year point the candidates diverged. Those who would go on to win climbed slowly during their fourth year in office to around 60%. Those who would go on to lose had fallen disastrously by the end of the first quarter of their fourth year to 40%.

(iv) Bush is, per this data, still looking distinctly more like a winner than a loser.

17

derPlau 03.31.04 at 6:28 am

jk, you seem to have missed a couple of correlations:

(v) presidents who won their second election tended to get to the mid-50’s at the end of their third year by moving up in the polls (or, in Nixon’s case, at least not moving down)

(vi) presidents who lost tended to get there by moving down.

18

ArchPundit 03.31.04 at 7:44 am

The real question that is important to answer in any statistical analysis whether interocular or not is whether the future or present is different than the past. It could well be that something has fundamentally changed about how we evaluate Presidents. I don’t think so, but it is interesting to not the GOP started making this argument when Bush went below 70.

19

dsquared 03.31.04 at 8:56 am

God knows I’ve had more than my share of harsh words for the Iowa Electronic Markets over time. But I think I prefer even them to the project of trying to do technical analysis on charts of poll data.

The interesting question to ask here is whether the most recent Gallup poll (the green dot at the right hand edge that puts Bush on a much more healthy rating) is noise or signal.

IEM seems to reckon it’s signal; Bush has been bid up substantially relative to Kerry over the last two weeks.

(btw, to join the pile-on on statboi, 50% is obviously the only sensible place to put a zero point on a chart of poll data, as it’s the point at which you either win or lose a bloody election!)

20

dsquared 03.31.04 at 8:58 am

Second thoughts: I suspect that the IEM numbers (which are for the vote-share market because the winner-takes-all market hasn’t opened yet) are picking up the effect of Nader entering the race rather than implicit movements in the Kerry/Bush share.

21

dave heasman 03.31.04 at 9:11 am

“It could well be that something has fundamentally changed “.

I don’t think that the chairman of the company manufacturing voting machines has declared that he wants to ensure the reelection of the current president in any previous contests.

22

Dave F 03.31.04 at 10:05 am

I saw that Iowa comparison chart (at Instapundit!). I know little of psephological analysis, but would Reynolds’s conclusion (that Bush gathered strength from the point where Kerry emerged as the likely nominee) be reasonable?

If so, prhaps this indicates partly that the election may well be decided by how the electorate feels about its security in the light of the performance of the incumbent and the stated strategy of the challenger in the “war on terror”.

23

biff3000 03.31.04 at 10:46 am

dquared: 50% is obviously the only sensible place to put a zero point on a chart of poll data, as it?s the point at which you either win or lose a bloody election!)

Sadly, not in American presidential elections….

24

Nasi Lemak 03.31.04 at 12:12 pm

Oooh, fame (of the barely anonymous sort). Anyhow, I didn’t want to post any interpretation of the graph in the original postings, but here goes: the key thing here seems to me *not* the absolute level that the approval ratings have reached, but rather that this administration has never had, at any point, steady increases in approval, nor has it had broadly stable approval ratings except perhaps at the very beginning. The ratings will, surely, at least stabilise a bit, but I’ve been saying that since the mean was about 65% and it hasn’t happened yet.

25

Ulex Gantry 03.31.04 at 1:00 pm

WE should form a brown shirt party and go around atacking male St Bernards who want to marrry, or something ridiculous.

OOPs! Just a thought.

26

Ulex Gantry 03.31.04 at 1:00 pm

WE should form a brown shirt party and go around atacking male St Bernards who want to marrry, or something ridiculous.

OOPs! Just a thought.

27

Ulex Gantry 03.31.04 at 1:00 pm

WE should form a brown shirt party and go around atacking male St Bernards who want to marrry, or something ridiculous.

OOPs! Just a thought.

28

Ulex Gantry 03.31.04 at 1:00 pm

WE should form a brown shirt party and go around atacking male St Bernards who want to marrry, or something ridiculous.

OOPs! Just a thought.

29

harry 03.31.04 at 1:28 pm

Just to point out that Clinton won, twice, with considerably less than 50%. His races were exceptional because of a strong third party candidate. How should his ratings be interpreted? (sincere question, I’ve no idea).

30

Del C 03.31.04 at 1:36 pm

One other remark for statboi. Tufte, in his classic _The Visual Display of Quantitative Information_, does disapprove of crossing the ordinate of a time series graph at a point other than the zero, because this can lead to confusing the crossing point with a zero point.

However, his proposed solution is not to include tick marks all the way down to the zero (apart from anything else, this would be mathematically impossible where the ordinate is a logarithmic scale).

Instead, he recommends truncating the axes before they have an opportunity to cross, leaving them floating in space below and to the left of the data, respectively. This conveys the correct impression while not wasting space.

Sadly, many graphing utilities (such as the ubiquitous MS Excel) are not well equipped to do this, but that’s not the original poster’s fault. statboi is not only making a fuss about a small matter, but he’s making it about the wrong small matter.

31

JamesW 03.31.04 at 1:44 pm

I noticed Nasi Lemak’s charts use the sensible European date format of dd/mm/yy rather than the idiosyncratic, penny-nails US standard of mm/dd/yy. Is this a litmus test for telling craven frog-lovers from red-blooded Americans?
Oddly, the pragmatic Swedes use a rigorous Cartesian system that goes yy/mm/dd(/hh/ss/….). But it hasn’t cottoned on.

32

Nat Whilk 03.31.04 at 2:31 pm

FWIW, the Executive Director of Democrats Abroad reports: “In January 1980 Carter led Reagan by 62-33% but lost in November by 10 points. In October 1983, Mondale led Reagan by 50-44% but lost 59 to 41%.” I envy those who have public sentiment figured out.

33

DJW 03.31.04 at 4:06 pm

Statboi’s complaint makes sense, as long as we assume that Bush’s performance has been so bad that it’s rather surprising his approval rating aren’t hovering around zero. What I see him so close to the bottom of the graph, I think, “great, everyone’s finally figured it out, and…oh, wait, that’s 47%?! What the…”

34

Nat Whilk 03.31.04 at 4:55 pm

To tie this into the thread on the “Book of Revelations(sic)”, polls suggest that about 2/3 of the American people believe in the Second Coming of Jesus.

35

Ed Zeppelin 03.31.04 at 5:16 pm

Just to point out that Clinton won, twice, with considerably less than 50%.

Clinton got over 50% in 1996. Perot was significantly weaker that time out (can’t recall the exact percentages, but I know Perot came nowhere near his 1992 showing).

IN 1992, though, Clinton got 43%, Bush I got 38%, and Perot got 19% (but no actual electoral votes, IIRC).

36

Nat Whilk 03.31.04 at 6:18 pm

Ed Zeppelin wrote:

“Clinton got over 50% in 1996.”

Nope. 49.24%.

37

Matt Weiner 03.31.04 at 6:58 pm

I’ll presumptuously say some stuff about Harry’s question: Clinton’s results show that in a three-way race a candidate can poll considerably lower than his approval ratings. GHWB was polling between 40% and 50% approval in 1992, and didn’t clear 40; in ’96, Clinton looks to have been polling around 58 on election day and didn’t make 50. This is more or less what you’d expect, I’d guess, when you have two choices in the approval poll and three choices in the vote.
I think the 50% mark is still a natural place to look for the baseline for approval; Nat Whilk is right about reading the future, although I wishfully note that Nixon and Reagan (and Clinton!) were trending up at the time of those polls.

38

Jim Miller 03.31.04 at 9:03 pm

If you are interested in what I would call a “fundamentals” approach to the question, rather than arguing about graphs, take a look at this post
which uses standard political science arguments to predict that — if the American economy grows as most economists predict — Bush will defeat Kerry by 59-41 in the two party vote.

I’ll be updating these predictions at least once a month, by the way.

Note that I made the prediction early in March, before Bush retook the lead.

I glanced at “Nasi Lemak”‘s chart and the discussion accompanying it, but saw no insights from political science. But then I’ve been out of the field for many years, so perhaps I missed something.

Those interested in my previous predictions can look in my November 2002 archives.

39

ArchPundit 04.01.04 at 8:22 am

Jim, if you are going to claim your prediction is based on political science, why not base it on the many political science models out there that have actually operationalized the variables you mention? I mean, you could start with Lewis-Beck and Rice from 1992 given they have summaries of many models and then pick up the modifications over the years.

40

Jim Miller 04.01.04 at 2:39 pm

Archpundit – That’s a fair question. Briefly, I am skeptical about the thinking behind the models, because they appear to have been constructed by experimenting to see which measures give the best fit. Unfortunately, that will, if you fiddle enough and have small data sets, always appear to fit the data well.

As for making my own argument more mathematical, I could do that but don’t think it worthwhile, given the poor measures of some of the important variables and small number of cases.

I have done longer critiques of those models in the past and will probably do one again at my site before the election. For those who want a hint, just ask yourself how many elections the models are tested against, 10 maybe?

41

Pedro Sans Pedro 04.02.04 at 5:01 am

belaborer:
A war deflects the populations ire away from the president towards The Enemy…?

42

Zizka 04.02.04 at 5:23 am

I think that this election will be decided by events occuring between now and the election. Both campaign events and events in Iraq and elsewhere. Trying to figure things out by measuring trends is delusory. (Yes, I mean you!) However, this graph does show me that Bush is vulnerable.

A friend of mine thinks that Osama will be captured at the appropriate time before the election. If Bush manipulates events successfully, he will probably win. If things swing out of his control, he won’t.

My feeling is that Bush-Rove look at excess approval ratings as uninvested capital. If Bush popped above 60% he’d probably do something unpopular on his agenda to bring things down to 52%, which is where he wants to be.

This thread does exemplify CT comments’ two worst traits: troll-driven threads, and second-order methodological wonk discussions. Let’s give a round of applause to statboi for her sterling red herring.

43

Zizka 04.02.04 at 5:23 am

I think that this election will be decided by events occuring between now and the election. Both campaign events and events in Iraq and elsewhere. Trying to figure things out by measuring trends is delusory. (Yes, I mean you!) However, this graph does show me that Bush is vulnerable.

A friend of mine thinks that Osama will be captured at the appropriate time before the election. If Bush manipulates events successfully, he will probably win. If things swing out of his control, he won’t.

My feeling is that Bush-Rove look at excess approval ratings as uninvested capital. If Bush popped above 60% he’d probably do something unpopular on his agenda to bring things down to 52%, which is where he wants to be.

This thread does exemplify CT comments’ two worst traits: troll-driven threads, and second-order methodological wonk discussions. Let’s give a round of applause to statboi for her sterling red herring.

44

Zizka 04.02.04 at 5:25 am

I think that this election will be decided by events occuring between now and the election. Both campaign events and events in Iraq and elsewhere. Trying to figure things out by measuring trends is delusory. (Yes, I mean you!) However, this graph does show me that Bush is vulnerable.

A friend of mine thinks that Osama will be captured at the appropriate time before the election. If Bush manipulates events successfully, he will probably win. If things swing out of his control, he won’t.

My feeling is that Bush-Rove look at excess approval ratings as uninvested capital. If Bush popped above 60% he’d probably do something unpopular on his agenda to bring things down to 52%, which is where he wants to be.

This thread does exemplify CT comments’ two worst traits: troll-driven threads, and second-order methodological wonk discussions. Let’s give a round of applause to statboi for her sterling red herring.

45

Zizka 04.02.04 at 5:33 am

Sorry, dialup problem.

But really, I’d like to reiterate how pointless I think that the statistical projections are. MY blog partner Dave Johnson looked at the spikes in the graph (all military-oriennted) and pointed out that Bush needs an event in late October or early November. He may guess wrong as to the right kind of event, but he certainly will make sure that there is an event.

Comparable events/non-events probably decided the 1980 and possibly the 1968 elections. Carter couldn’t get the hostages home, Johnson couldn’t get anything done in Vietnam. In both cases, I believe that the Republicans cut deals (with Khomeini and Diem).

46

Joshua W. Burton 04.02.04 at 2:36 pm

I think everyone is missing the point here by trying to project forward to 2004-11-02. If the numbers tell us anything, it’s that this administration’s future poll numbers are _less_ predictable than those of recent ex-presidents.

Why? Because, as Nasi Lemak observes above, the data do not probe GWB’s stable “baseline” support at all; they only define a lower bound for it at 50%. What they show is that (1) running away on 9/11, starting the wrong war, and shaving the Iraqi butcher are each worth a discontinuous boost, and (2) the Shrub has been losing support at 2% a month (comparable to his father’s slide, and, barring Watergate, about as fast as polls can fall) for _all the months he’s been in office_ except the initial honeymoon and the rally-round-the-flag spikes.

What these numbers must be telling Karl Rove (who is surely reading them without hazy Tuftean digression) is that his man has _only one proven way to win public approval_. In other words, that the crucial election-day number can be swayed _only_ by one more national wake-up, and that the November extrapolation without one is in the unplumbed upper 30s. This is _emphatically_ not a line of reasoning I want haunting the corridors of power through a long hot foreign-policy summer.

47

Joshua W. Burton 04.02.04 at 2:41 pm

“…they only define a lower bound for it at 50%.”

An upper bound, of course.

Comments on this entry are closed.