What’s wrong with happiness measurement ?

The reason we can say that “…in reality, height does increase with (absolute) age” is that we can, in fact, measure height in absolute terms. Happiness we can’t measure.

What we should do, then, is bring the average family from the 60s to the present (in a thought experiment), let them hang out here for a few years and then ask them to rate their happiness again.

To maybe make you feel happier, see where you rate on the world richness list.

http://www.globalrichlist.com/

“And of course, if you really want to go the revealed preference route, there’s migration.”

Which would tend to make the US a rather happier place than Oliver James seems to think it is in Affluenza.

John,

My intuition is that the idea of comparative scale renorming (what you seem to be arguing for) would suggest that we should see fairly static distributions of happiness over time (if people are reporting by just fitting themselves into a spot in the distribution of their peers). From what I know, that doesn’t seem to be the case, at least not everywhere. If we were to see fairly static distributions conditional on income, but changing distributions overall, then would that not provide support for the relative income hypothesis? (I should probably work this out more formally, but figured I might as well put it out there as a hunch.)

Similarly, the data doesn’t seem entirely consistent with a view of adaptive scale renorming either: the idea individuals adjust their reporting scale in response to their own past happiness levels. (I don’t think that’s the view you’re advocating, but it has been pushed elsewhere.) This would suggest that we should adapt (proportionately) to everything, not just money. But that too is inconsistent with the survey findings.

The point is that, if you’re willing to make plausible assumptions about the form that any scale renorming takes, that may allow you to identify “real” changes from the data, or at least establish bounds on such changes, or do some sort ofsensitivity analysis.

More generally, it’s possible to think that happiness surveys provide support for diminishing marginal utility, relative income, and/or adaptation without being convinced that prove any of those things. It just depends on what you think is a reasonable prior about the inherent relativity of survey responses. (And indeed, if you believe some of those claims on independent grounds, then you may also think that this provides some evidence for the survey responses being at least partly non-relative.)

The analogy strikes me as misleading. While people may rely somewhat on comparisons with others when answering ‘How happy are you, on a scale from 1 to 10?’, they do so much less than when answering ‘How tall are you, on a scale from 1 to 10?’. The latter is an inherently comparative question in a way that the second isn’t, since there’s no non-comparative concept of tallness. But people do have some conception of an absolute scale of happiness and some at least rough idea of where they fall on it. Would John suggest that if you asked the happiness question of people in a concentration camp you’d still get a median answer of 7?

tom hurka, wouldn’t that be a bit like asking the height question in a special summer camp for short kids or something? You wouldn’t get a high average there either. I can’t see that tallness and happiness differ in this regard.

There’s another problem you haven’t mentioned, namely, that what people say on surveys about happiness is bound to be heavily influenced by cultural norms. For instance, Americans think we are supposed to be happy, we think we are responsible for our own happiness, and so we think we are reporting failure if we report unhappiness. Whereas French people think that being very happy demonstrates that you are severely out of touch with reality.

(Pardon the cultural stereotypes.)

Once I was advising a nurse on a study of people who had suffered traumatic injuries; she mentioned a happiness instrument. I said that there’d probably be a problem with the distribution of scores, that they should clump at the lower end. She stated that this would not be a problem, that happiness scores rebounded to where they were, 6-12 months after the injury/onset of illness.

That made me very skeptical of that instrument.

Happiness is subjective and based both on experience and emotions that are sometimes coincidental vary in intensity based on componenents of recent and distant past. Unrelated experiences can change the intensity and make up of emotions that different events cause. Frequency, intensity, and variation are important in determining how events affect us emotionally.

Wouldn’t it make more sense to ask the teachers?

“wrongly think it is the city or statem”

What would be wrong about thinking that there are better places than Flint, Michigan?

Pablo,

What notion of subjective well-being are the devices you’re talking about intended to capture? As has already been noted, “happiness” can be distinguished from “life-satisfaction” and other subjective constructs. My impression was that they were geared at the former, but it’s probably the latter I’m more interested in.

Conchis,

You are right that these devices would provide us with information about ‘happiness’ rather than ‘life-satisfaction’; they would measure how people have felt across time rather than how people presently think they felt. But I count this as an added benefit to the approach I’m advocating. Retrospective assessments of subjective well-being are subject to a number of biases, introduced by the need to retrieve and aggregate past experience. Direct, continuous measurements of brain activity would overcome these problems as well, in addition to the difficulties John noted.

Pablo,

I’m certainly not dismissing the usefulness of the devices entirely, but I’m also very much a believer in Tukey’s dictum “better an approximate answer to the right question … than the exact answer to the wrong question”. For that reason, I’m inclined to see progress on these sorts of measurement more as a small instance of the sort of progress that will be needed on a lot of fronts, than the massive victory some people seem to think it is.

Maybe instead of questioning a happiness instrument that shows that people with physical disabilities (traumatic injuries, amputations, etc.) tend to be roughly as happy as everyone else (after they adapt to their current condition), you should question your own beliefs about disability. Why should we give more weight to the view of a nondisabled person about how disabled people “should” feel than to the experiences of people with disabilities?

Not really on-topic, but US suicide statistics are pretty interesting. The lowest rates are in liberal Democratic states, with Massachusetts and New York at the very bottom. The highest rates are all in western states, except for West Virginia.

http://www.suicidology.org/associations/1045/files/2004statedatapgv1.pdf

Good point Pablo, and here’s a better version of mine, minus dubious claims about inherently comparative concepts.

The issue is how people asked the 1-to-10 question fix what the 10 is. (I assume there are non-arbitrary zeroes, having no height or maybe that of a newborn baby in the one case, being at the neutral point between happiness and suffering in the other.)

John assumes that in the height case people will equate the 10 with something like the tallest height in their local community, i.e. class. They can’t equate it with the tallest height humans in general are capable of (7 feet 6?), because then the different classes would have different medians. And he must therefore assume that in the happiness case they’ll likewise equate the 10 with something like the maximum in their local group.

But there’s another possibility. Unlike height, happiness fluctuates. You have moments of intense happiness and other moments of misery. And an initial understanding of the relevantly greatest happiness is a state in which the most intense happiness you’ve experienced is present continuously, i.e. you’re always at that level. That understanding can be extended. You might think you haven’t yet experienced the most happiness you can; your peak is somewhat higher. Or looking at other people’s expressions of ecstasy you may think the general human peak is higher than you’ve yet experienced. But those are just variations.

The basic idea is that you can identify the 10 with something like your own happiest moments carried on continuously. And you can then ask you close your average level of happiness comes to that peak. If it’s close, because you’re most of the time very happy, you’re an 8; if you’re often miserable and only rarely happy, it may be a 2. But at no point are comparisons with others essential. You just ask how close your life, on the whole, comes to the maximum as understood based on your experience.

I doubt that peak moments of happiness have much to do with money. They involve love, or achievement, or rest after strenuous, rewarding work. So the peak can remain constant even though GDP per person changes, as from the 1950s to today. If people now think their average level of happiness is no closer to that peak than in the 1950s, that’s informative.

This isn’t to say there aren’t lots of problems with these subjective assessments, just that there isn’t the specific problem suggested by John’s analogy. A key reason, I’m suggesting, is that levels of happiness vary, so we can identify the relevant maximum as what our lives would be like if they were always at that maximum. But heights don’t vary, or at least John’s assuming that the way they do isn’t used in assessing height in his classroom analogy.

So as I’ve said, the analogy is unpersuasive. But arguments just by analogy are always tricky — though tempting — things.

#34 – using state data is tricky. The major correlation seems to be rural v. urban states. But you would really need much more fine-grained data to draw conclusions.

I second slocum’s reccommendation of Bounded vs. Unbounded. Essentially it questions how you can compare bounded statistics (happiness on a 1-10 scale) with unbounded statistics (say income) and hope to get useful information out of them over any long period of time.

eudoxis: the alternative explanation of the Kahneman et al. result is that perhaps satisfaction is just something different from an aggregation of moment to moment good mood.

I’m somewhat sympathetic to the focusing illusion argument. But the Science article doesn’t actually provide much evidence for it.

that happiness scores rebounded to where they were, 6-12 months after the injury/onset of illness. … made me very skeptical of that instrument.

I’ve seen my dearest friend (who has multiple sclerosis) go from being able to walk a block on crutches to being able to move only her head, left arm, and left thumb. I won’t say she’s as happy now as she was then (at some point the effects of disability on happiness presumably start to resemble those of low income), but she’s happier than many folks I know – thanks in part to her own indomitable spirit and in part to TV, the Internet and iTunes.

Late to the discussion and the most important points have already been made by conchis, John and others. One thing to note though is that most of these studies done in an international context run into translation problems. And I don’t mean just faulty translations, which I think are somewhat of a problem but something usually the studies try to limit, but also cultural translation.
The word “happy” or “satisfied” may not mean the same thing language to language, culture to culture.

For example, suppose that some Cimmerians decided to conduct a happiness survey, came to America and went around asking people “How merry are you?”, or, worse, “How gay are you?”. You gonna get some serious bias.

For another concrete example, I’ve seen surveys in Polish which translated “How happy are you?” into “How lucky are you?” the words for happiness and luck being pretty close (or even identical) in Slavic languages.

Yes, I know that this is something that the folks who carry out these studies try to address but I’ve never been really convinced that they do so succesfully.

This discussion reminds me of my experience with appendicitis last year. At the hospital both before and after the appendectomy I was asked to rate my level of pain on a 1-10 scale. I remember being totally baffled and the nurses getting increasingly impatient as I tried to mull out how to assign a number to that. Is 1 simply no pain? Or is 1 an experience of total pleasure? I finally decided that 10 represented a pain so intense that it would cause major psychological trauma, so I went with about a 5 even though I was in a considerable amount of pain, and probably analyzed myself out of some much appreciated demerol.

There was even a helpful chart hung on the wall that helped normalize the scale versus facial expressions, so 1 was a blue smily face, 5 had a sad green face, and red was a crying, very sad face. Of course, this simply implies that more stoic people experience less pain, which strikes me as dubious and unhelpful.

Sebastian: Comparing bounded and unbounded statistics is a well-understood problem in econometrics.

But in reality, height does increase with (absolute) age and the problem is with the scaling of the question. A question of this kind can only give relative answers.

Hmm. You mean in principle or just in practice?

In principle, this must be false. Clearly you could give people much more rigorous survey guidelines (e.g. “a ‘7’ means that you experience blah blah at least 20 times a day…”), such that the measurements were accurate and absolute, not relative.

In practice, that’s hard. Surely if you just say “pick a number!” without much explanation, who knows what you’ll get.

You are putting your finger on a problem, but the problem is this:

1) We don’t know how people interpret the survey question.
2) So, we don’t know what their criteria are in answering it.
3) In particular, we don’t know whether they answer using relative or absolute criteria.

You are positing that they answer using relative criteria, which is indeed a problem because it means that people in different groups do not have comparable scores.

The problem is not with the 1 to 10 scale, though. Those are just numerical endpoints, you can of course find a bijection between [1,10] and any other 1-dimensional numerical scale you wanted to use. It may be necessary to map ’10’ to infinity. Oh well, you throw out a value.

Of course, we may someday find out that 1 to 10 is a little silly. That depends on whether we find something we really think is a great thing to measure and call ‘happiness’ (like, the percentage of time your left PFC activity is greater than right, or vice versa, I forget). Chances are if we find some actual thing we want to call the numerical measurement of happiness, the 1-10 scale will not be the most natural scale for it.

The problem with 1 to 10 is not the numbers, or the boundedness of the numbers, but rather the lack of explanation on how to give yourself a number.

If you accept Runcimans concept of Relative Deprivation developed in Britain in the late sixties, basically that as wealth rises people continually raise the bar below which they feel deprived, then I cannot see why you would have a problem with Relative happiness. Yes we have more holidays, free time and material goods but most of this is what we expect. So we don’t rate ourselves as extremely happy. Or overly satisfied.

It is also worth mentioning that the 5, 7, 9, here are not really numbers. They’re categories, akin to “very happy” or “moderately happy” or what have you.

In that sense, even if the criteria were clear and non-relative, the “numbers” may obscure more than they reveal. But if you set the survey up correctly, it may still be useful to help determine whether happiness increases with income, etc.

(A methodological problem with the “video” example above is that respondents may have their own personal relative rankings of how happy all the people in the video are! So you might dictate that a “7” means as happy as this guy, but someone might think that the person dictated as a “6” actually looked more happy. You could try to work around this maybe, by having them perform the relative rankings of the 10 people themselves…)

“Sebastian: Comparing bounded and unbounded statistics is a well-understood problem in econometrics.”

Oh thank GOD you cleared that up! Want to clue us in on the understanding?

On revealed preference, won’t this fail to answer the important question? It’s probably fairly well accepted that most people believe that greater income means more happiness, but if what we want to know is if they’re correct on this, then looking at whether or not people actually pursue greater incomes isn’t going to help.

Recall Mae West’s famous line people “Are you happy to see me or is that a pickle in your pocket?” and take your cue that this has nothing to do with numbers.
You are happy or you’re not.
Don’t give me this waffling nonsense about 6 out of 10 –that’s a ‘no’. Bone up and confess you are in the doldrums just trying to fashion a response. Now you may think that Mae’s “happiness” is so different from the economist’s, but you, unhappy buzzard, would be wrong.

Is it transient? [only an idiot is happy all the time, yes?] Can your case be made more or less transient? [Can Mae make a difference? can she!]
And we are going to entertain the notion of measuring this?
Not me. And not Mae. What about you, econometrician (stick-in-the-mud)?
This elusive but well worth pursuing Happy person is someone who is motivated, inspired and an inspiration to others. She is both your best citizen and your best worker. So policies that make more people happy, more often for longer periods of time are worth pursuing. We can tell (like Mae) when you’re happy, but measuring it in the aggregate…I don’t think so.