Improving your health by pursuing meaning in your life versus happiness

pnasjpgA recent article in the prestigious Proceedings of the National Academy of Sciences(PNAS) claims to explore genomic aspects of the pursuit of meaning versus happiness. The considerable press coverage the article generated accepted it provided a scientific basis for resolving a classical philosophical question: should we pursue meaning ( termed “eudaimonism”) in our lives or happiness (“hedonism”)?

As the Atlantic Monthly notes

The terms hedonism and eudaimonism bring to mind the great philosophical debate, which has shaped Western civilization for over 2,000 years, about the nature of the good life. Does happiness lie in feeling good, as hedonists think, or in doing and being good, as Aristotle and his intellectual descendants, the virtue ethicists, think? From the evidence of this study, it seems that feeling good is not enough. People need meaning to thrive. In the words of Carl Jung, “The least of things with a meaning is worth more in life than the greatest of things without it.” Jung’s wisdom certainly seems to apply to our bodies, if not also to our220px-Sanzio_01_Plato_Aristotle hearts and our minds.

The title of the Atlantic Monthly article announces the resolution this research offers

Meaning Is Healthier Than Happiness

And the header further declares

People who are happy but have little-to-no sense of meaning in their lives have the same gene expression patterns as people who are enduring chronic adversity.

If, as an Academic Editor for PLOS One I had received this article as a manuscript, I would probably have recommended Rejection without sending it out for further review. But if I had sent the manuscript out for review, I would have chosen at least some reviewers with relevant psychometric backgrounds.  I would be disappointed if they did not immediately notice some fatal flaws hiding in plain sight.

The article is highly technical, with basic details presented in unnecessarily ponderous sentences:

Primary analyses examined the relationships of hedonic and eudaimonic well-being to expression of a 53-gene contrast score summarizing three a priori-defined components of the CTRA profile (12, 33–35): up-regulated expression of proinflammatory genes, down-regulated expression of genes mediating type I IFN responses, and down-regulated expression of genes involved in antibody synthesis.

There are many such sentences, some even more complex.

I doubt that many people who have an opinion about the article have actually read it in its entirety. But with such a formidably complex article making such strong claims, it is all the more important to start with checking basic statistics and measurement issues. Problems often start in these details that preclude important questions being addressed. Think of the aliens invading Earth in the War of the Worlds being brought down by common microbes.

In the case of this article, the validity of sophisticated methodologies, analyses, and interpretations depend entirely on some distinctions being captured in self-report assessments of hedonic versus eudaimonic well-being with the Short Flourishing Scale. What we can say about all the complicated biomedical assessments is limited by what we can say about some very simple questions delivered by Internet So, I started by examining the correlation between these two subscales and it was r = 0.79, ( p < 0.0001).

This correlation is as high as the reliability of these two subscales allows, meaning they are essentially interchangeable and measure the same thing. Except for by random variance, that finding is highly unlikely that something will be related to one variable and not the other.

The high correlation between these variables is reflected in shared associations with other variables:

Analyses found both forms of well-being to show similarly strong inverse relationships to symptoms of depression [Center for Epidemiological Studies–Depression (CES-D) correlation with hedonic well-being, r = −0.67, p < 0.0001; correlation with eudaimonic well-being, r = −0.66, p< 0.0001; difference in dependent correlations, p = 0.8550]. Similarly strong inverse relationships were also observed for CES-D subscales assessing affective symptoms of depression (hedonic, r = −0.75, P < 0.001; eudaimonic, r = −0.71, P < 0.001; difference, P = 0.3228) and vegetative symptoms of depression (hedonic, r = −0.45, P < 0.001; eudaimonic, r = −0.48, P < 0.001; difference, P = 0.6297).

The investigators attempted to escape their problems by introducing statistical controls:

each well-being dimension treated as a continuous measure and adjusted for correlation with the other dimension of well-being and for age, sex, race/ethnicity, body mass index (BMI), smoking, alcohol consumption, recent minor illness symptoms, and leukocyte subset prevalence.

But given the close association between the two variables, what is portrayed in the multivariate analyses, eudaimonic-well-being-controlling-for-hedonic-well-being-and-many-other-things is very different than eudaimonic well-being without such controls. If we were talking about people, we probably couldn’t even recognize a family resemblance between the two. Shared similarities were removed, so one would not be recognizable from a photo of the other.

This represents statistical malpractice and we are well on our way to nonsense. Any divergent associations with genomic transcriptome profiles are likely to be artificial and not replicated in future studies. The investigators’ particular choice of variables to control was arbitrary. There was no particular reason given for this combination. Other variables were simply left out. Other choices of what to control would lead to other results. But this is not a meaningful exercise for another reason: there are too many variables being controlled with too few research participants.

And more generally, we cannot just dump possible confounding variables into an equation expect to find anything meaningful

Statistical adjustment by an excessive number of variables or parameters, uninformed by substantive knowledge (e.g. lacking coherence with biologic, clinical, epidemiological, or social knowledge)…can obscure a true effect or create an apparent effect when none exists.

Faced with such highly correlated measures of the crucial variables of hedonic versus eudaimonic well-being, what could the investigators possibly have done to remedy matters and proceed? I don’t think that there is anything. It should have been mission aborted.

When I encounter such high correlations, I look to the items that went into the construction of scales, deliberately ignoring the labeling of the scales. In the case of these subscales:

Participants completed online assessments of hedonic and eudaimonic well-being [Short Flourishing Scale, e.g., in the past week, how often did you feel. . . happy? (hedonic), satisfied? (hedonic), that your life has a sense of direction or meaning to it? (eudaimonic), that you have experiences that challenge you to grow and become a better person? (eudaimonic), that you had something to contribute to society? (eudaimonic); answered on a six-point frequency metric whereby 0 indicates never, 1 indicates once or twice, 2 indicates approximately once per week, 3 indicates two or three times per week, 4 indicates almost every day, and 5 indicates every day]

These questions are odd, vague, and unlikely to be encountered in everyday life unless someone happens to be Barack Obama or Bill Gates. I don’t know about you, but my wife has not recently asked me at dinner nor have I been asked another friend at a happy hour get together, “hey Jim, what did you do today to contribute to society?” “Did you happen to run into any problems that challenge you to grow and become a better person?”

It’s not surprising that research participants requested to answer these questions came up with something vague and affectively toned, i.e., related to their mood at the moment. I’m sure that if investigators had done a cognitive interview, it would’ve revealed that respondents struggle with trying to find answers and the basis of the answers vary widely from responded to respond. This is a particularly poorly constructed assessment instrument. And what ever solid biomedical science was done, it is sustained or falls on an empirically indefensible distinction derived from poor assessment of what people have to say about themselves on the Internet.

Press coverage for this study is pure hokum. Over centuries, lots of people have offered advice about whether we should pursue meaning or happiness. They will undoubtedly continue to do so, but let’s have no illusions that Barbara Fredrickson’s study can do anything to settle the issue.

Distorted press coverage can often be traced to distorted abstracts, and so I always recommend that skeptical readers compare press coverage to the abstracts of scientific articles. I know well that the abstracts too are often distorted, but it is sort of a rule-out assessment. If the press coverage does not fit with what is said in the abstract, it could be that the journalist is making something up. On the other hand if the two seem to fit, we might have to proceed to the laborious process of checking the abstract against what is said in the rest of the paper. And maybe we can establish that exaggerated coverage in the press is churnaled from a quick read of the abstract.

The abstract states

Hedonic and eudaimonic well-being showed similar affective correlates but highly divergent transcriptome profiles.

Translation? The two scales had similar correlates with self-report measures, but widely different profiles associated with them.

Perhaps, but there is a sleight of hand going on here, or if you like, a bait and switch. The correlates of Hedonic and eudaimonic well-being were examined at the simple bivariate level and their association with transcriptome profiles was examined in dubious multivariate analyses.

I am extremely doubtful that with transcriptome profiles of hedonic and eudaimonic well-being would be that different if statistical controls had not been applied.  I challenge the authors to present the relevant data. It is quite statistically improbable that to self-report variable so highly correlated could have distinctive different profiles.

Note that this is the same Barbara Fredrickson who brought us the 2.900 positivity/negativity ratio. Check out the NeuroSkeptic’s straight-shooting coverage of the debunking of that, as well as his call for a retraction.

In a recent posting at my other blog, I cited John Ioannidis to suggest that hot areas likemeaning_of_life_17632451 genomics tend to bring lowered standards, false discoveries, and exaggerations and very little real advancement of science. This application of genomics to philosophical questions about how we should lead our life certainly seems to fit the bill. But I think something else is going on here. A huge market has been generated by people desperately searching in self-help books for answers that religion once provided. I wouldn’t be surprised if the contract for another self-help book has been signed already.


18 thoughts on “Improving your health by pursuing meaning in your life versus happiness”

  1. I spent some time in the analytical supplement because the PNAS format doesn’t require as much in the methods as I like to read. The authors give this model for their 80 participants with complete data;

    Log2 transcript abundance =Intercept+ Hedonic+ Eudaimonic + Age+ Sex
    + White/Non+ Alcohol+ Smoking +IllnessSymptoms+CD3D+CD3E+CD4+CD8A
    +CD19+FCGR3A+NCAM1+CD14+ residual;

    While you are able to run linear regression models on any continuous variable you may not always want to. The authors should share with us why this is not one of those times. How are they accounting for collinearity in the included continuous variables? Are all of the cells created by the categorical variables filled and distributed well? These models were run to calculate point estimates for later use and not for probability testing but establishing that your linear regression model is valid means that the point estimates the authors use later can be seen as valid.

    I’d be very interested to understand power however, related to this statement on page 3 of the Supporting Information; “Thus, despite the known negative bias of split-half replication analyses (as a result of loss of statistical power in half-sized samples),”. Fredrickson et al acknowledge statistical power as an issue but do not provide us with the initial power calculations on which they based their sample size. This is a complicated set of analyses that build on each other. I would be very interested to see these calculation because I would personally have no idea where to look, for instance, for the effect size estimates given the outcome. I realize that this is an easy critique because the study is asking a new and interesting question where there is little existing literature. Still a power analysis plot across varying levels of effect size, accounting for the analytical methods and covariates would be very informative for the reader. This statement;

    “Statistical reliability of TELiS results was also assessed
    by empirical split-half replication analyses and Monte Carlo
    analyses of statistical power and replication rates as described
    later (Figs. S1–S5).”

    suggests that power was considered post hoc and conducted on the data that the investigators had in hand, not on how many subjects they needed to recruit in order to test their hypothesis. The arguments around post hoc power analysis are many.

    An important thing to note in this study is that bioinformatic approaches are different from classical statistical approaches in the questions they can ask and answer and this paper is based on the former. From the perspective of epidemiology the critique is straightforward; “the sample size is to small for the number of variables used (34,592)” but from the bioinformatics perspective the explanation may be quite different; “the number of variables compensates for the small sample size”.

    We live in interesting times.


  2. I just simulated their experiment (I probably didn’t get all the detail right but this is based on my understanding of their statistical methods), using random gene expressions:
    for(j in 1:100) {
    #make random well-being scores and make them highly correlated
    #make some random covariates
    data = data.frame(eudo=eudo,hedo=hedo,sex=runif(140)>.5,race=runif(140)>.8,bmi=rnorm(140),age=rnorm(140),leuco=rnorm(140),illnes=runif(140)>.7,smoking=runif(140)>.6,alcohol=rnorm(140))
    #make 53 random gene expressions
    for(i in 1:53) {
    #adjust well-being measures for each other and for covariates
    data$hedoadjust = lm(hedo~eudo+sex+race+bmi+age+leuco+illnes+smoking+alcohol,data=data)$residuals
    data$eudoadjust = lm(hedo~eudo+sex+race+bmi+age+leuco+illnes+smoking+alcohol,data=data)$residuals
    #fit well-being models for individual genes
    hedomodel =[,11:63],rep(1,140))),data$hedoadjust)
    eudomodel =[,11:63],rep(1,140))),data$eudoadjust)
    #contruct coefficient-wieghted gene expression averages
    data$weightedgenes.hedo = as.matrix(data[,11:63]) %*% hedomodel$coefficients[1:53]
    data$weightedgenes.eudo = as.matrix(data[,11:63]) %*% eudomodel$coefficients[1:53]
    #test correlation with well-being

    In 100 experiments I never got a p-value larger than 10^-11.


  3. I don’t know about you, but my wife has not recently asked me at dinner nor have I been asked another friend at a happy hour get together, “hey Jim, what did you do today to contribute to society?” “Did you happen to run into any problems that challenge you to grow and become a better person?”

    Time to find more interesting friends?


    1. My friends are exceptionally interesting and intelligent. They just don’t talk in the stilted, phony manner the positive psychology gurus seem to assume that everybody talks. Do your friends really talk that way?


  4. I just finished analyzing their raw data and I am going to throw up. It is so unbelievably bad that I have not seen anything like that in my whole life, even in a low-quality undergraduate project. Yet, the language of the paper is quite sophisticated to fool any reader, who does not make an effort to see what is behind those claims.

    Like you mentioned, as soon as I opened the data file from GSE, I noticed high degree of correlation between ‘hedonic’ and ‘eudaimonic’ scores. Not only that, for most participants the scores were around 4. Only 16 out of 78 had scores below 2.5 on a (1-5) scale. I wonder whether those five or six who scored around 1 got ticket before getting to the experimental lab, flunked an exam on the previous day or were joking with the analyst. Moreover, given such high correlation between two data sets, their experimental results are from only a small subset of persons. So, clearly there is very high degree of over-fitting going on.


  5. I suspect that the correlation of hedonic (H) and eudaimonic (E) happiness might have been even higher, were it not for the fact that their large granularity (aka, minimal information content) means that a difference of one single point in one single answer (“hmmm, did I feel good three times last week, or only twice?”) on their respective scales corresponds to 0.3 SD (H) or 0.1 SD (E).


    1. Excellent point, I wish I had seen this in time to include in my blog post. It looks like the whole study has unraveled due to ignored problems in the assessment of hedonic (H) and eudaimonic (E) happiness by self-report. Looks like results were driven by a few outliers but, regardless, difficulties with the self-report measures strongly indicate more complicated analyses should have been abandoned before they were attempted.

      Thanks for catching this.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s