Can we predict suicide from Twitter language?

Can we predict county-level death by suicide from Twitter data? We tried. Our surprising results added weight to results of our re-analyses of Twitter data attempting to predict death from heart disease.  Analyzing Twitter data in bulk does not add to our understanding geographical variations in health outcomes.

mind the brain logo

Can we predict county-level death by suicide from Twitter data? We tried. Our surprising results added weight to results of our re-analyses of Twitter data attempting to predict death from heart disease.  Analyzing Twitter data in bulk does not add to our understanding geographical variations in health outcomes.

Nick Brown and I (*) recently posted a preprint:

No Evidence That Twitter Language Reliably Predicts Heart Disease: A Reanalysis of Eichstaedt et al. (2015a)

We reanalyze Eichstaedt et al.’s (2015a) claim to have shown that language patterns among Twitter users, aggregated at the level of U.S. counties, predicted county-level mortality rates from atherosclerotic heart disease (AHD), with “negative” language being associated with higher rates of death from AHD and “positive” language associated with lower rates…We conclude that there is no evidence that analyzing Twitter data in bulk in this way can add anything useful to our ability to understand geographical variation in AHD mortality rates.

You can find the original article here:

Eichstaedt JC, Schwartz HA, Kern ML, Park G, Labarthe DR, Merchant RM, Jha S, Agrawal M, Dziurzynski LA, Sap M, Weeg C. Psychological language on Twitter predicts county-level heart disease mortality. Psychological Science. 2015 Feb;26(2):159-69.


A press release from Association for Psychological Science heaped lavish praise on the original article. It can be found here.

“Twitter seems to capture a lot of the same information that you get from health and demographic indicators,” co-author Gregory Park said, “but it also adds something extra. So predictions from Twitter can actually be more accurate than using a set of traditional variables.

 Our overarching conclusion:

… There is a very large amount of noise in the measures of the meaning of Twitter data used by Eichstaedt et al., and these authors’ complex analysis techniques (involving, for example, several steps to deal with high multicollinearity) are merely modeling this noise to produce the illusion of a psychological mechanism that acts at the level of people’s county of residence.

Our look at key assumptions and re-analyses

The choice of atherosclerotic heart disease (AHD) as the health outcome fits with lay understanding of what causes heart attacks of interest, but was unfortunate.

Folk beliefs about negative emotion causing heart attacks had been bolstered by some initial promising findings in small samples suggesting a link between Type A behavior pattern (TABP) and cardiac events and mortality. In our preprint, we discuss how subsequent, better controlled studies did not confirm these results.

Type A behavior pattern cannot readily be distinguished from other negative emotion variables. These negative emotion variables converge in what is been called by Paul Meehl a “crud factor” or by others, a “big mess.” Such negative affect variables are non-informative risk markers, not true risk factors. These variables have too many correlates in background, pre-existing variables, including poor physical health; and in concurrent variables that cannot readily be separated in statistical analyses, even with prospective data. See “Negative emotions and health: why do we keep stalking bears when we only find scat”for a further discussion.

While we were finishing up our manuscript, an article came out that analyzed and succinctly summarized this issue:

A substantial part of the distress–IHD [ischaemic heart disease] association is explained by confounding and functional limitations . . . . Emphasis should be on psychological distress as a marker of healthcare need and IHD risk, rather than a causative factor.”

AHD is actually a chronic condition, slowly developing over a lifetime. Many of the crucial determinants of whether someone later shows signs and symptoms of AHD occur in childhood or adolescence.

Americans are a highly mobile population, and when they reach middle age with its increase in heart attacks, they may have moved geographically far away from where they lived when their chronic disease developed. The counties in which participants are identified for the purposes of this Twitter study are not the counties in which they developed their condition.

Most of the people who are tweeting in a county are younger than the people likely to be dying from AHD. So, we are assessing one population to predict health events in another.

Some of our other findings that are discussed more fully in our preprint:

Coding of AHD as the cause of death in this study was highly unreliable and subject to major variability across counties.

The process for selecting counties to be included in the study was biased.

The Twitter-based dictionaries used for coding appear not to be a faithful summary of the words that were actually typed by users. There were puzzling omissions.

Arbitrary and presumably post-hoc choices were apparently made in some of the dictionary-based analyses and these choices strengthened the appearance of an association between Twitter language and death from AHD.

There were numerous problems associated with the use of counties as the unit of analysis, which vary greatly in size (between) as well as heterogeneity (within) of sociodemographic or socioemotional factors, as well as the proportion of county residents who were actually on Twitter.

The predictive power of the model, including the associated maps, appears to be questionable.

While we were working on the manuscript that became a preprint, another relevant paper came out:

Jensen, E. A. (2017). Putting the methodological brakes on claims to measure national happiness through Twitter: Methodological limitations in social media analytics. PLOS ONE, 12(9), e0180080.

We  endorse its conclusion:

When researchers approach a data set, they need to understand and publicly account for not only the limits of the data set, but also the limits of which questions they can ask . . . and what interpretations are appropriate (p. 6).

Using Twitter data to predict death by suicide

Ok, I have already spoiled the story by giving up front the argument that trying to predict health outcomes from big Twitter data is not a good idea.

But a case can be made that if we are going to predict a health outcome from Twitter, suicide is a better candidate than AHD. This was Nick’s idea, but I wanted to emphasize it more than he did.

Although suicide can be the result of long-term mental health problems and other stressors, a person’s psychological state in the months and days leading up to the point at which they take their own life clearly has a substantial degree of relevance to their decision. Hence, we might expect any county-level psychological factors that act directly on the health and welfare of members of the local community to be more closely reflected in the mortality statistics for suicide than those for a chronic disease such as AHD.

We [collective “we” the authors, but actually Nick] also downloaded comparable mortality data for the ICD-10 categories X60–X84, collectively labeled “Intentional self-harm”—in order to test the idea that suicide might be at least as well predicted by Twitter language as AHD—as well as the data for several other causes of death (including all-cause mortality) for comparison purposes.

We therefore examined the relationship of the set of causes of death listed by the CDC as “self-harm” with Twitter language usage, using the procedures reported in the first subsections entitled “Language variables from Twitter” and “Statistical analysis” of Eichstaedt et al.’s (2015a, p. 161) Method section. Because of the limitation of the CDC Wonder database, noted earlier, whereby mortality rates are only available when at least 10 deaths per year are recorded in a given county, data for self-harm were only available for 741 counties; however, these represented 89.9% of the population of Eichstaedt et al.’s set of 1,347 counties.

Our findings

self-harm and twitter


In the “Dictionaries” analysis, we found that mortality from self-harm was negatively correlated with all five “negative” language factors, with three of these correlations (for anger, negative-relationship, and negative-emotion words) being statistically significant at the .05 level (see our Table 1). That is, counties whose residents made greater use of negative language on Twitter had lower rates of suicide, or, to borrow Eichstaedt et al.’s (2015a, p. 162) words, use of negative language was “significantly protective” against self-harm; this statistical significance was unchanged when income and education were added as covariates. In a further contrast to AHD mortality, two of the three positive language factors (positive relations and positive emotions) were positively correlated with mortality from self-harm, although these correlations were not statistically significant.

Next, we analyzed the relationship between Twitter language and self-harm outcomes at the “Topics” level. Among the topics most highly correlated with increased risk of self-harm were those associated with spending time surrounded by nature (e.g., grand, creek, hike; r = .214, CI[1] = [.144, .281]), romantic love (e.g., beautiful, love, girlfriend; r = .176, CI = [.105, .245]), and positive evaluation of one’s social situation (e.g., family, friends, wonderful; r = .175, CI = [.104, .244]). There were also topics of discussion that appeared to be strongly “protective” against the risk of self-harm, such as baseball (e.g., game, Yankees, win; r = −.317, CI = [−.381, −.251]), binge drinking (e.g., drunk, sober, hungover; r = −.249, CI = [−.316, −.181]), and watching reality TV (e.g., Jersey, Shore, episode; r = −.200, CI = [−.269, −.130]). All of the correlations between these topics and self-harm outcomes, both positive and negative, were significant at the same Bonferroni-corrected significance level (i.e., .05/2,000 = .000025) used by Eichstaedt et al. (2015a), and remained significant at that level after adjusting for income and education. That is, several topics that were ostensibly associated with “positive,” “eudaimonic” approaches to life predicted higher rates of county-level self-harm mortality, whereas apparently hedonistic topics were associated with lower rates of self-harm mortality, and the magnitude of these associations was at least as great—and in a few cases, even greater—than those found by Eichstaedt et al. These topics are shown in “word cloud” form (generated at in our Figure 2 (cf. Eichstaedt et al.’s Figure 1).

time spent with nature


If anyone insists on giving this finding a substantive interpretation…

This discovery would seem to pose a problem for Eichstaedt et al.’s (2015a, p. 166) claim to have shown the existence of “community-level psychological factors that are important for the cardiovascular health of communities.” Apparently the “positive” versions of these factors, while acting via some unspecified mechanism to make the community as a whole less susceptible to developing hardening of the arteries, also simultaneously manage to make the same people more likely to commit suicide, and vice versa. It seems that more research into the possible risks of increased levels of self-harm would seem to be needed before any program to enhance these “community-level psychological factors” were to be undertaken.

But actually, no, we don’t want to do that.

Of course, there is no suggestion that the study of the language used on Twitter by the inhabitants of any particular county has any real predictive value for the local suicide rate; we believe that such associations are likely to be the entirely spurious results of imperfect measurements and chance factors, and to use Twitter data to predict which areas might be about to experience higher suicide rates is likely to prove extremely inaccurate (and perhaps ethically questionable as well).


*When published, this preprint will serve as one of the articles that will be bundled in Nick Brown’s PhD thesis submitted to University Medical Centre., Groningen. As Nick’s adviser, I was pleased to have a role that justified an authorship. I want to be clear, however, my role was more like a midwife observing a natural birth than an OBGyn having to induce labor. Nick can’t say what I can say: there is some real brilliance to this paper. The brilliance belongs to Nick, not me.  And I mean brilliance in the restricted American sense, not promiscuous British sense, like that is a brilliant dessert.

I encourage you to dig in and enjoy. There are lots of treats and curious observations. Nick notably retrieved and analyzed the data, but also did some programming to capture the color depiction of counties and ADHD rates. He identified some anomalies and then developed his own depiction with some corrections to the original. Truly amazing.

map differences


Using F1000 “peer review” to promote politics over evidence about delivering psychosocial care to cancer patients

The F 1000 platform allowed authors and the reviewers whom they nominated to collaborate in crafting more of their special interest advocacy that they have widely disseminated elsewhere. Nothing original in this article and certainly not best evidence!


mind the brain logo

A newly posted article on the F1000 website raises questions about what the website claims is a “peer-reviewed” open research platform.

Infomercial? The F1000 platform allowed authors and the reviewers whom they nominated to collaborate in crafting more of their special interest advocacy that they have widely disseminated elsewhere. Nothing original in this article and certainly not best evidence!

I challenge the authors and the reviewers they picked to identify something said in the F1000 article that they have not said numerous times before either alone or in papers co-authored by some combination of authors and the reviewers they picked for this paper.

F1000 makes the attractive and misleading claim that versions of articles that are posted on its website reflect the response to reviewers.

Readers should be aware of uncritically accepting articles on the F 1000 website as having been peer-reviewed in any conventional sense of the term.

Will other special interests groups exploit this opportunity to brand their claims as “peer-reviewed” without the risk of having to tone down their claims in peer review? Is this already happening?

In the case of this article, reviewers were all chosen by the authors and have a history of co-authoring papers with the authors of the target paper in active advocacy of a shared political perspective, one that is contrary to available evidence.

Cynically, future authors might be motivated to divide their team, with some remaining authors and others dropping off to become nominated as reviewers. They could then suggest content that had already been agreed would be included, but was left off for the purposes being suggested in the review process


F1000Research bills itself as

An Open Research publishing platform for life scientists, offering immediate publication of articles and other research outputs without editorial bias. All articles benefit from transparent refereeing and the inclusion of all source data.

Material posted on this website is labeled as having received rapid peer-review:

Articles are published rapidly as soon as they are accepted, after passing an in-house quality check. Peer review by invited experts, suggested by the authors, takes place openly after publication.

My recent Google Scholar alert call attention to an article posted on F1000

Advancing psychosocial care in cancer patients [version 1; referees: 3 approved]

 Who were the reviewers?

open peer review of Advancing psychosocial care

Google the names of authors and reviewers. You will discover a pattern of co-authorship; leadership positions in international Psycho-Oncology society, a group promoting the mandating of specially mental health services for cancer patients, and lots of jointly and separately authored articles making a pitch for increased involvement of mental health professionals in routine cancer care. This article adds almost nothing to what is multiply available elsewhere in highly redundant publications

Given a choice of reviewers, these authors would be unlikely to nominate me. Nonetheless, here is my review of the article.

 As I might do in a review of a manuscript, I’m not providing citations for these comments, but support can readily be found by a search of blog posts at my website and Google Scholar search of my publications. I welcome queries from anybody seeking documentation of these points below.

 Fighting Spirit

The notion that cancer patients having a fighting spirit improves survival is popular in the lay press and in promoting the power of the mind over cancer, but it has thoroughly been discredited.

Early on, the article identifies fighting spirit as an adaptive coping style. In actuality, fighting spirit was initially thought to predict mortality in a small methodologically flawed study. But that is no longer claimed.

Even one of the authors of the original study, Maggie Watson,  expressed relief when her own larger, better designed study failed to confirm the impression that a fighting spirit extended life after diagnosis  of cancer. Why? Dr. Watson was concerned that the concept was being abused in blaming cancer patients who were dying there was to their personal deficiency of not having enough fighting spirit.

Fighting spirit is rather useless as a measure of psychological adaptation. It confounds severity of cancer enrolled dysfunction with efforts to cope with cancer.

Distress as the sixth vital sign for cancer patients

distress thermometerBeware of a marketing slogan posing as an empirical statement. Its emptiness is similar to that of to “Pepsi is the one.” Can you imagine anyone conducting a serious study in which they conclude “Pepsi is not the one”?

Once again in this article, a vacuous marketing slogan is presented in impressive, but pseudo-medical terms. Distress cannot be a vital sign in the conventional sense. Thr  vital signs are objective measurements that do not depend on patient self-report: body temperature, pulse rate, and respiration rate (rate of breathing) (Blood pressure is not considered a vital sign, but is often measured along with the vital signs.).

Pain was declared a fifth vital sign, with physicians mandated  by guidelines to provide routine self-report screening of patients, regardless of their reasons for visit. Pain being the fifth vital sign seems to have been the inspiration for declaring distress as the sixth vital sign for cancer patients. However policy makers declaring pain  as the fifth vital sign did not result in improved patient levels of pain. Their subsequent making intervention mandatory for any reports of pain led to a rise in unnecessary back and knee surgery, with a substantial rise in associated morbidity and loss of function. The next shift to prescription of opioids that were claimed not to be addictive was the beginning of the current epidemic of addiction to prescription opioids. Making pain the fifth vital sign is killed a lot of patients and  turned others into addicts craving drugs on the street because they have lost their prescriptions for the opioids that addicted them.

pain as 5th vital signCDC launches

 Cancer as a mental health issue

There is a lack of evidence that cancer carries a risk of psychiatric disorder more than other chronic and catastrophic illnesses. However, the myth that there is something unique or unusual about cancer’s threat to mental health is commonly cited by mental health professional advocacy groups is commonly used to justify increased resources to them for specialized services.

The article provides an inflated estimate of psychiatric morbidity by counting adjustment disorders as psychiatric disorders. Essentially, a cancer patient who seeks mental health interventions for distress qualifies by virtue of help seeking being defined as impairment.

The conceptual and empirical muddle of “distress” in cancer patients

The article repeats the standard sloganeering definition of distress that the authors and reviewers have circulated elsewhere.

It has been very broadly defined as “a multifactorial, unpleasant, emotional experienceof a psychological (cognitive, behavioural, emotional), social and/or spiritual nature that may interfere with the ability to cope effectively with cancer, its physical symptoms and its treatment and that extends along a continuum, ranging from common normalfeelings of vulnerability, sadness and fears to problems that can become disabling, such as depression, anxiety, panic, social isolation and existential and spiritual crisis”5

[You might try googling this. I’m sure you’ll discover an amazing number of repetitions in similar articles advocating increasing psychosocial services for cancer patients organized around this broad definition.]

Distress is so broadly defined and all-encompassing, that there can be no meaningful independent validation of distress measures except for by other measures of distress, not conventional measures of adaptation or mental health. I have discussed that in a recent blog post.

If we restrict “distress” to the more conventional meaning of stress or negative affect, we find that any elevation in distress (usually 35% or so) associated with onset diagnosis of cancer tends to follow a natural trajectory of decline without formal intervention. Elevations in distress for most cancer patients, are resolved within 3 to 6 months without intervention. A residual 9 to 11% of cancer patients having elevated distress is likely attributed to pre-existing psychiatric disorder.

Routine screening for distress

The slogan “distress is the sixth vital sign” is used to justify mandatory routine screening of cancer patients for distress. In the United States, surgeons cannot close their electronic medical records for a patient and go on to the next patient without recording whether they had screened patients for distress, and if the patient reports distress, what intervention has been provided. Clinicians simply informally asking patients if they are distressed and responding to a “yes” by providing the patient with an antidepressant without further follow up allows surgeons to close the medical records.

As I have done so before, I challenge advocates of routine screening of cancer patients for distress to produce evidence that simply introducing routine screening without additional resources leads to better patient outcomes.

Routine screening for distress as uncovering unmet needs among cancer patients

 Studies in the Netherlands suggest that there is not a significant increase in need for services from mental health or allied health professionals associated with diagnosis of cancer. There is some disruption of such services that patients were receiving before diagnosis. It doesn’t take screening and discussion to suggest that patients that they at some point resume those services if they wish. There is also some increased need for physical therapy and nutritional counseling

If patients are simply asked a question whether they want a discussion of the services (in Dutch: Zou u met een deskundige willen praten over uw problemen?)  that are available, many patients will decline.

Much of demand for supportive services like counseling and support groups, especially among breast cancer patients is not from among the most distressed patients. One of the problems with clinical trials of psychosocial interventions is that most of the patients who seek enrollment are not distressed, and less they are prescreened. This poses dilemma: if you require elevated distress on a screening instrument, we end up rationing services and excluding many of the patients who would otherwise be receiving them.

I welcome clarification from F 1000 just what they offer over other preprint repositories. When one downloads a preprint from some other repositories, it clearly displays “not yet peer-reviewed.” F 1000 carries the advantage of the label of “peer-reviewed, but does not seem to be hard earned.


Slides are from two recent talks at Dutch International Congress on Insurance Medicine Thursday, November 9, 2017, Almere, Netherlands   :

Will primary care be automated screening and procedures or talking to patients and problem-solving? Invited presentation


Why you should not routinely screen your patients for depression and what you should do instead. Plenary Presentation






Complex PTSD, STAIR, Social Ecology and lessons learned from 9/11- a conversation with Dr. Marylene Cloitre

Dr. Marylene Cloitre is the Associate Director of Research of the National Center for PTSD Dissemination and Training Division and a research Professor of Psychiatry and Child and Adolescent Psychiatry at the New York University, Langone Medical Center in New York City. She is a recipient of several honors related to her service in New York City following 9-11 and was an advisory committee member for the National September 11 Memorial Museum. She has specific expertise in complex PTSD and for the development and dissemination of STAIR (Skills Training in Affective and Interpersonal Regulation), a psychological therapy designed to help survivors of trauma.

Dr. Jain: What exactly is complex PTSD?

Dr. Cloitre:
Complex PTSD has a very long history, really pushed primarily by clinicians who looked at their patients and thought there’s something more going on here than PTSD.
In DSM-4, complex PTSD was recognized in the additional features where there is a mix of problems related to emotion regulation, self-concept and interpersonal relationships. After that, there was really no funding around investigating this further and the research for it has been spotty and it was sort of dying on the vine.

But with the development of a new version of ICD-11, there was an opportunity really to refresh consideration about complex PTSD. I was part of a work group that started in 2012, we looked at the literature and thought there seems to be enough data to support two different forms of PTSD , the classic fear circuitry disturbance and then this more general kind of disturbance in these three core areas of emotion regulation, self-concept and interpersonal relationships.

We proposed that there should be two distinct disorders: PTSD and complex PTSD and it looks like it’s been accepted and it will part of the ICD-11 coming out in the 2018.

Since the initial proposal, I’ve been working with many people, mostly Europeans, where ICD is more prominent than in the United States and there are now about nine published papers providing supporting evidence that these two distinct disorders.

Dr. Jain:
Can you summarize in which ways they’re distinct? So on a clinical level what would you see in complex PTSD?

Dr. Cloitre: Mostly we’ve been looking at latent class analysis which is a newish kind of data analytic technique which looks at how people cluster together and you look at their symptom profile. There are a group of people who very distinctly have PTSD in terms of re-experiencing, avoidance and hyperarousal and then they’re fine on everything else. Then you have another group of people who have these problems as well as problems in these three other areas.And then there are another group of people who, despite exposure to trauma, do fairly well.

What we’ve been seeing are these three groups in clinical populations as well as in community populations and adults as well as in children.

Overall, these latent class analyses are really showing that people cluster together in very distinctly different ways. I think the important thing about this distinction is, what’s next? Perhaps there are different clinical interventions that we want to look at to maximize good outcome. Some people may do very well with exposure therapy. I would say the PTSD clustered folks will do very well and have optimal outcome because that’s all that bothers them. For the other folks, they have a lot of other problems that really contribute to their functional impairment.

For me as a clinician as well as a researcher, I’ve always been worried not so much about the diagnosis of the person in front of me but about how well they’re functioning in the world. What I have noticed is you can get rid of the PTSD symptoms, for people with complex PTSD, but they’re still very impaired.
My motivation for thinking about a different diagnosis and different treatment is to identify these other problems and then to provide interventions, that target these other problems, for the goal of improving our day to day life functioning. If you don’t have ability to relate well to people because you mistrust them or are highly avoidant or if you think poorly about yourself these are huge issues then we need to target these issues in treatment.

Dr. Jain
Have you noticed that different types of trauma contribute to PTSD v complex PTSD?

Dr. Cloitre Yes, it does and it kind of makes sense that people who have had sustained and repeated trauma (e.g. multiple and sustained trauma doing childhood) are the ones who have complex PTSD.

Dr. Jain: Can you tell us a little bit about the fundamental philosophy that drove you to come up with STAIR and what evidence is there for it’s effectiveness?

Dr. Cloitre I came to develop STAIR as a result of paying attention to what my patients were telling me they wanted help with, that was the driving force. It wasn’t a theoretical model, it was that patients came and said,” I’m really having problems with my relationships and that’s what I need help with” or “I really have problems with my moods and I need help with that”.

So, I thought, why don’t we start there? That is why I developed STAIR and developed it as a sequence therapy while respecting the importance of getting into the trauma and doing exposure based work, I also wanted to engage the patient and respect their presented needs. That what it’s all about for me.
Overtime I saw a secondary benefit, that an improved sense of self and improved emotion regulation could actually impact the value of exposure therapy in a positive way.

In my mind, the real question is: What kind of treatments work best for whom? That is the question. There will be some people for whom going straight to exposure therapy is the most effective and efficient way to get them functioning and they’ll be happy with three or four sessions, just like some 9/11 survivors I saw. They only needed three or four sessions.

Other people might do better with combination therapies .

Dr. Jain The studies that you’ve done with STAIR can you summarize the populations you have used it for?

Dr. Cloitre: I began using STAIR + exposure with the population I thought would most need it which is people with histories of childhood abuse. In fact, our data show that the combination of skills training, plus exposure was significantly better than skills alone or exposure alone. So that’s very important. It also reduced dropout very significantly as compared to exposure, which is a continuing problem with exposure therapy especially for this population

Dr. Jain Can you speak to the social ecology/social bonds and PTSD, what the research world can tell us about the social dimensions of PTSD and how we can apply this to returning military members and veterans?

Dr. Cloitre: I think that social support is critical to the recovery of people who have been exposed to trauma and who are vulnerable to symptoms .We have enough studies showing that it’s the critical determinant of return to health.

I think we have done a very poor job of translating this observation into something meaningful for returning veterans. There is general attention that families are part of the solution and communities are part of the solution but it is vague –there isn’t really a sense of what are we going to do about it.

I think these wars (Afghanistan and Iraq) are very different than Vietnam, that’s where soldiers came back and they were called baby killers and had tomatoes and garbage thrown at them. You can really understand why a vulnerable person would spiral downwards into pretty significant PTSD and substance abuse.

I think we need to be more thoughtful and engage veterans in discussions about what’s most helpful in the reintegration progress, because there are probably really explicit things like, being welcomed home, but also very subtle things that we haven’t figured out about the experience.
I think on a community or family level, there’s a general awareness but we haven’t really gotten clear thinking or effective about what to do. I think that’s our next step. The parade and the welcome home signs are not enough.

I’ll give an example of what I witnessed after 9/11. The community around survivors feels awkward and doesn’t know what to do, so they start moving away. Combine this with the survivor who is sad or being irritable and so not the most attractive person to engage with. I say to patients sometimes, it’s a really unfair and unfortunate circumstance, that in a way, not only are you suffering but you’re also kind of responsible for making people around you comfortable with you.

I used to do STAIR because patients ask for it and also I thought,” oh well some people never had these social skills in the first place, which is why they are vulnerable with PTSD” but then I noticed that STAIR was useful for everybody with PTSD because I think the traumatized patient has an unfair burden to actually reach out to people in a process of re-engagement because the community and the family is confused. Others, strangers or say employers are scared. So they have to kind of compensate for the discomfort of others, which is asking a lot.

I think in our therapies we can say look, it’s not fair, but people feel uncomfortable around the veteran. They don’t know how to act and in a way you not only have to educate yourself about your circumstance but, in the end, educate others.

Dr. Jain Survivor perception of social support really matters. If you take a group of disaster survivors, we may feel well we’re doing this for them and we’re doing that for them but if the survivors, for whatever reason, don’t perceive it as being helpful it doesn’t matter. When I think about marginalized populations in our society, I don’t think to communicate to others about how to help you or how to support you is that simple.

Dr. Cloitre It’s very complicated because it is a dynamic. I think we need to talk to trauma survivors and understand what their needs are so that the community can respond effectively and be a match. Not everybody wants the same thing. That’s the real challenge. I think if survivors can be a little bit more compassionate, not only towards themselves for what they have been through but to others who are trying to communicate with them and failing.

Dr. Jain That can be hard to do when you’re hurting. The social ecology of PTSD is really important but it’s really complicated and we are not there, in terms of harnessing social ecology to improve lives.

Dr. Cloitre No. I think we’re just groping around in the dark, in a room that says the social ecology of PTSD is important. We don’t know how to translate that observation into actionable plans either in our individual therapies or in our family therapies and then in our community actions or policies.
But I do think that, in the individual therapy, recognizing the importance of trying to enhance perception of support where they’re real. Secondly, recognizing the burden that they have that’s unfair and try to enhance skills for communicating with people. Thirdly, having compassion for people who are out there who are trying to communicate but failing.
I have had a lot of patients who come, into therapy, and say,
” This is so ridiculous. They’re saying stupid things to me”.
And, I say,
“well at least they’re trying”
I think it’s important for the affected community to have the voice and take leadership, instead of people kind of smothering them with social support that they may or may not need.

Dr. Jain
I know you’re a native New Yorker and you provided a lot of service to New York City following 9/11. Can you speak about that work? And in particular what I’m really interested in is that body of research that emerged after 9/11 because I feel like that has helped us understand so much about disaster related PTSD.

Dr. Cloitre We found out was most people are very resilient. We were able to get prevalence rates of PTSD following 9/11, that in of itself was very important. I think that’s the strongest research that came out.

I think on a social level it broke open awareness, in this country and maybe globally, about the impact of trauma and about PTSD because it came with very little shame or guilt.
Some people say what was so different about 9/11? Well, because it happened to the most powerful country and the most powerful city then if it could happen to them it could happen anywhere. That was the response, there was not this marginalization, ”Well this is a victim circumstance and it couldn’t happen to me and they must have done something to bring it on themselves”.
There was a hugely different response and that was so key to the shift in recognition of the diagnosis of PTSD which then led to more general research about it. I think that that was huge.
Before 9/11, I would say I do research in PTSD and people would say, what is that? Now I say I do research in PTSD, not a single person ever asks me what that is. I mean I’m sure they don’t really know what it is but they never looked confused. It’s a term that is now part and parcel of American society.
9/11 revolutionized the awareness of PTSD and also the acceptability of adverse effects, as a result of trauma. There was new knowledge gained and also just a transformation in awareness that was national and probably global because the impact it had and the ripple effects on another countries.
I think those are the two main things.
I don’t think it’s really done very much for our thinking about treatment. I think we continue to do some of our central treatments and we didn’t get too far in really advancing or diversifying.
For me personally, I learned a lot about the diversity of kinds of trauma survivors. Very different people, very different reactions.
I think probably the other important academic or scholarly advance, was in the recognition of this blend of loss and trauma and how they come together. That people’s responses to death ,under circumstances of unexpected and violent death, has also advanced. In fact now ICD-11 there will be a traumatic grief diagnosis, which I think has moved forward because of 9/11. That’s pretty big.

Why PhD students should not evaluate a psychotherapy for their dissertation project

  • Things some clinical and health psychology students wish they had known before they committed themselves to evaluating a psychotherapy for their dissertation study.
  • A well designed pilot study addressing feasibility and acceptability issues in conducting and evaluating psychotherapies is preferable to an underpowered study which won’t provide a valid estimate of the efficacy of the intervention.
  • PhD students would often be better off as research parasites – making use of existing published data – rather than attempting to organize their own original psychotherapy study, if their goal is to contribute meaningfully to the literature and patient care.
  • Reading this blog, you will encounter a link to free, downloadable software that allows you to make quick determinations of the number of patients needed for an adequately powered psychotherapy trial.

I so relish the extra boost of enthusiasm that many clinical and health psychology students bring to their PhD projects. They not only want to complete a thesis of which they can be proud, they want their results to be directly applicable to improving the lives of their patients.

Many students are particularly excited about a new psychotherapy about which extravagant claims are being made that it’s better than its rivals.

I have seen lots of fad and fashions come and go, third wave, new wave, and no wave therapies. When I was a PhD student, progressive relaxation was in. Then it died, mainly because it was so boring for therapists who had to mechanically provide it. Client centered therapy was fading with doubts that anyone else could achieve the results of Carl Rogers or that his three facilitative conditions of unconditional positive regard, genuineness,  and congruence were actually distinguishable enough to study.  Gestalt therapy was supercool because of the charisma of Fritz Perls, who distracted us with his showmanship from the utter lack of evidence for its efficacy.

I hate to see PhD students demoralized when their grand plans prove unrealistic.  Inevitably, circumstances force them to compromise in ways that limit any usefulness to their project, and maybe even threaten their getting done within a reasonable time period. Overly ambitious plans are the formidable enemy of the completed dissertation.

The numbers are stacked against a PhD student conducting an adequately powered evaluation of a new psychotherapy.

This blog post argues against PhD students taking on the evaluation of a new therapy in comparison to an existing one, if they expect to complete their projects and make meaningful contribution to the literature and to patient care.

I’ll be drawing on some straightforward analysis done by Pim Cuijpers to identify what PhD students are up against when trying to demonstrate that any therapy is better than treatments that are already available.

Pim has literally done dozens of meta-analyses, mostly of treatments for depression and anxiety. He commands a particular credibility, given the quality of this work. The way Pim and his colleagues present a meta-analysis is so straightforward and transparent that you can readily examine the basis of what he says.

Disclosure: I collaborated with Pim and a group of other authors in conducting a meta-analysis as to whether psychotherapy was better than a pill placebo. We drew on all the trials allowing a head-to-head comparison, even though nobody ever really set out to pit the two conditions against each other as their first agenda.

Pim tells me that the brief and relatively obscure letter, New Psychotherapies for Mood and Anxiety Disorders: Necessary Innovation or Waste of Resources? on which I will draw is among his most unpopular pieces of work. Lots of people don’t like its inescapable message. But I think that if PhD students should pay attention, they might avoid a lot of pain and disappointment.

But first…

Note how many psychotherapies have been claimed to be effective for depression and anxiety. Anyone trying to make sense of this literature has to contend with claims being based on a lot of underpowered trials– too small in sample size to be expected reasonably to detect the effects that investigators claim – and that are otherwise compromised by methodological limitations.

Some investigators were simply naïve about clinical trial methodology and the difficulties doing research with clinical populations. They may have not understand statistical power.

But many psychotherapy studies end up in bad shape because the investigators were unrealistic about the feasibility of what they were undertaken and the low likelihood that they could recruit the patients in the numbers that they had planned in the time that they had allotted. After launching the trial, they had to change strategies for recruitment, maybe relax their selection criteria, or even change the treatment so it was less demanding of patients’ time. And they had to make difficult judgments about what features of the trial to drop when resources ran out.

Declaring a psychotherapy trial to be a “preliminary” or a “pilot study” after things go awry

The titles of more than a few articles reporting psychotherapy trials contain the apologetic qualifier after a colon: “a preliminary study” or “a pilot study”. But the studies weren’t intended at the outset to be preliminary or pilot studies. The investigators are making excuses post-hoc – after the fact – for not having been able to recruit sufficient numbers of patients and for having had to compromise their design from what they had originally planned. The best they can hope is that the paper will somehow be useful in promoting further research.

Too many studies from which effect sizes are entered into meta-analyses should have been left as pilot studies and not considered tests of the efficacy of treatments. The rampant problem in the psychotherapy literature is that almost no one treats small scale trials as mere pilot studies. In a recent blog post, I provided readers with some simple screening rules to identify meta-analyses of psychotherapy studies that they could dismiss from further consideration. One was whether there were sufficient numbers of adequately powered studies,  Often there are not.

Readers take their inflated claims of results of small studies seriously, when these estimates should be seen as unrealistic and unlikely to be replicated, given a study’s sample size. The large effect sizes that are claimed are likely the product of p-hacking and the confirmation bias required to get published. With enough alternative outcome variables to choose from and enough flexibility in analyzing and interpreting data, almost any intervention can be made to look good.

The problem is is readily seen in the extravagant claims about acceptance and commitment therapy (ACT), which are so heavily dependent on small, under-resourced studies supervised by promoters of ACT that should not have been used to generate effect sizes.

Back to Pim Cuijpers’ brief letter. He argues, based on his numerous meta-analyses, that it is unlikely that a new treatment will be substantially more effective than an existing credible, active treatment.  There are some exceptions like relaxation training versus cognitive behavior therapy for some anxiety disorders, but mostly only small differences of no more than d= .20 are found between two active, credible treatments. If you search the broader literature, you can find occasional exceptions like CBT versus psychoanalysis for bulimia, but most you find prove to be false positives, usually based on investigator bias in conducting and interpreting a small, underpowered study.

You can see this yourself using the freely downloadable G*power program and plug in d= 0.20 for calculating the number of patients needed for a study. To be safe, add more patients to allow for the expectable 25% dropout rate that has occurred across trials. The number you get would require a larger study than has ever been done in the past, including the well-financed NIMH Collaborative trial.

G power analyses

Even more patients would be needed for the ideal situation in which a third comparison group allowed  the investigator to show the active comparison treatment had actually performed better than a nonspecific treatment that was delivered with the same effectiveness that the other had shown in earlier trials. Otherwise, a defender of the established therapy might argue that the older treatment had not been properly implemented.

So, unless warned off, the PhD student plans a study to show not only that now hypothesis can be rejected that the new treatment is no better than the existing one, but that in the same study the existing treatment had been shown to be better than wait list. Oh my, just try to find an adequately powered, properly analyzed example of a comparison of two active treatments plus a control comparison group in the existing published literature. The few examples of three group designs in which a new psychotherapy had come out better than an effectively implemented existing treatment are grossly underpowered.

These calculations so far have all been based on what would be needed to reject the null hypothesis of no difference between the active treatment and a more established one. But if the claim is that the new treatment is superior to the existing treatment, our PhD student now needs to conduct a superiority trial in which some criteria is pre-set (such as greater than a moderate difference, d= .30) and the null hypothesis is that the advantage of the new treatment is less. We are now way out into the fantasyland of breakthrough, but uncompleted dissertation studies.

Two take away messages

 The first take away message is that we should be skeptical of claims of the new treatment is better than past ones except when the claim occurs in a well-designed study with some assurance that it is free of investigator bias. But the claim also has to arise in a trial that is larger than almost any psychotherapy study is ever been done. Yup, most comparative psychotherapy studies are underpowered and we cannot expect robust claims are robust that one treatment is superior to another.

But for PhD students been doing a dissertation project, the second take away message is that they should not attempt to show that one treatment is superior to another in the absence of resources they probably don’t have.

The psychotherapy literature does not need another study with too few patients to support its likely exaggerated claims.

An argument can be made that it is unfair and even unethical to enroll patients in a psychotherapy RCT with insufficient sample size. Some of the patients will be randomized to the control condition that is not what attracted them to the trial. All of the patients will be denied having been in a trial makes a meaningful contribution to the literature and to better care for patients like themselves.

What should the clinical or health psychology PhD student do, besides maybe curb their enthusiasm? One opportunity to make meaningful contributions to literature by is by conducting small studies testing hypotheses that can lead to improvement in the feasibility or acceptability of treatments to be tested in studies with more resources.

Think of what would’ve been accomplished if PhD students had determined in modest studies that it is tough to recruit and retain patients in an Internet therapy study without some communication to the patients that they are involved in a human relationship – without them having what Pim Cuijpers calls supportive accountability. Patients may stay involved with the Internet treatment when it proves frustrating only because they have the support and accountability to someone beyond their encounter with an impersonal computer. Somewhere out there, there is a human being who supports them and sticking it out with the Internet psychotherapy and will be disappointed if they don’t.

A lot of resources have been wasted in Internet therapy studies in which patients have not been convinced that what they’re doing is meaningful and if they have the support of a human being. They drop out or fail to do diligently any homework expected of them.

Similarly, mindfulness studies are routinely being conducted without anyone establishing that patients actually practice mindfulness in everyday life or what they would need to do so more consistently. The assumption is that patients assigned to the mindfulness diligently practice mindfulness daily. A PhD student could make a valuable contribution to the literature by examining the rates of patients actually practicing mindfulness when the been assigned to it in a psychotherapy study, along with barriers and facilitators of them doing so. A discovery that the patients are not consistently practicing mindfulness might explain weaker findings than anticipated. One could even suggest that any apparent effects of practicing mindfulness were actually nonspecific, getting all caught up in the enthusiasm of being offered a treatment that has been sought, but not actually practicing mindfulness.

An unintended example: How not to recruit cancer patients for a psychological intervention trial

Randomized-controlled-trials-designsSometimes PhD students just can’t be dissuaded from undertaking an evaluation of a psychotherapy. I was a member of a PhD committee of a student who at least produced a valuable paper concerning how not to recruit cancer patients for a trial evaluating problem-solving therapy, even though the project fell far short of conducting an adequately powered study.

The PhD student was aware that  claims of effectiveness of problem-solving therapy reported in in the prestigious Journal of Consulting and Clinical Psychology were exaggerated. The developer of problem-solving therapy for cancer patients (and current JCCP Editor) claimed  a huge effect size – 3.8 if only the patient were involved in treatment and an even better 4.4 if the patient had an opportunity to involve a relative or friend as well. Effect sizes for this trial has subsequently had to be excluded from at least meta-analyses as an extreme outlier (1,2,3,4).

The student adopted the much more conservative assumption that a moderate effect size of .6 would be obtained in comparison with a waitlist control. You can use G*Power to see that 50 patients would be needed per group, 60 if allowance is made for dropouts.

Such a basically inert control group, of course, has a greater likelihood of seeming to demonstrate a treatment is effective than when the comparison is another active treatment. Of course, such a control group also has the problem of not allowing a determination if it was the active ingredient of the treatment that made the difference, or just the attention, positive expectations, and support that were not available in the waitlist control group.

But PhD students should have the same option as their advisors to contribute another comparison between an active treatment and a waitlist control to the literature, even if it does not advance our knowledge of psychotherapy. They can take the same low road to a successful career that so many others have traveled.

This particular student was determined to make a different contribution to the literature. Notoriously, studies of psychotherapy with cancer patients often fail to recruit samples that are distressed enough to register any effect. The typical breast cancer patient, for instance, who seeks to enroll in a psychotherapy or support group trial does not have clinically significant distress. The prevalence of positive effects claimed in the literature for interventions with cancer patients in published studies likely represents a confirmation bias.

The student wanted to address this issue by limiting patients whom she enrolled in the study to those with clinically significant distress. Enlisting colleagues, she set up screening of consecutive cancer patients in oncology units of local hospitals. Patients were first screened for self-reported distress, and, if they were distressed, whether they were interested in services. Those who met both criteria were then re-contacted to see if that be willing to participate in a psychological intervention study, without the intervention being identified. As I reported in the previous blog post:

  • Combining results of  the two screenings, 423 of 970 patients reported distress, of whom 215 patients indicated need for services.
  • Only 36 (4% of 970) patients consented to trial participation.
  • We calculated that 27 patients needed to be screened to recruit a single patient, with 17 hours of time required for each patient recruited.
  • 41% (n= 87) of 215 distressed patients with a need for services indicated that they had no need for psychosocial services, mainly because they felt better or thought that their problems would disappear naturally.
  • Finally, 36 patients were eligible and willing to be randomized, representing 17% of 215 distressed patients with a need for services.
  • This represents 8% of all 423 distressed patients, and 4% of 970 screened patients.

So, the PhD student’s heroic effort did not yield the sample size that she anticipated. But she ended up making a valuable contribution to the literature that challenges some of the basic assumptions that were being made about how cancer patients in psychotherapy research- that all or most were distressed. She also ended up producing some valuable evidence that the minority of cancer patients who report psychological distress are not necessarily interested in psychological interventions.

Fortunately, she had been prepared to collect systematic data about these research questions, not just scramble within a collapsing effort at a clinical trial.

Becoming a research parasite as an alternative to PhD students attempting an under-resourced study of their own

research parasite awardPsychotherapy trials represent an enormous investment of resources, not only the public funding that is often provided for them,be a research parasite but in the time, inconvenience, and exposure to ineffective treatments experienced by patients who participate in the trials. Increasingly, funding agencies require that investigators who get money to do a psychotherapy study some point make their data available for others to use.  The 14 prestigious medical journals whose editors make up the International Committee of Medical Journal Editors (ICMJE) each published in earlier in 2016 a declaration that:

there is an ethical obligation to responsibly share data generated by interventional clinical trials because participants have put themselves at risk.

These statements proposed that as a condition for publishing a clinical trial, investigators would be required to share with others appropriately de-identified data not later than six months after publication. Further, the statements proposed that investigators describe their plans for sharing data in the registration of trials.

Of course, a proposal is only exactly that, a proposal, and these requirements were intended to take effect only after the document is circulated and ratified. The incomplete and inconsistent adoption of previous proposals for registering of  trials in advance and investigators making declarations of conflicts of interest do not encourage a lot of enthusiasm that we will see uniform implementation of this bold proposal anytime soon.

Some editors of medical journals are already expressing alarmover the prospect of data sharing becoming required. The editors of New England Journal of Medicine were lambasted in social media for their raising worries about “research parasites”  exploiting the availability of data:

a new class of research person will emerge — people who had nothing to do with the design and execution of the study but use another group’s data for their own ends, possibly stealing from the research productivity planned by the data gatherers, or even use the data to try to disprove what the original investigators had posited. There is concern among some front-line researchers that the system will be taken over by what some researchers have characterized as “research parasites.”

 Richard Lehman’s  Journal Review at the BMJ ‘s blog delivered a brilliant sarcastic response to these concerns that concludes:

I think we need all the data parasites we can get, as well as symbionts and all sorts of other creatures which this ill-chosen metaphor can’t encompass. What this piece really shows, in my opinion, is how far the authors are from understanding and supporting the true opportunities of clinical data sharing.

However, lost in all the outrage that The New England Journal of Medicine editorial generated was a more conciliatory proposal at the end:

How would data sharing work best? We think it should happen symbiotically, not parasitically. Start with a novel idea, one that is not an obvious extension of the reported work. Second, identify potential collaborators whose collected data may be useful in assessing the hypothesis and propose a collaboration. Third, work together to test the new hypothesis. Fourth, report the new findings with relevant coauthorship to acknowledge both the group that proposed the new idea and the investigative group that accrued the data that allowed it to be tested. What is learned may be beautiful even when seen from close up.

The PLOS family of journals has gone on record as requiring that all data for papers published in their journals be publicly available without restriction.A February 24, 2014 PLOS’ New Data Policy: Public Access to Data  declared:

In an effort to increase access to this data, we are now revising our data-sharing policy for all PLOS journals: authors must make all data publicly available, without restriction, immediately upon publication of the article. Beginning March 3rd, 2014, all authors who submit to a PLOS journal will be asked to provide a Data Availability Statement, describing where and how others can access each dataset that underlies the findings. This Data Availability Statement will be published on the first page of each article.

Many of us are aware of the difficulties in achieving this lofty goal. I am holding my breath and turning blue, waiting for some specific data.

The BMJ has expanded their previous requirements for data being available:

Loder E, Groves T. The BMJ requires data sharing on request for all trials. BMJ. 2015 May 7;350:h2373.

The movement to make data from clinical trials widely accessible has achieved enormous success, and it is now time for medical journals to play their part. From 1 July The BMJ will extend its requirements for data sharing to apply to all submitted clinical trials, not just those that test drugs or devices. The data transparency revolution is gathering pace.

I am no longer heading dissertation committees after one that I am currently supervising is completed. But if any PhD students asked my advice about a dissertation project concerning psychotherapy, I would strongly encourage them to enlist their advisor to identify and help them negotiate access to a data set appropriate to the research questions they want to investigate.

Most well-resourced psychotherapy trials have unpublished data concerning how they were implemented, with what bias and with which patient groups ending up underrepresented or inadequately exposed to the intensity of treatment presumed to be needed for benefit. A story awaits to be told. The data available from a published trial are usually much more adequate than then any graduate student could collect with the limited resources available for a dissertation project.

I look forward to the day when such data is put into a repository where anyone can access it.

until youre done In this blog post I have argued that PhD students should not take on responsibility for developing and testing a new psychotherapy for their dissertation project. I think that using data from existing published trials is a much better alternative. However, PhD students may currently find it difficult, but certainly not impossible to get appropriate data sets. I certainly am not recruiting them to be front-line infantry in advancing the cause of routine data sharing. But they can make an effort to obtain such data and they deserve all support they can get from their dissertation committees in obtaining data sets and in recognizing when realistically that data are not being made available, even when the data have been promised to be available as a condition for publishing. Advisors, please request the data from published trials for your PhD students and protect them from the heartache of trying to collect such data themselves.


Advocating CBT for Psychosis: “Ultimately it is all political.”

Political… Or just cynical?

Frida Kahlo, “Without Hope”
Frida Kahlo, “Without Hope”

Professor Paul Salkovskis and his colleagues organized a lively, thought-provoking conference at University of Bath “Understanding Psychosis and Schizophrenia: How well do we understand and what should we do to improve how we help?”

Presenters and members of the roundtable discussion panel included a number of authors of the British Psychological Society’s Understanding Psychosis and Schizophrenia. But they noticeably  avoided engaging anyone outside their tight knit group, especially speakers disagreeing with their manifesto. The Understanding Psychosis and Schizophrenia authors appeared glum and dyspeptic throughout lively discussions. The conference nonetheless went on around them. Highlights included presentations by Professors Robin Murray and Clive Adams.

In his “Genes, Social Adversity and Cannabis: how do they interact?” Professor Robin Murray gently chided the authors of the British Psychological Society’s Understanding Psychosis and Schizophrenia for their insensitivity to the suffering, debilitation, and sometimes terror posed by schizophrenia. For me, his talk clarified confusion caused by the authors of Understanding Psychosis repeatedly claiming Professor Robin Murray had endorsed their document. He did not. He is an exceptionally kind and well-mannered person and I think his polite comments at the earlier launch meeting for Understanding Psychosis were misinterpreted. His presentation at the Bath conference left no doubt where he stood.

A diagnosis of schizophrenia encompasses a wide range of conditions that will undoubtedly by sorted into a tighter, more useful categories as we use existing categories to organize the evidence we accumulate. As Joe McCleary summarized in comments on my FB wall, if we use existing – admittedly imperfect and provisional – categories, we can learn about

the nature of the individuals symptoms and experience, the likelihood and time course of improvement, recovery, and/or relapse, persistence of difficulties in particular domains (intellectual, social, emotional, adaptive functioning), which interventions might be most useful to try, what co-occurring disorders and risks are high and low (e.g., suicide, aggression, dissociation), likely levels of dependence vs independence, impacts on family, reliance on family, impacts on society, reliance on society, risk for harm (e.g., being taken advantage of or abused), etc., etc., etc.

These correlates of a diagnosis of schizophrenia check out well when we go to the available literature.

Professor Peter Kinderman who is President-Elect of the British Psychological Society, as well as an author of Understanding Psychosis was a member of the afternoon roundtable panel at Bath But he mostly sat in silence. He rejects the idea that the diagnosis has led to any progress:

Diagnostic systems in psychiatry have always been criticized for their poor reliability, validity, utility, epistemology and humanity.


The poor validity of psychiatric diagnoses—their inability to map onto any entity discernable in the real world—is demonstrated by their failure to predict course or indicate which treatment options are beneficial, and by the fact that they do not map neatly onto biological findings, which are often nonspecific and cross diagnostic boundaries.

Kinderman repeats these points in every forum he’s given until he lapses into self-plagiarism. Compare Imagine there’s no diagnosis, it’s easy if you try  to Drop the language of disorder.

What does Kinderman offer in place of diagnosis? That we respond to patients in terms ofmy paradigm their nonspecific distress, which is a “normal, not abnormal, part of human life.” This insight, according to Kinderman, places us on the “cusp of a major paradigm shift in our thinking about psychiatric disorders.”

Kinderman leaves us with sweeping declarations and no evidence to support them. He gets quite fussy when challenged. During the Roundtable Discussion, he went off on one of his usual rants, peppered by a torrent of clichés, allusions to unnamed professionals describing schizophrenia as a genetic disease, and argument by anecdote.

But what if we took seriously his suggestion that we drop diagnosis and substitute a generic distress? He concedes that many patients are helped by antipsychotic medication. But getting the best candidates for this treatment depends on the diagnostic label schizophrenia. And just as importantly, keeping patients who are likely to be poor candidates and for whom it will be ineffective, also depends on using the criteria associated with the label schizophrenia to rule out this treatment is appropriate. Unless Kinderman can come up with something else, it would seem that we risk both undermedication of those who desperately need it and overmedication of those who get more harm than benefit, if we abandon such labels.

And turning to Professor Clive Adam’s presentation organizing the available literature around the diagnostic label of schizophrenia, we can see from Cochrane reviews the likelihood that treatment with cognitive behavior therapy in the absence of medication is likely to be ineffective and not at all based on available evidence.

Clive Adams delivered a take-no-prisoners “CBT-P and medication in the treatment of psychosis: summarising best evidence.”  Adams’ presentation is captured in a blog post but its message can be succinctly stated

I just cannot see that this approach (CBTp), on average, is reaping enough benefits for people.

None of the authors of Understanding Psychosis responded to Adams’ strictly data oriented presentation. They simply mumbled among themselves.

Maybe we should simply accept that when the authors of Understanding Psychosis call for extensive discussion and dialogue, it is not what would be usually meant by those terms. They don’t want their monologue  interrupted by anything but  applause.

What the authors of Understanding Psychosis  get is that with Twitter and blogs, you cannot not engage in a dialogue when you put outrageous claims out there. You can only risk your social media identity being defined what others say.

Let’s examine what Peter Kinderman says in another monologic blog post, strikingly free of any reference to evidence, Three phrases. The post discusses three phrases that organized an international meeting concerning cognitive behavior therapy held in Philadelphia in May, 2015.

It’s probably better to read the outcomes of our discussions in peer-reviewed scientific papers and in the policy documents of our various nations. For me, however, three phrases stood out as we discussed our shared interests.

I can’t wait! But until then we have his blog.

The first phrase “Trauma-informed practice” is described

In all kinds of ways, we’re learning how psychotic experiences can relate to trauma – in childhood and as adults. And we’re learning how the ways in which we purport to care for people – with the labels that we attach to their problems, with the explanations (and non-explanations) that we propose, and especially with the treatments that we use (and occasionally impose, even forcefully) – can potentiate experiences of trauma. So I welcome the fact that there appears to be increasing discussion of how we might base our therapies, and indeed our whole service design philosophy, on an appreciation of the role of trauma, for many people, in the development of their difficulties.

Presumably the forthcoming “peer-reviewed scientific papers” will allow us to evaluate the evidence for the efficacy of “trauma-informed” treatment of schizophrenia. I can’t find it. I don’t see where any of the randomized trials of CBT for psychosis that have been conducted are organized around this concept. Does Kinderman have any sense of the history or usage of “trauma-informed” in the United States and elsewhere?

mindbody connection“Trauma informed practice” typically refers to an approach that is more hermeneutic than scientific. The assumption is made that psychological trauma causes both mental disorder and physical illnesses.

Understanding Psychosis takes for granted that traumatic experiences are at the root of most psychotic disturbance. When they invoke evidence at all, it is the work of one of its authors, Richard Bentall. The literature concerning the role of child adversity in psychotic disturbance is  methodologically flawed, but even if we accepted at face, the effect sizes it generates would not justify the assumptions that trauma is behind all psychotic experiences.

In the United States, evidence-based, research-oriented clinicians are skeptical of the slippery slope whereby calls for “trauma-informed practice” too often lead down to nonsense about trauma being embodied in organs and peripheral tissue, not just the nervous system. Untrained and incompetent therapists insist that conditions like diabetes and asthma are linked to trauma, and if patients cannot report relevant traumatic experiences, there should be an effort to recover their repressed memories. Serious damage was done to a lot of patients and their families before the fad of recovering memories of sexual abuse and participation in devil worshiping cults was put down with legal action.

Kinderman’s second phrase is “CBT-informed practice”

It’s hardly a surprise that the acronym ‘CBT’ means slightly different things to different people.

There’s a valuable debate about ‘fidelity’ (whether a therapist is or is not adherent to the accepted elements of CBT). But there’s also an appreciation that, in the field of psychosocial interventions in mental health care, common therapeutic factors, the fundamental role of a good ‘therapeutic alliance’ (a relationship based on respect) and the heterogeneity of individual experiences means that we are now much more likely to talk about “CBT-informed practice”. Again, for me, this is welcome. I believe that it not only allows for valuable innovation and development of psychosocial interventions, but also permits an appreciation of the uniqueness of each person’s experience.

The retreat from any claim to being evidence-based continues. If a therapy carries the branding of evidence-based, it is assumed that it is delivered with some fidelity to what has been tested in clinical trials. Branding as “evidence-based” cannot be retained unless the innovations and further development are themselves subjected to clinical trials. “Evidence-based, is not a branding they can be casually transferred to new products without testing.

Kinderman’s final phrase is “_ ultimately, it’s all political_.”

The attendees of these meetings are all applied scientists (although some have some influential roles in shaping healthcare policies). But it was interesting that many of our discussions referred back to the social circumstances of those people accessing our services, and on the political decisions taken about how those services are commissioned, planned and delivered We discussed, for instance, the role of social determinants of health generally and mental health in particular. We discussed how different psychological and social problems seem to have similar social determinants (and the implications of this). We talked about how trauma, discrimination, racism, the struggles of undocumented migrants and the pressures on unemployed people can affect their mental health. We discussed how people access high-quality healthcare in different states and nations, and we discussed how political decisions – such as those related to involuntary detention and compulsory treatment, the funding of healthcare and provision of different forms of care – impact on our clients. We also discussed how, as a group of professionals, we are increasingly being asked to contribute to these debates.

So for me, it was a very positive and encouraging trip. I am – I remain – confident that conventional CBT, a form of one-to-one therapy that of course has its limitations, can be very positive for people experiencing psychosis. But, given the views I hold about the fundamental nature of mental heath and wellbeing, the phrases that echo most encouragingly from last week’s meeting are “trauma-informed practice”, CBT-informed practice” and “ultimately, it’s all political.”

I think I finally get it. Kinderman is saying that his followers should hold on to claims of being evidence-based, even in the face of clinical trials and meta-analyses providing evidence to the contrary. And they should incorporate elements of “trauma-based practice.” This is not taking  seriously principles of evidence-based evaluation of best practices. but that is not what Understanding Psychosis is about.

Advocating CBT is political, not evidence-based, but we need the latter label for credibility and controlling credentialing.

This is cynical, not political.

Distress- the 6th vital sign for cancer patients?

A third or more of cancer patients experience significant psychological distress following their diagnosis. Yet a much smaller proportion receives any psychosocial services.

Cancer-Care-for-the-Whole-Patient-Meeting-Psychosocial-Health-NeedsThis situation is counter to recommendations from a number of sources, including the US Institute of Medicine report, Care of the Whole Patient, Meeting Psychosocial Health Needs. The gap between the proportion of patients experiencing distress and those getting services has led to widespread calls for implementing routine screening for distress. The assumption is that the breakdown between being distressed and getting services is patients getting identified and being referred to appropriate services.

There have been intense national and international campaigns by professional organizations first to recommend implementation of screening and then to mandate it as a condition of accreditation of cancer care settings. Increasingly, campaigns are organized around the slogan “distress is the sixth vital sign.” Promoters have sufficient political clout to get the slogan into the titles of articles in scientific journals, often with leaders of the field of psycho-oncology lending support as co-authors.

Holland, J. C., & Bultz, B. D. (2007). The NCCN guideline for distress management: a case for making distress the sixth vital sign. Journal of the National Comprehensive Cancer Network, 5(1), 3-7.

Bultz, B. D., & Johansen, C. (2011). Screening for distress, the 6th vital sign: where are we, and where are we going? PsychoOncology, 20(6), 569-57.

Watson, M., & Bultz, B. D. (2010). Distress, the 6th vital sign in cancer care. Psycho-oncologie, 4(3), 159-163.

The push to make distress accepted as the sixth vital sign for cancer patients is modeled after efforts to get pain designated as the fifth vital sign for general medical patients. In the late 1980s, it was recognized that pain management was grossly inadequate, with many patients’ pain going undetected and untreated. Pain was designated the fifth vital sign in general medical settings, with guidelines set for assessment and improved referral and treatment. These guidelines were first simply recommendations, but they grew to be mandated, with performance goals and quality of care indicators established to monitor their implementation in particular settings.

What is distress? Maybe not what you think

distressDistress was originally defined narrowly in terms of stress and anxiety and depressive symptoms. Early screening recommendations were for quick assessment with a distress thermometer modeled after the simple pain rating scale adopted in the campaign for pain as the fifth vital sign. Efforts were made to validate distress thermometers in terms of their equivalence to longer self-report measures of anxiety and depression, and in turn, interview-based measures of psychiatric disorder. Ratings on a distress thermometer were not seen as a substitute for longer assessments, but rather as a screening for whether longer assessments were needed. Patients who screened above a cutpoint on a distress thermometer would be identified for further assessment.

“Distress” was the preferred term, rather than stress or anxiety or depressive symptoms, because it was viewed as more neutral and not necessarily indicating that someone who wanted services had a psychiatric disorder. It was believed that many cancer patients did not seek services out of fear of being stigmatized and this vague term might avoid that raising that barrier to treatment.

However, the definition has now been extended to include a much broader range of needs and concerns. The widely cited definition of the NCCN of distress is

An unpleasant emotional experience of a psychological, social and/or spiritual nature which extends on a continuum from normal feelings of vulnerability, sadness and fears to disabling problems  such as depression, anxiety, panic, social isolation and spiritual crisis.

The emphasis is still on quick screening, but the distress thermometer is typically supplemented with a checklist of common problems. Particular items vary across recommendations for screening, but the checklist is meant to be a brief supplement to the distress thermometer. Canadians who have taken the lead in promoting distress as the sixth vital sign recommend a Modified Problem Checklist (PCL).

Click to Enlarge
Click to Enlarge

This list contains the 7 most common practical problems in our  settings (accommodation, transportation, parking, drug coverage, work/school, income/finances, and groceries); and 13 psychosocial problems (burden to others, worry about family/friends, talking with family, talking with medical team, family conflict, changes  in appearance; alcohol/drugs, smoking, coping, sexuality, spirituality, treatment decisions and  sleep). Participants indicate the presence or absence of each problem in the preceding week.

Despite the range of problems being assessed, the assumption is still that the predominant form of services that patients will seek as a result of being screened is some form of psychological counseling or psychotherapy.

Interventions usually assume one of four common forms: psychoeducation, cognitive-behavioural training (group or individual), group supportive therapy, and individual supportive therapy.

Support by professional organizations for screening of distress is quite widespread. Well-organized proponents exercise their political clout in making expression of enthusiasm for screening an obligatory condition for publishing in psycho-oncology journals. Dissenting critics and embarrassing data can be summarily sandbagged. Even when embarrassing data makes it through the review process, authors may be surprised to discover that their articles are accompanied by disparaging commentaries, to which they have not been offered an opportunity to respond. I have been publishing for over thirty years, and I have never before encountered the level of bullying that I have seen with papers concerning screening.

screeningYet, promotion of implementing routine screening for distress is based only on consensus of like-minded professionals, not the systematic evaluation of evidence that medical screening generally requires. Screening tests are common in medicine. The standard criteria for adopting a recommendation for screening is that it can be shown to improve patient outcomes beyond simply making available to patients and their providers the resources that are accessed by screening.

It is not sufficient that screening address a serious problem, it must also lead to improved outcomes. For instance, it is now recognized that although prostate cancer is a common and threatening disease, routine screening for prostate cancer of men without symptoms is likely to lead to overtreatment and side effects incontinence and loss of a sexual functioning, without any improvement in overall survival. Campaigns to promote  prostate specific antigen (PSA) testing have largely been discontinued. More generally, Choosing Wisely committees are systematically reevaluating the evidence for common forms of screening and withdrawing recommendations. Some well established screening procedures are losing their recommendation.

We undertook a systematic review of the literature concerning screening for distress and failed to find sufficient evidence to recommend it.

scrrening art
Click to Enlarge

Because of the lack of evidence of beneficial effects of screening cancer patients for distress, it is premature to recommend or mandate implementation of routine screening.

Other reviews have come to more favorable evaluations of screening,  arrived at by including nonrandomized trials or trials that chose number of referrals being made is a surrogate outcome in place of distress being reduced. However, for a patient getting referred to experience reduced distress, a complex pathway has to be crossed, starting with acceptance and completion of the referral and including receiving appropriate treatment at an appropriate level of intensity and frequency. Referrals, particularly when they are to outside the cancer’s care setting, are notoriously black holes. There are a few systematic data concerning the fate of referrals, but the general consensus is that many, perhaps most are not completed.

My colleagues and I also noted that the professional recommendations for screening have not been developed according to the standard procedures for making such recommendations. Development of professional guidelines is supposed to follow an orderly process of

  • Assembling a diverse and representative group of relevant stakeholders,
  • Systematically reviewing the literature with transparency in search criteria and method of synthesis and interpretation.
  • Publication of preliminary recommendations for public comment.
  • Revision of preliminary recommendations taking into account feedback.

In contrast, recommendations for screening have simply been released by professional groups assembled on the basis of previously known loyalty to the conclusion and with inattention to the lack of evidence showing that screening would improve outcomes. Notably missing from the “consensus” groups recommending screening are patients, frontline clinicians who will have to implement screening, and professionals from the community who would have to receive and evaluate referrals, notably primary-care physicians in most countries.

Nonetheless, the push for routine screening of cancer patients for psychological distress continues to gain momentum, with goals being set in many settings for the minimal proportion of cancer patients who must be screened.

The slogan “distress is the sixth vital sign” is not a testable empirical statement of the order of “depression is a significant risk factor for cardiac mortality.” Rather, it is best evaluated in terms of the use of terms, particularly “vital sign” and whether adopting slogan improves patient outcomes. Let us look at the notion of vital sign and then at the outcome of efforts to adopt pain has the fifth vital sign.

What is a vital sign?

According to Wikipedia

vital_signs_bot_image_blkVital signs are measures of various physiological statistics, often taken by health professionals, in order to assess the most basic body functions. Vital signs are an essential part of a case presentation. The act of taking vital signs normally entails recording body temperature, pulse rate (or heart rate), blood pressure, and respiratory rate, but may also include other measurements. Vital signs often vary by age.

In an excellent discussion of vital signs, Lucy Hornstein states

If someone other than the patient can’t see, hear, palpate, percuss, or measure it, it’s a symptom. Anything that can be perceived by someone else is a sign.


Vital signs are measured…and yield numeric results. Normal ranges are defined; values that fall outside those normal ranges are described with specific words (eg, bradycardia, tachypnea, hypothermia, hypertension).

A cough is a sign, but a headache is a symptom. Not that headaches are less “real,” they are just different.

With the standard definitions, pain cannot be a vital sign, only a symptom. The labeling of pain as a vital sign is metaphoric involves twisting the definition of a sign.

Something I will note now, but come back to later in the blog: patients cannot really argue against results of assessment of a vital sign. If repeated blood pressure readings indicate dystolic pressure above established cutoff, it is not left for the patient to argue “no, I do not have hypertension.” The notion is that a vital sign is objective and not subject to validation by patient self-report, although what to do about the vital sign may be interpreted in terms of other data about the patient, some of it from self-report. The point is that calling a measurement a “vital sign” claims particular authority for the results.

There have been numerous proposals for designating other vital signs in need of routine assessment and recording in medical records. These include (here I have extended the list from Wikipedia):

Pain as the fifth vital sign

Okay, so it is only metaphorically that pain can be considered a fifth vital sign. But what was nonetheless accomplished by designating it so in terms of improving its management? What can be learned that can be applied to designating distress as the sixth vital sign?

pain buttonThe pain-as-fifth-vital-sign (P5VS) campaign simply started with publicizing of high levels of untreated pain in medical populations, which in turn led to the consensus statements and mandated screening, similar to what is now occurring with declaring distress as the sixth vital sign.

1992. US Agency for Healthcare Quality Research (AHQR), issued guidelines documenting that half of surgical patients did not receive adequate post-surgical medication. The guidelines declared patients had a “right” to adequate pain measurement.

1995. American Pain Society (APS) issued a landmark consensus statement with guidelines for a quality improvement approach towards the treatment of acute and cancer pain [10], expanding upon its 1989 guidelines [11] for the treatment of pain. In his Presidential Address, Dr. James Campbell presented the idea of evaluating pain as a vital sign.

1998. The Veterans Health Administration (VHA) launched a national  P5VS initiative to pain thermometerimprove pain management for its patients, starting with documentation in the electronic medical record of assessment of patients’ self-report of pain.  It required use of a Numeric Rating Scale (NRS) in all clinical encounters.

P5VS campaigns were based on professional consensus, not appraisal of best evidence. When studies were later conducted, results did not demonstrate that the fifth vital sign campaign improved patient outcomes.

The title of a study of 300 consecutive patients before the start of the VA initiative and 300 afterwards says it all:

Measuring Pain as the 5th Vital Sign Does Not Improve Quality of Pain Management

The study examined 7 process indicators of quality of pain management fail to identify any improvement in

  • Subjective provider assessment (49.3% before versus 48.7% after).
  • Occurrence of a pain exam (26.3% versus 26.0%).
  • Orders to assess pain (11.7% versus 8.3%).
  • New pain medication prescribed (8.7% versus 11.0%)
  • Change in existing pain medication prescription (6.7%, 4.3%)
  • Other pain treatment (11.7% versus 13.7%).
  • Recording of follow-up plans (10.0%, 8.7%).

The initiative required that “patients indicating a score of four or about on a pain scale should receive comprehensive pain assessment and prompt intervention . . ’’

In a subsample of patients who reported substantial pain:

  • 22% had no attention to pain documented in the medical record of the visit which they reported.
  • 52% received no new therapy for pain at that visit.
  • 27% had no further assessment documented.

Our investigation of the P5VS…initiative at a single VA site has shown the routine documentation of pain levels, even with system-wide support and broad-based provider education, was ineffective in improving the quality-of-care.

Carly Simon – Haven’t Got Time for the Pain

What was going wrong? It appears that front line clinicians making the pain assessments lacked the time or skills to take effective action. Pain assessments were typically conducted in encounters for which there are other reasons for the visit. Furthermore, pain assessments were collected outside of a clinical interaction in which the context or history could be discussed, which could likely lead to invalid ratings. When patients were asked only to rate current pain, it was unknown whether they took into account how much the pain bothered them, whether it was acute or chronic, or whether it reflected any change from past levels, all meaningful considerations. Other clinicians in the system either did not receive the ratings in a timely fashion or lacked the context to interpret the ratings.

Other studies [1, 2] similarly demonstrated that the P5VS campaign was by itself ineffective:

One potential reason for the insensitivity of routine pain screening in these studies is that all were conducted in outpatient primary and specialty care settings where chronic persistent or intermittent pain is much more common than acute pain. Routine pain screening that focuses on pain intensity “now” may not be sufficiently sensitive to detect important chronic pain that occurs episodically or varies with activity. In the VA primary care, the vast majority of pain problems are longstanding in nature, so sensitivity for chronic pain is important for any pain screening strategy in this setting.

The mandate that unrelieved pain must be addressed soon led to ineffective, inappropriate treatment that was not based on proper assessment. There was an increase in diagnostic tests that only confirmed existence of pain that was difficult to treat, notably chronic back pain. In the decade after the campaign to reduce pain was launched, costs of treating chronic back pain escalated without any demonstrated improvement in patient outcomes.

The guidelines had been promulgated with claims that addiction to opiates prescribed for acute pain was rare. But the evidence for that was only a brief 1980 letter in the New England Journal of Medicine indicating only four instances in treatment of 12,000 patients.

The campaign to improve pain control using routine ratings had an effect unanticipated by its proponents.

Dispensing opioids has almost doubled according to National Health and Nutrition Examination Survey data indicating that from 1988 to 1994 a total of 3.2% of Americans reported using opioids for pain, whereas from 2005 to 2008 a total of 5.7% reported use.

This significant increase has been associated with serious consequences, including an estimated 40 deaths per day due to prescription opioids.

Put simply

Improving pain care may require attention to all aspects of pain management, not just screening.

The pain as the fifth vital sign campaign involved mandated assessment of pain with a simple numerical scale at every clinical encounter, regardless of the reason for visit. The rating was typically obtained in the absence of any effort to talk to patients or to examine them in an effort to determine what were likely the multiple sources of their pain, its history, and their goals. Indeed, collecting these simple ratings may have become a substitute for having such discussions. The number on the rating scale came to characterize the patient for purposes of clinical decision-making, and may have led to overtreatment including escalating prescription of pain medication.

Is a good idea to consider distress the sixth vital sign?

Like pain, distress is not a vital sign by the conventional definition of the term. Yet to label it as such suggests that there is some sort of objective procedure involved in collecting ratings on a distress thermometer from patients.

Generally ignored in the promotion of screening is that most patients who indicate distress above established thresholds do not wish to receive a psychosocial service that they are not already receiving. Current guidelines for screening do not have a requirement of asking patients where they have any need for services. Instead, their response to the distress thermometer is used to tell them that they need intervention, with an emphasis on counseling.

When asked directly, most distressed patients reject the need for psychosocial services that they are not already getting, often outside the cancer center.  A rather typical study found that 14% definitely and an additional 29% maybe wanted to talk with a professional about their problems. Patients variously report

  • Already receiving services.
  • Believing they can solve the problems themselves.
  • Concentrating on the treating their physical illness takes precedence over receiving psychosocial and supportive services.
  • Services being offered to them are not needed, timely, or what they preferred.

A heightened score on a distress thermometer is a poor indication of whether patients are interested in receiving services that are listed on a screening sheet. Most do not want to receive a service, but most receiving services are not distressed. Think about it, looking at the problems listed on the screening form above. Many of the problems would be endorsed without patients having a heightened distress score. This poses for the dilemma for any interpretation of a score on a distress thermometer as if it were a vital sign.

Overall, thinking about distress as the sixth vital sign creates the illusion that a score on a distress thermometer is an authoritative, objective standalone indicator, much like a blood pressure reading. Actually, scores on a distress thermometer need to be discussed and interpreted. If distress is taken too seriously as the sixth vital sign, there is a risk that patients who do not meet the cut off for clinically significant distress will be denied an opportunity to discuss the problems that they might otherwise seek.

My colleagues and I undertook a study where we used results of screening for distress to attempt to recruit a sample of patients for an intervention trial evaluating problem-solving therapy as a way of reducing distress. It proved to be a discouragingly inefficient process.

  • We screened 970 cancer patients, of whom 423 were distressed, and, of these, 215 indicated a need for services. However, only 36 (4%) consented to participate in the intervention study.
  • 51% of the distressed patients reported having no need for psychosocial services and an additional 25% were already receiving services for their needs.
  • Overall, we had to screen 27 patients in order to recruit a single patient, with 17 hours of time required for each patient recruited.

Consider could have been accomplished if these 17 hours of were used instead to talk to patients who had indicated they wanted to talk to someone about their problems.

Designating distress as the sixth vital sign suggest false objectivity and validity to a procedure that has not demonstrated improvement in patient outcomes. It is an advertising slogan that is likely to prove ineffective and misdirect resources, just as the P5VS did.