A science-based medicine skeptic struggles with his as-yet medically unexplained pain and resists alternative quack treatments

Paul: “For three years I kept my faith that relief had to be just around the corner, but my disappointment is now as chronic as my pain. Hope has become a distraction.”

mind the brain logo

Chronic pain and tragic irony…

Paul: “For three years I kept my faith that relief had to be just around the corner, but my disappointment is now as chronic as my pain. Hope has become a distraction.”

Paul Ingraham is quite important in the Science-Based Skeptics movement and in my becoming involved in it. He emailed me after a long spell without contact. He wanted to explain how he had been out of touch. His life had been devastated by as-yet medically unexplained pain and other mysterious symptoms.

Paul  modestly describes himself at his blog site as “a health writer in Vancouver, Canada, best known for my work debunking common myths about treating common pain problems on PainScience.com. I actually make a living doing that. On this blog, I just mess around.  ~ Paul Ingraham (@painsci, Facebook).”

Some of Paul’s posts at his own blog site


on fire


Paul’s Big Self-Help Tutorials for Pain Problems are solidly tied to the best peer-reviewed evidence.

Detailed, readable tutorials about common stubborn pain problems & injuries, like back pain or runner’s knee.

Many common painful problems are often misunderstood, misdiagnosed, and mistreated. Made for patients, but strong enough for professionals, these book-length tutorials are crammed with tips, tricks, and insights about what works, what doesn’t, and why. No miracle cures are for sale here — just sensible information, scientifically current, backed up by hundreds of free articles and a huge pain and injury science bibliography.



Paul offered me invaluable assistance and support when I began blogging at the prestigious Science Based Medicine. See for instance, my:

Systematic Review claims acupuncture as effective as antidepressants: Part 1: Checking the past literature


Is acupuncture as effective as antidepressants? Part 2. Blinding readers who try to get an answer

I have not consistently blogged there, because my topics don’t always fit. Whenever I do blog there, I learn a lot from  the wealth of thoughtful comments I received.

I have great respect for Science Based Medicine’s authoritative, well documented and evidence-based analyses. I highly recommend the blog for those who are looking for sophistication  delivered in a way that an intelligent lay person could understand.

What’s the difference between Sciencebased medicine (SBM) versus evidence-based medicine (EBM)?

I get some puzzlement every time I bring up this important distinction – Bloggers at SBM frequently make a distinction between science-based- and evidence-based- medicine. They offer careful analyses of unproven treatments like acupuncture and homeopathy. Proponents of these treatment increasingly sell them as evidence-based, citing randomized trials that do not involve an active treatment. The illusion of efficacy is often created by the positive expectations and mysterious rituals with which these treatments are delivered. Comparison treatments in these studies often lack this boost, particularly when tested in in unblinded comparisons.

The SBM bloggers like to point out that there are no plausible tested scientific mechanisms by which these treatments might conceivably work. The name of  blog,  Science-Based Medicine calls  attention to their higher standards for considering treatments efficacious: to be considered science based medicine, they have to be proven as effective as evidence-based active treatments, and have to have a mechanism beyond nonspecific, placebo effects.

Paul Ingram reappears from a disappearance.

Paul mysteriously disappeared for a while. Now he’s reemerged with a tale that is getting a lot of attention. He gave me permission to blog about excerpts. I enclose a link to the full story that I strongly recommend.

Paul Ingram title


A decade ago I devoted myself to helping people with chronic pain, and now it’s time to face my ironic new reality: I have serious unexplained chronic pain myself. It may never stop, and I need to start learning to live with it rather than trying to fix it.

I have always been “prone” to aches and pains, and that’s why I became a massage therapist and then moved on to publishing PainScience.com. But that tendency was a pain puppy humping my leg compared to the Cerberus of suffering that’s mauling me now. I’ve graduated to the pain big leagues.

For three years I kept my faith that relief had to be just around the corner, but my disappointment is now as chronic as my pain. Hope has become a distraction. I’ve been like a blind man waiting for my sight to return instead of learning braille. It’s acceptance time.

Paul describes how is pain drove him into hiding.

… why I’ve become one of those irritating people who answers every invitation with a “maybe” and bails on half the things I commit to. I never know what I’m going to be able to cope with on a given day until it’s right in front of me.

He struggled to define the problem:

Mostly widespread soreness and joint pain like the early stages of the flu, a parade of agonizing hot spots that are always on the verge of breaking my spirit, and a lot of sickly fatigue. All of which is easily provoked by exercise.

But there was a dizzying array of other symptoms…

Any diagnosis would be simply a label, not an explanation.

Nothing turned up in a few phases of medical investigation in 2015 and 2016. My “MS hug” is not caused by MS. My thunderclap headaches are not brain bleeds. My tremors are not Parkinsonian. I am not deficient in vitamins B or D. There is no tumour lurking in my chest or skull, nor any markers of inflammation in my blood. My heart beats as steadily as an atomic clock, and my nerves conduct impulses like champs.

Paul was not seriously tempted by alternative and complementary medicine

I am not tempted to try alternative medicine. The best of alt-med is arguably not alternative at all — e.g. nutrition, mindfulness, relaxation, massage, and so on — and the rest of what alt-med offers ranges from dubious at best to insane bollocks at the worst. You can’t fool a magician with his own tricks, and you can’t give false hope to an alt-med apostate like me: I’ve seen how the sausage is made, and I feel no surge of false hope when someone tells me (and they have) “it’s all coming from your jaw, you should see this guy in Seattle, he’s a Level 17 TMJ Epic Master, namaste.” Most of what sounds promising to the layperson just sounds like a line of bull to me.

Fascinating how many people clearly think Paul’s story was almost identical to their own.

All these seemingly “identical” cases have got me pondering: syndromes consist of non-specific symptoms by definition, and batches of such symptoms will always seem more similar than they actually are… because blurry pictures look more alike than sharp and clear ones. Non-specific symptoms are generalized biological reactions to adversity. Anxiety can cause any of them, and so can cancer. Any complex cases without pathognomic (specific, defining) symptoms are bound to have extensive overlap of their non-specific symptoms.

There are many ways to be sick, and relatively few ways to feel bad.

Do check out his full blog post. http://www.paulingraham.com/chronic-pain-tragic-irony.html

Flawed meta-analysis reveals just how limited the evidence is mapping meditation into specific regions of the brain

The article put meaningless, but reassuring effect sizes into the literature where these numbers will be widely and uncritically cited.

mind the brain logo

“The only totally incontrovertible conclusion is that much work remains to be done…”.

lit up brain not in telegraph article PNG

Authors of a systematic review and meta-analysis of functional neuroanatomical studies (fMRI and PET) of meditation were exceptionally frank in acknowledging problems relating the practice of meditation to differences in specific regions of the brain. However, they did not adequately deal with problems hiding in plain sight. These problems should have discouraged integration of this literature into a meta-analysis and the authors’ expressing the strength of the association between meditation and the brain in terms of a small set of moderate effect sizes.

The article put meaningless, but reassuring effect sizes into the literature where these numbers will be widely and uncritically cited.

An amazing set of overly small studies with evidence that null findings are being suppressed.

Many in the multibillion mindfulness industry are naive or simply indifferent to what constitutes quality evidence. Their false confidence that “meditation changes the brain*” can be bolstered by selective quotes from this review seemingly claiming that the associations are well-established and practically significant. Readers who are more sophisticated may nonetheless be mislead by this review, unless they read beyond the abstract and with appropriate skepticism.

Read on. I suspect you will be surprised as I was about the small quantity and poor quality of the literature relating the practice of meditation to specific areas of the brain. The colored pictures of the brain widely used to illustrate discussions of meditation are premature and misleading.

As noted in another article :

Brightly coloured brain scans are a media favourite as they are both attractive to the eye and apparently easy to understand but in reality they represent some of the most complex scientific information we have. They are not maps of activity but maps of the outcome of complex statistical comparisons of blood flow that unevenly relate to actual brain function. This is a problem that scientists are painfully aware of but it is often glossed over when the results get into the press.

The article is

Fox KC, Dixon ML, Nijeboer S, Girn M, Floman JL, Lifshitz M, Ellamil M, Sedlmeier P, Christoff K. Functional neuroanatomy of meditation: A review and meta-analysis of 78 functional neuroimaging investigations. Neuroscience & Biobehavioral Reviews. 2016 Jun 30;65:208-28.


Keep in mind how few readers go beyond an abstract in forming an impression of what an article shows. More readers “know” what the meta analysis found solely based on their reading the abstract , relative to the fewer people who read both the article and the supplementary material).

Meditation is a family of mental practices that encompasses a wide array of techniques employing distinctive mental strategies. We systematically reviewed 78 functional neuroimaging (fMRI and PET) studies of meditation, and used activation likelihood estimation to meta-analyze 257 peak foci from 31 experiments involving 527 participants. We found reliably dissociable patterns of brain activation and deactivation for four common styles of meditation (focused attention, mantra recitation, open monitoring, and compassion/loving-kindness), and suggestive differences for three others (visualization, sense-withdrawal, and non-dual awareness practices). Overall, dissociable activation patterns are congruent with the psychological and behavioral aims of each practice. Some brain areas are recruited consistently across multiple techniques—including insula, pre/supplementary motor cortices, dorsal anterior cingulate cortex, and frontopolar cortex—but convergence is the exception rather than the rule. A preliminary effect-size meta-analysis found medium effects for both activations (d = 0.59) and deactivations (d = −0.74), suggesting potential practical significance. Our meta-analysis supports the neurophysiological dissociability of meditation practices, but also raises many methodological concerns and suggests avenues for future research.

The positive claims in the abstract

“…Found reliably dissociable patterns of brain activation and deactivation for four common styles of meditation.”

“Dissociable activation patterns are congruent with the psychological and behavioral aims of each practice.”

“Some brain areas are recruited consistently across multiple techniques”

“A preliminary effect-size meta-analysis found medium effects for both activations (d = 0.59) and deactivations (d = −0.74), suggesting potential practical significance.”

“Our meta-analysis supports the neurophysiological dissociability of meditation practices…”

 And hedges and qualifications in the abstract

“Convergence is the exception rather than the rule”

“[Our meta-analysis] also raises many methodological concerns and suggests avenues for future research.

Why was this systematic review and meta-analysis undertaken now?

A figure provided in the article showed a rapid accumulation of studies of mindfulness in the brain in the past few years, with over 100 studies now available.

However, the authors systematic search yielded “78 functional neuroimaging (fMRI and PET) studies of meditation, and used activation likelihood estimation to meta-analyze 257 peak foci from 31 experiments involving 527 participants.” About a third of the studies identified in a search provided usable data.

What did the authors want to accomplish?

Taken together, our central aims were to: (i) comprehensively review and meta-analyze the existing functional neuroimaging studies of meditation (using the meta-analytic method known as activation likelihood estimation, or ALE), and compare consistencies in brain activation and deactivation both within and across psychologically distinct meditation techniques; (ii) examine the magnitude of the effects that characterize these activation patterns, and address whether they suggest any practical significance; and (iii) articulate the various methodological challenges facing the emerging field of contemplative neuroscience (Caspi and Burleson, 2005; Thompson, 2009; Davidson, 2010; Davidson and Kaszniak, 2015), particularly with respect to functional neuroimaging studies of meditation.

Said elsewhere in the article:

Our central hypothesis was a simple one: meditation practices distinct at the psychological level (Ψ) may be accompanied by dissociable activation patterns at the neurophysiological level (Φ). Such a model describes a ‘one-to-many’ isomorphism between mind and brain: a particular psychological state or process is expected to have many neurophysiological correlates from which, ideally, a consistent pattern can be discerned (Cacioppo and Tassinary, 1990).

The assumption is meditating versus not-meditating brains should be characterized by  distinct, observable neurophysiological pattern. There should also be distinct, enduring changes in the brain in people who have been practicing meditation for some time.

I would wager that many meditation enthusiasts believe that links to specific regions are already well established. Confronted with evidence to the contrary, they would suggest that links between the experience of meditating and changes in the brain are predictable and are waiting to be found. It is that kind of confidence that leads to the significance chasing and confirmatory bias currently infecting this literature.

Types of meditation available for study

Quantitative analyses focused on four types of meditation. Additional terms of meditation did not have sufficient studies and so were examined qualitatively. Some studies of the four provided within-group effect size, whereas other studies provided between-group effect sizes.

Focused attention (7 studies)

Directing attention to one specific object (e.g., the breath or a mantra) while monitoring and disengaging from extraneous thoughts or stimuli (Harvey, 1990, Hanh, 1991, Kabat-Zinn, 2005, Lutz et al., 2008b, Wangyal and Turner, 2011).

Mantra recitation (8 studies)

Repetition of a sound, word, or sentence (spoken aloud or silently in one’s head) with the goals of calming the mind, maintaining focus, and avoiding mind-wandering.

Open monitoring (10 studies)

Bringing attention to the present moment and impartially observing all mental contents (thoughts, emotions, sensations, etc.) as they naturally arise and subside.

Loving-kindness/compassion (6 studies)

L-K involves:

Generating feelings of kindness, love, and joy toward themselves, then progressively extend these feelings to imagined loved ones, acquaintances, strangers, enemies, and eventually all living beings (Harvey, 1990, Kabat_Zinn, 2005, Lutz et al., 2008a).

Similar but not identical, compassion meditation

Takes this practice a step further: practitioners imagine the physical and/or psychological suffering of others (ranging from loved ones to all humanity) and cultivate compassionate attitudes and responses to this suffering.

In addition to these four types of meditation, three others can be identified, but so far have only limited studies of the brain: Visualization, Sense-withdrawal and Non-dual awareness practices.

A dog’s breakfast: A table of the included studies quickly reveals a meta-analysis in deep trouble

studies included

This is not a suitable collection of studies to enter into a meta-analysis with any expectation that a meaningful, generalizable effect size will be obtained.

Most studies (14) furnish only pre-post, within-group effects for mindfulness practiced by long time practitioners. Of these 14 studies, there are two outliers with 20 and 31 practitioners. Otherwise the sample size ranges from 4 to 14.

There are 11 studies furnishing between-group comparisons between experienced and novice meditators. The number of participants in the smaller cell is key for the power of between-group effect sizes, not the overall sample size. In these 11 studies, this ranged from 10 to 22.

It is well-known that one should not combine within- and between- group effect sizes in meta analysis.  Pre-/post-within-group differences capture not only the effects of the active ingredients of an intervention, but nonspecific effects of the conditions under which data are gathered, including regression to the mean. These within-group differences will typically overestimate between-group differences. Adding a  comparison group and calculating between-group differences has the potential for  controlling nonspecific effects, if the comparison condition is appropriate.

The effect sizes based on between-group differences in these studies have their own problems as estimates of the effects of meditation on the brain. Participants were not randomized to the groups, but were selected because they were already either experienced or novice meditators. Yet these two groups could differ on a lot of variables that cannot be controlled: meditation could be confounded with other lifestyle variables: sleeping better or having a better diet. There might be pre-existing differences in the brain that made it easier for the experienced meditators to have committed to long term practice. The authors acknowledge these problems late in the article, but they do so only after discussing the effect sizes they obtained as having substantive importance.

There is good reason to be skeptical that these poorly controlled between-group differences are directly comparable to whatever changes would occur in experienced meditators’ brains in the course of practicing meditation.

It has been widely appreciated that neuroimaging studies are typically grossly underpowered, and that the result is low reproducibility of findings. Having too few participants in a  study will likely yield false negatives because of an inability to achieve the effects needed to obtain significant findings. Small sample size means a stronger association is needed to be significant.

Yet, what positive findings (i.e., significant) are obtained will of necessity be larger likely to be exaggerated and not reproducible with a larger sample.

Another problem with such small cell sizes is that it cannot be assumed that effects are due to one or more participants’ differences in brain size or anatomy. One or a small subgroup of outliers could drive all significant findings in an already small sample. The assumption that statistical techniques can smooth these interindividual differences depends on having much larger samples.

It has been noted elsewhere:

Brains are different so the measure in corresponding voxels across subjects may not sample comparable information.

How did the samples get so small? Neuroanatomical studies are expensive, but why did Lazar et al (2000) have 5 rather 6 participants, or only the 4 participants that Davanger et had? Were from some participants dropped after a peeking at the data? Were studies compromised by authors not being able to recruit intended numbers of participants and having to relax entry criteria? What selection bias is there in these small samples? We just don’t know.

I am reminded of all the contentious debate that has occurred when psychoanalysts insisted on mixing uncontrolled case-series with randomized trials in the same meta-analyses of psychotherapy. My colleagues and I showed this introduces great distortion  into the literature . Undoubtedly, the same is occurring in these studies of meditation, but there is so much else wrong with this meta analysis.

The authors acknowledge that in calculating effect sizes, they combined studies measuring cerebral blood flow (positron emission tomography; PET) and blood oxygenation level (functional magnetic resonance imaging; fMRI). Furthermore, the meta-analyses combined studies that varied in the experimental tasks for which neuroanatomical data were obtained.

One problem is that even studies examining a similar form of meditation might be comparing a meditation practice to very different baseline or comparison tasks and conditions. However, collapsing across numerous different baselines or control conditions is a common (in fact, usually inevitable) practice in meta_analyses of functional neuroimaging studies…

So, there are other important sources of heterogeneity between these studies.

A generic forest plot. This article did not provide one.

It’s a pity that the authors did not provide a forest plot [How to read  a forest plot.]  graphically showing the confidence intervals around the effect sizes being entered into the meta-analysis.

But the authors did provide a funnel plot that I found shocking. [Recommendations for examining and interpreting funnel plot] I have never seen one like, except when someone has constructed an artificial funnel plot to make a point.

funnel plot

Notice two things about this funnel plot. Rather than a smooth, unbroken distribution, studies with effect sizes between -.45 and +.45 are entirely missing. Studies with smaller sample sizes have the largest effect sizes, whereas the smallest effect sizes all come from the larger samples.

For me, this adds to the overwhelming evidence there is something gone wrong in this literature and any effect sizes should be ignored. There must have been considerable suppression of null findings so large effects from smaller studies will not generalize. Yet, the authors find the differences between small and larger sample studies encouraging

This suggests, encouragingly, that despite potential publication bias or inflationary bias due to neuroimaging analysis methods, nonetheless studies with larger samples tend to converge on similar and more reasonable (medium) effect sizes. Although such a conclusion is tentative, the results to date (Fig. 6) suggest that a sample size of approximately n = 25 is sufficient to reliably produce effect sizes that accord with those reported in studies with much larger samples (up to n = 46).

I and others have long argued that studies of this small sample size in evaluating psychotherapy should be left as pilot feasibility studies and not used to generate effect sizes. I think the same logic applies to this literature.

Distinctive patterns of regional activation and deactivation

The first part of the results section is devoted to studies examining particular forms of meditation. Seeing the apparent consistency of results, one needs to keep in mind the small number of studies being examined and the considerable differences among them. For instance, results presented for focused attention combine three between-group comparisons with four within-group studies. Focused attention includes both pre-post meditation differences from experienced Tibetan Buddhist practitioners to differences between novice and experienced practitioners of mindfulness-based stress reduction (MBSR). In almost all cases, meaningful statistically significant differences are found in both activation and deactivation regions that would make a lot of sense in terms of the functions that are known to be associated with them. There is not much noting of anomalous brain regions being identified by significant effects There is a high ratio of significant findings to number of participants comparisons. There is little discussion of anomalies.

Meta-analysis of focused attention studies resulted in 2 significant clusters of activation, both in prefrontal cortex (Table 3;Fig. 2). Activations were observed in regions associated with the voluntary regulation of thought and action, including the premotor cortex (BA 6; Fig. 2b) and dorsal anterior cingulate cortex (BA24; Fig. 2a). Slightly sub-threshold clusters were also observed in the dorsolateral prefrontal cortex (BA 8/9; Fig. 2c) and left midinsula (BA 13; Fig. 2e); we display these somewhat sub-threshod results here because of the obvious interest of these findings in practices that involve top-down focusing of attention, typically focused on respiration. We also observed clusters of deactivation in regions associated with episodic memory and conceptual processing, including the ventral posterior cingulate cortex (BA 31; Fig. 2d)and left inferior parietal lobule (BA 39; Fig. 2f).

How can such meaningful, practically significant findings obtains when so many conditions mitigate against finding them? John Ioannidis once remarked that in hot areas of research, consistency of positive findings from small studies often reflects only the strength of bias with which they are sought. The strength of findings will decrease when larger, more methodologically sophisticated studies become available, conducted by investigators who are less committed to having to get confirmation.

The article concludes:

Many have understandably viewed the nascent neuroscience of meditation with skepticism (Andresen, 2000; Horgan, 2004), burecent years have seen an increasing number of high-quality, controlled studies that are suitable for inclusion in meta-analyses and that can advance our cumulative knowledge of the neural basis of various meditation practices (Tang et al., 2015). With nearly a hundred functional neuroimaging studies of meditation now reported, we can conclude with some confidence that different practices show relatively distinct patterns of brain activity, and that the magnitude of associated effects on brain function may have some practical significance. The only totally incontrovertible conclusion, however, is that much work remains to be done to confirm and build upon these initial findings.

“Increasing number of high-quality, controlled studies that are suitable for inclusion in meta-analyses” ?…” “Conclude with some confidence…? “Relatively distinct patterns”?… “Some practical significance”?

In all of this premature enthusiasm about findings relating the practice of meditation to activation of particular regions of the brain and deactivation of others, we should not lose track of some other issues.

Although the authors talk about mapping one-to-one relationships between psychological states and regions of the brain, none of the studies would be of sufficient size to document some relationships, given the expected size of the relationship, based on what is typically found between psychological states and other biological variables.

Many differences between techniques could be artifactual –due to the technique altering breathing, involving verbalization, or focused attention. Observed differences in the brain regions activated and deactivated might simply reflect these differences without them being related to psychological functioning.

Even if the association were found, it would be a long way to establishing that the association reflected a causal mechanism, rather than simply being correlational or even artifactual. Think of the analogy of discovering a relationship between the amount of sweat while exercising in concluding that any weight loss was due to sweating it out.

We still have not established that meditation has more psychological and physical health benefits than other active interventions with presumably different mechanisms. After lots of studies, we still don’t know whether mindfulness meditation is anything more than a placebo. While I was finishing up this blog post, I came across a new study:

The limited prosocial effects of meditation: A systematic review and meta-analysis. 

Although we found a moderate increase in prosociality following meditation, further analysis indicated that this effect was qualified by two factors: type of prosociality and methodological quality. Meditation interventions had an effect on compassion and empathy, but not on aggression, connectedness or prejudice. We further found that compassion levels only increased under two conditions: when the teacher in the meditation intervention was a co-author in the published study; and when the study employed a passive (waiting list) control group but not an active one. Contrary to popular beliefs that meditation will lead to prosocial changes, the results of this meta-analysis showed that the effects of meditation on prosociality were qualified by the type of prosociality and methodological quality of the study. We conclude by highlighting a number of biases and theoretical problems that need addressing to improve quality of research in this area. [Emphasis added].




When psychotherapy trials have multiple flaws…

Multiple flaws pose more threats to the validity of psychotherapy studies than would be inferred when the individual flaws are considered independently.

mind the brain logo

Multiple flaws pose more threats to the validity of psychotherapy studies than would be inferred when the individual flaws are considered independently.

We can learn to spot features of psychotherapy trials that are likely to lead to exaggerated claims of efficacy for treatments or claims that will not generalize beyond the sample that is being studied in a particular clinical trial. We can look to the adequacy of sample size, and spot what Cochrane collaboration has defined as risk of bias in their handy assessment tool.

We can look at the case-mix in the particular sites where patients were recruited.  We can examine the adequacy of diagnostic criteria that were used for entering patients to a trial. We can examine how blinded the trial was in terms of whoever assigned patients to particular conditions, but also what the patients, the treatment providers, and their evaluaters knew which condition to which particular patients were assigned.

And so on. But what about combinations of these factors?

We typically do not pay enough attention multiple flaws in the same trial. I include myself among the guilty. We may suspect that flaws are seldom simply additive in their effect, but we don’t consider whether they may be even synergism in the negative effects on the validity of a trial. As we will see in this analysis of a clinical trial, multiple flaws can provide more threats to the validity trial than what we might infer when the individual flaws are considered independently.

The particular paper we are probing is described in its discussion section as the “largest RCT to date testing the efficacy of group CBT for patients with CFS.” It also takes on added importance because two of the authors, Gijs Bleijenberg and Hans Knoop, are considered leading experts in the Netherlands. The treatment protocol was developed over time by the Dutch Expert Centre for Chronic Fatigue (NKCV, http://www.nkcv.nl; Knoop and Bleijenberg, 2010). Moreover, these senior authors dismiss any criticism and even ridicule critics. This study is cited as support for their overall assessment of their own work.  Gijs Bleijenberg claims:

Cognitive behavioural therapy is still an effective treatment, even the preferential treatment for chronic fatigue syndrome.


Not everybody endorses these conclusions, however their objections are mostly baseless.

Spoiler alert

This is a long read blog post. I will offer a summary for those who don’t want to read through it, but who still want the gist of what I will be saying. However, as always, I encourage readers to be skeptical of what I say and to look to my evidence and arguments and decide for themselves.

Authors of this trial stacked the deck to demonstrate that their treatment is effective. They are striving to support the extraordinary claim that group cognitive behavior therapy fosters not only better adaptation, but actually recovery from what is internationally considered a physical condition.

There are some obvious features of the study that contribute to the likelihood of a positive effect, but these features need to be considered collectively, in combination, to appreciate the strength of this effort to guarantee positive results.

This study represents the perfect storm of design features that operate synergistically:

perfect storm

 Referral bias – Trial conducted in a single specialized treatment setting known for advocating psychological factors maintaining physical illness.

Strong self-selection bias of a minority of patients enrolling in the trial seeking a treatment they otherwise cannot get.

Broad, overinclusive diagnostic criteria for entry into the trial.

Active treatment condition carry strong message how patients should respond to outcome assessment with improvement.

An unblinded trial with a waitlist control lacking the nonspecific elements (placebo) that confound the active treatment.

Subjective self-report outcomes.

Specifying a clinically significant improvement that required only that a primary outcome be less than needed for entry into the trial

Deliberate exclusion of relevant objective outcomes.

Avoidance of any recording of negative effects.

Despite the prestige attached to this trial in Europe, the US Agency for Healthcare Research and Quality (AHRQ) excludes this trial from providing evidence for its database of treatments for chronic fatigue syndrome/myalgic encephalomyelitis. We will see why in this post.

factsThe take away message: Although not many psychotherapy trials incorporate all of these factors, most trials have some. We should be more sensitive to when multiple factors occur in the same trial, like bias in the site for patient recruitment; lacking of blinding; lack of balance between active treatment and control condition in terms of nonspecific factors, and subjective self-report measures.

The article reporting the trial is

Wiborg JF, van Bussel J, van Dijk A, Bleijenberg G, Knoop H. Randomised controlled trial of cognitive behaviour therapy delivered in groups of patients with chronic fatigue syndrome. Psychotherapy and Psychosomatics. 2015;84(6):368-76.

Unfortunately, the article is currently behind a pay wall. Perhaps readers could contact the corresponding author Hans.knoop@radboudumc.nl  and request a PDF.

The abstract

Background: Meta-analyses have been inconclusive about the efficacy of cognitive behaviour therapies (CBTs) delivered in groups of patients with chronic fatigue syndrome (CFS) due to a lack of adequate studies. Methods: We conducted a pragmatic randomised controlled trial with 204 adult CFS patients from our routine clinical practice who were willing to receive group therapy. Patients were equally allocated to therapy groups of 8 patients and 2 therapists, 4 patients and 1 therapist or a waiting list control condition. Primary analysis was based on the intention-to-treat principle and compared the intervention group (n = 136) with the waiting list condition (n = 68). The study was open label. Results: Thirty-four (17%) patients were lost to follow-up during the course of the trial. Missing data were imputed using mean proportions of improvement based on the outcome scores of similar patients with a second assessment. Large and significant improvement in favour of the intervention group was found on fatigue severity (effect size = 1.1) and overall impairment (effect size = 0.9) at the second assessment. Physical functioning and psychological distress improved moderately (effect size = 0.5). Treatment effects remained significant in sensitivity and per-protocol analyses. Subgroup analysis revealed that the effects of the intervention also remained significant when both group sizes (i.e. 4 and 8 patients) were compared separately with the waiting list condition. Conclusions: CBT can be effectively delivered in groups of CFS patients. Group size does not seem to affect the general efficacy of the intervention which is of importance for settings in which large treatment groups are not feasible due to limited referral

The trial registration


Who was enrolled into the trial?

Who gets into a psychotherapy trial is a function of the particular treatment setting of the study, the diagnostic criteria for entry, and patient preferences for getting their care through a trial, rather than what is being routinely provided in that setting.

 We need to pay particular attention to when patients enter psychotherapy trials hoping they will receive a treatment they prefer and not to be assigned to the other condition. Patients may be in a clinical trial for the betterment of science, but in some settings, they are willing to enroll because of a probability of getting treatment they otherwise could not get. This in turn also affects the evaluation of both the condition in which they get the preferred treatment, but also their evaluation of the condition in which they are denied it. Simply put, they register being pleased with what they wanted or not being pleased if they did not get what they wanted.

The setting is relevant to evaluating who was enrolled in a trial.

The authors’ own outpatient clinic at the Radboud University Medical Center was the site of the study. The group has an international reputation for promoting the biopsychosocial model, in which psychological factors are assumed to be the decisive factor in maintaining somatic complaints.

All patients were referred to our outpatient clinic for the management of chronic fatigue.

There is thus a clear referral bias  or case-mix bias but we are not provided a ready basis for quantifying it or even estimating its effects.

The diagnostic criteria.

The article states:

In accordance with the US Center for Disease Control [9], CFS was defined as severe and unexplained fatigue which lasts for at least 6 months and which is accompanied by substantial impairment in functioning and 4 or more additional complaints such as pain or concentration problems.

Actually, the US Center for Disease Control would now reject this trial because these entry criteria are considered obsolete, overinclusive, and not sufficiently exclusive of other conditions that might be associated with chronic fatigue.*

There is a real paradigm shift happening in America. Both the 2015 IOM Report and the Centers for Disease Control and Prevention (CDC) website emphasize Post Exertional Malaise and getting more ill after any effort with M.E. CBT is no longer recommended by the CDC as treatment.

cdc criteriaThe only mandatory symptom for inclusion in this study is fatigue lasting 6 months. Most properly, this trial targets chronic fatigue [period] and not the condition, chronic fatigue syndrome.

Current US CDC recommendations  (See box  7-1 from the IoM document, above) for diagnosis require postexertional malaise for a diagnosis of myalgic encephalomyelitis (ME). See below.

pemPatients meeting the current American criteria for ME would be eligible for enrollment in this trial, but it’s unclear what proportion of the patients enrolled actually met the American criteria. Because of the over-inclusiveness of the entry diagnostic criteria, it is doubtful whether the results would generalize to American sample. A look at patient flow into the study will be informative.

Patient flow

Let’s look at what is said in the text, but also in the chart depicting patient flow into the trial for any self-selection that might be revealed.

In total, 485 adult patients were diagnosed with CFS during the inclusion period at our clinic (fig. 1). One hundred and fifty-seven patients were excluded from the trial because they declined treatment at our clinic, were already asked to participate in research incompatible with inclusion (e.g. research focusing on individual CBT for CFS) or had a clinical reason for exclusion (i.e. they received specifically tailored interventions because they were already unsuccessfully treated with individual CBT for CFS outside our clinic or were between 18 and 21 years of age and the family had to be involved in the therapy). Of the 328 patients who were asked to engage in group therapy, 99 (30%) patients indicated that they were unwilling to receive group therapy. In 25 patients, the reason for refusal was not recorded. Two hundred and four patients were randomly allocated to one of the three trial conditions. Baseline characteristics of the study sample are presented in table 1. In total, 34 (17%) patients were lost to follow-up. Of the remaining 170 patients, 1 patient had incomplete primary outcome data and 6 patients had incomplete secondary outcome data.

flow chart

We see that the investigators invited two thirds of patients attending the clinic to enroll in the trial. Of these, 41% refused. We don’t know the reason for some of the refusals, but almost a third of the patients approached declined because they did not want group therapy. The authors left being able to randomize 42% of patients coming to the clinic or less than two thirds of patients they actually asked. Of these patients, a little more than two thirds received the treatment to which were randomized and were available for follow-up.

These patients receiving treatment to which they were randomized and who were available for follow-up are self-selected minority of the patients coming to the clinic. This self-selection process likely reduced the proportion of patients with myalgic encephalomyelitis. It is estimated that 25% of patients meeting the American criteria a housebound and 75% are unable to work. It’s reasonably to infer that patients being the full criteria would opt out of a treatment that require regular attendance of a group session.

The trial is biased to ambulatory patients with fatigue and not ME. Their fatigue is likely due to some combinations of factors such as multiple co-morbidities, as-yet-undiagnosed medical conditions, drug interactions, and the common mild and subsyndromal  anxiety and depressive symptoms that characterize primary care populations.

The treatment being evaluated

Group cognitive behavior therapy for chronic fatigue syndrome, either delivered in a small (4 patients and 1 therapist) or larger (8 patients and 2 therapists) group format.

The intervention consisted of 14 group sessions of 2 h within a period of 6 months followed by a second assessment. Before the intervention started, patients were introduced to their group therapist in an individual session. The intervention was based on previous work of our research group [4,13] and included personal goal setting, fixing sleep-wake cycles, reducing the focus on bodily symptoms, a systematic challenge of fatigue-related beliefs, regulation and gradual increase in activities, and accomplishment of personal goals. A formal exercise programme was not part of the intervention.

Patients received a workbook with the content of the therapy. During sessions, patients were explicitly invited to give feedback about fatigue-related cognitions and behaviours to fellow patients. This aspect was introduced to facilitate a pro-active attitude and to avoid misperceptions of the sessions as support group meetings which have been shown to be insufficient for the treatment of CFS.

And note:

In contrast to our previous work [4], we communicated recovery in terms of fatigue and disabilities as general goal of the intervention.

Some impressions of the intensity of this treatment. This is a rather intensive treatment with patients having considerable opportunities for interactions with providers. This factor alone distinguishes being assigned to the intervention group versus being left in the wait list control group and could prove powerful. It will be difficult to distinguish intensity of contact from any content or active ingredients of the therapy.

I’ll leave for another time a fuller discussion of the extent to which what was labeled as cognitive behavior therapy in this study is consistent with cognitive therapy as practiced by Aaron Beck and other leaders of the field. However, a few comments are warranted. What is offered in this trial does not sound like cognitive therapy as Americans practice it. What is often in this trial seems emphasize challenging beliefs, pushing patients to get more active, along with psychoeducational activities. I don’t see indications of the supportive, collaborative relationship in which patients are encouraged to work on what they want to work on, engage in outside activities (homework assignments) and get feedback.

What is missing in this treatment is what Beck calls collaborative empiricism, “a systemic process of therapist and patient working together to establish common goals in treatment, has been found to be one of the primary change agents in cognitive-behavioral therapy (CBT).”

Importantly, in Beck’s approach, the therapist does not assume cognitive distortions on the part of the patient. Rather, in collaboration with the patient, the therapist introduces alternatives to the interpretations that the patient has been making and encourages the patient to consider the difference. In contrast, rather than eliciting goal statements from patients, therapist in this study imposes the goal of increased activity. Therapists in this study also seem ready to impose their views that the patients’ fatigue-related beliefs are maladaptive.

The treatment offered in this trial is complex, with multiple components making multiple assumptions that seem quite different from what is called cognitive therapy or cognitive behavioral therapy in the US.

The authors’ communication of recovery from fatigue and disability seems a radical departure not only from cognitive behavior therapy for anxiety and depression and pain, but for cognitive behavior therapy offered for adaptation to acute and chronic physical illnesses. We will return to this “communication” later.

The control group

Patients not randomized to group CBT were placed on a waiting list.

Think about it! What do patients think about having gotten involved in all the inconvenience and burden of a clinical trial in hope that they would get treatment and then being assigned to the control group with just waiting? Not only are they going to be disappointed and register that in their subjective evaluations of the outcome assessments patients may worry about jeopardizing the right to the treatment they are waiting for if they overly endorse positive outcomes. There is a potential for  nocebo effect , compounding the placebo effect of assignment to the CBT active treatment groups.

What are informative comparisons between active treatments and  control conditions?

We need to ask more often what inclusion of a control group accomplishes for the evaluation of a psychotherapy. In doing so, we need to keep in mind that psychotherapies do not have effect sizes, only comparisons of psychotherapies and control condition have effect sizes.

A pre-post evaluation of psychotherapy from baseline to follow-up includes the effects of any active ingredient in the psychotherapy, a host of nonspecific (placebo) factors, and any changes that would’ve occurred in the absence of the intervention. These include regression to the mean– patients are more likely to enter a clinical trial now, rather than later or previously, if there has been exacerbation of their symptoms.

So, a proper comparison/control condition includes everything that the patients randomized to the intervention group get except for the active treatment. Ideally, the intervention and the comparison/control group are equivalent on all these factors, except the active ingredient of the intervention.

That is clearly not what is happening in this trial. Patients randomized to the intervention group get the intervention, the added intensity and frequency of contact with professionals that the intervention provides, and all the support that goes with it; and the positive expectations that come with getting a therapy that they wanted.

Attempts to evaluate the group CBT versus the wait-list control group involved confounding the active ingredients of the CBT and all these nonspecific effects. The deck is clearly being stacked in favor of CBT.

This may be a randomized trial, but properly speaking, this is not a randomized controlled trial, because the comparison group does not control for nonspecific factors, which are imbalanced.

The unblinded nature of the trial

In RCTs of psychotropic drugs, the ideal is to compare the psychotropic drug to an inert pill placebo with providers, patients, and evaluate being blinded as to whether the patients received psychotropic drug or the comparison pill.

While it is difficult to achieve a comparable level of blindness and a psychotherapy trial, more of an effort to achieve blindness is desirable. For instance, in this trial, the authors took pains to distinguish the CBT from what would’ve happened in a support group. A much more adequate comparison would therefore be CBT versus either a professional or peer-led support group with equivalent amounts of contact time. Further blinding would be possible if patients were told only two forms of group therapy were being compared. If that was the information available to patients contemplating consenting to the trial, it wouldn’t have been so obvious from the outset to the patients being randomly assigned that one group was preferable to the other.

Subjective self-report outcomes.

The primary outcomes for the trial were the fatigue subscale of the Checklist Individual Strength;  the physical functioning subscale of the Short Health Survey 36 (SF-36); and overall impairment as measured by the Sickness Impact Profile (SIP).

Realistically, self-report outcomes are often all that is available in many psychotherapy trials. Commonly these are self-report assessments of anxiety and depressive symptoms, although these may be supplemented by interviewer-based assessments. We don’t have objective biomarkers with which to evaluate psychotherapy.

These three self-report measures are relatively nonspecific, particularly in a population that is not characterized by ME. Self-reported fatigue in a primary care population lacks discriminative validity with respect to pain, anxiety and depressive symptoms, and general demoralization.  The measures are susceptible to receipt of support and re-moralization, as well as gratitude for obtaining a treatment that was sought.

Self-report entry criteria include a score 35 or higher on the fatigue severity subscale. Yet, a score of less than 35 on this scale at follow up is part of what is defined as a clinically significant improvement with a composite score from combined self-report measures.

We know from medical trials that differences can be observed with subjective self-report measures that will not be found with objective measures. Thus, mildly asthmatic patients will fail to distinguish in their subjective self-reports between [  between the effective inhalant albuterol, an inert inhalant, and sham acupuncture, but will rate improvement better than getting no intervention.  However,  there will be a strong advantage over the other three conditions with an objective measure, maximum forced expiratory volume in 1 second (FEV1) as assessed  with spirometry.

The suppression of objective outcome measures

We cannot let these the authors of this trial off the hook in their dependence on subjective self-report outcomes. They are instructing patients that recovery is the goal, which implies that it is an attainable goal. We can reasonably be skeptical about acclaim of recovery based on changes in self-report measures. Were the patients actually able to exercise? What was their exercise capacity, as objectively measured? Did they return to work?

These authors have included such objective measurements in past studies, but not included them as primary outcomes, nor, even in some cases, reported them in the main paper reporting the trial.

Wiborg JF, Knoop H, Stulemeijer M, Prins JB, Bleijenberg G. How does cognitive behaviour therapy reduce fatigue in patients with chronic fatigue syndrome? The role of physical activity. Psychol Med. 2010 Jan 5:1

The senior authors’ review fails to mention their three studies using actigraphy that did not find effects for CBT. I am unaware of any studies that did find enduring effects.

Perhaps this is what they mean when they say the protocol has been developed over time – they removed what they found to be threats to the findings that they wanted to claim.

Dismissing of any need to consider negative effects of treatment

Most psychotherapy fail to assess any adverse effects of treatment, but this is usually done discretely, without mention. In contrast, this article states

Potential harms of the intervention were not assessed. Previous research has shown that cognitive behavioural interventions for CFS are safe and unlikely to produce detrimental effects.

Patients who meet stringent criteria for ME would be put at risk for pressure to exert themselves. By definition they are vulnerable to postexertional malaise (PEM). Any trail of this nature needs to assess that risk. Maybe no adverse effects would be found. If that were so, it would strongly indicate the absence of patients with appropriate diagnoses.

Timing of assessment of outcomes varied between intervention and control group.

I at first did not believe what I was reading when I encountered this statement in the results section.

The mean time between baseline and second assessment was 6.2 months (SD = 0.9) in the control condition and 12.0 months (SD = 2.4) in the intervention group. This difference in assessment duration was significant (p < 0.001) and was mainly due to the fact that the start of the therapy groups had to be frequently postponed because of an irregular patient flow and limited treatment capacities for group therapy at our clinic. In accordance with the treatment manual, the second assessment was postponed until the fourteenth group session was accomplished. The mean time between the last group session and the second assessment was 3.3 weeks (SD = 3.5).

So, outcomes were assessed for the intervention group shortly after completion of therapy, when nonspecific (placebo) effects would be stronger, but a mean of six months later than for patients assigned to the control condition.

Post-hoc statistical controls are not sufficient to rescue the study from this important group difference, and it compounds other problems in the study.

Take away lessons

Pay more attention to how limitations any clinical trial may compound each other in terms of the trial provide exaggerated estimates of the effects of treatment or the generalizability of the results to other settings.

Be careful of loose diagnostic criteria because a trial may not generalize to the same criteria being applied in settings that are different either in terms of patient population of the availability of different treatments. This is particularly important when a treatment setting has a bias in referrals and only a minority of patients being invited to participate in the trial actually agree and are enrolled.

Ask questions about just what information is obtained in comparing active treatment group and the study to its control/comparison. For start, just what is being controlled and how might that affect the estimates of the effectiveness of the active treatment?

Pay particular attention to the potent combination of the trial being unblinded, a weak comparision/control, and an active treatment that is not otherwise available to patients.


*The means of determining whether the six months of fatigue might be accounted for by other medical factors was specific to the setting. Note that a review of medical records for sufficient for an unknown proportion of patients, with no further examination or medical tests.

The Department of Internal Medicine at the Radboud University Medical Center assessed the medical examination status of all patients and decided whether patients had been sufficiently examined by a medical doctor to rule out relevant medical explanations for the complaints. If patients had not been sufficiently examined, they were seen for standard medical tests at the Department of Internal Medicine prior to referral to our outpatient clinic. In accordance with recommendations by the Centers for Disease Control, sufficient medical examination included evaluation of somatic parameters that may provide evidence for a plausible somatic explanation for prolonged fatigue [for a list, see [9]. When abnormalities were detected in these tests, additional tests were made based on the judgement of the clinician of the Department of Internal Medicine who ultimately decided about the appropriateness of referral to our clinic. Trained therapists at our clinic ruled out psychiatric comorbidity as potential explanation for the complaints in unstructured clinical interviews.


Is Donald Trump suffering from Pick’s Disease (frontotemporal dementia)?

Changing the conversation about Donald Trump’s fitness for office from whether he has a personality disorder to whether he has an organic brain disorder.

mind the brain logoChanging the conversation about Donald Trump’s fitness for office from whether he has a personality disorder to whether he has an organic brain disorder.

Trump.jpgFor a long while there has been an ongoing debate about whether Donald Trump suffers from a personality disorder that might contribute to his being unfit the President of the United States. Psychiatrists have ethical constraints in what they say because of the so-called Goldwater rule, barring them from commenting on the mental health of political figures that they have not personally  interviewed.

I am a clinical psychologist, not a psychiatrist. I feel the need to speak out that the behavior of Donald Trump is abnormal and we should caution against normalizing it. The problem with settling on his behavior being simply that of a bad person or con man is it doesn’t prepare us for just how erratic his behavior can be.

I’ll refrain from making a formal psychiatric diagnosis. I actually think that in clinical practice, a lot of mental health professionals too casually make diagnoses of personality disorders for patients (or privately, even for colleagues) they find difficult or annoying.  If they ever gave these people a structured interview,  I suspect they would be found to fall  below the threshold for any particular personality disorder.

Changing the conversation

But now an article in Stat has changed the conversation to whether Donald Trump suffers from personality disorder to whether he is developing an organic brain disorder.

I’m a brain specialist. I think Trump should be tested for a degenerative brain disease

When President Trump slurred his words during a news conference this week, some Trump watchers speculated that he was having a stroke. I watched the clip and, as a physician who specializes in brain function and disability, I don’t think a stroke was behind the slurred words. But having evaluated the chief executive’s remarkable behavior through my clinical lens for almost a year, I do believe he is displaying signs that could indicate a degenerative brain disorder.

As the president’s demeanor and unusual decisions raise the potential for military conflict in two regions of the world, the questions surrounding his mental competence have become urgent and demand investigation.


I see worrisome symptoms that fall into three main categories: problems with language and executive function; problems with social cognition and behavior; and problems with memory, attention, and concentration. None of these are symptoms of being a bad or mean person. Nor do they require spelunking into the depths of his psyche to understand. Instead, they raise concern for a neurocognitive disease process in the same sense that wheezing raises the alarm for asthma.

In addition to being a medical journalist, the author Ford Vox of the article is a neurorehabilitation physician who is board-certified physical medicine and rehabilitation physician with additional subspecialty board certification in brain injury medicine.

I was alerted by the possibility of a diagnosis of frontotemporal dementia by a tweet by Barney Carroll. He is a senior psychiatrist whom I have come to trust as a mentor on social media, even though we’ve never overlapped in the same department at the same time.

barney forget psychnoanalysis

And then there was this tweet about the Stat story, but I could judge its credibility because I did not know the tweeter or her source:

trump's disease

I followed up with a Google search and came across an article from August 2016, before the election:

Finally figured out Trump’s medical diagnosis after watching this:

It’s called Pick’s Disease, or frontotemporal dementia

Look at the symptoms, all of these which fit Trump quite closely:

  • Impulsivity and poor judgment
  • Extreme restlessness (early stages)
  • Overeating or drinking to excess
  • Sexual exhibitionism or promiscuity
  • Decline in function at work and home
  • Repetitive or obsessive behavior

And especially these, listed earlier in the article:

Excess protein build-up causes the frontal and temporal lobes of the brain, which control speech and personality, to slowly atrophy. 

Then I followed up with more Google searches, hitting MedLine Plus,  the website maintained by the National Institutes of Health’s Web site for patients and their families and friends and produced by the National Library of Medicine.

Pick disease

Pick disease is a rare form of dementia that is similar to Alzheimer disease, except that it tends to affect only certain areas of the brain.


People with Pick disease have abnormal substances (called Pick bodies and Pick cells) inside nerve cells in the damaged areas of the brain.

Pick bodies and Pick cells contain an abnormal form of a protein called tau. This protein is found in all nerve cells. But some people with Pick disease have an abnormal amount or type of this protein.

The exact cause of the abnormal form of the protein is unknown. Many different abnormal genes have been found that can cause Pick disease. Some cases of Pick disease are passed down through families.

Pick disease is rare. It can occur in people as young as 20. But it usually begins between ages 40 and 60. The average age at which it begins is 54.


The disease gets worse slowly. Tissues in parts of the brain shrink over time. Symptoms such as behavior changes, speech difficulty, and problems thinking occur slowly and get worse.

Early personality changes can help doctors tell Pick disease apart from Alzheimer disease. (Memory loss is often the main, and earliest, symptom of Alzheimer disease.)

People with Pick disease tend to behave the wrong way in different social settings. The changes in behavior continue to get worse and are often one of the most disturbing symptoms of the disease. Some persons have more difficulty with decision making, complex tasks, or language (trouble finding or understanding words or writing).

The website notes

A brain biopsy is the only test that can confirm the diagnosis.

However, some alternative diagnoses can be ruled out:

Your doctor might order tests to help rule out other causes of dementia, including dementia due to metabolic causes. Pick disease is diagnosed based on symptoms and results of tests, including:

Assessment of the mind and behavior (neuropsychological assessment)

Brain MRI

Electroencephalogram (EEG)

Examination of the brain and nervous system (neurological exam)

Examination of the fluid around the central nervous system (cerebrospinal fluid) after a lumbar puncture

Head CT scan

Tests of sensation, thinking and reasoning (cognitive function), and motor function

Back to Ford Vox in his Stats article:

In Trump’s case, we have no relevant testing to review. His personal physician issued a thoroughly unsatisfying letter before the election that didn’t contain much in the way of hard data. That’s a situation many people want to correct via an independent medical panel that can objectively evaluate the president’s fitness to serve. But the prospects for getting Congress to use the 25th Amendment in this way seem poor at the moment.

What we do have are a growing array of signs and symptoms displayed in public for all to see. It’s time to discuss these issues in a clinical context, even if this is a very atypical form of examination. It’s all we have. And even if the president has a physical exam early next year and releases the records, as announced by the White House, what he really needs is thorough cognitive testing.


Before biting the bullet, I also spoke with Dr. Dennis Agliano, who chairs the AMA’s Council on Ethical and Judicial Affairs, the panel that wrote the new ethical guidance. He advised me to be careful: “You can get yourself into hot water, since there are people who like Trump, and they may submit a complaint to the AMA,” the Tampa otolaryngologist told me. Ultimately, he reassured me that I should just do what I think is right.

Which is warn the president that he needs to be evaluated for a brain disease.

Good luck, Dr Vox, but at least we have a reasonable hypothesis on the table. As Barney Carroll says “Time will tell.”

slurred speech

Using F1000 “peer review” to promote politics over evidence about delivering psychosocial care to cancer patients

The F 1000 platform allowed authors and the reviewers whom they nominated to collaborate in crafting more of their special interest advocacy that they have widely disseminated elsewhere. Nothing original in this article and certainly not best evidence!


mind the brain logo

A newly posted article on the F1000 website raises questions about what the website claims is a “peer-reviewed” open research platform.

Infomercial? The F1000 platform allowed authors and the reviewers whom they nominated to collaborate in crafting more of their special interest advocacy that they have widely disseminated elsewhere. Nothing original in this article and certainly not best evidence!

I challenge the authors and the reviewers they picked to identify something said in the F1000 article that they have not said numerous times before either alone or in papers co-authored by some combination of authors and the reviewers they picked for this paper.

F1000 makes the attractive and misleading claim that versions of articles that are posted on its website reflect the response to reviewers.

Readers should be aware of uncritically accepting articles on the F 1000 website as having been peer-reviewed in any conventional sense of the term.

Will other special interests groups exploit this opportunity to brand their claims as “peer-reviewed” without the risk of having to tone down their claims in peer review? Is this already happening?

In the case of this article, reviewers were all chosen by the authors and have a history of co-authoring papers with the authors of the target paper in active advocacy of a shared political perspective, one that is contrary to available evidence.

Cynically, future authors might be motivated to divide their team, with some remaining authors and others dropping off to become nominated as reviewers. They could then suggest content that had already been agreed would be included, but was left off for the purposes being suggested in the review process


F1000Research bills itself as

An Open Research publishing platform for life scientists, offering immediate publication of articles and other research outputs without editorial bias. All articles benefit from transparent refereeing and the inclusion of all source data.

Material posted on this website is labeled as having received rapid peer-review:

Articles are published rapidly as soon as they are accepted, after passing an in-house quality check. Peer review by invited experts, suggested by the authors, takes place openly after publication.

My recent Google Scholar alert call attention to an article posted on F1000

Advancing psychosocial care in cancer patients [version 1; referees: 3 approved]

 Who were the reviewers?

open peer review of Advancing psychosocial care

Google the names of authors and reviewers. You will discover a pattern of co-authorship; leadership positions in international Psycho-Oncology society, a group promoting the mandating of specially mental health services for cancer patients, and lots of jointly and separately authored articles making a pitch for increased involvement of mental health professionals in routine cancer care. This article adds almost nothing to what is multiply available elsewhere in highly redundant publications

Given a choice of reviewers, these authors would be unlikely to nominate me. Nonetheless, here is my review of the article.

 As I might do in a review of a manuscript, I’m not providing citations for these comments, but support can readily be found by a search of blog posts at my website @CoyneoftheRealm.com and Google Scholar search of my publications. I welcome queries from anybody seeking documentation of these points below.

 Fighting Spirit

The notion that cancer patients having a fighting spirit improves survival is popular in the lay press and in promoting the power of the mind over cancer, but it has thoroughly been discredited.

Early on, the article identifies fighting spirit as an adaptive coping style. In actuality, fighting spirit was initially thought to predict mortality in a small methodologically flawed study. But that is no longer claimed.

Even one of the authors of the original study, Maggie Watson,  expressed relief when her own larger, better designed study failed to confirm the impression that a fighting spirit extended life after diagnosis  of cancer. Why? Dr. Watson was concerned that the concept was being abused in blaming cancer patients who were dying there was to their personal deficiency of not having enough fighting spirit.

Fighting spirit is rather useless as a measure of psychological adaptation. It confounds severity of cancer enrolled dysfunction with efforts to cope with cancer.

Distress as the sixth vital sign for cancer patients

distress thermometerBeware of a marketing slogan posing as an empirical statement. Its emptiness is similar to that of to “Pepsi is the one.” Can you imagine anyone conducting a serious study in which they conclude “Pepsi is not the one”?

Once again in this article, a vacuous marketing slogan is presented in impressive, but pseudo-medical terms. Distress cannot be a vital sign in the conventional sense. Thr  vital signs are objective measurements that do not depend on patient self-report: body temperature, pulse rate, and respiration rate (rate of breathing) (Blood pressure is not considered a vital sign, but is often measured along with the vital signs.).

Pain was declared a fifth vital sign, with physicians mandated  by guidelines to provide routine self-report screening of patients, regardless of their reasons for visit. Pain being the fifth vital sign seems to have been the inspiration for declaring distress as the sixth vital sign for cancer patients. However policy makers declaring pain  as the fifth vital sign did not result in improved patient levels of pain. Their subsequent making intervention mandatory for any reports of pain led to a rise in unnecessary back and knee surgery, with a substantial rise in associated morbidity and loss of function. The next shift to prescription of opioids that were claimed not to be addictive was the beginning of the current epidemic of addiction to prescription opioids. Making pain the fifth vital sign is killed a lot of patients and  turned others into addicts craving drugs on the street because they have lost their prescriptions for the opioids that addicted them.

pain as 5th vital signCDC launches

 Cancer as a mental health issue

There is a lack of evidence that cancer carries a risk of psychiatric disorder more than other chronic and catastrophic illnesses. However, the myth that there is something unique or unusual about cancer’s threat to mental health is commonly cited by mental health professional advocacy groups is commonly used to justify increased resources to them for specialized services.

The article provides an inflated estimate of psychiatric morbidity by counting adjustment disorders as psychiatric disorders. Essentially, a cancer patient who seeks mental health interventions for distress qualifies by virtue of help seeking being defined as impairment.

The conceptual and empirical muddle of “distress” in cancer patients

The article repeats the standard sloganeering definition of distress that the authors and reviewers have circulated elsewhere.

It has been very broadly defined as “a multifactorial, unpleasant, emotional experienceof a psychological (cognitive, behavioural, emotional), social and/or spiritual nature that may interfere with the ability to cope effectively with cancer, its physical symptoms and its treatment and that extends along a continuum, ranging from common normalfeelings of vulnerability, sadness and fears to problems that can become disabling, such as depression, anxiety, panic, social isolation and existential and spiritual crisis”5

[You might try googling this. I’m sure you’ll discover an amazing number of repetitions in similar articles advocating increasing psychosocial services for cancer patients organized around this broad definition.]

Distress is so broadly defined and all-encompassing, that there can be no meaningful independent validation of distress measures except for by other measures of distress, not conventional measures of adaptation or mental health. I have discussed that in a recent blog post.

If we restrict “distress” to the more conventional meaning of stress or negative affect, we find that any elevation in distress (usually 35% or so) associated with onset diagnosis of cancer tends to follow a natural trajectory of decline without formal intervention. Elevations in distress for most cancer patients, are resolved within 3 to 6 months without intervention. A residual 9 to 11% of cancer patients having elevated distress is likely attributed to pre-existing psychiatric disorder.

Routine screening for distress

The slogan “distress is the sixth vital sign” is used to justify mandatory routine screening of cancer patients for distress. In the United States, surgeons cannot close their electronic medical records for a patient and go on to the next patient without recording whether they had screened patients for distress, and if the patient reports distress, what intervention has been provided. Clinicians simply informally asking patients if they are distressed and responding to a “yes” by providing the patient with an antidepressant without further follow up allows surgeons to close the medical records.

As I have done so before, I challenge advocates of routine screening of cancer patients for distress to produce evidence that simply introducing routine screening without additional resources leads to better patient outcomes.

Routine screening for distress as uncovering unmet needs among cancer patients

 Studies in the Netherlands suggest that there is not a significant increase in need for services from mental health or allied health professionals associated with diagnosis of cancer. There is some disruption of such services that patients were receiving before diagnosis. It doesn’t take screening and discussion to suggest that patients that they at some point resume those services if they wish. There is also some increased need for physical therapy and nutritional counseling

If patients are simply asked a question whether they want a discussion of the services (in Dutch: Zou u met een deskundige willen praten over uw problemen?)  that are available, many patients will decline.

Much of demand for supportive services like counseling and support groups, especially among breast cancer patients is not from among the most distressed patients. One of the problems with clinical trials of psychosocial interventions is that most of the patients who seek enrollment are not distressed, and less they are prescreened. This poses dilemma: if you require elevated distress on a screening instrument, we end up rationing services and excluding many of the patients who would otherwise be receiving them.

I welcome clarification from F 1000 just what they offer over other preprint repositories. When one downloads a preprint from some other repositories, it clearly displays “not yet peer-reviewed.” F 1000 carries the advantage of the label of “peer-reviewed, but does not seem to be hard earned.


Slides are from two recent talks at Dutch International Congress on Insurance Medicine Thursday, November 9, 2017, Almere, Netherlands   :

Will primary care be automated screening and procedures or talking to patients and problem-solving? Invited presentation


Why you should not routinely screen your patients for depression and what you should do instead. Plenary Presentation






In a classic study of early childhood abuse and neglect, effects on later mental health nearly disappeared when….

Controls were introduced for life events that occurred between the childhood abuse and neglect and assessment of mental health outcomes in adulthood.

Adults who had objective reports of abuse and neglect in their childhood were matched with adults who were from similar backgrounds, but without such abuse and neglect.

mind the brain logo

Apparent effects of early experience largely disappeared when:

Controls were introduced for life events that occurred between the childhood abuse and neglect and assessment of mental health outcomes in adulthood.

Adults who had objective reports of abuse and neglect in their childhood were matched with adults who were from similar backgrounds, but without such abuse and neglect.

The study had a lot of influence on my thinking. You can click on link below to obtain a PDF.

Horwitz AV, Widom CS, McLaughlin J, White HR. The impact of childhood abuse and neglect on adult mental health: A prospective study. Journal of Health and Social Behavior. 2001 Jun 1:184-201.


This classic paper examined the impact of three types of victimization in childhood sexual abuse, physical abuse, and neglect on lifetime measures of mental health among adults.

It replicated some past findings obtained with checklists:

Men who were abused and neglected as children have more dysthymia and antisocial personality disorder as adults than matched controls, but they did not have more alcohol problems. Abused and neglected women report more symptoms of dysthymia, antisocial personality disorder, and alcohol problems than controls.

However, unlike past studies, this study introduced controls for subsequent life events:

After controlling for stressful life events, however, childhood victimization had little direct impact on any lifetime mental health outcome.

The study also matched adults with objective records of abuse and neglect in their childhood with adults who had similar backgrounds but without such abuse and neglect.

Mental health outcomes were similar between these matched groups.

What else was different:

Rather than a respondent-completed checklist, objective data were obtained.

The study used a prospective sample gathered from records of documented court cases of childhood abuse and neglect in a Midwestern city around 1970.

These subjects then interviewed about twenty years later to assess adult mental health.

The study was unusual in having a matched control group. 

The study:

Compared outcomes of the 641 members of the abuse and neglect group with a matched control group of 510 persons who did not have documented cases of abuse or neglect.

Unlike past research, the study introduced controls for subsequent life events.

The study was important because it indicated:

The importance of adopting an approach that places childhood victimization in the context of other life stressors and of prospective changes over the life course.

The study addressed two major theoretical shortcomings in past literature.

In past research, victimization had been considered as an isolated event with little regard to the context in which it occurs.

As Briere (1992) notes  “there has been a tendency for investigators to examine sexual abuse in a relative vacuum” (p. 199).

The study instead adopted a life course, developmental perspective.

Childhood victimization is typically part of a matrix of environmental problems such as poverty, unemployment, parental alcohol and drug problems, and inadequate family functioning…The unique contribution of childhood victimization to later symptomatology, after taking into account conditions such as family disruptions and stressors, persistent poverty, and broader patterns of social deprivation, is not  well understood.

The study started with the recognition of the theoretical deficiency of most studies.

Past research assumed a simple causal direction leading from childhood abuse and neglect  to mental health outcomes in later life.

Instead, this study made the basic assumption:

The influence of childhood experiences is contingent upon historical changes across the lives of affected individuals…Traumatic events that occur in early stages of the life course are unlikely to have uniform and straightforward mental health impacts in later stages of life, regardless of subsequent social conditions.

[Any] lasting impact on adults of childhood experiences depends on later factors such as the strength of the marriages and other social relationships, educational and occupational attainment, and the adequacy of family functioning.

This study was intended to contribute some knowledge of these later factors.

We know very little, however, about how subsequent stressors in life trajectories shaped later mental health consequences of childhood victimization (Bifulco, Brown, and Adler,  1991).

The study addressed important methodological limitations of past research:

Most past research depend upon adult responses to questions about their experience of abuse events as children.

Retrospective reporting should not have a major impact on the recall of screening and objective events such as loss of a parent or divorce.


Abuse events in early life are not encoded in memory as objective occurrences, but recollections of what constitutes abuse and experienced in the past change in light of later events definitions of abuse.


Most research about the traumatic impact of childhood abuse is not only retrospective but is also cross-sectional. Because studies obtain measures of prior abuse events in current states of mental health at the same time, present states of mental health may influence people’s recollections of recurrence has dramatic events (Brown Harris 1978).

In general, people with poor mental health have a bleaker of the world, including their prior life course, those with with higher psychological well-being (Coyne 1976; Beck et al, 1979; Burbach and Bourdin 1986).

Uniformly high correlation between later mental health and past child abuse might impart results from factors having to do with the past and the present, thus blurring the temporal order between recall of childhood events and subsequent psychological outcomes. Prospective studies, however, can overcome some of the problems stem from retrospective reports of childhood adversity (Kessler et al. 1997).

The study addressed the inadequate samples in most past studies:

Most studies used samples of college students, patients in clinical treatment, or responses to newspaper advertisements.

Few included adequate control groups of equivalent but non-victimized children. The high rates of psychopathology among adults who were abused as children are only meaningful when these rates are higher than comparable, non-abuse groups.

Shared characteristics between victimized and unvictimized children from similar backgrounds might explain the poorer mental health usually attributed to victimization.

Third factors may be influential.

Another factor such as disadvantaged social economic circumstances or family adversities may lead to both abuse in childhood and to poor adult mental health. The lack of control groups of non-abuse children comparable backgrounds precludes establishing the effects of abuse, as opposed to the impact of the matrix of social economic disadvantage within which abuse may occur on later states of mental health.


Findings showed both men and women who were victimized as children report more stressful life events over their lifetimes than a control sample, suggesting that early childhood abuse and neglect is part of a broader constellation of life stressors.


In addition, abused and neglected males and females are more likely than controls to have grown up in families receiving welfare, possibly indicating the officially reported childhood victimization co-occurs with particularly disadvantaged circumstances. Find there are no racial or age differences between the abused and neglected and control groups.


As a life course literature emphasizes, these results indicate the experiences occurring early in life do not have uniform consequences for mental health outcomes in later life (Harris, Brown, and Bifulco 1990, Elder at L 1996). Instead, the influence of these early childhood experiences vary depending on what happened in subsequent stages of the life course. In particular, stressful life events that occur later in the life course influence how much effect child victimization will have on subsequent outcomes. When childhood victims of abuse and neglect did not experience more stressors than controls, they do not have worse mental health outcomes (alcohol problems, dysthymia, or antisocial personality disorder, as adults. Thus, not only do early childhood events affect the life experiences, but these later experiences also affect how consequential these earlier events will be the subsequent health.

Limitations of this study need to be addressed in the future.

Although the results of this study clearly indicate the subsequent impact of childhood victimization on the mental health of adults grounded in a broader context and the life course trajectories, they do not specify the causal links among childhood victimization, lifetime stressors, and subsequent mental health.


One possible interpretation of these findings is the childhood victimization produces poor mental health outcomes among children. Poor early mental health could elevate the risk of experiencing subsequent life events such as getting fired from jobs, unemployment, and divorce, which in turn strongly relate for mental health among adults.

Another possible interpretation of our findings is the childhood victimization does not precede life stressors but is the correlate of other lifetime stressors such as family isolation and disorganization… [If so, pre-existing familial context can be critical for both the child and victimization for subsequent life stressors.

In addition, children from disorganized families might have weakened social support networks as adults, exacerbating their vulnerability to stressful life conditions.

Our findings only indicate a general relationship between childhood victimization, subsequent stressors, and mental health impacts.

Our findings do not allow the causal statements about the relationship among these factors.

It is important that future research specifies both the pathways through which childhood victimization elevates the risk of suffering subsequent stressors and the possible mediating buffering factors that protect some victims of child abuse and neglect from the adverse consequences in later stages of their lives.

Stop using the Adverse Childhood Experiences Checklist to make claims about trauma causing physical and mental health problems

Scores on the adverse childhood experiences (ACE) checklist (or ACC) are widely used in making claims about the causal influence of childhood trauma on mental and physical health problems. Does anyone making these claims bother to look at how the checklist is put together and consider what a summary score might mean?


mind the brain logo

Scores on the adverse childhood experiences (ACE) checklist (or ACC) are widely used in making claims about the causal influence of childhood trauma on mental and physical health problems. Does anyone making these claims bother to look at how the checklist is put together and consider what a summary score might mean?

In this issue of Mind the Brain, we begin taking a skeptical look at the ACE checklist. We ponder some of the assumptions implicit in what items were included and how summary scores of the number of items checked are interpreted. Readers will be left with profound doubts that the ACE is suitable for making claims about trauma.

This blog will eventually be followed by another that presents the case that scores on the ACC do not represent a risk factor for health problems, only a relatively uninformative risk marker. In contrast to potentially modifiable risk factors, risk markers are best interpreted as calling attention to the influence of some combination of other risk factors, many of as yet unspecified, but undoubtedly of an entirely different nature than what is being studied. What?!! You will have to stay tuned, but I’ll give some hints about what I am talking about in the current blog post.

Summary of key points

 The ACE checklist is a collection of very diverse and ambiguous items that cannot be presumed to necessarily represent traumatic experiences.

Items variously

  • Represent circumstances that are not typically traumatic.
  • Reflect the respondent’s past or current psychopathology.
  • Make equivalent and traumatic vastly different experiences, many neutral and some that are positive.
  • Reinterpret a personal vulnerability due to familial transmission of psychopathology, either direct or indirect, rather than simply an exposure to events.
  • Ignore crucial contextual information, including timing of events.

There is reason not to assume that higher summed scores for the ACE represent more exposure to trauma than lower scores.

Are professionals misinterpreting the ACE checklist just careless or are they ideologues selectively identifying “evidence” for their positions which don’t depend on evidence at all?

ace-7Witness claims based on research with the ACE that migraines are caused by sexual abuse   and that psychotherapy addressing that abuse should be first line treatment. Or claims that childhood trauma is as strong a risk factor for psychosis and schizophrenia as smoking is for lung cancer [* ] and so psychotherapy is equivalent to medication in its effects. Or claims that myalgic encephalomyelitis, formerly known as chronic fatigue syndrome, is caused by childhood trauma and the psychological treatments can be recommended as the treatment of choice. These claims share a speculative, vague neo-cryptic pseudopsychoanalytic set of assumptions that is seldom articulated or explicitly confronted with evidence. Authors typically leap from claims about childhood trauma causing later problems to non sequitur claims about the efficacy of psychological intervention in treating these problems by addressing trauma. These claims about efficacy of trauma-focused treatment are not borne out in actually examining effects observed in randomized controlled trials.

Rather than attempting to address a provocative question about investigator motivation without a ready way of answering it, I will show most claims about trauma causing mental and physical health problems are, at best, based on very weak evidence, if they depend solely on the ACE checklist.

I will leave for my readers to decide if some authors who make such a fuss about the ACE have bothered to look at the instrument or care that is so inappropriate for the purposes to which they put it.

The ACE is reproduced at the bottom of this post and it is a good idea to compare what I’m saying about it to the actual checklist.

What “science” is behind such speculations?

The ACE was originally intended for educational purposes, not as a scientific instrument. Perhaps that explains its gross deficiencies as a key measure of psychological and epidemiological constructs.

The ACE checklist is a collection of very different and ambiguous items that cannot be presumed to represent traumatic experiences.

The ACE consists of ten dichotomous items for which the respondent is asked to indicate no/yes whether an experience occurred before the age of 18.  However, for six of the 10 items, the respondent is given further choices  that often differ greatly in the kind of experience to which the items refer. Scoring of the instrument does not take which of these experiences is the basis of a response. For example,

5. Did you often feel that … You didn’t have enough to eat, had to wear dirty clothes, and had no one to protect you? or

Your parents were too drunk or high to take care of you or take you to the doctor if you needed it?

Yes   No     If yes enter 1     ________

This item treats some very different circumstances as equivalent. The first half is complex, but largely covers the experience of living in poverty, but combines that with “having no one to protect you.” In contrast, the second half refers to substance abuse on the part of parents. In neither case, is there any room for interpreting what mitigating circumstances in the respondent’s life might have influenced effects of exposure. Presumably, the timing of this exposure would be important. If the exposure only occurred at the end of the 18 year period covered by the checklist, effects could be mitigated by other individual and social resources the respondent had.

Single items that are added together in a summary score.  We have to ask whether there is an equivalency between the two halves of the item that will be treated as the same. This will be an accumulating concern as we go through the 10 item questionnaire

The items vary greatly in the likelihood that they refer to an experience that was traumatic. Seldom do any of the researchers who use the ACE explain what they mean by trauma. If they did, I doubt that they could make a good argument that in endorsing many of these items would indicate that a respondent had faced a trauma.

From the third edition of the American Psychiatric Association Diagnostic and Statistical Manual (DSM-III) onward to DSM-5, the assumption has been that a traumatic event is a catastrophic stressor outside the range of usual human experience.

With that criteria in mind we have to ask if items are likely to represent a traumatic experience for most people. In answering this question, we also have to ask how we willing to consider a particular item is equivalent to other items in arriving at an overall score reflecting exposure to trauma before age 18. Yet, if summary scores are to be meaningful, assumption has to be made that items contribute equally if they are endorsed

6. Were your parents ever separated or divorced?

Yes   No     If yes enter 1     ________

The item refers to a highly prevalent and complex event, the nature and consequences of which are likely to unfold over time. Importantly, we need a sense of context to judge whether the event is traumatic and, if so how severe. Presumably, it would matter greatly when, across the 18 year span, the event that occurred. No timing or other information is asked of the respondent, only whether or not this event occurred. Neither the respondent nor anyone interpreting a score on the inventory has further information as to what is meant.

Other problems with ambiguous items.

Questions can be raised about the validity of all the individual items and the wisdom of combining them as equivalent in creating a summary score.

Items 1 and 2: Items raise questions about what role the respondent played eliciting the event.

 Did an event simply befall a respondent? Was it related to some pre-existing characteristic of the respondent? Or did the respondent have an active role in generating the event?

Did a parent or other adult member of the household often…

Swear at you, insult you, put you down, or humiliate you?


Act in a way that made you afraid that you might be physically hurt?

Yes   No     If yes enter 1     ________


Did a parent or other adult in the household often …

Push, grab, slap, or throw something at you?


Ever hit you so hard that you had marks or were injured?

Yes   No     If yes enter 1     ________

 Here, as throughout the rest of the checklist, questions can be raised about whether these items refer simply to an environmental exposure in epidemiological terms, say, equivalent to asbestos or tobacco. We don’t know the frequency, intensity or context of a the behavior in question, all of which may be crucial in evaluating whether a trauma occurred. For instance, it matters greatly if the behavior happened frequently when the respondent as a toddler or was limited to a struggle that occurred when the respondent was a teen high on drugs  attempting to take the car keys and go for a after midnight drive.

Like most of the rest of the questionnaire, there is the question of timing.

Item 3: There is so much ambiguity in endorsments of (ostensible) sexual abuse. Maybe it was a positive, liberating experience.

This is a crucial item and discussions of the ACE often assume that it is endorsed and represents a traumatic experience:

Did an adult or person at least 5 years older than you ever…

Touch or fondle you or have you touch their body in a sexual way?


Try to or actually have oral, anal, or vaginal sex with you?

Note that this is a complex item for which endorsement could be on the basis of a single instance of a person at least 5 years older touching or fondling the respondent. What if the presumed “perpetrator” is the 20 year old boyfriend or girlfriend of a 14 year old?

Are we willing to treat as equivalent “touch” or ‘fondle you” and “having anal sex” in all instances?

Arguably, the event which construed as trauma could actually be quite positive, as in the respondent  forming a secure attachment with a somewhat older, but nonetheless appropriate partner. All that is unconventional is not traumatic. What if the respondent and  alleged “perpetrator” were in a deeply intimate relationship or already married?

The research that attempts to link endorsement of such an item to lasting mental and physical health problems is remarkably contradictory and inconsistent 

Item 4:  Does this  item reflect the respondent’s serious clinical depression or other mental disorder before age 18 or currently, when the checklist is being completed?

Did you often feel that …  No one in your family loved you or thought you were important or special?    or

Your family didn’t look out for each other, feel close to each other, or support each other?

Yes   No     If yes enter 1     ________

As elsewhere in the checklist, there is no place for the respondent or someone interpreting a “yes” response for taking into account timing or contextual factors that might mitigate or compound effects of this “exposure.”

Item 5: Is this a  traumatic exposure or an enduring set of circumstances conferring multiple known risks to mental and physical health?

Did you often feel that …

You didn’t have enough to eat, had to wear dirty clothes, and had no one to protect you?


Your parents were too drunk or high to take care of you or take you to the doctor if you needed it?

Yes   No     If yes enter 1     ________

This item has already been discussed above, but is worth revisiting in terms of raising issues whether particular items refer either directly or indirectly to enduring sets of circumstances that pose their own enduring threat. The relevant question is whether items which ostensibly represent “traumatic events” and risk for subsequent problems are not risk factors, but only risk indicators, and not particularly informative ones.

Item 7: Could an ostensibly a traumatic exposure actually be no actual exposure?

Was your mother or stepmother:

Often pushed, grabbed, slapped, or had something thrown at her?    or

Sometimes or often kicked, bitten, hit with a fist, or hit with something hard?    or

Ever repeatedly hit over at least a few minutes or threatened with a gun or knife?

Yes   No     If yes enter 1     ________

Like item four, which refers to ostensible sexual abuse, this item seems to be one of the least ambiguous in terms of representing exposure to risk. But does it? We don’t know the timing, duration, or context. For instance, the mother might no longer be in the home and the respondent might not have known what happened at the time. There is even the possibility that the respondent was the “perpetrator” of such violence against the mother.

Items 8 and 9: Are traumatic exposures or indications of familial transmission of psychopathology?

Did you live with anyone who was a problem drinker or alcoholic or who used street drugs?

Yes   No

If yes enter 1     ________


Was a household member depressed or mentally ill or did a household member attempt suicide?    Yes   No     If yes enter 1     ________

These items are highly ambiguous. They don’t take in consideration whether the person was a biological relative, or whether they were a parent, sibling, or someone not biologically related. They don’t take into account timing. There may not have even been any direct exposure to the substance misuse or the attempted suicide, but the respondent only later learned of something that was closeted.

Item 10: traumatic exposure or relief from exposure?

Did a household member go to prison?

Yes   No

If yes enter 1     ________

The implications of endorsement of this item depend greatly on whom the household member was and the circumstances of them going to prison.

There may be a familial relationship with this person, but it could have been an abusive stepparents or stepsiblings, with the incarceration representing a lasting relief from some impressive situations. Or the person who became incarcerated was not an immediate family member, but somewhat more transient, maybe someone who was just renting a room or given a place to stay. We just don’t know.

Does adding up all these endorsements in a summary score clarify or confuse further?

Now add up your “Yes” answers:   _______   This is your ACE Score

 It would be useful to briefly review the assumptions involved in summing across items of a checklist and entering the summary score as a continuous variable in statistical analyses.

Classical test theory recognizes that the individual items may imperfectly reflect the underlying construct, in this case, traumatic exposure. However, in constructing a sum, the expectation is that the imperfections or errors of measurement in particular items cancel each other out. The summed score becomes a purer a representation of the underlying construct than any of the original items. Thus, the summary score will be more reliable and valid than any of the individual items would be.

There are a number of problems in applying this assumption to a summary ACE score. The items are quite heterogeneous, i.e., they vary wildly in whether they are likely to represent a traumatic exposure, and if so, the severity of that exposure. More importantly, there is a huge amount of variation in what these brief items would represent for particular individuals in the contexts they found themselves in the first 18 years of their lives. Undoubtedly, most endorsements of these items would represent false positives, if we hold ourselves to any strict definitions of trauma. If we don’t do so, we risk equating the only normative experiences that may have neutral or even positive effects on the respondent with serious exposures to traumatic events with lasting consequences

We are not in a position to know whether a score of five or even eight necessarily represents more traumatic exposure than a score of one.

Moreover, there is important empirical research of the clustering of events. We certainly cannot consider them random and unrelated. One classic study found 

In our data, total CCA was related to depressive symptoms, drug use, and antisocial behavior in a quadratic manner. Without further elucidation, this higher order relationship could have been interpreted as support for a sensitization process in which the long-term impact of each additional adversity on mental health compounds as childhood adversity accumulates. However, further analysis revealed that this acceleration effect was an artifact of the confounding of high cumulative adversity scores with the experience of more severe events. Thus, respondents with higher total CCA had disproportionately poorer emotional and behavioral functioning because of both the number and severity of the adversities they were exposed to, not the cumulative number of different types of adversities experienced.


Because low-impact adversities did not present a cumulative hazard to young adult mental health, they functioned as suppressor events in the total sum score, consistent with Turner and Wheaton’s (1997) expectation. Their inclusion increased the “noise” in the score and greatly watered down the influence of high-impact events. Thus, in addition to decreasing efficiency, total scores may seriously underestimate the cumulative effects of severe forms of childhood adversity, such as abuse and serious neglect.

But what if many or most of the high scores in a particular sample represent only a clustering of low- or no-impact adversities?

Another large-sample, key study cautioned:

Significant effects of parental separation}divorce in predicting subsequent mood disorders and addictive disorders are powerfully affected by whether or not there was parental violence and psychopathology in the household prior to the break-up and whether exposure to these adversities was reduced as a result of the separation (Kessler et al. 1997a). There are some situations – such as one in which the father was a violent alcoholic – where our data suggest that parental divorce and subsequent removal of the respondent from exposure to the father might actually be associated with a significant improvement in the respondent’s subsequent disorder risk profile, a possibility that has important social policy implications.

Finding Your ACE Score-page-0


*Richard Bentall commonly interprets summed ACE scores in peer reviewed articles  as having a traditional dose-response association with mental health outcomes, and therefore as representing a modifiable causal factor in psychosis. In books and in social media, his claims become simply absurd.


I don’t think his interpretations withstand a scrutiny of the items and what a summed score might conceivably represent.

eBook_Mindfulness_345x550Preorders are being accepted for e-books providing skeptical looks at mindfulness and positive psychology, and arming citizen scientists with critical thinking skills. 

I will also be offering scientific writing courses on the web as I have been doing face-to-face for almost a decade. I want to give researchers the tools to get into the journals where their work will get the attention it deserves.

Sign up at my website to get advance notice of the forthcoming e-books and web courses, as well as upcoming blog posts at this and other blog sites. Get advance notice of forthcoming e-books and web courses. Lots to see at CoyneoftheRealm.com.