Unintended consequences of universal mindfulness training for schoolchildren?

the mindful nationThis is the first installment of what will be a series of occasional posts about the UK Mindfulness All Party Parliamentary Group report,  Mindful Nation.

  • Mindful Nation is seriously deficient as a document supposedly arguing for policy based on evidence.
  • The professional and financial interests of lots of people involved in preparation of the document will benefit from implementation of its recommendations.
  • After an introduction, I focus on two studies singled in Mindful Nation out as offering support for the benefits of mindfulness training for school children.
  • Results of the group’s cherrypicked studies do not support implementation of mindfulness training in the schools, but inadvertently highlight some issues.
  • Investment in universal mindfulness training in the schools is unlikely to yield measurable, socially significant results, but will serve to divert resources from schoolchildren more urgently in need of effective intervention and support.
  • Mindfulness Nation is another example of  delivery of  low intensity  services to mostly low risk persons to the detriment of those in greatest and most urgent need.

The launch event for the Mindful Nation report billed it as the “World’s first official report” on mindfulness.

Mindful Nation is a report written by the UK Mindfulness All-Party Parliamentary Group.

The Mindfulness All-Party Parliamentary Group (MAPPG)  was set up to:

  • review the scientific evidence and current best practice in mindfulness training
  • develop policy recommendations for government, based on these findings
  • provide a forum for discussion in Parliament for the role of mindfulness and its implementation in public policy.

The Mindfulness All-Party Parliamentary Group describes itself as

Impressed by the levels of both popular and scientific interest, and launched an inquiry to consider the potential relevance of mindfulness to a range of urgent policy challenges facing government.

Don’t get confused by this being a government-commissioned report. The report stands in sharp contrast to one commissioned by the US government in terms of unbalanced constitution of the committee undertaking the review, and lack  of transparency in search for relevant literature,  and methodology for rating and interpreting of the quality of available evidence.

ahrq reportCompare the claims of Mindful Nation to a comprehensive systematic review and meta-analysis prepared for the US Agency for Healthcare Research and Quality (AHRQ) that reviewed 18,753 citations, and found only 47 trials (3%) that included an active control treatment. The vast majority of studies available for inclusion had only a wait list or no-treatment control group and so exaggerated any estimate of the efficacy of mindfulness.

Although the US report was available to those  preparing the UK Mindful Nation report, no mention is made of either the full contents of report or a resulting publication in a peer-reviewed journal. Instead, the UK Mindful Nation report emphasized narrative and otherwise unsystematic reviews, and meta-analyses not adequately controlling for bias.

When the abridged version of the AHRQ report was published in JAMA: Internal Medicine, an accompanying commentary raises issues even more applicable to the Mindful Nation report:

The modest benefit found in the study by Goyal et al begs the question of why, in the absence of strong scientifically vetted evidence, meditation in particular and complementary measures in general have become so popular, especially among the influential and well educated…What role is being played by commercial interests? Are they taking advantage of the public’s anxieties to promote use of complementary measures that lack a base of scientific evidence? Do we need to require scientific evidence of efficacy and safety for these measures?

The members of the UK Mindfulness All-Party Parliamentary Group were selected for their positive attitude towards mindfulness. The collection of witnesses they called to hearings were saturated with advocates of mindfulness and those having professional and financial interests in arriving at a positive view. There is no transparency in terms of how studies or testimonials were selected, but the bias is notable. Many of the scientific studies were methodologically poor, if there was any methodology at all. Many were strongly stated, but weakly substantiated opinion pieces. Authors often included those having  financial interests in obtaining positive results, but with no acknowledgment of conflict of interest. The glowing testimonials were accompanied by smiling photos and were unanimous in their praise of the transformative benefits of mindfulness.

As Mark B. Cope and David B. Allison concluded about obesity research, such a packing of the committee and a highly selective review of the literature leads to a ”distortion of information in the service of what might be perceived to be righteous ends.” [I thank Tim Caulfield for calling this quote to my attention].

Mindfulness in the schools

The recommendations of Mindfulness Nation are

  1. The Department for Education (DfE) should designate, as a first step, three teaching schools116 to pioneer mindfulness teaching,co-ordinate and develop innovation, test models of replicability and scalability and disseminate best practice.
  2. Given the DfE’s interest in character and resilience (as demonstrated through the Character Education Grant programme and its Character Awards), we propose a comparable Challenge Fund of £1 million a year to which schools can bid for the costs of training teachers in mindfulness.
  3. The DfE and the Department of Health (DOH) should recommend that each school identifies a lead in schools and in local services to co-ordinate responses to wellbeing and mental health issues for children and young people117. Any joint training for these professional leads should include a basic training in mindfulness interventions.
  4. The DfE should work with voluntary organisations and private providers to fund a freely accessible, online programme aimed at supporting young people and those who work with them in developing basic mindfulness skills118.
Payoff of Mindful Nation to Oxford Mindfulness Centre will be huge.
Payoff of Mindful Nation to Oxford Mindfulness Centre will be huge.

Leading up to these recommendations, the report outlined an “alarming crisis” in the mental health of children and adolescents and proposes:

Given the scale of this mental health crisis, there is real urgency to innovate new approaches where there is good preliminary evidence. Mindfulness fits this criterion and we believe there is enough evidence of its potential benefits to warrant a significant scaling-up of its availability in schools.

Think of all the financial and professional opportunities that proponents of mindfulness involved in preparation of this report have garnered for themselves.

Mindfulness to promote executive functioning in children and adolescents

For the remainder of the blog post, I will focus on the two studies cited in support of the following statement:

What is of particular interest is that those with the lowest levels of executive control73 and emotional stability74 are likely to benefit most from mindfulness training.

The terms “executive control” and “emotional stability” were clarified:

Many argue that the most important prerequisites for child development are executive control (the management of cognitive processes such as memory, problem solving, reasoning and planning) and emotion regulation (the ability to understand and manage the emotions, including and especially impulse control). These main contributors to self-regulation underpin emotional wellbeing, effective learning and academic attainment. They also predict income, health and criminality in adulthood69. American psychologist, Daniel Goleman, is a prominent exponent of the research70 showing that these capabilities are the biggest single determinant of life outcomes. They contribute to the ability to cope with stress, to concentrate, and to use metacognition (thinking about thinking: a crucial skill for learning). They also support the cognitive flexibility required for effective decision-making and creativity.

Actually, Daniel Goleman is the former editor of the pop magazine Psychology Today and an author of numerous pop books.

The first cited paper.

73 Flook L, Smalley SL, Kitil MJ, Galla BM, Kaiser-Greenland S, Locke J, et al. Effects of mindful  awareness practices on executive functions in elementary school children. Journal of Applied School Psychology. 2010;26(1):70-95.

Journal of Applied School Psychology is a Taylor-Francis journal, formerly known as Special Services in the Schools (1984 – 2002).  Its Journal Impact Factor is 1.30.

One of the authors of the article, Susan Kaiser-Greenland is a mindfulness entrepreneur as seen in her website describing her as an author, public speaker, and educator on the subject of sharing secular mindfulness and meditation with children and families. Her books are The Mindful Child: How to Help Your Kid Manage Stress and Become Happier, Kinder, and More Compassionate and Mindful Games: Sharing Mindfulness and Meditation with Children, Teens, and Families and the forthcoming The Mindful Games Deck: 50 Activities for Kids and Teens.

This article represents the main research available on Kaiser-Greenfield’s Inner Kids program and figures prominently in her promotion of her products.

The sample consisted of 64 children assigned to either mindful awareness practices (MAPs; n = 32) or a control group consisting of a silent reading period (n = 32).

The MAPs training used in the current study is a curriculum developed by one of the authors (SKG). The program is modeled after classical mindfulness training for adults and uses secular and age appropriate exercises and games to promote (a) awareness of self through sensory awareness (auditory, kinesthetic, tactile, gustatory, visual), attentional regulation, and awareness of thoughts and feelings; (b) awareness of others (e.g., awareness of one’s own body placement in relation to other people and awareness of other people’s thoughts and feelings); and (c) awareness of the environment (e.g., awareness of relationships and connections between people, places, and things).

A majority of exercises involve interactions among students and between students and the instructor.

Outcomes.

The primary EF outcomes were the Metacognition Index (MI), Behavioral Regulation Index (BRI), and Global Executive Composite (GEC) as reported by teachers and parents

Wikipedia presents the results of this study as:

The program was delivered for 30 minutes, twice per week, for 8 weeks. Teachers and parents completed questionnaires assessing children’s executive function immediately before and following the 8-week period. Multivariate analysis of covariance on teacher and parent reports of executive function (EF) indicated an interaction effect baseline EF score and group status on posttest EF. That is, children in the group that received mindful awareness training who were less well regulated showed greater improvement in EF compared with controls. Specifically, those children starting out with poor EF who went through the mindful awareness training showed gains in behavioral regulation, metacognition, and overall global executive control. These results indicate a stronger effect of mindful awareness training on children with executive function difficulties.

The finding that both teachers and parents reported changes suggests that improvements in children’s behavioral regulation generalized across settings. Future work is warranted using neurocognitive tasks of executive functions, behavioral observation, and multiple classroom samples to replicate and extend these preliminary findings.”

What I discovered when I scrutinized the study.

 This study is unblinded, with students and their teachers and parents providing the subjective ratings of the students well aware of which group students are assigned. We are not given any correlations among or between their ratings and so we don’t know whether there is just a global subjective factor (easy or difficult child, well-behaved or not) operating for either teachers or parents, or both.

It is unclear for what features of the mindfulness training the comparison reading group offers control or equivalence. The two groups are  different in positive expectations and attention and support that are likely to be reflected the parent and teacher ratings. There’s a high likelihood of any differences in outcomes being nonspecific and not something active and distinct ingredient of mindfulness training. In any comparison with the students assigned to reading time, students assigned to mindfulness training have the benefit of any active ingredient it might have, as well as any nonspecific, placebo ingredients.

This is exceedingly weak design, but one that dominates evaluations of mindfulness.

With only 32 students per group, note too that this is a seriously underpowered study. It has less than a 50% probability of detecting a moderate sized effect if one is present. And because of the larger effect size needed to achieve statistical significance with such a small sample size, and statistically significant effects will be large, even if unlikely to replicate in a larger sample. That is the paradox of low sample size we need to understand in these situations.

Not surprisingly, there were no differences between the mindfulness and reading control groups on any outcomes variable, whether rated by parents or teachers. Nonetheless, the authors rescued their claims for an effective intervention with:

However, as shown by the significance of interaction terms, baseline levels of EF (GEC reported by teachers) moderated improvement in posttest EF for those children in the MAPs group compared to children in the control group. That is, on the teacher BRIEF, children with poorer initial EF (higher scores on BRIEF) who went through MAPs training showed improved EF subsequent to the training (indicated by lower GEC scores at posttest) compared to controls.

Similar claims were made about parent ratings. But let’s look at figure 3 depicting post-test scores. These are from the teachers, but results for the parent ratings are essentially the same.

teacher BRIEF quartiles

Note the odd scaling of the X axis. The data are divided into four quartiles and then the middle half is collapsed so that there are three data points. I’m curious about what is being hidden. Even with the sleight-of-hand, it appears that scores for the intervention and control groups are identical except for the top quartile. It appears that just a couple of students in the control group are accounting for any appearance of a difference. But keep in mind that the upper quartile is only a matter of eight students in each group.

This scatter plot is further revealing:

teacher BRIEF

It appears that the differences that are limited to the upper quartile are due to a couple of outlier control students. Without them, even the post-hoc differences that were found in the upper quartile between intervention control groups would likely disappear.

Basically what we are seeing is that most students do not show any benefit whatsoever from mindfulness training over being in a reading group. It’s not surprising that students who were not particularly elevated on the variables of interest do not register an effect. That’s a common ceiling effect in such universally delivered interventions in general population samples

Essentially, if we focus on the designated outcome variables, we are wasting the students’ time as well as that of the staff. Think of what could be done if the same resources could be applied in more effective ways. There are a couple of students in in this study were outliers with low executive function. We don’t know how else they otherwise differ.Neither in the study, nor in the validation of these measures is much attention given to their discriminant validity, i.e., what variables influence the ratings that shouldn’t. I suspect strongly that there are global, nonspecific aspects to both parent and teacher ratings such that they are influenced by the other aspects of these couple of students’ engagement with their classroom environment, and perhaps other environments.

I see little basis for the authors’ self-congratulatory conclusion:

The present findings suggest that mindfulness introduced in a general  education setting is particularly beneficial for children with EF difficulties.

And

Introduction of these types of awareness practices in elementary education may prove to be a viable and cost-effective way to improve EF processes in general, and perhaps specifically in children with EF difficulties, and thus enhance young children’s socio-emotional, cognitive, and academic development.

Maybe the authors stared with this conviction and it was unshaken by disappointing findings.

Or the statement made in Mindfulness Nation:

What is of particular interest is that those with the lowest levels of executive control73 and emotional stability74 are likely to benefit most from mindfulness training.

But we have another study that is cited for this statement.

74. Huppert FA, Johnson DM. A controlled trial of mindfulness training in schools: The importance of practice for an impact on wellbeing. The Journal of Positive Psychology. 2010; 5(4):264-274.

The first author, Felicia Huppert is a  Founder and Director – Well-being Institute and Emeritus Professor of Psychology at University of Cambridge, as well as a member of the academic staff of the Institute for Positive Psychology and Education of the Australian Catholic University.

This study involved 173 14- and 15- year old  boys from a private Catholic school.

The Journal of Positive Psychology is not known for its high methodological standards. A look at its editorial board suggests a high likelihood that manuscripts submitted will be reviewed by sympathetic reviewers publishing their own methodologically flawed studies, often with results in support of undeclared conflicts of interest.

The mindfulness training was based on the program developed by Kabat-Zinn and colleagues at the University of Massachusetts Medical School (Kabat-Zinn, 2003). It comprised four 40 minute classes, one per week, which presented the principles and practice of mindfulness meditation. The mindfulness classes covered the concepts of awareness and acceptance, and the mindfulness practices included bodily awareness of contact points, mindfulness of breathing and finding an anchor point, awareness of sounds, understanding the transient nature of thoughts, and walking meditation. The mindfulness practices were built up progressively, with a new element being introduced each week. In some classes, a video clip was shown to highlight the practical value of mindful awareness (e.g. “The Last Samurai”, “Losing It”). Students in the mindfulness condition were also provided with a specially designed CD, containing three 8-minute audio files of mindfulness exercises to be used outside the classroom. These audio files reflected the progressive aspects of training which the students were receiving in class. Students were encouraged to undertake daily practice by listening to the appropriate audio files. During the 4-week training period, students in the control classes attended their normal religious studies lessons.

A total of 155 participants had complete data at baseline and 134 at follow-up (78 in the mindfulness and 56 in the control condition). Any student who had missing data are at either time point was simply dropped from the analysis. The effects of this statistical decison are difficult to track in the paper. Regardless, there was a lack of any difference between intervention and control group and any of a host of outcome variables, with none designated as primary outcome.

Actual practicing of mindfulness by students was inconsistent.

One third of the group (33%) practised at least three times a week, 34.8% practised more than once but less than three times a week, and 32.7% practised once a week or less (of whom 7 respondents, 8.4%, reported no practice at all). Only two students reported practicing daily. The practice variable ranged from 0 to 28 (number of days of practice over four weeks). The practice variable was found to be highly skewed, with 79% of the sample obtaining a score of 14 or less (skewness = 0.68, standard error of skewness = 0.25).

The authors rescue their claim of a significant effect for the mindfulness intervention with highly complex multivariate analyses with multiple control variables in which outcomes within-group effects for students assigned to mindfulness  were related to the extent of students actually practicing mindfulness. Without controlling for the numerous (and post-hoc) multiple comparisons, results were still largely nonsignificant.

One simple conclusion that can be drawn is that despite a lot of encouragement, there was little actual practice of mindfulness by the relatively well-off students in a relatively highly resourced school setting. We could expect results to improve with wider dissemination to schools with less resources and less privileged students.

The authors conclude:

The main finding of this study was a significant improvement on measures of mindfulness and psychological well-being related to the degree of individual practice undertaken outside the classroom.

Recall that Mindful Nation cited the study in the following context:

What is of particular interest is that those with the lowest levels of executive control73 and emotional stability74 are likely to benefit most from mindfulness training.

These are two methodologically weak studies with largely null findings. They are hardly the basis for launching a national policy implementing universal mindfulness in the schools.

As noted in the US AHRQ report, despite a huge number of studies of mindfulness having been conducted, few involved a test with an adequate control group, and so there’s little evidence that mindfulness has any advantage over any active treatment. Neither of these studies disturbed that conclusion, although they are spun both in the original studies and in the Mindful Nation report to be positive. Both papers were published in journals where the reviewers were likely to be overly sympathetic and not at him tentative to serious methodological and statistical problems.

The committee writing Mindful Nation arrived at conclusions consistent with their prior enthusiasm for mindfulness and their vested interest in it. They sorted through evidence to find what supported their pre-existing assumptions.

Like UK resilience programs, the recommendations of Mindful Nation put considerable resources in the delivery of services to a large population and likely to have the threshold of need to register a socially in clinically significant effect. On a population level, results of the implementation are doomed to fall short of its claims. Those many fewer students in need more timely, intensive, and tailored services are left underserved. Their presence is ignored or, worse, invoked to justify the delivery of services to the larger group, with the needy students not benefiting.

In this blog post, I mainly focused on two methodologically poor studies. But for the selection of these particular studies, I depended on the search of the authors of Mindful Nation and the emphasis that were given to these two studies for some sweeping claims in the report. I will continue to be writing about the recommendations of Mindful Nation. I welcome reader feedback, particularly from readers whose enthusiasm for mindfulness is offended. But I urge them not simply to go to Google and cherry pick an isolated study and ask me to refute its claims.

Rather, we need to pay attention to the larger literature concerning mindfulness, its serious methodological problems, and the sociopolitical forces and vested interests that preserve a strong confirmation bias, both in the “scientific” literature and its echoing in documents like Mindful Nation.

Why PhD students should not evaluate a psychotherapy for their dissertation project

  • Things some clinical and health psychology students wish they had known before they committed themselves to evaluating a psychotherapy for their dissertation study.
  • A well designed pilot study addressing feasibility and acceptability issues in conducting and evaluating psychotherapies is preferable to an underpowered study which won’t provide a valid estimate of the efficacy of the intervention.
  • PhD students would often be better off as research parasites – making use of existing published data – rather than attempting to organize their own original psychotherapy study, if their goal is to contribute meaningfully to the literature and patient care.
  • Reading this blog, you will encounter a link to free, downloadable software that allows you to make quick determinations of the number of patients needed for an adequately powered psychotherapy trial.

I so relish the extra boost of enthusiasm that many clinical and health psychology students bring to their PhD projects. They not only want to complete a thesis of which they can be proud, they want their results to be directly applicable to improving the lives of their patients.

Many students are particularly excited about a new psychotherapy about which extravagant claims are being made that it’s better than its rivals.

I have seen lots of fad and fashions come and go, third wave, new wave, and no wave therapies. When I was a PhD student, progressive relaxation was in. Then it died, mainly because it was so boring for therapists who had to mechanically provide it. Client centered therapy was fading with doubts that anyone else could achieve the results of Carl Rogers or that his three facilitative conditions of unconditional positive regard, genuineness,  and congruence were actually distinguishable enough to study.  Gestalt therapy was supercool because of the charisma of Fritz Perls, who distracted us with his showmanship from the utter lack of evidence for its efficacy.

I hate to see PhD students demoralized when their grand plans prove unrealistic.  Inevitably, circumstances force them to compromise in ways that limit any usefulness to their project, and maybe even threaten their getting done within a reasonable time period. Overly ambitious plans are the formidable enemy of the completed dissertation.

The numbers are stacked against a PhD student conducting an adequately powered evaluation of a new psychotherapy.

This blog post argues against PhD students taking on the evaluation of a new therapy in comparison to an existing one, if they expect to complete their projects and make meaningful contribution to the literature and to patient care.

I’ll be drawing on some straightforward analysis done by Pim Cuijpers to identify what PhD students are up against when trying to demonstrate that any therapy is better than treatments that are already available.

Pim has literally done dozens of meta-analyses, mostly of treatments for depression and anxiety. He commands a particular credibility, given the quality of this work. The way Pim and his colleagues present a meta-analysis is so straightforward and transparent that you can readily examine the basis of what he says.

Disclosure: I collaborated with Pim and a group of other authors in conducting a meta-analysis as to whether psychotherapy was better than a pill placebo. We drew on all the trials allowing a head-to-head comparison, even though nobody ever really set out to pit the two conditions against each other as their first agenda.

Pim tells me that the brief and relatively obscure letter, New Psychotherapies for Mood and Anxiety Disorders: Necessary Innovation or Waste of Resources? on which I will draw is among his most unpopular pieces of work. Lots of people don’t like its inescapable message. But I think that if PhD students should pay attention, they might avoid a lot of pain and disappointment.

But first…

Note how many psychotherapies have been claimed to be effective for depression and anxiety. Anyone trying to make sense of this literature has to contend with claims being based on a lot of underpowered trials– too small in sample size to be expected reasonably to detect the effects that investigators claim – and that are otherwise compromised by methodological limitations.

Some investigators were simply naïve about clinical trial methodology and the difficulties doing research with clinical populations. They may have not understand statistical power.

But many psychotherapy studies end up in bad shape because the investigators were unrealistic about the feasibility of what they were undertaken and the low likelihood that they could recruit the patients in the numbers that they had planned in the time that they had allotted. After launching the trial, they had to change strategies for recruitment, maybe relax their selection criteria, or even change the treatment so it was less demanding of patients’ time. And they had to make difficult judgments about what features of the trial to drop when resources ran out.

Declaring a psychotherapy trial to be a “preliminary” or a “pilot study” after things go awry

The titles of more than a few articles reporting psychotherapy trials contain the apologetic qualifier after a colon: “a preliminary study” or “a pilot study”. But the studies weren’t intended at the outset to be preliminary or pilot studies. The investigators are making excuses post-hoc – after the fact – for not having been able to recruit sufficient numbers of patients and for having had to compromise their design from what they had originally planned. The best they can hope is that the paper will somehow be useful in promoting further research.

Too many studies from which effect sizes are entered into meta-analyses should have been left as pilot studies and not considered tests of the efficacy of treatments. The rampant problem in the psychotherapy literature is that almost no one treats small scale trials as mere pilot studies. In a recent blog post, I provided readers with some simple screening rules to identify meta-analyses of psychotherapy studies that they could dismiss from further consideration. One was whether there were sufficient numbers of adequately powered studies,  Often there are not.

Readers take their inflated claims of results of small studies seriously, when these estimates should be seen as unrealistic and unlikely to be replicated, given a study’s sample size. The large effect sizes that are claimed are likely the product of p-hacking and the confirmation bias required to get published. With enough alternative outcome variables to choose from and enough flexibility in analyzing and interpreting data, almost any intervention can be made to look good.

The problem is is readily seen in the extravagant claims about acceptance and commitment therapy (ACT), which are so heavily dependent on small, under-resourced studies supervised by promoters of ACT that should not have been used to generate effect sizes.

Back to Pim Cuijpers’ brief letter. He argues, based on his numerous meta-analyses, that it is unlikely that a new treatment will be substantially more effective than an existing credible, active treatment.  There are some exceptions like relaxation training versus cognitive behavior therapy for some anxiety disorders, but mostly only small differences of no more than d= .20 are found between two active, credible treatments. If you search the broader literature, you can find occasional exceptions like CBT versus psychoanalysis for bulimia, but most you find prove to be false positives, usually based on investigator bias in conducting and interpreting a small, underpowered study.

You can see this yourself using the freely downloadable G*power program and plug in d= 0.20 for calculating the number of patients needed for a study. To be safe, add more patients to allow for the expectable 25% dropout rate that has occurred across trials. The number you get would require a larger study than has ever been done in the past, including the well-financed NIMH Collaborative trial.

G power analyses

Even more patients would be needed for the ideal situation in which a third comparison group allowed  the investigator to show the active comparison treatment had actually performed better than a nonspecific treatment that was delivered with the same effectiveness that the other had shown in earlier trials. Otherwise, a defender of the established therapy might argue that the older treatment had not been properly implemented.

So, unless warned off, the PhD student plans a study to show not only that now hypothesis can be rejected that the new treatment is no better than the existing one, but that in the same study the existing treatment had been shown to be better than wait list. Oh my, just try to find an adequately powered, properly analyzed example of a comparison of two active treatments plus a control comparison group in the existing published literature. The few examples of three group designs in which a new psychotherapy had come out better than an effectively implemented existing treatment are grossly underpowered.

These calculations so far have all been based on what would be needed to reject the null hypothesis of no difference between the active treatment and a more established one. But if the claim is that the new treatment is superior to the existing treatment, our PhD student now needs to conduct a superiority trial in which some criteria is pre-set (such as greater than a moderate difference, d= .30) and the null hypothesis is that the advantage of the new treatment is less. We are now way out into the fantasyland of breakthrough, but uncompleted dissertation studies.

Two take away messages

 The first take away message is that we should be skeptical of claims of the new treatment is better than past ones except when the claim occurs in a well-designed study with some assurance that it is free of investigator bias. But the claim also has to arise in a trial that is larger than almost any psychotherapy study is ever been done. Yup, most comparative psychotherapy studies are underpowered and we cannot expect robust claims are robust that one treatment is superior to another.

But for PhD students been doing a dissertation project, the second take away message is that they should not attempt to show that one treatment is superior to another in the absence of resources they probably don’t have.

The psychotherapy literature does not need another study with too few patients to support its likely exaggerated claims.

An argument can be made that it is unfair and even unethical to enroll patients in a psychotherapy RCT with insufficient sample size. Some of the patients will be randomized to the control condition that is not what attracted them to the trial. All of the patients will be denied having been in a trial makes a meaningful contribution to the literature and to better care for patients like themselves.

What should the clinical or health psychology PhD student do, besides maybe curb their enthusiasm? One opportunity to make meaningful contributions to literature by is by conducting small studies testing hypotheses that can lead to improvement in the feasibility or acceptability of treatments to be tested in studies with more resources.

Think of what would’ve been accomplished if PhD students had determined in modest studies that it is tough to recruit and retain patients in an Internet therapy study without some communication to the patients that they are involved in a human relationship – without them having what Pim Cuijpers calls supportive accountability. Patients may stay involved with the Internet treatment when it proves frustrating only because they have the support and accountability to someone beyond their encounter with an impersonal computer. Somewhere out there, there is a human being who supports them and sticking it out with the Internet psychotherapy and will be disappointed if they don’t.

A lot of resources have been wasted in Internet therapy studies in which patients have not been convinced that what they’re doing is meaningful and if they have the support of a human being. They drop out or fail to do diligently any homework expected of them.

Similarly, mindfulness studies are routinely being conducted without anyone establishing that patients actually practice mindfulness in everyday life or what they would need to do so more consistently. The assumption is that patients assigned to the mindfulness diligently practice mindfulness daily. A PhD student could make a valuable contribution to the literature by examining the rates of patients actually practicing mindfulness when the been assigned to it in a psychotherapy study, along with barriers and facilitators of them doing so. A discovery that the patients are not consistently practicing mindfulness might explain weaker findings than anticipated. One could even suggest that any apparent effects of practicing mindfulness were actually nonspecific, getting all caught up in the enthusiasm of being offered a treatment that has been sought, but not actually practicing mindfulness.

An unintended example: How not to recruit cancer patients for a psychological intervention trial

Randomized-controlled-trials-designsSometimes PhD students just can’t be dissuaded from undertaking an evaluation of a psychotherapy. I was a member of a PhD committee of a student who at least produced a valuable paper concerning how not to recruit cancer patients for a trial evaluating problem-solving therapy, even though the project fell far short of conducting an adequately powered study.

The PhD student was aware that  claims of effectiveness of problem-solving therapy reported in in the prestigious Journal of Consulting and Clinical Psychology were exaggerated. The developer of problem-solving therapy for cancer patients (and current JCCP Editor) claimed  a huge effect size – 3.8 if only the patient were involved in treatment and an even better 4.4 if the patient had an opportunity to involve a relative or friend as well. Effect sizes for this trial has subsequently had to be excluded from at least meta-analyses as an extreme outlier (1,2,3,4).

The student adopted the much more conservative assumption that a moderate effect size of .6 would be obtained in comparison with a waitlist control. You can use G*Power to see that 50 patients would be needed per group, 60 if allowance is made for dropouts.

Such a basically inert control group, of course, has a greater likelihood of seeming to demonstrate a treatment is effective than when the comparison is another active treatment. Of course, such a control group also has the problem of not allowing a determination if it was the active ingredient of the treatment that made the difference, or just the attention, positive expectations, and support that were not available in the waitlist control group.

But PhD students should have the same option as their advisors to contribute another comparison between an active treatment and a waitlist control to the literature, even if it does not advance our knowledge of psychotherapy. They can take the same low road to a successful career that so many others have traveled.

This particular student was determined to make a different contribution to the literature. Notoriously, studies of psychotherapy with cancer patients often fail to recruit samples that are distressed enough to register any effect. The typical breast cancer patient, for instance, who seeks to enroll in a psychotherapy or support group trial does not have clinically significant distress. The prevalence of positive effects claimed in the literature for interventions with cancer patients in published studies likely represents a confirmation bias.

The student wanted to address this issue by limiting patients whom she enrolled in the study to those with clinically significant distress. Enlisting colleagues, she set up screening of consecutive cancer patients in oncology units of local hospitals. Patients were first screened for self-reported distress, and, if they were distressed, whether they were interested in services. Those who met both criteria were then re-contacted to see if that be willing to participate in a psychological intervention study, without the intervention being identified. As I reported in the previous blog post:

  • Combining results of  the two screenings, 423 of 970 patients reported distress, of whom 215 patients indicated need for services.
  • Only 36 (4% of 970) patients consented to trial participation.
  • We calculated that 27 patients needed to be screened to recruit a single patient, with 17 hours of time required for each patient recruited.
  • 41% (n= 87) of 215 distressed patients with a need for services indicated that they had no need for psychosocial services, mainly because they felt better or thought that their problems would disappear naturally.
  • Finally, 36 patients were eligible and willing to be randomized, representing 17% of 215 distressed patients with a need for services.
  • This represents 8% of all 423 distressed patients, and 4% of 970 screened patients.

So, the PhD student’s heroic effort did not yield the sample size that she anticipated. But she ended up making a valuable contribution to the literature that challenges some of the basic assumptions that were being made about how cancer patients in psychotherapy research- that all or most were distressed. She also ended up producing some valuable evidence that the minority of cancer patients who report psychological distress are not necessarily interested in psychological interventions.

Fortunately, she had been prepared to collect systematic data about these research questions, not just scramble within a collapsing effort at a clinical trial.

Becoming a research parasite as an alternative to PhD students attempting an under-resourced study of their own

research parasite awardPsychotherapy trials represent an enormous investment of resources, not only the public funding that is often provided for them,be a research parasite but in the time, inconvenience, and exposure to ineffective treatments experienced by patients who participate in the trials. Increasingly, funding agencies require that investigators who get money to do a psychotherapy study some point make their data available for others to use.  The 14 prestigious medical journals whose editors make up the International Committee of Medical Journal Editors (ICMJE) each published in earlier in 2016 a declaration that:

there is an ethical obligation to responsibly share data generated by interventional clinical trials because participants have put themselves at risk.

These statements proposed that as a condition for publishing a clinical trial, investigators would be required to share with others appropriately de-identified data not later than six months after publication. Further, the statements proposed that investigators describe their plans for sharing data in the registration of trials.

Of course, a proposal is only exactly that, a proposal, and these requirements were intended to take effect only after the document is circulated and ratified. The incomplete and inconsistent adoption of previous proposals for registering of  trials in advance and investigators making declarations of conflicts of interest do not encourage a lot of enthusiasm that we will see uniform implementation of this bold proposal anytime soon.

Some editors of medical journals are already expressing alarmover the prospect of data sharing becoming required. The editors of New England Journal of Medicine were lambasted in social media for their raising worries about “research parasites”  exploiting the availability of data:

a new class of research person will emerge — people who had nothing to do with the design and execution of the study but use another group’s data for their own ends, possibly stealing from the research productivity planned by the data gatherers, or even use the data to try to disprove what the original investigators had posited. There is concern among some front-line researchers that the system will be taken over by what some researchers have characterized as “research parasites.”

 Richard Lehman’s  Journal Review at the BMJ ‘s blog delivered a brilliant sarcastic response to these concerns that concludes:

I think we need all the data parasites we can get, as well as symbionts and all sorts of other creatures which this ill-chosen metaphor can’t encompass. What this piece really shows, in my opinion, is how far the authors are from understanding and supporting the true opportunities of clinical data sharing.

However, lost in all the outrage that The New England Journal of Medicine editorial generated was a more conciliatory proposal at the end:

How would data sharing work best? We think it should happen symbiotically, not parasitically. Start with a novel idea, one that is not an obvious extension of the reported work. Second, identify potential collaborators whose collected data may be useful in assessing the hypothesis and propose a collaboration. Third, work together to test the new hypothesis. Fourth, report the new findings with relevant coauthorship to acknowledge both the group that proposed the new idea and the investigative group that accrued the data that allowed it to be tested. What is learned may be beautiful even when seen from close up.

The PLOS family of journals has gone on record as requiring that all data for papers published in their journals be publicly available without restriction.A February 24, 2014 PLOS’ New Data Policy: Public Access to Data  declared:

In an effort to increase access to this data, we are now revising our data-sharing policy for all PLOS journals: authors must make all data publicly available, without restriction, immediately upon publication of the article. Beginning March 3rd, 2014, all authors who submit to a PLOS journal will be asked to provide a Data Availability Statement, describing where and how others can access each dataset that underlies the findings. This Data Availability Statement will be published on the first page of each article.

Many of us are aware of the difficulties in achieving this lofty goal. I am holding my breath and turning blue, waiting for some specific data.

The BMJ has expanded their previous requirements for data being available:

Loder E, Groves T. The BMJ requires data sharing on request for all trials. BMJ. 2015 May 7;350:h2373.

The movement to make data from clinical trials widely accessible has achieved enormous success, and it is now time for medical journals to play their part. From 1 July The BMJ will extend its requirements for data sharing to apply to all submitted clinical trials, not just those that test drugs or devices. The data transparency revolution is gathering pace.

I am no longer heading dissertation committees after one that I am currently supervising is completed. But if any PhD students asked my advice about a dissertation project concerning psychotherapy, I would strongly encourage them to enlist their advisor to identify and help them negotiate access to a data set appropriate to the research questions they want to investigate.

Most well-resourced psychotherapy trials have unpublished data concerning how they were implemented, with what bias and with which patient groups ending up underrepresented or inadequately exposed to the intensity of treatment presumed to be needed for benefit. A story awaits to be told. The data available from a published trial are usually much more adequate than then any graduate student could collect with the limited resources available for a dissertation project.

I look forward to the day when such data is put into a repository where anyone can access it.

until youre done In this blog post I have argued that PhD students should not take on responsibility for developing and testing a new psychotherapy for their dissertation project. I think that using data from existing published trials is a much better alternative. However, PhD students may currently find it difficult, but certainly not impossible to get appropriate data sets. I certainly am not recruiting them to be front-line infantry in advancing the cause of routine data sharing. But they can make an effort to obtain such data and they deserve all support they can get from their dissertation committees in obtaining data sets and in recognizing when realistically that data are not being made available, even when the data have been promised to be available as a condition for publishing. Advisors, please request the data from published trials for your PhD students and protect them from the heartache of trying to collect such data themselves.

 

Creating the illusion that mindfulness improves the survival of cancer patients

  • A demonstration of just how unreliable investigators’ reports of mindfulness studies can be.
  • Exaggerations of efficacy combined with self-contradiction in the mindfulness literature pose problems for any sense being made of the available evidence by patients, clinicians, and those having responsibility for clinical and public policy decisions.

Despite thousands of studies, mindfulness-based stress reduction (MBSR) and related meditation approaches have not yet been shown  to be more efficacious than other active treatments for reducing stress. Nonetheless many cancer patients seek MBSR or mindfulness-based cancer recovery (MBCR) believing that they are improving their immune system and are on their way to a better outcome in “fighting” their cancer.

UVa Cancer Center
UVa Cancer Center

This unproven claim leads many cancer patients to integrative cancer centers. Once patients begin receiving treatment at these centers, they are offered a variety of other services that can be expensive, despite being unproven or having been proven ineffective. Services provided by integrative cancers treatments can discourage patients from seeking conventional treatments that are more effective, but that come with serious side effects and disfigurement. Moreover, integrative treatments give false hope to patients who would otherwise accept the limits of treatments for cancer and come to terms with their own mortality. And integrative treatments can lead to patients blaming themselves when they do not benefit.

Mindfulness studies keep being added to the literature, often in quality journals, that cultivate these illusions of vulnerable cancer patients. This psychoneuroimmunology(PNI) literature is self-perpetuating in its false claims, exaggerations, and spin. The literature ignores some basic findings:

  1. Psychotherapy and support groups have not been shown to improve the survival of cancer patients.
  2. The contribution of stress to the onset, progression, and outcome of cancer is likely to be minimal, if at all.
  3. Effects of psychological interventions like MBSR/MBCR on the immune system are weak or nonexistent, and the clinical significance of any effects is not established.

Evidence-based oncologists and endocrinologists would not take seriously the claims regularly appearing in the PNI literature. Such clinician-scientists would find bizarre many of the supposed mechanisms by which MBCR supposedly affects cancer. Yet, investigators create the illusion of accumulating evidence, undaunted by negative findings and the lack of plausible mechanisms by which MBCR could conceivably influence basic disease processes in cancer.

This blog post debunks a study by one of the leading proponents of MBCR for cancer patients, showing how exaggerated and outright false claims are created and amplified across publications.

Responsible scientists and health care providers should dispel myths that patients may have about the effectiveness of psychosocial treatments in extending life. But in the absence of responsible professionals speaking out, patients can be intimidated by how these studies are headlined in the popular media, particularly when they believe that they are dealing with expert opinion based on peer-reviewed studies.

Mindfulness-based cancer recovery (MBCR)

The primary report for study was published in the prestigious Journal of Clinical Oncology and is available as a downloadable PDF 

Carlson LE, Doll R, Stephen J, Faris P, Tamagawa R, Drysdale E, Speca M. Randomized controlled trial of mindfulness-based cancer recovery versus supportive expressive group therapy for distressed survivors of breast cancer (MINDSET).  Journal of Clinical Oncology. 2013 Aug 5:JCO-2012.

The authors compared the efficacy of what they describe as“two empirically supported group interventions to help distressed survivors of breast cancer”: mindfulness-based cancer recovery (MBCR) and supportive-expressive group therapy (SET). Each of these active treatments was delivered in 8 weekly 90 minute sessions plus a six-hour workshop. A six-hour, one day didactic seminar served as the comparison/control condition.

The 271 participants were Stage I, II, or III breast patients who had completed all cancer treatment a mean of two years ago. Patients also had to meet a minimal level of distress and not have a psychiatric diagnosis.  Use of psychotropic medication was not an exclusion, because of the high prevalence of antidepressants and anxiolytics in this population.

One hundred thirteen patients were randomized to MBSR, 104 to SET, and 54 to the didactic seminar control group.

A full range of self-measures was collected, along with saliva samples at four times (awakening, noon,  5 PM, and bedtime) over three days. The trial registration for this study is basically useless. It is lacking in basic detail.  Rather than declaring one or maybe two outcomes as primary, the authors specify broad classes – mood, stress, post-traumatic growth, social support, quality of life, spirituality and cortisol levels (stress hormone). Yet,

A later report  states that ”The sample size estimate was based on the primary outcome measure (POM TMD)” – Profile of Mood Total Mood score. The saliva collection was geared to assessing cortisol, although in such studies saliva can provide a full range of biological variables, including immune function.

Why bring up the lack of registration and multiple outcome measures?

The combination of a vague trial registration and multiple outcome measures allows investigators considerable flexibility in which outcome they pick. They can wait to make a choice until after results are known, but that is considered a questionable research practice. The collection of saliva was obviously geared to assessing saliva cortisol. However, a recent comprehensive review  of salivary diurnal cortisol as an outcome measure at least three parameters (the cortisol awakening response, diurnal slope an area under the curve), each reflecting different aspects of hypothalamus pituitary adrenal) HPA axis function.

So, the authors have a lot of options from which to choose data points and analyses best suggesting that MBCR is effective.

Results

Cortisol-levels-400x210Modest effects on POMS TMS disappeared in corrected pairwise comparisons between MBSR in SET. So, according to the evidence presented, mood was not improved.

Baseline cortisol data were only available for 242 patients, and only 172 had data for post intervention slopes. Uncorrected group differences in cortisol slope across the day are not reported. However, when cancer severity number of cigarettes smoked per day, and sleep quality were entered as control variables, a group X time difference was found (p< .009).

We should beware of studies that do not present uncorrected group differences, but depend on only data adjusted for covariates, the appropriateness of which is not established.

But going further, there was no difference between MBCR and SEM. Actually, any difference between these two groups and the control with due to an unexpected increase in the control group slope while the patients in the MBCR and SEM remained unchanged. I can’t see how this would have been predicted. The assumption guiding the study had been that cortisol slope should decrease one or both of the active intervention groups.

The authors searched for more positive findings from cortisol and found:

There were no significant group x time interaction effects for cortisol concentrations at any single collection point, but a time x group contrast between MBCR and SMS was significant for bedtime cortisol concentrations (P =.044; Table 3), which were elevated after SMS (mean change, 0.11) but slightly decreased after MBCR (mean change,=0.02; Fig 2D).

These are weak findings revealed by a post hoc search of a number of different cortisol measures. Aside from the analysis been post-hoc,  I would not place much confidence in a cherry-picked p = .044.

How the authors discuss the results

 Ignoring the null results for the primary measure, the Profile of Mood States Total Score (POM TS), the authors jump to secondary outcomes to proclaim the greater effectiveness of MBCR:

As predicted, MBCR emerged as superior for decreasing symptoms of stress and also for improving overall quality of life and social support in these women, even though we hypothesized that SET might be superior on social support. Improvements were clinically meaningful and similar to those reported in our previous work with mixed groups of patients with cancer.

Keep in mind the disappointing result for cortisol profiles when reading their closing claims for “significantly altered” cortisol:

Cortisol profiles were significantly altered after program completion. Participants in both MBCR and SET maintained the initial steepness of cortisol slopes, whereas SMS participants evidenced increasingly flatter diurnal cortisol slopes, with a medium between-group effect size. Hence, the two interventions buffered unfavorable biologic changes that may occur without active psychosocial intervention. Because abnormal or flattened cortisol profiles have been related to both poorer psychological functioning and shorter survival time in breast,16,17,45,46 lung,47 and renal cell48 carcinoma, this finding may point to the potential for these psychosocial interventions to improve biologic processes related to both patient-reported outcomes and more objective indices. More work is needed to fully understand the clinical meaning of these parameters in primary breast cancer.

The authors set out to demonstrate this psychological interventions decreased cortisol slopes and found no evidence that they did. However, they seized on the finding of increasingly flatter cortisol slopes in the control group. But all these breast cancer patients are receiving MBCR and SET two years after their cancer treatment ended. For most patients, distress levels have receded by then to what they were before cancer was detected. One has to ask the authors if they are taking seriously this continuing decline in cortisol slopes, where are cortisol levels heading?  And when did the decline start?

I attach no credibility to the authors’ claims unless they provide us with an understanding of how they occurred. Do the authors assume they have an odd group of patients who have been declining since diagnosis or maybe since the end of active cancer treatment, but have somehow ended up at the same level of cortisol as the other patients in the sample? There was, you know, random assignment and, there were no baseline differences at the start of this study.

The attempt to relate their findings to shorter survival time in a variety of cancers is dodgy and irresponsible. Their overview of the literature is highly selective, depends on small samples, and there is no evidence that the alleged flattened cortisol profiles are causes rather than being an effect of disease parameters associated with shorter survival.

The authors have not demonstrated an effect of their psychological interventions on survival. No previous study ever has.

Interestingly, a classic small study by Spiegel prompted a whole line of research in which an effect of psychological intervention on survival was sought. However, a careful look at the graphs in his original study reveals that the survival curves for the patients receiving the intervention approximated with other patients with advanced breast cancer in the larger community in the absence of intervention. Compared to the large population from which they were drawn, the patients receiving the intervention in Spiegel’s study were no better off.

survival curve-page-0In the contrast, there were unexplainable deaths in Spiegel’s  control group that generated the illusion that his intervention was increasing survival. Given how small his control group was (39 patients at the outset), it only took the sudden death of four patients in the control group to create an effect where previously there was none. So, it is not that psychotherapy extended survival,  but that a small cluster of patients in the control group died suddenly, years after randomization. Go figure, but keep in mind that the study was never designed to test the effects of psychological intervention on survival. That hypothesis was generated after data were available and Spiegel claimed surprise that they were positive findings.

Spiegel himself has never been able to replicate this finding. You can read more about this study here.

From Hilda Bastian
From Hilda Bastian

The present authors did not identify survival has a primary outcome for the trial, nor did they assess it. They are essentially depending on spun data that assumes cortisol slope not just as a biological variable, but a surrogate for survival. See a blog post by Hilda Bastian’s Statistically funny: Biomarkers Unlimited: Accept Only OUR Substitutes!  for an explanation of why this is sheer folly. Too many promising medical treatments for cancer have been accepted as efficacious on the basis of surrogate outcomes, only to be later shown to have no effect on survival. But these psychological treatments are not even in the running.

This is the kind of nonsense that encourages cancer patients to continue with the false hope that mindfulness-based treatment will extend their lives.

1681869-slide-aforanimation-sitonmyface-1The fish gets bigger with each telling.

A follow up paper  makes stronger claims and makes new claims of telomere length,  the clinical implications of which the authors ultimately concede they don’t understand.

Carlson LE, Beattie TL, Giese‐Davis J, Faris P, Tamagawa R, Fick LJ, Degelman ES, Speca M. Mindfulness‐based cancer recovery and supportive‐expressive therapy maintain telomere length relative to controls in distressed breast cancer survivors. Cancer. 2015 Feb 1;121(3):476-84.

The authors opening summary of their previously reported results we have been discussing:

We recently reported primary outcomes of the MINDSET trial, which compared 2 empirically supported psychosocial group interventions, mindfulness-based cancer recovery (MBCR) and supportive-expressive group therapy (SET), with a minimal-intervention control condition on mood, stress symptoms, quality of life, social support, and diurnal salivary cortisol in distressed breast cancer survivors.[4] Although MBCR participation resulted in the most psychosocial benefit, including improvements across a range of psychosocial outcomes, both MBCR and SET resulted in healthier cortisol profiles over time compared with the control condition.

Endocrinologists would scratch their heads and laugh at the claim that intervention resulted in “healthier cortisol profiles.” There is a wide range of cortisol values in the general population, and these are well within the normal range. The idea that they are somehow “healthier” is as bogus as claims made for super foods and supplements. You have to ask, “healthier” in what sense?

In this secondary analysis of MINDSET trial data, we collected and stored blood samples taken from a subset of women to further investigate the effects of these interventions on potentially important biomarkers. Telomeres are specialized nucleoprotein complexes that form the protective ends of linear chromosomes and provide genomic stability through several mechanisms.

The authors justify the study with speculations that stop just short of claiming their intervention increased survival:

Telomere dysfunction and the loss of telomere integrity may result in DNA damage or cell death; when a critically short telomere length (TL) is reached, cells enter senescence and have reduced viability, and chromosomal fusions appear.[6] Shorter TL has been implicated in several disease states, including cardiovascular disease, diabetes, dyskeritosis congenita, aplastic anemia, and idiopathic pulmonary fibrosis.[7] Shorter TL also was found to be predictive of earlier mortality in patients with chronic lymphocytic leukemia,[8] promyelocytic leukemia,[9] and breast cancer.[10-12] However, the relationships between TL and the clinical or pathological features of tumors are still not clearly understood.[13].

They waffle some more and then acknowledge there are few relevant data concerning cancer:

Telomere dysfunction and the loss of telomere integrity may result in DNA damage or cell death; when a critically short telomere length (TL) is reached, cells enter senescence and have reduced viability, and chromosomal fusions appear.[6] Shorter TL has been implicated in several disease states, including cardiovascular disease, diabetes, dyskeritosis congenita, aplastic anemia, and idiopathic pulmonary fibrosis.[7] Shorter TL also was found to be predictive of earlier mortality in patients with chronic lymphocytic leukemia,[8] promyelocytic leukemia,[9] and breast cancer.[10-12] However, the relationships between TL and the clinical or pathological features of tumors are still not clearly understood.[13].

Too small a sample to find anything clinically significant and generalizable

Correlational studies of telomere length and disease require very large samples. These epidemiologic findings in no way encourage anticipating finding effects in a modest sized trial of a psychological intervention. Moreover, significant results from smaller studies exaggerate associations because they have to be larger to be statistically significant.  They not be expected to replicate in a larger study. The authors’ sample has shrunk considerably from recruitment and randomization to a sample of women provided two blood samples with which they hope to find differences among two interventions and one control group.

Due to the availability of resources, blood samples were only collected in Calgary. Of the 128 women in Calgary, 5 declined to donate their blood. Thirty-one women provided their blood only at the preintervention time period; therefore, the current study included 92 women who donated a blood sample before and after the intervention.

Not surprisingly, no differences between groups were found, but that inspires some creativity in analysis.

The results of ANCOVA demonstrated no statistical evidence of differences in postintervention TL between the MBCR and SET interventions after adjusting the impact of the preintervention log10 T/S ratios. The mean difference was −0.12 (95% confidence interval [95% CI], −0.74 to 0.50). Because the 2 interventions shared similar nonspecific components and no significant differences emerged in their baseline-adjusted postintervention T/S ratios, the 2 intervention groups were subsequently combined to allow greater power for detecting any effects on TL related to participation in a psychosocial intervention compared with the control condition.

The authors initially claimed that MBCR and SET were so different that an expensive large scale RCT was justified. Earlier in the present paper they claimed MBCR was superior. But now they are claiming there is so little difference between  treatments that a post hoc combining is justified to see if null findings can be overturned.

Their tortured post hoc analyses revealed a tiny effect that they fail to acknowledge was nonsignificat – confidence intervals (-0.01 to 1.35)  include 0:

After adjustment for the baseline log10 T/S ratio, there was a statistical trend toward a difference in posttreatment log10 T/S ratios between treatment and control subjects (statistics shown in Table 2). The adjusted mean difference was 0.67 (95% CI, -0.01 to 1.35). The effect size of g2 was 0.043 (small to medium).

There was no association between psychological outcomes and telomere length. Yet differences would be expected if interventions targeting psychological variables somehow influenced telomere length.

Nonetheless, the authors concluded they had a pattern in the results of the primary and secondary studies encouraging more research:

Together, these changes suggest an effect of the interventions on potentially important biomarkers of psychosocial stress. Given the increasingly well-documented association between TL and cancer initiation46 and survival,47 this finding adds to the literature supporting the potential for stress-reducing interventions to impact important disease-regulating processes and ultimately disease outcome.

They end with a call for bigger, more expensive studies, even if they cannot understand what is going on (or for that matter, whether anything of interest occurred in their study):

Future investigators should power studies of intervention effects on TL and telomerase as primary outcomes, and follow participants over time to better understand the clinical implications of group differences. The interpretation of any changes in TL in patients with breast cancer is difficult. One study that analyzed TL in breast tumor tissue found no relations between TL and any clinical or pathological features or disease or survival outcomes,13 whereas other studies have shown that TL was related to breast cancer risk46,51 and survival.10,46,47 Although interpretation remains difficult,the results of the current study nonetheless provide provocative new data that suggest it is possible to influence TL in cancer survivors through the use of psychosocial interventions involving group support, emotional expression, stress reduction, and mindfulness meditation.

This is not serious research. At the outset, the authors had to know that the sample was much too small and there been too much nonrandom attrition to make robust and generalizable conclusions concerning effects on telomere length. And the authors knew ahead of time, they had no idea how they would interpret such effect. But they didn’t find them. They delivered an intervention, administered questionnaires, took spit and blood samples, but this is not “research” in which they were willing to concede hypotheses were confirmed, this is an experimercial for mindfulness programs.

exaggeration-300x290But the power of MBCR gets even greater with yet another telling

A recent review:

Carlson LE. Mindfulness‐based interventions for coping with cancer. Annals of the New York Academy of Sciences. 2016 Mar

One of the authors of the two articles we have been discussing uses them as the main basis for even stronger claims and about MBCR specifically.

Our adaptation, mindfulness-based cancer recovery (MBCR), has resulted in improvements across a range of psychological and biological outcomes, including cortisol slopes, blood pressure, and telomere length, in various groups of cancer survivors.

Wow! Specifically,

Overall, women in the MBCR group showed more improvement on stress symptoms compared with women in both the SET and control groups, on QOL compared with the control group, and in social support compared with the SET group,[28] but both active-intervention groups’ cortisol slopes (a marker of stress responding) were maintained over time relative to the control group, whose cortisol slopes became flatter. Steeper slopes are generally considered to be healthier. The two intervention groups also maintained their telomere length, a potentially important marker of cell aging, over time compared to controls,

But wait! The superiority of MBCR gets even better with a follow-up study.

The publication of long term follow up data become the occasion for describing the superiority of MBCR over SET as ever greater.

Carlson LE, Tamagawa R, Stephen J, Drysdale E, Zhong L, Speca M. Randomized‐controlled trial of mindfulness‐based cancer recovery versus supportive expressive group therapy among distressed breast cancer survivors (MINDSET): long‐term follow‐up results. Psycho‐Oncology. 2016 Jan 1.

 The abstract describes the outcomes at the end of the intervention:

Immediately following the intervention, women in MBCR reported greater reduction in mood disturbance (primarily fatigue, anxiety and confusion) and stress symptoms including tension, sympathetic arousal and cognitive symptoms than those in SET. They also reported increased emotional and functional quality of life, emotional, affective and positive social support, spirituality (feelings of peace and meaning in life) and post-traumatic growth (appreciation for life and ability to see new possibilities) relative to those in SET, who also improved to a lesser degree on many outcomes.

A search for “cortisol” in this report finds it is never mentioned.

The methods section clarifies that the 54 women in the seminar control group were offered randomization to the two active treatments and 35 accepted, with 21 going to MBCR and 14 to SET. However, 8 of the women newly assigned to MBCR and 9 of the women newly assigned to SET did not provide post-intervention data.  The authors nonetheless used two-level piecewise hierarchical linear modelling (HLM) with random intercepts for intent-to-treat analyses for the full sample. The authors acknowledge a high attrition rate of over half of the patients being lost to follow up, but argue these hierarchical analyses were a solution. While this is often done, the analyses assumes attrition is random and validity is vulnerable to such high rates of attrition. I don’t know why a reviewer did not object to the analyses or the strong conclusions drawn from them.

Recognize what is being done here: the authors are including a small amount of new data in analyses, but with so much attrition by the end of treatment  that analyses depend more on estimating from data available from a minority of patients to what the authors claim would be obtained if the full sample were involved. This is statistically dodgy, but apparently acceptable to the stats editor of this journal. What the authors did is not considered fraud, but it is making up data.

The follow up study concludes:

In sum, these results represent the first demonstration in a comparative effectiveness approach that MBCR is superior to another active intervention, SET, which also showed lesser benefit to distressed survivors of breast cancer. Our previous report also showed that MBCR was superior to a minimal intervention control condition pre-intervention to post-intervention. Benefits were accrued across outcomes measuring stress, mood, quality of life and PTG, painting a picture of women who were more able to cope with cancer survivorship and to fully embrace and enjoy life.

I pity the poor detached investigator attempting to use these data in a meta-analysis. Do they go with the original, essentially null results, or do they rely on these voodoo statistics that post-hoc give a better picture. They would have to write to the authors anyway, because on corrected results are presented in the paper.

This is not science, it is promotion of a treatment by enthusiastic proponents who are strongly committed to demonstrating that the treatment is superior to alternatives, in defiance of contradictory data they have generated.

Terribly disappointing, but this effort is actually better than much of the studies of mindfulness for cancer patients. It is a randomized trial, and started with a reasonably large sample, even if it has substantial attrition – i.e., most patients were lost to follow-up.

For those you who have actually read this longread blog post from start to finish, would you have expected this kind of background if you’d only stumbled upon the authors’ glowing praise of their own work in the prestigious Annals of New York Academy of Sciences? I don’t think so.

Dammit! It shouldn’t be so hard to figure out what went on studies. We should be able to depend on authors to provide more transparent, consistent reports of the results they obtain. While mindfulness research has no monopoly on such contrary practices, it is exceptionally rich with exaggerated and even false claims and suppression of evidence to the contrary. Consumers be very skeptical of what they read!

Let’s get more independent re-evaluations of the claims made by promoters of mindfulness by those who don’t profit professionally or financially from exaggerating benefits. And please, clinicians, start dispelling the myths of cancer patients who think that they are obtaining effects on their disease from practicing mindfulness.

For further discussion, see Mindfulness-based stress reduction for improving sleep among cancer patients: A disappointing look.

 

 

 

 

Mindfulness research’s huge problem with uninformative control groups

Are enthusiasts protecting cherished beliefs about the power of mindfulness from disconfirmation?

Do any advantages of mindfulness training disappear in a fairly matched cage fight with a treatment of comparable frequency and intensity?

  • Very few of the 1000s of articles retrieved in a literature search with the keyword “mindfulness” represent advances in the limited evidence that mindfulness-based stress reduction (MBSR) is effective for physical health problems.
  • Only a few randomized controlled trials with appropriate control groups are available and they do not offer strong evidence for the efficacy of MBSR.
  • This blog post demonstrates how uninformative and misleading comparisons with no treatment or treatment as usual/routine care can be.
  • While the lack of adequately controlled studies could have initially reflected the naïveté of MBSR researchers, increasing acknowledgment of the problem suggests enthusiasts’ avoidance of confronting cherished beliefs with disconfirming evidence.
  • When cage fights are arranged between MBSR and appropriate active control groups, the alternative treatments are often shown to be superior and more cost-effective, even when MBSR enthusiasts are the referees.

A comprehensive systematic review and meta-analysis prepared for the US Agency for Healthcare Research and Quality (AHRQ)

Goyal M, Singh S. Sibinga EMS, et al. Meditation programs for psychological stress and well-being: a systematic review and meta-analysis. JAMA Intern Med. Epub Jan 6 2014. doi:10.1001/jamainternmed.2013.13018.

Reviewed 18,753 citations, and found only 47 trials (3%) with 3515 participants that included an active control treatment.

Mindfulness meditation programs had moderate evidence of improved anxiety (effect size, 0.38 [95%CI, 0.12-0.64] at 8 weeks and 0.22 [0.02-0.43] at 3-6 months), depression (0.30 [0.00-0.59] at 8 weeks and 0.23 [0.05-0.42] at 3-6 months), and pain (0.33 [0.03- 0.62]) and low evidence of improved stress/distress and mental health–related quality of life. We found low evidence of no effect or insufficient evidence of any effect of meditation programs on positive mood, attention, substance use, eating habits, sleep, and weight. We found no evidence that meditation programs were better than any active treatment (ie, drugs, exercise, and other behavioral therapies).

An accompanying commentary on the review asked:

The modest benefit found in the study by Goyal et al begs the question of why, in the absence of strong scientifically vetted evidence, meditation in particular and complementary measures in general have become so popular, especially among the influential and well educated…What role is being played by commercial interests? Are they taking advantage of the public’s anxieties to promote use of complementary measures that lack a base of scientific evidence? Do we need to require scientific evidence of efficacy and safety for these measures?

A reminder: treatments do not have effect sizes.

MBSR does not have an effect size. Rather, comparisons of MBSR to other conditions have effect sizes, which will vary greatly with the comparison treatment and population being studied.

Not just any comparison/control condition will do.

A comparison/control condition must be suitably matched with MBSR in terms of frequency and intensity of contact, positive expectations, and overall levels of support and attention. MBSR treatments typically involve weekly meetings, daylong workshops or retreats, and the expectations that patients will practice mindfulness daily.

Construction of an adequate control condition that matches these features can be challenging.

Comparisons of MBSR with wait list controls and no treatment control conditions produce exaggerated effect sizes for active treatments and may produce positive findings were no differences would be found with an adequate control group.

The domination of the MBSR literature by nonrandomized trials and randomized trials with inadequate control groups represents one contribution to an exaggeration of the efficacy of MBSR.

Demonstrating how uninformative and even misleading poorly chosen control groups can be.

 Spirometry_NIHA study published in NEJM that did not evaluate MBSR nonetheless demonstrates how misleading poorly chosen control groups can be, especially for physical health outcomes.

Wechsler ME, Kelley JM, Boyd IO, Dutile S, Marigowda G, Kirsch I, Israel E, Kaptchuk TJ. Active albuterol or placebo, sham acupuncture, or no intervention in asthma. New England Journal of Medicine. 2011 Jul 14;365(2):119-26.

 This randomized, double-blind, crossover pilot study involved screening 79 patients, of whom 46 with mild-to-moderate asthma met the entry criteria, and were randomly assigned to one of four study interventions. An inhaled albuterol bronchodilator was compared to one of three control conditions placebo inhaler, sham acupuncture, or no intervention. Figure 4 from the article presents subjective outcomes for two self-report measures, perceived improvements in asthma symptoms on a visual-analogue scale and perceived credibility of treatment.

percent change subjectivePatients reported substantial improvement not only with inhaled albuterol (50% improvement) but also with inhaled placebo (45%) and with sham acupuncture (46%). In contrast, the improvement reported with no intervention was only 21%. The difference in the subjective drug effect between the active albuterol inhaler and the placebo inhaler was not significant (P=0.12), and the observed effect size was small (d=0.21). With respect to the placebo effects, however, the difference between the two placebo interventions and no intervention was large (d=1.07 for placebo inhaler and d=1.11 for sham acupuncture) and significant (P<0.001 for both comparisons). Treatment credibility was high, and most patients believed that they had received active treatment (73% for double-blind albuterol, 66% for double-blind placebo inhaler, and 85% for sham acupuncture). The two double-blind conditions did not differ significantly from each other, but sham acupuncture was significantly more credible than both inhaler conditions (P<0.05).

Figure 3 from the article  presents the outcomes for an objective measure physiological responses – improvement in forced expiratory volume (FEV1), measured with spirometry   to each intervention (albuterol inhaler, placebo inhaler, sham acupuncture, and no intervention) across the three study visits.

percent chane objective

The mean percent improvement in FEV1 was 20.1±1.6% with inhaled albuterol, as compared with 7.5±1.0% with inhaled placebo, 7.3±0.8% with sham acupuncture, and 7.1±0.8% with the no-intervention control. There were no significant differences between the three inactive interventions, none of which resulted in the degree of improvement observed with active albuterol. The difference in drug effect between the albuterol inhaler and the placebo inhaler, as indexed by the difference in mean percent improvement in FEV1, was significant (P<0.001) and large (d=1.48). In contrast, the placebo effects did not differ significantly between the two placebo interventions and the no-intervention control (P=0.65 for the comparison of placebo inhaler with no intervention, and P=0.75 for the comparison of sham acupuncture with no intervention).

The authors concluded:

In this repeated-measures pilot study in which active-drug and placebo effects were assessed in patients with asthma, two different types of placebo had no objective bronchodilator effect beyond the improvement that occurred when patients received no intervention of any kind and simply underwent repeated spirometry (no-intervention control). In contrast, the subjective improvement in asthma symptoms with both inhaled placebo and sham acupuncture was significantly greater than the subjective improvement with the no-intervention control and was similar to that with the active drug.

Relevance to Studies of MBSR.

 Claims for the efficacy of MBSR depend heavily on RCTs comparing MBSR to waitlist. I’m unaware of comparisons of the standard waitlist control condition to more appropriate comparison/control conditions.  However, this unusual pilot study provides some suggestive evidence that a waitlist is seriously deficient when compared to credible comparison/control conditions for which patients are likely to have positive expectations.

We should be cautious in interpreting these results because we will be comparing effect sizes across different kinds of studies. But with this caution in mind, we can see that for subjective self-report measures, the large difference between placebo conditions with positive expectations and no treatment is certainly greater than the differences typically found between the MBSR and a waitlist.

The difference between a waitlist control group and a blinded control with blinding group with positive expectations is considerably greater than the difference between MBSR and a waitlist control group. This spells trouble for anyone wanting to crow about MBSR.

It’s not an unreasonable inference that comparison with more appropriate comparison/control conditions will eliminate any advantage of MBSR. I would welcome a direct test of this hypothesis by pitting MBSR against a placebo condition with positive expectations and another comparison control condition like waitlist or no treatment.

The contrast between results from subjective self-report and objective outcomes should be troubling to those needing to evaluate MBSR or other psychological interventions for clinical or health policy applications.  If one relies on studies with subjective self-report as the primary outcome, the risk is that differences for objective health measures will be missed and ineffective treatments will be accepted as effective. Ouch!

For a large proportion of studies of psychological interventions for chronic health conditions, the primary outcomes are indeed subjective self-report. Even when conceptually possible, objective measures of health conditions are either not included or they are deemphasized as secondary outcomes. The message of this study, again delivered with appropriate caution, is that we should not be generalizing from results obtained with subjective self-report to objective health outcomes.

In defense of MBSR researchers, they might not just be defending against his confirmation of a cherished belief. They may also be avoiding a threat to continued funding. An investigator who conducted such a trial and got the expected result would jeopardize getting further funding for MBSR trials.

Cage fights between MBSR and active control conditions.

 corey nelsonComparisons between MBSR an active control conditions are the real test of whether MBSR is effective and distinctively so. Such “cage fights” become particularly important when MBSR enthusiasts are not the referee. Investigator allegiance is an important determinant of outcome. Yet even when cage fights are refereed by investigators rooting for MBSR, the results can be disappointing.

In a recent blog post, I examined a trial of MBSR for smoking cessation that was published too late to be included in the comprehensive systematic review and meta-analysis.

 

The well-designed study

Vidrine JI, Spears CA, Heppner WL, Reitzel LR, Marcus MT, Cinciripini PM, Waters AJ, Li Y, Nguyen NT, Cao Y, Tindle HA. Efficacy of Mindfulness-Based Addiction Treatment (MBAT) for Smoking Cessation and Lapse Recovery: A Randomized Clinical Trial. Journal of Consulting and Clinical Psychology. 2016 May.

Compared mindfulness-based abstinence therapy (MBAT) to cognitive behavior therapy, which was closely matched for frequency and intensity of contact and credibility. The control/comparison group was four  5-10 minute individual counseling sessions. Although the comparison was lopsided in terms of frequency and intensity of meetings, there were no differences among the three groups. The authors did not emphasize that  a reason for the finding that all three groups received a nicotine patch with instructions.

Another new large study 

Daubenmier J, Moran PJ, Kristeller J, Acree M, Bacchetti P, Kemeny ME, Dallman M, Lustig RH, Grunfeld C, Nixon DF, Milush JM. Effects of a mindfulness‐based weight loss intervention in adults with obesity: A randomized clinical trial. Obesity. 2016 Apr 1;24(4):794-804.

Compared mindfulness training to a 5.5 month active control condition that was carefully matched.

To control of attention, social support, expectations of benefit, food provided during the mindful eating exercises, and home practice time in the mindfulness intervention, the control intervention included additional nutrition and physical activity information, strength training with exercise bands, discussion of societal issues concerning weight loss, snacks, and home activities. We controlled for a mindfulness approach to stress management by including progressive muscle relaxation and cognitive-behavioral training in the control group, although at a lower dose than in the mindfulness intervention.

There were no differences in weight loss between the two groups. Questions can be raised about how different the two treatments actually were, but part of the problem is that it is difficult to design such a treatment of comparable frequency and intensity of contact with credible content that does not overlap.

I would anticipate that comparisons between MBSR and appropriate active control conditions will be slow to accumulate. But the results at this point are not encouraging of the notion that MBSR is distinctively more effective than other active control conditions when delivered with the same frequency of contact, intensity, and positive expectations.

Mindfulness-based stress reduction versus cognitive behavior therapy for chronic back pain

The most interesting things to be learned from a recent clinical trial of mindfulness-based stress reduction to cognitive behavior therapy for chronic back pain are not what the authors intend.

Noticing that some key information is missing from the study illustrates why we don’t need more studies like it.

  • We need more studies of mindfulness-based therapies with meaningful comparison/control groups.
  • We need evidence that patients assigned to mindfulness-based treatments actually practice mindfulness in their everyday lives.
  • We need to demonstrate that any efficacy of mindfulness depends upon patients assigned to it actually showing up.
  • We need to be alert how boundaries of the concept of mindfulness-based therapies are expanding. Reviewers should be cautious in integrating results from different studies claiming to evaluate “mindfulness.” There is growing clinical heterogeneity – different interventions, sometimes with very different components–that should be distinguished.

mindfulnessdefn4For only the second time in its history, the flagship journal of the American Medical Association, JAMA has published a clinical trial of mindfulness. [Apparently the only other trial of mindfulness was one for PTSD among veterans, with only modest differences over a present-centered group therapy comparison/control group].

audiovideo promotion mindfulnessThe importance of this study was underscored by (1) an accompanying editorial commentary, (2) free access and continue education credit for reading it, and (3) three multimedia links –a JAMA Report on the study, and audio in video interviews with the author.

The article is

Cherkin DC, Sherman KJ, Balderson BH, Cook AJ, Anderson ML, Hawkes RJ, Hansen KE, Turner JA. Effect of Mindfulness-Based Stress Reduction vs Cognitive Behavioral Therapy or Usual Care on Back Pain and Functional Limitations in Adults With Chronic Low Back Pain: A Randomized Clinical Trial. JAMA. 2016 Mar 22;315(12):1240-9.

The trial registration is Comparison of CAM and Conventional Mind-Body Therapies for Chronic Back Pain.

The protocol is available here.

The editorial commentary by Madhav Goyal and Jennifer Haythornthwaite JA asked:

Is It Time to Make Mind-Body Approaches Available for Chronic Low Back Pain?

My recent discussions [1]  [2]  of articles in JAMA network journals that are accompanied by editorial commentaries have contemplated why particular studies were chosen for JAMA journals and the conflicts of interest that characterize editorial commentaries. This discussion will be somewhat different.

This commentary is definitely written by authors who have reasons to promote mindfulness. The commentary ends with a predictable non sequitur:

High-quality studies such as the clinical trial by Cherkin et al create a compelling argument for ensuring that an evidence-based health care system should provide access to affordable mind-body therapies.

Not exactly, if you stick to the evidence.

I will eventually comment on my usual questions of:

  • Why was this article published in a prestigious, generalist medical journal?
  • Why was it accompanied by an invited editorial commentary?
  • Why were the particular authors chosen for the commentary?

But the commentary isn’t that bad. It makes some reasonable points that might be overlooked. I will mainly focus on the article itself.

Abstract

Importance. Mindfulness-based stress reduction (MBSR) has not been rigorously evaluated for young and middle-aged adults with chronic low back pain.

Objective. To evaluate the effectiveness for chronic low back pain of MBSR vs cognitive behavioral therapy (CBT) or usual care.

Design, Setting, and Participants.  Randomized, interviewer-blind, clinical trial in an integrated health care system in Washington State of 342 adults aged 20 to 70 years with chronic low back pain enrolled between September 2012 and April 2014 and randomly assigned to receive MBSR (n = 116), CBT (n = 113), or usual care (n = 113).

Interventions. CBT (training to change pain-related thoughts and behaviors) and MBSR (training in mindfulness meditation and yoga) were delivered in 8 weekly 2-hour groups. Usual care included whatever care participants received.

Main Outcomes and Measures. Coprimary outcomes were the percentages of participants with clinically meaningful (≥30%) improvement from baseline in functional limitations (modified Roland Disability Questionnaire [RDQ]; range, 0-23) and in self-reported back pain bothersomeness (scale, 0-10) at 26 weeks. Outcomes were also assessed at 4, 8, and 52 weeks.

Results. There were 342 randomized participants, the mean (SD) [range] age was 49.3 (12.3) [20-70] years, 224 (65.7%) were women, mean duration of back pain was 7.3 years (range, 3 months-50 years), 123 (53.7%) attended 6 or more of the 8 sessions, 294 (86.0%) completed the study at 26 weeks, and 290 (84.8%) completed the study at 52 weeks. In intent-to-treat analyses at 26 weeks, the percentage of participants with clinically meaningful improvement on the RDQ was higher for those who received MBSR (60.5%) and CBT (57.7%) than for usual care (44.1%) (overall P = .04; relative risk [RR] for MBSR vs usual care, 1.37 [95% CI, 1.06-1.77]; RR for MBSR vs CBT, 0.95 [95% CI, 0.77-1.18]; and RR for CBT vs usual care, 1.31 [95% CI, 1.01-1.69]). The percentage of participants with clinically meaningful improvement in pain bothersomeness at 26 weeks was 43.6% in the MBSR group and 44.9% in the CBT group, vs 26.6% in the usual care group (overall P = .01; RR for MBSR vs usual care, 1.64 [95% CI, 1.15-2.34]; RR for MBSR vs CBT, 1.03 [95% CI, 0.78-1.36]; and RR for CBT vs usual care, 1.69 [95% CI, 1.18-2.41]). Findings for MBSR persisted with little change at 52 weeks for both primary outcomes.

Conclusions and Relevance Among adults with chronic low back pain, treatment with MBSR or CBT, compared with usual care, resulted in greater improvement in back pain and functional limitations at 26 weeks, with no significant differences in outcomes between MBSR and CBT. These findings suggest that MBSR may be an effective treatment option for patients with chronic low back pain.

Among the interesting things to note in the abstract is that there were only modest (p <.04) differences between either MBSR or CBT and usual care, which was described as “whatever participants received.” The MBSR was augmented by yoga. We cannot distinguish the effects of mindfulness from this added component.

back-pain-in-seniors-helped-with-mindfulness-300x200Unfortunately, if you do a search for “usual care” or “yoga” in the article itself or in the trial registration or protocol, you won’t learn about what the nature of the usual care or yoga. You will learn, however, in the article that:

Thirty of the 103 (29%) participants attending at least 1 MBSR session reported an adverse event (mostly temporarily increased pain with yoga). Ten of the 100 (10%) participants who attended at least 1 CBT session reported an adverse event (mostly temporarily increased pain with progressive muscle relaxation). No serious adverse events were reported.

Secondary outcomes

Some outcomes that would be of interest to policy makers, clinicians, in patients are relegated to a secondary status: whether medication was used in the past week, whether back exercises were done for at least three days, and whether there was general exercise for more than three days.

There were no consistent effects of these interventions versus routine care for these variables.

Intensity of treatment

Unless a study is focusing simply on differences in intensity of treatment, comparisons of treatments should ensure that the conditions being compared are equivalent in the intensity and frequency of clinical contact. In this trial:

The interventions were comparable in format (group), duration (2 hours/week for 8 weeks, although the MBSR program also included an optional 6-hour retreat), frequency (weekly), and number of participants per group.

Only about a quarter of the patients assigned to MBSR attended the six hour retreat, compounding the problems of adherence (around half of patients assigned to either to MBSR or CBT attended at least six group sessions), which also suggests that the 20% of patients lost to follow-up may not be random. That poses issues for the fancy statistical techniques used to compensate for attrition, which assume the missing data are random.

But the bigger issue is that the interventions provide a lot more contact than is typically available in routine care for chronic pain. There are lots of opportunities for important differences between the interventions and control group in nonspecific factors, like supportive accountability.

More contact communicates the patients that they matter more. Getting more interaction with providers means patients have more of a sense that their adherence matters (i.e., they are accountable) to someone besides themselves for activities like daily back exercises. The more intensive treatment also influences self-reported subjective outcomes, even when effects are not shown for other important variables, like decreased use of medication.

Distinguishing MBSR from CBT

MSBR is described as

MBSR was modeled closely after the original MBSR program—adapted from the 2009 MBSR instructor’s manual by a senior MBSR instructor. The MBSR program does not focus specifically on a particular condition such as pain. All classes included didactic content and mindfulness practice (body scan, yoga, meditation [attention to thoughts, emotions, and sensations in the present moment without trying to change them, sitting meditation with awareness of breathing, and walking meditation]).

The original manual that is cited comes from the University of Massachusetts Medical School. If you go the website you can find Mindfulness-Based Stress Reduction (MBSR): Standards of Practice .

yoga posesThe standards describe the yoga component as

“Formal” Mindfulness Meditation Methods

Body Scan Meditation – a supine meditation

Gentle Hatha Yoga – practiced with mindful awareness of the body

Sitting Meditation – mindfulness of breath, body, feelings, thoughts, emotions, and choiceless awareness

Walking Meditation

My concern is that an RCT has been published in JAMA concludes that a combined mindfulness and yoga treatment “may be an effective treatment option for patients with chronic low back pain.” Past research by some of the authors this JAMA article suggests that yoga by itself provides only short-term benefits for patients with chronic pain. This particular study had worrisome adverse effect from the yoga component. Why add something unnecessary to treatments if they may have adverse effects?

Although the providers of MBSR are described as having training in MBSR, there is no mention of training specifically for yoga for patients with chronic back pain.

Practitioners of yoga who have intermittent chronic pain tell me that it has been very important for them to find yoga instructors who are competent to deal with pain. A single, ill-chosen exercise can inflict long-term damage on patient who already has chronic back pain.

CBT is described as

The CBT protocol included CBT techniques most commonly applied and studied for chronic low back pain. The intervention included (1) education about chronic pain, relationships between thoughts and emotional and physical reactions, sleep hygiene, relapse prevention, and maintenance of gains; and (2) instruction and practice in changing dysfunctional thoughts, setting and working toward behavioral goals, relaxation skills (abdominal breathing, progressive muscle relaxation, and guided imagery), activity pacing, and pain-coping strategies. Between-session activities included reading chapters of The Pain Survival Guide: How to Reclaim Your Life. Mindfulness, meditation, and yoga techniques were proscribed in CBT; methods to challenge dysfunctional thoughts were proscribed in MBSR.

Many stripped-down versions of CBT offered in primary care do not have all these components, leaving out the abdominal breathing, progressive muscle relaxation, and guided imagery. Many eclectic versions of mindfulness training incorporate progressive muscle relaxation.

Given the about 50% attendance to at least six sessions in the modest uptake of the mindfulness retreat, I’m not sure that that these two interventions often distinctly different experiences. It’s doubtful that questions of whether these two treatments are characterized by distinctly different mechanisms could be addressed in this trial.

Routine Care for Chronic Pain in the US

Routine care for chronic back pain differs widely in the United States. Episodes of care – a clustering of visits around a complaint – do not typically occur beyond a month or couple of visits.

Routine care can be no care at all after initial evaluation in which diagnosis of chronic back pain is recorded.

But routine care for chronic back pain that is guideline-congruent can ironically prove iatrogenic. It can involve overtreatment, unnecessary exposure to opioids and antidepressants without adequate evaluation or follow up, and unnecessary surgeries.

We are living in the aftermath of pain being identified as the Fifth Vital Sign. In some settings, every patient has to be assessed with a simple rating scale of pain, regardless of the reason for visit. Providers have to document that they asked about pain and what procedures or referrals they provided if the patient reported other than “no pain.” Providers are penalized for not recording interventions when there is any pain indicated. They may lose insurance reimbursement for the visit.

There is currently a campaign to overturn these ridiculous and harmful guidelines, which are not evidence-based. The effect of the guidelines having that prescribed opioid pain medications rivaled heroin in terms of its negative public health impact. There is also been an epidemic of unnecessary back surgery, sometimes with crippling adverse effects.

But the guidelines have also induced despair and an unwillingness to address a condition that often must be endured with minimal intervention, rather than burdening clinicians and patients with the unrealistic expectation that it will be cured or eliminated. Clinicians are not good at dealing with conditions for which they do not have solutions.

I suspect that many of the patients in this study who remained assigned to routine care were getting minimal or no care. They were being provided little or no monitoring or reassessment of pain medications; little encouragement to engage in back exercises with regularity needed for them to be effective; and little support in the face of success and failures of getting on with their life in the face of chronic back pain.

Once again, we have an expensive study of mindfulness that does not address the question of whether any apparent effectiveness is simply due to increased intensity and frequency contact with the medical system and support.

We don’t know if the intervention is simply correcting the inadequacies or lack of routine care.

We cannot determine whether a better use of funds would be to improve the overall quality of routine care for chronic pain, including for the bulk of patients who have no interest in devoting the necessary time in the daily lives to practicing mindfulness.

The editorial commentary

The intended answer to the question posed by the title is obviously yes: Is It Time to Make Mind-Body Approaches Available for Chronic Low Back Pain?

The assessment provided by the commentary is:

A compelling argument for ensuring that an evidence-based health care system should provide access to affordable mind-body therapies.

Like the authors of the trial itself, the commentators are trying to get reimbursement for treatment that is provided through a designated mind-body center. Whether or not mind-body centers improve patient outcomes, they are useful for the intensive competitive marketing of medical centers.

NCCIHLike the authors, the commentators are not only competing for funds from the National Center for Complementary and Integrative Health [NCCIH}, formerly known as The National Center for Complementary and Alternative Medicine [NCCAM], they hoping to get more funds to this National Institute of Health.

The authors of the trial are connected. They have previously co-authored a study of acupuncture for chronic back pain with NCCAM program officers who are listed in the article as influencing and revising interpretations of the data. We have ample evidence acupuncture is not a science based medicine intervention chronic back pain. Any apparent effects are nonspecific. An illusion of effectiveness is likely to emerge in a comparison with routine care that lacks these nonspecific effects. I can’t believe the authors don’t know that.

So we’ve come in another route, but we’ve arrived at the same old story.

  • Authors with connections get their articles into prestigious, generalist medical journals.
  • Even though the evidence does not report the strong claims that are made, they are amplified with goodies like the article been freely available, having free continuing education, and other promotions like audio and video links.
  • Authors of the invited commentaries are written by persons with similar connections and similar vested interest.

I don’t think this article should have made it into JAMA. I don’t think it deserved an editorial commentary. If one were nonetheless provided, it should interpret for a general medical audience issues of the inadequacies of routine care, and inadequacy of routine care as a comparison group, and the practical issues of allocating scarce resources. An accompanying editorial should be reserved for articles more special than this one, and should offer a more detached, objective assessment of the strengths and weaknesses of a study and their implications.

MBSR spans New Age religious and science, as well as, evidence-based versus alternative, non-evidence-based treatments. The new agey aspect is emphasized in the titling of the trial registration including a designation as “CAM [complementary and alternative medicine] and Conventional Mind-Body Therapies.”

We must be alert to MBSR being hyped, promoted beyond what is justified by available evidence, – and now – it leading the charge of non-evidence-based treatments into reimbursement and competition for scarce resources in an already overexpensive and malfunctioning health system.

Study: Switching from antidepressants to mindfulness meditation increases relapse

  • A well-designed recent study found that patients with depression in remission who switch from maintenance antidepressants to mindfulness meditation without continuing medication had an increase in relapses.
  • The study is better designed and more transparently reported than a recent British study, but will get none of the British study’s attention.
  • The well-orchestrated promotion of mindfulness raises issues about the lack of checks and balances between investigators’ vested interest, supposedly independent evaluation, and the making of policy.

The study

Huijbers MJ, Spinhoven P, Spijker J, Ruhé HG, van Schaik DJ, van Oppen P, Nolen WA, Ormel J, Kuyken W, van der Wilt GJ, Blom MB. Discontinuation of antidepressant medication after mindfulness-based cognitive therapy for recurrent depression: randomised controlled non-inferiority trial. The British Journal of Psychiatry. 2016 Feb 18:bjp-p.

The study is currently behind a pay wall and does not appear to have a press release. These two factors will not contribute to it getting the attention it deserves.

But the protocol for the study is available here.

Huijbers MJ, Spijker J, Donders AR, van Schaik DJ, van Oppen P, Ruhé HG, Blom MB, Nolen WA, Ormel J, van der Wilt GJ, Kuyken W. Preventing relapse in recurrent depression using mindfulness-based cognitive therapy, antidepressant medication or the combination: trial design and protocol of the MOMENT study. BMC Psychiatry. 2012 Aug 27;12(1):1.

And the trial registration is here

Mindfulness Based Cognitive Therapy and Antidepressant Medication in Recurrent Depression. ClinicalTrials.gov: NCT00928980

The abstract

Background

Mindfulness-based cognitive therapy (MBCT) and maintenance antidepressant medication (mADM) both reduce the risk of relapse in recurrent depression, but their combination has not been studied.

Aims

To investigate whether MBCT with discontinuation of mADM is non-inferior to MBCT+mADM.

Method

A multicentre randomised controlled non-inferiority trial (ClinicalTrials.gov: NCT00928980). Adults with recurrent depression in remission, using mADM for 6 months or longer (n = 249), were randomly allocated to either discontinue (n = 128) or continue (n = 121) mADM after MBCT. The primary outcome was depressive relapse/recurrence within 15 months. A confidence interval approach with a margin of 25% was used to test non-inferiority. Key secondary outcomes were time to relapse/recurrence and depression severity.

Results

The difference in relapse/recurrence rates exceeded the non-inferiority margin and time to relapse/recurrence was significantly shorter after discontinuation of mADM. There were only minor differences in depression severity.

Conclusions

Our findings suggest an increased risk of relapse/recurrence in patients withdrawing from mADM after MBCT.

Translation?

Meditating_Dog clay___4e7ba9ad6f13e

A comment by Deborah Apthorp suggested that the original title Switching from antidepressants to mindfulness meditation increases relapse was incorrect. Checking it I realized that the abstract provides the article was Confusing, but the study did indded show that mindfulness alone led to more relapses and continued medication plus mindfulness.

Here is what is said in the actual introduction to the article:

The main aim of this multicentre, noninferiority effectiveness trial was to examine whether patients who receive MBCT for recurrent depression in remission could safely withdraw from mADM, i.e. without increased relapse/recurrence risk, compared with the combination of these interventions. Patients were randomly allocated to MBCT followed by discontinuation of mADM or MBCT+mADM. The study had a follow-up of 15 months. Our primary hypothesis was that discontinuing mADM after MBCT would be non-inferior, i.e. would not lead to an unacceptably higher risk of relapse/ recurrence, compared with the combination of MBCT+mADM.

Here is what is said in the discussion:

The findings of this effectiveness study reflect an increased risk of relapse/recurrence for patients withdrawing from mADM after having participated in MBCT for recurrent depression.

So, to be clear, the sequence was that patients were randomized either to MBCT without antidepressant or to MBCT with continuing antidepressants. Patients were then followed up for 15 months. Patients who received MBCT without the antidepressants have significantly more relapses/recurrences In the follow-up period than those who received MBCT with antidepressants.

The study addresses the question about whether patients with remitted depression on maintenance antidepressants who were randomized to receive mindfulness-based cognitive therapy (MBCT) have poorer outcomes than those randomized to remaining on their antidepressants.

The study found that poorer outcomes – more relapses – were experienced by patients switching to MBCT verses those remaining on antidepressants plus MBCT.

Strengths of the study

The patients were carefully assessed with validated semi structured interviews to verify they had recurrent past depression, were in current remission, and were taking their antidepressants. Assessment has an advantage over past studies that depended on less reliable primary-care physicians’ records to ascertain eligibility. There’s ample evidence that primary-care physicians often do not make systematic assessments deciding whether or not to preparation on antidepressants.

The control group. The comparison/control group continued on antidepressants after they were assessed by a psychiatrist who made specific recommendations.

 Power analysis. Calculation of sample size for this study was based on a noninferiority design. That meant that the investigators wanted to establish that within particular limit (25%), whether switching to MBCT produce poor outcomes.

A conventional clinical trial is designed to see if the the null hypothesis can rejected of no differences between intervention and control group. As an noninferiority trial, this study tested the null hypothesis that the intervention, shifting patients to MBCT would not result in an unacceptable rise, set at 25% more relapses and recurrences. Noninferiority trials are explained here.

Change in plans for the study

The protocol for the study originally proposed a more complex design. Patients would be randomized to one of three conditions: (1) continuing antidepressants alone; (2) continuing antidepressants, but with MBCT; or (3) MBCT alone. The problem the investigators encountered was that many patients had a strong preference and did not want to be randomized. So, they conducted two separate randomized trials.

This change in plans was appropriately noted in a modification in the trial registration.

The companion study examined whether adding MBCT to maintenance antidepressants reduce relapses. The study was published first:

Huijbers MJ, Spinhoven P, Spijker J, Ruhé HG, van Schaik DJ, van Oppen P, Nolen WA, Ormel J, Kuyken W, van der Wilt GJ, Blom MB. Adding mindfulness-based cognitive therapy to maintenance antidepressant medication for prevention of relapse/recurrence in major depressive disorder: Randomised controlled trial. Journal of Affective Disorders. 2015 Nov 15;187:54-61.

A copy can be obtained from this depository.

It was a smaller study – 35 patients randomized to MBCT alone and 33 patients randomized to a combination of MBCT and continued antidepressants. There were no differences in relapse/recurrence in 15 months.

An important limitation on generalizability

 The patients were recruited from university-based mental health settings. The minority of patients who move from treatment of depression in primary care to a specially mental health settings proportionately include more with moderate to severe depression and with a more defined history of past depression. In contrast, the patients being treated for depression in primary care include more who were mild to moderate and whose current depression and past history have not been systematically assessed. There is evidence that primary-care physicians do not make diagnoses of depression based on a structured assessment. Many patients deemed depressed and in need of treatment will have milder depression and only meet the vaguer, less validated diagnosis of Depression Not Otherwise Specified.

Declaration of interest

The authors indicated no conflicts of interest to declare for either study.

Added February 29: This may be a true statement for the core Dutch researchers who led in conducted the study. However, it is certainly not true for the British collaborator who may have served as a consultant and got authorship as result. He has extensive conflicts of interest and gains a lot personally and professionally from promotion of mindfulness in the UK. Read on.

The previous British study in The Lancet

Kuyken W, Hayes R, Barrett B, Byng R, Dalgleish T, Kessler D, Lewis G, Watkins E, Brejcha C, Cardy J, Causley A. Effectiveness and cost-effectiveness of mindfulness-based cognitive therapy compared with maintenance antidepressant treatment in the prevention of depressive relapse or recurrence (PREVENT): a randomised controlled trial. The Lancet. 2015 Jul 10;386(9988):63-73.

I provided my extended critique of this study in a previous blog post:

Is mindfulness-based therapy ready for rollout to prevent relapse and recurrence in depression?

The study protocol claimed it was designed as a superiority trial, but the authors did not provide the added sample size needed to demonstrate superiority. And they spun null findings, starting in their abstract:

However, when considered in the context of the totality of randomised controlled data, we found evidence from this trial to support MBCT-TS as an alternative to maintenance antidepressants for prevention of depressive relapse or recurrence at similar costs.

What is wrong here? They are discussing null findings as if they had conducted a noninferiority trial with sufficient power to show that differences of a particular size could be ruled out. Lots of psychotherapy trials are underpowered, but should not be used to declare treatments can be substituted for each other.

Contrasting features of the previous study versus the present one

Spinning of null findings. According to the trial registration, the previous study was designed to show that MBCT was superior to maintenance antidepressant treatment and preventing relapse and recurrence. A superiority trial tests the hypothesis that an intervention is better than a control group by a pre-set margin. For a very cool slideshow comparing superiority to noninferiority trials, see here .

Rather than demonstrating that MBCT was superior to routine care with maintenance antidepressant treatment, The Lancet study failed to find significant differences between the two conditions. In an amazing feat of spin, the authors took to publicizing this has a success that MBCT was equivalent to maintenance antidepressants. Equivalence is a stricter criterion that requires more than null findings – that any differences be within pre-set (registered) margins. Many null findings represent low power to find significant differences, not equivalence.

Patient selection. Patients were recruited from primary care on the basis of records indicating they had been prescribed antidepressants two years ago. There was no ascertainment of whether the patients were currently adhering to the antidepressants or whether they were getting effective monitoring with feedback.

Poorly matched, nonequivalent comparison/control group. The guidelines that patients with recurrent depression should remain on antidepressants for two years when developed based on studies in tertiary care. It’s likely that many of these patients were never systematically assessed for the appropriateness of treatment with antidepressants, follow-up was spotty, and many patients were not even continuing to take their antidepressants with any regularit

So, MBCT was being compared to an ill-defined, unknown condition in which some proportion of patients do not need to be taken antidepressants and were not taking them. This routine care also lack the intensity, positive expectations, attention and support of the MBCT condition. If an advantage for MBCT had been found – and it was not – it might only a matter that there was nothing specific about MBCT, but only the benefits of providing nonspecific conditions that were lacking in routine care.

The unknowns. There was no assessment of whether the patients actually practiced MBCT, and so there was further doubt that anything specific to MBCT was relevant. But then again, in the absence of any differences between groups, we may not have anything to explain.

  • Given we don’t know what proportion of patients were taking an adequate maintenance doses of antidepressants, we don’t know whether anything further treatment was needed for them – Or for what proportion.
  • We don’t know whether it would have been more cost-effective simply to have a depression care manager  recontact patients recontact patients, and determine whether they were still taking their antidepressants and whether they were interested in a supervised tapering.
  • We’re not even given the answer of the extent to which primary care patients provided with an MBCT actually practiced.

A well orchestrated publicity campaign to misrepresent the findings. Rather than offering an independent critical evaluation of The Lancet study, press coverage offered the investigators’ preferred spin. As I noted in a previous blog

The headline of a Guardian column  written by one of the Lancet article’s first author’s colleagues at Oxford misleadingly proclaimed that the study showed

freeman promoAnd that misrepresentation was echoed in the Mental Health Foundation call for mindfulness to be offered through the UK National Health Service –

 

calls for NHS mindfulness

The Mental Health Foundation is offering a 10-session online course  for £60 and is undoubtedly prepared for an expanded market

Declaration of interests

WK [the first author] and AE are co-directors of the Mindfulness Network Community Interest Company and teach nationally and internationally on MBCT. The other authors declare no competing interests.

Like most declarations of conflicts of interest, this one alerts us to something we might be concerned about but does not adequately inform us.

We are not told, for instance, something the authors were likely to know: Soon after all the hoopla about the study, The Oxford Mindfulness Centre, which is directed by the first author, but not mentioned in the declaration of interest publicize a massive effort by the Wellcome Trust to roll out its massive Mindfulness in the Schools project that provides mindfulness training to children, teachers, and parents.

A recent headline in the Times: US & America says it all.

times americakey to big bucks 

 

 

A Confirmation bias in subsequent citing

It is generally understood that much of what we read in the scientific literature is false or exaggerated due to various Questionable Research Practices (QRP) leading to confirmation bias in what is reported in the literature. But there is another kind of confirmation bias associated with the creation of false authority through citation distortion. It’s well-documented that proponents of a particular view selectively cite papers in terms of whether the conclusions support of their position. Not only are positive findings claimed original reports exaggerated as they progress through citations, negative findings receie less attention or are simply lost.

Huijbers et al.transparently reported that switching to MBCT leads to more relapses in patients who have recovered from depression. I confidently predict that these findings will be cited less often than the poorer quality The Lancet study, which was spun to create the appearance that it showed MBCT had equivalent  outcomes to remaining on antidepressants. I also predict that the Huijbers et al MBCT study will often be misrepresented when it is cited.

Added February 29: For whatever reason, perhaps because he served as a consultant, the author of The Lancet study is also an author on this paper, which describes a study conducted entirely in the Netherlands. Note however, when it comes to the British The Lancet study,  this article cites it has replicating past work when it was a null trial. This is an example of creating a false authority by distorted citation in action. I can’t judge whether the Dutch authors simply accepted the the conclusions offered in the abstract and press coverage of The Lancet study, or whether The Lancet author influenced their interpretation of it.

I would be very curious and his outpouring of subsequent papers on MBCT, whether The author of  The Lancet paper cites this paper and whether he cites it accurately. Skeptics, join me in watching.

What do I think is going on it in the study?

I think it is apparent that the authors have selected a group of patients who have remitted from their depression, but who are at risk for relapse and recurrence if they go without treatment. With such chronic, recurring depression, there is evidence that psychotherapy adds little to medication, particularly when patients are showing a clinical response to the antidepressants. However, psychotherapy benefits from antidepressants being added.

But a final point is important – MBCT was never designed as a primary cognitive behavioral therapy for depression. It was intended as a means of patients paying attention to themselves in terms of cues suggesting there are sliding back into depression and taking appropriate action. It’s unfortunate that been oversold as something more than this.

 

Is mindfulness-based therapy ready for rollout to prevent relapse and recurrence in depression?

Doubts that much of clinical or policy significance was learned from a recent study published in Lancet

Dog-MindfulnessPromoters of Acceptance and Commitment Therapy (ACT) notoriously established a record for academics endorsing a psychotherapy as better than alternatives, in the absence of evidence from adequately sized, high quality studies with suitable active control/comparison conditions. The credibility of designating a psychological interventions as “evidence-based” took a serious hit with the promotion of ACT, before its enthusiasts felt they attracted enough adherents to be able to abandon claims of “best” or “better than.”

But the tsunami of mindfulness promotion has surpassed anything ACT ever produced, and still with insufficient quality and quantity of evidence.

Could that be changing?

Some might think so with a recent randomized controlled trial reported in the Lancet of mindfulness-based cognitive therapy (MBCT) to reduce relapse and recurrence in depression. The headline of a Guardian column  by one of the Lancet article’s first author’s colleagues at Oxford misleadingly proclaimed that the study showed

freeman promoAnd that misrepresentation was echoed in the Mental Health Foundation call for mindfulness to be offered through the UK National Health Service –

calls for NHS mindfulnessThe Mental Health Foundation is offering a 10-session online course  for £60 and is undoubtedly prepared for an expanded market.

andrea-on-mindfulness
Patient testimonial accompanying Mental Health Foundation’s call for dissemination.

 

 

 

The Declaration of Conflict of Interest for the Lancet article mentions the first author and one other are “co-directors of the Mindfulness Network Community Interest Company and teach nationally and internationally on MBCT.” The first author notes the marketing potential of his study in comments to the media.

revising NICETo the authors’ credit, they modified the registration of their trial to reduce the likelihood of it being misinterpreted.

Reworded research question. To ensure that readers clearly understand that this trial is not a direct comparison between antidepressant medication (ADM) and Mindfulness-based cognitive therapy (MBCT), but ADM versus MBCT plus tapering support (MBCT-TS), the primary research question has been changed following the recommendation made by the Trial Steering Committee at their meeting on 24 June 2013. The revised primary research question now reads as follows: ‘Is MBCT with support to taper/discontinue antidepressant medication (MBCT-TS) superior to maintenance antidepressant medication (m-ADM) in preventing depression over 24 months?’ In addition, the acronym MBCT-TS will be used to emphasise this aspect of the intervention.

1792c904fbbe91e81ceefdd510d46304I would agree and amplify: This trial adds nothing to  the paucity of evidence from well-controlled trials that MBCT is a first-line treatment for patients experiencing a current episode of major depression. The few studies to date are small and of poor quality and are insufficient to recommend MBCT as a first line treatment of major depression.

I know, you would never guess that from promotions of MBCT for depression, especially not in the current blitz promotion in the UK.

The most salient question is whether MBCT can provide an effective means of preventing relapse in depressed patients who have already achieved remission and seek discontinuation.

Despite a chorus of claims in the social media to the contrary, the Lancet trial does not demonstrate that

  • Formal psychotherapy is needed to prevent relapse and recurrence among patients previously treated with antidepressants in primary care.
  • Any less benefit would have been achieved with a depression care manager who requires less formal training than a MBCT therapist.
  • Any less benefit would have been achieved with primary care physicians simply tapering antidepressant treatment that may not even have been appropriate in the first place.
  • The crucial benefit to patients being assigned to the MBCT condition was their acquisition of skills.
  • That practicing mindfulness is needed or even helpful in tapering from antidepressants.

We are all dodos and everyone gets a prize

dodosSomething also lost in the promotion of the trial is that it was originally designed to test the hypothesis that MBCT was better than maintenance antidepressant therapy in terms of relapse and recurrence of depression. That is stated in the registration of the trial, but not in the actual Lancet report of the trial outcome.

Across the primary and secondary outcome measures, the trial failed to demonstrate that MBCT was superior. Essentially the investigators had a null trial on their hands. But in a triumph of marketing over accurate reporting of a clinical trial, they shifted the question to whether MBCT is inferior to maintenance antidepressant therapy and declared the success demonstrating that it was not.

We saw a similar move in a MBCT trial  that I critiqued just recently. The authors here opted for the noninformative conclusion that MBCT was “not inferior” to an ill-defined routine primary care for a mixed sample of patients with depression and anxiety and adjustment disorders.

An important distinction is being lost here. Null findings in a clinical trial with a sample size set to answer the question whether one treatment is better than another is not the same as demonstrating that the two treatments are equivalent. The latter question requires a non-inferiority design with a much larger sample size in order to demonstrate that by some pre-specified criteria two treatments do not differ from each other in clinically significant terms.

Consider this analogy: we want to test whether yogurt is better than aspirin for a headache. So we do a power analysis tailored to the null hypothesis of no difference between yogurt and aspirin, conduct a trial, and find that yogurt and aspirin do not differ. But if we were actually interested in the question whether yogurt can be substituted for aspirin in treating headaches, we would have to estimate what size of a study would leave us comfortable with that conclusion the treating aspirin with yogurt versus aspirin makes no clinically significant difference. That would require a much larger sample size, typically several times the size of a clinical trial designed to test the efficacy of an intervention.

The often confusing differences between standard efficacy trials and noninferiority and superiority trials are nicely explained here.

Do primary care patients prescribed an antidepressant need to continue?

Patients taking antidepressants should not stop without consulting their physician and agreeing on a plan for discontinuation.

NICE Guidelines, like many international guidelines, recommend that patients with recurrent depression continue their medication for at least two years, out of concerned for a heightened risk of relapse and recurrence. But these recommendations are based on research in specialty mental health settings conducted with patients with an established diagnosis of depression. The generalization to primary care patients may not be appropriate best evidence.

Major depression is typically a recurrent, episodic condition with onset in the teens or early 20s. Many currently adult depressed patients beyond that age would be characterized as having a recurrent depression. In a study conducted at primary care practices associated with the University of Michigan, we found that most patients in waiting rooms identified as depressed on the basis of a two stage screening and formal diagnostic interview had recurrent depression, with the average patient having over six episodes before our point of contact.

However, depression in primary care may have less severe symptoms in a given episode and an overall less severe course then the patients who make it to specialty mental health care. And primary care physicians’ decisions about placing patients on antidepressants in primary care are typically not based upon a formal, semi structured interview in which there are symptom counts to ascertain whether patients have the necessary number of symptoms (5 for the Diagnostic and Statistical Manual-5) to meet diagnostic criteria.

My colleagues in Germany and I conducted another relevant study in which we randomized patients to either antidepressant, behavior therapy, or the patient preference of antidepressant versus behavior therapy. However, what was unusual was that we relied on primary care physician diagnosis, not our formal research criteria. We found that many patients enrolling in the trial would not meet criteria for major depression and, at least by DSM-IV-R criteria, would be given the highly ambiguous diagnosis of Depression, Not Otherwise Specified. The patients identified by the primary care physicians as requiring treatment for depression were quite different than those typically entering clinical trials evaluating treatment options. You can find out more about the trial here .

It is thus important to note that patients in the Lancet study were not originally prescribed antidepressants based on a formal, research diagnosis of major depression. Rather, the decisions of primary care physicians to prescribe the antidepressants, are not usually based on a systematic interview aimed at a formal diagnosis based on a minimal number of symptoms being present. This is a key issue.

The inclusion criteria for the Lancet study were that patients currently be in full or partial remission from a recent episode of depression and have had at least three episodes, counting the recent one. But their diagnosis at the time they were prescribed antidepressants was retrospectively reconstructed and may have biased by them having received antidepressants

Patients enrolled in the study were thus a highly select subsample of all patients receiving antidepressants in the UK primary care. A complex recruitment procedure involving not only review of GP records, but advertisement in the community means that we cannot tell what the overall proportion of patients receiving antidepressants and otherwise meeting criteria would have agreed to be in the study.

The study definitely does not provide a basis for revising guidelines for determining when and if primary care physicians should raise the issue of tapering antidepressant treatment. But that’s a vitally important clinical question.

skeptical-cat-is-fraught-with-skepticismQuestions not answered by the study:

  • We don’t know the appropriateness of the prescription of antidepressants to these patients in the first place.
  • We don’t know what review of the appropriateness of prescription of antidepressants had been conducted by the primary care physicians in agreeing that their patients participate in the study.
  • We don’t know the selectivity with which primary care physicians agreed for their patients to participate. To what extent are the patients to whom they recommended the trial representative of other patients in the maintenance phase of treatment?
  • We don’t know enough about how the primary care physicians treating the patients in the control groups reacted to the advice from the investigator group to continue medication. Importantly, how often were there meetings with these patients and did that change as a result of participation in this trial? Like every other trial of CBT in the UK that I have reviewed, this one suffers from an ill defined control group that was nonequivalent in terms of the contact time with professionals and support.
  • The question persists whether any benefits claimed for cognitive behavior therapy or MBCT from recent UK trials could have been achieved with nonspecific supportive interventions. In this particular Lancet study, we don’t know whether the same results could been achieved by simply tapering antidepressants assisted by a depression care manager less credentialed than what is required to provide MBCT.

The investigators provided a cost analysis. They concluded that there were no savings in health care costs of moving patients in full or partial remission off antidepressants to MBCT. But the cost analysis did not take into account the added patient time invested in practicing MBCT. Indeed, we don’t even know whether the patients assigned to MBCT actually practiced it with any diligence or will continue to do after treatment.

The authors promise a process analysis that will shed light on what element of MBCT contributed to the equivalency of outcomes with the maintenance of antidepressant medication.

But this process analysis will be severely limited by the inability to control for nonspecific factors such as contact time with the patient and support provided to the primary care physician and patient in tapering medication.

The authors seem intent on arguing that MBCT should be disseminated into the UK National Health Services. But a more sober assessment is that this trial only demonstrates that a highly select group of patients currently receiving antidepressants within the UK health system could be tapered without heightened risk of relapse and recurrence. There may be no necessity or benefit of providing MBCT per se during this process.

The study is not comparable to other noteworthy studies of MBCT to prevent remission, like Zindel Segal’s complex study . That study started with an acutely depressed patient population defined by careful criteria and treated patients with a well-defined algorithm for choosing and making changes in medications. Randomization to continued medication, MBCT, or pill placebo occurred on in the patients who remitted. It is unclear how much the clinical characteristics of the patients in the present Lancet study overlapped with those in Segal’s study.

What would be the consequences of disseminating and implementing MBCT into routine care based on current levels of evidence?

There are lots of unanswered questions concerning whether MBCT should be disseminated and widely implemented in routine care for depression.

One issue is where would the resources come from for this initiative? There already are long waiting list for cognitive behavior therapy, generally 18 weeks. Would disseminating MBCT draw therapists away from providing conventional cognitive behavior therapy? Therapists are often drawn to therapies based on their novelty and initial, unsubstantiated promises rather than strength of evidence. And the strength of evidence for MBCT is not such that we could recommend substituting it for CBT for treatment of acute, current major depression.

Another issue is whether most patients would be willing to commit not only the time for sessions of training and MBCT but to actually practicing it in their everyday life. Of course, again, we don’t even know from this trial whether actually practicing MBCT matters.

There hasn’t been a fair comparison of MBCT to equivalent time with a depression manager who would review patients currently receiving antidepressants and advise physicians has to whether and how to taper suitable candidates for discontinuation.

If I were distributing scarce resources to research to reduce unnecessary treatment with antidepressants, I would focus on a descriptive, observational study of the clinical status of patients currently receiving antidepressants, the amount of contact time their receiving with some primary health care professional, and the adequacy of their response in terms of symptom levels, but also adherence. Results could establish the usefulness of targeting long term use of antidepressants and the level of adherence of patients to taking the medication and to physicians monitoring their symptom levels and adherence. I bet there is a lot of poor quality maintenance care for depression in the community

When I was conducting NIMH-funded studies of depression in primary care, I never could get review committees interested in the issue of overtreatment and unnecessarily continued treatment. I recall one reviewer’s snotty comment that that these are not pressing public health issues.

That’s too bad, because I think they are key in considering how to distribute scarce resources to study and improve care for depression in the community. Existing evidence suggest a substantial cost of treatment of depression with antidepressants in general medical care is squandered on patients who do not meet guideline criteria for receiving antidepressants or who do not receive adequate monitoring.