Unintended consequences of universal mindfulness training for schoolchildren?

the mindful nationThis is the first installment of what will be a series of occasional posts about the UK Mindfulness All Party Parliamentary Group report,  Mindful Nation.

  • Mindful Nation is seriously deficient as a document supposedly arguing for policy based on evidence.
  • The professional and financial interests of lots of people involved in preparation of the document will benefit from implementation of its recommendations.
  • After an introduction, I focus on two studies singled in Mindful Nation out as offering support for the benefits of mindfulness training for school children.
  • Results of the group’s cherrypicked studies do not support implementation of mindfulness training in the schools, but inadvertently highlight some issues.
  • Investment in universal mindfulness training in the schools is unlikely to yield measurable, socially significant results, but will serve to divert resources from schoolchildren more urgently in need of effective intervention and support.
  • Mindfulness Nation is another example of  delivery of  low intensity  services to mostly low risk persons to the detriment of those in greatest and most urgent need.

The launch event for the Mindful Nation report billed it as the “World’s first official report” on mindfulness.

Mindful Nation is a report written by the UK Mindfulness All-Party Parliamentary Group.

The Mindfulness All-Party Parliamentary Group (MAPPG)  was set up to:

  • review the scientific evidence and current best practice in mindfulness training
  • develop policy recommendations for government, based on these findings
  • provide a forum for discussion in Parliament for the role of mindfulness and its implementation in public policy.

The Mindfulness All-Party Parliamentary Group describes itself as

Impressed by the levels of both popular and scientific interest, and launched an inquiry to consider the potential relevance of mindfulness to a range of urgent policy challenges facing government.

Don’t get confused by this being a government-commissioned report. The report stands in sharp contrast to one commissioned by the US government in terms of unbalanced constitution of the committee undertaking the review, and lack  of transparency in search for relevant literature,  and methodology for rating and interpreting of the quality of available evidence.

ahrq reportCompare the claims of Mindful Nation to a comprehensive systematic review and meta-analysis prepared for the US Agency for Healthcare Research and Quality (AHRQ) that reviewed 18,753 citations, and found only 47 trials (3%) that included an active control treatment. The vast majority of studies available for inclusion had only a wait list or no-treatment control group and so exaggerated any estimate of the efficacy of mindfulness.

Although the US report was available to those  preparing the UK Mindful Nation report, no mention is made of either the full contents of report or a resulting publication in a peer-reviewed journal. Instead, the UK Mindful Nation report emphasized narrative and otherwise unsystematic reviews, and meta-analyses not adequately controlling for bias.

When the abridged version of the AHRQ report was published in JAMA: Internal Medicine, an accompanying commentary raises issues even more applicable to the Mindful Nation report:

The modest benefit found in the study by Goyal et al begs the question of why, in the absence of strong scientifically vetted evidence, meditation in particular and complementary measures in general have become so popular, especially among the influential and well educated…What role is being played by commercial interests? Are they taking advantage of the public’s anxieties to promote use of complementary measures that lack a base of scientific evidence? Do we need to require scientific evidence of efficacy and safety for these measures?

The members of the UK Mindfulness All-Party Parliamentary Group were selected for their positive attitude towards mindfulness. The collection of witnesses they called to hearings were saturated with advocates of mindfulness and those having professional and financial interests in arriving at a positive view. There is no transparency in terms of how studies or testimonials were selected, but the bias is notable. Many of the scientific studies were methodologically poor, if there was any methodology at all. Many were strongly stated, but weakly substantiated opinion pieces. Authors often included those having  financial interests in obtaining positive results, but with no acknowledgment of conflict of interest. The glowing testimonials were accompanied by smiling photos and were unanimous in their praise of the transformative benefits of mindfulness.

As Mark B. Cope and David B. Allison concluded about obesity research, such a packing of the committee and a highly selective review of the literature leads to a ”distortion of information in the service of what might be perceived to be righteous ends.” [I thank Tim Caulfield for calling this quote to my attention].

Mindfulness in the schools

The recommendations of Mindfulness Nation are

  1. The Department for Education (DfE) should designate, as a first step, three teaching schools116 to pioneer mindfulness teaching,co-ordinate and develop innovation, test models of replicability and scalability and disseminate best practice.
  2. Given the DfE’s interest in character and resilience (as demonstrated through the Character Education Grant programme and its Character Awards), we propose a comparable Challenge Fund of £1 million a year to which schools can bid for the costs of training teachers in mindfulness.
  3. The DfE and the Department of Health (DOH) should recommend that each school identifies a lead in schools and in local services to co-ordinate responses to wellbeing and mental health issues for children and young people117. Any joint training for these professional leads should include a basic training in mindfulness interventions.
  4. The DfE should work with voluntary organisations and private providers to fund a freely accessible, online programme aimed at supporting young people and those who work with them in developing basic mindfulness skills118.
Payoff of Mindful Nation to Oxford Mindfulness Centre will be huge.
Payoff of Mindful Nation to Oxford Mindfulness Centre will be huge.

Leading up to these recommendations, the report outlined an “alarming crisis” in the mental health of children and adolescents and proposes:

Given the scale of this mental health crisis, there is real urgency to innovate new approaches where there is good preliminary evidence. Mindfulness fits this criterion and we believe there is enough evidence of its potential benefits to warrant a significant scaling-up of its availability in schools.

Think of all the financial and professional opportunities that proponents of mindfulness involved in preparation of this report have garnered for themselves.

Mindfulness to promote executive functioning in children and adolescents

For the remainder of the blog post, I will focus on the two studies cited in support of the following statement:

What is of particular interest is that those with the lowest levels of executive control73 and emotional stability74 are likely to benefit most from mindfulness training.

The terms “executive control” and “emotional stability” were clarified:

Many argue that the most important prerequisites for child development are executive control (the management of cognitive processes such as memory, problem solving, reasoning and planning) and emotion regulation (the ability to understand and manage the emotions, including and especially impulse control). These main contributors to self-regulation underpin emotional wellbeing, effective learning and academic attainment. They also predict income, health and criminality in adulthood69. American psychologist, Daniel Goleman, is a prominent exponent of the research70 showing that these capabilities are the biggest single determinant of life outcomes. They contribute to the ability to cope with stress, to concentrate, and to use metacognition (thinking about thinking: a crucial skill for learning). They also support the cognitive flexibility required for effective decision-making and creativity.

Actually, Daniel Goleman is the former editor of the pop magazine Psychology Today and an author of numerous pop books.

The first cited paper.

73 Flook L, Smalley SL, Kitil MJ, Galla BM, Kaiser-Greenland S, Locke J, et al. Effects of mindful  awareness practices on executive functions in elementary school children. Journal of Applied School Psychology. 2010;26(1):70-95.

Journal of Applied School Psychology is a Taylor-Francis journal, formerly known as Special Services in the Schools (1984 – 2002).  Its Journal Impact Factor is 1.30.

One of the authors of the article, Susan Kaiser-Greenland is a mindfulness entrepreneur as seen in her website describing her as an author, public speaker, and educator on the subject of sharing secular mindfulness and meditation with children and families. Her books are The Mindful Child: How to Help Your Kid Manage Stress and Become Happier, Kinder, and More Compassionate and Mindful Games: Sharing Mindfulness and Meditation with Children, Teens, and Families and the forthcoming The Mindful Games Deck: 50 Activities for Kids and Teens.

This article represents the main research available on Kaiser-Greenfield’s Inner Kids program and figures prominently in her promotion of her products.

The sample consisted of 64 children assigned to either mindful awareness practices (MAPs; n = 32) or a control group consisting of a silent reading period (n = 32).

The MAPs training used in the current study is a curriculum developed by one of the authors (SKG). The program is modeled after classical mindfulness training for adults and uses secular and age appropriate exercises and games to promote (a) awareness of self through sensory awareness (auditory, kinesthetic, tactile, gustatory, visual), attentional regulation, and awareness of thoughts and feelings; (b) awareness of others (e.g., awareness of one’s own body placement in relation to other people and awareness of other people’s thoughts and feelings); and (c) awareness of the environment (e.g., awareness of relationships and connections between people, places, and things).

A majority of exercises involve interactions among students and between students and the instructor.

Outcomes.

The primary EF outcomes were the Metacognition Index (MI), Behavioral Regulation Index (BRI), and Global Executive Composite (GEC) as reported by teachers and parents

Wikipedia presents the results of this study as:

The program was delivered for 30 minutes, twice per week, for 8 weeks. Teachers and parents completed questionnaires assessing children’s executive function immediately before and following the 8-week period. Multivariate analysis of covariance on teacher and parent reports of executive function (EF) indicated an interaction effect baseline EF score and group status on posttest EF. That is, children in the group that received mindful awareness training who were less well regulated showed greater improvement in EF compared with controls. Specifically, those children starting out with poor EF who went through the mindful awareness training showed gains in behavioral regulation, metacognition, and overall global executive control. These results indicate a stronger effect of mindful awareness training on children with executive function difficulties.

The finding that both teachers and parents reported changes suggests that improvements in children’s behavioral regulation generalized across settings. Future work is warranted using neurocognitive tasks of executive functions, behavioral observation, and multiple classroom samples to replicate and extend these preliminary findings.”

What I discovered when I scrutinized the study.

 This study is unblinded, with students and their teachers and parents providing the subjective ratings of the students well aware of which group students are assigned. We are not given any correlations among or between their ratings and so we don’t know whether there is just a global subjective factor (easy or difficult child, well-behaved or not) operating for either teachers or parents, or both.

It is unclear for what features of the mindfulness training the comparison reading group offers control or equivalence. The two groups are  different in positive expectations and attention and support that are likely to be reflected the parent and teacher ratings. There’s a high likelihood of any differences in outcomes being nonspecific and not something active and distinct ingredient of mindfulness training. In any comparison with the students assigned to reading time, students assigned to mindfulness training have the benefit of any active ingredient it might have, as well as any nonspecific, placebo ingredients.

This is exceedingly weak design, but one that dominates evaluations of mindfulness.

With only 32 students per group, note too that this is a seriously underpowered study. It has less than a 50% probability of detecting a moderate sized effect if one is present. And because of the larger effect size needed to achieve statistical significance with such a small sample size, and statistically significant effects will be large, even if unlikely to replicate in a larger sample. That is the paradox of low sample size we need to understand in these situations.

Not surprisingly, there were no differences between the mindfulness and reading control groups on any outcomes variable, whether rated by parents or teachers. Nonetheless, the authors rescued their claims for an effective intervention with:

However, as shown by the significance of interaction terms, baseline levels of EF (GEC reported by teachers) moderated improvement in posttest EF for those children in the MAPs group compared to children in the control group. That is, on the teacher BRIEF, children with poorer initial EF (higher scores on BRIEF) who went through MAPs training showed improved EF subsequent to the training (indicated by lower GEC scores at posttest) compared to controls.

Similar claims were made about parent ratings. But let’s look at figure 3 depicting post-test scores. These are from the teachers, but results for the parent ratings are essentially the same.

teacher BRIEF quartiles

Note the odd scaling of the X axis. The data are divided into four quartiles and then the middle half is collapsed so that there are three data points. I’m curious about what is being hidden. Even with the sleight-of-hand, it appears that scores for the intervention and control groups are identical except for the top quartile. It appears that just a couple of students in the control group are accounting for any appearance of a difference. But keep in mind that the upper quartile is only a matter of eight students in each group.

This scatter plot is further revealing:

teacher BRIEF

It appears that the differences that are limited to the upper quartile are due to a couple of outlier control students. Without them, even the post-hoc differences that were found in the upper quartile between intervention control groups would likely disappear.

Basically what we are seeing is that most students do not show any benefit whatsoever from mindfulness training over being in a reading group. It’s not surprising that students who were not particularly elevated on the variables of interest do not register an effect. That’s a common ceiling effect in such universally delivered interventions in general population samples

Essentially, if we focus on the designated outcome variables, we are wasting the students’ time as well as that of the staff. Think of what could be done if the same resources could be applied in more effective ways. There are a couple of students in in this study were outliers with low executive function. We don’t know how else they otherwise differ.Neither in the study, nor in the validation of these measures is much attention given to their discriminant validity, i.e., what variables influence the ratings that shouldn’t. I suspect strongly that there are global, nonspecific aspects to both parent and teacher ratings such that they are influenced by the other aspects of these couple of students’ engagement with their classroom environment, and perhaps other environments.

I see little basis for the authors’ self-congratulatory conclusion:

The present findings suggest that mindfulness introduced in a general  education setting is particularly beneficial for children with EF difficulties.

And

Introduction of these types of awareness practices in elementary education may prove to be a viable and cost-effective way to improve EF processes in general, and perhaps specifically in children with EF difficulties, and thus enhance young children’s socio-emotional, cognitive, and academic development.

Maybe the authors stared with this conviction and it was unshaken by disappointing findings.

Or the statement made in Mindfulness Nation:

What is of particular interest is that those with the lowest levels of executive control73 and emotional stability74 are likely to benefit most from mindfulness training.

But we have another study that is cited for this statement.

74. Huppert FA, Johnson DM. A controlled trial of mindfulness training in schools: The importance of practice for an impact on wellbeing. The Journal of Positive Psychology. 2010; 5(4):264-274.

The first author, Felicia Huppert is a  Founder and Director – Well-being Institute and Emeritus Professor of Psychology at University of Cambridge, as well as a member of the academic staff of the Institute for Positive Psychology and Education of the Australian Catholic University.

This study involved 173 14- and 15- year old  boys from a private Catholic school.

The Journal of Positive Psychology is not known for its high methodological standards. A look at its editorial board suggests a high likelihood that manuscripts submitted will be reviewed by sympathetic reviewers publishing their own methodologically flawed studies, often with results in support of undeclared conflicts of interest.

The mindfulness training was based on the program developed by Kabat-Zinn and colleagues at the University of Massachusetts Medical School (Kabat-Zinn, 2003). It comprised four 40 minute classes, one per week, which presented the principles and practice of mindfulness meditation. The mindfulness classes covered the concepts of awareness and acceptance, and the mindfulness practices included bodily awareness of contact points, mindfulness of breathing and finding an anchor point, awareness of sounds, understanding the transient nature of thoughts, and walking meditation. The mindfulness practices were built up progressively, with a new element being introduced each week. In some classes, a video clip was shown to highlight the practical value of mindful awareness (e.g. “The Last Samurai”, “Losing It”). Students in the mindfulness condition were also provided with a specially designed CD, containing three 8-minute audio files of mindfulness exercises to be used outside the classroom. These audio files reflected the progressive aspects of training which the students were receiving in class. Students were encouraged to undertake daily practice by listening to the appropriate audio files. During the 4-week training period, students in the control classes attended their normal religious studies lessons.

A total of 155 participants had complete data at baseline and 134 at follow-up (78 in the mindfulness and 56 in the control condition). Any student who had missing data are at either time point was simply dropped from the analysis. The effects of this statistical decison are difficult to track in the paper. Regardless, there was a lack of any difference between intervention and control group and any of a host of outcome variables, with none designated as primary outcome.

Actual practicing of mindfulness by students was inconsistent.

One third of the group (33%) practised at least three times a week, 34.8% practised more than once but less than three times a week, and 32.7% practised once a week or less (of whom 7 respondents, 8.4%, reported no practice at all). Only two students reported practicing daily. The practice variable ranged from 0 to 28 (number of days of practice over four weeks). The practice variable was found to be highly skewed, with 79% of the sample obtaining a score of 14 or less (skewness = 0.68, standard error of skewness = 0.25).

The authors rescue their claim of a significant effect for the mindfulness intervention with highly complex multivariate analyses with multiple control variables in which outcomes within-group effects for students assigned to mindfulness  were related to the extent of students actually practicing mindfulness. Without controlling for the numerous (and post-hoc) multiple comparisons, results were still largely nonsignificant.

One simple conclusion that can be drawn is that despite a lot of encouragement, there was little actual practice of mindfulness by the relatively well-off students in a relatively highly resourced school setting. We could expect results to improve with wider dissemination to schools with less resources and less privileged students.

The authors conclude:

The main finding of this study was a significant improvement on measures of mindfulness and psychological well-being related to the degree of individual practice undertaken outside the classroom.

Recall that Mindful Nation cited the study in the following context:

What is of particular interest is that those with the lowest levels of executive control73 and emotional stability74 are likely to benefit most from mindfulness training.

These are two methodologically weak studies with largely null findings. They are hardly the basis for launching a national policy implementing universal mindfulness in the schools.

As noted in the US AHRQ report, despite a huge number of studies of mindfulness having been conducted, few involved a test with an adequate control group, and so there’s little evidence that mindfulness has any advantage over any active treatment. Neither of these studies disturbed that conclusion, although they are spun both in the original studies and in the Mindful Nation report to be positive. Both papers were published in journals where the reviewers were likely to be overly sympathetic and not at him tentative to serious methodological and statistical problems.

The committee writing Mindful Nation arrived at conclusions consistent with their prior enthusiasm for mindfulness and their vested interest in it. They sorted through evidence to find what supported their pre-existing assumptions.

Like UK resilience programs, the recommendations of Mindful Nation put considerable resources in the delivery of services to a large population and likely to have the threshold of need to register a socially in clinically significant effect. On a population level, results of the implementation are doomed to fall short of its claims. Those many fewer students in need more timely, intensive, and tailored services are left underserved. Their presence is ignored or, worse, invoked to justify the delivery of services to the larger group, with the needy students not benefiting.

In this blog post, I mainly focused on two methodologically poor studies. But for the selection of these particular studies, I depended on the search of the authors of Mindful Nation and the emphasis that were given to these two studies for some sweeping claims in the report. I will continue to be writing about the recommendations of Mindful Nation. I welcome reader feedback, particularly from readers whose enthusiasm for mindfulness is offended. But I urge them not simply to go to Google and cherry pick an isolated study and ask me to refute its claims.

Rather, we need to pay attention to the larger literature concerning mindfulness, its serious methodological problems, and the sociopolitical forces and vested interests that preserve a strong confirmation bias, both in the “scientific” literature and its echoing in documents like Mindful Nation.

Study: Switching from antidepressants to mindfulness meditation increases relapse

  • A well-designed recent study found that patients with depression in remission who switch from maintenance antidepressants to mindfulness meditation without continuing medication had an increase in relapses.
  • The study is better designed and more transparently reported than a recent British study, but will get none of the British study’s attention.
  • The well-orchestrated promotion of mindfulness raises issues about the lack of checks and balances between investigators’ vested interest, supposedly independent evaluation, and the making of policy.

The study

Huijbers MJ, Spinhoven P, Spijker J, Ruhé HG, van Schaik DJ, van Oppen P, Nolen WA, Ormel J, Kuyken W, van der Wilt GJ, Blom MB. Discontinuation of antidepressant medication after mindfulness-based cognitive therapy for recurrent depression: randomised controlled non-inferiority trial. The British Journal of Psychiatry. 2016 Feb 18:bjp-p.

The study is currently behind a pay wall and does not appear to have a press release. These two factors will not contribute to it getting the attention it deserves.

But the protocol for the study is available here.

Huijbers MJ, Spijker J, Donders AR, van Schaik DJ, van Oppen P, Ruhé HG, Blom MB, Nolen WA, Ormel J, van der Wilt GJ, Kuyken W. Preventing relapse in recurrent depression using mindfulness-based cognitive therapy, antidepressant medication or the combination: trial design and protocol of the MOMENT study. BMC Psychiatry. 2012 Aug 27;12(1):1.

And the trial registration is here

Mindfulness Based Cognitive Therapy and Antidepressant Medication in Recurrent Depression. ClinicalTrials.gov: NCT00928980

The abstract

Background

Mindfulness-based cognitive therapy (MBCT) and maintenance antidepressant medication (mADM) both reduce the risk of relapse in recurrent depression, but their combination has not been studied.

Aims

To investigate whether MBCT with discontinuation of mADM is non-inferior to MBCT+mADM.

Method

A multicentre randomised controlled non-inferiority trial (ClinicalTrials.gov: NCT00928980). Adults with recurrent depression in remission, using mADM for 6 months or longer (n = 249), were randomly allocated to either discontinue (n = 128) or continue (n = 121) mADM after MBCT. The primary outcome was depressive relapse/recurrence within 15 months. A confidence interval approach with a margin of 25% was used to test non-inferiority. Key secondary outcomes were time to relapse/recurrence and depression severity.

Results

The difference in relapse/recurrence rates exceeded the non-inferiority margin and time to relapse/recurrence was significantly shorter after discontinuation of mADM. There were only minor differences in depression severity.

Conclusions

Our findings suggest an increased risk of relapse/recurrence in patients withdrawing from mADM after MBCT.

Translation?

Meditating_Dog clay___4e7ba9ad6f13e

A comment by Deborah Apthorp suggested that the original title Switching from antidepressants to mindfulness meditation increases relapse was incorrect. Checking it I realized that the abstract provides the article was Confusing, but the study did indded show that mindfulness alone led to more relapses and continued medication plus mindfulness.

Here is what is said in the actual introduction to the article:

The main aim of this multicentre, noninferiority effectiveness trial was to examine whether patients who receive MBCT for recurrent depression in remission could safely withdraw from mADM, i.e. without increased relapse/recurrence risk, compared with the combination of these interventions. Patients were randomly allocated to MBCT followed by discontinuation of mADM or MBCT+mADM. The study had a follow-up of 15 months. Our primary hypothesis was that discontinuing mADM after MBCT would be non-inferior, i.e. would not lead to an unacceptably higher risk of relapse/ recurrence, compared with the combination of MBCT+mADM.

Here is what is said in the discussion:

The findings of this effectiveness study reflect an increased risk of relapse/recurrence for patients withdrawing from mADM after having participated in MBCT for recurrent depression.

So, to be clear, the sequence was that patients were randomized either to MBCT without antidepressant or to MBCT with continuing antidepressants. Patients were then followed up for 15 months. Patients who received MBCT without the antidepressants have significantly more relapses/recurrences In the follow-up period than those who received MBCT with antidepressants.

The study addresses the question about whether patients with remitted depression on maintenance antidepressants who were randomized to receive mindfulness-based cognitive therapy (MBCT) have poorer outcomes than those randomized to remaining on their antidepressants.

The study found that poorer outcomes – more relapses – were experienced by patients switching to MBCT verses those remaining on antidepressants plus MBCT.

Strengths of the study

The patients were carefully assessed with validated semi structured interviews to verify they had recurrent past depression, were in current remission, and were taking their antidepressants. Assessment has an advantage over past studies that depended on less reliable primary-care physicians’ records to ascertain eligibility. There’s ample evidence that primary-care physicians often do not make systematic assessments deciding whether or not to preparation on antidepressants.

The control group. The comparison/control group continued on antidepressants after they were assessed by a psychiatrist who made specific recommendations.

 Power analysis. Calculation of sample size for this study was based on a noninferiority design. That meant that the investigators wanted to establish that within particular limit (25%), whether switching to MBCT produce poor outcomes.

A conventional clinical trial is designed to see if the the null hypothesis can rejected of no differences between intervention and control group. As an noninferiority trial, this study tested the null hypothesis that the intervention, shifting patients to MBCT would not result in an unacceptable rise, set at 25% more relapses and recurrences. Noninferiority trials are explained here.

Change in plans for the study

The protocol for the study originally proposed a more complex design. Patients would be randomized to one of three conditions: (1) continuing antidepressants alone; (2) continuing antidepressants, but with MBCT; or (3) MBCT alone. The problem the investigators encountered was that many patients had a strong preference and did not want to be randomized. So, they conducted two separate randomized trials.

This change in plans was appropriately noted in a modification in the trial registration.

The companion study examined whether adding MBCT to maintenance antidepressants reduce relapses. The study was published first:

Huijbers MJ, Spinhoven P, Spijker J, Ruhé HG, van Schaik DJ, van Oppen P, Nolen WA, Ormel J, Kuyken W, van der Wilt GJ, Blom MB. Adding mindfulness-based cognitive therapy to maintenance antidepressant medication for prevention of relapse/recurrence in major depressive disorder: Randomised controlled trial. Journal of Affective Disorders. 2015 Nov 15;187:54-61.

A copy can be obtained from this depository.

It was a smaller study – 35 patients randomized to MBCT alone and 33 patients randomized to a combination of MBCT and continued antidepressants. There were no differences in relapse/recurrence in 15 months.

An important limitation on generalizability

 The patients were recruited from university-based mental health settings. The minority of patients who move from treatment of depression in primary care to a specially mental health settings proportionately include more with moderate to severe depression and with a more defined history of past depression. In contrast, the patients being treated for depression in primary care include more who were mild to moderate and whose current depression and past history have not been systematically assessed. There is evidence that primary-care physicians do not make diagnoses of depression based on a structured assessment. Many patients deemed depressed and in need of treatment will have milder depression and only meet the vaguer, less validated diagnosis of Depression Not Otherwise Specified.

Declaration of interest

The authors indicated no conflicts of interest to declare for either study.

Added February 29: This may be a true statement for the core Dutch researchers who led in conducted the study. However, it is certainly not true for the British collaborator who may have served as a consultant and got authorship as result. He has extensive conflicts of interest and gains a lot personally and professionally from promotion of mindfulness in the UK. Read on.

The previous British study in The Lancet

Kuyken W, Hayes R, Barrett B, Byng R, Dalgleish T, Kessler D, Lewis G, Watkins E, Brejcha C, Cardy J, Causley A. Effectiveness and cost-effectiveness of mindfulness-based cognitive therapy compared with maintenance antidepressant treatment in the prevention of depressive relapse or recurrence (PREVENT): a randomised controlled trial. The Lancet. 2015 Jul 10;386(9988):63-73.

I provided my extended critique of this study in a previous blog post:

Is mindfulness-based therapy ready for rollout to prevent relapse and recurrence in depression?

The study protocol claimed it was designed as a superiority trial, but the authors did not provide the added sample size needed to demonstrate superiority. And they spun null findings, starting in their abstract:

However, when considered in the context of the totality of randomised controlled data, we found evidence from this trial to support MBCT-TS as an alternative to maintenance antidepressants for prevention of depressive relapse or recurrence at similar costs.

What is wrong here? They are discussing null findings as if they had conducted a noninferiority trial with sufficient power to show that differences of a particular size could be ruled out. Lots of psychotherapy trials are underpowered, but should not be used to declare treatments can be substituted for each other.

Contrasting features of the previous study versus the present one

Spinning of null findings. According to the trial registration, the previous study was designed to show that MBCT was superior to maintenance antidepressant treatment and preventing relapse and recurrence. A superiority trial tests the hypothesis that an intervention is better than a control group by a pre-set margin. For a very cool slideshow comparing superiority to noninferiority trials, see here .

Rather than demonstrating that MBCT was superior to routine care with maintenance antidepressant treatment, The Lancet study failed to find significant differences between the two conditions. In an amazing feat of spin, the authors took to publicizing this has a success that MBCT was equivalent to maintenance antidepressants. Equivalence is a stricter criterion that requires more than null findings – that any differences be within pre-set (registered) margins. Many null findings represent low power to find significant differences, not equivalence.

Patient selection. Patients were recruited from primary care on the basis of records indicating they had been prescribed antidepressants two years ago. There was no ascertainment of whether the patients were currently adhering to the antidepressants or whether they were getting effective monitoring with feedback.

Poorly matched, nonequivalent comparison/control group. The guidelines that patients with recurrent depression should remain on antidepressants for two years when developed based on studies in tertiary care. It’s likely that many of these patients were never systematically assessed for the appropriateness of treatment with antidepressants, follow-up was spotty, and many patients were not even continuing to take their antidepressants with any regularit

So, MBCT was being compared to an ill-defined, unknown condition in which some proportion of patients do not need to be taken antidepressants and were not taking them. This routine care also lack the intensity, positive expectations, attention and support of the MBCT condition. If an advantage for MBCT had been found – and it was not – it might only a matter that there was nothing specific about MBCT, but only the benefits of providing nonspecific conditions that were lacking in routine care.

The unknowns. There was no assessment of whether the patients actually practiced MBCT, and so there was further doubt that anything specific to MBCT was relevant. But then again, in the absence of any differences between groups, we may not have anything to explain.

  • Given we don’t know what proportion of patients were taking an adequate maintenance doses of antidepressants, we don’t know whether anything further treatment was needed for them – Or for what proportion.
  • We don’t know whether it would have been more cost-effective simply to have a depression care manager  recontact patients recontact patients, and determine whether they were still taking their antidepressants and whether they were interested in a supervised tapering.
  • We’re not even given the answer of the extent to which primary care patients provided with an MBCT actually practiced.

A well orchestrated publicity campaign to misrepresent the findings. Rather than offering an independent critical evaluation of The Lancet study, press coverage offered the investigators’ preferred spin. As I noted in a previous blog

The headline of a Guardian column  written by one of the Lancet article’s first author’s colleagues at Oxford misleadingly proclaimed that the study showed

freeman promoAnd that misrepresentation was echoed in the Mental Health Foundation call for mindfulness to be offered through the UK National Health Service –

 

calls for NHS mindfulness

The Mental Health Foundation is offering a 10-session online course  for £60 and is undoubtedly prepared for an expanded market

Declaration of interests

WK [the first author] and AE are co-directors of the Mindfulness Network Community Interest Company and teach nationally and internationally on MBCT. The other authors declare no competing interests.

Like most declarations of conflicts of interest, this one alerts us to something we might be concerned about but does not adequately inform us.

We are not told, for instance, something the authors were likely to know: Soon after all the hoopla about the study, The Oxford Mindfulness Centre, which is directed by the first author, but not mentioned in the declaration of interest publicize a massive effort by the Wellcome Trust to roll out its massive Mindfulness in the Schools project that provides mindfulness training to children, teachers, and parents.

A recent headline in the Times: US & America says it all.

times americakey to big bucks 

 

 

A Confirmation bias in subsequent citing

It is generally understood that much of what we read in the scientific literature is false or exaggerated due to various Questionable Research Practices (QRP) leading to confirmation bias in what is reported in the literature. But there is another kind of confirmation bias associated with the creation of false authority through citation distortion. It’s well-documented that proponents of a particular view selectively cite papers in terms of whether the conclusions support of their position. Not only are positive findings claimed original reports exaggerated as they progress through citations, negative findings receie less attention or are simply lost.

Huijbers et al.transparently reported that switching to MBCT leads to more relapses in patients who have recovered from depression. I confidently predict that these findings will be cited less often than the poorer quality The Lancet study, which was spun to create the appearance that it showed MBCT had equivalent  outcomes to remaining on antidepressants. I also predict that the Huijbers et al MBCT study will often be misrepresented when it is cited.

Added February 29: For whatever reason, perhaps because he served as a consultant, the author of The Lancet study is also an author on this paper, which describes a study conducted entirely in the Netherlands. Note however, when it comes to the British The Lancet study,  this article cites it has replicating past work when it was a null trial. This is an example of creating a false authority by distorted citation in action. I can’t judge whether the Dutch authors simply accepted the the conclusions offered in the abstract and press coverage of The Lancet study, or whether The Lancet author influenced their interpretation of it.

I would be very curious and his outpouring of subsequent papers on MBCT, whether The author of  The Lancet paper cites this paper and whether he cites it accurately. Skeptics, join me in watching.

What do I think is going on it in the study?

I think it is apparent that the authors have selected a group of patients who have remitted from their depression, but who are at risk for relapse and recurrence if they go without treatment. With such chronic, recurring depression, there is evidence that psychotherapy adds little to medication, particularly when patients are showing a clinical response to the antidepressants. However, psychotherapy benefits from antidepressants being added.

But a final point is important – MBCT was never designed as a primary cognitive behavioral therapy for depression. It was intended as a means of patients paying attention to themselves in terms of cues suggesting there are sliding back into depression and taking appropriate action. It’s unfortunate that been oversold as something more than this.