Using F1000 “peer review” to promote politics over evidence about delivering psychosocial care to cancer patients

The F 1000 platform allowed authors and the reviewers whom they nominated to collaborate in crafting more of their special interest advocacy that they have widely disseminated elsewhere. Nothing original in this article and certainly not best evidence!

 

mind the brain logo

A newly posted article on the F1000 website raises questions about what the website claims is a “peer-reviewed” open research platform.

Infomercial? The F1000 platform allowed authors and the reviewers whom they nominated to collaborate in crafting more of their special interest advocacy that they have widely disseminated elsewhere. Nothing original in this article and certainly not best evidence!

I challenge the authors and the reviewers they picked to identify something said in the F1000 article that they have not said numerous times before either alone or in papers co-authored by some combination of authors and the reviewers they picked for this paper.

F1000 makes the attractive and misleading claim that versions of articles that are posted on its website reflect the response to reviewers.

Readers should be aware of uncritically accepting articles on the F 1000 website as having been peer-reviewed in any conventional sense of the term.

Will other special interests groups exploit this opportunity to brand their claims as “peer-reviewed” without the risk of having to tone down their claims in peer review? Is this already happening?

In the case of this article, reviewers were all chosen by the authors and have a history of co-authoring papers with the authors of the target paper in active advocacy of a shared political perspective, one that is contrary to available evidence.

Cynically, future authors might be motivated to divide their team, with some remaining authors and others dropping off to become nominated as reviewers. They could then suggest content that had already been agreed would be included, but was left off for the purposes being suggested in the review process

F1000

F1000Research bills itself as

An Open Research publishing platform for life scientists, offering immediate publication of articles and other research outputs without editorial bias. All articles benefit from transparent refereeing and the inclusion of all source data.

Material posted on this website is labeled as having received rapid peer-review:

Articles are published rapidly as soon as they are accepted, after passing an in-house quality check. Peer review by invited experts, suggested by the authors, takes place openly after publication.

My recent Google Scholar alert call attention to an article posted on F1000

Advancing psychosocial care in cancer patients [version 1; referees: 3 approved]

 Who were the reviewers?

open peer review of Advancing psychosocial care

Google the names of authors and reviewers. You will discover a pattern of co-authorship; leadership positions in international Psycho-Oncology society, a group promoting the mandating of specially mental health services for cancer patients, and lots of jointly and separately authored articles making a pitch for increased involvement of mental health professionals in routine cancer care. This article adds almost nothing to what is multiply available elsewhere in highly redundant publications

Given a choice of reviewers, these authors would be unlikely to nominate me. Nonetheless, here is my review of the article.

 As I might do in a review of a manuscript, I’m not providing citations for these comments, but support can readily be found by a search of blog posts at my website @CoyneoftheRealm.com and Google Scholar search of my publications. I welcome queries from anybody seeking documentation of these points below.

 Fighting Spirit

The notion that cancer patients having a fighting spirit improves survival is popular in the lay press and in promoting the power of the mind over cancer, but it has thoroughly been discredited.

Early on, the article identifies fighting spirit as an adaptive coping style. In actuality, fighting spirit was initially thought to predict mortality in a small methodologically flawed study. But that is no longer claimed.

Even one of the authors of the original study, Maggie Watson,  expressed relief when her own larger, better designed study failed to confirm the impression that a fighting spirit extended life after diagnosis  of cancer. Why? Dr. Watson was concerned that the concept was being abused in blaming cancer patients who were dying there was to their personal deficiency of not having enough fighting spirit.

Fighting spirit is rather useless as a measure of psychological adaptation. It confounds severity of cancer enrolled dysfunction with efforts to cope with cancer.

Distress as the sixth vital sign for cancer patients

distress thermometerBeware of a marketing slogan posing as an empirical statement. Its emptiness is similar to that of to “Pepsi is the one.” Can you imagine anyone conducting a serious study in which they conclude “Pepsi is not the one”?

Once again in this article, a vacuous marketing slogan is presented in impressive, but pseudo-medical terms. Distress cannot be a vital sign in the conventional sense. Thr  vital signs are objective measurements that do not depend on patient self-report: body temperature, pulse rate, and respiration rate (rate of breathing) (Blood pressure is not considered a vital sign, but is often measured along with the vital signs.).

Pain was declared a fifth vital sign, with physicians mandated  by guidelines to provide routine self-report screening of patients, regardless of their reasons for visit. Pain being the fifth vital sign seems to have been the inspiration for declaring distress as the sixth vital sign for cancer patients. However policy makers declaring pain  as the fifth vital sign did not result in improved patient levels of pain. Their subsequent making intervention mandatory for any reports of pain led to a rise in unnecessary back and knee surgery, with a substantial rise in associated morbidity and loss of function. The next shift to prescription of opioids that were claimed not to be addictive was the beginning of the current epidemic of addiction to prescription opioids. Making pain the fifth vital sign is killed a lot of patients and  turned others into addicts craving drugs on the street because they have lost their prescriptions for the opioids that addicted them.

pain as 5th vital signCDC launches

 Cancer as a mental health issue

There is a lack of evidence that cancer carries a risk of psychiatric disorder more than other chronic and catastrophic illnesses. However, the myth that there is something unique or unusual about cancer’s threat to mental health is commonly cited by mental health professional advocacy groups is commonly used to justify increased resources to them for specialized services.

The article provides an inflated estimate of psychiatric morbidity by counting adjustment disorders as psychiatric disorders. Essentially, a cancer patient who seeks mental health interventions for distress qualifies by virtue of help seeking being defined as impairment.

The conceptual and empirical muddle of “distress” in cancer patients

The article repeats the standard sloganeering definition of distress that the authors and reviewers have circulated elsewhere.

It has been very broadly defined as “a multifactorial, unpleasant, emotional experienceof a psychological (cognitive, behavioural, emotional), social and/or spiritual nature that may interfere with the ability to cope effectively with cancer, its physical symptoms and its treatment and that extends along a continuum, ranging from common normalfeelings of vulnerability, sadness and fears to problems that can become disabling, such as depression, anxiety, panic, social isolation and existential and spiritual crisis”5

[You might try googling this. I’m sure you’ll discover an amazing number of repetitions in similar articles advocating increasing psychosocial services for cancer patients organized around this broad definition.]

Distress is so broadly defined and all-encompassing, that there can be no meaningful independent validation of distress measures except for by other measures of distress, not conventional measures of adaptation or mental health. I have discussed that in a recent blog post.

If we restrict “distress” to the more conventional meaning of stress or negative affect, we find that any elevation in distress (usually 35% or so) associated with onset diagnosis of cancer tends to follow a natural trajectory of decline without formal intervention. Elevations in distress for most cancer patients, are resolved within 3 to 6 months without intervention. A residual 9 to 11% of cancer patients having elevated distress is likely attributed to pre-existing psychiatric disorder.

Routine screening for distress

The slogan “distress is the sixth vital sign” is used to justify mandatory routine screening of cancer patients for distress. In the United States, surgeons cannot close their electronic medical records for a patient and go on to the next patient without recording whether they had screened patients for distress, and if the patient reports distress, what intervention has been provided. Clinicians simply informally asking patients if they are distressed and responding to a “yes” by providing the patient with an antidepressant without further follow up allows surgeons to close the medical records.

As I have done so before, I challenge advocates of routine screening of cancer patients for distress to produce evidence that simply introducing routine screening without additional resources leads to better patient outcomes.

Routine screening for distress as uncovering unmet needs among cancer patients

 Studies in the Netherlands suggest that there is not a significant increase in need for services from mental health or allied health professionals associated with diagnosis of cancer. There is some disruption of such services that patients were receiving before diagnosis. It doesn’t take screening and discussion to suggest that patients that they at some point resume those services if they wish. There is also some increased need for physical therapy and nutritional counseling

If patients are simply asked a question whether they want a discussion of the services (in Dutch: Zou u met een deskundige willen praten over uw problemen?)  that are available, many patients will decline.

Much of demand for supportive services like counseling and support groups, especially among breast cancer patients is not from among the most distressed patients. One of the problems with clinical trials of psychosocial interventions is that most of the patients who seek enrollment are not distressed, and less they are prescreened. This poses dilemma: if you require elevated distress on a screening instrument, we end up rationing services and excluding many of the patients who would otherwise be receiving them.

I welcome clarification from F 1000 just what they offer over other preprint repositories. When one downloads a preprint from some other repositories, it clearly displays “not yet peer-reviewed.” F 1000 carries the advantage of the label of “peer-reviewed, but does not seem to be hard earned.

Notes

Slides are from two recent talks at Dutch International Congress on Insurance Medicine Thursday, November 9, 2017, Almere, Netherlands   :

Will primary care be automated screening and procedures or talking to patients and problem-solving? Invited presentation

and

Why you should not routinely screen your patients for depression and what you should do instead. Plenary Presentation

        

                                  

 

 

 

Creating illusions of wondrous effects of yoga and meditation on health: A skeptic exposes tricks

The tour of the sausage factory is starting, here’s your brochure telling you’ll see.

 

A recent review has received a lot of attention with it being used for claims that mind-body interventions have distinct molecular signatures that point to potentially dramatic health benefits for those who take up these practices.

What Is the Molecular Signature of Mind–Body Interventions? A Systematic Review of Gene Expression Changes Induced by Meditation and Related Practices.  Frontiers in Immunology. 2017;8.

Few who are tweeting about this review or its press coverage are likely to have read it or to understand it, if they read it. Most of the new agey coverage in social media does nothing more than echo or amplify the message of the review’s press release.  Lazy journalists and bloggers can simply pass on direct quotes from the lead author or even just the press release’s title, ‘Meditation and yoga can ‘reverse’ DNA reactions which cause stress, new study suggests’:

“These activities are leaving what we call a molecular signature in our cells, which reverses the effect that stress or anxiety would have on the body by changing how our genes are expressed.”

And

“Millions of people around the world already enjoy the health benefits of mind-body interventions like yoga or meditation, but what they perhaps don’t realise is that these benefits begin at a molecular level and can change the way our genetic code goes about its business.”

[The authors of this review actually identified some serious shortcomings to the studies they reviewed. I’ll be getting to some excellent points at the end of this post that run quite counter to the hype. But the lead author’s press release emphasized unwarranted positive conclusions about the health benefits of these practices. That is what is most popular in media coverage, especially from those who have stuff to sell.]

Interpretation of the press release and review authors’ claims requires going back to the original studies, which most enthusiasts are unlikely to do. If readers do go back, they will have trouble interpreting some of the deceptive claims that are made.

Yet, a lot is at stake. This review is being used to recommend mind-body interventions for people having or who are at risk of serious health problems. In particular, unfounded claims that yoga and mindfulness can increase the survival of cancer patients are sometimes hinted at, but occasionally made outright.

This blog post is written with the intent of protecting consumers from such false claims and providing tools so they can spot pseudoscience for themselves.

Discussion in the media of the review speaks broadly of alternative and complementary interventions. The coverage is aimed at inspiring  confidence in this broad range of treatments and to encourage people who are facing health crises investing time and money in outright quackery. Seemingly benign recommendations for yoga, tai chi, and mindfulness (after all, what’s the harm?) often become the entry point to more dubious and expensive treatments that substitute for established treatments.  Once they are drawn to centers for integrative health care for classes, cancer patients are likely to spend hundreds or even thousands on other products and services that are unlikely to benefit them. One study reported:

More than 72 oral or topical, nutritional, botanical, fungal and bacterial-based medicines were prescribed to the cohort during their first year of IO care…Costs ranged from $1594/year for early-stage breast cancer to $6200/year for stage 4 breast cancer patients. Of the total amount billed for IO care for 1 year for breast cancer patients, 21% was out-of-pocket.

Coming up, I will take a skeptical look at the six randomized trials that were highlighted by this review.  But in this post, I will provide you with some tools and insights so that you do not have to make such an effort in order to make an informed decision.

Like many of the other studies cited in the review, these randomized trials were quite small and underpowered. But I will focus on the six because they are as good as it gets. Randomized trials are considered a higher form of evidence than simple observational studies or case reports [It is too bad the authors of the review don’t even highlight what studies are randomized trials. They are lumped with others as “longitudinal studies.]

As a group, the six studies do not actually add any credibility to the claims that mind-body interventions – specifically yoga, tai chi, and mindfulness training or retreats improve health by altering DNA.  We can be no more confident with what the trials provide than we would be without them ever having been done.

I found the task of probing and interpreting the studies quite labor-intensive and ultimately unrewarding.

I had to get past poor reporting of what was actually done in the trials, to which patients, and with what results. My task often involved seeing through cover ups with authors exercising considerable flexibility in reporting what measures were they actually collected and what analyses were attempted, before arriving at the best possible tale of the wondrous effects of these interventions.

Interpreting clinical trials should not be so hard, because they should be honestly and transparently reported and have a registered protocol and stick to it. These reports of trials were sorely lacking, The full extent of the problems took some digging to uncover, but some things emerged before I got to the methods and results.

The introductions of these studies consistently exaggerated the strength of existing evidence for the effects of these interventions on health, even while somehow coming to the conclusion that this particular study was urgently needed and it might even be the “first ever”. The introductions to the six papers typically cross-referenced each other, without giving any indication of how poor quality the evidence was from the other papers. What a mutual admiration society these authors are.

One giveaway is how the introductions  referred to the biggest, most badass, comprehensive and well-done review, that of Goyal and colleagues.

That review clearly states that the evidence for the effects of mindfulness is poor quality because of the lack of comparisons with credible active treatments. The typical randomized trial of mindfulness involves a comparison with no-treatment, a waiting list, or patients remaining in routine care where the target problem is likely to be ignored.  If we depend on the bulk of the existing literature, we cannot rule out the likelihood that any apparent benefits of mindfulness are due to having more positive expectations, attention, and support over simply getting nothing.  Only a handful  of hundreds of trials of mindfulness include appropriate, active treatment comparison/control groups. The results of those studies are not encouraging.

One of the first things I do in probing the introduction of a study claiming health benefits for mindfulness is see how they deal with the Goyal et al review. Did the study cite it, and if so, how accurately? How did the authors deal with its message, which undermines claims of the uniqueness or specificity of any benefits to practicing mindfulness?

For yoga, we cannot yet rule out that it is better than regular exercising – in groups or alone – having relaxing routines. The literature concerning tai chi is even smaller and poorer quality, but there is the same need to show that practicing tai chi has any benefits over exercising in groups with comparable positive expectations and support.

Even more than mindfulness, yoga and tai chi attract a lot of pseudoscientific mumbo jumbo about integrating Eastern wisdom and Western science. We need to look past that and insist on evidence.

Like their introductions, the discussion sections of these articles are quite prone to exaggerating how strong and consistent the evidence is from existing studies. The discussion sections cherry pick positive findings in the existing literature, sometimes recklessly distorting them. The authors then discuss how their own positively spun findings fit with what is already known, while minimizing or outright neglecting discussion of any of their negative findings. I was not surprised to see one trial of mindfulness for cancer patients obtain no effects on depressive symptoms or perceived stress, but then go on to explain mindfulness might powerfully affect the expression of DNA.

If you want to dig into the details of these studies, the going can get rough and the yield for doing a lot of mental labor is low. For instance, these studies involved drawing blood and analyzing gene expression. Readers will inevitably encounter passages like:

In response to KKM treatment, 68 genes were found to be differentially expressed (19 up-regulated, 49 down-regulated) after adjusting for potentially confounded differences in sex, illness burden, and BMI. Up-regulated genes included immunoglobulin-related transcripts. Down-regulated transcripts included pro-inflammatory cytokines and activation-related immediate-early genes. Transcript origin analyses identified plasmacytoid dendritic cells and B lymphocytes as the primary cellular context of these transcriptional alterations (both p < .001). Promoter-based bioinformatic analysis implicated reduced NF-κB signaling and increased activity of IRF1 in structuring those effects (both p < .05).

Intimidated? Before you defer to the “experts” doing these studies, I will show you some things I noticed in the six studies and how you can debunk the relevance of these studies for promoting health and dealing with illness. Actually, I will show that even if these 6 studies got the results that the authors claimed- and they did not- at best, the effects would trivial and lost among the other things going on in patients’ lives.

Fortunately, there are lots of signs that you can dismiss such studies and go on to something more useful, if you know what to look for.

Some general rules:

  1. Don’t accept claims of efficacy/effectiveness based on underpowered randomized trials. Dismiss them. The rule of thumb is reliable to dismiss trials that have less than 35 patients in the smallest group. Over half the time, true moderate sized effects will be missed in such studies, even if they are actually there.

Due to publication bias, most of the positive effects that are published from such sized trials will be false positives and won’t hold up in well-designed, larger trials.

When significant positive effects from such trials are reported in published papers, they have to be large to have reached significance. If not outright false, these effect sizes won’t be matched in larger trials. So, significant, positive effect sizes from small trials are likely to be false positives and exaggerated and probably won’t replicate. For that reason, we can consider small studies to be pilot or feasibility studies, but not as providing estimates of how large an effect size we should expect from a larger study. Investigators do it all the time, but they should not: They do power calculations estimating how many patients they need for a larger trial from results of such small studies. No, no, no!

Having spent decades examining clinical trials, I am generally comfortable dismissing effect sizes that come from trials with less than 35 patients in the smaller group. I agree with a suggestion that if there are two larger trials are available in a given literature, go with those and ignore the smaller studies. If there are not at least two larger studies, keep the jury out on whether there is a significant effect.

Applying the Rule of 35, 5 of the 6 trials can be dismissed and the sixth is ambiguous because of loss of patients to follow up.  If promoters of mind-body interventions want to convince us that they have beneficial effects on physical health by conducting trials like these, they have to do better. None of the individual trials should increase our confidence in their claims. Collectively, the trials collapse in a mess without providing a single credible estimate of effect size. This attests to the poor quality of evidence and disrespect for methodology that characterizes this literature.

  1. Don’t be taken in by titles to peer-reviewed articles that are themselves an announcement that these interventions work. Titles may not be telling the truth.

What I found extraordinary is that five of the six randomized trials had a title that indicating a positive effect was found. I suspect that most people encountering the title will not actually go on to read the study. So, they will be left with the false impression that positive results were indeed obtained. It’s quite a clever trick to make the title of an article, by which most people will remember it, into a false advertisement for what was actually found.

For a start, we can simply remind ourselves that with these underpowered studies, investigators should not even be making claims about efficacy/effectiveness. So, one trick of the developing skeptic is to confirm that the claims being made in the title don’t fit with the size of the study. However, actually going to the results section one can find other evidence of discrepancies between what was found in what is being claimed.

I think it’s a general rule of thumb that we should be careful of titles for reports of randomized that declare results. Even when what is claimed in the title fits with the actual results, it often creates the illusion of a greater consistency with what already exists in the literature. Furthermore, even when future studies inevitably fail to replicate what is claimed in the title, the false claim lives on, because failing to replicate key findings is almost never a condition for retracting a paper.

  1. Check the institutional affiliations of the authors. These 6 trials serve as a depressing reminder that we can’t go on researchers’ institutional affiliation or having federal grants to reassure us of the validity of their claims. These authors are not from Quack-Quack University and they get funding for their research.

In all cases, the investigators had excellent university affiliations, mostly in California. Most studies were conducted with some form of funding, often federal grants.  A quick check of Google would reveal from at least one of the authors on a study, usually more, had federal funding.

  1. Check the conflicts of interest, but don’t expect the declarations to be informative. But be skeptical of what you find. It is also disappointing that a check of conflict of interest statements for these articles would be unlikely to arouse the suspicion that the results that were claimed might have been influenced by financial interests. One cannot readily see that the studies were generally done settings promoting alternative, unproven treatments that would benefit from the publicity generated from the studies. One cannot see that some of the authors have lucrative book contracts and speaking tours that require making claims for dramatic effects of mind-body treatments could not possibly be supported by: transparent reporting of the results of these studies. As we will see, one of the studies was actually conducted in collaboration with Deepak Chopra and with money from his institution. That would definitely raise flags in the skeptic community. But the dubious tie might be missed by patients in their families vulnerable to unwarranted claims and unrealistic expectations of what can be obtained outside of conventional medicine, like chemotherapy, surgery, and pharmaceuticals.

Based on what I found probing these six trials, I can suggest some further rules of thumb. (1) Don’t assume for articles about health effects of alternative treatments that all relevant conflicts of interest are disclosed. Check the setting in which the study was conducted and whether it was in an integrative [complementary and alternative, meaning mostly unproven.] care setting was used for recruiting or running the trial. Not only would this represent potential bias on the part of the authors, it would represent selection bias in recruitment of patients and their responsiveness to placebo effects consistent with the marketing themes of these settings.(2) Google authors and see if they have lucrative pop psychology book contracts, Ted talks, or speaking gigs at positive psychology or complementary and alternative medicine gatherings. None of these lucrative activities are typically expected to be disclosed as conflicts of interest, but all require making strong claims that are not supported by available data. Such rewards are perverse incentives for authors to distort and exaggerate positive findings and to suppress negative findings in peer-reviewed reports of clinical trials. (3) Check and see if known quacks have prepared recruitment videos for the study, informing patients what will be found (Serious, I was tipped off to look and I found that).

  1. Look for the usual suspects. A surprisingly small, tight, interconnected group is generating this research. You could look the authors up on Google or Google Scholar or  browse through my previous blog posts and see what I have said about them. As I will point out in my next blog, one got withering criticism for her claim that drinking carbonated sodas but not sweetened fruit drinks shortened your telomeres so that drinking soda was worse than smoking. My colleagues and I re-analyzed the data of another of the authors. We found contrary to what he claimed, that pursuing meaning, rather than pleasure in your life, affected gene expression related to immune function. We also showed that substituting randomly generated data worked as well as what he got from blood samples in replicating his original results. I don’t think it is ad hominem to point out a history for both of the authors of making implausible claims. It speaks to source credibility.
  1. Check and see if there is a trial registration for a study, but don’t stop there. You can quickly check with PubMed if a report of a randomized trial is registered. Trial registration is intended to ensure that investigators commit themselves to a primary outcome or maybe two and whether that is what they emphasized in their paper. You can then check to see if what is said in the report of the trial fits with what was promised in the protocol. Unfortunately, I could find only one of these was registered. The trial registration was vague on what outcome variables would be assessed and did not mention the outcome emphasized in the published paper (!). The registration also said the sample would be larger than what was reported in the published study. When researchers have difficulty in recruitment, their study is often compromised in other ways. I’ll show how this study was compromised.

Well, it looks like applying these generally useful rules of thumb is not always so easy with these studies. I think the small sample size across all of the studies would be enough to decide this research has yet to yield meaningful results and certainly does not support the claims that are being made.

But readers who are motivated to put in the time of probing deeper come up with strong signs of p-hacking and questionable research practices.

  1. Check the report of the randomized trial and see if you can find any declaration of one or two primary outcomes and a limited number of secondary outcomes. What you will find instead is that the studies always have more outcome variables than patients receiving these interventions. The opportunities for cherry picking positive findings and discarding the rest are huge, especially because it is so hard to assess what data were collected but not reported.
  1. Check and see if you can find tables of unadjusted primary and secondary outcomes. Honest and transparent reporting involves giving readers a look at simple statistics so they can decide if results are meaningful. For instance, if effects on stress and depressive symptoms are claimed, are the results impressive and clinically relevant? Almost in all cases, there is no peeking allowed. Instead, authors provide analyses and statistics with lots of adjustments made. They break lots of rules in doing so, especially with such a small sample. These authors are virtually assured to get results to crow about.

Famously, Joe Simmons and Leif Nelson hilariously published claims that briefly listening to the Beatles’ “When I’m 64” left students a year and a half older younger than if they were assigned to listening to “Kalimba.”  Simmons and Leif Nelson knew this was nonsense, but their intent was to show what researchers can do if they have free reign with how they analyze their data and what they report and  . They revealed the tricks they used, but they were so minor league and amateurish compared to what the authors of these trials consistently did in claiming that yoga, tai chi, and mindfulness modified expression of DNA.

Stay tuned for my next blog post where I go through the six studies. But consider this, if you or a loved one have to make an immediate decision about whether to plunge into the world of woo woo unproven medicine in hopes of  altering DNA expression. I will show the authors of these studies did not get the results they claimed. But who should care if they did? Effects were laughably trivial. As the authors of this review about which I have been complaining noted:

One other problem to consider are the various environmental and lifestyle factors that may change gene expression in similar ways to MBIs [Mind-Body Interventions]. For example, similar differences can be observed when analyzing gene expression from peripheral blood mononuclear cells (PBMCs) after exercise. Although at first there is an increase in the expression of pro-inflammatory genes due to regeneration of muscles after exercise, the long-term effects show a decrease in the expression of pro-inflammatory genes (55). In fact, 44% of interventions in this systematic review included a physical component, thus making it very difficult, if not impossible, to discern between the effects of MBIs from the effects of exercise. Similarly, food can contribute to inflammation. Diets rich in saturated fats are associated with pro-inflammatory gene expression profile, which is commonly observed in obese people (56). On the other hand, consuming some foods might reduce inflammatory gene expression, e.g., drinking 1 l of blueberry and grape juice daily for 4 weeks changes the expression of the genes related to apoptosis, immune response, cell adhesion, and lipid metabolism (57). Similarly, a diet rich in vegetables, fruits, fish, and unsaturated fats is associated with anti-inflammatory gene profile, while the opposite has been found for Western diet consisting of saturated fats, sugars, and refined food products (58). Similar changes have been observed in older adults after just one Mediterranean diet meal (59) or in healthy adults after consuming 250 ml of red wine (60) or 50 ml of olive oil (61). However, in spite of this literature, only two of the studies we reviewed tested if the MBIs had any influence on lifestyle (e.g., sleep, diet, and exercise) that may have explained gene expression changes.

How about taking tango lessons instead? You would at least learn dance steps, get exercise, and decrease any social isolation. And so what if there were more benefits than taking up these other activities?

 

 

Unmasking Jane Brody’s “A Positive Outlook May Be Good for Your Health” in The New York Times

A recipe for coercing ill people with positive psychology pseudoscience in the New York Times

  • Judging by the play she gets in social media and the 100s of comments on her articles in the New York Times, Jane Brody has a successful recipe for using positive psychology pseudoscience to bolster down-home advice you might’ve gotten from your grandmother.
  • Her recipe might seem harmless enough, but her articles are directed at people struggling with chronic and catastrophic physical illnesses. She offers them advice.
  • The message is that persons with physical illness should engage in self-discipline, practice positive psychology exercises – or else they are threatening their health and shortening their lives.
  • People struggling with physical illness have enough to do already. The admonition they individually and collectively should do more -they should become more self-disciplined- is condescending and presumptuous.
  • Jane Brody’s carrot is basically a stick. The implied threat is simply coercive: that people with chronic illness are not doing what they can to improve the physical health unless they engage in these exercises.
  • It takes a careful examination Jane Brody’s sources to discover that the “scientific basis” for this positive psychology advice is quite weak. In many instances it is patently junk, pseudoscience.
  • The health benefits claimed for positivity are unfounded.
  • People with chronic illness are often desperate or simply vulnerable to suggestions that they can and should do more.  They are being misled by this kind of article in what is supposed to be the trusted source of a quality news outlet, The New York Times, not The Daily News.
  • There is a sneaky, ill-concealed message that persons with chronic illness will obtain wondrous benefits by just adopting a positive attitude – even a hint that cancer patients will live longer.

In my blog post about positive psychology and health, I try to provide  tools so that consumers can probe for themselves the usually false and certainly exaggerated claims that are being showered on them.

However, in the case of Jane Brody’s articles, we will see that the task is difficult because she draws on a selective sampling of the literature in which researchers generate junk self-promotional claims.

That’s a general problem with the positive psychology “science” literature, but the solution for journalists like Jane Brody is to seek independent evaluation of claims from outside the positive psychology community. Journalists, did you hear that message?

The article, along with its 100s of comments from readers, is available here:

A Positive Outlook May Be Good for Your Health by Jane E.Brody

The article starts with some clichéd advice about being positive. Brody seems to be on the side of the autonomy of her  readers. She makes seemingly derogatory comments  that the advice is “cockeyed optimism” [Don’t you love that turn of phrase? I’m sure to borrow it in the future]

“Look on the sunny side of life.”

“Turn your face toward the sun, and the shadows will fall behind you.”

“Every day may not be good, but there is something good in every day.”

“See the glass as half-full, not half-empty.”

Researchers are finding that thoughts like these, the hallmarks of people sometimes called “cockeyed optimists,” can do far more than raise one’s spirits. They may actually improve health and extend life.

See?  The clever putdown of this advice was just a rhetorical device, just a set up for what follows. Very soon Brody is delivering some coercive pseudoscientific advice, backed by the claim that “there is no longer any doubt” and that the links between positive thinking and health benefits are “indisputable.”

There is no longer any doubt that what happens in the brain influences what happens in the body. When facing a health crisis, actively cultivating positive emotions can boost the immune system and counter depression. Studies have shown an indisputable link between having a positive outlook and health benefits like lower blood pressure, less heart disease, better weight control [Emphasis added.].

I found the following passage particularly sneaky and undermining of people with cancer.

Even when faced with an incurable illness, positive feelings and thoughts can greatly improve one’s quality of life. Dr. Wendy Schlessel Harpham, a Dallas-based author of several books for people facing cancer, including “Happiness in a Storm,” was a practicing internist when she learned she had non-Hodgkin’s lymphoma, a cancer of the immune system, 27 years ago. During the next 15 years of treatments for eight relapses of her cancer, she set the stage for happiness and hope, she says, by such measures as surrounding herself with people who lift her spirits, keeping a daily gratitude journal, doing something good for someone else, and watching funny, uplifting movies. Her cancer has been in remission now for 12 years.

“Fostering positive emotions helped make my life the best it could be,” Dr. Harpham said. “They made the tough times easier, even though they didn’t make any difference in my cancer cells.”

Sure, Jane Brody is careful to avoid the explicit claim the positive attitude somehow is connected to the cancer being in remission for 12 years, but the implication is there. Brody pushes the advice with a hint of the transformation available to cancer patients, only if they follow the advice.

After all, Jane Brody had just earlier asserted that positive attitude affects the immune system and this well-chosen example happens to be a cancer of the immune system.

Jane Brody immediately launches into a description of a line of research conducted by a positive psychology group at Northwestern University and University of California San Francisco.

Taking her cue from the investigators, Brody blurs the distinction between findings based in correlational studies and the results of intervention studies in which patients actually practiced positive psychology exercises.

People with new diagnoses of H.I.V. infection who practiced these skills carried a lower load of the virus, were more likely to take their medication correctly, and were less likely to need antidepressants to help them cope with their illness.

But Brody sins as a journalist are worse than that. With a great deal of difficulty, I have chased her claims back into the literature. I found some made up facts.

In my literature search, I could find only one study from these investigators that seemed directly related to these claims. The mediocre retrospective correlational study was mainly focused on use of psychostimulants, but it included a crude 6-item summary measure  of positive states of mind.

The authors didn’t present the results in a simple way that allows direct independent examination of whether indeed positive affect is related to other outcomes in any simple fashion. They did not allow check of simple correlations needed to determine whether their measure was not simply a measure of depressive symptoms turned on its head. They certainly had the data, but did not report it. Instead, they present some multivariate analyses that do not show impressive links. Any direct links to viral load are not shown and presumably are not there, although the investigators tested statistically for them. Technically speaking, I would write off the findings to measurement and specification error, certainly not worthy of reporting in The New York Times.

Less technically speaking, Brody is leading up to using HIV as an exemplar illness where cultivating positivity can do so much. But if this study is worth anything at all, it is to illustrate that even correlationally, positive affect is not related to much, other than – no surprise – alternative measures of positive affect.

Brody then goes on to describe in detail an intervention study. You’d never know from her description that her source of information is not a report of the results of the intervention study, but a promissory protocol that supposedly describes how the intervention study was going to be done.

I previously blogged about this protocol. At first, I thought it was praiseworthy that a study of a positive psychology intervention for health had even complied with the requirement that studies be preregistered and have a protocol available. Most such studies do not, but they are supposed to do that. In plain English, protocols are supposed to declare ahead of time what researchers are going to do and precisely how they are going to evaluate whether an intervention works. That is because, notoriously, researchers are inclined to say later they were really trying to do something else and to pick another outcome that makes the intervention look best.

But then I got corrected by James Heathers on Facebook. Duh, he had looked at the date the protocol was published.

He pointed out that this protocol was actually published years after collection of data had begun. The researchers already had a lot to peek at. Rather than identifying just a couple of variables on which the investigators were prepared to stake their claim the intervention was affected, the protocol listed 25 variables that would be examined as outcomes (!) in order to pick one or two.

So I updated what I said in my earlier blog. I pointed out that the published protocol was misleading. It was posted after the fact of the researchers being able to see how their study was unfolding and to change their plains accordingly.  The vagueness of the protocol gave the authors lots of wiggle room for selectively reporting and hyping their findings with the confirmation bias. They would later take advantage of this when they actually published the results of their study.

The researchers studied 159 people who had recently learned they had H.I.V. and randomly assigned them to either a five-session positive emotions training course or five sessions of general support. Fifteen months past their H.I.V. diagnosis, those trained in the eight skills maintained higher levels of positive feelings and fewer negative thoughts related to their infection.

Brody is not being accurate here. When the  authors finally got around to publishing the results, they told a very different story if you probe carefully. Even with the investigators doing a lot of spinning, they showed null results, no effects for the intervention. Appearances the contrary were created by the investigators ignoring what they actually reported in their tables. If you go to my earlier blog post, I point this out in detail, so you can see for yourself.

Brody goes on to describe the regimen that was not shown in the published study validation to be effective.

An important goal of the training is to help people feel happy, calm and satisfied in the midst of a health crisis. Improvements in their health and longevity are a bonus. Each participant is encouraged to learn at least three of the eight skills and practice one or more each day. The eight skills are:

■ Recognize a positive event each day.

■ Savor that event and log it in a journal or tell someone about it.

■ Start a daily gratitude journal.

■ List a personal strength and note how you used it.

■ Set an attainable goal and note your progress.

■ Report a relatively minor stress and list ways to reappraise the event positively.

■ Recognize and practice small acts of kindness daily.

■ Practice mindfulness, focusing on the here and now rather than the past or future.

For chrissakes, this is a warmed over version of Émile Coué de la Châtaigneraie’s autosuggestion “Every day in every way, I’m getting better and better. Surely, contemporary positive psychology’s science of health can do better than that. To Coué’s credit, he gave away his advice for free. He did not charge for his coaching, even if he was giving away something for which he had no evidence would improve people’s physical health.

Dr. Moskowitz said she was inspired by observations that people with AIDS, Type 2 diabetes and other chronic illnesses lived longer if they demonstrated positive emotions. She explained, “The next step was to see if teaching people skills that foster positive emotions can have an impact on how well they cope with stress and their physical health down the line.”

She listed as the goals improving patients’ quality of life, enhancing adherence to medication, fostering healthy behaviors, and building personal resources that result in increased social support and broader attention to the good things in life.

Let me explain why I am offended here. None of these activities have been shown to improve the health of persons with newly diagnosed HIV. It’s reasonable to assume that newly diagnosed persons have a lot with which to contend. It’s a bad time to give them advice to clutter their life with activities that will not make a difference in their health.

The published study was able to recruit and retain a sample of persons with newly diagnosed HIV because it paid them well to keep coming. I’ve worked with this population before, in a study aiming at helping them solve specific practical problems that that they said got in the way of their adherence.

Many persons with newly diagnosed HIV are low income and are unemployed or marginally employed. They will enroll in studies to get the participant fees. When I lived in the San Francisco Bay area, I recall one patient telling a recruiter from UCSF that he was too busy and unable to make a regular visit to the medical center for the intervention, but he would be willing to accept being in the study if he was assigned to the control group. It did not involve attending intervention sessions and would give him a little cash.

Based on my clinical and research experience, I don’t believe that such patients would regularly show up for this kind of useless positive psychology treatment without getting paid. Paticularly if they were informed of the actual results of this misrepresented study.

Gregg De Meza, a 56-year-old architect in San Francisco who learned he was infected with H.I.V. four years ago, told me that learning “positivity” skills turned his life around. He said he felt “stupid and careless” about becoming infected and had initially kept his diagnosis a secret.

“When I entered the study, I felt like my entire world was completely unraveling,” he said. “The training reminded me to rely on my social network, and I decided to be honest with my friends. I realized that to show your real strength is to show your weakness. No pun intended, it made me more positive, more compassionate, and I’m now healthier than I’ve ever been.”

I object to this argument by quotes-from-an-unrepresentative-patient. The intervention did not have the intended effect, and it is misleading to find somebody who claim to turn their life around.

Jane Brody proceeds with some more fake facts.

In another study among 49 patients with Type 2 diabetes, an online version of the positive emotions skills training course was effective in enhancing positivity and reducing negative emotions and feelings of stress. Prior studies showed that, for people with diabetes, positive feelings were associated with better control of blood sugar, an increase in physical activity and healthy eating, less use of tobacco and a lower risk of dying.

The study was so small and underpowered, aside from being methodologically flawed, that even if such effects were actually present, most of the time they would be missed because the study did not have enough patients to achieve significance.

In a pilot study of 39 women with advanced breast cancer, Dr. Moskowitz said an online version of the skills training decreased depression among them. The same was true with caregivers of dementia patients.

“None of this is rocket science,” Dr. Moskowitz said. “I’m just putting these skills together and testing them in a scientific fashion.”

It’s not rocket science, it’s misleading hogwash.

In a related study of more than 4,000 people 50 and older published last year in the Journal of Gerontology, Becca Levy and Avni Bavishi at the Yale School of Public Health demonstrated that having a positive view of aging can have a beneficial influence on health outcomes and longevity. Dr. Levy said two possible mechanisms account for the findings. Psychologically, a positive view can enhance belief in one’s abilities, decrease perceived stress and foster healthful behaviors. Physiologically, people with positive views of aging had lower levels of C-reactive protein, a marker of stress-related inflammation associated with heart disease and other illnesses, even after accounting for possible influences like age, health status, sex, race and education than those with a negative outlook. They also lived significantly longer.

This is even deeper into the woo. Give me a break, Jane Brody. Stop misleading people with chronic illness with false claims and fake facts. Adopting these attitudes will not prevent dementia.

Don’t believe me? I previously debunked these patently false claims in detail. You can see my critique here.

Here is what the original investigators claimed about Alzheimer’s:

We believe it is the stress generated by the negative beliefs about aging that individuals sometimes internalize from society that can result in pathological brain changes,” said Levy. “Although the findings are concerning, it is encouraging to realize that these negative beliefs about aging can be mitigated and positive beliefs about aging can be reinforced, so that the adverse impact is not inevitable.”

I exposed some analysis of voodoo statistics on which this claim is based. I concluded:

The authors develop their case that stress is a significant cause of Alzheimer’s disease with reference to some largely irrelevant studies by others, but depend on a preponderance of studies that they themselves have done with the same dubious small samples and dubious statistical techniques. Whether you do a casual search with Google scholar or a more systematic review of the literature, you won’t find stress processes of the kind the authors invoke among the usual explanations of the development of the disease.

Basically, the authors are arguing that if you hold views of aging like “Old people are absent-minded” or “Old people cannot concentrate well,” you will experience more stress as you age, and this will accelerate development of Alzheimer’s disease. They then go on to argue that because these attitudes are modifiable, you can take control of your risk for Alzheimer’s by adopting a more positive view of aging and aging people

Nonsense, utter nonsense.

Let chronically ill people and those facing cancer adopt any attitude is comfortable or natural for them. It’s a bad time to ask for change, particularly when there isn’t any promised benefit in improved health or prolonged life.

Rather than Jane Brody’s recipe for positive psychology improving your health, I strongly prefer Lilia Downe’s  La Cumbia Del Mole.

It is great on chicken. If it does not extend your life, It will give you some moments of happiness, but you will have to adjust the spices to your personal taste.

I will soon be offering e-books providing skeptical looks at positive psychology, as well as mindfulness. As in this blog post, I will take claims I find in the media and trace them back to the scientific studies on which they are based. I will show you what I see so you can see it too.

 Sign up at my new website to get advance notice of the forthcoming e-books and web courses, as well as upcoming blog posts at this and other blog sites. You can even advance order one or all of the e-books.

 Lots to see at CoyneoftheRealm.com. Come see…

1 billion views! Why we should be concerned about PR campaign for 2 RCTs of psilocybin for cancer patients

According to the website of an advocacy foundation, coverage of two recent clinical trials published in in Journal of Psychopharmacology evaluating psilocybin for distress among cancer patients garnered over 1 billion views in the social media. To put that in context, the advocacy group claimed that this is one sixth of the attention that the Super Bowl received.

In this blog post I’ll review the second of the two clinical trials. Then, I will discuss some reasons why we should be concerned about the success of this public relations campaign in terms of what it means for both the integrity of scientific publishing, as well as health and science journalism.

The issue is not doubt that cancer patients will find benefit from the ingesting psychedelic mushroom in a safe environment. Nor that sale and ingestion of psilocybin is currently criminalized (Schedule 1, classified same as heroin).

We can appreciate the futility of the war on drugs, and the absurdity of the criminalization of psilocybin, but still object to how, we were strategically and effectively manipulated by this PR campaign.

Even if we approve of a cause, we need to be careful about subordinating the peer-review process and independent press coverage to the intended message of advocates.

Tolerating causes being promoted in this fashion undermines the trustworthiness of peer review and of independent press coverage of scientific papers.

To contradict a line from the 1964 acceptance speech of Republican Presidential Candidate Barry Goldwater, “Extremism in pursuit of virtue is no [a] vice. “

In this PR campaign –

We witnessed the breakdown of expected buffer of checks and balances between:

  • An advocacy group versus reporting of clinical trials in a scientific journal evaluating its claims.
  • Investigators’ exaggerated self-promotional claims versus editorial review and peer commentary.
  • Materials from the publicity campaign versus supposedly independent evaluation by journalists.

What if the next time the object of promotion is pharmaceuticals or medical devices by authors with conflicts of interest? But wait! Isn’t that what we’ve seen in JAMA Network journals on a smaller scale? Such as dubious claims about the wondrous effects of deep brain stimulation in JAMA: Psychiatry by promoters who “disappeared” failed trials? And claims in JAMA itself that suicides were eliminated at a behavioral health organization outside Detroit?

Is this part of a larger trend, where advocacy and marketing shape supposedly peer-reviewed publications in prestigious medical journals?

The public relations campaign for the psilocybin RCTs also left in tatters the credibility of altmetrics as an alternative to journal impact factors. The orchestrating of 1 billion views is a dramatic demonstration how altmetrics can be readily gamed. Articles published in a journal with a modest impact factor scored spectacularly, as seen in these altmetrics graphics the Journal of Psychopharmacology posted.

I reviewed in detail one of the clinical trials in my last blog post and will review the second in this one. They are both mediocre, poorly designed clinical trials that got lavishly praised as being highest quality by an impressive panel of commentators. I’ll suggest that in particular the second trial is best seen as what Barney Caroll has labeled  an experimercial, a clinical trial aimed at generating enthusiasm for a product, rather than a dispassionate evaluation undertaken with some possibility of not been able to reject the null hypothesis. If this sounds harsh, please indulge me and read on and be entertained and I think persuaded that this was not a clinical trial but an elaborate ritual, complete with psychobabble woo that has no place in the discussion of the safety and effectiveness of medicine.

After skeptically scrutinizing the second trial, I’ll consider the commentaries and media coverage of the two trials.

I’ll end with a complaint that this PR effort is only aimed at securing the right of wealthy people with cancer to obtain psilocybin under supervision of a psychiatrist and in the context of woo psychotherapy. The risk of other people in other circumstances ingesting psilocybin is deliberately exaggerated. If psilocybin is as safe and beneficial as claimed by these articles, why should use remain criminalized for persons who don’t have cancer or don’t want to get a phony diagnosis from a psychiatrist or don’t want to submit to woo psychotherapy?

The normally pay walled Journal of Psychopharmacology granted free access to the two articles, along with most but not all of the commentaries. However, extensive uncritical coverage in Medscape Medical News provides a fairly accurate summary, complete with direct quotes of lavish self-praise distributed by the advocacy-affiliated investigators and echoed in seemingly tightly coordinated commentaries.

The praise one of the two senior authors heaped upon their two studies as captured in Medscape Medical News and echoed elsewhere:

The new findings have “the potential to transform the care of cancer patients with psychological and existential distress, but beyond that, it potentially provides a completely new model in psychiatry of a medication that works rapidly as both an antidepressant and anxiolytic and has sustained benefit for months,” Stephen Ross, MD, director of Substance Abuse Services, Department of Psychiatry, New York University (NYU), Langone Medical Center, told Medscape Medical News.

And:

“That is potentially earth shattering and a big paradigm shift within psychiatry,” Dr Ross told Medscape Medical News.

The Hopkins Study

Griffiths RR, Johnson MW, Carducci MA, Umbricht A, Richards WA, Richards BD, Cosimano MP, Klinedinst MA. Psilocybin produces substantial and sustained decreases in depression and anxiety in patients with life-threatening cancer: A randomized double-blind trial. Journal of Psychopharmacology. 2016 Dec 1;30(12):1181-97.

The trial’s available registration is at ClinicalTrial.gov is available here.

The trial’s website is rather drab and typical for clinical trials. It contrasts sharply with the slick PR of the website for the NYU trial . The latter includes a gushy, emotional  video from a clinical psychologist participating as a patient in the study.  She delivers a passionate pitch for the “wonderful ritual” of the transformative experimental session. You can also get a sense of how session monitor structured the session and cultivated positive expectations. You also get a sense of the psilocybin experience being slickly marketed to appeal to the same well-heeled patients who pay out-of-pocket for complementary and alternative medicine at integrative medicine centers.

Conflict of interest

The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Roland Griffiths is on the Board of Directors of the Heffter Research Institute.

Heffter Research Institute is listed as one of the funders of the study.

The introduction.

 The Hopkins study starts with some familiar claims from psycho-oncology ] that portray cancer as a mental health issue. The exaggerated estimates of 40% of cancer patients experiencing a mood disorder is arrived at by lumping adjustment reactions with a smaller proportion of diagnoses of generalized anxiety and major depression.

The introduction contradicts a large body of literature that suggests that the prevalence of mental disorder in cancer patients is no greater than other chronic health conditions  and may approximate what is found in primary care waiting rooms . There is also a fundamental confusion of psychological distress associated with diagnosis of cancer with psychiatric disorder in need of treatment. Much of the initial psychological distress in cancer patients resolves in a short time, making it difficult to demonstrate benefits of treatment beyond this natural trajectory of decline. Prescription of an antidepressant would be ineffective and inappropriate.

The introduction ends with a strong claim to the rigor and experimental control exercised in the clinical trial:

The present study provides the most rigorous evaluation to date of the efficacy of a classic hallucinogen for treatment of depressed mood and anxiety in psychologically distressed cancer patients. The study evaluated a range of clinically relevant measures using a double-blind cross-over design to compare a very low psilocybin dose (intended as a placebo) to a moderately high psilocybin dose in 51 patients under conditions that minimized expectancy effects.

The methods and results

In a nutshell: Despite claims to the contrary, this study cannot be considered a blinded study. At the six month follow-up, which is the outcome assessment point of greatest interest, it could no longer meaningfully considered a randomized trial. All benefits of randomization were lost. In addition, the effects of psilocybin were confounded with a woo psychotherapy in which positive expectations and support were provided and reinforced in a way that likely influenced assessments of outcome. Outcomes at six months also reflected changes in distress which would’ve occurred in the absence of treatment. The sample is inappropriate for generalizations about the treatment of major depression and generalized anxiety. The characterization of patients as facing impending death is inaccurate.

 The study involved a crossover design, which provides a lower level of evidence than a placebo controlled comparison study. The study compared a high psilocybin dose (22 or 30 mg/70 kg) with a low dose (1 or 3 mg/70 kg) administered in identically appearing capsules. While the low dose might not be homeopathic, it can be readily distinguished soon after administration from the larger dosage. The second drug administration occurred approximately 5 weeks later. Not surprisingly, with the high difference in dosage, session monitors who were supposedly blinded readily identified the group to which the participant they were observing had been assigned.

Within a cross over design, the six month follow-up data basically attributed any naturalistic decline in distress to the drug treatments. As David Colquhoun would argue, any estimate of the effects of the drug was inflated by including regression to the mean and get-better anyway effects. Furthermore, the focus on outcomes at six months meant patients assigned to either group in the crossover design had received high dosage psilocybin by at least five weeks into the study. Any benefits of randomization were lost.

Like the NYU study, the study Johns Hopkins involves selecting a small, unrepresentative sample of a larger group responding to a mixed recruitment strategy utilizing flyers, the internet, and physician referral.

  • Less than 10% of the cancer patients calling in were randomized.
  • Almost half of the final sample were currently using marijuana and, similarly, almost half had used hallucinogens in the past.
  • The sample is relatively young for cancer patients and well educated. More than half had postgraduate education, almost all were white, but there were two black people.
  • The sample is quite heterogeneous with respect to psychiatric diagnoses, with almost half having an adjustment disorder, and the rest anxiety and mood disorders.
  • In terms of cancer diagnoses and staging, it was also a select and heterogeneous group with only about a quarter having recurrent/metastatic disease with less than two years of expected survival. This suggests the odd “life-threatening” in the title is misleading.

Any mental health effects of psilocybin as a drug are inseparable from the effects of accompanying psychotherapy designed by a clinical psychologist “with extensive experience in studies of classic hallucinogens.” Participants met with that “session monitor” several times before the session in which the psilocybin was ingested in the monitor guided and aided in the interpretation of the drug experience. Aside from providing therapy, the session monitor instructed the patient to have positive expectations before the ingestion of the drug and work to maintain these expectations throughout the experience.

I found this psychotherapeutic aspect of the trial strikingly similar to one that was included in a trial of homeopathy in Germany that I accepted for publication in PLOS One. [See here for my rationale for accepting the trial and the ensuing controversy.] Trials of alternative therapies notoriously have such an imbalance of nonspecific placebo factors favoring the intervention group.

The clinical trial registration indicates that the primary outcome was the Pahnke-Richards Mystical Experience Questionnaire. This measure is included among 20 participant questionnaires listed in the Table 3 in the article as completed seven hours after administration of psilocybin. Although I haven’t reviewed all of these measures, I’m skeptical about their psychometric development, intercorrelation, and validation beyond face validity. What possibly could be learned from administering such a battery?

The authors make unsubstantiated assumptions in suggesting that these measures either individually or collectively capture mediation of later response assessed by mental health measures. A commentary echoed this:

Mediation analysis indicates that the mystical experience was a significant mediator of the effects of psilocybin dose on therapeutic outcomes.

But one of the authors of the commentary later walked that back with a statement to Medscape Medical News:

As for the mystical experiences that some patients reported, it is not clear whether these are “a cause, consequence or corollary of the anxiolytic effect or unconstrained cognition.”

Clinical outcomes at six months are discussed in terms of multiple measures derived from the unblinded, clinician-rated Hamilton scales. However, there are repeated references to box scores of the number of significant findings from at least 17 clinical measures (for instance, significant effects for 11 of the 17 measures), in addition to other subjective patient and significant-other measures. It is unclear why the authors would choose to administer so many measures that are highly likely intercorrelated.

There were no adverse events attributed to administration of psilocybin, and while there were a number of adverse psychological effects during the session with the psilocybin, none were deemed serious.

My summary evaluation

The clinical trial registration indicates broad inclusion criteria which may suggest the authors anticipated difficulty in recruiting patients that had significant psychiatric disorder for which psychotropic medication would be appropriate, as well as difficulty obtaining cancer patients that actually had poorer prognoses. Regardless, descriptions of the study is focusing on anxiety and depression and on “life-threatening” cancer seem to be marketing. You typically do not see a mixed sample with a large proportion of adjustment reaction characterized in the title of a psychiatric journal as treatment of “anxiety” and “depression”. You typically do not see a the adjective “life-threatening” in the title of an oncology article with such a mixed sample of cancer patients.

The authors could readily have anticipated that at the six-month assessment point of interest that they no longer had a comparison they could have been described as a rigorous double-blind, randomized trial. They should have thought through exactly what was being controlled by a control comparison group of a minimal dose of psilocybin. They should have been clearer that they were not simply evaluating psilocybin, but psilocybin administered in the context of a psychotherapy and an induction of strong positive expectations and promise of psychological support.

The finding of a lack of adverse events is consistent with a large literature, but is contradicted in the way the study is described to the media.

The accompanying editorial and commentary

Medscape Medical News reports the numerous commentaries accompanies these two clinical trials were hastily assembled. Many of the commentaries read that way, with the authors uncritically passing on the psilocybin authors’ lavish self praise of their work, after a lot of redundant recounts of the chemical nature of psilocybin and its history in psychiatry. When I repeatedly encountered claims that these trials represented rigorous, double blinded clinical trials or suggestions that the cancer was in a terminal phase, I assumed that the authors had not read the studies, only the publicity material, or simply had suspended all commitment to truth.

harmsI have great admiration for David Nutt  and respect his intellectual courage in campaigning for the decriminalization of recreational drugs, even when he knew that it would lead to his dismissal as chairman of the UK’s Advisory Council on the Misuse of Drugs (ACMD). He has repeatedly countered irrationality and prejudice with solid evidence. His graph depicting the harms of various substances to the uses and others deserves the wide distribution that it has received.

He ends his editorial with praise for the two trials as “the most rigorous double-blind placebo-controlled trials of a psychedelic drug in the past 50 years.” I’ll give him a break and assume that that reflects his dismal assessment of the quality of the other trials. I applaud his declaration, available nowhere else in the commentaries that:

There was no evidence of psilocybin being harmful enough to be controlled when it was banned, and since then, it has continued to be used safely by millions of young people worldwide with a very low incidence of problems. In a number of countries, it has remained legal, for example in Mexico where all plant products are legal, and in Holland where the underground bodies of the mushrooms (so-called truffles) were exempted from control.

His description of the other commentaries accompanying the two trials is apt:

The honours list of the commentators reads like a ‘who’s who’ of American and European psychiatry, and should reassure any waverers that this use of psilocybin is well within the accepted scope of modern psychiatry. They include two past presidents of the American Psychiatric Association (Lieberman and Summergrad) and the past-president of the European College of Neuropsychopharmacology (Goodwin), a previous deputy director of the Office of USA National Drug Control Policy (Kleber) and a previous head of the UK Medicines and Healthcare Regulatory Authority (Breckenridge). In addition, we have input from experienced psychiatric clinical trialists, leading pharmacologists and cancer-care specialists. They all essentially say the same thing..

The other commentaries. I do not find many of the commentaries worthy of further comment. However, one by Guy M Goodwin, Psilocybin: Psychotherapy or drug? Is unusual in offering even mild skepticism about the way the investigators are marketing their claims:

The authors consider this mediating effect as ‘mystical’, and show that treatment effects correlate with a subjective scale to measure such experience. The Oxford English Dictionary defines mysticism as ‘belief that union with or absorption into the Deity or the absolute, or the spiritual apprehension of knowledge inaccessible to the intellect, may be attained through contemplation and self-surrender’. Perhaps a scale really can measure a relevant kind of experience, but it raises the caution that the investigation of hallucinogens as treatments may be endangered by grandiose descriptions of their effects and unquestioning acceptance of their value.

The commentary by former president of the American Psychiatric Association Paul Summergrad, Psilocybin in end of life care: Implications for further research shamelessly echoes the psychobabble and self-promotion of the authors of the trials:

The experiences of salience, meaningfulness, and healing that accompanied these powerful spiritual experiences and that were found to be mediators of clinical response in both of these carefully performed studies are also important to understand in their own right and are worthy of further study and contemplation. None of us are immune from the transitory nature of human life, which can bring fear and apprehension or conversely a real sense of meaning and preciousness if we carefully number our days. Understanding where these experiences fit in healing, well-being, and our understanding of consciousness may challenge many aspects of how we think about mental health or other matters, but these well-designed studies build upon a recent body of work that confronts us squarely with that task.

Coverage in of the two studies in the media

The website for Heffter Research Institute  provides a handy set of links to some of the press coverage of the studies have received. There’s remarkable sameness to the portrayal of the study in the media, suggesting that journalists stayed closely to the press releases, except occasionally supplementing these with direct quotes from the authors. The appearance of a solicitation of independent evaluation of the trial almost entirely dependent on the commentaries published with the two articles.

There’s a lot of slick marketing by the two studies’ authors. In addition to what I wrote noted earlier in the blog, there are recurring unscientific statements marketing the psilocybin experience:

“They are defined by a sense of oneness – people feel that their separation between the personal ego and the outside world is sort of dissolved and they feel that they are part of some continuous energy or consciousness in the universe. Patients can feel sort of transported to a different dimension of reality, sort of like a waking dream.

There are also recurring distinct efforts to keep the psilocybin experience under the control of psychiatrists and woo clinical psychologists:

The new studies, however, suggest psilocybin be used only in a medical setting, said Dr. George Greer, co-founder, medical director and secretary at the Heffter Research Institute in Santa Fe, New Mexico, which funded both studies.

“Our focus is scientific, and we’re focused on medical use by medical doctors,” Greer said at the news conference. “This is a special type of treatment, a special type of medicine. Its use can be highly controlled in clinics with specially trained people.”

He added he doubts the drug would ever be distributed to patients to take home.

There are only rare admissions from an author of one of the studies that:

The results were similar to those they had found in earlier studies in healthy volunteers. “In spite of their unique vulnerability and the mood disruption that the illness and contemplation of their death has prompted, these participants have the same kind of experiences, that are deeply meaningful, spiritually significant and producing enduring positive changes in life and mood and behaviour,” he said.

If psilocybin is so safe and pleasant to ingest…

I think the motion of these studies puts ingestion of psilocybin on the path to being allowed in nicely furnished integrative cancer centers. In that sense psilocybin could become a gateway drug to quack services such as acupuncture, reiki, and energy-therapy therapeutic touch.

I’m not sure that demand would be great except among previous users of psychedelics and current users of cannabis.

But should psilocybin remain criminalized outside of cancer centers where wealthy patients can purchase a diagnosis of adjustment reaction from a psychiatrist? Cancer is not especially traumatic and PTSD is almost as common in the waiting rooms of primary care physicians. Why not extend to primary care physicians the option of prescribing psilocybin to their patients? What would be accomplished is that the purity could be assured. But why should psilocybin use being limited to mental health conditions, once we accept that a diagnosis of adjustment reaction is such a distorted extension of the term? Should we exclude patients who are atheists and only wants a satisfying experience, not a spiritual one?

Experience in other countries suggests that psilocybin can safely be ingested in a supportive, psychologically safe environment. Why not allow cancer patients and others to obtain psilocybin with assured purity and dosage? They could then ingest it in the comfort of friends and intimate partners who have been briefed on how the experience needs to be managed. The patients in the studies were mostly not facing immediate death from terminal cancer. But should we require that persons need to be dying in order to have a psilocybin experience without the risk of criminal penalties? Why not allow psilocybin to be ingested in the presence of pastoral counselors or priests whose religious beliefs are more congruent with the persons seeking such experiences than are New York City psychiatrists?

 

 

 

 

 

Psilocybin as a treatment for cancer patients who are not depressed: the NYU study

psilocybinThe well orchestrated promotion of a modest and misrepresented pair of studies of using psilocybin with cancer patients raises lots of issues and should leave lots of people embarrassed.

A breakdown in peer review allowed unmoderated and misleading statements about the studies to appear in commentaries from an impressive array of leading psychiatrists. They embarrassed themselves by claiming the studies were well-designed trials and that the patients were clinically depressed.

There was a breakdown in the press’s evaluation of press releases from the studies, with journalists embarrassing themselves by passing on outrageously self-congratulatory statements by the authors without independently checking them. Journalists covering the story almost exclusively drew on a rich set of promotional materials from the authors’ institutions and added little else.

A lot of commentators in the social media, especially Twitter, embarrassed themselves by tweeting and liking others’ tweets with unbridled, uncritical enthusiasm for the claims of the authors. They obviously did not read the papers or read them carefully.

From the 1960s, the United States has had an irrational policy of criminalizing use of psilocybin. The articles and publicity campaign do not directly challenge this policy or call for decriminalization. Rather, they propose establishing a loophole by which people of means can obtain a diagnosis of adjustment disorder from a psychiatrist, receive a prescription for psilocybin (4-phosphoryloxy-N,N-dimethyltryptamine), and purchase an elaborate ceremony with lots of pseudoscientific trappings and mystical explanations. Use by others outside of this context will remain criminalized.

The publicity campaign can be seen as a larger effort by psychiatry to define dying as a mental health issue, routinely requiring their specialty expertise and services.

The articles’ justification for the trials runs roughshod over the existing literature, exaggerating psychiatric aspects of cancer, and minimizing the needs and preferences of cancer patients.

Hopefully, by the end of this blog and the next, I will convince readers that my assessments are more reasonable than they first appear.

Although published in a pay wall journal, both articles were made freely available, along with some of the laudatory comments, mostly from psychiatrists.

The first article is

Ross S, Bossis A, Guss J, Agin-Liebes G, Malone T, Cohen B, Mennenga SE, Belser A, Kalliontzi K, Babb J, Su Z. Rapid and sustained symptom reduction following psilocybin treatment for anxiety and depression in patients with life-threatening cancer: a randomized controlled trial. Journal of Psychopharmacology. 2016 Dec 1;30(12):1165-80.

The trial’s freely available registration at ClinicalTrial.gov is available here.

In this first of two blog posts, I will center on this study. However, a forthcoming blog post will focus on the second article, comparing and contrasting it to another study in methods and interpretation :

Griffiths RR, Johnson MW, Carducci MA, Umbricht A, Richards WA, Richards BD, Cosimano MP, Klinedinst MA. Psilocybin produces substantial and sustained decreases in depression and anxiety in patients with life-threatening cancer: A randomized double-blind trial. Journal of Psychopharmacology. 2016 Dec 1;30(12):1181-97.

The trial’s available registration is at ClinicalTrial.gov is available here. https://clinicaltrials.gov/ct2/show/NCT00465595

Distortions in coverage of the literature in the introduction to Ross et al study at NYU

 The introduction states:

Enduring clinically significant anxiety and/or depressive symptoms are common in patients with cancer, present in 30–40% of patients in hospital settings (Mitchell et al., 2011).

I debated Alex Mitchell, the author of this review to a packed audience at the International Association for Psycho-oncology meeting in Rotterdam.  Here are my slides.

I pointed out that these estimates are of self-reported symptoms, not clinical diagnoses, which is substantially lower. Although there is a modest elevation in self-reported symptoms immediately after diagnosis, there is a trajectory of rapid decline. Only about 12% of patients have heightened distress at six months, and is likely that many of those that do, had pre-existing psychological disorders or past histories. After the initial stress of a diagnosis, rates of distress among patients in an oncology waiting room approximate the rates of patients in primary care waiting rooms.

These symptoms are associated with a variety of poor outcomes, including medication non-adherence, increased health care utilization, adverse medical outcomes, decreased quality of life, decreased social function, increased disability, hopelessness, increased pain, increased desire for hastened death, increased rates of suicide, and decreased survival rates (Arrieta et al., 2013; Brown et al., 2003; Jaiswal et al., 2014).

These correlations are not established as causal, and mostly come from cross-sectional studies. Residual confounding and reverse causality are highly likely. Patients with more debilitating physical symptoms inadequately controlled pain register more distress. There is no evidence that treating depression increases survival rates or slows progression.

The introduction goes on to mention that effects of antidepressants have a slow onset and they are incompletely effective. The author should have noted that antidepressant treatment would be inappropriate for the patients with adjustment disorders that dominated their study sample.

…With a growing body of evidence linking higher levels of existential/spiritual wellbeing (in cancer patients) with improved quality of life and decreased depression/hopelessness/suicidality (Breitbart et al., 2000; McClain et al., 2003; Nelson et al., 2002), the need to develop effective therapeutic approaches to mitigate this domain of distress has become increasingly recognized within the disciplines of palliative care and psycho-oncology.

Hold on! The authors are introducing some pretty shabby research that psychiatrists should be addressing existential/spiritual well-being.Since when are existential/spiritual well-being issues psychiatric?

A number of manualized “existentially oriented psychotherapies” for cancer patients have been developed to address these existential/spiritual issues, with some empirical support from clinical trials (Lemay and Wilson, 2008), and several of these approaches were integrated into the therapy platform developed for this study.

I will pick up on this discussion in my next blog post. But in a previous blog post, I examined the largest trial (with 441 palliative care patients) for one of these “existential/spiritual” therapies.

That study found no differences in 22 different measures of distress (!) between a mental health professional delivering an existential/spiritual “dignity therapy” over standard palliative care or a client centered care delivered by a nurse. Nonetheless, in New York, you can find expensive continued education courses offered by psychiatrist for this dignity therapy.Having been evaluated in a large clinical trial does not necessarily make a treatment “evidence-based,” particularly when it only achieved no findings.

One of the problems acknowledged in that trial is that only a small proportion of palliative care patients had clinically significant anxiety and depression scales. This study is a particularly interesting comparison compared to the study of psilocybin for cancer patients. Palliative care patients in the larger study are arguably more confronting of an impending death than the patients in the psilocybin study. Yet the authors of the psilocybin study make a broad assumption that end-of-life is a time of considerable distress and a mental health issue.

Methods and results of the Ross et al study at NYU

Patients in the NYU Psilocybin Cancer Anxiety Study begin their hallucinatory experience wearing eyeshades and headphones as doctors encourage them to bear witness to an inner world - Photo and description provided by NYU Alumni News
Patients in the NYU Psilocybin Cancer Anxiety Study begin their hallucinatory experience wearing eyeshades and headphones as doctors encourage them to bear witness to an inner world – Photo and description provided by NYU Alumni News

Representing only about a third of the patients screened for the study, 29 patients were randomized to either psilocybin or niacin (vitamin B3) in a cross over design. The randomized patients were highly educated and almost half had a history of use of hallucinogens. Almost all patients (93%) indicated “white” as their race. No patients endorsed black, Hispanic, Asian, or Native American. All but five patients had diagnoses of adjustment disorder, and these other five had diagnoses of general anxiety disorder. Although about two thirds of the patients had stage III or stage IV cancer, the rest had stage I or stage II. is not clear why they allow such heterogeneity of patients expected survival time in their study. I suspect they had trouble recruiting patients.

Regardless, my immediate observation is that this is a highly unrepresentative sample. For psychiatry journal, it would usually be considered a misrepresentation to use “anxiety “ and “depression” in a title when we’re sample did predominantly not have diagnoses of major depression in clinical anxiety disorders. When did you last see “life-threatening” in the title of a paper in an oncology or even psycho-oncology journal? With the title, we already are seeing a bit of dramatization in the service of attracting more attention than a more accurate and conventional title.

Any sense of a double blinded study is undone by the use of niacin as a comparison/control. Patients would shortly discover whether they had been given vitamin B3 or a hallucinogenic drug in any research assistants would notice the difference.

The abstract mentions that the psilocybin and the vitamin B3 were administered after a preparatory psychotherapy session. The therapy was then delivered at during both the psilocybin and vitamin B3 treatments. Thus, that this is not simply treatment with psilocybin, but with a psilocybin/psychotherapy combination. There is not even minimal description of this psychological treatment in the article.

However,  if you go to the NYU Alumni Magazine, you’ll find an extensive interview and photos of some of the investigators. You’ll learn that both psilocybin and vitamin B3 are administered with a special ceramic chalice. Description of the psychotherapy is certainly what you would not expect for a manualized therapy. But the following section of the article is enlightening:

The NYU Psilocybin Cancer Anxiety Study is conducted in a serene space that seems more like someone’s living room than part of a bustling research clinic. There’s a couch strewn with ethnic-printed pillows, shelves stocked with oversize books of Tibetan art, framed landscape photographs, warm pools of lamplight, Buddha figurines, and, on dosing days, fresh fruit and flowers.

Notwithstanding the lovely room, psilocybin does present risks. Although it is not known to induce addiction, overdose, or withdrawal symptoms, some research has suggested it can bring about prolonged psychosis in people with a personal or family history of major mental illness, so such patients are carefully screened out of the study. In the session itself, there may be some physical side effects, such as nausea, dizziness, and tremors, but the more pronounced hazards are psychological. Periods of transient anxiety can occur as patients navigate the challenging psychological material related to cancer. More extreme reactions, such as paranoia and panic, can occur, but are rare and safeguarded against through careful preparation and a highly structured therapy session, much of which is influenced by the rituals of indigenous cultures that utilize psychedelic medicines.

Subjects start with a series of meetings with a male-female therapist dyad to build trust, establish familiarity, and set intentions. On the dosing day—there is one for placebo and one for the real thing, set seven weeks apart—a small container of psilocybin synthesized in a government-monitored laboratory and weighed daily according to strict DEA regulations is taken from an 800-pound safe. Twenty milligrams of powder, an amount precisely judged to increase the likelihood of a mystical experience, are measured and pressed into a pill, which subjects swallow from a ceramic chalice.

The drug starts to take hold after about half an hour and the subjects are encouraged to put on eyeshades and headphones, lie down, and ride the waves of music on a carefully curated playlist. The therapists sit quietly nearby. There’s often sobbing and sometimes laughter. After a few hours, subjects usually remove their eyeshades and start bearing witness to the inner world they’ve traversed.

This investigator’s New Age depiction of mechanism falls short of conventional scientific standards”

“People come out with an acceptance of the cycles of life,” Bossis says. “We’re born, we live, we find meaning, we love, we die, and it’s all part of something perfect and fine. The emergent themes are love, and transcending the body and this existence. In oncology, we’re pretty good at advancing life and targeting chemotherapies, but we’re not so good at addressing deep emotional distress about mortality. So to see someone cultivate a sense of acceptance and meaning, something that we all hope to cultivate over a 90-year life, in six hours? It’s profound.”

This study evaluated as a clinical trial

This study is grossly underpowered study, involving an unrepresentative sample of 29 patients whose only diagnosis was an adjustment disorder, with the exception of a few patients with generalized anxiety. Diagnoses of adjustment disorders are not validated nor have they received extensive study. If a patient has some distress with recent onset and they want to see a mental health professional, they can be given a diagnosis of adjustment disorder to justify treatment. For a while, that was a common practice in the Netherlands, but treatments billed for adjustment reactions are no longer allowed.

I’m sure where I got this phrase, but I sometimes refer to adjustment disorder as an acute compensatory reaction of psychiatrists seeking to bill for services.

The idea that this is a blinded trial is ridiculous. Very soon after administration of the drug,  patients and clinical and research staff knew whether the patients had received psilocybin or vitamin B3.

chalice used in studyA chalice was used to administer both psilocybin and vitamin B3. Once patients became unblinded by noticing any effects of the drug they had been given, the chalice added to the sense of this supposed to be a mystical experience. There is more than a little hokum in play.

We’re told little about the psychotherapy, but enough to reveal that it has strong nonspecific factors and should induce strong positive expectations.

Crossover designs provide weak evidence, particularly ones that are readily unblinded and if we are interested in any effects of the first treatment after the crossover. Patients immediately realized in the first session whether they were assigned to psilocybin or vitamin B3. Awareness of their first treatment assignment further unblinded them to the next phase of the trial. Patients who received psilocybin in the first session (and clinical and research staff assigned to them) knew that they were going to get. Because psilocybin was expected to have lasting effects, it would be predicted that patients assigned to the vitamin B3 in the second session would nonetheless exhibit the lowered self-reported symptoms, due to the effects of earlier psilocybin. So what was the point?

The crossover design means that any expected natural decline in distress will be folded into effect of treatment. That is, it’s well-known that the distress of cancer patients has a strong natural declining trajectory. That natural decline in distress will be interpreted as part of the effects of the treatment.

For each of the six primary outcome measures (HADS T, HADS A, HADS D, BDI, STAI S, STAI T), there were significant differences between the experimental and control groups (prior to the crossover at 7 weeks post-dose 1) with the psilocybin group (compared to the active control) demonstrating immediate, substantial, and sustained (up to 7 weeks post-dosing) clinical benefits in terms of reduction of anxiety and depression symptoms. The magnitude of differences between the psilocybin and control groups (Cohen’s d effect sizes) was large across the primary outcome measures, assessed at 1 day/2 weeks/6 weeks/7 weeks post-dose 1.

And:

Compared to the control, psilocybin produced mystical-type experiences, consistent with prior trials of psilocybin administration in normal volunteers (Griffiths et al., 2006, 2008, 2011).

Conflict of interest

 The authors declared no conflict of interest. However they indicate that a private foundation advocating use of psilocybin funded the study and professionals associated with this foundation provided preliminary review of the manuscript before it was published.

The foundation’s involvement in the funding study in vetting of the manuscript does not necessarily invalidate results reported in the manuscript, but readers have the right to be informed of this potential conflict of interest.

Coming up next: the Johns Hopkins study and integrated discussion

 Undoubtedly, ingesting psilocybin in a setting in which expectations are well structured can be a positive experience. I can say that from experience. Yet it is criminalized in the United States to sell, purchase, or ingest psilocybin. The study seems clearly aimed at creating a loophole for cancer patients who engaged psychiatrists. What’s wrong with that? What’s wrong with anybody being able to obtain psilocybin of assured purity and consume it in a pleasant safe environment with a knowledgeable guide whom they trust?

Suppose the cancer patient was a skeptic like myself and did not want to submit to the mumble jumble psychotherapy offered in this trial. Particularly if they were experienced taking psychedelics, as almost half of the patients were in the study, shouldn’t they be able to go to a primary care physician and get a prescription for psilocybin of known doses and purity, and go home and take it with a trusted friend or two? What if someone did not have cancer, but like many patients in primary care waiting rooms, had distress elevated to the degree that participants in the study did. Should they be allowed to self-administer psilocybin?

It’s a wise idea to take psilocybin with a knowledgeable friend in a context conducive to a good experience. Why should somebody have to involve a psychiatrist? People experience was psilocybin are aware in someone else’s experience is becoming anxiety provoking, and can usually appropriately distract them away and into a more pleasant experience.

It’s a wise idea not to go scuba diving alone, but to be accompanied by a knowledgeable friend. Should scuba divers also be required to take their psychiatrists along?

Okay, you as a reader may personally have no interest in taking psilocybin out of the presence of a psychiatrist anymore than having any interest in scuba diving, with or without a psychiatrist. You may even object to people doing so and considerate it foolish. But should the people, whether suffering from cancer or not, taking psilocybin at home with friends be subject to steep fines and jail time

We will next discuss the other study of psilocybin administered to cancer patients. We will also examine the larger issues of the commentaries that accompanied this set of articles and the press coverage that they were given.

In the meantime and afterwards, I certainly welcome back talk in comments on this blog.

 

 

 

 

 

 

 

 

Why PhD students should not evaluate a psychotherapy for their dissertation project

  • Things some clinical and health psychology students wish they had known before they committed themselves to evaluating a psychotherapy for their dissertation study.
  • A well designed pilot study addressing feasibility and acceptability issues in conducting and evaluating psychotherapies is preferable to an underpowered study which won’t provide a valid estimate of the efficacy of the intervention.
  • PhD students would often be better off as research parasites – making use of existing published data – rather than attempting to organize their own original psychotherapy study, if their goal is to contribute meaningfully to the literature and patient care.
  • Reading this blog, you will encounter a link to free, downloadable software that allows you to make quick determinations of the number of patients needed for an adequately powered psychotherapy trial.

I so relish the extra boost of enthusiasm that many clinical and health psychology students bring to their PhD projects. They not only want to complete a thesis of which they can be proud, they want their results to be directly applicable to improving the lives of their patients.

Many students are particularly excited about a new psychotherapy about which extravagant claims are being made that it’s better than its rivals.

I have seen lots of fad and fashions come and go, third wave, new wave, and no wave therapies. When I was a PhD student, progressive relaxation was in. Then it died, mainly because it was so boring for therapists who had to mechanically provide it. Client centered therapy was fading with doubts that anyone else could achieve the results of Carl Rogers or that his three facilitative conditions of unconditional positive regard, genuineness,  and congruence were actually distinguishable enough to study.  Gestalt therapy was supercool because of the charisma of Fritz Perls, who distracted us with his showmanship from the utter lack of evidence for its efficacy.

I hate to see PhD students demoralized when their grand plans prove unrealistic.  Inevitably, circumstances force them to compromise in ways that limit any usefulness to their project, and maybe even threaten their getting done within a reasonable time period. Overly ambitious plans are the formidable enemy of the completed dissertation.

The numbers are stacked against a PhD student conducting an adequately powered evaluation of a new psychotherapy.

This blog post argues against PhD students taking on the evaluation of a new therapy in comparison to an existing one, if they expect to complete their projects and make meaningful contribution to the literature and to patient care.

I’ll be drawing on some straightforward analysis done by Pim Cuijpers to identify what PhD students are up against when trying to demonstrate that any therapy is better than treatments that are already available.

Pim has literally done dozens of meta-analyses, mostly of treatments for depression and anxiety. He commands a particular credibility, given the quality of this work. The way Pim and his colleagues present a meta-analysis is so straightforward and transparent that you can readily examine the basis of what he says.

Disclosure: I collaborated with Pim and a group of other authors in conducting a meta-analysis as to whether psychotherapy was better than a pill placebo. We drew on all the trials allowing a head-to-head comparison, even though nobody ever really set out to pit the two conditions against each other as their first agenda.

Pim tells me that the brief and relatively obscure letter, New Psychotherapies for Mood and Anxiety Disorders: Necessary Innovation or Waste of Resources? on which I will draw is among his most unpopular pieces of work. Lots of people don’t like its inescapable message. But I think that if PhD students should pay attention, they might avoid a lot of pain and disappointment.

But first…

Note how many psychotherapies have been claimed to be effective for depression and anxiety. Anyone trying to make sense of this literature has to contend with claims being based on a lot of underpowered trials– too small in sample size to be expected reasonably to detect the effects that investigators claim – and that are otherwise compromised by methodological limitations.

Some investigators were simply naïve about clinical trial methodology and the difficulties doing research with clinical populations. They may have not understand statistical power.

But many psychotherapy studies end up in bad shape because the investigators were unrealistic about the feasibility of what they were undertaken and the low likelihood that they could recruit the patients in the numbers that they had planned in the time that they had allotted. After launching the trial, they had to change strategies for recruitment, maybe relax their selection criteria, or even change the treatment so it was less demanding of patients’ time. And they had to make difficult judgments about what features of the trial to drop when resources ran out.

Declaring a psychotherapy trial to be a “preliminary” or a “pilot study” after things go awry

The titles of more than a few articles reporting psychotherapy trials contain the apologetic qualifier after a colon: “a preliminary study” or “a pilot study”. But the studies weren’t intended at the outset to be preliminary or pilot studies. The investigators are making excuses post-hoc – after the fact – for not having been able to recruit sufficient numbers of patients and for having had to compromise their design from what they had originally planned. The best they can hope is that the paper will somehow be useful in promoting further research.

Too many studies from which effect sizes are entered into meta-analyses should have been left as pilot studies and not considered tests of the efficacy of treatments. The rampant problem in the psychotherapy literature is that almost no one treats small scale trials as mere pilot studies. In a recent blog post, I provided readers with some simple screening rules to identify meta-analyses of psychotherapy studies that they could dismiss from further consideration. One was whether there were sufficient numbers of adequately powered studies,  Often there are not.

Readers take their inflated claims of results of small studies seriously, when these estimates should be seen as unrealistic and unlikely to be replicated, given a study’s sample size. The large effect sizes that are claimed are likely the product of p-hacking and the confirmation bias required to get published. With enough alternative outcome variables to choose from and enough flexibility in analyzing and interpreting data, almost any intervention can be made to look good.

The problem is is readily seen in the extravagant claims about acceptance and commitment therapy (ACT), which are so heavily dependent on small, under-resourced studies supervised by promoters of ACT that should not have been used to generate effect sizes.

Back to Pim Cuijpers’ brief letter. He argues, based on his numerous meta-analyses, that it is unlikely that a new treatment will be substantially more effective than an existing credible, active treatment.  There are some exceptions like relaxation training versus cognitive behavior therapy for some anxiety disorders, but mostly only small differences of no more than d= .20 are found between two active, credible treatments. If you search the broader literature, you can find occasional exceptions like CBT versus psychoanalysis for bulimia, but most you find prove to be false positives, usually based on investigator bias in conducting and interpreting a small, underpowered study.

You can see this yourself using the freely downloadable G*power program and plug in d= 0.20 for calculating the number of patients needed for a study. To be safe, add more patients to allow for the expectable 25% dropout rate that has occurred across trials. The number you get would require a larger study than has ever been done in the past, including the well-financed NIMH Collaborative trial.

G power analyses

Even more patients would be needed for the ideal situation in which a third comparison group allowed  the investigator to show the active comparison treatment had actually performed better than a nonspecific treatment that was delivered with the same effectiveness that the other had shown in earlier trials. Otherwise, a defender of the established therapy might argue that the older treatment had not been properly implemented.

So, unless warned off, the PhD student plans a study to show not only that now hypothesis can be rejected that the new treatment is no better than the existing one, but that in the same study the existing treatment had been shown to be better than wait list. Oh my, just try to find an adequately powered, properly analyzed example of a comparison of two active treatments plus a control comparison group in the existing published literature. The few examples of three group designs in which a new psychotherapy had come out better than an effectively implemented existing treatment are grossly underpowered.

These calculations so far have all been based on what would be needed to reject the null hypothesis of no difference between the active treatment and a more established one. But if the claim is that the new treatment is superior to the existing treatment, our PhD student now needs to conduct a superiority trial in which some criteria is pre-set (such as greater than a moderate difference, d= .30) and the null hypothesis is that the advantage of the new treatment is less. We are now way out into the fantasyland of breakthrough, but uncompleted dissertation studies.

Two take away messages

 The first take away message is that we should be skeptical of claims of the new treatment is better than past ones except when the claim occurs in a well-designed study with some assurance that it is free of investigator bias. But the claim also has to arise in a trial that is larger than almost any psychotherapy study is ever been done. Yup, most comparative psychotherapy studies are underpowered and we cannot expect robust claims are robust that one treatment is superior to another.

But for PhD students been doing a dissertation project, the second take away message is that they should not attempt to show that one treatment is superior to another in the absence of resources they probably don’t have.

The psychotherapy literature does not need another study with too few patients to support its likely exaggerated claims.

An argument can be made that it is unfair and even unethical to enroll patients in a psychotherapy RCT with insufficient sample size. Some of the patients will be randomized to the control condition that is not what attracted them to the trial. All of the patients will be denied having been in a trial makes a meaningful contribution to the literature and to better care for patients like themselves.

What should the clinical or health psychology PhD student do, besides maybe curb their enthusiasm? One opportunity to make meaningful contributions to literature by is by conducting small studies testing hypotheses that can lead to improvement in the feasibility or acceptability of treatments to be tested in studies with more resources.

Think of what would’ve been accomplished if PhD students had determined in modest studies that it is tough to recruit and retain patients in an Internet therapy study without some communication to the patients that they are involved in a human relationship – without them having what Pim Cuijpers calls supportive accountability. Patients may stay involved with the Internet treatment when it proves frustrating only because they have the support and accountability to someone beyond their encounter with an impersonal computer. Somewhere out there, there is a human being who supports them and sticking it out with the Internet psychotherapy and will be disappointed if they don’t.

A lot of resources have been wasted in Internet therapy studies in which patients have not been convinced that what they’re doing is meaningful and if they have the support of a human being. They drop out or fail to do diligently any homework expected of them.

Similarly, mindfulness studies are routinely being conducted without anyone establishing that patients actually practice mindfulness in everyday life or what they would need to do so more consistently. The assumption is that patients assigned to the mindfulness diligently practice mindfulness daily. A PhD student could make a valuable contribution to the literature by examining the rates of patients actually practicing mindfulness when the been assigned to it in a psychotherapy study, along with barriers and facilitators of them doing so. A discovery that the patients are not consistently practicing mindfulness might explain weaker findings than anticipated. One could even suggest that any apparent effects of practicing mindfulness were actually nonspecific, getting all caught up in the enthusiasm of being offered a treatment that has been sought, but not actually practicing mindfulness.

An unintended example: How not to recruit cancer patients for a psychological intervention trial

Randomized-controlled-trials-designsSometimes PhD students just can’t be dissuaded from undertaking an evaluation of a psychotherapy. I was a member of a PhD committee of a student who at least produced a valuable paper concerning how not to recruit cancer patients for a trial evaluating problem-solving therapy, even though the project fell far short of conducting an adequately powered study.

The PhD student was aware that  claims of effectiveness of problem-solving therapy reported in in the prestigious Journal of Consulting and Clinical Psychology were exaggerated. The developer of problem-solving therapy for cancer patients (and current JCCP Editor) claimed  a huge effect size – 3.8 if only the patient were involved in treatment and an even better 4.4 if the patient had an opportunity to involve a relative or friend as well. Effect sizes for this trial has subsequently had to be excluded from at least meta-analyses as an extreme outlier (1,2,3,4).

The student adopted the much more conservative assumption that a moderate effect size of .6 would be obtained in comparison with a waitlist control. You can use G*Power to see that 50 patients would be needed per group, 60 if allowance is made for dropouts.

Such a basically inert control group, of course, has a greater likelihood of seeming to demonstrate a treatment is effective than when the comparison is another active treatment. Of course, such a control group also has the problem of not allowing a determination if it was the active ingredient of the treatment that made the difference, or just the attention, positive expectations, and support that were not available in the waitlist control group.

But PhD students should have the same option as their advisors to contribute another comparison between an active treatment and a waitlist control to the literature, even if it does not advance our knowledge of psychotherapy. They can take the same low road to a successful career that so many others have traveled.

This particular student was determined to make a different contribution to the literature. Notoriously, studies of psychotherapy with cancer patients often fail to recruit samples that are distressed enough to register any effect. The typical breast cancer patient, for instance, who seeks to enroll in a psychotherapy or support group trial does not have clinically significant distress. The prevalence of positive effects claimed in the literature for interventions with cancer patients in published studies likely represents a confirmation bias.

The student wanted to address this issue by limiting patients whom she enrolled in the study to those with clinically significant distress. Enlisting colleagues, she set up screening of consecutive cancer patients in oncology units of local hospitals. Patients were first screened for self-reported distress, and, if they were distressed, whether they were interested in services. Those who met both criteria were then re-contacted to see if that be willing to participate in a psychological intervention study, without the intervention being identified. As I reported in the previous blog post:

  • Combining results of  the two screenings, 423 of 970 patients reported distress, of whom 215 patients indicated need for services.
  • Only 36 (4% of 970) patients consented to trial participation.
  • We calculated that 27 patients needed to be screened to recruit a single patient, with 17 hours of time required for each patient recruited.
  • 41% (n= 87) of 215 distressed patients with a need for services indicated that they had no need for psychosocial services, mainly because they felt better or thought that their problems would disappear naturally.
  • Finally, 36 patients were eligible and willing to be randomized, representing 17% of 215 distressed patients with a need for services.
  • This represents 8% of all 423 distressed patients, and 4% of 970 screened patients.

So, the PhD student’s heroic effort did not yield the sample size that she anticipated. But she ended up making a valuable contribution to the literature that challenges some of the basic assumptions that were being made about how cancer patients in psychotherapy research- that all or most were distressed. She also ended up producing some valuable evidence that the minority of cancer patients who report psychological distress are not necessarily interested in psychological interventions.

Fortunately, she had been prepared to collect systematic data about these research questions, not just scramble within a collapsing effort at a clinical trial.

Becoming a research parasite as an alternative to PhD students attempting an under-resourced study of their own

research parasite awardPsychotherapy trials represent an enormous investment of resources, not only the public funding that is often provided for them,be a research parasite but in the time, inconvenience, and exposure to ineffective treatments experienced by patients who participate in the trials. Increasingly, funding agencies require that investigators who get money to do a psychotherapy study some point make their data available for others to use.  The 14 prestigious medical journals whose editors make up the International Committee of Medical Journal Editors (ICMJE) each published in earlier in 2016 a declaration that:

there is an ethical obligation to responsibly share data generated by interventional clinical trials because participants have put themselves at risk.

These statements proposed that as a condition for publishing a clinical trial, investigators would be required to share with others appropriately de-identified data not later than six months after publication. Further, the statements proposed that investigators describe their plans for sharing data in the registration of trials.

Of course, a proposal is only exactly that, a proposal, and these requirements were intended to take effect only after the document is circulated and ratified. The incomplete and inconsistent adoption of previous proposals for registering of  trials in advance and investigators making declarations of conflicts of interest do not encourage a lot of enthusiasm that we will see uniform implementation of this bold proposal anytime soon.

Some editors of medical journals are already expressing alarmover the prospect of data sharing becoming required. The editors of New England Journal of Medicine were lambasted in social media for their raising worries about “research parasites”  exploiting the availability of data:

a new class of research person will emerge — people who had nothing to do with the design and execution of the study but use another group’s data for their own ends, possibly stealing from the research productivity planned by the data gatherers, or even use the data to try to disprove what the original investigators had posited. There is concern among some front-line researchers that the system will be taken over by what some researchers have characterized as “research parasites.”

 Richard Lehman’s  Journal Review at the BMJ ‘s blog delivered a brilliant sarcastic response to these concerns that concludes:

I think we need all the data parasites we can get, as well as symbionts and all sorts of other creatures which this ill-chosen metaphor can’t encompass. What this piece really shows, in my opinion, is how far the authors are from understanding and supporting the true opportunities of clinical data sharing.

However, lost in all the outrage that The New England Journal of Medicine editorial generated was a more conciliatory proposal at the end:

How would data sharing work best? We think it should happen symbiotically, not parasitically. Start with a novel idea, one that is not an obvious extension of the reported work. Second, identify potential collaborators whose collected data may be useful in assessing the hypothesis and propose a collaboration. Third, work together to test the new hypothesis. Fourth, report the new findings with relevant coauthorship to acknowledge both the group that proposed the new idea and the investigative group that accrued the data that allowed it to be tested. What is learned may be beautiful even when seen from close up.

The PLOS family of journals has gone on record as requiring that all data for papers published in their journals be publicly available without restriction.A February 24, 2014 PLOS’ New Data Policy: Public Access to Data  declared:

In an effort to increase access to this data, we are now revising our data-sharing policy for all PLOS journals: authors must make all data publicly available, without restriction, immediately upon publication of the article. Beginning March 3rd, 2014, all authors who submit to a PLOS journal will be asked to provide a Data Availability Statement, describing where and how others can access each dataset that underlies the findings. This Data Availability Statement will be published on the first page of each article.

Many of us are aware of the difficulties in achieving this lofty goal. I am holding my breath and turning blue, waiting for some specific data.

The BMJ has expanded their previous requirements for data being available:

Loder E, Groves T. The BMJ requires data sharing on request for all trials. BMJ. 2015 May 7;350:h2373.

The movement to make data from clinical trials widely accessible has achieved enormous success, and it is now time for medical journals to play their part. From 1 July The BMJ will extend its requirements for data sharing to apply to all submitted clinical trials, not just those that test drugs or devices. The data transparency revolution is gathering pace.

I am no longer heading dissertation committees after one that I am currently supervising is completed. But if any PhD students asked my advice about a dissertation project concerning psychotherapy, I would strongly encourage them to enlist their advisor to identify and help them negotiate access to a data set appropriate to the research questions they want to investigate.

Most well-resourced psychotherapy trials have unpublished data concerning how they were implemented, with what bias and with which patient groups ending up underrepresented or inadequately exposed to the intensity of treatment presumed to be needed for benefit. A story awaits to be told. The data available from a published trial are usually much more adequate than then any graduate student could collect with the limited resources available for a dissertation project.

I look forward to the day when such data is put into a repository where anyone can access it.

until youre done In this blog post I have argued that PhD students should not take on responsibility for developing and testing a new psychotherapy for their dissertation project. I think that using data from existing published trials is a much better alternative. However, PhD students may currently find it difficult, but certainly not impossible to get appropriate data sets. I certainly am not recruiting them to be front-line infantry in advancing the cause of routine data sharing. But they can make an effort to obtain such data and they deserve all support they can get from their dissertation committees in obtaining data sets and in recognizing when realistically that data are not being made available, even when the data have been promised to be available as a condition for publishing. Advisors, please request the data from published trials for your PhD students and protect them from the heartache of trying to collect such data themselves.

 

Relaxing vs Stimulating Acupressure for Fatigue Among Breast Cancer Patients: Lessons to be Learned

  • A chance to test your rules of thumb for quickly evaluating clinical trials of alternative or integrative  medicine in prestigious journals.
  • A chance to increase your understanding of the importance of  well-defined control groups and blinding in evaluating the risk of bias of clinical trials.
  • A chance to understand the difference between merely evidence-based treatments versus science-based treatments.
  • Lessons learned can be readily applied to many wasteful evaluations of psychotherapy with shared characteristics.

A press release from the University of Michigan about a study of acupressure for fatigue in cancer patients was churnaled  – echoed – throughout the media. It was reproduced dozens of times, with little more than an editor’s title change from one report to the next.

Fortunately, the article that inspired all the fuss was freely available from the prestigious JAMA: Oncology. But when I gained access, I quickly saw that it was not worth my attention, based on what I already knew or, as I often say, my prior probabilities. Rules of thumb is a good enough term.

So the article became another occasion for us to practice our critical appraisal skills, including, importantly, being able to make reliable and valid judgments that some attention in the media is worth dismissing out of hand, even when tied to an article in a prestigious medical journal.

The press release is here: Acupressure reduced fatigue in breast cancer survivors: Relaxing acupressure improved sleep, quality of life.

A sampling of the coverage:

sample coverage

As we’ve come to expect, the UK Daily Mail editor added its own bit of spin:

daily mailHere is the article:

Zick SM, Sen A, Wyatt GK, Murphy SL, Arnedt J, Harris RE. Investigation of 2 Types of Self-administered Acupressure for Persistent Cancer-Related Fatigue in Breast Cancer Survivors: A Randomized Clinical Trial. JAMA Oncol. Published online July 07, 2016. doi:10.1001/jamaoncol.2016.1867.

Here is the Trial registration:

All I needed to know was contained in a succinct summary at the Journal website:

key points

This is a randomized clinical trial (RCT) in which two active treatments that

  • Lacked credible scientific mechanisms
  • Were predictably shown to be better than
  • A routine care that lacked the positive expectations and support.
  • A primary outcome assessed by  subjectiveself-report amplified the illusory effectiveness of the treatments.

But wait!

The original research appeared in a prestigious peer-reviewed journal published by the American Medical Association, not a  disreputable journal on Beall’s List of Predatory Publishers.

Maybe  this means publication in a peer-reviewed prestigious journal is insufficient to erase our doubts about the validity of claims.

The original research was performed with a $2.65 million peer-reviewed grant from the National Cancer Institute.

Maybe NIH is wasting scarce money on useless research.

What is acupressure?

 According to the article

Acupressure, a method derived from traditional Chinese medicine (TCM), is a treatment in which pressure is applied with fingers, thumbs, or a device to acupoints on the body. Acupressure has shown promise for treating fatigue in patients with cancer,23 and in a study24 of 43 cancer survivors with persistent fatigue, our group found that acupressure decreased fatigue by approximately 45% to 70%. Furthermore, acupressure points termed relaxing (for their use in TCM to treat insomnia) were significantly better at improving fatigue than another distinct set of acupressure points termed stimulating (used in TCM to increase energy).24 Despite such promise, only 5 small studies24– 28 have examined the effect of acupressure for cancer fatigue.

290px-Acupuncture_point_Hegu_(LI_4)You can learn more about acupressure here. It is a derivative of acupuncture, that does not involve needles, but the same acupuncture pressure points or acupoints as acupuncture.

Don’t be fooled by references to traditional Chinese medicine (TCM) as a basis for claiming a scientific mechanism.

See Chairman Mao Invented Traditional Chinese Medicine.

Chairman Mao is quoted as saying “Even though I believe we should promote Chinese medicine, I personally do not believe in it. I don’t take Chinese medicine.”

 

Alan Levinovitz, author of the Slate article further argues:

 

In truth, skepticism, empiricism, and logic are not uniquely Western, and we should feel free to apply them to Chinese medicine.

After all, that’s what Wang Qingren did during the Qing Dynasty when he wrote Correcting the Errors of Medical Literature. Wang’s work on the book began in 1797, when an epidemic broke out in his town and killed hundreds of children. The children were buried in shallow graves in a public cemetery, allowing stray dogs to dig them up and devour them, a custom thought to protect the next child in the family from premature death. On daily walks past the graveyard, Wang systematically studied the anatomy of the children’s corpses, discovering significant differences between what he saw and the content of Chinese classics.

And nearly 2,000 years ago, the philosopher Wang Chong mounted a devastating (and hilarious) critique of yin-yang five phases theory: “The horse is connected with wu (fire), the rat with zi (water). If water really conquers fire, [it would be much more convincing if] rats normally attacked horses and drove them away. Then the cock is connected with ya (metal) and the hare with mao (wood). If metal really conquers wood, why do cocks not devour hares?” (The translation of Wang Chong and the account of Wang Qingren come from Paul Unschuld’s Medicine in China: A History of Ideas.)

Trial design

A 10-week randomized, single-blind trial comparing self-administered relaxing acupressure with stimulating acupressure once daily for 6 weeks vs usual care with a 4-week follow-up was conducted. There were 5 research visits: at screening, baseline, 3 weeks, 6 weeks (end of treatment), and 10 weeks (end of washout phase). The Pittsburgh Sleep Quality Index (PSQI) and Long-Term Quality of Life Instrument (LTQL) were administered at baseline and weeks 6 and 10. The Brief Fatigue Inventory (BFI) score was collected at baseline and weeks 1 through 10.

Note that the trial was “single-blind.” It compared two forms of acupressure, relaxing versus stimulating. Only the patient was blinded to which of these two treatments was being provided, except patients clearly knew whether or not they were randomized to usual care. The providers were not blinded and were carefully supervised by the investigators and provided feedback on their performance.

The combination of providers not being blinded, patients knowing whether they were randomized to routine care, and subjective self-report outcomes together are the makings of a highly biased trial.

Interventions

Usual care was defined as any treatment women were receiving from health care professionals for fatigue. At baseline, women were taught to self-administer acupressure by a trained acupressure educator.29 The 13 acupressure educators were taught by one of the study’s principal investigators (R.E.H.), an acupuncturist with National Certification Commission for Acupuncture and Oriental Medicine training. This training included a 30-minute session in which educators were taught point location, stimulation techniques, and pressure intensity.

Relaxing acupressure points consisted of yin tang, anmian, heart 7, spleen 6, and liver 3. Four acupoints were performed bilaterally, with yin tang done centrally. Stimulating acupressure points consisted of du 20, conception vessel 6, large intestine 4, stomach 36, spleen 6, and kidney 3. Points were administered bilaterally except for du 20 and conception vessel 6, which were done centrally (eFigure in Supplement 2). Women were told to perform acupressure once per day and to stimulate each point in a circular motion for 3 minutes.

Note that the control/comparison condition was an ill-defined usual care in which it is not clear that patients received any attention and support for their fatigue. As I have discussed before, we need to ask just what was being controlled by this condition. There is no evidence presented that patients had similar positive expectations and felt similar support in this condition to what was provided in the two active treatment conditions. There is no evidence of equivalence of time with a provider devoted exclusively to the patients’ fatigue. Unlike patients assigned to usual care, patients assigned to one of the acupressure conditions received a ritual delivered with enthusiasm by a supervised educator.

Note the absurdity of the  naming of the acupressure points,  for which the authority of traditional Chinese medicine is invoked, not evidence. This absurdity is reinforced by a look at a diagram of acupressure points provided as a supplement to the article.

relaxation acupuncture pointsstimulation acupressure points

 

Among the many problems with “acupuncture pressure points” is that sham stimulation generally works as well as actual stimulation, especially when the sham is delivered with appropriate blinding of both providers and patients. Another is that targeting places of the body that are not defined as acupuncture pressure points can produce the same results. For more elaborate discussion see Can we finally just say that acupuncture is nothing more than an elaborate placebo?

 Worth looking back at credible placebo versus weak control condition

In a recent blog post   I discussed an unusual study in the New England Journal of Medicine  that compared an established active treatment for asthma to two credible control conditions, one, an inert spray that was indistinguishable from the active treatment and the other, acupuncture. Additionally, the study involved a no-treatment control. For subjective self-report outcomes, the active treatment, the inert spray and acupuncture were indistinguishable, but all were superior to the no treatment control condition. However, for the objective outcome measure, the active treatment was more effective than all of the three comparison conditions. The message is that credible placebo control conditions are superior to control conditions lacking and positive expectations, including no treatment and, I would argue, ill-defined usual care that lacks positive expectations. A further message is ‘beware of relying on subjective self-report measures to distinguish between active treatments and placebo control conditions’.

Results

At week 6, the change in BFI score from baseline was significantly greater in relaxing acupressure and stimulating acupressure compared with usual care (mean [SD], −2.6 [1.5] for relaxing acupressure, −2.0 [1.5] for stimulating acupressure, and −1.1 [1.6] for usual care; P < .001 for both acupressure arms vs usual care), and there was no significant difference between acupressure arms (P  = .29). At week 10, the change in BFI score from baseline was greater in relaxing acupressure and stimulating acupressure compared with usual care (mean [SD], −2.3 [1.4] for relaxing acupressure, −2.0 [1.5] for stimulating acupressure, and −1.0 [1.5] for usual care; P < .001 for both acupressure arms vs usual care), and there was no significant difference between acupressure arms (P > .99) (Figure 2). The mean percentage fatigue reductions at 6 weeks were 34%, 27%, and −1% in relaxing acupressure, stimulating acupressure, and usual care, respectively.

These are entirely expectable results. Nothing new was learned in this study.

The bottom line for this study is that there was absolutely nothing to be gained by comparing an inert placebo condition to another inert placebo condition to an uninformative condition without clear evidence the control condition offered control of nonspecific factors – positive expectations, support, and attention. This was a waste of patient time and effort, as well as government funds, and produced results that were potentially misleading to patients. Namely, results are likely to be misinterpreted the acupressure is an effective, evidence-based treatment for cancer-related fatigue.

How the authors explained their results

Why might both acupressure arms significantly improve fatigue? In our group’s previous work, we had seen that cancer fatigue may arise through multiple distinct mechanisms.15 Similarly, it is also known in the acupuncture literature that true and sham acupuncture can improve symptoms equally, but they appear to work via different mechanisms.40 Therefore, relaxing acupressure and stimulating acupressure could elicit improvements in symptoms through distinct mechanisms, including both specific and nonspecific effects. These results are also consistent with TCM theory for these 2 acupoint formulas, whereby the relaxing acupressure acupoints were selected to treat insomnia by providing more restorative sleep and improving fatigue and the stimulating acupressure acupoints were chosen to improve daytime activity levels by targeting alertness.

How could acupressure lead to improvements in fatigue? The etiology of persistent fatigue in cancer survivors is related to elevations in brain glutamate levels, as well as total creatine levels in the insula.15 Studies in acupuncture research have demonstrated that brain physiology,41 chemistry,42 and function43 can also be altered with acupoint stimulation. We posit that self-administered acupressure may have similar effects.

Among the fallacies of the authors’ explanation is the key assumption that they are dealing with a specific, active treatment effect rather than a nonspecific placebo intervention. Supposed differences between relaxing versus stimulating acupressure arise in trials with a high risk of bias due to unblinded providers of treatment and inadequate control/comparison conditions. ‘There is no there there’ to be explained, to paraphrase a quote attributed to Gertrude Stein

How much did this project cost?

 According to the NIH Research Portfolios Online Reporting Tools website, this five-year project involved support by the federal government of $2,265,212 in direct and indirect costs. The NCI program officer for investigator-initiated  R01CA151445 is Ann O’Marawho serves ina similar role for a number of integrative medicine projects.

How can expenditure of this money be justified for determining whether so-called stimulating acupressure is better than relaxing acupressure for cancer-related fatigue?

 Consider what could otherwise have been done with these monies.

 Evidence-based versus science based medicine

Proponents of unproven “integrative cancer treatments” can claim on the basis of the study the acupressure is an evidence-based treatment. Future Cochrane Collaboration Reviews may even cite this study as evidence for this conclusion.

I normally label myself as an evidence-based skeptic. I require evidence for claims of the efficacy of treatments and am skeptical of the quality of the evidence that is typically provided, especially when it comes from enthusiasts of particular treatments. However, in other contexts, I describe myself as a science based medicine skeptic. The stricter criteria for this term is that not only do I require evidence of efficacy for treatments, I require evidence for the plausibility of the science-based claims of mechanism. Acupressure might be defined by some as an evidence-based treatment, but it is certainly not a science-based treatment.

For further discussion of this important distinction, see Why “Science”-Based Instead of “Evidence”-Based?

Broader relevance to psychotherapy research

The efficacy of psychotherapy is often overestimated because of overreliance on RCTs that involve inadequate comparison/control groups. Adequately powered studies of the comparative efficacy of psychotherapy that include active comparison/control groups are infrequent and uniformly provide lower estimates of just how efficacious psychotherapy is. Most psychotherapy research includes subjective patient self-report measures as the primary outcomes, although some RCTs provide independent, blinded interview measures. A dependence on subjective patient self-report measures amplifies the bias associated with inadequate comparison/control groups.

I have raised these issues with respect to mindfulness-based stress reduction (MBSR) for physical health problems  and for prevention of relapse in recurrence in patients being tapered from antidepressants .

However, there is a broader relevance to trials of psychotherapy provided to medically ill patients with a comparison/control condition that is inadequate in terms of positive expectations and support, along with a reliance on subjective patient self-report outcomes. The relevance is particularly important to note for conditions in which objective measures are appropriate, but not obtained, or obtained but suppressed in reports of the trial in the literature.