This classic paper examined the impact of three types of victimization in childhood sexual abuse, physical abuse, and neglect on lifetime measures of mental health among adults.
It replicated some past findings obtained with checklists:
Men who were abused and neglected as children have more dysthymia and antisocial personality disorder as adults than matched controls, but they did not have more alcohol problems. Abused and neglected women report more symptoms of dysthymia, antisocial personality disorder, and alcohol problems than controls.
However, unlike past studies, this study introduced controls for subsequent life events:
After controlling for stressful life events, however, childhood victimization had little direct impact on any lifetime mental health outcome.
The study also matched adults with objective records of abuse and neglect in their childhood with adults who had similar backgrounds but without such abuse and neglect.
Mental health outcomes were similar between these matched groups.
What else was different:
Rather than a respondent-completed checklist, objective data were obtained.
The study used a prospective sample gathered from records of documented court cases of childhood abuse and neglect in a Midwestern city around 1970.
These subjects then interviewed about twenty years later to assess adult mental health.
The study was unusual in having a matched control group.
Compared outcomes of the 641 members of the abuse and neglect group with a matched control group of 510 persons who did not have documented cases of abuse or neglect.
Unlike past research, the study introduced controls for subsequent life events.
The study was important because it indicated:
The importance of adopting an approach that places childhood victimization in the context of other life stressors and of prospective changes over the life course.
The study addressed two major theoretical shortcomings in past literature.
In past research, victimization had been considered as an isolated event with little regard to the context in which it occurs.
As Briere (1992) notes “there has been a tendency for investigators to examine sexual abuse in a relative vacuum” (p. 199).
The study instead adopted a life course, developmental perspective.
Childhood victimization is typically part of a matrix of environmental problems such as poverty, unemployment, parental alcohol and drug problems, and inadequate family functioning…The unique contribution of childhood victimization to later symptomatology, after taking into account conditions such as family disruptions and stressors, persistent poverty, and broader patterns of social deprivation, is not well understood.
The study started with the recognition of the theoretical deficiency of most studies.
Past research assumed a simple causal direction leading from childhood abuse and neglect to mental health outcomes in later life.
Instead, this study made the basic assumption:
The influence of childhood experiences is contingent upon historical changes across the lives of affected individuals…Traumatic events that occur in early stages of the life course are unlikely to have uniform and straightforward mental health impacts in later stages of life, regardless of subsequent social conditions.
[Any] lasting impact on adults of childhood experiences depends on later factors such as the strength of the marriages and other social relationships, educational and occupational attainment, and the adequacy of family functioning.
This study was intended to contribute some knowledge of these later factors.
We know very little, however, about how subsequent stressors in life trajectories shaped later mental health consequences of childhood victimization (Bifulco, Brown, and Adler, 1991).
The study addressed important methodological limitations of past research:
Most past research depend upon adult responses to questions about their experience of abuse events as children.
Retrospective reporting should not have a major impact on the recall of screening and objective events such as loss of a parent or divorce.
Abuse events in early life are not encoded in memory as objective occurrences, but recollections of what constitutes abuse and experienced in the past change in light of later events definitions of abuse.
Most research about the traumatic impact of childhood abuse is not only retrospective but is also cross-sectional. Because studies obtain measures of prior abuse events in current states of mental health at the same time, present states of mental health may influence people’s recollections of recurrence has dramatic events (Brown Harris 1978).
In general, people with poor mental health have a bleaker of the world, including their prior life course, those with with higher psychological well-being (Coyne 1976; Beck et al, 1979; Burbach and Bourdin 1986).
Uniformly high correlation between later mental health and past child abuse might impart results from factors having to do with the past and the present, thus blurring the temporal order between recall of childhood events and subsequent psychological outcomes. Prospective studies, however, can overcome some of the problems stem from retrospective reports of childhood adversity (Kessler et al. 1997).
The study addressed the inadequate samples in most past studies:
Most studies used samples of college students, patients in clinical treatment, or responses to newspaper advertisements.
Few included adequate control groups of equivalent but non-victimized children. The high rates of psychopathology among adults who were abused as children are only meaningful when these rates are higher than comparable, non-abuse groups.
Shared characteristics between victimized and unvictimized children from similar backgrounds might explain the poorer mental health usually attributed to victimization.
Third factors may be influential.
Another factor such as disadvantaged social economic circumstances or family adversities may lead to both abuse in childhood and to poor adult mental health. The lack of control groups of non-abuse children comparable backgrounds precludes establishing the effects of abuse, as opposed to the impact of the matrix of social economic disadvantage within which abuse may occur on later states of mental health.
Findings showed both men and women who were victimized as children report more stressful life events over their lifetimes than a control sample, suggesting that early childhood abuse and neglect is part of a broader constellation of life stressors.
In addition, abused and neglected males and females are more likely than controls to have grown up in families receiving welfare, possibly indicating the officially reported childhood victimization co-occurs with particularly disadvantaged circumstances. Find there are no racial or age differences between the abused and neglected and control groups.
As a life course literature emphasizes, these results indicate the experiences occurring early in life do not have uniform consequences for mental health outcomes in later life (Harris, Brown, and Bifulco 1990, Elder at L 1996). Instead, the influence of these early childhood experiences vary depending on what happened in subsequent stages of the life course. In particular, stressful life events that occur later in the life course influence how much effect child victimization will have on subsequent outcomes. When childhood victims of abuse and neglect did not experience more stressors than controls, they do not have worse mental health outcomes (alcohol problems, dysthymia, or antisocial personality disorder, as adults. Thus, not only do early childhood events affect the life experiences, but these later experiences also affect how consequential these earlier events will be the subsequent health.
Limitations of this study need to be addressed in the future.
Although the results of this study clearly indicate the subsequent impact of childhood victimization on the mental health of adults grounded in a broader context and the life course trajectories, they do not specify the causal links among childhood victimization, lifetime stressors, and subsequent mental health.
One possible interpretation of these findings is the childhood victimization produces poor mental health outcomes among children. Poor early mental health could elevate the risk of experiencing subsequent life events such as getting fired from jobs, unemployment, and divorce, which in turn strongly relate for mental health among adults.
Another possible interpretation of our findings is the childhood victimization does not precede life stressors but is the correlate of other lifetime stressors such as family isolation and disorganization… [If so, pre-existing familial context can be critical for both the child and victimization for subsequent life stressors.
In addition, children from disorganized families might have weakened social support networks as adults, exacerbating their vulnerability to stressful life conditions.
Our findings only indicate a general relationship between childhood victimization, subsequent stressors, and mental health impacts.
Our findings do not allow the causal statements about the relationship among these factors.
It is important that future research specifies both the pathways through which childhood victimization elevates the risk of suffering subsequent stressors and the possible mediating buffering factors that protect some victims of child abuse and neglect from the adverse consequences in later stages of their lives.
A recipe for coercing ill people with positive psychology pseudoscience in the New York Times
Judging by the play she gets in social media and the 100s of comments on her articles in the New York Times, Jane Brody has a successful recipe for using positive psychology pseudoscience to bolster down-home advice you might’ve gotten from your grandmother.
Her recipe might seem harmless enough, but her articles are directed at people struggling with chronic and catastrophic physical illnesses. She offers them advice.
The message is that persons with physical illness should engage in self-discipline, practice positive psychology exercises – or else they are threatening their health and shortening their lives.
People struggling with physical illness have enough to do already. The admonition they individually and collectively should do more -they should become more self-disciplined- is condescending and presumptuous.
Jane Brody’s carrot is basically a stick. The implied threat is simply coercive: that people with chronic illness are not doing what they can to improve the physical health unless they engage in these exercises.
It takes a careful examination Jane Brody’s sources to discover that the “scientific basis” for this positive psychology advice is quite weak. In many instances it is patently junk, pseudoscience.
The health benefits claimed for positivity are unfounded.
People with chronic illness are often desperate or simply vulnerable to suggestions that they can and should do more. They are being misled by this kind of article in what is supposed to be the trusted source of a quality news outlet, The New York Times, not The Daily News.
There is a sneaky, ill-concealed message that persons with chronic illness will obtain wondrous benefits by just adopting a positive attitude – even a hint that cancer patients will live longer.
In my blog post about positive psychology and health, I try to provide tools so that consumers can probe for themselves the usually false and certainly exaggerated claims that are being showered on them.
However, in the case of Jane Brody’s articles, we will see that the task is difficult because she draws on a selective sampling of the literature in which researchers generate junk self-promotional claims.
That’s a general problem with the positive psychology “science” literature, but the solution for journalists like Jane Brody is to seek independent evaluation of claims from outside the positive psychology community. Journalists, did you hear that message?
The article, along with its 100s of comments from readers, is available here:
The article starts with some clichéd advice about being positive. Brody seems to be on the side of the autonomy of her readers. She makes seemingly derogatory comments that the advice is “cockeyed optimism” [Don’t you love that turn of phrase? I’m sure to borrow it in the future]
“Look on the sunny side of life.”
“Turn your face toward the sun, and the shadows will fall behind you.”
“Every day may not be good, but there is something good in every day.”
“See the glass as half-full, not half-empty.”
Researchers are finding that thoughts like these, the hallmarks of people sometimes called “cockeyed optimists,” can do far more than raise one’s spirits. They may actually improve health and extend life.
See? The clever putdown of this advice was just a rhetorical device, just a set up for what follows. Very soon Brody is delivering some coercive pseudoscientific advice, backed by the claim that “there is no longer any doubt” and that the links between positive thinking and health benefits are “indisputable.”
There is no longer any doubt that what happens in the brain influences what happens in the body. When facing a health crisis, actively cultivating positive emotions can boost the immune system and counter depression. Studies have shown an indisputable link between having a positive outlook and health benefits like lower blood pressure, less heart disease, better weight control [Emphasis added.].
I found the following passage particularly sneaky and undermining of people with cancer.
Even when faced with an incurable illness, positive feelings and thoughts can greatly improve one’s quality of life. Dr. Wendy Schlessel Harpham, a Dallas-based author of several books for people facing cancer, including “Happiness in a Storm,” was a practicing internist when she learned she had non-Hodgkin’s lymphoma, a cancer of the immune system, 27 years ago. During the next 15 years of treatments for eight relapses of her cancer, she set the stage for happiness and hope, she says, by such measures as surrounding herself with people who lift her spirits, keeping a daily gratitude journal, doing something good for someone else, and watching funny, uplifting movies. Her cancer has been in remission now for 12 years.
“Fostering positive emotions helped make my life the best it could be,” Dr. Harpham said. “They made the tough times easier, even though they didn’t make any difference in my cancer cells.”
Sure, Jane Brody is careful to avoid the explicit claim the positive attitude somehow is connected to the cancer being in remission for 12 years, but the implication is there. Brody pushes the advice with a hint of the transformation available to cancer patients, only if they follow the advice.
After all, Jane Brody had just earlier asserted that positive attitude affects the immune system and this well-chosen example happens to be a cancer of the immune system.
Jane Brody immediately launches into a description of a line of research conducted by a positive psychology group at Northwestern University and University of California San Francisco.
Taking her cue from the investigators, Brody blurs the distinction between findings based in correlational studies and the results of intervention studies in which patients actually practiced positive psychology exercises.
People with new diagnoses of H.I.V. infection who practiced these skills carried a lower load of the virus, were more likely to take their medication correctly, and were less likely to need antidepressants to help them cope with their illness.
But Brody sins as a journalist are worse than that. With a great deal of difficulty, I have chased her claims back into the literature. I found some made up facts.
In my literature search, I could find only one study from these investigators that seemed directly related to these claims. The mediocre retrospective correlational study was mainly focused on use of psychostimulants, but it included a crude 6-item summary measure of positive states of mind.
The authors didn’t present the results in a simple way that allows direct independent examination of whether indeed positive affect is related to other outcomes in any simple fashion. They did not allow check of simple correlations needed to determine whether their measure was not simply a measure of depressive symptoms turned on its head. They certainly had the data, but did not report it. Instead, they present some multivariate analyses that do not show impressive links. Any direct links to viral load are not shown and presumably are not there, although the investigators tested statistically for them. Technically speaking, I would write off the findings to measurement and specification error, certainly not worthy of reporting in The New York Times.
Less technically speaking, Brody is leading up to using HIV as an exemplar illness where cultivating positivity can do so much. But if this study is worth anything at all, it is to illustrate that even correlationally, positive affect is not related to much, other than – no surprise – alternative measures of positive affect.
Brody then goes on to describe in detail an intervention study. You’d never know from her description that her source of information is not a report of the results of the intervention study, but a promissory protocol that supposedly describes how the intervention study was going to be done.
I previously blogged about this protocol. At first, I thought it was praiseworthy that a study of a positive psychology intervention for health had even complied with the requirement that studies be preregistered and have a protocol available. Most such studies do not, but they are supposed to do that. In plain English, protocols are supposed to declare ahead of time what researchers are going to do and precisely how they are going to evaluate whether an intervention works. That is because, notoriously, researchers are inclined to say later they were really trying to do something else and to pick another outcome that makes the intervention look best.
But then I got corrected by James Heathers on Facebook. Duh, he had looked at the date the protocol was published.
He pointed out that this protocol was actually published years after collection of data had begun. The researchers already had a lot to peek at. Rather than identifying just a couple of variables on which the investigators were prepared to stake their claim the intervention was affected, the protocol listed 25 variables that would be examined as outcomes (!) in order to pick one or two.
So I updated what I said in my earlier blog. I pointed out that the published protocol was misleading. It was posted after the fact of the researchers being able to see how their study was unfolding and to change their plains accordingly. The vagueness of the protocol gave the authors lots of wiggle room for selectively reporting and hyping their findings with the confirmation bias. They would later take advantage of this when they actually published the results of their study.
The researchers studied 159 people who had recently learned they had H.I.V. and randomly assigned them to either a five-session positive emotions training course or five sessions of general support. Fifteen months past their H.I.V. diagnosis, those trained in the eight skills maintained higher levels of positive feelings and fewer negative thoughts related to their infection.
Brody is not being accurate here. When the authors finally got around to publishing the results, they told a very different story if you probe carefully. Even with the investigators doing a lot of spinning, they showed null results, no effects for the intervention. Appearances the contrary were created by the investigators ignoring what they actually reported in their tables. If you go to my earlier blog post, I point this out in detail, so you can see for yourself.
Brody goes on to describe the regimen that was not shown in the published study validation to be effective.
An important goal of the training is to help people feel happy, calm and satisfied in the midst of a health crisis. Improvements in their health and longevity are a bonus. Each participant is encouraged to learn at least three of the eight skills and practice one or more each day. The eight skills are:
■ Recognize a positive event each day.
■ Savor that event and log it in a journal or tell someone about it.
■ Start a daily gratitude journal.
■ List a personal strength and note how you used it.
■ Set an attainable goal and note your progress.
■ Report a relatively minor stress and list ways to reappraise the event positively.
■ Recognize and practice small acts of kindness daily.
■ Practice mindfulness, focusing on the here and now rather than the past or future.
For chrissakes, this is a warmed over version of Émile Coué de la Châtaigneraie’s autosuggestion “Every day in every way, I’m getting better and better. Surely, contemporary positive psychology’s science of health can do better than that. To Coué’s credit, he gave away his advice for free. He did not charge for his coaching, even if he was giving away something for which he had no evidence would improve people’s physical health.
Dr. Moskowitz said she was inspired by observations that people with AIDS, Type 2 diabetes and other chronic illnesses lived longer if they demonstrated positive emotions. She explained, “The next step was to see if teaching people skills that foster positive emotions can have an impact on how well they cope with stress and their physical health down the line.”
She listed as the goals improving patients’ quality of life, enhancing adherence to medication, fostering healthy behaviors, and building personal resources that result in increased social support and broader attention to the good things in life.
Let me explain why I am offended here. None of these activities have been shown to improve the health of persons with newly diagnosed HIV. It’s reasonable to assume that newly diagnosed persons have a lot with which to contend. It’s a bad time to give them advice to clutter their life with activities that will not make a difference in their health.
Many persons with newly diagnosed HIV are low income and are unemployed or marginally employed. They will enroll in studies to get the participant fees. When I lived in the San Francisco Bay area, I recall one patient telling a recruiter from UCSF that he was too busy and unable to make a regular visit to the medical center for the intervention, but he would be willing to accept being in the study if he was assigned to the control group. It did not involve attending intervention sessions and would give him a little cash.
Based on my clinical and research experience, I don’t believe that such patients would regularly show up for this kind of useless positive psychology treatment without getting paid. Paticularly if they were informed of the actual results of this misrepresented study.
Gregg De Meza, a 56-year-old architect in San Francisco who learned he was infected with H.I.V. four years ago, told me that learning “positivity” skills turned his life around. He said he felt “stupid and careless” about becoming infected and had initially kept his diagnosis a secret.
“When I entered the study, I felt like my entire world was completely unraveling,” he said. “The training reminded me to rely on my social network, and I decided to be honest with my friends. I realized that to show your real strength is to show your weakness. No pun intended, it made me more positive, more compassionate, and I’m now healthier than I’ve ever been.”
I object to this argument by quotes-from-an-unrepresentative-patient. The intervention did not have the intended effect, and it is misleading to find somebody who claim to turn their life around.
Jane Brody proceeds with some more fake facts.
In another study among 49 patients with Type 2 diabetes, an online version of the positive emotions skills training course was effective in enhancing positivity and reducing negative emotions and feelings of stress. Prior studies showed that, for people with diabetes, positive feelings were associated with better control of blood sugar, an increase in physical activity and healthy eating, less use of tobacco and a lower risk of dying.
The study was so small and underpowered, aside from being methodologically flawed, that even if such effects were actually present, most of the time they would be missed because the study did not have enough patients to achieve significance.
In a pilot study of 39 women with advanced breast cancer, Dr. Moskowitz said an online version of the skills training decreased depression among them. The same was true with caregivers of dementia patients.
“None of this is rocket science,” Dr. Moskowitz said. “I’m just putting these skills together and testing them in a scientific fashion.”
It’s not rocket science, it’s misleading hogwash.
In a related study of more than 4,000 people 50 and older published last year in the Journal of Gerontology, Becca Levy and Avni Bavishi at the Yale School of Public Health demonstrated that having a positive view of aging can have a beneficial influence on health outcomes and longevity. Dr. Levy said two possible mechanisms account for the findings. Psychologically, a positive view can enhance belief in one’s abilities, decrease perceived stress and foster healthful behaviors. Physiologically, people with positive views of aging had lower levels of C-reactive protein, a marker of stress-related inflammation associated with heart disease and other illnesses, even after accounting for possible influences like age, health status, sex, race and education than those with a negative outlook. They also lived significantly longer.
This is even deeper into the woo. Give me a break, Jane Brody. Stop misleading people with chronic illness with false claims and fake facts. Adopting these attitudes will not prevent dementia.
Here is what the original investigators claimed about Alzheimer’s:
We believe it is the stress generated by the negative beliefs about aging that individuals sometimes internalize from society that can result in pathological brain changes,” said Levy. “Although the findings are concerning, it is encouraging to realize that these negative beliefs about aging can be mitigated and positive beliefs about aging can be reinforced, so that the adverse impact is not inevitable.”
I exposed some analysis of voodoo statistics on which this claim is based. I concluded:
The authors develop their case that stress is a significant cause of Alzheimer’s disease with reference to some largely irrelevant studies by others, but depend on a preponderance of studies that they themselves have done with the same dubious small samples and dubious statistical techniques. Whether you do a casual search with Google scholar or a more systematic review of the literature, you won’t find stress processes of the kind the authors invoke among the usual explanations of the development of the disease.
Basically, the authors are arguing that if you hold views of aging like “Old people are absent-minded” or “Old people cannot concentrate well,” you will experience more stress as you age, and this will accelerate development of Alzheimer’s disease. They then go on to argue that because these attitudes are modifiable, you can take control of your risk for Alzheimer’s by adopting a more positive view of aging and aging people
Nonsense, utter nonsense.
Let chronically ill people and those facing cancer adopt any attitude is comfortable or natural for them. It’s a bad time to ask for change, particularly when there isn’t any promised benefit in improved health or prolonged life.
Rather than Jane Brody’s recipe for positive psychology improving your health, I strongly prefer Lilia Downe’s La Cumbia Del Mole.
It is great on chicken. If it does not extend your life, It will give you some moments of happiness, but you will have to adjust the spices to your personal taste.
Iwill soon be offering e-books providing skeptical looks at positive psychology, as well as mindfulness. As in this blog post, I will take claims I find in the media and trace them back to the scientific studies on which they are based. I will show you what I see so you can see it too.
Sign up at my new website to get advance notice of the forthcoming e-books and web courses, as well as upcoming blog posts at this and other blog sites. You can even advance order one or all of the e-books.
A seriously flawed overview “systematic review “ of systematic reviews and meta-analyses of the effects of mindfulness on health and well-being alerts readers how they need to be skeptical of what they are told about the benefits of mindfulness.
Especially when the information comes those benefiting enormously from promoting the practice.
The glowing evaluation of the benefits of mindfulness presented in a PLOS One review is contradicted by a more comprehensive and systematic review which was cited but summarily dismissed. As we will see, the PLOS One article sidesteps substantial confirmation bias and untrustworthiness in the mindfulness literature.
The review was prepared by authors associated with the Benson-Henry Institute for Mind-Body Medicine, which is tied to Massachusetts General Hospital and Harvard Medical School. The institute directly markets mindfulness treatment to patients and training to professionals and organizations. Its website provides links to research articles such as this one, which are used to market a wide range of programs –
Recently PLOS One published corrections to five articles from this group concerning previous statements about the authors having no conflicts of interest to declare. The corrections acknowledged extensive conflicts of interest.
The Competing Interests statement is incorrect. The correct Competing Interests statement is: The following authors hold or have held positions at the Benson-Henry Institute for Mind-Body Medicine at Massachusetts General Hospital, which is paid by patients and their insurers for running the SMART-3RP and related relaxation/mindfulness clinical programs, markets related products such as books, DVDs, CDs and the like, and holds a patent pending (PCT/US2012/049539 filed August 3, 2012) entitled “Quantitative Genomics of the Relaxation Response.”
While the review we will be discussing was not corrected, it should have been.
The same conflicts of interest should have been disclosed to readers evaluating the trustworthiness of what is being presented to them.
Probing this review will demonstrate just how hard it is to uncover the bias and distortions that routinely is provided by promoters of mindfulness wanting to demonstrate the evidence base for what they offer.
The evidence supports the use of MBSR and MBCT to alleviate symptoms, both mental and physical, in the adjunct treatment of cancer, cardiovascular disease, chronic pain, depression, anxiety disorders and in prevention in healthy adults and children.
This evaluation is more emphatically stated near the end of the article:
This review provides an overview of more trials than ever before and the intervention effect has thus been evaluated across a broad spectrum of target conditions, most of which are common chronic conditions. Study settings in many countries across the globe contributed to the analysis, further serving to increase the generalizability of the evidence. Beneficial effects were mostly seen in mental health outcomes: depression, anxiety, stress and quality of life improved significantly after training in MBSR or MBCT. These effects were seen both in patients with medical conditions and those with psychological disorders, compared with many types of control interventions (WL, TAU or AT). Further evidence for effectiveness was provided by the observed dose-response relationship: an increase in total minutes of practice and class attendance led to a larger reduction of stress and mood complaints in four reviews [18,20,37,54].
Are you impressed? “More than ever before”? “Generalizability of the evidence”? Really?
And in wrap up summary comments:
Although there is continued scepticism in the medical world towards MBSR and MBCT, the evidence indicates that MBSR and MBCT are associated with improvements in depressive symptoms, anxiety, stress, quality of life, and selected physical outcomes in the adjunct treatment of cancer, cardiovascular disease, chronic pain, chronic somatic diseases, depression, anxiety disorders, other mental disorders and in prevention in healthy adults and children.
Compare and contrast these conclusions with a more balanced and comprehensive review.
The US Agency for Healthcare Research and Quality (AHCRQ) commissioned a report from Johns Hopkins University Evidence-based Practice Center.
The 439 page report is publicly available:
Goyal M, Singh S, Sibinga EMS, Gould NF, Rowland-Seymour A, Sharma R, Berger Z, Sleicher D, Maron DD, Shihab HM, Ranasinghe PD, Linn S, Saha S, Bass EB, Haythornthwaite JA. Meditation Programs for Psychological Stress and Well-Being. Comparative Effectiveness Review No. 124. (Prepared by Johns Hopkins University Evidence-based Practice Center under Contract No. 290-2007-10061–I.) AHRQ Publication No. 13(14)-EHC116-EF. Rockville, MD: Agency for Healthcare Research and Quality; January 2014.
A companion, less detailed article was also published in JAMA: Internal Medicine:
Consider how conclusions of this article were characterized in the Bensen-Henry PLOS One article. The article is briefly mentioned without detailing its methods and conclusions.
Recently, Goyal et al. published a review of mindfulness interventions compared to active control and found significant improvements in depression and anxiety.
A recent review compared meditation to only active control groups, and although lower, also found a beneficial effect on depression, anxiety, stress and quality of life. This review was excluded in our study for its heterogeneity of interventions .
What the Goyal et JAMA: Internal Medicine actually said:
After reviewing 18 753 citations, we included 47 trials with 3515 participants. Mindfulness meditation programs had moderate evidence of improved anxiety (effect size, 0.38 [95% CI, 0.12-0.64] at 8 weeks and 0.22 [0.02-0.43] at 3-6 months), depression (0.30 [0.00-0.59] at 8 weeks and 0.23 [0.05-0.42] at 3-6 months), and pain (0.33 [0.03- 0.62]) and low evidence of improved stress/distress and mental health–related quality of life. We found low evidence of no effect or insufficient evidence of any effect of meditation programs on positive mood, attention, substance use, eating habits, sleep, and weight. We found no evidence that meditation programs were better than any active treatment (ie, drugs, exercise, and other behavioral therapies).
The review also notes that evidence of the effectiveness mindfulness interventions is largely limited to trials in which it is compared to no treatment, wait list, or a usually ill-defined treatment as usual (TAU).
In our comparative effectiveness analyses (Figure 1B), we found low evidence of no effect or insufficient evidence that any of the meditation programs were more effective than exercise, progressive muscle relaxation, cognitive-behavioral group therapy, or other specific comparators in changing any outcomes of interest. Few trials reported on potential harms of meditation programs. Of the 9 trials reporting this information, none reported any harms of the intervention.
This solid JAMA: Internal Medicine review explains why its conclusions may differ from past reviews:
Reviews to date report a small to moderate effect of mindfulness and mantra meditation techniques in reducing emotional symptoms (eg, anxiety, depression, and stress) and improving physical symptoms (eg, pain).7– 26 These reviews have largely included uncontrolled and controlled studies, and many of the controlled studies did not adequately control for placebo effects (eg, waiting list– or usual care–controlled studies). Observational studies have a high risk of bias owing to problems such as self-selection of interventions (people who believe in the benefits of meditation or who have prior experience with meditation are more likely to enroll in a meditation program and report that they benefited from one) and use of outcome measures that can be easily biased by participants’ beliefs in the benefits of meditation. Clinicians need to know whether meditation training has beneficial effects beyond self-selection biases and the nonspecific effects of time, attention, and expectations for improvement.27,28
Basically, this article insists that mindfulness be evaluated in a head-to- head comparison to an active treatment. Failure to provide such a comparison means not being able to rule out that apparent effects of mindfulness are nonspecific, i.e., not due to any active ingredient of the practice.
An accompanying editorial commentary raised troubling issues about the state of the mindfulness literature. It noted that limiting inclusion to RCTs with an active control condition and a patient population experiencing mental or physical health problems left only 3% (47/18,753 of the citations that had been retrieved. Furthermore:
The modest benefit found in the study by Goyal et al begs the question of why, in the absence of strong scientifically vetted evidence, meditation in particular and complementary measures in general have become so popular, especially among the influential and well educated…What role is being played by commercial interests? Are they taking advantage of the public’s anxieties to promote use of complementary measures that lack a base of scientific evidence? Do we need to require scientific evidence of efficacy and safety for these measures?
How did the Bensen-Henry review arrive at a more favorable assessment?
The issue that dominated the solid Goyal et al systematic review and meta analysis is not prominent in the Bensen-Henry review. The latter article hardly mentions the importance of whether mindfulness is compared to an active treatment. It doesn’t mention if any difference in effect size for mindfulness can be expected when the comparison is an active treatment.
The Bensen-Henry review stated that it excluded systematic reviews and meta analyses if they did not focus on MBCT or MBSR. One has to search the supplementary materials to find that Goyal et al was excluded because it did not calculate separate effect sizes for mindfulness-based stress reduction (MBSR).
However, Bensen-Henry review included narrative systematic reviews that did not calculate effect sizes at all. Furthermore, the excluded Goyal et al JAMA: Internal Medicine article summarized MBSR separate from other forms of meditation and the more comprehensive AHCQR report provided detailed forest plots of effect sizes for MBSR with specific outcomes and patient populations.
Hmm, keeping out evidence that does fit with the sell-job story?
We need to keep in mind the poor manner in which MBSR was specified, particularly in the early studies that dominate the reviews covered by the Bensen – Henry article. Many of the treatments were not standardized and certainly not manualized. They sometimes, but not always incorporate psychoeducation, other cognitive behavioral techniques, and varying types of yoga.
The Bensen-Henry authors claimed to have performed quality assessments of the reviews included using a checklist based on the validated PRISMA guidelines. However, PRISMA evaluates the quality of reporting in reviews, not the quality of how the review was done. The checklist used by the Bensen-Henry authors was highly selective in terms of which PRISMA items it chose to include, left unvalidated, and simply eccentric. For instance, one item evaluated a review favorably if it interpreted studies “independent of funding source.”
A lack of independence of a study from its funding source is generally considered a high risk of bias. There is ample documentation of industry-funded studies and reviews exaggerating the efficacy of interventions supported by industry.
Our group received the Bill Silverman Prize from the Cochrane Collaboration for our identifying funding source as an overlooked source of bias in many meta analyses and, in particular, in Cochrane reviews. The Bensen-Henry checklist scores a review ignoring funding source as a virtue, not a vice! These authors are letting trials and reviews from promoters of mindfulness off the hook for potential conflict of interest, including their own studies and this review.
Examination of the final sample of reviews included in the Bensen-Henry analysis reveals that some are narrative reviews and could not contribute effect sizes. Some are older reviews that depend on a less developed literature. While optimistic about the promise of mindfulness, authors of these reviews frequently complained about the limits on the quantity and quality of available studies, calling for larger and better quality studies. When integrated and summarized by the Bensen-Henry authors, these reviews were given a more positive glow than the original authors conveyed.
Despite claims of being an “overview of more trials than ever before”, Bensen-Henry excluded all but 23 reviews. Some of those included do not appear to be recent or rigorous, particularly when contrasted with the quality and rigor of the excluded Goyal et al:
MJ, Norris RL, Bauer-Wu SM (2006) Mindfulness meditation for oncology patients: A discussion and critical review. Integr Cancer Ther 5: 98–108. pmid:16685074
Shennan C, Payne S, Fenlon D (2011) What is the evidence for the use of mindfulness-based interventions in cancer care? A review. Psycho-Oncology 20: 681–697.
Veehof MM, Oskam MJ, Schreurs KMG, Bohlmeijer ET (2011) Acceptance-based interventions for the treatment of chronic pain: A systematic review and meta-analysis. Pain 152: 533–542
Coelho HF, Canter PH, Ernst E (2007) Mindfulness-Based Cognitive Therapy: Evaluating Current Evidence and Informing Future Research. J Consult Clin Psychol 75: 1000–1005.
Ledesma D, Kumano H (2009) Mindfulness-based stress reduction and cancer: A meta-analysis. Psycho-Oncology 18: 571–579.
Ott MJ, Norris RL, Bauer-Wu SM (2006) Mindfulness meditation for oncology patients: A discussion and critical review. Integr Cancer Ther 5: 98–108.
Burke CA (2009) Mindfulness-Based Approaches with Children and Adolescents: A Preliminary Review of Current Research in an Emergent Field. J Child Fam Stud.
Do we get the most authoritative reviews of mindfulness from Holist Nurs Pract, Integr Cancer Ther, and Psycho-Oncology?
To cite just one example of the weakness of evidence being presented as strong, take the bold Bensen-Henry conclusion:
Further evidence for effectiveness was provided by the observed dose-response relationship: an increase in total minutes of practice and class attendance led to a larger reduction of stress and mood complaints in four reviews [18,20,37,54].
“Observed dose-response relationship”? This claim is based [check out with respect to the citations just above] on Ott et al, 18, Smith et al 20, Burke 37 and Proulx 54, which makes the evidence neither recent nor systematic. I am confident that other examples will not hold up if scrutinized.
Further contradiction of the too perfect picture of mindfulness therapy conveyed by the Bensen – Henry review.
A more recent PLOS One review of mindfulness studies exposed the confirmation bias in the published mindfulness literature. It suggested a too perfect picture has been created of uniformly positive studies.
A systematic search yielded 124 RCTs of mindfulness-based treatments:
108 (87%) of 124 published trials reported >1 positive outcome in the abstract, and 109(88%) concluded that mindfulness-based therapy was effective, 1.6 times greater than the expected number of positive trials based on effect size d = 0.55 (expected number positivetrials = 65.7). Of 21 trial registrations, 13 (62%) remained unpublished 30 months post-trial completion.
None of the 21 registrations, however, adequately specified a single primary outcome (or multiple primary outcomes with an appropriate plan for statistical adjustment) and specified the outcome measure, the time of assessment, and the metric (e.g., continuous, dichotomous). When we removed the metric requirement, only 2 (10%) registrations were classified as adequate.
There were only 3 trials that were presented unequivocally as negative trials without alternative interpretations or caveats to mitigate the negative results and suggest that the treatment might still be an effective treatment.
What we have is a picture of trials of mindfulness-based treatment having an excess of positive studies, given the study sample sizes. Selective reporting of positive outcomes likely contributed to this excess of published positive findings in the published literature. Most of the trials were not preregistered and so it’s unclear whether the positive outcomes that were reported were hypothesized to be the primary outcomes of interest. Most of the trials that were preregistered remained unpublished 30 months after the trials were completed.
The Goyal et al. study originally planned to conduct quantitative analyses of publication biases, but abandoned the effort when they couldn’t find sufficient numbers of the 47 studies that that reported most of the outcomes they evaluated.
The Bensen-Henry review produces a glowing picture of the quality of RCTs evaluating MSBR and the consistency of positive findings across diverse outcomes and populations. This is consistent with the message that they want to promote in marketing their products to patients, clinicians, and institutions. In this blog post I’ve uncovered substantial problems in internal to the Bensen-Henry review in terms of the studies that were included and the manner in which they were evaluated. But now we have external evidence in two reviews without obvious conflicts of interest come into markedly different appraisals of a literature that lacks appropriate control groups and seems to be reporting findings with a distinct confirmation bias.
I could have gone further, but what I found about the Bensen-Henry review seems sufficient for a serious challenge to the validity of its conclusions. Investigation of the claims made about dose-response relationships between amount of mindfulness practice and outcomes should encourage probing of other specific claims.
A correction to the Bensen-Henry PLOS One review is in order to clarify the obvious conflicts of interest of the authors. But the problem is not limited to reviews or original studies from Benson-Henry Institute for Mind-Body Medicine. It’s time that authors be required to answer more explicit questions about conflict of interest. Ruling out a conflict of interest should be based on authors having to endorse explicitly no conflicts, rather than on their basis of their not disclosing a conflict and then being able to claim it was an oversight that they did not report one.
Postscript Who was watching at PLOS One to keep out infomercials from promoters associated with Massachusetts General Hospital and Harvard Medical School? The Academic Editor was To avoid the appearance of a conflict of interest, should he have recused him from serving as editor?
I will soon be offering e-books providing skeptical looks at mindfulness and positive psychology, as well as scientific writing courses on the web as I have been doing face-to-face for almost a decade.
Sign up at my new website to get advance notice of the forthcoming e-books and web courses, as well as upcoming blog posts at this and other blog sites. Get advance notice of forthcoming e-books and web courses. Lots to see at CoyneoftheRealm.com.
I was asked by a Danish journalist to examine a randomized controlled trial (RCT) of cognitive behavior therapy (CBT) for functional somatic symptoms. I had not previously given the study a close look.
I was dismayed by how highly problematic the study was in so many ways.
I doubted that the results of the study showed any benefits to the patients or have any relevance to healthcare.
I then searched and found the website for the senior author’s clinical offerings. I suspected that the study was a mere experimercial or marketing effort of the services he offered.
Overall, I think what I found hiding in plain sight has broader relevance to scrutinizing other studies claiming to evaluate the efficacy of CBT for what are primarily physical illnesses, not psychiatric disorders. Look at the other RCTs. I am confident you will find similar problems. But then there is the bigger picture…
Was unblinded to patients, interventionists, and to the physicians continuing to provide routine care.
Had a grossly unmatched, inadequate control/comparison group that leads to any benefit from nonspecific (placebo) factors in the trial counting toward the estimated efficacy of the intervention.
Relied on subjective self-report measures for primary outcomes.
With such a familiar trio of design flaws, even an inert homeopathic treatment would be found effective, if it were provided with the same positive expectations and support as the CBT in this RCT. [This may seem a flippant comment that reflects on my credibility, not the study. But please keep reading to my detailed analysis where I back it up.]
The study showed an inexplicably high rate of deterioration in both treatment and control group. Apparent improvement in the treatment group might only reflect less deterioration than in the control group.
The study is focused on unvalidated psychiatric diagnoses being applied to patients with multiple somatic complaints, some of whom may not yet have a medical diagnosis, but most clearly had confirmed physical illnesses.
But wait, there is more!
It’s not CBT that was evaluated, but a complex multicomponent intervention in which what was called CBT is embedded in a way that its contribution cannot be evaluated.
The “CBT” did not map well on international understandings of the assumptions and delivery of CBT. The complex intervention included weeks of indoctrination of the patient with an understanding of their physical problems that incorporated simplistic pseudoscience before any CBT was delivered. We focused on goals imposed by a psychiatrist that didn’t necessarily fit with patients’ sense of their most pressing problems and the solutions.
And the kicker.
The authors switched primary outcomes – reconfiguring the scoring of their subjective self-report measures years into the trial, based on a peeking at the results with the original scoring.
Investigators have a website which is marketing services. Rather than a quality contribution to the literature, this study can be seen as an experimercial doomed to bad science and questionable results from before the first patient was enrolled. An undeclared conflict of interest in play? There is another serious undeclared conflict of interest for one of the authors.
For the uninformed and gullible, the study handsomely succeeds as an advertisement for the investigators’ services to professionals and patients.
Personally, I would be indignant if a primary care physician tried to refer me or friend or family member to this trial. In the absence of overwhelming evidence to the contrary, I assume that people around me who complain of physical symptoms have legitimate physical concerns. If they do not yet have a confirmed diagnosis, it serves little purpose to stop the probing and refer them to psychiatrists. This trial operates with an anachronistic Victorian definition of psychosomatic condition.
1.This is an unblinded trial, a particularly weak methodology for examining whether a treatment works.
The letter that alerted physicians to the trial had essentially encouraged them to refer patients they were having difficulty managing.
‘Patients with a long-term illness course due to medically unexplained or functional somatic symptoms who may have received diagnoses like fibromyalgia, chronic fatigue syndrome, whiplash associated disorder, or somatoform disorder.
Patients and the physicians who referred them subsequently got feedback about to which group patients were assigned, either routine care or what was labeled as CBT. This information could have had a strong influence on the outcomes that were reported, particularly for the patients left in routine care.
Patients’ learning that they did not get assigned to the intervention group was undoubtedly disappointing and demoralizing. The information probably did nothing to improve the positive expectations and support available to patients in routine. This could have had a nocebo effect. The feedback may have contributed to the otherwise inexplicably high rates of subjective deterioration [to be noted below] reported by patients left in the routine care condition. In contrast, the authors’ disclosure that patients had been assigned to the intervention group undoubtedly boosted the morale of both patients and physicians and also increased the gratitude of the patients. This would be reflected in the responses to the subjective outcome measures.
The gold standard alternative to an unblinded trial is a double-blind, placebo-controlled trial in which neither providers, nor patients, nor even the assessors rating outcomes know to which group particular patients were assigned. Of course, this is difficult to achieve in a psychotherapy trial. Yet a fair alternative is a psychotherapy trial in which patients and those who refer them are blind to the nature of the different treatments, and in which an effort is made to communicate credible positive expectations about the comparison control group.
Conclusion: A lack of blinding seriously biases this study toward finding a positive effect for the intervention, regardless of whether the intervention has any active, effective component.
2. A claim that this is a randomized controlled trial depends on the adequacy of the control offered by the comparison group, enhanced routine care. Just what is being controlled by the comparison? In evaluating a psychological treatment, it’s important that the comparison/control group offers the same frequency and intensity of contact, positive expectations, attention and support. This trial decidedly did not.
There were large differences between the intervention and control conditions in the amount of contact time. Patients assigned to the cognitive therapy condition received an additional 9 group sessions with a psychiatrist of 3.5 hour duration, plus the option of even more consultations. The over 30 hours of contact time with a psychiatrist should be very attractive to patients who wanted it and could not otherwise obtain it. For some, it undoubtedly represented an opportunity to have someone to listen to their complaints of pain and suffering in a way that had not previously happened. This is also more than the intensity of psychotherapy typically offered in clinical trials, which is closer to 10 to 15, 50-minute sessions.
The intervention group thus received substantially more support and contact time, which was delivered with more positive expectations. This wealth of nonspecific factors favoring the intervention group compromises an effort to disentangle the specific effects of any active ingredient in the CBT intervention package. From what has been said so far, the trials’ providing a fair and generalizable evaluation of the CBT intervention is nigh impossible.
Conclusion: This is a methodologically poor choice of control groups with the dice loaded to obtain a positive effect for CBT.
3.The primary outcomes, both as originally scored and after switching, are subjective self-report measures that are highly responsive to nonspecific treatments, alleviation of mild depressive symptoms and demoralization. They are not consistently related to objective changes in functioning. They are particularly problematic when used as outcome measures in the context of an unblinded clinical trial within an inadequate control group.
There have been consistent demonstrations that assigning patients to inert treatments and measuring the outcomes with subjective measures may register improvements that will not correspond to what would be found with objective measures.
For instance, a provocative New England Journal of Medicinestudy showed that sham acupuncture as effective as an established medical treatment – an albuterol inhaler – for asthma when judged with subjective measures, but there was a large superiority for the established medical treatment obtained with objective measures.
There have been a number of demonstrations that treatments such as the one offered in the present study to patient populations similar to those in the study produce changes in subjective self-report that are not reflected in objective measures.
Much of the improvement in primary outcomes occurred before the first assessment after baseline and not very much afterwards. The early response is consistent with a placebo response.
The study actually included one largely unnoticed objective measure, utilization of routine care. Presumably if the CBT was effective as claimed, it would have produced a significant reduction in healthcare utilization. After all, isn’t the point of this trial to demonstrate that CBT can reduce health-care utilization associated with (as yet) medically unexplained symptoms? Curiously, utilization of routine care did not differ between groups.
The combination of the choice of subjective outcomes, unblinded nature of the trial, and poorly chosen control group bring together features that are highly likely to produce the appearance of positive effects, without any substantial benefit to the functioning and well-being of the patients.
Conclusion: Evidence for the efficacy of a CBT package for somatic complaints that depends solely on subjective self-report measures is unreliable, and unlikely to generalize to more objective measures of meaningful impact on patients’ lives.
4. We need to take into account the inexplicably high rates of deterioration in both groups, but particularly in the control group receiving enhanced care.
There was an unexplained deterioration of 50% deterioration in the control group and 25% in the intervention group. Rates of deterioration are only given a one-sentence mention in the article, but deserve much more attention. These rates of deterioration need to qualify and dampen any generalizable clinical interpretation of other claims about outcomes attributed to the CBT. We need to keep in mind that the clinical trials cannot determine how effective treatments are, but only how different a treatment is from a control group. So, an effect claimed for a treatment and control can largely or entirely come from deterioration in the control group, not what the treatment offers. The claim of success for CBT probably largely depends on the deterioration in the control group.
One interpretation of this trial is that spending an extraordinary 30 hours with a psychiatrist leads to only half the deterioration experienceddoing nothing more than routine care. But this begs the question of why are half the patients left in routine care deteriorating in such a large proportion. What possibly could be going on?
Conclusion: Unexplained deterioration in the control group may explain apparent effects of the treatment, but both groups are doing badly.
5. The diagnosis of “functional somatic symptoms” or, as the authors prefer – Severe Bodily Distress Syndromes – is considered by the authors to be a psychiatric diagnosis. It is not accepted as a valid diagnosis internationally. Its validation is limited to the work done almost entirely within the author group, which is explicitly labeled as “preliminary.” This biased sample of patients is quite heterogeneous, beyond their physicians having difficulty managing them. They have a full range of subjective complaints and documented physical conditions. Many of these patients would not be considered primarily having a psychiatric disorder internationally and certainly within the US, except where they had major depression or an anxiety disorder. Such psychiatric disorders were not an exclusion criteria.
Once sent on the pathway to a psychiatric diagnosis by their physicians’ making a referral to the study, patients had to meet additional criteria:
To be eligible for participation individuals had to have a chronic (i.e. of at least 2 years duration) bodily distress syndrome of the severe multi-organ type, which requires functional somatic symptoms from at least three of four bodily systems, and moderate to severe impairment.in daily living.
The condition identified in the title of the article is not validated as a psychiatric diagnosis. Two papers to which the authors refer to their own studies ( 1 , 2 ) from a single sample. The title of one of these papers makes a rather immodest claim:
Fink P, Schröder A. One single diagnosis, bodily distress syndrome, succeeded to capture 10 diagnostic categories of functional somatic syndromes and somatoform disorders. Journal of Psychosomatic Research. 2010 May 31;68(5):415-26.
In neither the two papers nor the present RCT is there sufficient effort to rule out a physical basis for the complaints qualifying these patients for a psychiatric diagnosis. There is also a lack of follow-up to see if physical diagnoses were later applied.
Citation patterns of these papers strongly suggest the authors are not having got much traction internationally. The criteria of symptoms from three out of four bodily systems is arbitrary and unvalidated. Many patients with known physical conditions would meet these criteria without any psychiatric diagnosis being warranted.
The authors relate what is their essentially homegrown diagnosis to functional somatic syndromes, diagnoses which are themselves subject to serious criticism. See for instance the work of Allen Frances M.D., who had been the chair of the American Psychiatric Association ‘s Diagnostic and Statistical Manual (DSM-IV) Task Force. He became a harsh critic of its shortcomings and the failures of APA to correct coverage of functional somatic syndromes in the next DSM.
Unless DSM-5 changes these incredibly over inclusive criteria, it will greatly increase the rates of diagnosis of mental disorders in the medically ill – whether they have established diseases (like diabetes, coronary disease or cancer) or have unexplained medical conditions that so far have presented with somatic symptoms of unclear etiology.
The diagnosis of mental disorder will be based solely on the clinician’s subjective and fallible judgment that the patient’s life has become ‘subsumed’ with health concerns and preoccupations, or that the response to distressing somatic symptoms is ‘excessive’ or ‘disproportionate,’ or that the coping strategies to deal with the symptom are ‘maladaptive’.
“These are inherently unreliable and untrustworthy judgments that will open the floodgates to the overdiagnosis of mental disorder and promote the missed diagnosis of medical disorder.
The DSM 5 Task force refused to adopt changes proposed by Dr. Frances.
Dimsdale noted that applying this psychiatric diagnosis sidesteps the quality of medical examination that led up to it. Furthermore:
Many illnesses present initially with nonspecific signs such as fatigue, long before the disease progresses to the point where laboratory and physical findings can establish a diagnosis.
And such diagnoses may encompass far too varied a group of patients for any intervention to make sense:
One needs to acknowledge that diseases are very heterogeneous. That heterogeneity may account for the variance in response to intervention. Histologically, similar tumors have different surface receptors, which affect response to chemotherapy. Particularly in chronic disease presentations such as irritable bowel syndrome or chronic fatigue syndrome, the heterogeneity of the illness makes it perilous to diagnose all such patients as having MUS and an underlying somatoform disorder.
I tried making sense of a table of the additional diagnoses that the patients in this study had been given. A considerable proportion of patients had physical conditions that would not be considered psychiatric problems in the United States.. Many patients could be suffering from multiple symptoms not only from the conditions, but side effects of the medications being offered. It is very difficult to manage multiple medications required by multiple comorbidities. Physicians from the community found their competence and ability to spend time with these patients taxing.
The nongastrointestinal nonpsychiatric disorders with the best-documented association are fibromyalgia (median of 49% have IBS), chronic fatigue syndrome (51%), temporomandibular joint disorder (64%), and chronic pelvic pain (50%).
In the United States, many patients and specialists would consider considering irritable bowel syndrome as a psychiatric condition offensive and counterproductive. There is growing evidence that irritable bowel syndrome is a disturbance in the gut microbiota. It involves a gut-brain interaction, but the primary direction of influence is of the disturbance in the gut on the brain. Anxiety and depression symptoms are secondary manifestations, a product of activity in the gut influencing the nervous system.
Most of the patients in the sample had a diagnosis of fibromyalgia and over half of all patients in this study had a diagnosis of chronic fatigue syndrome.
Other patients had diagnosable anxiety and depressive disorders, which, particularly at the lower end of severity, are responsive to nonspecific treatments.
Undoubtedly many of these patients, perhaps most of them, are demoralized by not been able to get a diagnosis for what they have good basis to believe is a medical condition, aside from the discomfort, pain, and interference with their life that they are experiencing. They could be experiencing a demoralization secondary to physical illness.
These patients presented with pain, fatigue, general malaise, and demoralization. I have trouble imagining how their specific most pressing concerns could be addressed in group settings. These patients pose particular problems for making substantive clinical interpretation of outcomes that are highly general and subjective.
Conclusion: Diagnosing patients with multiple physical symptoms as having a psychiatric condition is highly controversial. Results will not generalize to countries and settings where the practice is not accepted. Many of the patients involved in the study had recognizable physical conditions, and yet they are being shunted to psychiatrists who focused only on their attitude towards the symptoms. They are being denied the specialist care and treatments that might conceivably reduce the impact of their conditions on their lives
6. The “CBT” offered in this study is as part of a complex, multicomponent treatment that does not resemble cognitive behavior therapy as it is practiced in the United States.
As seen in figure 1 in the article, The multicomponent intervention is quite complex and consists of more than cognitive behavior therapy. Moreover, at least in the United States, CBT has distinctive elements of collaborative empiricism. Patients and therapist work together selecting issues on which to focus, developing strategies, with the patients reporting back on efforts to implement them. From the details available in the article, the treatment sounded much more like an exhortation or indoctrination, even arguing with the patients, if necessary. An English version available on the web of the educational material used in initial sessions confirmed a lot of condescending pseudoscience was presented to convince the patients that their problems were largely in their heads.
Without a clear application of learning theory, behavioral analysis, or cognitive science, the “CBT” treatment offered in this RCT has much more in common with the creative novation therapy offered by Hans Eysenck, which is now known to have been justified with fraudulent data. Indeed, the educational materials for this study to what is offered in Eysenck’s study reveal striking similarities. Eysenck was advancing the claim that his intervention could prevent cardiovascular disease and cancer and overcome the iatrogenic effects. I know, this sounds really crazy, but see my careful documentation elsewhere.
Conclusion: The embedding of an unorthodox “CBT” in a multicomponent intervention in this study does not allow isolating any specific, active component ofCBT that might be at work.
7. The investigators disclose having altered their scoring of their primary outcome years after the trial began, and probably after a lot of outcome data had been collected.
I found a casual disclosure in the method section of this article unsettling, particularly noting that the original trial registration was:
We found an unexpected moderate negative correlation of the physical and mental component summary measures, which are constructed as independent measures. According to the SF-36 manual, a low or zero correlation of the physical and mental components is a prerequisite of their use.23 Moreover, three SF-36 scales that contribute considerably to the PCS did not fulfil basic scaling assumptions.31 These findings, together with a recent report of problems with the PCS in patients with physical and mental comorbidity,32 made us concerned that the PCS would not reliably measure patients’ physical health in the study sample. We therefore decided before conducting the analysis not to use the PCS, but to use instead the aggregate score as outlined above as our primary outcome measure. This decision was made on 26 February 2009 and registered as a protocol change at clinical trials. gov on 11 March 2009. Only baseline data had been analysed when we made our decision and the follow-up data were still concealed.
Switching outcomes, particularly after some results are known, constitutes a serious violation of best research practices and leads to suspicion of the investigators refining their hypotheses after they had peeked at the data. See How researchers dupe the public with a sneaky practice called “outcome switching”.
The authors had originally proposed a scoring consistent with a very large body of literature. Dropping the original scoring precludes any direct comparison with this body of research, including basic norms. They claim that they switched scoring because two key subscales were correlated in the opposite direction of what is reported in the larger literature. This is troubling indication that something has gone terribly wrong in authors’ recruitment of a sample. It should not be pushed under the rug.
Even if the authors of the present RCT did not peekat their outcome data before deciding to switch scoring of the primary outcome, they certainly had strong indications from other sources that the original scoring would produce weak or null findings. In 2009, one of the authors, Michael Sharpe had access to results of a relevant trial. What is called the FINE trial had null findings, which affected decisions to switch outcomes in the PACE trial. Is it just a coincidence that the scoring of the outcomes was then switched for the present RCT?
Conclusion: The outcome switching for the present trial represents bad research practices. For the trial to have any credibility, the investigators should make their data publicly available so these data could be independently re-analyzed with the original scoring of primary outcomes.
The senior author’s clinic
I invite readers to take a virtual tour of the website for the senior author’s clinical services ]. Much of it is available in English. Recently, I blogged about dubious claims of a health care system in Detroit achieving a goal of “zero suicide.” . I suggested that the evidence for this claim was quite dubious, but was a powerful advertisement for the health care system. I think the present report of an RCT can similarly be seen as an infomercial for training and clinical services available in Denmark.
If results of this are accepted at face, they will lend credibility to the claims that effective interventions are available to reduce social disability. It doesn’t matter that the intervention is not effective. Rather persons receiving social disability payments can be disqualified because they are not enrolled in such treatment.
Effects on the credibility of Cochrane collaboration report
The switched outcomes of the trial were entered into a Cochrane systematic review, to which primary care health professionals look for guidance in dealing with a complex clinical situation. The review gives no indication of the host of problems that I exposed here. Furthermore, I have glanced at some of the other trials included and I see similar difficulties.
I been unable to convince the Cochrane to clean up conflicts of interest that are attached to switched outcomes being entered in reviews. Perhaps some of my readers will want to approach Cochrane to revisit this issue.
I think this post raises larger issues about whether Cochrane has any business conducting and disseminating reviews of such a bogus psychiatric diagnosis, medically unexplained symptoms. These reviews do patients no good, and may sidetrack them from getting the medical care they deserve. The reviews do serve the interest of special interests, including disability insurance companies.
Special thanks to John Peters and to Skeptical Cat for their assistance with my writing this blog. However, I have sole responsibility for any excesses or distortions.
Cortisol, a stress hormone, is a key player in the subtle hormonal changes that have come to be associated with PTSD, and Dr. Rachel Yehuda, a neuroscientist and the director of the traumatic stress studies division at Mount Sinai School of Medicine in New York, has played a major role in advancing our scientific understanding of the role of cortisol in PTSD.
More recently, Dr. Yehuda also offered the PTSD scientific community a novel and intriguing idea: that the children of traumatized parents are at risk for similar problems due to changes that occurred in the biology of their parents, as a consequence of their trauma exposure. It is these epigenetic changes that are then transmitted to their children via a process called “intergenerational transmission.”
Recently, I spoke with Dr. Yehuda about cortisol, intergenerational transmission of stress, and the future of PTSD treatment and research.
Dr. Jain: You played a key role in re-conceptualizing the neuro-endocrine basis for PTSD after it became apparent that individuals with PTSD consistently have low cortisol levels. Can you speak a little bit about how robust a finding this is and what this means for clinical settings? How can we use cortisol levels in the diagnosis of PTSD? Can we use it to track if people are getting better?
Dr. Yehuda: The first published observation on cortisol in PTSD was in 1986 by John Mason and colleagues at Yale. The group was interested to see if tracking stress hormone levels in patients admitted to the psych unit would aid in determining when patients might be safely discharged, so they measured cortisol levels in a wide range of psychiatric patients. Generally, cortisol levels were higher for patients at admission and then were much lower at discharge, which is what one would expect if cortisol is a marker of stress. However there were two groups of patients, one being patients with post-traumatic stress disorder, for whom this did not appear to be the case. The authors were surprised to find that in fact PTSD patients showed significantly lower cortisol levels at admission and discharge compared to patients with other diagnoses. I joined Yale a year after that finding appeared in the literature. Like many others, I found it curious that cortisol levels would be low and thought for sure there had be some mistake, because we would expect, if anything, that cortisol levels would be high in a stress disorder, particularly one in which there was comorbidity of depression. So I attempted a replication with Mason and his colleagues, and of course, we were able to replicate the low cortisol findings in several studies in the early 90’s.
What was so interesting, however, was how long it took the field to accept that the finding may reflect a reality. At the same time, and in the same patients, Mason and I observed elevated catecholamines. Neither Mason nor I had any trouble with the very first publication that catecholamine levels were higher in PTSD. No one thought to question the finding because it was something expected—that people who are aroused and under stress have high levels of catecholamines, like norepinephrine. Yet the cortisol data from the samples were difficult for people to believe. I guess when we hear something that makes sense to us, we do not need a lot of data. But we all questioned the low cortisol finding because it didn’t make sense, and then we questioned the methodology and so on. The reason the finding did not make sense, in the early 90s, was because the field of PTSD was new, and we didn’t really understand PTSD yet. There were really no epidemiological studies until the early 90’s, and even this very well accepted idea that PTSD only occurred in a subset of trauma survivors was not yet known. The prevailing concept was that PTSD always occurred following trauma exposure. But once there was a body of literature that showed that a lot of people are trauma exposed and only a smaller subset of those people get PTSD, the field could start speculating that perhaps low cortisol signals an abnormality that helps explain why recovery has not occurred. And when that happened, we began to ask what is cortisol’s role in stress, anyway? In turns out that one of the things that cortisol does in response to stress is that it helps contain the catecholamine system—it helps bring down the high levels of adrenaline that are released during fight or flight. Since we all know that adrenaline and norepinephrine are responsible for memory formation and arousal, not having enough cortisol to completely bring down the sympathetic nervous system, at the time when it is very important for a person to calm down, may partially explain the formation of traumatic memory or generalized triggers.
The second part of your question is what does this mean in a clinical setting and how can we use cortisol levels in the diagnosis of PTSD? At this moment, we cannot use cortisol levels to aid in diagnoses. They are too variable, and although there is a mean difference between PTSD and other groups, in every study that has been performed to date, there are a lot of overlapping data. Furthermore, even the low cortisol levels in PTSD are well within the normal endocrinological range. The reason the low cortisol finding was important was that it led us down a trail of trying to understand why cortisol levels were low. Then it took us into the dynamics of the way that the hypothalamic–pituitary–adrenal (HPA) axis works and is regulated by the brain. Cortisol levels show natural variation during the day, and are affected by environmental perturbations. It is adaptive that cortisol levels vary , because cortisol helps regulate many bodily functions when we are stressed, and when we are not stressed. What we have been doing for the last 25 years is studying the underlying dynamics of cortisol levels. We have examined circadian rhythm changes that may determine how the brain regulates the release of cortisol over a diurnal cycle. We have looked at cortisol metabolism, to try to understand how cortisol is broken down into its various metabolites in the brain, liver, and kidney. But most of our studies have involved the glucocorticoid receptor and all of the genes and proteins that are involved in regulating the activity and sensitivity of that receptor. These studies have begun to give us an understanding that there is something really different about the stress system in PTSD, or in specific subtypes of people with PTSD,, but it is not going to be cortisol levels per se that are going to be useful to a clinician.
Dr. Jain: So the picture is much more complicated than what may have been originally conceptualized?
Dr. Yehuda: When we say low cortisol levels, an endocrinologist would cringe. In PTSD, cortisol levels are not lower than normal range. They are significantly lower on average compared to persons without PTSD, but the levels themselves are not abnormal. The cortisol levels in PTSD do not suggest that the adrenal gland is broken in any way or not releasing cortisol, but rather, given the normal range of cortisol, which is large—between 20 to 90 micrograms per 24 hours of urine—the means we would get in PTSD were in the 40s. Whereas, a straight mean would be more like in the 50s and 60s. We are not talking about an endocrine problem. We are talking about a tendency to be at the lower end which is within normal variability. Why this was newsworthy, again, was that we were expecting that it would be higher in a stress disorder, because cortisol is associated with stress. I personally would not use cortisol levels, not even 24-hour urinary cortisol levels, as a diagnostic marker. I would want to know a lot more about how the glucocorticoid receptor works. Is it more sensitive? What is the circadian rhythm like? What about cortisol metabolism? What about the genes that control cortisol and glucocorticoid functioning? So there is a potential to find biomarkers that relate to cortisol that may be clinically applicable—we have not given up on that idea at all. It’s just important to understand what kind of neuroendocrine or molecular neuroendocrine information is most relevant.
Dr. Jain: But it is not as simplistic as doing a blood test to diagnose PTSD.
Dr. Yehuda: Would that it were!
Dr. Jain: I know! But you offer a very important clarification: the pattern in PTSD is of lowER cortisol levels, not low cortisol.
Dr. Yehuda: Somehow statistically lower became low, but the devil is in the detail.
Dr. Jain: Absolutely, and that is why it is so valuable to talk to people like yourself.
Dr. Yehuda: Furthermore, the effect size of cortisol differences is small, too. In the Boscarino study (1995), he reported that cortisol was lower in PTSD, but there was a very small effect size. So it is not a diagnostic test. It is just a clue, and we used it exactly as a clue to unravel a deeper mystery.
Dr. Jain: I totally see that. My next question is about the potential role of cortisol in the treatment of PTSD. Maybe if you could speak about that a little bit. From a clinician point of view, that is really intriguing. It feels like immediate clinical applications might be on the horizon.
Dr. Yehuda: I see at least three or four ways that we could think about cortisol-based interventions. The first one might be prevention. That is the Zohar study, which is a study being conducted in Tel Hashomer hospital in Israel, headed by Dr. Joseph Zohar. When I first heard his idea of using cortisol in the ER to prevent PTSD, I have to admit I was skeptical, even though we are the ones that published that cortisol levels are lower in the immediate aftermath in persons who are more likely to develop PTSD. What Dr. Zohar said was, if that is true then we should be able to give cortisol during the “golden hours.” But I was nervous. Why? Because I think that hormonal response is something that you want to be very careful about changing, because the body has a wisdom. That is my general view of the world, but he convinced me that if you give a single really high dose of glucocorticoids within a 4-hour window of a trauma, then the effect that that might have would be to recalibrate the HPA axis in a way that provides enough cortisol to quiet down the sympathetic nervous system in a very organic and permanent way. Also, Dr. Hagit Cohen’s in Ben Gurion Medical School in Beer Sheva work with animal studies had shown that this might actually work to prevent PTSD if given during the “golden hours.”
Dr. Jain: By “golden hours” you refer to that 4-hour window after the trauma?
Dr. Yehuda: We do not know what the window is. In our study we said 4 hours. I do not know if it is 8 hours or 12 hours! We do not know if it is 2 days! Ironically, when people give benzodiazepines in the acute aftermath of a trauma, they are doing the opposite thing, as benzodiazepines lower cortisol levels. So even though in the short run, you may experience some relief, in the long run it just kicks the can down the road. Dr. Zohar’s idea is that by intervening early you can set a pathway towards recovery.
There have been other studies like this. In fact, the first observation of this was by a physician in Germany named Dr. Gustav Schelling. He was treating septic shock and using hydrocortisone as a treatment for septic shock. What he noticed was that those who had received high levels of glucocorticoids, which not everyone did, had fewer complaints of traumatic memories from their traumatic experience of being critically ill. He searched for an explanation and finally did a randomized clinical trial. He concluded that there were beneficial effects of administering high doses of glucocorticoids in the early aftermath of a trauma. So prevention is certainly one potential avenue.
But there are people who have given glucocorticoids not during the “golden hours,” but in a more sustained way over several weeks. They have also found potentially beneficial effects. We have just completed our study with Dr. Zohar and eagerly await the results. In this study we also measured biomarkers to see if treatment could be predicted.
Another way to effect changes in the HPA axis might actually be to block the glucocorticoid receptor. There is a trial that is ongoing now using a drug called mifepristone, which is a glucocorticoid receptor antagonist. You might know this drug by a different name. This study is being run by my colleague Dr. Julia Golier. You might know mifepristone as RU-486, or the abortion pill. RU-486 obviously has effects on the progesterone receptor, which is why it is an effective treatment to prevent pregnancy, but it also has effects on the glucocorticoid receptor. There is a trial that is ongoing now, ending August. The pilot study showed some benefit. What happens with that treatment is that you can block the glucocorticoid receptor and really recalibrate the ratio of peripheral to central cortisol. The beauty of that treatment is again you give it once or you give it for a very short period of time, and you look for recalibration effects. People like to take medications that way as opposed to every single day.
Another way to think about glucocorticoid treatments is to use cortisol as an augmenter of psychotherapy. We have been doing some studies where you give moderate doses of cortisol or hydrocortisone about half an hour before an exposure based treatment. The rationale for that is that glucocorticoids facilitate new learning. They facilitate extinction, and it could be that administration of moderate doses of hydrocortisone could really set the stage for doing better in exposure therapies. We found that in case reports in a small trial we conducted. What we found was that there were fewer drop-outs out of prolonged exposure therapy if they were given hydrocortisone compared to placebo. If that continues, that is a big deal, because we know that a lot of patients drop out of these treatments prematurely. Anything that makes somebody just stay in treatment is probably good.
Dr. Jain: Moving on to the next question then. There is this whole issue regarding lower cortisol levels being a pre-traumatic trait, like, somebody already has this and then they are trauma exposed and have a higher chance of developing PTSD. What are the implications of this for screening and resiliency programs in clinical settings?
Dr. Yehuda: We have an artificial view of what “pre-trauma” means. Pre-trauma of the event that we happen to be thinking about now? Many of us don’t consider enough what kind of early environmental events people have experienced before they present for effects of the trauma that they are coping with now.
We know that many people in the military have had traumatic experiences prior to being in the military, yet we define their pre-trauma cortisol as being pre- combat, as opposed to before they ever experienced any adversity.
I think this is a tough nut to crack. In our studies, we found that lower cortisol levels were present in rape victims who had had a prior assault. They are more likely to develop PTSD, but was their cortisol level already low? Is that why it did not climb up higher than it could have?
I think that these are important issues. Now, there was a fascinating study that was published by Mirjam van Zuiden and her group in the Netherlands that basically took a thousand soldiers, before they went into combat, and looked at cortisol and glucocorticoids receptor measures and markers, as well as genes and epigenetic markers of the glucocorticoid receptor. They found that low cortisol and enhanced glucocorticoid receptor sensitivity were predictors of people that had PTSD a few months later.
Now, of course, we do not know if they also had prior trauma. We do not know that, but that was a very elegant demonstration.
It is exactly as you say, but it is hard to unpack these things. At least we are getting closer to understanding that not all the action occurs at the time of the trauma. That the stage might be set in advance, we are actually an accumulation of our experiences, and we hold biologic changes and then use them to respond differently to traumatic events as they emerge in our lives.
Dr. Jain: That is very true. I like that phrase—it is setting the stage for subsequent trauma reactions. We have not figured out exactly how all those pieces come together.
Dr. Yehuda: There are a lot of people that are studying the effects of child abuse and early trauma even in the absence of PTSD. Their work is also supporting lower cortisol levels. It may be that low cortisol will impacts whether someone gets PTSD to a later trauma. The problem can be that when you study someone at one point in time and they have low cortisol but they don’t have PTSD, that does not mean that they will not develop PTSD if exposed to a trauma in the future. We do not know whether low cortisol measures are markers or predictors of the future, but I would suspect that there is a genetic component as well as an early environmental component that would make these markers predictors. That is one of the difficulties in conducting such studies. The challenge of clinical research is that we are looking at a few points in time and trying to make decisions as if we were looking at stable phenotypes, when we know that there is an awful lot of change that occurs within individuals in terms of their mental state, not to mention the fact that people often have really complex lives with a lot of things going on. So, you might be resilient following the first three events, and then the fourth one occurs and then you develop PTSD. We do not really know how useful these measures are, but there is probably a way that we can do more longitudinal prospective studies to get a flavor of that. I know that those are studies that are ongoing in the VA system, which is really good.
Dr. Jain: That is great. Related to that and transitioning to this concept of this intergenerational transmission of stress: Your 2005 study with the women who were pregnant in the World Trade Center, it was fascinating to read that study. I thought that it was an elegant demonstration of this concept of intergenerational transmission of stress. It would be great if you could talk a little bit about that study. One question that came to mind was a question about the pre-trauma cortisol level in the women. I wondered if that was measured, and did you gather data on their earlier experiences with trauma? That was just one particular question I had, but if you could just discuss the study in general, because I think it was really a fantastic contribution to the literature.
Dr. Yehuda: We did not have a lot of information on the women. In fact, this whole study was post-hoc in a sense that the study was designed for a completely different reason. It was to monitor pregnant women to make sure they gave birth to healthy babies. Everyone was really concerned about the level of environmental toxins after 9/11. Somebody from the environmental medicine group reached out to me because they noticed that a lot of women were really not doing very well emotionally and psychologically.
So by the time I was involved, some of the women had already given birth, but there had been a lot of information about what trimester they were in, about any pregnancy complications, exposure to toxins, etc. etc. So we added to that an evaluation of PTSD. Then when they came in for their 7 month to 1 year wellness baby evaluation, we were able to get salivary samples from the mother and the child. By then it did not surprise us to see that mothers with PTSD had lower cortisol levels than mothers without PTSD. But what did fascinate us was that in the mothers that had lower cortisol, the babies also had lower cortisol, but that this was a trimester dependent effect and that it seemed to split out in the second and third trimester in mothers who had been exposed in the middle of the second trimester or exposed in the third trimester.
When we had those findings, a lot of possibilities opened up in terms of how cortisol levels might be transmitted from parents to child or from mother to child. We were not the first people to make this observation. There has been a literature that that has demonstrated that mothers who are exposed to under feeding before puberty have children and grandchildren that have metabolic problems. Since we knew that the women exposed to starvation during pregnancy also tend to give birth to children who were more prone to hypertension as adults, we knew that there was the possibility of in utero effects.
But what seemed to happen here was an example of glucocorticoid programming. In the middle of the second trimester of pregnancy, there is an enzyme that becomes expressed in the placenta. It is an enzyme that blocks the conversion of cortisol to its inactive metabolite, cortisone. The induction of this enzyme really helps protect the fetus from detrimental effects of maternal glucocorticoids, because the cortisol is broken down into its inactive metabolite, cortisone. The enzyme is called 11β-Hydroxysteroid dehydrogenase type 2. We had already been interested in studying this enzyme just because we were interested in cortisol metabolism. But it turns out that in mothers who are under stress, it is very possible that their enzyme levels and the amount of glucocorticoids they have could overwhelm the body’s ability to metabolize cortisol into cortisone and affect the fetus. That was one idea that we had, that there might be a transmission based on offspring response in utero to maternal levels of stress hormones.
The message is straightforward: mothers who are stressed during pregnancy can program the stress response of their offspring, in utero, and the offspring accommodates somehow to the level of stress hormone. That has become a very important issue also in our intergenerational studies. It has become one viable mechanism through which mothers may “transmit” different vulnerabilities (or resilience) to their offspring. One does not need to have actual trauma experiences post-natally in order to have some of the neuroendocrine features associated with PTSD and PTSD risk. And this means that pregnancy is an important time with great social implications for our society. I do not think that we think about pregnancy as the very important developmental event that it really is. Otherwise, we would be really taking much better care of traumatized pregnant women than we do.
Dr. Jain: Obstetrics care involves screening for gestational diabetes, congenital defects in the baby, and even screening for postpartum depression……
Dr. Yehuda: Yes, and we should screen for trauma, too.
Dr. Jain: Given how high the rates of trauma exposure are in the population, it is worthwhile screening for trauma in pregnant women.
Dr. Yehuda: Exactly.
Dr Jain: The other thing I wanted to ask about was early data indicating that exposure to trauma can impact the psychosocial functioning of second, maybe third generation offspring. I think there were some studies done with holocaust survivors. If you could speak a little bit to that, because obviously that has very widespread societal implications, too.
Dr. Yehuda: Yes, we have found that in the adult children of holocaust survivors, they are more vulnerable to psychopathology and this is true of offspring who have parents with psychiatric symptoms. In one study we were able to measure biological and epigenetic markers showing that there are effects on holocaust offspring, based on either maternal and in utero developmental factors, maternal exposure, or maternal and paternal PTSD.
Dr. Jain: In general, what would you feel are the important questions for trauma scientists to answer in the next one to two decades? What would be top on your list to prioritize?
Dr. Yehuda: Many decades ago when the field first conceptualized the diagnosis of PTSD, our response was to emphasize the commonalities in trauma survivors regardless of what their exposures were. But I think it is important now to go back and see in a more clear way whether combat veterans are or are not different than other trauma survivors, or if interpersonal violence leaves a unique biological scar compared to a natural disaster, or whether age at traumatization matters or duration of trauma matters.
We basically have a threshold phenomenon where if you are over the threshold of what constitutes a trauma, you could be in the category depending on if you have the symptoms that are the symptoms of PTSD, but that is not very nuanced. In my experience, although there are similarities between trauma survivors in their mental health profile, there are also really important differences.
Some of the treatments that we have developed may really work better for some groups rather than others. For example, it seems like prolonged exposure is a fantastic treatment for interpersonal violence in women, and then the question becomes, is it as good for combat veterans? Have we studied this carefully enough? Should we be tailoring treatments based on trauma type and not just whether or not a threshold for trauma and symptoms has been met? We have to start customizing this.
The other thing that I think is really important is this idea that the designation of PTSD is a static one, or that it is binary or not dynamic. We have to rethink that. Now that I have the perspective of having years in the field and seeing the same trauma survivors over a period of many years, even decades, I understand that the same person can at sometimes meet diagnostic criteria for PTSD while at other times, that person may not. Do we view the person as always at risk after s/he has recovered? Especially when you have recovered from something and you are asked about having had it in the past, your memory is not so good for how much you have suffered in the past when you are feeling good right now.
Sometimes, I have had the ability to actually do a diagnostic interview of someone, meet them 10 years later, ask them about their worst episode of PTSD, and if they are feeling fine today they won’t remember how bad it was. What does that mean for biological studies, for biomarkers, and for risk? Just the idea of whether the categories are binary or not, I think is something that we really want to look at.
Finally, I think we have been paying a lot of attention to the psychological aspect of trauma and not enough to the physical illness part—the fact that people who are exposed to combat may die at an earlier age, make poor behavioral health choices, and are more prone to hypertension, metabolic syndrome, inflammatory illness, cardiovascular disease, and cancer. These cannot be coincidences, but may either be part of the trauma effects, or part of the PTSD effects. Why are we not more focused on the biomarkers that might help explain and reverse some of these illnesses? When will we start seeing PTSD and trauma exposure as the multisystem condition that it is and really try to integrate care plans that not only assess for nightmares, hyper vigilance, and concentration, but diet and exercise and hemoglobin A1c? These are markers for trauma survivors because they are at greater risk for all these issues, not to mention cognitive decline. What I would like to see is us incorporating a much more holistic approach to understanding the effect of trauma that does not divide the mind and the body into different spheres and really focuses on wellness in a much more broad way.
Dr. Jain: So that integration between the physical and the mental, even in the way we treat them. Right now, it is separated out into mental health and physical health.
Dr. Yehuda: It does not make sense. Many veterans that come for care do not take such good care of themselves. It is not a priority for them. They do not maybe eat as well as they could or they have really disrupted sleep. I would like us to think about trauma as something that really does affect the whole body and our behavioral health choices. We should think broad, because those are the things that are really very important to ward off long-term diseases.
Dr. Jain: Yes, and enhance overall quality of life, too.
Dr. Yehuda: I think patients talk about what we (as healthcare professionals) want to talk about, and we lead the conversation in a symptom focused way. The symptoms of PTSD are impairing, don’t get me wrong, I am just saying there is a greater range of problems than are contained in the PTSD diagnosis.
Dr. Jain: I could not agree with you more. I feel like it is in the air. We are on the verge of embracing it that way. We are just not quite there yet.
Dr. Yehuda: I completely agree with you, and I think that the reason for that is that as we do our research on a genome wide level, we identify that so many of the biomarker pathways that seem to be altered relate to inflammatory immune functions. The pathways that are being identified in people with PTSD are not just those that associate with psychiatric symptoms, but really affect much more bodily functioning. I think that is also a lesson, just to close the loop on this that has been learned from the glucocorticoid story in PTSD. Cortisol is not just about mental health. There are glucocorticoid receptors in almost every cell in the body. Cortisol has a myriad of different functions in different target tissues, mostly in the metabolic systems promoting fuel and energy. It is silly to just think about cortisol’s role in traumatic memory when cortisol is a ubiquitous hormone that has so many different roles.
Unwarranted claims that “modifiable” negative beliefs cause Alzheimer’s disease lead to blaming persons who develop Alzheimer’s disease for not having been more positive.
Lesson: A source’s impressive credentials are no substitute for independent critical appraisal of what sounds like junk science and is.
More lessons on how to protect yourself from dodgy claims in press releases of prestigious universities promoting their research.
If you judge the credibility of health-related information based on the credentials of the source, this article is a clear winner:
Levy BR, Ferrucci L, Zonderman AB, Slade MD, Troncoso J, Resnick SM. A Culture–Brain Link: Negative Age Stereotypes Predict Alzheimer’s Disease Biomarkers. Psychology and Aging. Dec 7 , 2015, No Pagination Specified. http://dx.doi.org/10.1037/pag0000062
As noted in the press release from Yale University, two of the authors are from Yale School of Medicine, another is a neurologist at Johns Hopkins School of Medicine, and the remaining three authors are from the US National Institute on Aging (NIA), including NIA’s Scientific Director.
“Newly published research led by the Yale School of Public Health demonstrates that individuals who hold negative beliefs about aging are more likely to have brain changes associated with Alzheimer’s disease.
“The study suggests that combatting negative beliefs about aging, such as elderly people are decrepit, could potentially offer a way to reduce the rapidly rising rate of Alzheimer’s disease, a devastating neurodegenerative disorder that causes dementia in more than 5 million Americans.
The press release posited a novel mechanism:
“We believe it is the stress generated by the negative beliefs about aging that individuals sometimes internalize from society that can result in pathological brain changes,” said Levy. “Although the findings are concerning, it is encouraging to realize that these negative beliefs about aging can be mitigated and positive beliefs about aging can be reinforced, so that the adverse impact is not inevitable.”
A Google search reveals over 40 stories about the study in the media. Provocative titles of the media coverage suggest a children’s game of telephone or Chinese whispers in which distortions accumulate with each retelling.
Negative beliefs about aging tied to Alzheimer’s (Waltonian)
Distain for the elderly could increase your risk of Alzheimer’s (FinancialSpots)
Lack of respect for elderly may be fueling Alzheimer’s epidemic (Telegraph)
Negative thoughts speed up onset of Alzheimer’s disease (Tech Times)
Karma bites back: Hating on the elderly may put you at risk of Alzheimer’s (LA Times)
How you feel about your grandfather may affect your brain health later in life (Men’s Health News)
Young people pessimistic about aging more likely to develop Alzheimer’s later on (Health.com)
Looking forward to old age can save you from Alzheimer’s (Canonplace News)
If you don’t like old people, you are at higher risk of Alzheimer’s, study says (RedOrbit)
If you think elderly people are icky, you’re more likely to get Alzheimer’s (HealthLine)
In defense of the authors of this article as well as journalists, it is likely that editors added the provocative titles without obtaining approval of the authors or even the journalists writing the articles. So, let’s suspend judgment and write off sometimes absurd titles to editors’ need to establish they are offering distinctive coverage, when they are not necessarily doing so. That’s a lesson for the future: if we’re going to criticize media coverage, better focus on the content of the coverage, not the titles.
However, a number of these stories have direct quotes from the study’s first author. Unless the media coverage is misattributing direct quotes to her, she must have been making herself available to the media.
Was the article such an important breakthrough offering new ways in which consumers could take control of their risk of Alzheimer’s by changing beliefs about aging?
No, not at all. In the following analysis, I’ll show that judging the credibility of claims based on the credentials of the sources can be seriously misleading.
What is troubling about this article and its well-organized publicity effort is that information is being disseminated that is misleading and potentially harmful, with the prestige of Yale and NIA attached.
Before we go any further, you can take your own look at a copy of the article in the American Psychological Association journal Psychology and Aging here, the Yale University press release here, and a fascinating post-publication peer review at PubPeer that I initiated as peer 1.
Ask yourself: if you encountered coverage of this article in the media, would you have been skeptical? If so what were the clues?
The article is yet another example of trusted authorities exploiting entrenched cultural beliefs about the mind-body connection being able to be harnessed in some mysterious way to combat or prevent physical illness. As Ann Harrington details in her wonderful book, The Cure Within, this psychosomatic hypothesis has a long and checkered history, and gets continually reinvented and misapplied.
We see an example of this in claims that attitude can conquer cancer. What’s the harm of such illusions? If people can be led to believe they have such control, they are set up for blame from themselves and from those around them when they fail to fend off and control the outcome of disease by sheer mental power.
The myth of “fighting spirit” overcoming cancer that has survived despite the accumulation of excellent contradictory evidence. Cancer patients are vulnerable to blaming themselves for being blamed by loved ones when they do not “win” the fight against cancer. They are also subject to unfair exhortations to fight harder as their health situation deteriorates.
What I saw when I skimmed the press release and the article
The first alarm went off when I saw that causal claims were being made from a modest sized correlational study. This should set off anyone’s alarms.
The press release refers to this as a “first ever” d discussion section of the article refer to this as a “first ever” study. One does not seek nor expect to find robust “first ever” discoveries in such a small data set.
The authors do not provide evidence that their key measure of “negative stereotypes” is a valid measure of either stereotyping or likelihood of experiencing stress. They don’t even show it is related to concurrent reports of stress.
Like a lot of measures with a negative tone to items, this one is affected by what Paul Meehl calls the crud factor. Whatever is being measured in this study cannot be distinguished from a full range of confounds that not even being assessed in this study.
The mechanism by which effects of this self-report measure somehow get manifested in changes in the brain lacks evidence and is highly dubious.
There was no presentation of actual data or basic statistics. Instead, there were only multivariate statistics that require at least some access to basic statistics for independent evaluation.
The authors resorted to cheap statistical strategies to fool readers with their confirmation bias: reliance on one tailed rather than two-tailed tests of significance; use of a discredited backwards elimination method for choosing control variables; and exploring too many control/covariate variables, given their modest sample size.
The analyses that are reported do not accurately depict what is in the data set, nor generalize to other data sets.
The authors develop their case that stress is a significant cause of Alzheimer’s disease with reference to some largely irrelevant studies by others, but depend on a preponderance of studies that they themselves have done with the same dubious small samples and dubious statistical techniques. Whether you do a casual search with Google scholar or a more systematic review of the literature, you won’t find stress processes of the kind the authors invoke among the usual explanations of the development of the disease.
Basically, the authors are arguing that if you hold views of aging like “Old people are absent-minded” or “Old people cannot concentrate well,” you will experience more stress as you age, and this will accelerate development of Alzheimer’s disease. They then go on to argue that because these attitudes are modifiable, you can take control of your risk for Alzheimer’s by adopting a more positive view of aging and aging people
The authors used their measure of negative aging stereotypes in other studies, but do not provide the usual evidence of convergent and discriminant validity needed to establish the measure assesses what is intended. Basically, we should expect authors to show that a measure that they have developed is related to existing measures (convergent validity) in ways that one would expect, but not related to existing measures (discriminate validity) with which it should have associations.
The article reported two studies. The first tested whether participants holding more negative age stereotypes would have significantly greater loss of hippocampal volume over time. The study involved 52 individuals selected from a larger cohort enrolled in the brain-neuroimaging program of the Baltimore Longitudinal Study of Aging.
Readers are given none of the basic statistics that would be needed to interpret the complex multivariate analyses. Ideally, we would be given an opportunity to see how the independent variable, negative age stereotypes, is related to other data available on the subjects, and so we could get some sense if we are starting with some basic, meaningful associations.
Instead the authors present the association between negative age stereotyping and hippocampal volume only in the presence of multiple control variables:
Covariates consisted of demographics (i.e., age, sex, and education) and health at time of baseline-age-stereotype assessment, (number of chronic conditions on the basis of medical records; well-being as measured by a subset of the Chicago Attitude Inventory); self-rated health, neuroticism, and cognitive performance, measured by the Benton Visual Retention Test (BVRT; Benton, 1974).
Readers get cannot tell why these variables and not others were chosen. Adding or dropping a few variables could produce radically different results. But there are just too many variables being considered. With only 52 research participants, spurious findings that do not generalize to other samples are highly likely.
I was astonished when the authors announced that they were relying on one-tailed statistical tests. This is widely condemned as unnecessary and misleading.
Basically, every time the authors report a significance level in this article, you need to double the number to get what is obtained with a more conventional two-tailed test. So, if they proudly declare that results are significant p = .046, then the results are actually (non)significant, p= .092. I know, we should not make such a fuss about significance levels, but journals do. We’re being set up to be persuaded the results are significant, when they are not by conventional standards.
So the authors’ accumulating sins against proper statistical techniques and transparent reporting: no presentation of basic associations; reporting one tailed tests; use of multivariate statistics inappropriate for a sample that is so small. Now let’s add another one, in their multivariate regressions, the authors relied on a potentially deceptive backwards elimination:
Backward elimination, which involves starting with all candidate variables, testing the deletion of each variable using a chosen model comparison criterion, deleting the variable (if any) that improves the model the most by being deleted, and repeating this process until no further improvement is possible.
The authors assembled their candidate control/covariate variables and used a procedure that checks them statistically and drop some from consideration, based on whether they fail to add to the significance of the overall equation. This procedure is condemned because the variables that are retained in the equation capitalize on chance. Particular variables that could be theoretically relevant are eliminated simply because they fail to add anything statistically in the context of the other variables being considered. In the context of other variables, these same discarded variables would have been retained.
The final regression equation had fewer control/covariates then when the authors started. Statistical significance will be calculated on the basis of the small number of variables remaining, not the number that were picked over and so results will artificially appear stronger. Again, potentially quite misleading to the unwary reader.
The authors nonetheless concluded:
As predicted, participants holding more-negative age stereotypes, compared to those holding more-positive age stereotypes, had a significantly steeper decline in hippocampal volume
The second study:
examined whether participants holding more negative age stereotypes would have significantly greater accumulation of amyloid plaques and neurofibrillary tangles.
The outcome was a composite-plaques-and-tangles score and the predictor was the same negative age stereotypes measure from the first study. These measurements were obtained from 74 research participants upon death and autopsy. The same covariates were used in stepwise regression with backward elimination. Once again, the statistical test was one tailed.
As predicted, participants holding more-negative age stereotypes, compared to those holding more-positive age stereotypes, had significantly higher composite-plaques-and-tangles scores, t(1,59) = 1.71 p = .046, d = 0.45, adjusting for age, sex, education, self-rated health, well-being, and number of chronic conditions.
Aha! Now we see why the authors commit themselves to a one tailed test. With a conventional two-tailed test, these results would not be significant. Given a prevailing confirmation bias, aversion to null findings, and obsession with significance levels, this article probably would not have been published without the one tailed test.
The authors’ stirring overall conclusion from the two studies:
By expanding the boundaries of known environmental influences on amyloid plaques, neurofibrillary tangles, and hippocampal volume, our results suggest a new pathway to identifying mechanisms and potential interventions related to Alzheimer’s disease
Comments accumulated for a couple of days on PubPeer after I posted some concerns about the first study. All of the comments were quite smart, some directly validated points that I been thinking about, but others took the discussion in new directions either statistically or because the commentators knew more about neuroscience.
Using a mechanism available at PubPeer, I sent emails to the first author of the paper, the statistician, and one of the NIA personnel inviting them to make comments also. None have responded so far.
Tom Johnstone, a commentator who exercise the option of identifying himself noted the reliance on inferential statistics in the absence of reporting basic relationships. He also noted that the criterion used to drop covariates was lax. Apparently familiar with neuroscience, he expressed doubts that the results had any clinical significance or relevance to the functioning of the research participants.
Another commentator complained of the small sample size, use of one tailed statistical tests without justification, the “convoluted list of covariates,” and “taboo” strategy for selecting covariates to be retained in the regression equation. This commentator also noted that the authors had examined the effect of outliers, conducting analyses both with and without the inclusion of the most extreme case. While it didn’t affect the overall results, exclusion dramatically change the significance level, highlighting the susceptibility of such a small sample to chance variation or sampling error.
Who gets the blame for misleading claims in this article?
There’s a lot of blame to go around. By exaggerating the size and significance of any effects, the first author increases the chance of publication and also further funding to pursue what is seen as a “tantalizing” association. But it’s the job of editors and peer reviewers to protect the readership from such exaggerations and maybe to protect the author from herself. They failed, maybe because exaggerated findings are consistent with the journal‘s agenda of increasing citations by publishing newsworthy rather than trustworthy findings. The study statistician, Martin Slade obviously knew that misleading, less than optimal statistics were used, why didn’t he object? Finally, I think the NIA staff, particularly Luigi Ferrucci, the Scientific Director of NIA should be singled out for the irresponsibility of attaching their names to such misleading claims. Why they do so? Did they not read the manuscript? I will regularly present instances of NIH staff endorsing dubious claims, such as here. The mind-over-disease, psychosomatic hypothesis, gets a lot of support not warranted by the evidence. Perhaps NIH officials in general see this as a way of attracting research monies from Congress. Regardless, I think NIH officials have the responsibility to see that consumers are not misled by junk science.
This article at least provided the opportunity for an exercise that should raise skepticism and convince consumers at all levels – other researchers, clinicians, policymakers, and those who suffer from Alzheimer’s disease and those who care from them – we just cannot sit back and let trusted sources do our thinking for us.
Where is the HPA axis dysregulation? It is mainly in the minds of the authors, in service of their desired narrative. Were basal cortisol levels increased? No. Were peak cortisol levels increased? They didn’t say. Was the cortisol increment increased? Only if we accept a p value of 0.042 with no correction for multiple comparisons. Most importantly, was the termination of the stress cortisol response impaired? No, it wasn’t (Table 3). That variable is a feature of allostasis, about which co-author Wolkowitz is well informed. Termination of the stress response is a crucial component of HPA axis regulation (see PubMed #18282566), and it was no different between the two groups. So, where’s the beef? The weakness of this report tells us not only about the authors’ standards but also about the level of editorial tradecraft on display in Molecular Psychiatry. [Hyperlink added]
You also can see my response to Professor Carroll in the comments.
I transferred another comment to the blog from my Facebook wall. It gave me an opportunity to elaborate on why
we shouldn’t depend on small convenience samples to attempt to understand phenomena that must be examined in larger samples followed prospectively.
There are lots of unanswered questions about the authors’ sampling of adolescents. We don’t know what they are like when their mothers are not depressed. The young girls could also simply be reacting to environmental conditions contributing to their mother’s depression, not to their mother’s depression per se. We don’t know how representative this convenience sample is of other daughters of depressed mothers. Is it unusual or common that daughters of this age are not depressed concurrent with their mothers’ depression? What factors about the daughters, the mothers, or their circumstances determine that the mother and daughter depression does not occur at the same time? What about differences with him him dthe daughters of mothers who are prone to depression, but are not currently depressed? We need to keep in mind that most biomarkers associated with depression are state dependent, not trait dependent. And these daughters were chosen because they are not depressed…
But with no differences in cortisol response, what are we explaining anyway?
The Molecular Psychiatry article provides an excellent opportunity to learn to spot bad
science. I encourage interested readers to map what is said in that into the chart at the right.
This second installment of my two-part blog examines how the exaggerations and distortions of the article reverberate through a press release and then coverage in NIMH Director Thomas Insel’s personal blog.
The Stanford University press release headline is worthy of the trashy newspapers we find at supermarket checkouts:
The press release says things that didn’t appear in the article, but echoes the distorted literature review of the article’s introduction in claiming well-established links between shortened telomeres, frequent infections in chronic disease and death that just are not there.
The girls also had telomeres that were shorter by the equivalent of six years in adults. Telomeres are caps on the ends of chromosomes. Every time a cell divides the telomeres get a little shorter. Telomere length is like a biological clock corresponding to age. Telomeres also shorten as a result of exposure to stress. Scientists have uncovered links in adults between shorter telomeres and premature death, more frequent infections and chronic diseases.
And the claim of “the equivalent of six years” comes from direct quote from obtained from senior author Professor Ian Gotlib.
“It’s the equivalent in adults of six years of biological aging,” Gotlib said, but “it’s not at all clear that that makes them 18, because no one has done this measurement in children.”
Dr. Gotlib seems confused himself about what he mean by the 10 to 14-year-old girls having aged an additional six years. Does he really think that they are now 18? If so in what way? What could he possibly mean – do they look six years older than age matched controls? That would be really strange if they did.
I hope he lets us know when he figures out what he were saying, but he shouldn’t have given the statement to the Stanford press officer unless he was clear what he meant.
The press release noted that Dr. Gotlib had already moved on to intervention studies designed to prevent telomere shortening these girls.
In other studies, Gotlib and his team are examining the effectiveness of stress reduction techniques for girls. Neurofeedback and attention bias training (redirecting attention toward the positive) seem promising. Other investigators are studying techniques based on mindfulness training.
Gotlib’s claims play right into popular fantasies about rigging people up with some sort of apparatus that changes their brain. But everything changes the brain, even reading this blog post. I don’t think that reading this blog post has any less evidence for preventing later depression than neurofeedback. Nonetheless, I’m hoping that my blogging implants a healthy dose of skepticism in readers’ brains so that they are immunized against further confusion from exposure to such press releases. For an intelligent, consumer oriented discussion of neurofeedback, see Christian Jarrett’s
Attention bias training is a curious choice. It is almost as trendy as neurofeedback, but would it work? We have the benefit of a systematic review and recent meta-analysis that suggests a lack of evidence for attention bias training in treating depression and no evidence for preventing it. If it’s ineffectual in treating depression, how could we possibly expect it to prevent depression? Evidence please!
Let’s speculate about the implications if the authors found the cortisol differences between the daughters of the depressed mothers and daughters of controls that they had hypothesized but did not find. What then could have been done for these young girls? Note that the daughters of depressed mothers were chosen because they were functioning well, not currently depressed themselves. Just because they were different from the control girls would not necessarily indicate that any cortisol variables were in the abnormal range. Cortisol levels are not like blood pressure – we cannot specify a level below which cortisol levels have to be brought down for better health and functioning.
Note also that these daughters were selected on the basis of their mothers being depressed and that could mean the daughters themselves were facing a difficult situation. We can’t make the mother-bashing assumption that their mother’s depression was inflicting stress on them. Maybe any psychobiological stress response that was evident was due to the circumstances that led to the depression of their mother. We don’t know enough to specify what levels of cortisol variables would be optimal and consistent with good coping with the situation – we let even specify what is normal. And we don’t know how the daughters would recover from any abnormalities without formal treatment when their circumstances changed.
Bottom line is that these investigators did not get the results they hypothesized. Even if they had, results would not necessarily to lead to clinical applications.
Nonetheless, the director of NIMH saw fit to single this paper out or maybe he was just picking up on the press release.
Thomas Insel’s Personal Blog: Depression, Daughters, and Telomeres.
Thomas Insel’s Director’s Blog starts by acknowledging that there are no genetic or imaging markers predicting risk for depression, but research by Stanford Psychology Professor Ian Gotlib and colleagues in Molecular Psychiatry is “worth watching.”
Insel describes Gotlib’s “longitudinal” research as following depressed mothers’ early adolescent daughters.
The young girls have not yet developed depression, but 60 percent will become depressed by the age of 18.
I can find no basis in the article for Insel’s claim that Gotlib has found 60 per cent of these girls will be depressed by age 18. The estimate seems exaggerated, particularly given the case mix of mothers of these girls. It appears that some or most of the mothers were drawn from the community. We cannot expect severe course and biological correlates of depression that we would expect from a more inpatient sample.
Searching the papers coming out of this lab, I could only find one study involving a 30 month follow-up of 22 daughters of depressed mothers in the same age range as the sample in the Molecular Psychiatry article. That’s hardly a basis for the strong claim of 60% becoming depressed by 18.
Insel embellishes the importance of differences in telomere length. He perpetuates the illusion that we can be confident that differences in telomere length suggest these girls were experiencing accelerated aging and what have high risk for disease when the girls reached middle and late age. Without the backing of data from the paper or the existing literature, Insel zeros in on
Troubling early sign of risk for premature biological aging and possibly age-related chronic diseases, such as cardiovascular disease. Investigating the cause and timing of decreased telomere length—to what extent it may result from abnormalities in stress responses or is genetically influenced, for example—will be important for understanding the relationship between cellular aging, depression, and other medical conditions.
Insel ponders how such young, healthy girls could possibly show signs of aging. According to him the answer is not clear, but it might be tied to the increased stress reactivity these girls show in performing laboratory tasks.
But as Professor Caroll noted, the study just does not much evidence of “increased stress reactivity.”
Nonetheless, Insel indicates that Gotlib’s next step is
Using neurofeedback to help these girls retrain their brain circuits and hopefully their stress responses. It will be a few years before we will know how much this intervention reduces risk for depression, but anything that prevents or slows the telomere shortening may be an early indication of success.
It’s interesting that Insel sidestepped the claim in the press release that Gotlib was trying out a cognitive behavioral intervention to affect stress reactivity. Instead he presents a fanciful notion that neural feedback will somehow retrain these girls’ brain circuits and reduce their stress response throughout their time at home and prevent them getting depressed by their mother’s depression.
Oh, if that were only so: Insel would be vindicated in his requiring for funding that researchers get down to basic mechanisms and simply bypass existing diagnoses with limited reliability, but at least some ties to patients’ verbal reports of why they are seeking treatment. In his world of science fiction, patients, or at least these young girls, which come in to have their brains retrained to forestall the telomere shortening that is threatening them not only with becoming depressed later, but with chronic diseases and middle and late life and early death.
So, let’s retrace what was said in the original Molecular Psychiatry article to what was claimed in the Stanford University press release and what was disseminated in the social media of Dr. Insel’s personal blog. Authors’ spin bad science in a peer-reviewed article. They collaborate with their university’s press relations department by providing even more exaggerated claims. And Dr. Insel’s purpose is served by simply passing them on and social media.
There’s a lot in Dr. Insel’s Personal Blog to disappoint and even outrage
Researchers seeking guidance for funding priorites.
Clinicians in the trenches needing to do something now to deal with the symptoms and simple misery that are being presented to them.
Consumers looking for guidance from the Director of NIMH as to whether they should be concerned about their daughters and what they should do about it.
A lot of bad science and science fiction is being served to back up false promises about anything likely to occur in our lifetimes, if ever.
Taxpayers need to appreciate where Dr. Insel is taking funding of mental health with research. He will no longer fund grants that will explore different psychotherapeutic strategies for common mental health problems as they are currently understood – you know, diagnoses tied to what patients complain about. Instead he is offering a futuristic vision in which we no longer have to pay for primary care physicians or mental health clinicians spending time talking to patients about the problems in their lives. Rather, patients can bring in a saliva sample to assess the telomere length. They then can be rigged up to a videogame providing a social stress challenge. They will then be given neurofeedback and asked to provide another saliva sample. If the cortisol levels aren’t where they are supposed to be, they will come back and get some more neurofeedback and videogames.
But wait! We don’t even need to wait until people develop problems in their lives. We can start collecting spit samples when they are preteens and head off any problems developing in their life with neural feedback.
Presumably all this could be done by technicians who don’t need to be taught communication skills. And if the technicians are having problems, we can collect spit samples from them and maybe give them some neurofeedback.
Sure, mild to moderate depression in the community is a large and mixed grouping. The diagnostic category major depression loses some of its already limited reliability and validity when applied to this level of severity. But I still have a lot more confidence in this diagnosis than relying on some unproven notions about treating telomere length and cortisol parameters in people who do not currently complain about mental health or their circumstances. And the lamer notion that this can be done without any empathy or understanding.
He acknowledged some of the serious barriers to the development of valid, clinically useful biomarkers:
Patients with mental disorders show many biological abnormalities which distinguish them from normal volunteers; however, few of these have led to tests with clinical utility. Several reasons contribute to this delay: lack of a biological ‘gold standard’ definition of psychiatric illnesses; a profusion of statistically significant, but minimally differentiating, biological findings;‘approximate replications’ of these findings in a way that neither confirms nor refutes them; and a focus on comparing prototypical patients to healthy controls which generates differentiations with limited clinical applicability. Overcoming these hurdles will require a new approach. Rather than seek biomedical tests that can ‘diagnose’ DSM-defined disorders, the field should focus on identifying biologically homogenous subtypes that cut across phenotypic diagnosis—thereby sidestepping the issue of a gold standard.
All but the last sentence could have been part of a negative review of the Molecular Psychiatry article or the grant that provided funding for it. But the last sentence is the kind of nonsense that a director of NIMH can lay on the and research community and expect it to be reflected in their grant applications.
But just what was the theme of this other blog post from Dr. Insel? P-hacking and the crisis concerning results of biomedical research not being consistently reproducible.
The relentless quest for a significant “P” value is only one of the many problems with data analysis that could contribute to the reproducibility problem. Many mistakenly believe that “P” values convey information about the size of the difference between two groups. P values are actually only a way of estimating the likelihood that the difference you observe could have occurred by chance. In science, “significance” usually means a P value of less than 0.05 or 1 in 20, but this does not mean that the difference observed between two groups is functionally important. Perhaps the biggest problem is the tendency for scientists to report data that have been heavily processed rather than showing or explaining the details. This suggests one of the solutions for P-hacking and other problems in data analysis: provide the details, including what comparisons were planned prior to running the experiment.
Maybe because Insel is Director of NIMH, he doesn’t expect anybody to call him on the contradictions in what he is requesting. In the p-hacking blog post, he endorsed a call to action to address the problem of a lot of federal money being wasted on research that can’t lead to improvements in the health and well-being of the population because the research is simply unreliable and depends on “heavily processed” data for which investigators don’t provide the details. Yet in the Depression, Daughters, and Telomeres post he grabs an outrageous example of this being done and tells the research community he wants to see more of it.