The PACE PLOS One data will not be released and the article won’t be retracted

PLOS One has bought into discredited arguments about patient consent forms not allowing sharing of anonymized data. PLOS One is no longer at the vanguard of open science through routine data sharing.

mind the brain logo

Two years have passed since I requested release of the PLOS One PACE data, eight months since the Expression of Concern was posted. What can we expect?

expression of concern-page-0

9 dot problem
Solving the 9-dot problem involves paying attention and thinking outside the box.

If we spot some usually unrecognized connections, we can see the PLOS One editors are biased towards the PACE investigators, favoring them over other stakeholders in whether the data are released as promised..

Spoiler: The PLOS One Senior Editors completed the pre-specified process of deciding what to do about the data not being shared.  They took no action. Months later the Senior Editors reopened the process and invited one of PACE investigators Trudy Chalder’s outspoken co-authors to help them reconsider.

A lot of us weren’t cynical enough to notice.

International trends will continue toward making uploading data into publicly accessible repositories a requirement for publication. PLOS One has slowed down by buying into discredited arguments about patient consent forms not allowing sharing of anonymized data.

PLOS One is no longer at the vanguard of open science through routine data sharing.

The expression of concern

actual display of expression of concern on PLOS article
Actual Expression of Concern on display on PLOS One article.

The editors’ section of the Expression of Concern ends with:

In spite of requests to the authors and Queen Mary University of London, we have not yet received confirmation that an institutional process compatible with the existing PLOS data policy at the time has been developed or implemented for the independent evaluation of requests for data from this study. We conclude that the lack of resolution towards release of the dataset is not in line with the journal’s editorial policy and we are thus issuing this Expression of Concern to alert readers about the concerns raised about this article.

This is followed by the PACE investigators’ response:

Statement from the authors

We disagree with the Expression of Concern about our health economic paper that PLOS ONE has issued and do not accept that it is justified. We believe that data should be made available and have shared data from the PACE trial with other researchers previously, in line with our data sharing policy. This is consistent with the data sharing policies of Queen Mary University of London, and the Medical Research Council, which funded the trial. The policy allows for the sharing of data with other researchers, so long as safeguards are agreed regarding confidentiality of the data and consent as specified by the Research Ethics Committee (REC). We have also pointed out to PLOS ONE that our policy includes an independent appeal process, if a request is declined, so this policy is consistent with the journal’s policy when the paper was published.

During negotiations with the journal over these matters, we have sought further guidance from the PACE trial REC. They have advised that public release, even of anonymised data, is not appropriate. As a consequence, we are unable to publish the individual patient data requested by the journal. However, we have offered to provide key summarised data, sufficient to provide an independent re-analysis of our main findings, so long as it is consistent with the REC decision, on the PLOS ONE website. As such we are surprised by and question the decision by the journal to issue this Expression of Concern.

Check out my critique of their claim to have shared data from the PACE trial with other researchers-

Don’t bother to apply: PACE investigators issue guidance for researchers requesting access to data.

Nothing_to_DeclareConflict of interest: Nothing to declare?

 The PACE authors were thus given an extraordinary opportunity to undermine the editors’ Expression of Concern.

It is just as extraordinary that there is no disclosure of conflict of interest. After all, it is their paper is receiving expression of concern because of the authors’ failure to provide data as promised.

In contrast, when the PLOS One editors placed a discreet Editors Note in 2015 in the comment section of the article about the data not being shared when requested, it carried a COI declaration:

Competing interests declared: PLOS ONE Staff

That COI aroused the curiosity of Retraction Watch who asked PLOS One:

We weren’t sure what the last line was referring to, so contacted Executive Editor Veronique Kiermer. She told us that staff sometimes include their byline under “competing interests,” so the authorship is immediately clear to readers who may be scanning a series of comments.

Commentary from Retraction Watch

PLOS upgrades flag on controversial PACE chronic fatigue syndrome trial; authors “surprised”

Notable excerpts:

A spokesperson for PLOS told us this is the first time the journal has included a statement from the authors in an EOC:

This has been a complex case involving many stakeholders and we wanted to document the different aspects of the case in a fair manner.

And

We asked if the journal plans to retract the paper if the authors fail to provide what it’s asked for; the spokesperson explained:

At this time, PLOS stands by its Expression of Concern. For now, we have exhausted the options to make the data available in accordance with our policy at the time, but PLOS still seeks a positive outcome to this case for all parties. It is our intention to update this notice when a mechanism is established that allows concerns about the article’s analyses to be addressed while protecting patient privacy. PLOS has not given the authors a deadline.

Note: “PLOS did not given the authors a deadline.”

One of the readers who has requested the data is James Coyne, a psychologist at the University Medical Center, Groningen, who submitted his request 18 months ago (and wrote about it on the PLOS blog site). Although some of the data have been released (to one person under the Freedom of Information Act), it’s not nearly enough to conduct an analysis, Coyne told us:

This small data set does not allow recalculation of original primary outcomes but did allow recalculation of recovery data. Release of the PLOS data is crucial for a better understanding of what went on in that trial. That’s why the investigators are fighting so hard.

Eventually, Coyne began suggesting to PLOS that he would organize public protests and scientific meetings attended by journal representatives.

I think it is the most significant issue in psychotherapy today, in terms of data sharing. It’s a flagrant violation of international standards.

The Retraction Watch article cited a 2015 STAT article that was written by Retraction Watch co-founders Ivan Oransky and Adam Marcus. That article was sympathetic to my request:

If the information Coyne is seeking is harmful and distressing to the staff of the university — and that’s the university’s claim, not ours — that’s only because the information is in fact harmful and distressing. In other words, revealing that you have nothing to hide is much less embarrassing than revealing that you’re hiding something.

The STAT article also said:

To be clear, Coyne’s not asking for sex tapes or pictures of lab workers taking bong hits. He’s asking for raw data so that he can evaluate whether what a group of scientists reported in print is in fact what those data show. It’s called replication, and as Richard Smith, former editor of The BMJ (and a member of our board of directors), put it last week, the refusal goes “against basic scientific principles.” But, unfortunately, stubborn researchers and institutions have used legal roadblocks before to prevent scrutiny of science.

The PLOS One Editors’ blog  post.

The Expression of Concern was accompanied by a blog post from PLOS Iratxe Puebla, Managing Editor for PLOS ONE and Joerg Heber, Editor-in-Chief on May 2, 2017

Data sharing in clinical research: challenges and open opportunities

Since we feel we have exhausted the options to make the data available responsibly, and considering the questions that were raised about the validity of the article’s conclusions, we have decided to post an Expression of Concern [5] to alert readers that the data are not available in line with the journal’s editorial policy. It is our intention to update this notice when a mechanism is established that allows concerns about the article’s analyses to be addressed while protecting patient privacy.

This statement seems to suggest that the ball is in the PACE investigators’ court and that PLOS One editors are prepared to wait. But reading the rest of the blog post, it becomes apparent that PLOS One is wavering on the data sharing policy

Current challenges and opportunities ahead

During our follow up it became clear that there is little consensus of opinion on the sharing of this particular dataset. Experts from the Data Advisory Board whom we consulted expressed different views on the stringency of the journal reaction. Overall they agreed on the need to consider the risk to confidentiality of the trial participants and on the relevance of developing mechanisms for consideration of data requests by an independent body or committee. Interestingly, the ruling of the FOI Tribunal also indicated that the vote did not reflect a consensus among all committee members.

Fact checking the PLOS One’s Editors’ blog and a rebuttal

John Peter fact checked  the PLOS One editors’ blog. It came up short on a number of points.

“Interestingly, the ruling of the FOI Tribunal also indicated that the vote did not reflect a consensus among all committee members.”

This line is misleading and reveals either ignorance or misunderstanding of the decision in Matthees.

The Information Tribunal (IT) is not a committee. It is part of the courts system of England and Wales.

…the IT’s decisions may be appealed to a higher court. As QMUL chose not to exercise this right but to opt instead to accept the decision, then clearly it considered there were no grounds for appeal. The decision stands in its entirety and applies without condition or caveat.

And

The court had two decisions to make:

First, could and should trial data be released and if so what test should apply to determine whether particular data should be made public? Second, when that test is applied to this particular set of data, do they meet that test?

The unanimous decision on the first question was very clear: there is no legal or ethical consideration which prevents release; release is permitted by the consent forms; there is a strong public interest in the release; making data available advances legitimate scientific debate; and the data should be released.

The test set by this unanimous decision was simple: whether data can be anonymized. Furthermore, again unanimously, the Tribunal stated that the test for anonymization is not absolute. It is whether the risk of identification is reasonably likely, not whether it is remote, and whether patients can be identified without prior knowledge, specialist knowledge or equipment, or resort to criminality.

It was on applying this test to the data requested, on whether they could be properly anonymized, that the IT reached a majority decision.

On the principles, on how these decisions should be made, on the test which should be applied and on the nature of that test, the court was unanimous.

It should also be noted that to share data which have not been anonymized would be in breach of the Data Protection Act. QMUL has shared these data with other researchers. QMUL should either report itself to the Information Commissioner’s Office or accept that the data can be anonymized. In which case, the unanimous decision of the IT is very clear: the data should be shared.

PLOS ONE should apply the IT decision and its own regulations and demand the data be shared or the paper retracted.

Data Advisory Board

The Editors’ blog referred to “Experts from the Data Advisory Board.. express[ing] different views on the stringency of the journal reaction.”

That was a source of puzzlement for me. Established procedures make no provision for an advisory board as part of the process or any appeal.

A Google Search clarified. I had been to this page a number of times before and did not remember seeing this statement. There is no date or any indication it was added after the rest of the statement.

PLOS has formed an external board of advisors across many fields of research published in PLOS journals. This board will work with us to develop community standards for data sharing across various fields, provide input and advice on especially complex data-sharing situations submitted to the journals, define data-sharing compliance, and proactively work to refine our policy. If you have any questions or feedback, we welcome you to write to us at data@plos.org.

The availability of data from reanalysis and independent probing has lots of stakeholders. Independent investigators, policymakers, and patients all have a stake. I don’t recognize the names on this list and see no indication that consumers affected by what is reported in clinical and health services papers have role in making decisions about the release of data. But one name stands out.

Who is Malcolm Macleod and what is he doing in this decision-making process?

Malcolm Macleod is quoted in the Science Media Centre reaction to the PACEgate special issue:

 Expert reaction to Journal of Health Psychology’s Special Issue on The PACE Trial

Prof. Malcolm Macleod, Professor of Neurology and Translational Neuroscience, University of Edinburgh, said:

“The PACE trial, while not perfect, provides far and away the best evidence for the effectiveness of any intervention for chronic fatigue; and certainly is more robust than any of the other research cited. Reading the criticisms, I was struck by how little actual meat there is in them; and wondered where some of the authors came from. In fact, one of them lists as an institution a research centre (Soerabaja Research Center) which only seems to exist as an affiliation on papers he wrote criticising the PACE trial.

“Their main criticisms seem to revolve around the primary outcome was changed halfway through the trial: there are lots of reasons this can happen, some justifiable and others not; the main think is whether it was done without knowledge of the outcomes already accumulated in the trial and before data lock – which is what was done here.

“So I don’t think there is really a story here, apart from a group of authors, some of doubtful provenance, kicking up dust about a study which has a few minor wrinkles (as all do) but still provides information reliable enough to shape practice. If you substitute ‘CFS’ for ‘autism’ and ‘PACE trial’ for ‘vaccination’ you see a familiar pattern…”

The declaration of interest is revealing in what it says and what it does not say.

Prof. MacLeod: “Prof Sharpe used to have an office next to my wife’s; and I sit on the PLoS Data board that considered what to do about one of their other studies.

The declaration fails to reveal a recent publication co-authored by Macleod and Trudy  Chalder.

Wu S, Mead G, Macleod M, Chalder T. Model of understanding fatigue after stroke. Stroke. 2015 Mar 1;46(3):893-8.

This press release comes from an organization strongly committed to the protection of the PACE trial from independent scrutiny. The SMC even organized a letter writing campaign headed by Peter White to petition Parliament to exclude universities for Freedom of Information Act requests. Of course, that will effectively block request for data.

Why would the PLOS One editors involved such a person to reconsider what been a decision in favor of releasing the data?

Connect the dots.

Trends will continue toward making uploading data into publicly accessible repositories a requirement for publication. PLOS One has bought into discredited arguments about patient consent forms not allowing sharing of anonymized data. PLOS One is no longer at the vanguard of open science through routine data sharing.

Better days: When PLOS Blogs honored my post about fatal flaws in the PACE chronic fatigue syndrome follow-up study (2015)

The back story on my receiving this honor was that PLOS Blogs only days before had shut down the blog site because of complaints from someone associated with the PACE trial. I was asked to resign. I refused. PLOS Blogs relented when I said it would be a publicity disaster for PLOS Blogs.

mind the brain logoThe back story on my receiving this honor was that PLOS Blogs only days before had shut down the blog site because of complaints from someone associated with the PACE trial. I was asked to resign. I refused. PLOS Blogs relented when I said it would be a publicity disaster for PLOS Blogs.

screen shot 11th most accessedA Facebook memory of what I was posting two years ago reminded me of better days when PLOS Blogs honored my post about the PACE trial.

Your Top 15 in ’15: Most popular on PLOS BLOGS Network

I was included in a list of the most popular blog posts in a network that received over 2.3 million visitors reading more than 600 new posts. [It is curious that the sixth and seventh most popular posts were omitted from this list, but that’s another story]

I was mentioned for number 11:

11) Uninterpretable: Fatal flaws in PACE Chronic Fatigue Syndrome follow-up study Mind the Brain 10/29/15

Investigating and sharing potential errors in scientific methods and findings, particularly involving psychological research, is the primary reason Clinical Health Psychologist (and PLOS ONE AE) Jim Coyne blogs on Mind the Brain and elsewhere. This closely followed post is one such example.

Earlier decisions by the investigator group preclude valid long-term follow-up evaluation of CBT for chronic fatigue syndrome (CFS). At the outset, let me say that I’m skeptical whether we can hold the PACE investigators responsible… Read more

The back story was that only days before, I had gotten complaints from readers of Mind the Brain who found they were blocked from leaving comments at my blog site. I checked and found that I couldn’t even access the blog as an author.

I immediately emailed Victoria Costello and asked her what it happened. We agreed to talk by telephone, even though it was already late night where I was in Philadelphia. She was in the San Francisco PLOS office.

In the telephone conversation,  I was reminded me that there were some topics about which was not supposed to blog. Senior management at PLOS found me in violation of that prohibition and wanted me to stop blogging.

As is often the case with communication with the senior management of PLOS, no specifics had been given.  There was no formal notice or disclosure about what topics I couldn’t blog or who had complained. And there had been no warning when my access to the blog site was cut. Anything that I might say publicly could be met with a plausible denial.

I reminded Victoria that I had never received any formal specification about what I could blog nor from whom the complaint hand come. There had been a vague communication from her about not blogging about certain topics. I knew that complaints from either Gabrielle Oettingen or her family members had led to request the blog about the flaws in her book,  Rethinking Positive Thinking . That was easy to do because I was not planning another post about that dreadful self-help book.  Any other prohibition was left so vague that had no idea that I couldn’t blog about the PACE trial. I had known that the authors of the British Psychological Society’s Understanding Psychosis were quite upset with what I had said in heavily accessed blog posts. Maybe that was the source of the other prohibition, but no one made that clear. And I wasn’t sure I wanted to honor it, anyway.

I pressed Victoria Costello for details. She said an editor had complained. When I asked if it was Richard Horton, she paused and mumbled something that I took as an affirmative. Victoria then suggested that  it would be best for the blog network and myself if we had a mutually agreed-upon parting of ways. I told her that I would probably publicly comment that the breakup was not mutual and it would be a publicity disaster for the blog.

igagged_jpg-scaled500Why I was even blogging for PLOS Blogs? Victoria Costello had recruited me over after I expressed discontent with the censorship that I was receiving at Psychology Today. The PT editors there had complained that some of my blogging about antidepressants might discourage ads from pharmaceutical companies for which they depended for revenue. The editors had insisted on  the right to approve my posts before I uploaded them. In inviting me to PLOS Blogs, Victoria told me that she too was a refugee from blogging at Psychology Today.  I wouldn’t have to worry about restrictions on what I could say at Mind the Brain, beyond avoiding libel.

I ended the conversation accepting the prohibition about blogging about the PACE trial. This is was despite disagreeing with the rationale that it would be a conflict of interest for me to blog about it after requesting the data from the PLOS One paper.

Since then, I repeatedly requested that the PLOS management acknowledge the prohibition on my blogging or at least put it in writing. My request was met with repeated refusals from Managing Editor Iratxe Puebla, who always cited my conflict of interest.

In early 2017, I began publicly tweeting about the issue, stimulating some curiosity others about whether there was a prohibition. InJuly 2017, the entire Mind the Brain site, not just my blog, was shut.

In early 2018, I will provide more backstory on that shutdown and dispute what was said in the blog post below. And more about the collusion between PLOS One senior management and the PACE investigators in the data not being available 2 years after I requested it.

Message for Mind the Brain readers from PLOSBLOGS

blank plos blogs thumb nail
This strange thumbnail is the default for when no preferred image is provided. It could indicate the haste with which this blog was posted.

Posted July 31, 2017 by Victoria Costello in Uncategorized

After five years and over a hundred posts, PLOSBLOGS is retiring its psychology blog, Mind the Brain, from our PLOS-hosted blog network. By mutual agreement with the primary Mind the Brain blogger, James Coyne, Professor Coyne will retain the name of this blog and will take his archive of posts for reuse on his independent website, http://www.coyneoftherealm.com.

According to PLOSBLOGS’ policy for all our retired (inactive) blogs, any and all original posts published on Mind the Brain will retain their PLOS web addresses as intact urls, so links made previously from other sites will not be broken. In addition, PLOS will supply the archive of his posts directly to Prof Coyne so that he may repost them anywhere he may wish.

PLOS honors James Coyne’s voice as an important one in peer-to-peer scientific criticism. As discussed with Professor Coyne in recent days, after careful consideration PLOSBLOGS has concluded that it does not have the staff resources required to vet the sources, claims and tone contained in his posts, to assure they are aligned with our PLOSBLOGS Community Guidelines. This has lead us to the conclusion that Professor Coyne and his content would be better served on his own independent blog platform. We wish James Coyne the best with his future blogging.

—Victoria Costello, Senior Editor, PLOSBLOGS & Communities

Bollocks!

Is Donald Trump suffering from Pick’s Disease (frontotemporal dementia)?

Changing the conversation about Donald Trump’s fitness for office from whether he has a personality disorder to whether he has an organic brain disorder.

mind the brain logoChanging the conversation about Donald Trump’s fitness for office from whether he has a personality disorder to whether he has an organic brain disorder.

Trump.jpgFor a long while there has been an ongoing debate about whether Donald Trump suffers from a personality disorder that might contribute to his being unfit the President of the United States. Psychiatrists have ethical constraints in what they say because of the so-called Goldwater rule, barring them from commenting on the mental health of political figures that they have not personally  interviewed.

I am a clinical psychologist, not a psychiatrist. I feel the need to speak out that the behavior of Donald Trump is abnormal and we should caution against normalizing it. The problem with settling on his behavior being simply that of a bad person or con man is it doesn’t prepare us for just how erratic his behavior can be.

I’ll refrain from making a formal psychiatric diagnosis. I actually think that in clinical practice, a lot of mental health professionals too casually make diagnoses of personality disorders for patients (or privately, even for colleagues) they find difficult or annoying.  If they ever gave these people a structured interview,  I suspect they would be found to fall  below the threshold for any particular personality disorder.

Changing the conversation

But now an article in Stat has changed the conversation to whether Donald Trump suffers from personality disorder to whether he is developing an organic brain disorder.

I’m a brain specialist. I think Trump should be tested for a degenerative brain disease

When President Trump slurred his words during a news conference this week, some Trump watchers speculated that he was having a stroke. I watched the clip and, as a physician who specializes in brain function and disability, I don’t think a stroke was behind the slurred words. But having evaluated the chief executive’s remarkable behavior through my clinical lens for almost a year, I do believe he is displaying signs that could indicate a degenerative brain disorder.

As the president’s demeanor and unusual decisions raise the potential for military conflict in two regions of the world, the questions surrounding his mental competence have become urgent and demand investigation.

And

I see worrisome symptoms that fall into three main categories: problems with language and executive function; problems with social cognition and behavior; and problems with memory, attention, and concentration. None of these are symptoms of being a bad or mean person. Nor do they require spelunking into the depths of his psyche to understand. Instead, they raise concern for a neurocognitive disease process in the same sense that wheezing raises the alarm for asthma.

In addition to being a medical journalist, the author Ford Vox of the article is a neurorehabilitation physician who is board-certified physical medicine and rehabilitation physician with additional subspecialty board certification in brain injury medicine.

I was alerted by the possibility of a diagnosis of frontotemporal dementia by a tweet by Barney Carroll. He is a senior psychiatrist whom I have come to trust as a mentor on social media, even though we’ve never overlapped in the same department at the same time.

barney forget psychnoanalysis

And then there was this tweet about the Stat story, but I could judge its credibility because I did not know the tweeter or her source:

trump's disease

I followed up with a Google search and came across an article from August 2016, before the election:

Finally figured out Trump’s medical diagnosis after watching this:

It’s called Pick’s Disease, or frontotemporal dementia

Look at the symptoms, all of these which fit Trump quite closely:

  • Impulsivity and poor judgment
  • Extreme restlessness (early stages)
  • Overeating or drinking to excess
  • Sexual exhibitionism or promiscuity
  • Decline in function at work and home
  • Repetitive or obsessive behavior

And especially these, listed earlier in the article:

Excess protein build-up causes the frontal and temporal lobes of the brain, which control speech and personality, to slowly atrophy. 

Then I followed up with more Google searches, hitting MedLine Plus,  the website maintained by the National Institutes of Health’s Web site for patients and their families and friends and produced by the National Library of Medicine.

Pick disease

Pick disease is a rare form of dementia that is similar to Alzheimer disease, except that it tends to affect only certain areas of the brain.

Causes

People with Pick disease have abnormal substances (called Pick bodies and Pick cells) inside nerve cells in the damaged areas of the brain.

Pick bodies and Pick cells contain an abnormal form of a protein called tau. This protein is found in all nerve cells. But some people with Pick disease have an abnormal amount or type of this protein.

The exact cause of the abnormal form of the protein is unknown. Many different abnormal genes have been found that can cause Pick disease. Some cases of Pick disease are passed down through families.

Pick disease is rare. It can occur in people as young as 20. But it usually begins between ages 40 and 60. The average age at which it begins is 54.

Symptoms

The disease gets worse slowly. Tissues in parts of the brain shrink over time. Symptoms such as behavior changes, speech difficulty, and problems thinking occur slowly and get worse.

Early personality changes can help doctors tell Pick disease apart from Alzheimer disease. (Memory loss is often the main, and earliest, symptom of Alzheimer disease.)

People with Pick disease tend to behave the wrong way in different social settings. The changes in behavior continue to get worse and are often one of the most disturbing symptoms of the disease. Some persons have more difficulty with decision making, complex tasks, or language (trouble finding or understanding words or writing).

The website notes

A brain biopsy is the only test that can confirm the diagnosis.

However, some alternative diagnoses can be ruled out:

Your doctor might order tests to help rule out other causes of dementia, including dementia due to metabolic causes. Pick disease is diagnosed based on symptoms and results of tests, including:

Assessment of the mind and behavior (neuropsychological assessment)

Brain MRI

Electroencephalogram (EEG)

Examination of the brain and nervous system (neurological exam)

Examination of the fluid around the central nervous system (cerebrospinal fluid) after a lumbar puncture

Head CT scan

Tests of sensation, thinking and reasoning (cognitive function), and motor function

Back to Ford Vox in his Stats article:

In Trump’s case, we have no relevant testing to review. His personal physician issued a thoroughly unsatisfying letter before the election that didn’t contain much in the way of hard data. That’s a situation many people want to correct via an independent medical panel that can objectively evaluate the president’s fitness to serve. But the prospects for getting Congress to use the 25th Amendment in this way seem poor at the moment.

What we do have are a growing array of signs and symptoms displayed in public for all to see. It’s time to discuss these issues in a clinical context, even if this is a very atypical form of examination. It’s all we have. And even if the president has a physical exam early next year and releases the records, as announced by the White House, what he really needs is thorough cognitive testing.

So?

Before biting the bullet, I also spoke with Dr. Dennis Agliano, who chairs the AMA’s Council on Ethical and Judicial Affairs, the panel that wrote the new ethical guidance. He advised me to be careful: “You can get yourself into hot water, since there are people who like Trump, and they may submit a complaint to the AMA,” the Tampa otolaryngologist told me. Ultimately, he reassured me that I should just do what I think is right.

Which is warn the president that he needs to be evaluated for a brain disease.

Good luck, Dr Vox, but at least we have a reasonable hypothesis on the table. As Barney Carroll says “Time will tell.”

slurred speech

Using F1000 “peer review” to promote politics over evidence about delivering psychosocial care to cancer patients

The F 1000 platform allowed authors and the reviewers whom they nominated to collaborate in crafting more of their special interest advocacy that they have widely disseminated elsewhere. Nothing original in this article and certainly not best evidence!

 

mind the brain logo

A newly posted article on the F1000 website raises questions about what the website claims is a “peer-reviewed” open research platform.

Infomercial? The F1000 platform allowed authors and the reviewers whom they nominated to collaborate in crafting more of their special interest advocacy that they have widely disseminated elsewhere. Nothing original in this article and certainly not best evidence!

I challenge the authors and the reviewers they picked to identify something said in the F1000 article that they have not said numerous times before either alone or in papers co-authored by some combination of authors and the reviewers they picked for this paper.

F1000 makes the attractive and misleading claim that versions of articles that are posted on its website reflect the response to reviewers.

Readers should be aware of uncritically accepting articles on the F 1000 website as having been peer-reviewed in any conventional sense of the term.

Will other special interests groups exploit this opportunity to brand their claims as “peer-reviewed” without the risk of having to tone down their claims in peer review? Is this already happening?

In the case of this article, reviewers were all chosen by the authors and have a history of co-authoring papers with the authors of the target paper in active advocacy of a shared political perspective, one that is contrary to available evidence.

Cynically, future authors might be motivated to divide their team, with some remaining authors and others dropping off to become nominated as reviewers. They could then suggest content that had already been agreed would be included, but was left off for the purposes being suggested in the review process

F1000

F1000Research bills itself as

An Open Research publishing platform for life scientists, offering immediate publication of articles and other research outputs without editorial bias. All articles benefit from transparent refereeing and the inclusion of all source data.

Material posted on this website is labeled as having received rapid peer-review:

Articles are published rapidly as soon as they are accepted, after passing an in-house quality check. Peer review by invited experts, suggested by the authors, takes place openly after publication.

My recent Google Scholar alert call attention to an article posted on F1000

Advancing psychosocial care in cancer patients [version 1; referees: 3 approved]

 Who were the reviewers?

open peer review of Advancing psychosocial care

Google the names of authors and reviewers. You will discover a pattern of co-authorship; leadership positions in international Psycho-Oncology society, a group promoting the mandating of specially mental health services for cancer patients, and lots of jointly and separately authored articles making a pitch for increased involvement of mental health professionals in routine cancer care. This article adds almost nothing to what is multiply available elsewhere in highly redundant publications

Given a choice of reviewers, these authors would be unlikely to nominate me. Nonetheless, here is my review of the article.

 As I might do in a review of a manuscript, I’m not providing citations for these comments, but support can readily be found by a search of blog posts at my website @CoyneoftheRealm.com and Google Scholar search of my publications. I welcome queries from anybody seeking documentation of these points below.

 Fighting Spirit

The notion that cancer patients having a fighting spirit improves survival is popular in the lay press and in promoting the power of the mind over cancer, but it has thoroughly been discredited.

Early on, the article identifies fighting spirit as an adaptive coping style. In actuality, fighting spirit was initially thought to predict mortality in a small methodologically flawed study. But that is no longer claimed.

Even one of the authors of the original study, Maggie Watson,  expressed relief when her own larger, better designed study failed to confirm the impression that a fighting spirit extended life after diagnosis  of cancer. Why? Dr. Watson was concerned that the concept was being abused in blaming cancer patients who were dying there was to their personal deficiency of not having enough fighting spirit.

Fighting spirit is rather useless as a measure of psychological adaptation. It confounds severity of cancer enrolled dysfunction with efforts to cope with cancer.

Distress as the sixth vital sign for cancer patients

distress thermometerBeware of a marketing slogan posing as an empirical statement. Its emptiness is similar to that of to “Pepsi is the one.” Can you imagine anyone conducting a serious study in which they conclude “Pepsi is not the one”?

Once again in this article, a vacuous marketing slogan is presented in impressive, but pseudo-medical terms. Distress cannot be a vital sign in the conventional sense. Thr  vital signs are objective measurements that do not depend on patient self-report: body temperature, pulse rate, and respiration rate (rate of breathing) (Blood pressure is not considered a vital sign, but is often measured along with the vital signs.).

Pain was declared a fifth vital sign, with physicians mandated  by guidelines to provide routine self-report screening of patients, regardless of their reasons for visit. Pain being the fifth vital sign seems to have been the inspiration for declaring distress as the sixth vital sign for cancer patients. However policy makers declaring pain  as the fifth vital sign did not result in improved patient levels of pain. Their subsequent making intervention mandatory for any reports of pain led to a rise in unnecessary back and knee surgery, with a substantial rise in associated morbidity and loss of function. The next shift to prescription of opioids that were claimed not to be addictive was the beginning of the current epidemic of addiction to prescription opioids. Making pain the fifth vital sign is killed a lot of patients and  turned others into addicts craving drugs on the street because they have lost their prescriptions for the opioids that addicted them.

pain as 5th vital signCDC launches

 Cancer as a mental health issue

There is a lack of evidence that cancer carries a risk of psychiatric disorder more than other chronic and catastrophic illnesses. However, the myth that there is something unique or unusual about cancer’s threat to mental health is commonly cited by mental health professional advocacy groups is commonly used to justify increased resources to them for specialized services.

The article provides an inflated estimate of psychiatric morbidity by counting adjustment disorders as psychiatric disorders. Essentially, a cancer patient who seeks mental health interventions for distress qualifies by virtue of help seeking being defined as impairment.

The conceptual and empirical muddle of “distress” in cancer patients

The article repeats the standard sloganeering definition of distress that the authors and reviewers have circulated elsewhere.

It has been very broadly defined as “a multifactorial, unpleasant, emotional experienceof a psychological (cognitive, behavioural, emotional), social and/or spiritual nature that may interfere with the ability to cope effectively with cancer, its physical symptoms and its treatment and that extends along a continuum, ranging from common normalfeelings of vulnerability, sadness and fears to problems that can become disabling, such as depression, anxiety, panic, social isolation and existential and spiritual crisis”5

[You might try googling this. I’m sure you’ll discover an amazing number of repetitions in similar articles advocating increasing psychosocial services for cancer patients organized around this broad definition.]

Distress is so broadly defined and all-encompassing, that there can be no meaningful independent validation of distress measures except for by other measures of distress, not conventional measures of adaptation or mental health. I have discussed that in a recent blog post.

If we restrict “distress” to the more conventional meaning of stress or negative affect, we find that any elevation in distress (usually 35% or so) associated with onset diagnosis of cancer tends to follow a natural trajectory of decline without formal intervention. Elevations in distress for most cancer patients, are resolved within 3 to 6 months without intervention. A residual 9 to 11% of cancer patients having elevated distress is likely attributed to pre-existing psychiatric disorder.

Routine screening for distress

The slogan “distress is the sixth vital sign” is used to justify mandatory routine screening of cancer patients for distress. In the United States, surgeons cannot close their electronic medical records for a patient and go on to the next patient without recording whether they had screened patients for distress, and if the patient reports distress, what intervention has been provided. Clinicians simply informally asking patients if they are distressed and responding to a “yes” by providing the patient with an antidepressant without further follow up allows surgeons to close the medical records.

As I have done so before, I challenge advocates of routine screening of cancer patients for distress to produce evidence that simply introducing routine screening without additional resources leads to better patient outcomes.

Routine screening for distress as uncovering unmet needs among cancer patients

 Studies in the Netherlands suggest that there is not a significant increase in need for services from mental health or allied health professionals associated with diagnosis of cancer. There is some disruption of such services that patients were receiving before diagnosis. It doesn’t take screening and discussion to suggest that patients that they at some point resume those services if they wish. There is also some increased need for physical therapy and nutritional counseling

If patients are simply asked a question whether they want a discussion of the services (in Dutch: Zou u met een deskundige willen praten over uw problemen?)  that are available, many patients will decline.

Much of demand for supportive services like counseling and support groups, especially among breast cancer patients is not from among the most distressed patients. One of the problems with clinical trials of psychosocial interventions is that most of the patients who seek enrollment are not distressed, and less they are prescreened. This poses dilemma: if you require elevated distress on a screening instrument, we end up rationing services and excluding many of the patients who would otherwise be receiving them.

I welcome clarification from F 1000 just what they offer over other preprint repositories. When one downloads a preprint from some other repositories, it clearly displays “not yet peer-reviewed.” F 1000 carries the advantage of the label of “peer-reviewed, but does not seem to be hard earned.

Notes

Slides are from two recent talks at Dutch International Congress on Insurance Medicine Thursday, November 9, 2017, Almere, Netherlands   :

Will primary care be automated screening and procedures or talking to patients and problem-solving? Invited presentation

and

Why you should not routinely screen your patients for depression and what you should do instead. Plenary Presentation

        

                                  

 

 

 

In a classic study of early childhood abuse and neglect, effects on later mental health nearly disappeared when….

Controls were introduced for life events that occurred between the childhood abuse and neglect and assessment of mental health outcomes in adulthood.

Adults who had objective reports of abuse and neglect in their childhood were matched with adults who were from similar backgrounds, but without such abuse and neglect.

mind the brain logo

Apparent effects of early experience largely disappeared when:

Controls were introduced for life events that occurred between the childhood abuse and neglect and assessment of mental health outcomes in adulthood.

Adults who had objective reports of abuse and neglect in their childhood were matched with adults who were from similar backgrounds, but without such abuse and neglect.

The study had a lot of influence on my thinking. You can click on link below to obtain a PDF.

Horwitz AV, Widom CS, McLaughlin J, White HR. The impact of childhood abuse and neglect on adult mental health: A prospective study. Journal of Health and Social Behavior. 2001 Jun 1:184-201.

reconsidering-the-roots-of-addiction-implications-of-the-adverse-childhood-experiences-study-15-638

This classic paper examined the impact of three types of victimization in childhood sexual abuse, physical abuse, and neglect on lifetime measures of mental health among adults.

It replicated some past findings obtained with checklists:

Men who were abused and neglected as children have more dysthymia and antisocial personality disorder as adults than matched controls, but they did not have more alcohol problems. Abused and neglected women report more symptoms of dysthymia, antisocial personality disorder, and alcohol problems than controls.

However, unlike past studies, this study introduced controls for subsequent life events:

After controlling for stressful life events, however, childhood victimization had little direct impact on any lifetime mental health outcome.

The study also matched adults with objective records of abuse and neglect in their childhood with adults who had similar backgrounds but without such abuse and neglect.

Mental health outcomes were similar between these matched groups.

What else was different:

Rather than a respondent-completed checklist, objective data were obtained.

The study used a prospective sample gathered from records of documented court cases of childhood abuse and neglect in a Midwestern city around 1970.

These subjects then interviewed about twenty years later to assess adult mental health.

The study was unusual in having a matched control group. 

The study:

Compared outcomes of the 641 members of the abuse and neglect group with a matched control group of 510 persons who did not have documented cases of abuse or neglect.

Unlike past research, the study introduced controls for subsequent life events.

The study was important because it indicated:

The importance of adopting an approach that places childhood victimization in the context of other life stressors and of prospective changes over the life course.

The study addressed two major theoretical shortcomings in past literature.

In past research, victimization had been considered as an isolated event with little regard to the context in which it occurs.

As Briere (1992) notes  “there has been a tendency for investigators to examine sexual abuse in a relative vacuum” (p. 199).

The study instead adopted a life course, developmental perspective.

Childhood victimization is typically part of a matrix of environmental problems such as poverty, unemployment, parental alcohol and drug problems, and inadequate family functioning…The unique contribution of childhood victimization to later symptomatology, after taking into account conditions such as family disruptions and stressors, persistent poverty, and broader patterns of social deprivation, is not  well understood.

The study started with the recognition of the theoretical deficiency of most studies.

Past research assumed a simple causal direction leading from childhood abuse and neglect  to mental health outcomes in later life.

Instead, this study made the basic assumption:

The influence of childhood experiences is contingent upon historical changes across the lives of affected individuals…Traumatic events that occur in early stages of the life course are unlikely to have uniform and straightforward mental health impacts in later stages of life, regardless of subsequent social conditions.

[Any] lasting impact on adults of childhood experiences depends on later factors such as the strength of the marriages and other social relationships, educational and occupational attainment, and the adequacy of family functioning.

This study was intended to contribute some knowledge of these later factors.

We know very little, however, about how subsequent stressors in life trajectories shaped later mental health consequences of childhood victimization (Bifulco, Brown, and Adler,  1991).

The study addressed important methodological limitations of past research:

Most past research depend upon adult responses to questions about their experience of abuse events as children.

Retrospective reporting should not have a major impact on the recall of screening and objective events such as loss of a parent or divorce.

But

Abuse events in early life are not encoded in memory as objective occurrences, but recollections of what constitutes abuse and experienced in the past change in light of later events definitions of abuse.

Furthermore:

Most research about the traumatic impact of childhood abuse is not only retrospective but is also cross-sectional. Because studies obtain measures of prior abuse events in current states of mental health at the same time, present states of mental health may influence people’s recollections of recurrence has dramatic events (Brown Harris 1978).

In general, people with poor mental health have a bleaker of the world, including their prior life course, those with with higher psychological well-being (Coyne 1976; Beck et al, 1979; Burbach and Bourdin 1986).

Uniformly high correlation between later mental health and past child abuse might impart results from factors having to do with the past and the present, thus blurring the temporal order between recall of childhood events and subsequent psychological outcomes. Prospective studies, however, can overcome some of the problems stem from retrospective reports of childhood adversity (Kessler et al. 1997).

The study addressed the inadequate samples in most past studies:

Most studies used samples of college students, patients in clinical treatment, or responses to newspaper advertisements.

Few included adequate control groups of equivalent but non-victimized children. The high rates of psychopathology among adults who were abused as children are only meaningful when these rates are higher than comparable, non-abuse groups.

Shared characteristics between victimized and unvictimized children from similar backgrounds might explain the poorer mental health usually attributed to victimization.

Third factors may be influential.

Another factor such as disadvantaged social economic circumstances or family adversities may lead to both abuse in childhood and to poor adult mental health. The lack of control groups of non-abuse children comparable backgrounds precludes establishing the effects of abuse, as opposed to the impact of the matrix of social economic disadvantage within which abuse may occur on later states of mental health.

Findings

Findings showed both men and women who were victimized as children report more stressful life events over their lifetimes than a control sample, suggesting that early childhood abuse and neglect is part of a broader constellation of life stressors.

 

In addition, abused and neglected males and females are more likely than controls to have grown up in families receiving welfare, possibly indicating the officially reported childhood victimization co-occurs with particularly disadvantaged circumstances. Find there are no racial or age differences between the abused and neglected and control groups.

Discussion

As a life course literature emphasizes, these results indicate the experiences occurring early in life do not have uniform consequences for mental health outcomes in later life (Harris, Brown, and Bifulco 1990, Elder at L 1996). Instead, the influence of these early childhood experiences vary depending on what happened in subsequent stages of the life course. In particular, stressful life events that occur later in the life course influence how much effect child victimization will have on subsequent outcomes. When childhood victims of abuse and neglect did not experience more stressors than controls, they do not have worse mental health outcomes (alcohol problems, dysthymia, or antisocial personality disorder, as adults. Thus, not only do early childhood events affect the life experiences, but these later experiences also affect how consequential these earlier events will be the subsequent health.

Limitations of this study need to be addressed in the future.

Although the results of this study clearly indicate the subsequent impact of childhood victimization on the mental health of adults grounded in a broader context and the life course trajectories, they do not specify the causal links among childhood victimization, lifetime stressors, and subsequent mental health.

 

One possible interpretation of these findings is the childhood victimization produces poor mental health outcomes among children. Poor early mental health could elevate the risk of experiencing subsequent life events such as getting fired from jobs, unemployment, and divorce, which in turn strongly relate for mental health among adults.

Another possible interpretation of our findings is the childhood victimization does not precede life stressors but is the correlate of other lifetime stressors such as family isolation and disorganization… [If so, pre-existing familial context can be critical for both the child and victimization for subsequent life stressors.

In addition, children from disorganized families might have weakened social support networks as adults, exacerbating their vulnerability to stressful life conditions.

Our findings only indicate a general relationship between childhood victimization, subsequent stressors, and mental health impacts.

Our findings do not allow the causal statements about the relationship among these factors.

It is important that future research specifies both the pathways through which childhood victimization elevates the risk of suffering subsequent stressors and the possible mediating buffering factors that protect some victims of child abuse and neglect from the adverse consequences in later stages of their lives.

Stop using the Adverse Childhood Experiences Checklist to make claims about trauma causing physical and mental health problems

Scores on the adverse childhood experiences (ACE) checklist (or ACC) are widely used in making claims about the causal influence of childhood trauma on mental and physical health problems. Does anyone making these claims bother to look at how the checklist is put together and consider what a summary score might mean?

 

mind the brain logo

Scores on the adverse childhood experiences (ACE) checklist (or ACC) are widely used in making claims about the causal influence of childhood trauma on mental and physical health problems. Does anyone making these claims bother to look at how the checklist is put together and consider what a summary score might mean?

In this issue of Mind the Brain, we begin taking a skeptical look at the ACE checklist. We ponder some of the assumptions implicit in what items were included and how summary scores of the number of items checked are interpreted. Readers will be left with profound doubts that the ACE is suitable for making claims about trauma.

This blog will eventually be followed by another that presents the case that scores on the ACC do not represent a risk factor for health problems, only a relatively uninformative risk marker. In contrast to potentially modifiable risk factors, risk markers are best interpreted as calling attention to the influence of some combination of other risk factors, many of as yet unspecified, but undoubtedly of an entirely different nature than what is being studied. What?!! You will have to stay tuned, but I’ll give some hints about what I am talking about in the current blog post.

Summary of key points

 The ACE checklist is a collection of very diverse and ambiguous items that cannot be presumed to necessarily represent traumatic experiences.

Items variously

  • Represent circumstances that are not typically traumatic.
  • Reflect the respondent’s past or current psychopathology.
  • Make equivalent and traumatic vastly different experiences, many neutral and some that are positive.
  • Reinterpret a personal vulnerability due to familial transmission of psychopathology, either direct or indirect, rather than simply an exposure to events.
  • Ignore crucial contextual information, including timing of events.

There is reason not to assume that higher summed scores for the ACE represent more exposure to trauma than lower scores.

Are professionals misinterpreting the ACE checklist just careless or are they ideologues selectively identifying “evidence” for their positions which don’t depend on evidence at all?

ace-7Witness claims based on research with the ACE that migraines are caused by sexual abuse   and that psychotherapy addressing that abuse should be first line treatment. Or claims that childhood trauma is as strong a risk factor for psychosis and schizophrenia as smoking is for lung cancer [* ] and so psychotherapy is equivalent to medication in its effects. Or claims that myalgic encephalomyelitis, formerly known as chronic fatigue syndrome, is caused by childhood trauma and the psychological treatments can be recommended as the treatment of choice. These claims share a speculative, vague neo-cryptic pseudopsychoanalytic set of assumptions that is seldom articulated or explicitly confronted with evidence. Authors typically leap from claims about childhood trauma causing later problems to non sequitur claims about the efficacy of psychological intervention in treating these problems by addressing trauma. These claims about efficacy of trauma-focused treatment are not borne out in actually examining effects observed in randomized controlled trials.

Rather than attempting to address a provocative question about investigator motivation without a ready way of answering it, I will show most claims about trauma causing mental and physical health problems are, at best, based on very weak evidence, if they depend solely on the ACE checklist.

I will leave for my readers to decide if some authors who make such a fuss about the ACE have bothered to look at the instrument or care that is so inappropriate for the purposes to which they put it.

The ACE is reproduced at the bottom of this post and it is a good idea to compare what I’m saying about it to the actual checklist.

e5fc302ac1fabf0757e62a935b27800d
What “science” is behind such speculations?

The ACE was originally intended for educational purposes, not as a scientific instrument. Perhaps that explains its gross deficiencies as a key measure of psychological and epidemiological constructs.

The ACE checklist is a collection of very different and ambiguous items that cannot be presumed to represent traumatic experiences.

The ACE consists of ten dichotomous items for which the respondent is asked to indicate no/yes whether an experience occurred before the age of 18.  However, for six of the 10 items, the respondent is given further choices  that often differ greatly in the kind of experience to which the items refer. Scoring of the instrument does not take which of these experiences is the basis of a response. For example,

5. Did you often feel that … You didn’t have enough to eat, had to wear dirty clothes, and had no one to protect you? or

Your parents were too drunk or high to take care of you or take you to the doctor if you needed it?

Yes   No     If yes enter 1     ________

This item treats some very different circumstances as equivalent. The first half is complex, but largely covers the experience of living in poverty, but combines that with “having no one to protect you.” In contrast, the second half refers to substance abuse on the part of parents. In neither case, is there any room for interpreting what mitigating circumstances in the respondent’s life might have influenced effects of exposure. Presumably, the timing of this exposure would be important. If the exposure only occurred at the end of the 18 year period covered by the checklist, effects could be mitigated by other individual and social resources the respondent had.

Single items that are added together in a summary score.  We have to ask whether there is an equivalency between the two halves of the item that will be treated as the same. This will be an accumulating concern as we go through the 10 item questionnaire

The items vary greatly in the likelihood that they refer to an experience that was traumatic. Seldom do any of the researchers who use the ACE explain what they mean by trauma. If they did, I doubt that they could make a good argument that in endorsing many of these items would indicate that a respondent had faced a trauma.

From the third edition of the American Psychiatric Association Diagnostic and Statistical Manual (DSM-III) onward to DSM-5, the assumption has been that a traumatic event is a catastrophic stressor outside the range of usual human experience.

With that criteria in mind we have to ask if items are likely to represent a traumatic experience for most people. In answering this question, we also have to ask how we willing to consider a particular item is equivalent to other items in arriving at an overall score reflecting exposure to trauma before age 18. Yet, if summary scores are to be meaningful, assumption has to be made that items contribute equally if they are endorsed

6. Were your parents ever separated or divorced?

Yes   No     If yes enter 1     ________

The item refers to a highly prevalent and complex event, the nature and consequences of which are likely to unfold over time. Importantly, we need a sense of context to judge whether the event is traumatic and, if so how severe. Presumably, it would matter greatly when, across the 18 year span, the event that occurred. No timing or other information is asked of the respondent, only whether or not this event occurred. Neither the respondent nor anyone interpreting a score on the inventory has further information as to what is meant.

Other problems with ambiguous items.

Questions can be raised about the validity of all the individual items and the wisdom of combining them as equivalent in creating a summary score.

Items 1 and 2: Items raise questions about what role the respondent played eliciting the event.

 Did an event simply befall a respondent? Was it related to some pre-existing characteristic of the respondent? Or did the respondent have an active role in generating the event?

Did a parent or other adult member of the household often…

Swear at you, insult you, put you down, or humiliate you?

or

Act in a way that made you afraid that you might be physically hurt?

Yes   No     If yes enter 1     ________

And

Did a parent or other adult in the household often …

Push, grab, slap, or throw something at you?

or

Ever hit you so hard that you had marks or were injured?

Yes   No     If yes enter 1     ________

 Here, as throughout the rest of the checklist, questions can be raised about whether these items refer simply to an environmental exposure in epidemiological terms, say, equivalent to asbestos or tobacco. We don’t know the frequency, intensity or context of a the behavior in question, all of which may be crucial in evaluating whether a trauma occurred. For instance, it matters greatly if the behavior happened frequently when the respondent as a toddler or was limited to a struggle that occurred when the respondent was a teen high on drugs  attempting to take the car keys and go for a after midnight drive.

Like most of the rest of the questionnaire, there is the question of timing.

Item 3: There is so much ambiguity in endorsments of (ostensible) sexual abuse. Maybe it was a positive, liberating experience.

This is a crucial item and discussions of the ACE often assume that it is endorsed and represents a traumatic experience:

Did an adult or person at least 5 years older than you ever…

Touch or fondle you or have you touch their body in a sexual way?

or

Try to or actually have oral, anal, or vaginal sex with you?

Note that this is a complex item for which endorsement could be on the basis of a single instance of a person at least 5 years older touching or fondling the respondent. What if the presumed “perpetrator” is the 20 year old boyfriend or girlfriend of a 14 year old?

Are we willing to treat as equivalent “touch” or ‘fondle you” and “having anal sex” in all instances?

Arguably, the event which construed as trauma could actually be quite positive, as in the respondent  forming a secure attachment with a somewhat older, but nonetheless appropriate partner. All that is unconventional is not traumatic. What if the respondent and  alleged “perpetrator” were in a deeply intimate relationship or already married?

The research that attempts to link endorsement of such an item to lasting mental and physical health problems is remarkably contradictory and inconsistent 

Item 4:  Does this  item reflect the respondent’s serious clinical depression or other mental disorder before age 18 or currently, when the checklist is being completed?

Did you often feel that …  No one in your family loved you or thought you were important or special?    or

Your family didn’t look out for each other, feel close to each other, or support each other?

Yes   No     If yes enter 1     ________

As elsewhere in the checklist, there is no place for the respondent or someone interpreting a “yes” response for taking into account timing or contextual factors that might mitigate or compound effects of this “exposure.”

Item 5: Is this a  traumatic exposure or an enduring set of circumstances conferring multiple known risks to mental and physical health?

Did you often feel that …

You didn’t have enough to eat, had to wear dirty clothes, and had no one to protect you?

or

Your parents were too drunk or high to take care of you or take you to the doctor if you needed it?

Yes   No     If yes enter 1     ________

This item has already been discussed above, but is worth revisiting in terms of raising issues whether particular items refer either directly or indirectly to enduring sets of circumstances that pose their own enduring threat. The relevant question is whether items which ostensibly represent “traumatic events” and risk for subsequent problems are not risk factors, but only risk indicators, and not particularly informative ones.

Item 7: Could an ostensibly a traumatic exposure actually be no actual exposure?

Was your mother or stepmother:

Often pushed, grabbed, slapped, or had something thrown at her?    or

Sometimes or often kicked, bitten, hit with a fist, or hit with something hard?    or

Ever repeatedly hit over at least a few minutes or threatened with a gun or knife?

Yes   No     If yes enter 1     ________

Like item four, which refers to ostensible sexual abuse, this item seems to be one of the least ambiguous in terms of representing exposure to risk. But does it? We don’t know the timing, duration, or context. For instance, the mother might no longer be in the home and the respondent might not have known what happened at the time. There is even the possibility that the respondent was the “perpetrator” of such violence against the mother.

Items 8 and 9: Are traumatic exposures or indications of familial transmission of psychopathology?

Did you live with anyone who was a problem drinker or alcoholic or who used street drugs?

Yes   No

If yes enter 1     ________

And

Was a household member depressed or mentally ill or did a household member attempt suicide?    Yes   No     If yes enter 1     ________

These items are highly ambiguous. They don’t take in consideration whether the person was a biological relative, or whether they were a parent, sibling, or someone not biologically related. They don’t take into account timing. There may not have even been any direct exposure to the substance misuse or the attempted suicide, but the respondent only later learned of something that was closeted.

Item 10: traumatic exposure or relief from exposure?

Did a household member go to prison?

Yes   No

If yes enter 1     ________

The implications of endorsement of this item depend greatly on whom the household member was and the circumstances of them going to prison.

There may be a familial relationship with this person, but it could have been an abusive stepparents or stepsiblings, with the incarceration representing a lasting relief from some impressive situations. Or the person who became incarcerated was not an immediate family member, but somewhat more transient, maybe someone who was just renting a room or given a place to stay. We just don’t know.

Does adding up all these endorsements in a summary score clarify or confuse further?

Now add up your “Yes” answers:   _______   This is your ACE Score

 It would be useful to briefly review the assumptions involved in summing across items of a checklist and entering the summary score as a continuous variable in statistical analyses.

Classical test theory recognizes that the individual items may imperfectly reflect the underlying construct, in this case, traumatic exposure. However, in constructing a sum, the expectation is that the imperfections or errors of measurement in particular items cancel each other out. The summed score becomes a purer a representation of the underlying construct than any of the original items. Thus, the summary score will be more reliable and valid than any of the individual items would be.

There are a number of problems in applying this assumption to a summary ACE score. The items are quite heterogeneous, i.e., they vary wildly in whether they are likely to represent a traumatic exposure, and if so, the severity of that exposure. More importantly, there is a huge amount of variation in what these brief items would represent for particular individuals in the contexts they found themselves in the first 18 years of their lives. Undoubtedly, most endorsements of these items would represent false positives, if we hold ourselves to any strict definitions of trauma. If we don’t do so, we risk equating the only normative experiences that may have neutral or even positive effects on the respondent with serious exposures to traumatic events with lasting consequences

We are not in a position to know whether a score of five or even eight necessarily represents more traumatic exposure than a score of one.

Moreover, there is important empirical research of the clustering of events. We certainly cannot consider them random and unrelated. One classic study found 

In our data, total CCA was related to depressive symptoms, drug use, and antisocial behavior in a quadratic manner. Without further elucidation, this higher order relationship could have been interpreted as support for a sensitization process in which the long-term impact of each additional adversity on mental health compounds as childhood adversity accumulates. However, further analysis revealed that this acceleration effect was an artifact of the confounding of high cumulative adversity scores with the experience of more severe events. Thus, respondents with higher total CCA had disproportionately poorer emotional and behavioral functioning because of both the number and severity of the adversities they were exposed to, not the cumulative number of different types of adversities experienced.

And

Because low-impact adversities did not present a cumulative hazard to young adult mental health, they functioned as suppressor events in the total sum score, consistent with Turner and Wheaton’s (1997) expectation. Their inclusion increased the “noise” in the score and greatly watered down the influence of high-impact events. Thus, in addition to decreasing efficiency, total scores may seriously underestimate the cumulative effects of severe forms of childhood adversity, such as abuse and serious neglect.

But what if many or most of the high scores in a particular sample represent only a clustering of low- or no-impact adversities?

Another large-sample, key study cautioned:

Significant effects of parental separation}divorce in predicting subsequent mood disorders and addictive disorders are powerfully affected by whether or not there was parental violence and psychopathology in the household prior to the break-up and whether exposure to these adversities was reduced as a result of the separation (Kessler et al. 1997a). There are some situations – such as one in which the father was a violent alcoholic – where our data suggest that parental divorce and subsequent removal of the respondent from exposure to the father might actually be associated with a significant improvement in the respondent’s subsequent disorder risk profile, a possibility that has important social policy implications.

Finding Your ACE Score-page-0

NOTE

*Richard Bentall commonly interprets summed ACE scores in peer reviewed articles  as having a traditional dose-response association with mental health outcomes, and therefore as representing a modifiable causal factor in psychosis. In books and in social media, his claims become simply absurd.

bentall

I don’t think his interpretations withstand a scrutiny of the items and what a summed score might conceivably represent.

eBook_Mindfulness_345x550Preorders are being accepted for e-books providing skeptical looks at mindfulness and positive psychology, and arming citizen scientists with critical thinking skills. 

I will also be offering scientific writing courses on the web as I have been doing face-to-face for almost a decade. I want to give researchers the tools to get into the journals where their work will get the attention it deserves.

Sign up at my website to get advance notice of the forthcoming e-books and web courses, as well as upcoming blog posts at this and other blog sites. Get advance notice of forthcoming e-books and web courses. Lots to see at CoyneoftheRealm.com.
 

 

 

School-Based Mindfulness Based Stress-Reduction Program (MBSR) fails to deliver positive results

No positive effects found for Jon Kabat-Zinn’s Mindfulness Based Stress-Reduction Program with middle and high school students. Evidence of deterioration was found in some subgroup analyses.

mind the brain logo

No positive effects found for Jon Kabat-Zinn’s Mindfulness Based Stress-Reduction Program with middle and high School Students. Evidence of deterioration was found in some subgroup analyses.

mindfulness in schoolsWe should be cautious about interpreting negative effects that are confined to subgroup analyses. They may well be due to chance. But we should be concerned about the lack of positive findings across measures in the primary analyses. MBSR (a mindfulness training product trademarked and controlled by Jon Kabat-Zinn) and other mindfulness programs have heavily promoted as having wondrous benefits and mandated in many school settings.

The study [with link to the PDF]

Johnson C, Burke C, Brinkman S, Wade T. Effectiveness of a school-based mindfulness program for transdiagnostic prevention in young adolescents. Behaviour Research and Therapy. 2016 Jun 30;81:1-1.

Abstract

Anxiety, depression and eating disorders show peak emergence during adolescence and share common risk factors. School-based prevention programs provide a unique opportunity to access a broad spectrum of the population during a key developmental window, but to date, no program targets all three conditions concurrently. Mindfulness has shown promising early results across each of these psychopathologies in a small number of controlled trials in schools, and therefore this study investigated its use in a randomised controlled design targeting anxiety, depression and eating disorder risk factors together for the first time. Students (M age 13.63; SD = .43) from a broad band of socioeconomic demographics received the eight lesson, once weekly.b (“Dot be”) mindfulness in schools curriculum (N = 132) or normal lessons (N = 176). Anxiety, depression, weight/shape concerns and wellbeing were the primary outcome factors. Although acceptability measures were high, no significant improvements were found on any outcome at post-intervention or 3-month follow-up. Adjusted mean differences between groups at post-intervention were .03 (95% CI: -.06 to -.11) for depression, .01 (-.07 to -.09) for anxiety, .02 (-.05 to -.08) for weight/shape concerns, and .06 (-.08 to -.21) for wellbeing. Anxiety was higher in the mindfulness than the control group at follow-up for males, and those of both genders with low baseline levels of weight/shape concerns or depression. Factors that may be important to address for effective dissemination of mindfulness-based interventions in schools are discussed. Further research is required to identify active ingredients and optimal dose in mindfulness-based interventions in school settings.

The discussion noted:

The design of this study addresses several shortcomings identified in the literature (Britton et al., 2014; Burke, 2010; Felver et al., 2015; Meiklejohn et al., 2012; Tan, 2015; Waters et al., 2014). First, it was a multi-site, randomised controlled design with a moderately large sample size based on a priori power calculations. Second, it included follow-up (three months). Third, it sought to replicate an existing mindfulness-based intervention for youth. Fourth, socioeconomic status was not only reported but a broad range of socioeconomic bands included, although it was unfortunate that poor opt-in consent rates resulted in high data wastage in the lower range schools. Use of the same instructor for all classes in the intervention arm represents a strength (consistency) and a limitation (generalisability of findings).

Coverage in Scientific American

Mindfulness Training for Teens Fails Important Test

A large trial in schools showed no evidence of benefits, and hints it could even cause problems

The fact that this carefully-controlled investigation showed no benefits of mindfulness for any measure, and furthermore indicated an adverse effect for some participants, indicates that mindfulness training is not a universal solution for addressing anxiety or depression in teens, nor does it qualify as a replacement for more traditional psychotherapy or psychopharmacology, at least not as implemented in this school-based paradigm.

eBook_Mindfulness_345x550Preorders are being accepted for e-books providing skeptical looks at mindfulness and positive psychology, and arming citizen scientists with critical thinking skills. Right now there is a special offer for free access to a Mindfulness Master Class. But hurry, it won’t last.

I will also be offering scientific writing courses on the web as I have been doing face-to-face for almost a decade. I want to give researchers the tools to get into the journals where their work will get the attention it deserves.

Sign up at my website to get advance notice of the forthcoming e-books and web courses, as well as upcoming blog posts at this and other blog sites. Get advance notice of forthcoming e-books and web courses. Lots to see at CoyneoftheRealm.com.