Did the struggle to get the Cochrane review withdrawn kill Robert Courtney? Or the denial of his basic human rights by the medical system?
An incomplete story that urgently needs to be told. We need to get some conversations going.
Did the struggle to get the Cochrane review withdrawn kill Robert Courtney? Or did the denial of his basic human rights by the medical system?
LONDON, Oct 17 (Reuters) – A respected science journal is to withdraw a much-cited review of evidence on an illness known as chronic fatigue syndrome (CFS) amid fierce criticism and pressure from activists and patients.
Citizen scientists and patient advocates Tom Kindlon and Robert Courtney played a decisive role in getting the Cochrane review withdrawn.
In the next few days, I will provide the cover letter email sent by Robert Courtney to Senior Cochrane Editor David Tovey that accompanied his last decisive contribution. Robert is now deceased.
I will also provide links to Tom Kindlon’s contributions that are just as important.
Readers will be able to see from what David Tuller calls their cogent, persuasive and unassailable submissions that the designation of these two as citizen scientists is well-deserved.
Since 2015, I have kept in touch with an advisory group of about a dozen patients with myalgic encephalomyelitis/chronic fatigue syndrome (ME/cfs). I send emails to myself with this group blind copied. The rationale was that any one of them could respond to me and not have the response revealed to anyone else. A number of patients requested that kind of confidentiality, given the divisions within the patient community.
Robert Courtney was a valued, active member of that group, but then he mysteriously disappeared in January 2017. Patients have their own reasons for entering and withdrawing from social engagement. Sometimes they announce taking leave, sometimes not. I’ve learned to respect absences without challenge, but I sometimes ask around. In the case of Robert, I could learn nothing from the community except he was not well.
Then in February 2018, Robert reemerged with the email message below. I had assumed his recovery would continue and he would participate in telling his story. Obviously there were a lot more details to tell, but he died by suicide a few weeks later.
Long, unbroken periods of being housebound and often bedridden is one of the curses of having severe ME/cfs. Able-bodied persons need to understand the reluctance of patients to invite them into their homes. Even able-bodied persons who believe that they have forged strong bonds with patients on social media.
I nonetheless occasionally make such offers to meet, as I travel through Europe. I’m typically told things like “sorry, I only leave my house for medical appointments and a twice a year holiday with my family.”
We have to learn not to be offended.
Consequently, few people who were touched by Robert Courtney and his efforts have ever met him. Most know little about him beyond his strong presence in social media.
Robert Courtney (d. March 7, 2018) was a patient advocate for myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS) and an outspoken critic of the PACE trial and the biopsychosocial model of chronic fatigue syndrome. He authored numerous published letters in medical journals regarding the PACE trial and, also, filed freedom of information requests in an attempt to get the authors of the PACE trial to release the full trial data to the public for scrutiny.
The day after I received the email below, Robert Courtney sent off to David Tovey of the Senior Editor Cochrane his final comments.
The email describes the horrible conditions of his last year and his mistreatment and the denial of basic human rights by the medical system. I think airing his story as a wake up call can become another of his contributions to the struggle for the dignity and rights of the patient community.
An excerpt from the email, repeated below.
It seems that this type of mistreatment is all too typical for ME patients. Since I’ve been out of hospital, many patients have told me that they have similar nutritional difficulties, and that they are too scared to seek medical assistance, and that quite a lot of them have been threatened with detention or indeed have been detained under the mental health act. It is a much worse situation than I ever realised.-Robert “Bob” Courtney
We can never know whether Bob’ determined effort to get the review withdrawn led to his medical collapse. The speculation is not just a mindless invoking of “stress kills.” One of the cardinal, defining symptoms of myalgic encephalomyelitis is post exertion malaise.
We usually think of the “exertion” as being physical, but patients with severe form of the illness learn to anticipate that sustained emotional arousal can, within 48 hours or so, put them in their beds for weeks. That applies to positive emotion, like a birthday party, and certainly to negative emotion. Aside from the stress, frustration, and uncertainty of trying to get bad science out of the literature, Bob and other members of the patient community had to contend with enormous vilification and gaslighting, which still continues today.
After the anorexia diagnosis, they rediagnosed my ME symptoms as being part of a somatoform disorder, and placed me on an eating disorders unit..-Robert “Bob” Courtney
On Sat, Feb 17, 2018 at 2:44 PM, Bob <firstname.lastname@example.org> wrote:
I don’t know if you’ll remember me. I am an ME patient who was in regular contact with you in 2016. Unfortunately I had a health crisis in early 2017 and I was hospitalised for most of the year. I had developed severe food intolerances and associated difficulties with eating and nutrition. When I admitted myself to hospital they quickly decided there was nothing medically wrong with me and then diagnosed me with anorexia ( to my shock and bewilderment ), and subsequently detained me under the mental health act. I’m not anorexic. The level of ignorance, mistreatment, neglect, abuse, and miscommunication was staggering. After the anorexia diagnosis, they rediagnosed my ME symptoms as being part of a somatoform disorder, and placed me on an eating disorders unit. Then they force-fed me. It is a very long and troubling story and I’ll spare you the details. I’d quite like a journalist to write up my story but that will have to wait while I address my ongoing health issues.
Unfortunately, it seems that this type of mistreatment is all too typical for ME patients. Since I’ve been out of hospital, many patients have told me that they have similar nutritional difficulties, and that they are too scared to seek medical assistance, and that quite a lot of them have been threatened with detention or indeed have been detained under the mental health act. It is a much worse situation than I ever realised. It is only by sharing my story that people have approached me and been able to tell me what had happened to them. It is such an embarrassing situation both to have eating difficulties and to be detained. The detention is humiliating and the eating difficulties are also excruciatingly embarrassing. Having difficulties with food makes one feel subhuman. So I have discovered that many patients keep their stories to themselves.
You might remember that in 2016 I submitted four lengthy comments to Cochrane with respect to the exercise therapy for chronic fatigue syndrome review. . Before hospital, I had also written an incomplete draft complaint to follow up my submitted comments, but my health crisis interrupted the process and so I haven’t yet sent it .
I am out of hospital now and have finished editing the complaint and I am about to send it. I am going to blind copy you into the complaint so this email is just to let you know to expect it. I’ll probably send it within the next 24 hours. The complaint isn’t as concise or carefully formatted as it could be because I’m still unwell and I have limited capacity.
Anyway this is just to give you some advance notice. I hope this email finds you in good spirits. I haven’t been keeping up to date with the news and activities, while I’ve been away, but I see there’s been a lot of activity. Thanks so much your ongoing efforts.
Bob (Robert Courtney)
James Coyne <email@example.com>
Feb 17, 2018, 2:50 PM
Bob, I remember you well as one of the heroes of the patient movement, and a particularly exemplary hero because you so captured my idea or of the citizen scientist gathering the data and the sense of methodology to understand the illness and battle the PACE people. I’m so excited to see your reemergence. I look forward to what you send.
James Coyne <firstname.lastname@example.org>
Feb 17, 2018, 3:11 PM
Your first goal must be to look after yourself and keep yourself as active and well as possible. You know, the patient conception of pacing. You are an important model and resource for lots of people
But when you are ready, I look forward to your telling your story and how it fits with others.
A well-orchestrated publicity campaign for a Lancet Psychiatry article promoted the view that locked inpatient wards are ineffective in reducing suicide.
This interpretation is not supported by data in the actual paper, but plays to some entrenched political stances and prejudices.
Hype and distortions in conventional and social media about this article are traceable directly to quotes from the authors in press releases from Lancet and from their university.
Mental Elf posted a blog the day the embargo on reporting this study was lifted. The blog post and an associated Twitter campaign generated lots of social media attention. Yet, there is no indication that the blogger went beyond what was in press releases or compared the press releases to what was in the actual article.
Not many of the re-tweets and “likes” were likely from people who had read the original research.
The publicity orchestrated for this study raises issues about the ethics of promoting clinical and public policy with claims of being evidence-based when the audience does not have the ability to evaluate independently the claims by actually reading the peer-reviewed article.
I obtained the article from interlibrary loan and the supplementary material from the authors. I appreciate the authors’ immediate responsiveness to my request.
[I delayed this blog post for a week because of indications that the article would be released from behind the pay wall, but apparently it has not been freed.]
In this blog post I identify important contradictions between the authors’ claims in the article and what they promoted in the media. The contradictions are obvious enough that someone other than the authors – the Lancet Psychiatry editor and reviewers – should have immediately caught them.
Spoiler: Claims supposedly based on sophisticated multivariate techniques that were applied to data from hundreds of thousands of patients were actually based on a paltry 75 completed suicides. These were a subsample of at least 174 that occurred in 21 hospital settings in the course of 15 years. Throwing way a chunk of the data and the application of multivariate analyses to such a small, arbitrarily chosen subsample is grossly inappropriate.Any interpretations are likely to be invalid and unreliable.
No one else seems to be commenting on these key features of the study, nor the other serious problems of the study that I uncovered when I actually examine the paper and supplements. Join me in the discovery process and see if you agree with me. Please let me know if you don’t agree with my assessment.
The promotion of the study can be seen as a matter of ideologically-driven mistreatment of data with the intention of promoting clinical and public policies that put severely disturbed persons at risk for suicide.
Regardless of where one stands as to whether severely disturbed persons should be prevented from hurting or killing themselves, this attempted manipulation of public policy should be viewed as objectionable.
In presenting what may be controversial points, I’ll start with editorials that were easily accessible. I’ll then delve into the paywalled article itself.
The press release from the authors’ University of Basil
The study’s last author declared his agenda in promoting the study:
Focus on ethical standards
“Our results are important for the destigmatization, participation and emancipation of patients, as well as for psychiatric care in general,” comments last author Undine Lang, Director of the Adult Psychiatric Clinic at UPK Basel. The results will also have an influence on legal issues that arise when clinics adopt an open door policy. In future, treatment should focus more on ethical standards that ensure patients retain their autonomy as far as possible, says Undine Lang. Efforts should also be made to strengthen the therapeutic relationship and joint decision-making with patients.
The press release from The Lancet
Distributed while the article was still embargoed, Locking doors in mental health hospitals does not lower suicide rate provided more details of the study, but more editorializing grounded in direct quotes from the authors:
Locking the doors of mental health hospitals does not reduce the risk of suicide or of patients leaving without permission, according to a study published in The Lancet Psychiatry.
Authorities around the world are increasingly using locked-door policies to keep patients safe from harm, but locked doors also restrict personal freedom.
European countries tend to follow traditional approaches in caring for patients in psychiatric care, because there has been little evidence so far that one method is better than another.
Similar outcomes whether doors are open or locked.
Of 349,574 patients, they selected 72,869 cases from each hospital type, or 145,738 cases altogether. Creating matched pairs enabled a direct comparison between hospitals.
Translation: to prepare the data for the statistical analyses the authors had planned, they threw away 203,836 cases, or 58.3% of the available cases.
And they concluded:
Findings revealed similar rates of suicide and attempted suicide, regardless of whether a hospital had a locked door policy or not. Furthermore, hospitals with an open door policy did not have higher rates of absconding, either with or without return. Patients who left an open door hospital without permission were more likely to return than those from a closed facility.
The press release next raised a dramatic question. But could these data answer it?
Do locked doors unnecessarily create a sense of oppression?
Given the similarity of outcomes between the two types of hospital, the researchers propose that an open door policy might be preferable.
“These findings suggest that locked door policies may not help to improve the safety of patients in psychiatric hospitals, and are not generally successful in preventing people from absconding. In fact, a locked door policy probably imposes a more oppressive atmosphere, which could reduce the effectiveness of treatments, resulting in longer stays in hospital. The practice may even lend motivation for patients to abscond.” -Dr. Christian Huber, of the Universitäre Psychiatrische Kliniken Basel, Switzerland
Of course, the study did not assess anything like “sense of oppression” and so cannot answer this question. As we will see when I discuss what I found in the actual paper, Dr. Huber’s characterization of his findings is untrue. Patients on locked wards did actually not have longer stays.
Since each hospital serves a specific location, there was no chance of higher-risk patients being allocated to hospitals with locked wards. This reduced the risk of bias.
This is also not true. An unknown proportion of the hospitals, probably most, had both locked and unlocked wards. There could easily have been strong selection bias by which patients was referred to a locked ward. We are not told whether patients could be referred into other catchment areas, but this information would be useful in interpreting the authors’ claims.
The authors warn that an open door strategy might not be appropriate everywhere, as mental health care provision differs in other ways, too, for example, how many beds are available, the percentage of acutely ill patients, and how long they are treated for.
Germany has around 1.1 psychiatric care beds for every 1,000 people, compared with 0.5 beds per 1,000 in the United Kingdom and 0.3 in the United States. Where there are fewer beds, patients who receive treatment are more likely to be severely ill and more at risk.
So, Germany has more than 3 times the beds/100 people than the USA and more than the twice the availability of beds in the UK. We can learn from other sources:
Germany is one of the countries with highest expenditure for mental health care in the world. However, in contrast to other western European countries, psychiatric treatment in Germany is still mainly provided by psychiatric hospitals, outpatient clinics and office based psychiatrists and only rarely by community mental health teams. As mental health policy, except the provision of pharmaceutical treatment, is the responsibility of the federal states, no national mental health plan exists. Therefore, community mental health care systems vary widely with regard to conceptual, organisational and economic conditions across the country. Moreover, the fact that different components of community mental health care are funded by different payers (and on different legal bases) hampers coordination and integration of services.
Studies largely conducted in other countries with organizations of care different than in Germany have consistently concluded that Assertive Community Treatment (ACT) programs are effective in reducing the need for inpatient treatment.
In order to keep the level of psychiatric inpatient treatment and institutional care as low as possible these services should be provided by multi-professional community mental health teams organized according to the principles of Assertive Community Treatment (ACT).
ACT programs keep persons with psychosis from being placed in psychiatric inpatient units like those studied in the Lancet Psychiatry and they lead to shorter hospital stays.
The Lancet Psychiatry article makes no mention of ACT in Germany. My inference is that implementation was not widespread during the study. If there are ACT programs in Germany, their influence on this data set is through an invisible hand.
Inpatient psychiatric beds are quite scarce in the US, even for patients and families willing to pay out of pocket. To deal with demand that is not met by psychiatric facilities, the Los Angeles jail has become the largest locked facility. Whether it not it was the intention of the Lancet Psychiatry, the ideology with which it is infused has served to make inpatient beds less available in the United States and greater reliance on jails instead of least restrictive, and more supportive settings for protecting persons with psychosis.
Inpatient hospitalizations in the United States are much shorter than in Germany. In some states, the mean length of stay is five days. Hospitalization has different goals in the US- only stabilization of the patient’s condition.
The means of killing oneself are also different between the US and Germany. Firearms are much more readily available in the US than in Germany, suggesting different means-restriction strategies for reducing suicide.
So, I cannot see the generalizability of the findings from the Lancet Psychiatry study to the US – or the UK, for that matter. Can you?
The Mental Elf advertises itself as offering “no bias, no misinformation, just what you need.” Its coverage of the study occurred the same day the embargo was lifted. Its coverage uncritically echoed what was in the press releases, adding some emotional and ideologically-driven amplification.
The reason usually given for wards being locked is that the people within them need to be kept safe; safe from harming themselves and safe from committing harm to others. Of course these are very real fears, but they are often wrongly magnified by a still sadly stigmatising media and public perception of severe mental illness.
There is certainly an uneasy tension between the Mental Health Act Code of Practice and the reality of locking up severely ill mental health patients, which is brought into sharp focus when we consider the lack of evidence for locked wards. The literature is primarily made up of expert opinion that insists safety is paramount, but fails to provide any compelling evidence that locking people up actually increases safety.
Let’s examine Mental Elf’s claim of the lack of “any compelling evidence that locking people up actually increases safety.” Presumably, he is referring to the lack of RCTs.
I have been a scientific advisor to experimental studies like the US PROSPECT study and quasi-experimental European studies attempting to test whether suicidality could be reduced. Any such studies suffer from the serious practical limitation that suicide is an infrequent event. But to say there is no compelling evidence for restricting opportunities for acutely suicidal persons to hurt themselves is akin to BMJ’s spoof systematic review finding no evidence from RCTs that parachutes reduce deaths when jumping of planes.
Neither RCTs nor the propensity analyses of administrative data that Mental Elf favors can produce “compelling data.” As I will soon show, this study displays the pitfalls of propensity analyses.
We can systematically examine the contextual circumstances of particular deaths by suicide when they do occur, and make suggestions whether some sort of means restriction, including access to a locked inpatient unit would have made a difference. We can also hold professionals in a decision making capacity legally responsible when they fail to avail themselves of such facilities, and we should.
The Mental Elf wrapped on a rousing, uncritical, and ultimately nonsensical note:
This is a novel and compelling study, conducted in Germany, but very relevant to any Western country that has a secure system for mentally ill inpatients.
Our obsession with security and safety in an ever more dangerous world is justified if you watch the TV news channels for any prolonged period of time. The world is after all full of war, terrorism, violent crime, child abuse; or so we’re led to believe.
I spent a very enjoyable day at City University last week, participating in the #COCAPPimpact discussions, which included some rich and very constructive conversations about therapeutic relationships. It doesn’t take much to appreciate that relationships (therapeutic or otherwise) are stronger and more equitable on open wards.
The Mental Elf website claims (8/5/20016) 215 responses to this post. All but a very few were approving tweets that did not depend on the tweeter having read the study.
The reference to TV news channels is at the level of evidence of a Donald Trump tweet in which he refers to something he saw on TV.
Taking a look at the actual article and its supplementary information.
Christian G. Huber, Andres R. Schneeberger, Eva Kowalinski, Daniela Fröhlich, Stefanie von Felten, Marc Walter, Martin Zinkler, Karl Beine, Andreas Heinz, Stefan Borgwardt, and Undine E. Lang. Suicide Risk and Absconding in Psychiatric Hospitals with and without Open Door Policies: A 15-year Naturalistic Observational Study. The Lancet Psychiatry, 2016 DOI: 10.1016/S2215-0366(16)30168-7
At the time of the media campaign, most people who wanted to access the article could only obtain its abstract, which you can click here .
Why were there only 75 suicides being explained?
Much ado is being made of 75 suicides that occurred over a 15 year period across 21 hospitals. Suicides are an infrequent event, even in high risk populations. But why were only 75 available for analysis from a sample that initially consisted of 350,000 in this amount of time?
Let’s start with the 350,000 admissions that are misrepresented as “cases” in the official press releases. The article states:
The resulting dataset contained 349 574 hospital admissions from 177 295 patients.
Presumably, a considerable proportion of these patients had multiple admissions over the 15 years. Suicides were probably concentrated in the group with multiple admissions. But some patients had only one admission. Moreover, some patients may have been admitted to different types of facilities – locked versus unlocked – on different occasions. Confusion is being generated, bias is being introduced, and valuable information is being lost about the non-independence of observations – i.e., admissions.
How many suicides occurred among these 349 574 hospital admissions? Readers cannot tell from the article. Table 4 states that multivariate analyses were based on predicting 79 suicides. Yet, going to supplementary materials, Table S1 indicates that the analyses were done without the matching requirements imposed by propensity analyses, there were 174 suicides to it explain. The authors aren’t particularly clear, but it appears that in order to meet the requirements of their propensity analysis, they threw away data on most of the suicides.
The exaggerated power of propensity analyses
The authors extol the virtues of propensity analyses:
We used propensity score matching and generalised linear mixed-eﬀects models to achieve the strongest causal inference possible without an experimental design. Since patients were not randomly allocated to the diﬀerent hospital types, causal inference between hospital type and outcomes might be biased—potential confounders could aﬀect both the probability of relevant outcomes and the probability of a case having been admitted to a speciﬁc hospital type. The propensity score of patients reﬂects their probability of having been admitted to a hospital with an open-door policy rather than one with a locked-door policy.15 By matching cases from both hospital types based on their propensity score, datasets with similar distributions of confounders can be generated. These allow stronger causal inference when analysed.15
A full discussion of propensity analyses is beyond the scope of this blog post. I worry that I would lose a lot of readers here if I attempted one. But here is a very readable, accessible source:
It remains unclear whether, and if so when, use of propensity scores provides estimates of drug effects that are less biased than those obtained from conventional multivariate models. In the great majority of published studies that have used both approaches, estimated effects from propensity score and regression methods have been similar.
Use of propensity scores will not correct biases from unmeasured confounders, but can aid in understanding determinants of drug use and lead to improved estimates of drug effects in some settings.
One problem with applying analysis of propensity scores to the data set used in the Lancet Psychiatry is that there was a great deal of difficulty matching the admissions to different settings. Moreover, because it was an administrative data set, there are numerous unmeasured, but particularly crucial confounds that could not be included in the propensity matching or in the generalised linear mixed-eﬀects model analyses thereafter. So, in using propensity analysis, the authors threw way most of their data without been able to achieve adequate statistical control for confounds.
We calculated propensity scores for all cases based on a model that included all clinical characteristics before admission as exploratory variables (age, sex, marital status, housing situation, living together with others, employment situation, main diagnosis, comorbid substance use disorder, comorbid personality disorder, comorbid mental retardation, self-injuring behaviour before admission, suicidal ideation before admission, suicide attempt before admission, type of admission, and voluntary admission). These calculations were done on a complete case basis, therefore 36 300 (10·4%) cases with missing covariate were excluded.
There is the temptation to ask “what is the harm in adjustments that involve the loss of only 10.4% of cases, particularly if better statistical control is achieved?” Well,
Overall, 72,869 pairs of matched cases could be created, resulting in a total matched set consisting of 145,738 cases from 87,640 individual patients for the analyses themselves.
So, the authors have lost a nonrandom selection of more than half the admissions with which they started, and they’ve lost the nonindependence of observations in this shrunken data set. Just look at the ratio of 145,738 “cases” to the 87,640 individual patients from which they came. There is a lot of valuable data being suppressed concerning the fate of individual patients when hospitalized in different settings.
How complete is the data available for matching and control of statistical confounds?
We calculated propensity scores for all cases based on a model that included all clinical characteristics before admission as exploratory variables (age, sex, marital status, housing situation, living together with others, employment situation, main diagnosis, comorbid substance use disorder, comorbid personality disorder, comorbid mental retardation, self-injuring behaviour before admission, suicidal ideation before admission, suicide attempt before admission, type of admission, and voluntary admission.
Let’s look at baseline characteristics in Table 1 of the Lancet Psychiatry article. These are the only variables that are available for matching or controlling for statistical confounds.
Recall that the effectiveness statistical controls assumes that all relevant variables have been measured with perfect precision. Statistical control is supposed to eliminate crucial differences among patients so they can be assumed to be otherwise equivalent in likelihood of being admitted to a locked or unlocked ward for the basis of analysis and interpretation. Statistical control is supposed to equip us to make “all-other-things-being equal” judgments about the effects of being in a locked or unlocked ward.
Zero in on main and comorbid diagnoses. What kind of statistical voodoo can possibly be expected to level other differences between patients at higher risk for suicide like the 49% minority with schizophrenia spectrum or affective of disorder versus the others at considerably lower risk? How does it help that this large minority of higher risk patient is thrown in with lower risk patients with organic mental disorder (dementia or mental retardation) and “neurotic, stress-related and somatoform disorders”?*
If there’s any rationality to the German system of care (and I assume there is), at least some crude risk assessment would guide patients with lower risk into less restrictive settings.
And then there is the question of substance use disorder, which was the primary diagnosis for 67,811 (25·5%) of the patients going into locked facilities and 14,621 (18·7%);
Substance use disorder was the comorbidity for another 100 128 (36·9%) going into locked facilities and 28 363 (36·2%) going into unlocked facilities. Issues for substance use disorder and exit security on psychiatric wards are very different than for patients without such disorders. These issues in relationship to absconding or dying by suicide are not going to be sorted by entering diagnosis into a propensity analysis or generalised linear mixed-eﬀects model analyses of a data set shrunken by matching in a propensity analysis.
I conclude that the data set is much less impressive and relevant than it first appears. There are not a lot of suicides. They occur in a heterogeneous population in a length of time in which the patterning of circumstances associated with these characteristics likely changed. Because it was the administrative data set, there were restricted opportunities for matching of patients or control of confounds. Any substantive interpretation of multivariate results requires dubious, unsubstantiated assumptions.
But more importantly, the data set does not provide much evidence for the ideologically saturated claims of the authors or their promoter, Mental Elf. They can pound their drums, but it is not evidence that they are announcing. And patients and their families in both Germany and elsewhere could suffer if the recommendations are taking seriously.
*The “neurotic, stress-related and somatoform disorders” admissions to inpatient units are a distinctly German phenomenon. Persons from the community claiming “burnout” can be admitted to facilities overseen by departments of psychotherapy and psychosomatics. There is ample insurance coverage for what can be a spa-like experience with massage and integrative medicine approaches.