Is there benefit to adding psychotherapy to antidepressants?

depressed person

Special thanks to Don Klein, MD and Bruce Thyer, PhD for helpful discussions, but all opinions expressed are the author’s alone.

Is there any benefit to adding psychotherapy to well-managed treatment with antidepressants? This clinically important question was addressed in a large-scale, exceptionally well-resourced study.

Despite appearing in the respected JAMA Psychiatry, the article will not get the attention it deserves. Its results are complex and nuanced. I had to read it carefully a number of times, along with its accompanying editorial to grasp its full significance. However, the study’s disappointing, downright disconcerting findings will keep it from getting widely disseminated.

There was no press release for the study and very little press coverage so far.  One of the few mentions in the media is balanced —once you get passed the hyped title — and includes quotes from the lead author:

“We know they both work so you assume when you put them together it’s going to work better,” says lead author Steven D. Hollon of the psychology department of Vanderbilt University in Nashville, Tennessee.

He would have liked to see that additive effect for the whole group of depressed patients, but for about two thirds of patients, adding cognitive therapy didn’t matter, Hollon said.

Imagine a study evaluating the benefit of adding antidepressant medication to well-delivered cognitive therapy and that the results were similarly disappointing. The study would be well-publicized (“Depressed persons don’t need meds if they are getting adequate therapy”) in part because of the cognitive therapy lobby, but also because the message resonates with the anti-medication side in the antidepressant wars. Unlike results of the present study, these hypothetical results would be trumpeted because they are consistent with entrenched opinions.

The silence greeting the article has much in common with supporters of a soccer team not wanting to discuss a disappointing loss. Opinions about antidepressants and psychotherapy are as partisan as loyalties to soccer teams. There is nothing sinister going on here. But it makes for a bad progression from the availability of evidence to changing practice.

In this post, I will examine some of the specifics of the study and their broader implications. There are some sobering things to be learned. Among them:

  • State-of-the-art treatment combining antidepressants and cognitive therapy continued over a long period of time leaves many patients still depressed.
  • Adding psychotherapy does not improve outcomes for many patients if they are already receiving well-managed, personalized treatment with antidepressants.
  • Whatever cognitive therapy contributes might be achieved cheaply and more simply with supportive therapy or enhanced clinical management of the antidepressants.
  • Therapists need guidance as to what to do when manualized psychotherapy is not having its intended effect, including how to inform and discuss with patients.

But to begin such discussions we need to dive into the details of the methods and the particular interventions being evaluated. And bring in what we already know about treatment of depression, particularly the gross inadequacies in routine care.

The abstract to the paper is available here. As with other papers behind pay walls, you will have to access this one through a University library or email the corresponding author, steven.d.hollon@vanderbilt.edu. The excellent editorial by Michael Thase is also behind a pay wall, but you can email him at thase@mail.med.upenn.edu.

Finally, the registration of the trial is available here.

jama psychiatryThe study

The objective of the study was

To determine the effects of combining cognitive therapy (CT) with ADM [antidepressant medication] vs ADM alone on remission and recovery in major depressive disorder (MDD).

Overall design

The trial design was exceptionally complex and involved providing acute treatment of up to 18 months, removal of patients who did not meet criteria for remission with 18 months, and transitioning of the remaining patients into continuation treatment.

Acute treatment lasted until the patient met the criteria for remission, defined as 4 consecutive weeks of minimal symptoms; continuation treatment lasted to the point of recovery, defined as another 26 consecutive weeks without relapse. Patients did not need to maintain the symptom levels required for remission to meet the criteria for recovery. Participants who experienced relapse during continuation were required to meet remission criteria again before they were eligible to meet the criteria for recovery. Patients who did not meet the symptomatic criteria for remission within 18 months of treatment were removed from the study and referred for other treatment, as were patients who did not meet criteria for recovery within 36 months. Patients who met only the symptomatic criterion for remission at month 18 (or recovery at month 36) continued treatment until it was determined whether they also met the temporal criteria. Thus, up to 19 months were allowed for remission and up to 42 months for recovery.

As a Phase 4 trial, the investigators assumed that the efficacy of both the ADM and CT have already been established so that the focus could be on whether these two efficacious treatments could be usefully combined. All patients received antidepressant treatment and half were randomized to receiving cognitive therapy as well. There was no pill placebo or other comparison group. The decision not to have a condition controlling for attention and support makes sense, but it introduces ambiguity in the interpretability of the ultimate results, as we will see.

The trial registration

The registration is entitled “Preventing the Recurrence of Depression With Drugs and Psychotherapy” and occurred after the first patients were accruing, not before. The title of the registration is discrepant with the actual published study, which does not mention prevention and downplays recurrence as an outcome.

The patient population had to have recurrent or chronic major depressive disorder, with the exclusion criteria were a current diagnosis of a psychotic disorder, a history of nonaffective psychotic disorder, substance abuse during the last three months requiring detoxification, and having either a schizotypal, antisocial, or borderline personality disorder.

There were three primary outcomes declared:

  1.  Time to remission
  2. Time to recovery
  3. Time to recurrence

Psychopharmacotherapy

All patients received acute treatment until they met criteria for remission. Continuation treatment was provided until the point of recovery…Dosages were raised as rapidly as possible and kept at maximum tolerated levels for at least 4 weeks. Treatment in patients who exhibited only a partial response was augmented with additional medications, and treatment in those who showed minimal response (or little additional response following augmentation) was switched to another ADM. Most patients were given multiple trials with easier-to-manage selective serotonin reuptake inhibitors or serotonin-norepinephrine reuptake inhibitors before treatment was switched to more difficult-to-manage tricyclic antidepressants or monoamine oxidase inhibitors.

So, unlike many ADM trials, this one involved providers being able to switch between antidepressants or add additional medications, not just adjust dosage. This is impressive, state-of-the-art, algorithm-based treatment, involving regularly assessing patient outcomes and making decisions about intensifying or changing treatments, based on set rules. You can find more about algorithm-based treatment here.

cognitive therapy depressionCognitive Therapy

The therapists met weekly for 90 minutes at each site to review cases, with onsite supervision provided by 3 of the authors (R.J.D., P.R.Y., and S.D.H.). The therapists followed the procedures outlined in the original treatment manual for CT of depression, augmented when indicated for patients with comorbid Axis II disorders. The protocol called for 50-minute sessions to be held twice weekly for at least the first 2weeks, at least weekly thereafter during acute treatment, and then at least monthly during continuation. Therapists were free to vary the session frequency to meet the needs of the patient.

This too is state-of-the-art treatment. The three supervisors, including the first author and principal investigator Steve Hollon, have been very involved in the promotion of cognitive therapy for depression and could be expected to provide expert implementation and supervision.

Results

Patients treated with antidepressants alone had a recovery rate of 62.5%, which was raised to 73.5% among those who received CT as well.

  • There were no differences in remission rate between patients assigned to ADM alone (60.3% by month 12) and those assigned to combined treatment (63.6%).
  • Fewer patients assigned to combined treatment dropped out and this group also had fewer adverse events, which the authors attribute to their less time in an episode of depression.
  • Recall that the trial registration indicated the study was supposedly aimed at preventing relapse. You have to search to find that there were no differences between the two groups in relapse, 80 relapses in 54 patients retained in the ADM alone group versus 71 in 48 patients vs in the combined group. Note the modest size of the samples of the two groups for which risk of relapse could even be calculated.

These are not impressive results for CT. The authors performed post hoc subgroup analyses

in which they found no effect for rate of recovery for the two thirds of patients with less severe or less chronic depression, but a sizable effect for the remaining patients who met these criteria. Basically, the number needed to treat (NNT) was 3 in this subgroup of patients with severe, nonchronic depression [Update: 10/19/2014 Corrected from earlier “chronic”]. That is impressive, but needs to be replicated, because analyses were post-hoc and underpowered. Such effects tend to be weaker or disappear all together when replication is attempted in a larger sample.

  • There were still no differences for remission in these subgroup analyses.
  • Recall patients with schizotypal, antisocial, or borderline were excluded. But those with other personality disorders took longer to recover than did patients without a personality disorder.

My interpretation of the results

Three things to keep in mind as we begin discussion of some unsettling results:

  1. The study design does not address whether antidepressants add anything to what is achieved with cognitive therapy. To do so would require a study in which all patients receive cognitive therapy and only some were randomized to antidepressants.
  2. This study did not have an inert control group such as wait list or no treatment and so any naturalistically occurring recovery in the absence of treatment gets attributed to the treatments. To some unknown degree, apparent effects of treatment are actually naturally occurring recovery that would have occurred in the absence of treatment.
  3. The study also does not have a psychotherapeutic control group like supportive therapy. We cannnot know whether any benefits achieved by adding cognitive therapy could have been obtained with a less intensive treatment, like supportive counseling or therapy or even simple encouragement and support for adherence.
  4. The study involved an extraordinary amount of patient-provider a contact time. It is unfortunate that exactly how much is not documented, but this is relevant to evaluating the cost effectiveness of prolonged treatment of depression in the absence of improvement.

The glass-is-half-empty interpretation of the study is that even when given state-of-the-art treatment that is more intensive and long-term than is typical, a quarter of depressed patients do not achieve remission or recovery. The quality and intensity of both pharmacological management and psychotherapy far exceeds what is routinely available in the community and probably what is even reimbursable by insurance.

Routine treatment for depression in the community is quite poor. The mean number of visits in a year for persons with a diagnosis of depression is only eight. Most depression treatment is with antidpressant medication, and most medication is given in non-mental health medical care settings, like primary care. Most primary care patients discontinue ADM treatment shortly after starting. Only 20-30% of depressed persons being treated exclusively in primary care settings receive adequate care and follow up. Said differently, 40% of depressed patients are administered treatment with little or no benefit over what would be obtained by remaining on a wait list, representing about 20% of the total cost of treating depression.

The treatment offered in this study could be seen as a  Rolls Royce. If so, routine careflat tire remains a bicycle with a flat tire.

Some of the problems of routine care lie in poor reimbursement and provider indifference to practice guidelines. The guidelines for meds minimally require a follow-up visit in 5 to 7 weeks to determine whether progress is being made, adherence and patient education are adequate, and whether any adjustment or change in medication is needed. That does not typically happen.

But another part of the problem lies in patients’ perception that such an investment in time and effort is not worth the benefits they received. That may also reflect the inadequacies of the care they get, but it is a cost/benefit analysis that could lead patients to refuse more intensive treatment.

The bottom line, is doing the best we can, treatment for depression will leave many patients dissatisfied and continuing to be depressed. We need to be careful about misleading depressed patients about what they can expect.

More details are needed about how much treatment was provided to whom in this study. It is such an ambitious and costly study, and unlikely to be done again anytime soon, but it leaves a lot of questions on answered. We could at least begin to formulate some hypotheses knowing more what went on.

For instance, what did the cognitive therapists do, what specific interventions did they provide in seeing patients so regularly for so long in the absence of apparent benefit? Were the therapists even aware of the lack of a benefit? The therapists surely had to improvise in going well beyond what is indicated in the manual, which is most adapted to shorter term therapy. Did they simply resort to being supportive?

We cannot rule out that any benefits of cognitive therapy in the study are simply due to nonspecific support, reinforcement of positive expectations, and encouragement to adhere to medication. The cognitive therapy had impressively credentialed and carefully supervised therapists. But was this required for the effects that were obtained?

Finally, providers managing medication in the present study relied on algorithms to make decisions about whether and when to make changes in medication, including switching, augmenting, or simply changing dosage. Many of the specific algorithms do not have strong empirical validation, but the notion does have empirical support that at some point clinicians have a responsibility to change what they are doing does. For instance, there are practice guidelines recommending that after around five weeks, and positive clinical change is not evident, the current treatment should be re-examined.

Manualized psychotherapy has guidelines for to do within the therapeutic model when change is not occurring. But there are typically no guidelines as to when medication should be suggested, a different therapeutic approach or referral to another therapist offered, or therapy should be terminated as futile.

Certainly we can conceive of situations where such judgments are warranted, but there is almost no discussion in the psychotherapy literature. In the case of cognitive therapy for depression, observational data derived from clinical trials could be used to suggest when change should be occurring, and if it is not, the likelihood that it will occur later. Out of respect for patient autonomy and informed consent, I think it is incumbent on psychotherapists to evaluate the evidence they have and come up with provisional recommendations for switching or stopping treatment that can be empirically tested.

 

 

 

 

20 thoughts on “Is there benefit to adding psychotherapy to antidepressants?”

  1. The question of whether CT adds anything to treatment with antidepressants is part of a larger question in depression treatment about whether combined treatment adds substantially to the efficacy of an already established monotherapy. It would appear that combination treatment leads some somewhat increased outcomes in depression treatment but this effect is small. As this study suggests, it’s likely that the effect is non-existent in some patients groups but large in other groups. Although meta-analyses do not provide conclusive answers, the existent data suggests that there’s probably about as much benefit to adding psychotherapies to antidepressants as there is to adding antidepressants to psychotherapy:

    Cuijpers, P., van Straten, A., Warmerdam, L., & Andersson, G. (2009). Psychotherapy versus the combination of psychotherapy and pharmacotherapy in the treatment of depression: a meta‐analysis. Depression and anxiety, 26(3), 279-288.

    Cuijpers, P., Van Straten, A., Hollon, S. D., & Andersson, G. (2010). The contribution of active medication to combined treatments of psychotherapy and pharmacotherapy for adult depression: A meta‐analysis. Acta Psychiatrica Scandinavica, 121(6), 415-423.

    Like

    1. I agree with your larger points, but we need to stay sensitive to how incomparable the present study is to the ones collected in these meta-analyses. It involves extraordinary clinical management in terms of intensity and length of time, as well as length of time of cognitive therapy. It also allowed much greater flexibility in terms of prescriber flexibility with respect to choice and combination of medications.

      Like

  2. Thank you for this interesting and informative post. I’d like to offer two additional observations about this study that may be relevant in interpreting its implications for the efficacy of adding psychotherapy to antidepressants in the treatment of depression.

    First, it’s interesting that cognitive therapy was chosen for this study given that some of the authors previously published a clinical trial demonstrating that behavioral activation was more effective than cognitive therapy. This leaves open the question of whether behavioural activation would have fared better as an augmentation strategy in this study.

    Second, to your point that the treatment offered to participants in this study was a “rolls royce,” I offer the following quote from p. 1160 of the article:

    “The protocol called for patients to meet with their prescribing practitioner weekly for the first month, biweekly thereafter during acute treatment, and monthly during continuation. The initial session lasted 30 to 45 minutes, with
    subsequent sessions approximately 20 minutes.”

    This is far more prescriber contact than ever occurs in real-world settings, and as such this treatment might best be construed as a combination of medication management and supportive therapy. Is it possible that antidepressant treatment delivered in this manner might be especially effective, and therefore especially unlikely to be augmented by psychotherapy, relative to the real-world practice of pharmacotherapy?

    I’d be interested in hearing your thoughts. Again, thank you for this informative blog post.

    Like

    1. Thanks for your thoughtful comments. Dimidjian et al did not credibly show behavior activation was superior to CT, although the spin of the article encourages that interpretation. Furthermore, Hollon was an author, but not an original investigator on that study. He became involved after the untimely death of the principal investigator, Neil Jacobson. As reported, the study is an atrocity. First, Neil broke blind and discussing the strong findings of the study before the final data were even in. Second, in the chaos after his death, investigators were able to look at the data before deciding to continue. There is no stopping rules on the final study. Third, the apparent superiority of behavior activation occurs only in underpowered, post hoc subgroup analyses. They were odd things going on, including on do influenced by some outliers. The small cell sizes in the subgroup analyses were susceptible to that.

      If you look at the effect sizes is of this past study that are entered into meta-analyses, along with the quality ratings, you get a different picture of it.

      I agree that there is much more prescriber contact then in the typical study of antidepressants or routine care. I think it is amazing that the investigators kept patients coming back for so long, and the supportive contact provided by the psychotherapy may have been a factor. Another interpretation is that this intensive clinical management had many of the features of psychotherapy and so not much could be added. Finally, I am just not sure what cognitive therapists could have been doing after such a long term treatment because I think there manual adapted to shorter treatment runs out of suggestions except for to do what is not working.

      Like

  3. I think Deacon makes a powerful point. It would appear that all patients were getting psychotherapy. Does biweekly mean twice a week? Or every other week? If it means twice a week, the study seems to me to be meaningless. If every other week, not sure.

    Like

  4. I think that there was some flexibility, as seen in the quote below, but it appears that a lot of therapy got delivered:

    The protocol called for 50-minute sessions to be held twice weekly for at least the first 2weeks, at least weekly thereafter during acute treatment, and then at least monthly during continuation. Therapists were free to vary the session frequency to meet the needs of the patient.

    Like

  5. Also important to consider that this study specifically selected chronic or recurrent patients. Contrary to popular belief, most depressed patients are not chronically depressed or have recurrent histories. Those who do represent a challenge in terms of treatment and account for the bulk of the public health expenditure on depression and also take up a lot of provider contact…

    Like

  6. jim i think you missed the point. the medication only group was receiving psychotherapy. the protocol reads: “The protocol called for patients to meet with their prescribing practitioner weekly for the first month, biweekly thereafter during acute treatment, and monthly during continuation. The initial session lasted 30 to 45 minutes, with subsequent sessions approximately 20 minutes.” the study design is flawed and assumes that cbt is different then counseling with a prescriber. but this assumption is the assumption made by the church of CBT, not the finding of science–dodo hypothesis–that shows that nonspecific factors are the active ingredient in therapy. so the comparison was between one group–med only– that got a healthy amount of psychotherapy and the group–combined tx– that got more.
    Neither condition reflects the real world. Who funds this stuff?

    Like

    1. That is the nature of clinical management in these kind of trials and I think there was particular concern about retaining patients without it. I agree that it really shrinks the possibility of a specific contribution of CT because it contains many of the nonspecific features of it.

      The protocol was funded by NIMH but note that the principal investigator is one of the most prominent promoters of CT, he’s not a psychiatrist advocating medication.

      I think what you are implicitly getting at is whether we really should move on to phase 4 trials that don’t involve any psychotherapy control group. We left struggling to figure out whether there is anything active about the CBT in play. I suspect the principal investigator was being responsive to what he perceived were pressures from NIMH to move out of phase 3 trials.

      Like

  7. i think an irony of his study is that it distorts the evidence for psychotherapy and erodes the case for psychotherapy. All patients in the study were getting psychotherapy. So the comparison is not between medication and medication and psychotherapy, it is between medication and either low dose v high dose psychotherapy. But Hollon doesn’t recognize that because he is an ideologue committed to his sect.

    Like

  8. Although the Hollon et al study has already received much invalidating criticism there are a few more substantive points to make , of some general import.
    The design does not face the question whether CBT is any better than placebo in the treatment of depression . Why should it ? Certainly the conventional wisdom is that CBT is a useful intervention . But if it doesn’t beat pill placebo why should anyone be interested? However, I was alerted to this question by being on the Scientific Advisory Board of the classical TDCRP study (Elkin I et al Arch Gen Psychiatry. 1989, but see Klein D and Ross DC Neuropsychopharmacology 1993 and Stewart J et al Journal of Clinical Psychopharmacology1998).
    Unsystematically , I happened on 6 other studies that included both CBT ,a pill placebo group and an active medication group The inclusion of an active medicine group allowed both medication and pill placebo to be conducted double blind. Of course that is not the case for psychotherapies.
    In all 7 studies ,CBT was not found to differ from placebo. Rather than detail these studies , can somebody provide a reference to such a study where CBT exceeded placebo by statistical significance ? I won’t even require the more sensible clinical significance. I have searched diligently and can’t find one. Remarkably, Hollon was second author on the most interesting of those studies ,Dimidgian S et al J Consult Clin Psychol. 2006 .

    Another issue is the sample composition. In one place it is stipulated that a score => 14 on the 17 item Hamilton was required. This is already an atypically low threshold . However later it is stated that the 17 item Hamilton was modified by an additional three items. So it is no longer 17 items and the threshold has dropped from conventional grasp.

    The major statistical approach is the esoteric “subdistribution hazard model”. This may be entirely correct but confidence is somewhat shaken by their statement , “All evaluations were recorded and a subset was rated across sites to establish interrater reliability”. These calculations resulted in the remarkable intraclass correlation of 0.96. However using ratings across sites cannot yield a reliability coefficient since that requires multiple independent ratings of the same subjects.

    The authors find no significant differences with regard to remission and a “modest (10%) increment” with regard to recovery. The article eschews the use of easily understood effect sizes . The cited t and p values obscure the importance of sample size in such calculations.
    A very useful equation is r^2= t^2/( t^2 +degrees of freedom). This amounts to a point biserial correlation. Think , 1 variable is treatment ,coded zero for 1 group and 1 for the other. This is then correlated with the respective outcomes. The degrees of freedom are the total sample size minus two. Applying this formula to the central recovery rate difference, where t=2.45,degrees of freedom = 451, then despite the impressive P=.01 it turns out the r=0.12, which diminishes impact.

    It should be noted that the authors , having spent less than half a page on their meager findings spend two figures and about two pages on the severity by treatment interaction , which was not hypothesized or even referred to in the introduction.
    This is post hoc data massage and serves at best an exploratory function. However,that is not made clear.
    Shifting focus, Jim refers to Michael Thase’s “excellent editorial” that accompanied this JAMA paper. Unfortunately the editorial accepts the “severity by treatment” finding as solid and makes “plausible that the studies that observed the smallest effects in favor of combined treatment “ had inappropriate samples. Thase also refers to the difference that the “treatment by severity” analysis reveals as a very large effect : 81.3% vs 51.7% , t=3.96,P=0.001,df=145 but r=0.32 which diminishes enthusiasm. Thase states ,”There is no debate about whether CT should be thought of as a first line option” but I think it is well worth debate.

    Like

  9. Donald Klein wrote, “The inclusion of an active medicine group allowed both medication and pill placebo to be conducted double blind.” Inclusion of a pill placebo is necessary but not sufficient for blinding. Even in a placebo-controlled medication trial, participants and/or investigators may guess at a level that exceeds chance which treatment was received, which invalidates the trial because the drug effect is confounded with expectancy effects. Remarkably, the integrity of the blind is almost never assessed in antidepressant trials, and this omission is considered acceptable by the FDA and academic psychiatry. Dr. Klein, is there reliable evidence to support the integrity of the double blind antidepressant trials? If so, I would be very interested to see it.

    Like

    1. Maybe Don Klein would be willing to revisit the issue. In the past, he has done an excellent job of demolishing this objection to the pill-placebo condition. First, relying on guessing would be confounded with the well-known early effects of antidepressants. Second, it suggests that foul tasting versus neutral placebo pills would be more efficacious than neutral tasting ones. Not the case. But I will leave to Dr. Klein to decide whether he wants to repeat his arguments elsewhere.

      Like

  10. Just wanted to clarify a basic finding of the paper. From the abstract (parenthetical text removed for clarity, emphasis mine):

    Combined treatment enhanced the rate of recovery vs treatment with ADM alone. This effect was conditioned on interactions with severity and chronicity such that the advantage for combined treatment was limited to patients with severe, nonchronic MDD.

    However, you wrote:

    The authors performed post hoc subgroup analyses in which they found no effect for rate of recovery for the two thirds of patients with less severe or less chronic depression, but a sizable effect for the remaining patients who met these criteria. Basically, the number needed to treat (NNT) was 3 in this subgroup of patients with severe or chronic depression.

    Since I don’t have access to their full paper, did they get their abstract wrong, or should your text have the chronicity reversed?

    Like

  11. Colleagues,
    The question is : does the double blind remains blind after treatment starts and isn’t this an important confound ,destroying the benefits of randomization. Difficult to answer from the data produced from the usual positive study because of the confound of improvement and the guess about what you are on.
    But this question can be turned into a relevant review of ineffective agents.
    The expectation of those holding that piercing the blind would result in a fallacious estimate of efficacy would be that the this would result in a flood of ineffective drugs appearing efficacious because side effects contribute to blind piercing and increases the expectation of success by patients and raters.
    However the experience of double blind placebo controlled phase 3 studies is that almost all fail efficacy determination , despite side effects— so they can’t be marketed, leading to a lot of disappointed Pharma.
    What about active controls? Some believe that side effects positively bias the rater while others believe that side effects cause the patient to guess they are on active medication ,so their expectation of success is heightened.
    Quitkin and I reviewed the literature where active controls were found equivalent to supposedly active psychotropics. The usual expectation is that the supposed active agent would be measured as well as generally reported-say 60% considered improved , but active controls would aso be measured at 60% so there would no significant difference. Our review of such reports agreed that in these trials , active controls could not be discriminated from supposedly effective psychotropes .
    However,that was because in these trials the reportedly effective psychotrope did very poorly -say 30% ,while the active placebo did the same as is usually reported for inactive placebo,also about 30 %.
    So our conclusion is that piercing the blind, as it occurs in clinical trials designed to be double blind , is of minor or nonexistent consequence.
    One other study by Judy Rabkin et al of our group seems relevant. In a series of trials she developed a sample of patients given placebo who had done well.
    Randomly they were told :1) this agent seems good for you let’s continue and monitor or 2) you were on placebo which shows you can do well without medication. However we will check up on you to see how you do.
    If we believe that receiving placebo somehow activates recovery,a usual inference is that,over time, group 1 should do better than 2.. In fact Judy et al found no difference.
    Those concerned that the loss of double blind is an important confound for clinical trials , should produce some relevant data. It remains a speculation that does not fit the available facts.

    Like

    1. Dr. Klein, your thorough replies are impressive. I have some thoughts for you to consider:

      On, “The question is : does the double blind remains blind after treatment starts and isn’t this an important confound ,destroying the benefits of randomization”.
      A good CNS study that requires blinding will have an ‘exit-analysis’ at the end of a trial to confirm the blind was maintained. If not, the study may be discarded, and/or the further development of the drug may be terminated.

      On, “However the experience of double blind placebo controlled phase 3 studies is that almost all fail efficacy determination , despite side effects— so they can’t be marketed, leading to a lot of disappointed Pharma.”
      Sometimes the business dept of a drug company wants to push thru the registration and they go to Phase 3 in spite of the inability to maintain a blind in phase 2. That does not mean the science of blinding is wrong, it’s just wrongly applied by the business dept. and either those drugs do not work, or those studies should have been redone. It has nothing to do with the value-and necessity-of filtering out the bias resulting from large random error in psychiatric drug studies where endpoints are almost always SUBJECTIVE. How else can you filter out expectation and hope? Double blinding, that is shown to be maintained by exit analysis is much better than a psychotherapy study where it is clearly impossible to have a single (=subject) blind, or double blind. Blind raters are misnamed as “single-blind” (only blind subjects are defined as actually being single-blind), and blind raters only rate whatever bias has come out of the subject-treater system. Also, you can not mix blind arms and unblind arms in the same study and compare them. The data is born from completely different designs-essentially handicapping the blinded arms in comparison- and invalidates study logic.

      On, “Quitkin and I reviewed the literature where active controls were found equivalent to supposedly active psychotropics. The usual expectation is that the supposed active agent would be measured as well as generally reported-say 60% considered improved , but active controls would also be measured at 60% so there would no significant difference.”
      I think you and Dr. Q, have replicated the holy grail of “non-inferiority”, frequently taken advantage of by drug companies when they want to position a new drug vs old in psychiatry, e.g., its not a superiority study. When study arms are unblind then many subjects “respond” (response is usually defined as 50% improvement, it’s not “recovery”) in all groups. You are studying psychiatric conditions with large random error in subjective endpoints. There is no mystery if a blind is not maintained as shown by exit analysis. If you study an objective endpoint that has a measurable biological parameter, i.e., MI, stroke, death, then blinding is less crucial. That’s not the case in psychiatry, unless you study suicide rates, but these are too few to study in a controlled study anyway.

      On, “One other study by Judy Rabkin et al of our group seems relevant. In a series of trials she developed a sample of patients given placebo who had done well. Randomly they were told :1) this agent seems good for you let’s continue and monitor or 2) you were on placebo which shows you can do well without medication. However we will check up on you to see how you do.
      If we believe that receiving placebo somehow activates recovery,a usual inference is that,over time, group 1 should do better than 2.. In fact Judy et al found no difference.”
      It is not necessarily true that knowing one received placebo will cause one to lose enough effect to discern difference if you are looking for non-inferiority. If you are looking for superiority, the hurdle is so high statistically, that these kinds of studies will not have enough statistical power to show superiority. More seriously, in Rabkin’s arm 2 the subjects were even told by treaters that “you can do well without medication” thus giving them hope and expectation and invalidating the study. They could have equally be told, “you have not been on active medication and now we expect this effect not to last”. There are too many potentials for random error in this study where endpoints are subjective so that the study is not robust in any way to make conclusions.

      I think these points can explain the facts you mention that the “data don’t fit the facts”.

      Like

    1. a large-scale, exceptionally well-resourced study.
      From the retraction of the paper and replacement with a new set of results, it would seem that the “well-resouced” part did not extend as far as counting patients and remissions correctly.

      Like

  12. This paper : http://f1000research.com/articles/4-639/v1
    goes further in describing blinding problems with the Hollon et al. paper. Also includes a colorful rebuff by Hollon that show the problems in the way he conceptualizes clinical trial science logic.

    In addition, there is a conflict of interest/academic nepotism issue because the director of Dr. Hollon’s Dept. at Vanderbilt is Dr. Stephan Heckers, Editor-in-Chief of JAMA Psychiatry. He was on the Editorial Board of JAMA Psychiatry at the time of Hollon et al.’s submission.

    The Editorial by Thase in the same issue:
    Large-Scale Study Suggests Specific Indicators for Combined Cognitive Therapy and Pharmacotherapy in Major Depressive Disorder JAMA Psychiatry. 2014;71(10):1101-1102.

    also seems to be a conflict. Thase and Hollon have previously worked together on the efficacy of medication and cognitive therapy in depression:

    1. Thase ME, Friedman ES, Biggs MM, Wisniewski SR, Trivedi MH, Luther JF, Fava M, Nierenberg AA, McGrath PJ, Warden D, Niederehe G, Hollon SD, Rush AJ. Cognitive therapy versus medication in augmentation and switch strategies as second-step treatments: a STAR-D report. Am J Psychiatry. 2007 May;164(5):739-52.
    2. Hollon SD, Jarrett RB, Nierenberg AA, Thase ME, Trivedi M, Rush AJ. Psychotherapy and medication in the treatment of adult and geriatric depression: which monotherapy or combined treatment? J Clin Psychiatry. 2005 Apr;66(4):455-68. Review.

    and one more recent article this time with Hollon as 2nd author published in JAMA Psychiatry where his Dept. Chair Heckers is Editor-in-Chief of JAMA Psychiatry:

    Erica S. Weitz, MA; Steven D. Hollon, PhD; et al. Baseline Depression Severity as Moderator of Depression Outcomes Between Cognitive Behavioral Therapy vs Pharmacotherapy. An Individual Patient Data Meta-analysis JAMA Psychiatry. Published online September 23, 2015.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s