Psychotherapy vs. Placebos
By Garry Cooper and Kathleen Smith
Over the past 25 years in the typically low-key, conflict-avoidant world of academic psychology, University of Pennsylvania professor James Coyne has been no stranger to controversy and debate. Although a therapist himself and a proponent of psychotherapy, he specializes in debunking what he considers the hype and exaggerated claims for the efficacy of a range of treatment approaches, advocating for higher standards of scientific rigor than is generally accepted as evidence for therapy’s effectiveness. For example, he’s challenged the findings purporting to show that support groups and alternative psychological treatment extend the life of cancer patients, cast doubt on some of the positive findings in the field of behavioral medicine, and raised the provocative question of whether the benefits of positive psychology—an approach that often ignores the influence of class and social factors on behavior—are “mainly for rich white people.”
Coyne’s latest target is the influential work of psychologist Irving Kirsch, bestselling author of The Emperor’s New Drugs, an expose of the psycho-pharmaceutical industry, and author of two much-cited meta-analyses testing the effectiveness of antidepressants against placebos for treating depression. Rather than relying solely on published studies that rarely include reports of unsuccessful clinical drug trials, Kirsch used the raw data that pharmaceutical companies are required to submit to the Food and Drug Administration. By eliminating the skew toward exclusively positive antidepressant research, he concluded that there was no significant difference between placebos and drugs in the treatment of depression, a finding that garnered him a great deal of media attention, including a featured spot on 60 Minutes.
In his 2008 study, Kirsch and his coresearchers had found an effect size for antidepressants of .32 in comparison with placebos. Since this was below the .50 effect size that they claimed was the cut-off for clinically effective results, they concluded that antidepressants were only negligibly more effective than sugar pills. In other words, a little over seven patients would’ve had to be treated to find one who was helped more by antidepressants. In a June blog post for Mind the Brain, Coyne criticized Kirsch’s .50 figure as an arbitrary and unrealistically high bar, and even quoted a noted British researcher who said that any study finding an effect size of .50 or greater is likely to have a flawed or biased methodology. Thus, says Coyne, Kirsch’s meta-analysis actually made a moderate case for antidepressants.
In his blog posts, Coyne challenged the complacency of his clinical colleagues who believed that Kirsch’s studies, by debunking antidepressants, somehow vindicated psychotherapy. He suspected that if therapy were subjected to a comparison trial against placebos, it wouldn’t do any better than antidepressants. So he contacted Pim Cuijpers of the University of Amsterdam, a researcher with an extensive database of psychotherapy and depression clinical trials, and suggested they do a meta-analysis that matched an antidepressant placebo against psychotherapies. They chose 10 studies comprising 12 therapies that had done this—including cognitive behaviorally based therapies (straight cognitive behavioral therapy, problem-solving, and behavioral activation) and short-term psychodynamically oriented therapies (supportive-expressive and interpersonal psychotherapy). Their meta-analysis, published in April as an online advance article for Psychological Medicine, found that the effect size of psychotherapy over a placebo came in at about .25—worse than what Kirsch had found in his comparison of antidepressants to placebos, but in statistical terms, roughly the same effect size as antidepressants over placebos.
Ever the intellectual provocateur, Coyne insists that he isn’t dismissing psychotherapy in the same way that Kirsch downplayed the effect of antidepressants by concluding they’re no better than a placebo. In fact, he notes that the statistical leveling of meta-analyses can disguise the fact that within a large sample size, some people are helped hugely by a treatment. His study suggests, he says, that psychotherapy is as effective as antidepressants, and that both outperform placebos. Although that’s a modest claim for psychotherapy, it is, he insists, better than Kirsch’s conclusion that the effective ingredients of both psychotherapy and placebos—the positive expectations of patients and, in the case of therapy, the therapist’s belief in the efficacy of therapy—are the same. Taking a cognitive behavioral bent, Coyne goes on to say that therapy also works by helping clients actively test the environment against their depressing beliefs and discover that their thoughts and gloomy expectations don’t match up with reality. According to Coyne, this kind of reality-based learning enhances the possibility of future change in a way that placebos cannot.
In fact, Cuijpers and Coyne’s meta-analysis found an overall effect size near .50 for psychotherapy that shrinks to .25 only when the highest quality of the 10 studies are considered. That effect size is still less than .8—an impressive average effect size, which psychotherapy researcher and Family Process editor Jay Lebow says many studies of psychotherapy typically achieve when comparing it to other active treatments or to no treatment.
Ultimately, however, many psychotherapy advocates agree with psychologist and author Michael Yapko, a strong proponent of short-term therapy for depression, who says that any study focused on broad methods and general treatment effects is missing the point. According to Yapko, there are too many variables involved in doing therapy for depressed individuals to set up objective research conditions that ignore the overriding impact of the practitioner on the results—give a half dozen therapists the same manual to follow, and their effectiveness with clients will be widely divergent. Says Yapko, “You can’t treat depression globally or apply psychotherapy as a global construct and expect to find any meaningful results or differences.”
J. Coyne. “Is Psychotherapy for Depression Any Better than a Sugar Pill?” Mind the Brain Blog, June 25, 2013; P. Cuijpers and others.
“Comparison of Psychotherapies for Adult Depression to Pill Placebo Control Groups: A Meta-analysis,” Psychological Medicine/FirstView Article, April 2013; I. Kirsch and others.
“Initial Severity and Antidepressant Benefits: A Meta-Analysis of Data Submitted to the Food and Drug Administration, PLoS Med 5, February 2008.