There has been a tremendous movement toward evidence-based treatment in clinical psychology over the past decade. On its face, this is a good thing — the idea that we should use scientific findings to make sure the types of treatment we’re using in psychotherapy actually work. My own clinical training included a wide array of these empirically based treatments, and I happily use many of their key techniques. I also continue to pay close attention to new clinical research, and frequently review the literature in the course of formulating treatments for my patients.
However, I see some really serious problems with evidence-based psychotherapy, problems that I believe are actually harmful to the patients, and to the credibility of clinical psychology.
In an effort to eliminate confounding variables that could interfere with study results, research psychologists try to exclude patients suffering from any ailments outside of the single diagnosis that the treatment protocol is designed for. This is in line with the medical model of psychology, which attempts to isolate discrete psychological disease processes and treat them directly, much the same way that a physician might take a throat culture and then prescribe antibiotics to treat strep throat.
The problem is that psychological issues don’t really work that way. The diagnoses used in clinical psychology are generally not isolated disease processes the way viral or bacterial infections are. Most people don’t present with a single discrete disorder, but rather a spectrum of symptoms, personality characteristics, and social factors which can all be considered a part of the syndrome which has led them to seek treatment. This requires holistic treatment that identifies and addresses the underlying causes.
In many ways, mental illness is similar to metabolic syndrome — another problem for which the medical model is proving inadequate. All day long I see patients with separate diagnoses of diabetes, hypertension, hyperlipidemia, cardiovascular disease, obesity, sleep apnea. Each one of these identified problems are then treated by suppressing the symptoms. For diabetes there are pills to lower blood sugar, for apnea there is the CPAP device, for high blood pressure there are pills to lower the blood pressure. Even the preventive advice that is rendered is symptom-focused: switch to whole grains so the sugar will enter your system more slowly, take less salt to reduce blood volume. But all of the symptoms are actually related. They all result from the same underlying problem of metabolic dysregulation, and until that total syndrome is addressed, all of the symptoms will continue to get worse over time no matter how well they are managed.
Mental illness is like that. There are genetic, social, and cultural predispositions that contribute, but in order for the syndrome to be expressed a disruption in the management of psychological resources — such as emotions, thoughts, internal imagery, relationships, etc. — must take place. A cascade of compensatory actions are then set in motion, resulting in the symptoms we see on the surface: depression, anxiety, nightmares, panic attacks, hallucinations, substance abuse, etc.
I’ve treated substance abuse, and rarely seen a case where depression or anxiety were not also present. I’ve treated PTSD, and it’s the same story. Would you believe that most sufferers of severe, adult-onset, chronic pain are survivors of some form of childhood abuse? Even your basic top-level disorders, anxiety and depression, actually share many symptoms and frequently develop simultaneously. And many patients’ suffering simply doesn’t fit any currently available psychiatric diagnosis.
So what happens when you pick a single disorder, cherry-pick patients who don’t meet criteria for any other problem, and study their response to an intervention? They get better! In fact, they mostly get better even when you just put them on a waiting list. Because these are the easiest, most uncommonly uncomplicated cases. They are not a good representation of the patients that actually come in for treatment. Therefore, the studies are not generalizable; they tell us very little about how well a treatment will work in the real world.
Because there are so many potential confounding factors in research with live humans, you can never read too much in to any one study. Once a study has been repeated a few times, preferably by different researchers under different conditions, then we can start to get a little more confident that we’re seeing a real treatment effect and not just an artifact of some unpredictable condition of the study. But that’s not happening very much in the field of clinical psychology. Most psychotherapy outcome studies are simply never replicated.
Part of the problem is publication bias: for a variety of reasons, papers describing a failure to replicate previous studies don’t get published as often. Many of the findings could not be replicated regardless, because they are false positives — the study designs do not adequately discern between treatment effects, placebo effects, and random effects. There is tremendous pressure on academic researchers to produce positive results, and this leads to both conscious and unconscious bias in the way research is carried out.
Finally, psychological treatments, like medical treatments, are subject to decline effects. Over time, They get less effective. Later studies find lesser effects than earlier studies, even when the research design is sound. This could be due to changes in researcher enthusiasm, changes in the patient populations that the treatments are being tested on, or changes in the cultural context in which treatments are being administered. Regardless, most published research findings are false. It is therefore highly problematic to make those findings the basis of clinical treatment.
Here are some excellent articles about these problems:
A large number of controlled and naturalistic studies have found that the therapist providing treatment is more important to the outcome than the type of treatment being provided. This makes a certain kind of natural sense to most people, yet psychologists remain resistant to the idea and the reasons for it have not been adequately studied. We don’t really know what makes the difference between a really great therapist, a mediocre therapist, and an ineffective therapist. Treatment adherence does not explain the difference.