The Importance of Measuring Outcomes
Achieving a sense of healing involvement requires a continual evaluation of where we are compared to where we’ve been. We must keep examining our clinical experiences, looking for evidence of our therapeutic mastery and mining our sessions for the golden moments that replenish us. But if our sense of healing involvement with clients is tied to our ongoing sense of making a difference, how do we know we’re truly helping? You know when a roof is tarred or a tank drained, but how do you know when psychotherapy is beneficial? Therapeutic outcomes are hard to define and harder to measure.
The research literature offers strong evidence that therapists aren’t good judges of their own performance. Consider a study by Vanderbilt University researcher Leonard Bickman and associates reported in 2005 in the Journal of Clinical Psychology: In Session in which clinicians of all types were asked to rate their job performance from A+ to F. About 66 percent ranked themselves A or better. Not one therapist rated him- or herself as being below average! If you remember how the Bell Curve works, you know that this isn’t logically possible.
Further evidence of therapists’ self-assessment difficulties is found in a study by Brigham Young University’s Corinne Hannan, Michael Lambert, and colleagues, reported in the same issue of the Journal of Clinical Psychology: In Session. They compared therapist judgments of client deterioration with actuarial predictions for 550 clients (algorithms based on a large database of clients who completed the Outcome Questionnaire 45.2). The average deterioration rate for psychotherapy clients is about 8 percent, so about 40 clients in this study of 550 would likely worsen with treatment. Therapists accurately predicted deterioration in only 1 out of 550 cases. Thus, of the 40 clients who deteriorated, psychotherapists missed 39. In contrast, the actuarial method only missed 4.
It’s not that we’re naïve or stupid; it’s simply hard, if not impossible, to accurately assess your effectiveness on a client-by-client basis. For this, you need some quantitative standard as a reference point—you need to measure outcomes. I can hear you groan, but I’m not talking about outcome measurement for the sake of bureaucratic “accountability” to funding sources or for justifying your existence by demonstrating your “proof of value” or “return on investment.” Rather, measuring outcomes allows you to cut through the ambiguity of therapy, using objective evidence from your practice to help you discern your clinical development without falling prey to that perennial bugaboo of the therapeutic endeavor: wishful thinking. Taking the time to measure outcomes relates directly to both having an awareness of our mastery over time and experiencing a sense of current growth.
How does outcome measurement further cumulative career development and currently experienced growth—the two keys to greater healing involvement with clients? First, cumulative career development is another way of saying that we’re “getting better all the time.” The routine collection of outcome data allows you to determine your effectiveness over time, and gives you a base for trying out and accurately evaluating new strategies. Begin simply by entering your outcome scores into a database, and keeping track of them on an ongoing basis: intake and final session scores, average change score (the difference between average intake and final session scores), and, ultimately, the percent of your clients who benefit. If you can review and assess your clinical work through the years, you can actually learn from your experience, rather than simply repeating it and hoping for the best.
Of course, finding out how effective you really are can be risky business. What if you find out that you’re not so good? What if you discover that you’re—say it isn’t so!—just average? Measuring outcomes takes courage, but so did walking into a consulting room for the first time to counsel someone in distress—and so does doing it day in and day out.
There are some good reasons to take the risk, however. Consider the results of a 2009 investigation of client-outcome feedback that I conducted in Norway with psychologist Morten Anker and family therapy professor Jacqueline Sparks and published in the Journal of Consulting and Clinical Psychology. The largest randomized clinical trial of couples therapy ever done, it found that clients who gave their therapists feedback about the benefit and fit of services on two brief, four-item forms reached clinically significant change nearly four times more than non-feedback couples did. (Both measures are available to download for free at www.heartandsoulofchange.com.)
So it’s clear that clients benefit from the use of feedback forms, but so do we. Tracking outcomes improved the results of 9 out of 10 therapists in this study. In fact, Anne, a therapist in the low-effectiveness group without feedback became the therapist with the best results with feedback. This heartening finding suggests that, regardless of where you start in terms of your effectiveness, you, too, can be among the most successful therapists if you’re proactive about tracking your development.
As for the relationship of measuring outcomes to currently experienced growth, as Orlinsky and Rønnestad have shown, the old therapeutic cliché is true: therapists really do believe that clients are their best teachers. Clients provide the opportunity for constant learning about the human condition, different cultures, and worldviews, as well as the myriad ways that people transcend adversity and cope with the unthinkable. But while we learn a great deal almost by osmosis from our clients, tracking outcomes takes the notion that “the client is the best teacher” to a different, higher, and more immediately practical level. Tracking outcomes with clients not only focuses us more precisely on the here-and-now of sessions, it takes us beyond mere intuition and subjective impressions to quantifiable feedback about how the client is doing. We get unambiguous data about whether clients are benefiting and whether our services are a good fit for them. From their reactions and reflections, we receive information that we can use in figuring out the next step to take in therapy. In short, tracking outcomes enables your clients—especially those who aren’t responding well to your therapeutic business-as-usual—to teach you how to work better. In fact, clients who aren’t benefiting offer us the most opportunity for learning by helping us to step outside our comfort zones.
Recall Anne, one of the lowest-scoring Norwegian therapists, who became the best therapist when she collected client outcome and alliance feedback. Here are her reflections about the relationship between her clients’ feedback and her sense of currently experienced growth:
Discussing when clients were not benefiting helped me be more straightforward, more courageous. I inquired more directly about what we could do together. . . . Clients taught me how to handle it when I was not useful. Clients and I reflected more on their changes and on the sessions. We got more concrete regarding change, how it started, and what else would be helpful.
In all, collecting outcome data with clients helped me take risks and invite negative feedback. So I asked for it, showed I could handle it, validated it, and then incorporated it in the work. That’s what therapy’s all about—real collaboration.