Are Licensing Exams Failing Clinicians?

On a cool June morning, Christian kissed his wife Julia* goodbye, stepped off the New York subway, and made his way toward a tall, nondescript office building in midtown Manhattan. He arrived almost 30 minutes early for his appointment, but a long line of fellow aspiring therapists was already waiting outside the front door. They were all there for the same reason: to take the single most important test of their clinical career. Passing the Examination for Professional Practice in Psychology, also known as the EPPP, was the last milestone in a multiyear journey, one that would launch them outside the stratosphere of junior clinicianhood. No longer would they be just interns, or postdoctoral students, or make a small fraction of what their senior colleagues were being paid. Finally, they’d be real therapists.

Christian was nervous. He’d barely slept the night before. His back ached too, a byproduct of having spent most of the last two months’ nights and weekends hunched over a tall stack of test prep materials on a not-so-ergonomically friendly IKEA couch—on top of spending his daytime hours doing more sitting with clients.

After a few minutes, the front door cracked open, and a testing official beckoned the line of students forward. They walked up a flight of stairs until they reached the testing center, where each student was photographed, finger­printed, and pointed toward a locker where they’d stow their personal belongings for the next four hours. Christian placed his wallet, keys, and phone—still buzzing with well-wishes from friends and family—into one and closed the door.

The testing room was spartan, almost dystopian: there were no windows, just cubicles, each filled with a single computer. Wall-mounted security cameras pointed down onto the students below.

Christian took a seat and waited until the proctor gave the green light, then hit the start button on his screen. But almost immediately, he saw questions he didn’t recognize, and his mind began to race. I don’t even know what this one is talking about! he thought. I’ve never seen anything phrased like this. Did I prepare enough?

Failing the exam wouldn’t just mean paying another $700 to retake it. It would mean months of additional study. It would potentially delay his start date at the job he’d recently accepted at Bellevue Hospital, costing him even more money. Equally devastating, it would mean the shame of having to tell his friends, family, coworkers, and Julia—everyone who’d been rooting for him—that he’d failed.

Christian isn’t alone. Many test-takers say the EPPP is notoriously difficult to prepare for. For one, they say, the Association of State and Provincial Psychology Boards (ASPPB), which oversees the EPPP, keeps an unnecessarily tight grip on exam content and information about how it’s developed. They claim there’s a shortage of adequate study materials available, in part due to the association’s secrecy. Perhaps most importantly, they claim that many test questions are detached from the realities of therapeutic practice.

These criticisms haven’t just been levied at the EPPP, which tests therapists at the PsyD and PhD level, but at the variety of licensing exams on the market, including the Association of Social Work Boards’ (ASWB) exam for social workers, the National Board for Certified Counselors’ National Clinical Mental Health Counseling Examination and National Counselor Examination for counselors, and the Association of Marital and Family Therapy Regulatory Boards’ (AMFTRB) MFT exam for marriage and family therapists.

These tests have also come under fire for costs that critics have called prohibitive, especially since none of the associations offer financial assistance. If that wasn’t enough, some of these exams’ opponents have levied an even more serious allegation: bias and racism. In August, ASWB released an 88-page report, which revealed that between 2018 and 2021, 91 percent of white students had passed the exam, compared to 80 percent of Asian students, 77 percent of Hispanic students, and 57 percent of Black students.

“Variations in exam performance across different racial/ethnic groups are not unique to the ASWB examinations,” reads a summary at the end of the report, which goes on to mention similar gaps in teaching, nursing, and law exams. “Historically marginalized groups may be more likely to experience challenges in the period leading up to exam administration,” the summary continues, “including but not limited to access to comprehensive, accurate, and effective exam preparation resources; sufficient time or availability to prepare for taking an exam; and adequate financial resources to pay for the exam.”

Not everyone was convinced. Shortly after the report was released, a group of 12 therapists created a change.org petition, calling for ASWB’s exams to be discontinued, for state boards to immediately adopt an alternate licensing model, and for ASWB to make reparations using its $33 million in net assets. ASWB, the petition states, “continues to use an unvalidated exam with dramatic biases that prevent Black, Latin/Hispanic, and Indigenous social workers from becoming licensed.” The petition also claims that routine evaluations of the exam have disproportionately solicited feedback from white social workers, and that ASWB had previously refused to provide demographic data related to exam pass rates and even denied having it.

“We stress that ASWB has never shown that their tests are related to actual performance as a social worker,” the petition states. “We see these exams as harmful, racist gatekeeping tools.” Taking aim at the wider testing industry, the authors continued: “More broadly, we also call into question the use of exams at all, as standardized testing is known to be biased.”

This petition has more than 8,000 signatures to date.

All of these concerns—about a lack of test transparency, a shortage of study resources, high test costs, high prep costs, and racial bias—raise important questions: are these exams and their associated stressors placing unnecessary burdens on people who, without them, would still be perfectly capable therapists? And with these tests having been in place for decades, can anything change?

Gatekeeping, but for Whom?

Julia knows that the road to a licensing exam is a long one, even under the best of circumstances. Given their small intern and postdoc salaries, she and Christian had to borrow money from family to pay for the EPPP—which she acknowledges is a privilege.

The hurdles didn’t stop there. Christian says it was tough to find reputable study materials. That’s not surprising. Although all the licensing associations offer online practice tests—for a fee—none of them makes previous exams available to applicants, nor do they endorse books or other study materials. Some, like AMFTRB, explicitly disavow tutors and exam resources that claim to be associated with them. “The sole official examination preparation product that is offered by the AMFTRB,” reads an online disclaimer in a bright red box, “is a practice examination which is available for purchase on the AMFTRB’s website.”

For Christian and Julia, the few test-prep companies that looked promising were prohibitively expensive. Plus, some of their marketing felt like fear-baiting, Christian recalls, as if they were overpromising results and taking advantage of desperate students’ anxiety. They began to wonder how much of the exam and prep offerings were just money-making ventures.

The study process had an element of elitism, Julia adds, “as if you needed to have a certain educational background, or a certain vocabulary, or be a seasoned test-taker in order to score better.” People who don’t fit the mold, she suspects, are facing an uphill battle. “I knew some students taking the test who spoke English as a second language,” she explains. “They were having to translate phrases you’d only hear in American culture on top of focusing on the content. That’s a whole other barrier.”

In the end, Christian and Julia compiled their own study guide, mostly made up of hand-me-down study materials from students who’d taken the exam years before—a common strategy. But almost immediately, Julia says she noticed that many of the sample test questions sounded antiquated, as if they’d been pulled from the DSM-III and IV. “The language about gender, sexuality, and intellectual disabilities felt outdated and problematic,” she says.

When her time came to take the test, a few weeks before Christian, she’d run into another disconnect: “The vast majority of questions felt irrelevant to actual practice,” she recalls. “I was surprised there weren’t more questions focused on actual clinical information and actual approaches to therapy and theoretical orientations. These exams are for people becoming clinicians, not research professors.”

These concerns are no surprise to Eddy Ameen, who oversaw the Committee on Early Career Psychologists between 2011 and 2020, serving as a liaison between the committee, graduate students, and licensing associations. “A lot of the students I heard from were up in arms,” he says. “’Why do we need these tests? Why are they so expensive? And who are they going to leave behind?’ Others had a problem with them being a valid indicator of somebody’s competence. Students would say we were overly self-policing without any credible problem. Some described it as a test in search of a problem.”

Ameen understands the counterargument, though: the associations need a way to assess competency. They have a duty to protect the public, and people need some way of knowing that their therapist has been properly vetted. But he says these exams may have been created for another, less virtuous reason: to gain the same recognition and legitimacy as the field of medicine. “When these tests were created,” he says, “there was this mentality of, ‘Well they’re doing it, so we need to as well.’ For a long time, psychotherapy has been trying to play in the same sandbox as medicine, trying to stay competitive.”

Although not a practicing therapist, Ameen was trained as a psychologist. He’s also a researcher. Curious about whom these exams might be excluding, he began collecting data on EPPP pass rates, pulling from an APA survey of 2,109 students who’d graduated between 2005 and 2015. In 2018, he detailed his findings in APA’s quarterly news journal Communique. What he found was concerning: 92 percent of white psychologists surveyed reported passing the exam on the first try, compared to 83 percent of psychologists of color, with some racial groups as low as 67 percent. For Ameen, it’s an alarm bell. “We already have a diversity problem in the field,” he says. “This shows that there needs to be accountability. How do we make sure we don’t continue to homogenize the field?”

Researcher, professor, and clinical psychologist Brian Sharpless has spent much of his career, too, looking at whom licensing exams are excluding and what might be done about it. Back in 2007, shortly after taking the EPPP, he noticed that little information about pass rates was publicly available, even though the EPPP dates back to 1965. “It was puzzling,” he says. “When I dug into the research, I couldn’t find certain basic information.”

Sharpless was curious about something else: the same disconnect Julia had noticed between clinical training and what the test actually assessed. The EPPP is designed to be comprehensive, with eight domains, including biological, cognitive, and social and cultural bases of behavior, assessment and diagnosis, research methods, and ethical and legal issues. But a significant portion is devoted to niche areas like Industrial Organ­ization (IO) psychology, which focuses on the psychology of humans in the workplace.

Sharpless had never learned about this in graduate school, nor would he need to use it in clinical practice. Looking for answers about how many people were shut out by the exam, he spent the next 17 years researching the EPPP, at one point using a Freedom of Information Act request to gather data from state psychology boards. In 2019, after sorting through 25 years of EPPP scores, broken down by gender, ethnicity, and degree type, he found not only a discrepancy in pass rates along racial lines, with Black and Hispanic students failing 38 percent and 35 percent of the time, respectively, compared to 14 percent for white students, but that after more than half a century, the test structure hadn’t really changed much at all.

“The basic organization and scope of the test is more or less the same since the field was far less specialized,” he says. “The exam adheres to a generalist model, and from my perspective, that’s one of its weaknesses. I think it’s more important to specifically assess people on the skills and knowledge they’re going to use. Otherwise, it could just be measuring things like test-taking ability and memory.” Ideally, Sharpless says, a variety of exams would match the test-taker’s unique clinical focus, but he adds that this would not only be a logistical hurdle, but likely drive up costs.

That doesn’t mean a change isn’t overdue. “As is, it would be helpful to know why we need these exams and what they incrementally add to licensure,” he says. “At this point, we don’t know if it assesses anything beyond the vetting measures that are already available.”

A Glimpse Behind the Curtain

For Mariann Burnetti-Atwell, CEO of the ASPPB, the EPPP is a living document, even if it doesn’t seem like one. It’s more flexible than people give it credit for, she says, and undergoes regular annual assessments by trained psychologists across disciplines to ensure it’s up to date—what’s known at ASPPB as the Job Task Analysis.

“This is where the development process begins,” she explains. “Every year we have psychologists who work on committees develop questions based on their expertise as it relates to the domains. There are multiple levels that go into the writing of a question before it ever hits the official test.”

Burnetti-Atwell says she’s aware of the criticisms that parts of the exam, like those on IO, aren’t relevant to some therapists. But she says that time after time, the Job Task Analysis team’s assessment finds that it remains relevant to a sizeable number of mental health professionals, even if it’s not to most in private practice.

There’s less flexibility in other areas. When it comes to costs, for example, Burnetti-Atwell says they’re as low as they can go. “The development of an exam is expensive because it goes through multiple committees and processes of review,” she says. “But I think the association has really tried to balance the work that’s required to produce a reliable exam and take into consideration the cost of the exam for those who are taking it.”

Burnetti-Atwell demurs when asked about ASPPB stepping up its test-prep offerings. Outside of practice tests and a handbook that lays out basic information about the exam’s different domains, she says there are no plans to produce other materials or work with test-prep companies. Currently, she says, “ASPPB does not have an active relationship with them.”

So is there enough transparency around the exam? Is the association doing enough to put all students on a path to success? Burnetti-Atwell thinks so. She says she and other ASPPB officials try to meet regularly with students through their member organizations to hear their concerns and answer questions when they can. “We’re committed to serving our member jurisdictions and their work to ensure competent professionals are licensed to safely provide services to those with mental health treatment needs,” she explains. “But we also strive to be a positive resource for those working to become licensed mental health professionals.”

On that June day, as Christian sat in the testing center, feeling more and more perplexed with each passing question, he remembered some advice he’d received from colleagues who’d taken the exam: Half the time, it’s going to feel like you’re failing. Talk yourself through that feeling and try to be kind to yourself.

Four hours passed, and despite plenty of positive self-talk, Christian still wasn’t sure he’d passed. He clicked a button to submit his answers, pushed back his chair, and walked into the next room, where one of the test officials was tallying scores and printing results.

I don’t know how I’m going to pay for this again, but I’ll figure it out, he thought to himself. I’ll take a few days off, and then try again.

The official took Christian’s name, printed out several sheets of paper, and handed them to him from across the table. He’d need a 75 percent or above to pass.

It took him a moment to clear his head, but then he saw it: a 77 percent. He’d passed.

Christian put both hands on his knees and gasped, trying his best to control his elation. There were other people around him waiting for their scores. Some might’ve not been so lucky. So he turned down the hallway, tears running down his cheeks, went into the bathroom around the corner, and jumped up and down in one of the stalls.

Then, Christian remembered that by now, Julia would be waiting for him downstairs. He took the elevator down and walked outside. Sure enough, there she was, waiting across the street and holding a bouquet of flowers. Their eyes met. Hers widened, searching for an answer. And when Christian put both hands in the air triumphantly, they ran to each other, crying and hugging, surrounded by strangers in the middle of downtown Manhattan.

“There’s just so much emotion in this one test,” Christian says. “It’s a culmination of everything you’ve been through, getting into grad school, going through your program. It felt like I’d run a marathon and finally crossed the finish line.”

Christian and Julia have crossed that finish line. But they haven’t forgotten what it took to get there, how they struggled, and the many capable, aspiring therapists who are disproportionately affected by an imperfect testing system.

“What can we do for people who just want to get out there and help and make a living—but have to take this exam a second or third time?” Christian wonders. “This career is hard enough as it is.”

* Christian and Julia’s last names have been removed for privacy.

PHOTO © PEXELS/Monstera