It’s Important to Maintain Perspective

Just at the end of 2020, both USMLE and NBOME made announcements about Pass-Fail changes. USMLE reiterated their intention to make the change to Step 1 “no earlier than January 2022”. That phrase still leaves open the door that the date might be later.  

As I wrote last year, it is important to maintain perspective. Most students currently in their third year have already passed Step 1 or Level 1 and received a numeric score. MD students in their second year, need to prepare for a numeric score test just in case the date changes. DO students know precisely what kind of test they are taking unless their school policies change. 

NBOME gave a very specific date (May 1st, 2022) that the COMLEX-USA Level 1 exam will become Pass-Fail. More details are expected to be released by NBOME between now this July.

Where Does This Leave Students? 

The reason for the change was to keep the exams focused on their primary purposes, which is to qualify physicians to be licensed and, directly related to that, to protect the public. By removing the secondary purpose of interview competition it was hoped that students would focus on “becoming competent physicians” rather than “becoming high scorers”. This should reduce anxiety about the exam, but not necessarily the match process. 

As long as Program Directors are limited in how many interviews they can give, they will be looking for an easy measure to make decisions. As long as students believe competition for interviews is fierce, they will be over-applying to increase their chances of getting an interview. These two motivations are locked in a dysfunctional spiral with more and more applications and an objective measure that becomes higher and higher stakes. Match anxiety will remain and this alternate objective measure will be the new focus.

How Can We Reduce Student Anxiety and Increase Institutional Trust?

A reduction in anxiety for students is dependent on the trust of program directors and schools in each other. Program directors need to trust they can share the performance statistics of interviewed and accepted applicants without worrying that schools and other program directors might use this information against them. That will address the over-application problem as students direct their applications in a realistic way. Schools need to trust they can defend the objectivity of their assessments and MSPE’s in a way that is convincing to program directors and students. Both schools and programs face the other side of this issue when confronted with disgruntled students or residents who feel they were unfairly evaluated. This is not a trivial problem to solve. 

The Coalition for Physician Accountability was meant to explore how to improve the whole process but 2020 was spent solving more immediate and practical problems (like should students be allowed to do 4th year away electives during a pandemic?). They have gotten started on the original task and have put out a survey which I encourage any and all to respond to. 

Three Things to Consider for Students Taking Step 1 or Level 1 in 2021

It seems inevitable, at least for current 1st-year students, that Step 2 CK will become the new high anxiety measure to worry about. So schools have three things to worry about for their students who will be taking their Step 1 or Level 1 exams in 2021:

1.) How can we help students taking the USMLE exam in 2021 stay motivated to treat it like a numeric-scored exam until USMLE makes a definitive announcement? 

2.) Should schools consider changes to give students the best chance of performing as well as possible (e.g. dedicated study period) as early as possible (e.g. accelerate schedule) on Step 2 CK and Level 2 CE?

3.) Can schools find new objective measures that can be included on the MSPEs that will give program directors alternative data to make interview decisions as easy as possible?  

That last phrase is problematic because the ERAS application conveys a lot of information in an organized what but certain parts (particularly test scores) are much easier to see (and filter by) than others. Still, beyond scores, we know there are certain parts of the application that program directors look at most: scores, grades, MSPE, and LORs. 

What Are The Options for Moving Forward?

One option is that alternative standardized tests are available (e.g. NBME CBSSA and CCSSA). Schools have been called upon to certify their students’ clinical skills in the absence of the Step 2 CS and Level 2-PE. To the extent the details of these assessments can be shown to be objective, this would be a data point more useful than the licensure exams have been (which provide little detail beyond pass-fail). There are other exams available from third parties (including Kaplan) that might alleviate one of the complaints of program directors which is a suspicion of how objective schools really are about their own students. 

None of these approaches will reduce the anxiety associated with the match process. Substituting one for another just shifts the focus. However, providing multiple approaches gives program directors more data and allows students to demonstrate competency in other ways instead of worrying about a single assessment; hopefully, the total anxiety is split across the various measures and not multiplied. 

Another option is to look for additional attributes to measure and document in an objective way that enhances the students’ applications. Research, community engagement, and medical leadership opportunities are all things that could serve this purpose and additionally shift the focus on “becoming competent physicians” rather than “becoming high scorers”. 

Maybe this is the crisis that is going to be the opportunity to transform medical education?