What does the USMLE Test Change Mean to Current and Future Students?
by Christopher Cimino, MD, FACMI, VP of Medical Academics, Kaplan Medical | April 15, 2020
Is there still a test change?
In the space of one month the entire medical education world has shifted its attention to how to assist with health care delivery in this crisis, how to provide clinical experience and proctored assessments in a socially distant manner, and what the pandemic means to the current graduating class of medical students. The idea of USMLE Step 1 becoming Pass/Fail seems almost moot by comparison.
And yet there is no doubt that the change will occur, sometime on or after January 2022. The slightly less significant changes are also still on someone’s roadmap; reducing the permissible number of exam retakes to 4 and the requirement to pass Step 1 before sitting for Step 2 CS will occur on or after January and March 2021. How does this affect current students? How will this affect future students? Realistically when will these changes take effect?
How will this affect current students?
The initial buzz in student communities was mixed. Some saw this as a great relief and stress reduction. Others saw this as a disadvantage for applications to competitive residencies. In both camps were people discussing whether they could speed up or slow down their studies to take advantage of their preferred grading method. The reality is few students have the flexibility to shift their studying enough to really have a choice.
If we assume the changes occur by the suggested dates and we take into account that schools have fixed schedules and policy requirements, then the timing for current students is such that they will be taking a numerically graded exam.
Even for international medical graduates (IMGs) who theoretically can take the test whenever they want, their reality is their life dictates the schedule of how much time they have to study. Once they start their dedicated study, trying to speed things up risks a poor performance. Trying to stretch things out risks study fatigue which can also reduce their score because they are testing after their peak.
The timeline below shows that Class of 2023 students at the small number of schools that take Step 1 around the 30 month mark might have a choice but it will probably be dictated by their schools’ policies. The same will hold true for Class of 2024 students (not shown) at the small number of schools that take Step 1 at the 18 month mark.
IMGs who are planning on starting their studying between July and December of 2020 might be the only ones with real flexibility to choose, given that typical study times are 12 to 18 months.
So if most current students will be taking a numerically scored test, does that mean they don’t need to change their approach? Looking ahead, Program Directors (PDs) are going to be looking at other indicators for selecting applicants for interviews when USMLE Step 1 becomes Pass/Fail. In 2023, some PDs, especially at competitive programs, will be starting to look at new indicators even though that cohort of students will still have numeric scores (from 2020 and 2021). The main way this change affects current students is they still have to work to perform as well as they possibly can but they will also have to worry about the new ways of being judged as future students.
How will this affect future students?
There are two main ways this test change will affect future students (Class of 2025 or IMGs who start studying in 2021): how to study for a Pass/Fail test and how to prepare for residency applications.
The approach to a Pass/Fail test should be very similar to the approach to a numerically scored test, at least in the first year of the change. Medical educators agree that a solid basic science foundation is essential to gaining clinical knowledge expertise. While it might be possible to learn clinical knowledge as it is applied today with minimal basic science knowledge, it is almost impossible to adapt to changes in clinical practice without that basic science background. Immunology or pharmacology are fast changing fields and appreciating these changes requires being facile with the science. There is also a degree of scientific reasoning that becomes useful when presented with an atypical patient or an unexpected treatment reaction. Schools are working to focus on critical reasoning skills but it is a challenge to teach such skills out of context. Basic science provides a rich context for applying such skills. Overall, the number of hours a typical student is likely to need to acquire the necessary basic science knowledge is going to be about the same. One change that can be accommodated is a shift toward school specific approaches to the organization of knowledge. That will help with being ready to pass school exams and will serve well for a Pass/Fail exam.
There is also an uncertainty of when the test change will occur. It would be a shame for someone to take it easy and slack off on their studying only to discover that the change has been pushed back and so now they will be taking a numeric exam. It is certain to be easier to adapt a numeric study approach to a Pass/Fail exam than to adapt a Pass/Fail study approach to a numeric exam.
Let’s consider what a Pass/Fail study approach would look like. Knowing when one is ready to take the exam and having experience of past students who have taken the Pass/Fail exam will be very useful to future students in knowing if they can scale back their study efforts. If so, future students should be thinking about how to use this “extra time” to be better prepared for residency applications. PDs will be looking for a replacement to help them decide who to invite for interviews and the most likely candidate is USMLE Step 2 CK.
Dovetailing Step 1 studying with Step 2 CK studying may be doubly useful. The clinical relevance will help make the Step 1 material more memorable and getting a jump on the clinical material will be helpful in clinical rotations. There are other possibilities like research or volunteer experiences that will help you get strong letters of recommendation or broaden your application. But these are not likely to take the place of an objective measure of performance. Might there be some new, as yet unknown, measure of performance? It is certainly possible but it is likely to be either similar to Step 1 or a mix of Step 1 and Step 2 CK content. The thought introduces yet another element of uncertainty into everyone’s decision making.
Realistically when will these changes take place?
The uncertainties make planning difficult. Will there be a new test solely for PDs to help them with their decision? Will it be a direct replacement for Step 1? How will other organizations like AAMC, AACOM, ACGME, FSMB, NRMP, and many more weigh in? All of these organizations are members of the Coalition for Physician Accountability which has been charged with making recommendations on improving the residency application and match process. That report may help clarify what other changes will occur which in turn may influence USMLE regarding when the test change will occur.
And when exactly the change will occur remains the second biggest uncertainty. They have stated that these changes will occur “on or after” 2021 and 2022. They have left the door open to having the changes occur later. If they are going to move ahead with the 2021 changes then they need to provide confirmation of that with some lead time. A logical time would be about 6 months before, in June or July. The merger of Osteopathic and Allopathic residencies in the National Residency Match Program has indirectly tied the USMLE and COMLEX-USA exams. Having applicants with Pass/Fail USMLE scores and a numeric COMLEX-USA scores in the same Match cycle would be awkward. The Osteopathic examine organization, NBOME, is independent from NBME, but they have already said they would be making an announcement this July.
The biggest uncertainty is how COVID-19 will affect everything. We have already seen a 2 month delay in examinations. At some point it will become impossible to “catch up” if these delays extend much longer. I have no inside knowledge but I believe USMLE picked the January and March dates to fall as far away as possible from the heaviest testing times. If they need to push things back a few months then they likely will push them back a full year. And so the uncertainty will continue.
But that’s okay. Dealing with uncertainty is good practice for clinical medicine.
Dr. Cimino has earned a reputation internationally as an award-winning medical educator. He was the founding Assistant Dean for Educational Informatics at Albert Einstein College of Medicine and former Associate Dean for Student Affairs at New York Medical College. He is board certified in Neurology and Clinical Informatics. He served as a member of the NBME Step 1 Behavioral Science Committee and the NBME End of Life Care Task Force.