USMLE Step 1 change has unintended effects on residency selection - Healio

The quantifiable quality of the United States Medical Licensing Examination Step 1 people lends itself to secondary uses, including aesculapian acquisition appraisal and evaluating students for promotion and graduation.

Scores betwixt 200 and 210 person been recovered to correlate with passing the committee introspection successful galore specialties, however, nary correlation betwixt higher (United States Medical Licensing Examination) USMLE Step 1 scores, and show during residency has been found. Although the grounds supporting the inferior of the Step 1 exam to foretell occurrence successful residency grooming is scant, historically, a precocious Step 1 people has been a captious origin successful obtaining a residency interview, careless of aesculapian specialty.

A precocious published survey of applicants applying to orthopedic country showed that 21.2% of applicants were successful favour of the pass/fail change. Source: Arguello A, et al. J Am Acad Orthop Surg. 2022;doi:10.5435/JAAOS-D-21-00615.

Some judge aesculapian students’ well-being has been negatively affected by the value placed connected the Step 1 result, with astir 1 successful 4 students reporting mean to precocious anxiousness surrounding the exam. There is besides a content that the clip spent preparing for this introspection detracts from important aspects of students’ idiosyncratic and objective development, specified arsenic assemblage service, research, shadowing, workout and different activities, to trim accent and beforehand health.

Competition for residency positions tin beryllium high. Data from the 2018 National Resident Matching Program (NRMP) showed the fig of applicants per residency presumption transcend the disposable positions successful astir each specialties. The Step 1 scores are utilized to surface residency applicants to marque the reappraisal of thousands of applications much manageable. This signifier is seen successful each programs from those with historically precocious chances of matching, specified arsenic pediatrics (98.8% lucifer rate), to the much competitory specialties, specified arsenic orthopedic country (82.4% lucifer rate).

In February 2020, the National Board of Medical Examiners (NBME) and Federation of State Medical Boards (FSMB) announced that, starting successful 2022, the USMLE Step 1 people would alteration from a numeric people with associated percentiles to reporting lone pass/fail. A large crushed down this alteration was to lessen the “Step 1 climate” successful aesculapian school, wherever it was believed excessively overmuch absorption had been diverted from objective improvement to USMLE Step 1 preparation. Although those successful favour of the pass/fail alteration judge it volition person a affirmative effect connected stress/well-being, beforehand a holistic exertion review, and summation diversity, determination are besides arguments that switching to pass/fail volition person the other effects.

Stress connected applicants

Medical schoolhouse is simply a clip that students should not lone larn medicine and diligent care, but besides however to negociate accent effectively. High-stakes examinations are not overseas to aesculapian students arsenic the Medical College Aptitude Test (MCAT) is required for introduction into aesculapian school. Learning to woody with accent efficaciously should beryllium an integral portion of students’ acquisition and should not beryllium avoided. Students volition beryllium sick prepared for the aboriginal arsenic these stressors bash not extremity aft aesculapian school. It is good researched that dealing with stress, deadlines, balancing enactment and location beingness continues aft aesculapian school. With the USMLE Step 1 pass/fail change, aesculapian schools volition beryllium doing students a disservice by not preparing them for what lies ahead. Furthermore, accent volition beryllium placed connected the USMLE Step 2 numeric score, simply pushing this accent backmost 1 year. The concerns astir accent are simply postponed.

Holistic approach

Although a extremity of the grading alteration was to motivate holistic exertion reappraisal alternatively of relying connected a numeric people arsenic a filter, programs whitethorn beryllium challenged to measure the applications successful a holistic mode successful a constricted magnitude of time, resulting successful greater contention for residency interviews owed to an accrued fig of applications.

With the pass/fail change, the fig of applications per programme volition apt summation and effect successful adjacent much contention for an interrogation position. It should beryllium noted, that for the 2021 cycle, astir each specialty had an summation successful applications per applicant. In the 2022 exertion cycle, determination were 8,389 students who applied to interior medicine with an mean of 35 applications per pupil (an summation from 2018, 2019 and 2020). Orthopedic country had the highest fig of applications per applicant astatine 97 compared with 86 applications per applicant successful 2020 (total applicants 1,283).

There could beryllium tens of thousands of applications requiring a reappraisal for 2 to 30 spots depending connected the specialty.

Recommendations including ceremonial grooming to code implicit bias, quantification of criteria specified arsenic objective skills, professionalism, enactment skills, assemblage work and advocacy whitethorn marque the residency enactment process much holistic. These recommendations bash not code however to sift done the accrued fig of applications successful a finite period, nor bash these code the information that the people tin beryllium an businesslike screening instrumentality for debased outliers astatine hazard for not passing their specialty boards.

The pass/fail alteration volition effect successful the nonaccomplishment of an nonsubjective information constituent to comparison applicants arsenic which whitethorn pb programs to measurement different variables erstwhile deciding connected granting an interview. Factors, specified arsenic tier of aesculapian school, USMLE Step 2, tier of undergraduate education, MCAT and telephone calls made by programs, mentors, and friends of applicants successful the specialty whitethorn beryllium weighted higher. Other than the USMLE Step 2 score, nary of these variables person been shown to correlate with show during residency and immoderate would reason are fraught with inequities and biases that tally antagonistic to gains successful diverseness and inclusion.

Opinions of aesculapian students and residents regarding USMLE Step 1 numerical vs. pass/fail grading has been a treatment for astir a decade. A 2011 survey surveying aesculapian students and residents recovered that lone 26% of respondents were successful favour of a pass/fail grading system, citing that the numerical people is advantageous for nonmigratory selection. A precocious published survey of applicants applying to orthopedic country showed lone 21.2% of applicants were successful favour of the pass/fail change. This disparity brings up an important question – Why are applicants, particularly those funny successful competitory specialties, mostly not successful favour of transitioning to a grading strategy that is intended to alteration the load of accent related to the residency enactment process, let for much clip with family, wellness, probe and shadowing? Most orthopedic applicants surveyed judge the consequences of the pass/fail alteration volition person the other effect.

USMLE Step 2

Most programme directors (80.7%) from a New England Journal of Medicine survey study that USMLE Step 2 Clinical Knowledge scores volition go much important successful nonmigratory selection. Although the USMLE Step 2 exam is thought beryllium much indicative of show during residency, the introspection is administered astatine antithetic times during the world year, depending connected the institution. Some students whitethorn consciousness pressured to instrumentality the introspection anterior to completing their halfway clerkships and yet volition beryllium tested successful those specialties they person not yet been formally taught. For different students, the trial whitethorn not beryllium administered earlier residency applications are due, truthful these students volition not person scores to taxable for their residency application. The displacement of absorption to the USMLE Step 2 volition effect successful investigating accent to beryllium pushed backmost a twelvemonth and much importantly volition not beryllium equitable to each students based connected trial timing arsenic compared with the USMLE Step 1.

Improved diversity?

Although immoderate contend that the pass/fail alteration volition beryllium beneficial for historically underrepresented aesculapian students, particularly those applying to much competitory specialties that are mostly the slightest divers successful medicine, the pass/fail alteration whitethorn person the other effect.

Furthermore, with the pass/fail alteration comparing students from schools without grades/Alpha Omega Alpha (AOA) status/class rank, arsenic seen with galore of the upper-tier aesculapian schools, to students from middle- oregon lower-tier aesculapian schools wherever determination are people rank/clerkship grades/AOA whitethorn beryllium adjacent much challenging. There are nary nonsubjective information points, lone communicative evaluations which whitethorn beryllium fraught with bias.

The USMLE Step 1 numeric people tin service to “level the playing field” for applicants, arsenic it tin comparison students careless of wherever they attended aesculapian schoolhouse oregon college. How tin students beryllium compared arsenic if 1 attends a aesculapian schoolhouse with grades and the different pupil attends a aesculapian schoolhouse without grades? Who volition person the advantage?

The USMLE numeric people helped to springiness the middle- and lower-tiered students a accidental to vie with those from upper-tiered aesculapian schools. The students who volition endure with this alteration are the historically underrepresented aesculapian students, planetary aesculapian graduates, osteopathic students, and students of middle- and lower-tiered aesculapian schools.

It is our fearfulness that residency reappraisal committees whitethorn place immoderate applicants who would person different stood retired owed to their Step 1 scores and whitethorn present beryllium bypassed. The extremity of the USMLE Step 1 pass/fail alteration to summation diverseness volition autumn short.

Opportunities for improvement

Although well-intentioned, the pass/fail alteration volition not make a much equitable process and volition person unintended consequences. The pursuing are immoderate options to code the concerns raised by the proponents of the USMLE Step 1 pass/fail change:

Medical schools tin instrumentality an progressive relation and see curricula/strategies to assistance springiness students the indispensable skills to woody with stressors some during and aft aesculapian schoolhouse which whitethorn assistance with the excessive accent students are presently feeling.

Although a holistic attack to residency enactment is ideal, it is intolerable if nonsubjective information points are removed arsenic the fig of applicants volition lone summation making a holistic attack impossible. To holistically reappraisal thousands of applications is impractical; the fig of applications indispensable beryllium decreased to a manageable number. As determination are nary limits to the fig of programs students tin use to, programs volition crook to different subjective information to alteration the fig of applications, galore of which are fraught with inequities. Programs, however, are starting to person supplemental essays oregon pre-interview video questions, oregon immoderate specialties person begun utilizing “signaling” a finite fig of programs to alteration the fig of applications and let for much holistic review. The strategy feeds upward and addressing the inequities astatine the introduction constituent whitethorn assistance the diverseness successful aesculapian schoolhouse and residencies. Making the USMLE Step 1 trial pass/fail should not beryllium seen arsenic the cure to a occupation that starts earlier.

Medical schools should besides hold to uncover applicants’ Shelf introspection scores and hold to person people ranks to comparison applicants reasonably arsenic it is hard to comparison applicants from schools with nary grades to schools with grades.

Making the trial pass/fail volition not amended accent levels, volition not pb to a holistic reappraisal and volition not let for just comparisons of applicants from antithetic backgrounds and aesculapian schools. The hopes are that aesculapian students volition postgraduate aesculapian schoolhouse prepared for their aboriginal and the quality to woody with the stressors of beingness arsenic a physician, for programs to beryllium capable to supply a holistic reappraisal of applicants, for applicants from each programs to beryllium compared fairly, and for the diverseness that is lacking successful each specialties of medicine to improve; however, making the USMLE Step 1 pass/fail is not the reply and volition lone exacerbate these issues..

References:

Al Khalili K, et al. World Neurosurg. 2014;doi:10.1016/j.wneu.2013.08.019.

Arguello A, et al. J Am Acad Orthop Surg. 2022;doi:10.5435/JAAOS-D-21-00615.

Armstrong A, et al. Mil Med. 2007;doi:10.7205/milmed.172.6.640.

Bernstein AD, et al. J Bone Joint Surg Am. 2002;doi:10.2106/00004623-200211000-00026.

Chen DR, et al. Acad Med. 2019;doi:10.1097/ACM.0000000000002565.

Dillon GF, et al. J Grad Med Educ. 2013;doi:10.4300/JGME-D-12-00205.1.

Green M, et al. Acad Med. 2009; doi:10.1097/ACM.0b013e3181970c6b.

Green M, et al. Clin Teach. 2016;doi:10.1111/tct.12386.

Hoffman KI. Acad Med. 1993;doi:10.1097/00001888-199310000-00003.

Jones MD, Jr., et al. BMC Med Educ. 2018;doi:10.1186/s12909-018-1138-x.

Kenny S, et al. Med Educ. 2013;doi:10.1111/medu.12234.

Lee M, et al. Med Teach. 2018;doi:10.1080/0142159X.2018.1430353.

Lewis CE, et al. J Grad Med Educ. 2011;doi:10.4300/JGME-D-10-00121.1.

Makhoul AT, et al. N Engl J Med. 2020;doi:10.1056/NEJMp2006148.

McCaskill QE, et al. Ambul Pediatr. 2007;doi:10.1016/j.ambp.2007.01.002.

McGaghie WC, et al. Acad Med. 2011;doi:10.1097/ACM.0b013e3181ffacdb.

Prober CG, et al. Acad Med. 2016;doi:10.1097/ACM.0000000000000855.

Prober CG. N Engl J Med. 2020;doi:10.1056/NEJMp2003880.

Program NRM. National Resident Matching Program, Results and Data: 2020 Main Residency Match. Washington DC. 2020:128.

Scoring I-ICoU. Change to pass/fail people reporting for Step 12020.

USMLE Step 1 modulation to pass/fail lone people reporting. USMLE. Available at: https://www.usmle.org/usmle-step-1-transition-passfail-only-score-reporting.

Verret CI, et al. Clin Orthop Relat Res. 2021;doi:10.1097/CORR.0000000000001457

Weissbart SJ, et al. Urology. 2015;doi:10.1016/j.urology.2014.12.041.

Williams RG. Acad Med. 1993;doi:10.1097/00001888-199310000-00004.