Ready (or Not)—Here They Come
===============================

* Kate DuChene Hanrahan

*   Career Choice
*   Comprehensive Health Care
*   COVID-19
*   Family Medicine
*   Family Physicians
*   Pandemics
*   Residency
*   Scope of Practice

The COVID-19 pandemic impacted all aspects of our lives worldwide, including our medical education system. The impact this disruption had on graduate medical education and graduates’ future practices is still to be determined. The policy brief by Sonoda in this issue of the journal analyzed results from the American Board of Family Medicine (ABFM) Family Medicine Certification Examination Registration Questionnaire regarding family medicine residency graduates intended scope of practice.1,2 Data collected from 2017 to 2021 graduates show significant changes (both increased and decreased) of graduates intended scope of practice, however visual trajectory of these changes remained unchanged in the 2021 cohort. Intended future scope of practice is one outcome measure that we can monitor to determine the impact the pandemic had on our learners. Other outcome measures we can monitor include performance on the ABFM In-Training Examination (ITE), pass rates for first-time test takers of the Family Medicine Certification Examination, Accreditation Council for Graduate Medical Education (ACGME) milestone rankings of residents and medical student matriculation into family medicine residency programs.

The ACGME develops program requirements specific to each specialty. Family medicine residency training has traditionally been 36 months. The WHO declared COVID-19 a pandemic on March 11, 2020, with the first 3 surges impacting communities and health care systems for nearly 9 months, through December 2020.3 This represents 25% of the family medicine resident training time being directly impacted by the pandemic and various surges. The current family medicine program requirements have minimum required experiences that residents must have including 6 months of inpatient adult medicine, 2 months in the emergency department, 2 months of ambulatory pediatric medicine and 3 months of elective time.4 The program requirements are prescriptive and have an intentional goal to ensure that family physicians are trained broadly to provide care to a diverse patient population in the communities they will serve. At the completion of residency training, it is the Program Director’s (PD) sole responsibility to verify the resident’s completion of all ACGME requirements to the ABFM for graduates to sit for the ABFM Certification Examination and become a board-certified family physician. There is a lot of training for a residency program to provide in 36 months to ensure that resident physicians are competent to practice family medicine independently before being recommended for the Certification Examination. The ACGME program requirements also include having a Clinical Competency Committee (CCC) made of at least 3 faculty who regularly reviews resident evaluations and advises the PD on each resident’s progress semiannualy.4 The final CCC review and the PD verification to the ABFM, are the final times that a family physician will be reviewed by a group, for as long as they maintain their continuous certification through the ABFM.

With disruption of our health care systems, the WHO developed guidance on approaching the pandemic which included identifying essential versus nonessential health care services and redistributing the health care workforce through reassignment and task sharing.5 Awadallah and colleagues examined the impact the pandemic had on family medicine faculty and residents’ clinical responsibilities, training, and well-being. They found that more than 80% of faculty and residents had decreased clinical activity during the first phase of the pandemic and almost all implemented or expanded their clinical telehealth services.6 The implementation of telehealth services may result in less exposure to a wide variety of clinical medical scenarios.6 Interviews and surveys with family medicine residents identified that their patient panels became less diverse as those using telemedicine tended to be tech savvy, “white, middle class individuals,” and that they struggled with broader patient panel management.7 Some residents felt like they forfeited education experiences while others appreciated the opportunities for online curriculum that could be done on their own time and schedule.8

A Council of Academic Family Medicine (CAFM) Educational Research Alliance (CERA) Program Director survey found that residency curriculum was most disrupted in geriatrics, musculoskeletal conditions, pediatrics, surgery, as well as the family medicine model clinic; ICU training had no disruption.9 However, the same survey results found that majority of PDs did not have concerns about ACGME accreditation regarding their curriculum which could imply that PDs felt confident in their program’s ability to adapt educational training during the pandemic. A systematic review found that innovative solutions to teaching during the pandemic included development and sharing of online learning modules, tele-conferences and webinars, and use of social media which provided timely education material for clinicians to stay current.10 This same review found that service was prioritized over education, as clinical teams tended to be smaller, with some residents staying at home to ensure a viable back up system and that training time increased in needed areas of emergency medicine and ICU care.10

The COVID-19 pandemic impacted the education and curriculum experiences for family medicine residents graduating in 2020 and 2021, but what outcome measures do we have on this impact? Medical knowledge and clinical decision making are essential for a family physician. The ABFM offers all residents the opportunity to take the ITE annually, which can provide residents information about their trajectory in acquiring the necessary knowledge needed to be a family physician. The ITE content is similar to the ABFM Family Medicine Certification Examination and ITE scores can be used to estimate the probability of passing the certification examination.11 The ITE is scored using a scaled score which allows one to compare performance from year to year, even though the tests are different. Since 2019, there has been a decline in ITE scores by 30 to 40 points for residents in each level of training.12 The reasons for this decline in ITE scores is unknown, but certainly multifactorial and not only due to the pandemic. During the same time, there has been no change in the pass rate for the Family Medicine Certification Examination with first time test takers which has remained near 99%.13

Over the past 2 decades, the graduate medical education system has evolved to be competency based. ACGME milestones were crafted to provide a framework for assessment of residents in developing the 6 key dimensions of physician competency: patient care, medical knowledge, systems based practice, practice based learning and improvement, professionalism, and interpersonal and communication skills.14 Each core competency is divided into subcompetencies that have 5 levels of milestones with Level 1 being novice and Level 5 being expert. Level 4 has been determined to be a graduation goal but not a requirement.14 Every 6 months, the CCC reviews completed evaluations and observations of residents and selects a milestone level for each subcompetency which is reported to the PD who then reports this data to the ACGME. The ACGME publishes a National Milestone Report each year summarizing the milestones reported including the mean and median of milestone ratings, as well as the variance across resident classes for each specialty. Review of the family medicine milestones reported from 2018 to 2021 for year-end PGY3 graduates shows show similar ratings with most subcompetencies being marked a Level 4.15⇓–17 It seems that the pandemic had minimal impact on milestone ratings of graduating residents.

The pandemic impacted the physician workforce with approximately half of physicians reporting that they are considering an employment change due to COVID-19 related overwork.18 A review of matriculation and graduation rates during the pandemic would be one way to evaluate the student and resident workforce in family medicine. The 2022 Match was the lowest fill rate (90.6%) for family medicine since 2007 (76.2%); with the record high fill rate of 96.7% occurring in 2018.19 In 2022, the fill rate for US MD seniors (31.5%) was the lowest in history compared with the highest fill rate for US DO seniors (30.3%).19 There are many potential reasons for the increased number of unfilled positions in family medicine recently including the pandemic, consolidation of the MD and DO match in 2020 and no increased percentage of US MD graduates pursuing careers in family medicine.19 Residency program attrition rates and number of resident graduates are harder to track and not readily available to assess how these could have been impacted by the pandemic.

There is no doubt that our resident’s curriculum and learning experiences were altered due to the pandemic, with at least 9 months directly impacted by surges from COVID-19. The impact that this had on their knowledge acquisition, clinical reasoning and procedural skills is still unknown. In their policy brief, Sonoda states that the trajectories of intended scope of future practice of our resident graduates remained unchanged in 2021.2 In review of other outcome metrics, there seems to be no change in milestone ranking or board certification rates for first time test-takers, despite the dramatic decline in resident performance on the ITE.

What should we do with this information? The new ACGME Family Medicine curriculum requirements effective for July 2023 allow for more flexibility in curriculum including 6 months of required elective time.20 We must study the lessons learned about the best ways to teach and for our residents to learn as we adapt to creative curriculum solutions. We must be leaders in reporting outcomes for other learning models such as remote/online learning compared with in person patient care learning. We must also remember our core values as a discipline of continuous, compassionate, comprehensive, and personal care in which learning at the bedside of a patient cannot be replaced. The patient is central to the teaching process and should not be replaced by online or remote learning modules as stated by Sir William Osler in 1903,

> “In what may be called the natural method of teaching, the student begins with the patient, continues with the patient, and ends his study with the patient, using books and lectures as tools, as means to an end. For the junior student … it is a safe rule to have no teaching without a patient for the text, and the best teaching is that taught by the patient himself.” 21

## Notes

*   *Funding:* None.

*   *Conflict of interest:* None.

*   To see this article online, please go to: [http://jabfm.org/content/35/6/1187.full](http://jabfm.org/content/35/6/1187.full).

*   See Related Article on Page [1191](http://www.jabfm.org/lookup/volpage/35/1191).

## References

1.  1.Peterson LE, Fang B, Phillips RL Jr.., Avant R, Puffer JC. The American Board of Family Medicine's data collection method for tracking their specialty. J Am Board Fam Med 2019;32:89–95.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiamFiZnAiO3M6NToicmVzaWQiO3M6NzoiMzIvMS84OSI7czo0OiJhdG9tIjtzOjIxOiIvamFiZnAvMzUvNi8xMTg3LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

2.  2.Sonoda K. Pandemic disruption in residency did not alter trends in intended scope of practice. J Am Board Fam Med 2022;35:1191–1193.
    
    

3.  3.CDC Museum COVID-19 timeline. October 6, 2022. Available from: [https://www.cdc.gov/museum/timeline/covid19.html](https://www.cdc.gov/museum/timeline/covid19.html).
    
    

4.  4.ACGME Program requirements for graduate medical education in family medicine. 2022:1–63. Accessed October 5, 2022. Available from: [https://www.acgme.org/globalassets/pfassets/programrequirements/120\_familymedicine\_2022.pdf](https://www.acgme.org/globalassets/pfassets/programrequirements/120_familymedicine_2022.pdf).
    
    

5.  5.COVID-19: Operational guidance for maintaining essential health services during an outbreak: Interim guidance. 2020. 25 March. Available from: [https://apps.who.int/iris/bitstream/handle/10665/331561/WHO-2019-nCoV-essential\_health\_services-2020.1-eng.pdf?sequence=1](https://apps.who.int/iris/bitstream/handle/10665/331561/WHO-2019-nCoV-essential_health_services-2020.1-eng.pdf?sequence=1).
    
    

6.  6.Awadallah NS, Czaja AS, Fainstad T, et al. The impact of the COVID-19 pandemic on family medicine residency training. Fam Pract 2021;38:i9–i15. Aug 27.
    
    

7.  7.Staloff J, Jabbarpour Y. Reflections from family medicine residents on training during the COVID-19 Pandemic. Fam Med 2022;54:694–9.
    
    

8.  8.Messner E, Parascando JA, La Gamma C, Bone C, Clebak KT, Riley TD. Effects of early COVID-19 restrictions on resident well-being and burnout. Fam Med 2022;54:708–12.
    
    

9.  9.Fashner J, Espinoza A, Mainous IA. COVID-19 disruption to family medicine residency curriculum: results from a 2020 US programme directors survey. Fam Med Community Health 2021;9.
    
    

10. 10.Dedeilia A, Sotiropoulos MG, Hanrahan JG, Janga D, Dedeilias P, Sideris M. Medical and surgical education challenges and innovations in the COVID-19 era: a systematic review. In Vivo 2020;34:1603–11.
    
    [Abstract/FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NjoiaW52aXZvIjtzOjU6InJlc2lkIjtzOjE1OiIzNC8zX3N1cHBsLzE2MDMiO3M6NDoiYXRvbSI7czoyMToiL2phYmZwLzM1LzYvMTE4Ny5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

11. 11.American Board of Family Medicine In-Training Examination. Accessed October 5, 2022. Available from: [https://www.theabfm.org/become-certified/acgme-program/in-training-examination](https://www.theabfm.org/become-certified/acgme-program/in-training-examination).
    
    

12. 12.Newton WP, O'Neill TR, Wang T. High-stakes knowledge assessment at ABFM: what we have learned and how it is useful. Ann Fam Med 2022;20:186–8.
    
    [FREE Full Text](http://www.jabfm.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6ODoiMjAvMi8xODYiO3M6NDoiYXRvbSI7czoyMToiL2phYmZwLzM1LzYvMTE4Ny5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

13. 13.One Day Family Medicine Certifcation Examination Information Pass Rates. October 5, 2022. Accessed October 5, 2022, 2022. Available from: [https://www.theabfm.org/continue-certification/cognitive-expertise/one-day-fmc-exam/exam-info](https://www.theabfm.org/continue-certification/cognitive-expertise/one-day-fmc-exam/exam-info).
    
    

14. 14.Family Medicine Milestones. 2019:1–26. Accessed October 6, 2022. Available from: [https://www.acgme.org/globalassets/pdfs/milestones/familymedicinemilestones.pdf](https://www.acgme.org/globalassets/pdfs/milestones/familymedicinemilestones.pdf).
    
    

15. 15.1.  Yamazaki K, 
    2.  Shah H
    
    Hamstra SJ, Yamazaki K, Shah HB, et al. Milestones National Report 2019. 2019:1–722. Available from: [https://www.acgme.org/globalassets/pdfs/milestones/2019milestonesnationalreportfinal.pdf](https://www.acgme.org/globalassets/pdfs/milestones/2019milestonesnationalreportfinal.pdf).
    
    

16. 16.1.  Hamstra SJ, 
    2.  Hogan S
    
    Edgar L, Hamstra SJ, Hogan S, et al. Milestones National Report 2020. 2020:1–683. Available from: [https://www.acgme.org/globalassets/pdfs/milestones/2020-milestones-national-report-final.pdf](https://www.acgme.org/globalassets/pdfs/milestones/2020-milestones-national-report-final.pdf).
    
    

17. 17.1.  Hogan S, 
    2.  Holmboe ES
    
    Edgar L, Hogan S, Holmboe ES, et al. Milestones Report 2021. 2021:1–582. Available from: [https://www.acgme.org/globalassets/pdfs/milestones/milestonesnationalreportfinal093021.pdf](https://www.acgme.org/globalassets/pdfs/milestones/milestonesnationalreportfinal093021.pdf).
    
    

18. 18.2021 Physician Compensation Report. Fifth Annual Study.1-20. December 2021. Accessed October 6, 2022. Available from: [https://c8y.doxcdn.com/image/upload/v1/Press%20Blog/Research%20Reports/Doximity-Compensation-Report-2021.pdf](https://c8y.doxcdn.com/image/upload/v1/Press%20Blog/Research%20Reports/Doximity-Compensation-Report-2021.pdf).
    
    

19. 19.AAFP. 2022 match results for family medicine. 2022:1–10. Available from: [https://www.aafp.org/dam/AAFP/documents/medical\_education\_residency/the\_match/AAFP-2022-Match-Results-for-Family-Medicine.pdf](https://www.aafp.org/dam/AAFP/documents/medical_education_residency/the_match/AAFP-2022-Match-Results-for-Family-Medicine.pdf).
    
    

20. 20.ACGME Program Requirements for Graduate Medical Education in Family Medicine Revision. 2022:1–64. Available from: [https://www.acgme.org/globalassets/pfassets/programrequirements/120\_familymedicine\_2023.pdf](https://www.acgme.org/globalassets/pfassets/programrequirements/120_familymedicine_2023.pdf).
    
    

21. 21.Osler SW. *Aequanimitas; with other addresses*. Blakiston; 1932.