Abstract
Background: Residency program requirements in Family Medicine have long required scholarship of both faculty and residents as part of instilling a culture of discovery and inquiry, but the impact of the requirements on faculty and residency scholarship is unclear.
Methods: We gathered information on family medicine faculty scholarly activity between the years 2016 to 2021 from data routinely collected by the ACGME, including faculty presentations, faculty publications, and program citations for scholarship. We compared these data with dates corresponding to the development of ACGME requirements for scholarly activity to search for possible correlations.
Results: Peer reviewed publications, other publications and presentations increased substantially over the time studied, and this increase seems to be out of proportion to the growth in the number of programs and faculty and occurred at the same time as shifts in residency requirements. PubMed articles increased from 505 to 3617; conference presentations increased from 4673 to 13842; and the ratio of PMID publications per faculty has increased from 0.03 to 0.21 between 2016 and 2021.
Discussion: The shift of scholarship requirements from a “detail” to a “core” requirement, along with increasing specification of expectations may have contributed to growth in scholarly activity in family medicine residency programs. Strategy for building research capacity in the specialty should include attention to the content of residency training.
Introduction
Scholarship in Family Medicine is at a crossroads, with the challenge to craft a national strategy to increase research capacity. The natural focus of any research strategy will lean heavily on academic institutions, and Family Medicine residency programs will have a key impact in the efforts of unfolding such a strategy and in training future family physicians in key research skills. A key question is to what extent can and should residency requirements for scholarship contribute to that effort?
Residency programs are undergoing a generational shift toward competency-based education with the release of new program requirements, including the need to model communication skills, lifelong learning, and self-reflection.1 These skills form the bedrock of scholarship. The accreditation requirements regarding scholarship which target the development of research skills will, to a large degree, direct the formation of future Family Medicine researchers and scholars. Residency faculty scholarship is required of all programs and can serve as a major influence in the formative process of an environment of inquiry. There is evidence of imprinting in residency with respect to quality of care and cost effectiveness—that is that experience in residency has an effect for at least 10 to 20 years.2 Accreditation requirements can serve as a tool in the imprinting of critical appraisal and research skills for residents as they embark on their careers. This article will explore the role of accreditation requirements in Family Medicine residency and the influence they play in the productive demonstration of scholarship of Family Medicine as a discipline.
Background
The Accreditation Council for Graduate Medical Education (ACGME) and the Review Committee for Family Medicine (RCFM) have published program requirements (PRs) for scholarship for decades., These have changed over the years to be more specific about scholarship expectations and the forms of scholarship that are acceptable. (see Table 1) The educational purpose of these requirements has been clear and consistent over time: in family medicine, as with other specialties, scholarship is a critical foundational component of the learning environment, helping foster a culture of inquiry and promoting master adaptive learning. However, minimum requirements intended to be the floor can become the ceiling for many programs who aspire to avoid citations but not necessarily advance scholarship.
Summary of FM Scholarly Activity Program Requirements from 2001–2023: (Major Changes from Prior Requirements Highlighted by Authors)
Before 2016 the guiding principle in the PRs for scholarly activity the last 20+ years has been that the responsibility for establishing and maintaining an environment of inquiry and scholarship rests with the faculty, and an active research component must be included. It is important to differentiate between scholarship and research. To clarify what was meant by scholarship the PRs listed all forms of Boyer’s definition of scholarship (discovery, integration, application, teaching) as options to fulfill the requirements. However, programs were also required to provide opportunity for residents to participate in research or other scholarly activities. In the early 2000s, there was an expectation that residents were expected to analyze the quality of care in their practice. This was formalized in a requirement for QI projects in the family medicine requirements in 2011. It was to be complemented by instruction in the critical evaluation of medical literature, including assessing study validity and the applicability of studies to the residents’ patients. The participation of each resident in an active research program was to be encouraged as preparation for a lifetime of self-education after the completion of formal training, but research by residents was not required.3
Between 2011 and 2016 the FM PR’s only required evidence of faculty scholarship in a single domain, (see Table 2) and these requirements differed from other specialties.4 In 2013 with the adoption of the Common Program Requirements (CPRs) as part of the Next Accreditation System all PRs had to be classified as core or detail requirements.5 In a formal sense, core requirements are citable and must be met by all programs, while detail requirements can be cited in certain circumstances, programs on continued accreditation in good standing are free to innovate around these requirements. The intent was to allow flexibility for residencies in how they fulfill scholarship requirement.
ACGME Web ADS Scholarship Domains
After 2016, family medicine residency scholarship became a core requirement and specificity increased to include a minimum of 3 domains of scholarship by faculty in residencies, including peer-reviewed publications. This was reinforced in 2019 by the RCFM, with the intention of promoting the skills needed to maintain a culture of scholarship throughout the FM GME community.6 In its recent major revision, the RCFM maintained its expectations for scholarship as one tool to foster the development of master adaptive learners. Current program requirements for faculty state: Among their scholarly activity, programs must demonstrate accomplishments in at least 3 of the following domains (Table 2).
The background and intent of the 2023 requirements state: “For the purposes of education, metrics of scholarly activity represent one of the surrogates for the program’s effectiveness in the creation of an environment of inquiry that advances the residents’ scholarly approach to patient care. The Review Committee evaluates the dissemination of scholarship for the program as a whole, not for individual faculty members, for a 5-year interval, for both core and noncore faculty members, with the goal of assessing the effectiveness of the creation of such an environment. The ACGME recognizes that there may be differences in scholarship requirements between different specialties and between residencies and fellowships in the same specialty.”7 Given this underlying goal, residency research requirements should not only require the generation of faculty scholarship, but also help motivate ongoing development of a culture of scholarship.
Methods
The ACGME Web-based Accreditation Data System (Web ADS) is a central accreditation data collection system that all programs must update annually with data on the composition and roles of faculty and faculty scholarly activity. To better understand the impact of PRs and scholarship, the authors reviewed a summary of the quantity of scholarly activity by programs for current Family Medicine faculty as reported for the last 5 academic years from 2016 to 2021 in Web ADS. While all forms of scholarship were reviewed, we have chosen to focus on the 3 largest domains (peer reviewed publications, conference presentations and nonpeer reviewed/other publications). The ratios of each of these domains to the number of faculty and programs was then calculated. (Table 3) It should be noted that in 2018 the ACGME first asked for “other/nonpeer reviewed presentations” to be listed to allow for non-PMID publications to be counted as a separate category. The authors also analyzed the program requirements for scholarship in family medicine for the corresponding years. (Table 1) The authors also reviewed data on program citations related to scholarship scholarly activity citation data for the 10 years before the pandemic. Data on program citations related to scholarship was based on information from ACGME collected and summarized by RC-FM leadership for the Starfield Summit IV in preparation for the Shaping GME: Future of Family Medicine major program requirement revision process.8
Summary of Faculty Scholarship in FM Residencies by Academic Year
Results
The trends in scholarship for FM show that, despite expansion of the number of programs and to a lesser extent the number of faculty, all forms of scholarship increased more than expected based on calculated ratios. (Table 3 and Figure 1) Nonpeer reviewed, and other publications outnumber peer-reviewed publications, suggesting that publishing avenues that are not indexed on the NLM (such as the Family Practice Inquiries Network or FPIN) are utilized often by residency faculty. Conference and other presentations far exceed all other forms of scholarship, and grant leadership has slowly grown.
Select scholarship trends in FM residencies 2016 to 2021.
There have been relatively few (<50 per year for over 10 yrs. with 450 or more programs) program citations regarding scholarship until the pandemic. (Figure 2) Citation data suggest that residency reviewers interpreted the definition of scholarship loosely. Peer-reviewed original research was not the primary expectation, nor was having the majority of core faculty participating in scholarship. As long as some faculty participated in 1 of the Boyer’s domains (see Table 1, PRs for 2006), the program likely did not receive a citation. Data on program citations and Areas for Improvement (AFI’s) for 2020 to 2022 is still embargoed by the ACGME – the information presented here is based on public presentations one of the authors (GH) gave in his role as former chair of the RCFM. The number of programs that were flagged for inadequate scholarship and subsequently given a citation or AFI dramatically increased from 2020 to 2022. This likely reflects the lack of opportunity for conference and other presentations during the pandemic, which had leveled off after a rapid increase in the previous 3 years.
RCFM program citations for scholarship by academic year.
Discussion
Our data suggest that increasing rigor and specificity of ACGME standards for scholarship had a positive impact on the quantity of scholarship produced by family medicine faculty, even after correcting for the growth of family medicine residencies and faculty. Conference presentations and nonpeer reviewed scholarship grew the most, but peer reviewed articles and grants also grew substantially. Not unexpectedly, for the years we have data, the number of ACGME citations grew, but it seems clear that the ACGME RC’s approach was not rigid. Citations for scholarship remained relatively low until the pandemic, when those markedly increased, most likely due to lack of opportunity to travel to present at conferences and also due to prioritizing acute patient care needs and practice redesign over scholarship. The data do not, however, allow assessment of which aspects of the changes in the requirements were associated with the growth of scholarship. It is likely the most important shift was the change of the requirement to “core” and hence citable for all programs, perhaps in combination with broader changes in the specialty. Other important factors may be the 2019 additions that faculty must disseminate scholarly accomplishments outside of the program and the requirement to include 3 different kinds of scholarship. The explicit differentiation of types of scholarship may have played a role in broadening residencies understanding of and efforts in scholarship. This increased emphasis on scholarship in residency may mirror similar changes seen in other specialties when scholarship PRs are made more rigorous.9
It is important to keep in mind the limitations of our data. Data on scholarship was collated from Web ADS data submitted by programs online for accreditation purposes. This may not represent the total scholarship output of residency faculty, or it may also be overinflated to satisfy accreditation requirements. Importantly, assessing the quality or impact of the publications and presentations was not possible; quantity is not the only important metric. The data are limited to 5 years and were deidentified, limiting conclusions about earlier years, and not allowing assessment of the impact on different kinds of programs, the impact of the pandemic or longer-term impact on scholarship. Moreover, the data are limited to faculty; and what residents do, either alone or in teams, should be an important consideration. The data do not speak to resident involvement in research after residency., Nor do they address the impact of scholarship on future research productivity in the specialty. In addition, imprinting competency in the critical evaluation of medical literature, including assessing study validity and the applicability of studies to the residents’ patients cannot be deduced from this data. Finally, citation data after 2020 was unavailable, limiting the conclusions regarding the impact of the pandemic. Despite these limitations, however, we believe that the changes in residency requirements have made a significant substantial impact. Given what is known about imprinting of residents—what happens in residency has impacts for at least 15 to 20 years—the question becomes what should we recommend for the future of the residency requirements to support building research capacity for the specialty?
As a first premise, the name is important to keep in mind: this is about scholarship rather than research, and the RC’s use of Boyer’s framework seems appropriate. Moreover, the emphasis on types of scholarship and dissemination are appropriate.
An important additional question for the discipline is whether the outlets for faculty for dissemination are appropriate, particularly with the closure of the Journal of Family Practice in the Fall of 2023. PRiMER has provided an outlet for clinician teachers for educational scholarship: do we have enough capacity for online publications for faculty clinician scholars? STFM, NAPCRG and other conferences provide an outlet for presentation but are additional opportunities for presentation and dialog necessary? As demonstrated by the I3 and P4 collaboratives, Residency Learning networks provide opportunities for presentation as well as a structure for scholarship in collaboratives broader than a single residency. Finally, it is important to broaden the focus from just faculty. Should the standards for scholarship in residency be changed with the next minor revision? As a matter of protocol, it is important to understand that the RCFM has the sole authority responsible for any revisions to the program requirements. Given the need of the specialty to develop research capacity, and based on this data, we recommend that a minor revision be considered. Keeping the current PRs as core and requiring peer reviewed scholarship as a domain is important, but should resident scholarship PRs be more rigorous? Presentations for resident scholarship “in house” are the default for most programs – we recommend more emphasis on dissemination outside of residencies. Presentation outside requires more preparation and provides an important opportunity for faculty to mentor residents. A detail PR in this regard might generate some traction.
What is not represented in the citation data are whether or not a program received an Area for Improvement (AFI) regarding scholarly activity. AFIs are given when a program is deemed by the RCFM to have an area of the program requirements that does not meet the criteria for a citation, but still warrants attention by the program to avoid devolving into a citation in the future. AFIs for scholarly activity are usually given based on the peer judgment of the reviewers and are likely much more frequent than citations.
Based on the experience of one of the authors (GH), citations for scholarship most often fall into 2 groups: new programs who are applying or in initial accreditation phase, where a culture of scholarship has not yet matured; and then in those programs with multiple citations whose overall struggles and challenges are also evident in a lack of scholarship. Since the process of assigning citations and AFIs is one of peer judgment from the RCFM in consideration of the program through holistic review, there will also be variability in the application of accreditation decisions. Developing common understanding of what constitutes substantial compliance around scholarship in residency will important.
For example, is giving a presentation to residents as part of a regular didactic program adequate to count as fulfilling one of the domains of scholarly activity as a faculty member, or should only presentations given to a wider audience outside the program count? Should residencies be held to different standards depending on their resources? Such as a rural training track versus a university-based program.
Data from ACGME on scholarship is not granular enough to draw conclusions about the culture and commitment in individual programs, but it does indicate the influence of accreditation standards as a tool to promote scholarship as one piece of lifelong learning. The background and intent around several new requirements have explicitly stated that educational collaboration between programs is a means to meet the requirements and is strongly encouraged. It remains to be seen if this will come to fruition.
Notes
This article was externally peer reviewed.
Funding: This manuscript was not funded by any outside sources.
Conflict of interest: The authors have no conflicts of interest.
To see this article online, please go to: http://jabfm.org/content/37/S2/S41.full.
- Received for publication January 4, 2024.
- Revision received May 15, 2024.
- Accepted for publication September 24, 2024.
In this issue
Jump to section
Related Articles
Cited By...
- No citing articles found.