Abstract
Introduction: The Colorado State Innovation Model (SIM) initiative aimed to support primary care practices in implementing behavioral health integration and other aspects of advanced primary care. This project evaluated implementation of behavioral health integration and the Building Blocks of High Performing Primary Care in primary care practices.
Methods: A total of 334 practices participated in SIM in 3 cohorts for 1 year (cohort 3) or 2 years (cohorts 1 and 2). Practice transformation support for implementation of advanced primary care and behavioral health integration was provided. Measures included 1) Comprehensive Primary Care Practice Monitor, completed at baseline, 12, and 24 months to assess implementation of the building blocks of primary care and behavioral integration, and 2) Behavioral health clinical quality measures, reported quarterly.
Results: Practices improved on implementation of all building blocks, including leadership, data driven improvement, empanelment, team-based care, patient-team partnerships, population management, continuity of care access to care, care coordination, resource utilization, behavioral health integration, and general behavioral health improvement (all P < .0001). Onsite behavioral health integration was implemented by 78% of practices by the end of the intervention. Practices improved on depression screening and monitoring (P < .0001) and maternal depression screening (P < .001). Implementation of several building blocks mediated improvement in depression screening and monitoring and maternal depression screening (P < .05).
Discussion: Practices in the SIM initiative successfully implemented behavioral health integration and the building blocks of high performing primary care, yielding improved behavioral clinical quality measures. Implementation of the building blocks mediated improvements in clinical quality measures.
- Behavioral Medicine
- Colorado
- Evaluation Studies
- Family Medicine
- Integrated Health Care Systems
- Medical Home
- Mental Health
- Organizational Change
- Primary Health Care
- Quality Improvement
Introduction
Behavioral health integration in primary care settings has been shown to improve outcomes and decrease costs associated with patients who have a behavioral health condition or a chronic medical condition with behavioral health contributing factors.1,2 Systematic reviews have demonstrated the effectiveness of integrating behavioral health service delivery into the primary care setting.3⇓–5 Collaborative care models for depression involving management in primary care practices with regular consultation from a psychiatrist have been shown in multiple projects to be effective.6⇓⇓–9 Behavioral integration in primary care practices has been put forth as a model for improving care and reducing costs by health policy makers and as part of national demonstration projects.10,11 However, implementation of new evidence-based interventions and models including behavioral health integration has been challenging for primary care practices.12,13 Practice facilitation has been shown to be effective in assisting practices with implementing new interventions and care models.14⇓⇓⇓⇓–19
In 2015 Colorado was awarded a cooperative agreement by the Center for Medicare and Medicaid Innovation (CMMI) as part of the State Innovation Model (SIM) initiative.20 CMMI’s SIM program required states to develop and test innovative models for transforming health care payment and delivery systems in moving value-based payment systems forward. Colorado focused specifically on supporting primary care practices in integrating behavioral and physical health and to prepare for alternative payment models. Colorado SIM’s overarching goal was to improve the health of Coloradans by expanding access to integrated primary care and behavioral health services in coordinated community systems with value-based payment structures. Integrated behavioral health was defined as “the care that results from a practice team of primary care and behavioral health clinicians, working together with patients and families, using a systematic and cost-effective approach to provide patient-centered care for a defined population. This care may address mental health, substance abuse conditions, health behaviors (including their contribution to chronic medical illnesses), life stressors and crises, stress-related physical symptoms, and ineffective patterns of health care utilization.”21 Behavioral integration models were seen on a continuum from coordination of care, to colocated behavioral and primary care, to full integrated, team-based care.22 The goal for the Colorado SIM project was to move practices along the behavioral integration continuum, but with recognition that not all practices would have the resources or circumstances to achieve a fully integrated model.
The Colorado Multi-Payer Collaborative and project leaders chose to use the Bodenheimer Building Blocks of High Performing Primary Care (building blocks) as a framework for assisting practices through practice facilitation and other supports in preparing for new alternative payment models in the project.23 The building block framework draws concepts from the patient-centered medical home, the chronic care model, Starfield’s 4 pillars of primary care, and other models of advanced primary care to delineate essential functions of high performing primary care in a tiered approach that provides guidance to practice regarding the sequencing of implementation of these functions. For this project, the building blocks were modified slightly to include the focus on behavioral health integration.
There were multiple elements involved in Colorado SIM, including efforts to build the health information technology infrastructure, pilot primary care integration into community mental health centers, and promote population health improvement across multiple sectors. The Colorado SIM final report provides a broader, detailed description of the goals and activities of the initiative, and the final process evaluation report details results from the program-wide evaluation, including practice outcomes broken out by practice cohorts, not reporting results across all practices.24,25 This analysis focuses on the impact across all practices of the efforts to support primary care practices in implementing integrated behavioral health as part of a model of advanced primary care on practice progression along the integration continuum and clinical quality measures.
Methods
Design
Intervention
The Bodenheimer 10 Building Blocks of High Performing Primary Care describes steps for practices to implement the concepts of the patient centered medical home and other advanced primary care models.23 This framework, modified to include a stronger focus on integrated behavioral health, was chosen by project stakeholders to establish milestones for practices to accomplish during the project and beyond. The practice support intervention was designed and overseen by the Colorado Governor’s Office, the Practice Innovation Program at the University of Colorado, the Colorado Multi-Payer Collaborative, and multiple other stakeholders and included:
Practice facilitators provided assistance to practices in implementing the milestones, with monthly in-person meetings with a multidisciplinary improvement team at the practice and intermittent email and/or phone communications. The practice facilitators were trained professionals from various backgrounds who worked for various “practice transformation organizations” that provide practice transformation support to various groups of practices across Colorado. The practice facilitators were provided with additional training on advanced practice facilitation and supporting behavioral integration, and their performance and practice advancement were closely monitored through a quality assurance process. An extensive implementation guide was provided to guide the training and assist the practices and practice facilitators in their work.
Clinical health information technology support assisted practices for clinical quality measure reporting through their electronic health records and in using their data for population management.
Collaborative learning sessions, bi-annual meetings of clinicians and staff members from the practices, were provided for collaborative learning.
“Achievement payments” of up to $13,000 were provided to practices based on their level of implementation of the building blocks.
Small competitive grants of up to $40,000 were given to a limited number of practices to support implementation of integrated care.
Advanced payments to support integrated care that varied across the different health plans were also provided to practices. We were unable to track the amount or the specific framing of these payments due to anti-trust regulations.
Support was rolled out in cohorts of practices, with cohorts 1 and 2 receiving 2 years of support and cohort 3 receiving 1 year.
The project was reviewed by the Colorado Multiple Institutional Review Board (protocol #19 to 0598) and judged to be not human subjects research but a program evaluation of a quality improvement initiative.
Measures
Data collection as part of this project occurred from late 2016 through mid-2019. Each practice completed an application to participate in the project, which included extensive practice demographics, level of behavioral integration, participation in previous practice transformation efforts, and patient-centered medical home recognition.
The Comprehensive Primary Care Practice Monitor (the Monitor) is a validated instrument used in previous projects to assess implementation of elements of the patient-centered medical home (PCMH).26 Items from the earlier version were modified and new items developed by a team of physicians, researchers, and practice facilitators to reflect key activities and milestones within the Bodenheimer building blocks framework. For this project, building block 10 of Bodenheimer’s original building blocks, dealing with a “Template of the Future,” was modified to focus on resource utilization, and 2 additional building blocks were added to focus on general behavioral health improvement and behavioral health integration specifically. The Monitor tool is included as Appendix 1. The Monitor was completed at baseline and yearly by each practice improvement team, facilitated by the practice facilitator, to help practices in their self-assessment and planning in implementing the milestones, and to evaluate their progress. Each item was scored from 0 to 4 with zero indicating no activity implementation and 4 indicating complete implementation. The scores were then summed and converted to a scale of 0 to 100 for each building block.
The Integrated Practice Assessment Tool (IPAT) is a brief survey designed to assess the level of behavioral health integration in primary care practices as defined by the Standard Framework for Levels of Integrated Care.27,28 The IPAT defines a spectrum of increasing levels of collaboration and integration from level 1 (minimal collaboration) up to level 6 (full collaboration in a fully integrated practice). This self-assessment, 8-item instrument was administered to practices at baseline and yearly.
Practice-level clinical quality measures were collected quarterly through reports generated by electronic health records across the period of practice participation. Each measure was based on patients seen in the practice within the preceding 12 months. Quality assurance strategies were imposed for all measures, and any data points that deviated from expected values were verified and corrected, if necessary, by the data manager and the clinical health information technology advisor. The primary mental health measures were
Screening for Clinical Depression and Follow-Up Plan (NQF 0418): Percentage of patients aged 12 years and older screened for clinical depression using an age-appropriate standardized depression screening tool AND if positive, a follow-up plan is documented on the date of the positive screen.
Maternal Depression Screening (NQF 1401): The percentage of children who turned 6 months of age during the measurement year who had a maternal depression screening for the mother at least once between 0 and 6 months of life.
Data Analysis
Initially, descriptive statistics were computed for baseline practice characteristics, testing for differences between cohorts. To assess improvement in practice implementation of the building blocks and clinical quality measures over time, we employed general linear mixed effects models (practice random effect) that utilized all available data, assuming ignorable missingness. Outcomes for these analyses were building block scores over time and clinical quality measures over time. Practice-level covariates included cohort, organization type, specialty type, rural location, and practice size. In addition, we performed mediator analyses29 to explore whether change in implementation of the building blocks was associated with improvement in clinical quality measures. In general, we hypothesized that practices that had greater improvement in the building blocks would have greater improvement in clinical quality measures. Outcomes for mediator analyses were change scores (from baseline) for some key clinical quality measures (maternal depression screening, depression screening and treatment). Baseline values of the clinical quality measure, baseline scores for building blocks, 12 month change scores for building blocks (tested 1 at a time), and relevant covariates (cohort, rural/urban location, organization type, specialty type, and practice size) were included in the models. Hypothesis tests were 2-sided with α = 0.05 or P values reported. All statistical analyses were performed using SAS version 9.4 (SAS Institute Inc., Cary, N.C.).
Results
Practice Engagement
The CONSORT diagram can be found in Figure 1. 448 practices initially applied for participation. A selection committee screened to make sure the applicants were actually delivering primary care and then rated practices according to their demographic characteristics (to assure a diversity of sites across the state) and their alignment with project goals, with limited capacity for practices during the first 2 cohorts of the project. 344 practice sites were offered participation and initially enrolled for the project; 334 actually began the intervention and completed baseline surveys. Fifteen practices discontinued participation during the intervention, resulting in 319 practices completing the final 12- or 24-month interventions. Most practice characteristics were similar between dropouts and those that remained; hospital-based practices had higher dropout rates than other organization types (P < .05).
CONSORT diagram. Abbreviations: PPA, Practice Participation Agreement; MOU, Memorandum of Understanding.
Practice characteristics for the 334 baseline practice participants can be found in Table 1. Most of the practices that withdrew did so due to the practice closing, being purchased, or merging with another practice. Practice engagement resulted in a good reach geographically, with almost a third of the practices in rural areas and the rest in urban areas, and with representation across the full range of primary care specialties serving various age groups.
Practice Characteristics
Practice Implementation of Building Blocks
Practice Monitor subscales other than those addressing behavioral health had been previously mapped to the Bodenheimer building blocks in prior projects. Principal factor analysis was used to assess the new items addressing behavioral health in primary care, resulting in 2 distinct subscales, 1 related to general improvement in behavioral health care and 1 more specific to behavioral health integration. Cronbach’s α was computed for each subscale to confirm internal consistency as follows: Engaged Leadership (α = 0.83), Quality Improvement Process (α = 0.93), Data Driven Improvement (α = 0.83), Empanelment (α = 0.77), Team-based Care (0.82), Patient-Team Partnership (0.73), Population Management (0.74), Continuity of Care (0.43 (only 2 items)), Access to Care (α = 0.71), Care Coordination (α = 0.64), Resource Utilization (α = 0.74), Behavioral Health Improvement (α = 0.88), and Behavioral Health Integration (α = 0.90).
Table 2 shows the practices’ assessments of their implementation of core aspects of each building block using the Practice Monitor, with the number of points out of 100 possible. In addition to substantial improvement during the initial intervention year, improvement was sustained during the second year for all building blocks. Practices progressed across all areas, and change over time was highly significant at P < .0001 overall and for each subscale.
Change in Building Block Implementation Over Time
Clinical Quality Measures
Although practice support did not especially focus on quality improvement of specific clinical quality measures, practices reported on a set of such measures across the duration of their practice facilitation to monitor the impact of the changes they were implementing on the clinical quality measures. The primary mental health clinical quality measure for practices providing care for adults or all ages was the percentage of patients screened for depression and having documentation of follow-up for positive screens. The mean percentages for this measure improved significantly across the project (F(1306) = 89.04, P < .0001), as seen in Figure 2. Pediatric practices reported on the percentage of mothers of newborns screened for maternal depression (Figure 2), and the percentage of mothers screened increased substantially and significantly across the project (F(1,67) = 19.82, P < .0001).
Improvement in depression screening and treatment quality measures.
Implementation of Integrated Behavioral Health Provider
Although our goal was full behavioral integration with a behavioral health provider available onsite, we knew going into the project that some practices would not be able to implement full behavioral integration due to a shortage of behavioral health professionals (especially in rural areas), lack of funding or other resources, and a variety of other issues. Even for those practices unable to achieve full integration, there were multiple things that they could implement to improve their care of behavioral health issues, including implementation of collaborative care. However, helping practices accomplish having an integrated behavioral health professional was a goal of the project. The mean scores on the Integrated Practice Assessment Tool (IPAT), used as one measure to assess the practices’ level of integration, increased significantly from 3.48 (out of a maximum of 6) to 4.76 across the intervention (F(1554) = 132.07, P < .0001). Furthermore, out of the 319 practices completing the intervention, 248 or 78% reported they had an onsite behavioral health provider at the end of their participation. In addition, 12 practices reported full-time access to a behavioral health provider through telehealth.
Mediator Analyses
Mediator analyses were conducted with each building block separately to better understand the mechanism of improvement and to eliminate concerns about collinearity among subscales. We estimated how much additional change per year in outcomes was associated with a 10-point improvement in each of the building blocks as well as the overall score for all building blocks. Results are shown in Table 3. Improvement in resource utilization was significantly associated with improvement in maternal depression screening (P = .0194). Improvement in improvement in multiple building blocks was associated with depression screening and treatment, including Data Driven Improvement (P = .0159), Patient Team Partnership (P = .0187), Population Management (P = .0207), Continuity of Care (P = .0135), Access to Care (P = .0472), Care Coordination (P = .0051), Resource Utilization (P = .0115), Behavioral Health Improvement (P = .0071), and Behavioral Health Integration (P = .0001).
Additional Change per Year by 10-point Improvement in Building Block Implementation*
Discussion
In summary, the practice transformation support resources provided in this intervention were very successful in supporting practices in implementing behavioral health integration and other components of improved behavioral health care, and in achieving the milestones that were set out for them for other aspects of advanced primary care as delineated by the Bodenheimer building blocks. Practices also improved their care of patients with behavioral health issues, as demonstrated through improvements in clinical quality measures related to behavioral health. Most practices were able to implement some level of behavioral health integration, while others worked on other aspects of improving behavioral health care delivery.
This project is important as one of the largest, if not the largest, initiatives supporting the integration of behavioral health services in primary care practices. It further demonstrates the effectiveness of practice facilitation and other supports in assisting practices in implementing behavioral health integration and other key aspects of advanced primary care. Interestingly, implementation of various activities associated with advanced primary care, as delineated by the building blocks and measured by the Practice Monitor, were associated with greater improvements in the depression screening and follow-up clinical quality measure, which was the measure most consistently reported across practices, although not with maternal depression screening as reported by the pediatric practices. The results with depression screening and monitoring replicates similar findings in EvidenceNOW Southwest30 and provides further evidence that implementation of foundational activities aligned with advanced primary care models as framed by the building blocks can improve care in multiple specific clinical areas as measured by clinical quality measures and support sustainable change.
Limitations include practice self-reporting of key data, including the Practice Monitor, the IPAT, and the reporting of the presence of integrated behavioral health professionals. Practice facilitators were trained to hold practices accountable for their reporting, but some bias certainly could have influenced results. This project was designed as a real-world implementation project without a control or comparison group. Furthermore, practices participating in the project may not be representative of the broader group of primary care practices, despite the diverse characteristics of those that participated.
With increasing focus on behavioral health integration in primary care as part of advanced payment models, further study of the impact of behavioral health integration on patient outcomes and utilization patterns, the key elements of such integration, and the cost and necessary reimbursement for practices implementing behavioral health integration are needed.
Notes
This article was externally peer reviewed.
This is the Ahead of Print version of the article.
Funding: This project was supported by cooperative agreement from the U.S Department of Health and Human Services, Center for Medicare & Medicaid Innovation - Funding Opportunity Number CMS-1G1-14-001.
Conflict of interest: None.
To see this article online, please go to: http://jabfm.org/content/00/00/000.full.
- Received for publication February 21, 2024.
- Revision received May 2, 2024.
- Accepted for publication May 13, 2024.