Abstract
Background: Integrating behavioral health into primary care can improve care quality; however, most electronic health records are not designed to meet the needs of integrated teams. We worked with practices and behavioral health (BH) clinicians to design a suite of electronic health record tools to address these needs (“BH e-Suite”). The purpose of this article is to examine whether implementation of the BH e-Suite changes process of care, intermediate clinical outcomes, and patient experiences, and whether its use is acceptable to practice members and BH clinicians.
Methods: We conducted a convergent mixed-methods proof-of-concept study, implementing the BH e-Suite across 6 Oregon federally qualified community health centers (“intervention clinics”). We matched intervention clinics to 6 control clinics, based on location and patient panel characteristics, to assess whether process of care (Patient Health Questionnaire-9 [PHQ-9] and Generalized Anxiety Disorder-7 screening) and intermediate outcomes (PHQ-9, Generalized Anxiety Disorder-7 scores) changed postimplementation. Prepost patient surveys were used to assess changes in patient experience. To elucidate factors influencing implementation, we merged quantitative findings with structured observations, surveys, and interviews with practice members.
Results: Implementation improved process of care (PHQ-9 screening). During the course of the study, change in intermediate outcomes was not observed. Degree of BH e-Suite implementation varied: 2 clinics fully implemented, 2 partially implemented, and 2 practices did not implement at all. Initial practice conditions (eg, low resistance to change, higher capacity), process characteristics (eg, thoughtful planning), and individual characteristics (eg, high self-efficacy) were related to degree of implementation.
Conclusions: Health information technology tools designed for behavioral health integration must fit the needs of clinics for the successful uptake and improvement in patient experiences. Research is needed to further assess the effectiveness of this tool in improving patient outcomes and to optimize broader dissemination of this tool among primary care clinics integrating behavioral health.
- Community Health Centers
- Electronic Health Records
- Oregon
- Patient Health Questionnaire
- Primary Health Care
- Proof of Concept Study
- Surveys and Questionnaires
Each year, 26% of the United States population experiences an emotional, mental, or behavioral health problem, and the majority of these individuals are seen in primary care settings.1,2 Here we define behavioral health as encompassing “any behavioral problems bearing on health, including mental health and substance abuse conditions, stress-linked physical symptoms, patient activation, and health behaviors”.3 Research supports the inseparability of behavioral health care and medical care4⇓–6, and strong evidence shows that integration, bringing behavioral and medical care together, improves clinical outcomes, improves patient experiences of care, and reduces health care costs (ie, the “Triple Aim” of health care).7⇓–9
Less well known is how to effectively deliver integrated care in routine primary care practice. At many practices beginning to integrate care, a new professional joins the team, a behavioral health clinician (BHC). BHCs are embedded in primary care teams to provide brief, targeted treatment to patients with mild-to-moderate behavioral health conditions. Although most primary care practices now routinely use electronic health records (EHRs),10 these EHRs are not tailored to the specific tasks or workflows of BHCs embedded in primary care clinics. Technological improvements are needed to enable BHCs to use the EHR for integrated care delivery.11⇓–13
In Turning EHRs into Assets for Mental Health and Uniting Practice (TEAM-UP), we collaborated with OCHIN, a not-for-profit health information network that provides health information technology (IT) solutions to a collaborative of 510 community health centers (CHCs) on a single instance of Epic EHR. In the United States, CHCs comprise the country's health care safety-net by providing primary care and public health services to under or uninsured, low-income patients. A detailed description of OCHIN can be found elsewhere.14, 15 With input from practicing BHCs, we developed a suite of user-centered tools (the BH e-Suite) to address the information needs of BHCs practicing in integrated primary care practices. The BH e-Suite is incorporated into Epic and enables integrated care delivery by organizing the tools BHCs use into a single tab, with fast links to those tools. In addition, we added functionality that supports psychosocial assessment, information-gathering, goal-setting, documentation, monitoring, and tracking, by using a range of point-and-click functions and drop-down menus. It also includes templates for commonly used screeners (eg, Patient Health Questionnaire-9 [PHQ-9]; Generalized Anxiety Disorder-7 [GAD-7]) that auto-calculate and auto-populate progress notes and show changes in scores over time.
This study had the following aims:
Examine whether implementing the BH e-Suite changes process of care, intermediate clinical outcomes, and patient experiences postimplementation;
Assess acceptability of the BH e-Suite to BHCs; and
Identify clinic and practice member factors influencing the implementation of the BH e-Suite.
We hypothesize that BHCs will find the BH e-Suite acceptable to use. We also hypothesize that implementing the BH e-Suite will improve process measures (eg, rate of screening with PHQ-9) and patient experience of care. However, we do not expect improvements in patient depression symptoms (PHQ-9 scores) during the short time frame of this study.
Methods
Design
We used a convergent, iterative, mixed-methods design to accomplish the main aims of this study.
Clinic Sample
Six OCHIN CHCs (intervention clinics) were recruited to implement and use the BH e-Suite. A detailed description of the intervention clinics is provided in Table 1. Clinics were offered minimal support to implement the tool (ie, training manual, one 13-minute video) and no financial incentives. Although the functionalities we developed are organized into a single “suite” for BHCs, this functionality and the information documented were available to other users practice-wide. BHCs, however, were the primary users of the BH e-Suite in this study. Propensity scoring methods were used to match the intervention clinics to control clinics not using the behavioral health tool, based on 7 criteria: (1) geographic location (rural vs urban), (2) race (% White, log transformed), (3) ethnicity (% non-Hispanic White, log transformed), (4) insurance (% Medicaid), (5) percent of adults with serious mental health disease (log transformed), (6) number of active primary care providers, and (7) number of active patients. Thus, 6 control clinics were similar to intervention clinics on the above factors but did not use the BH e-Suite.
Description of Integration and Implementation Approach by Practice
Patient Sample
Eligible patients were frequent attenders (defined as those who were in 75th percentile for the number of outpatient visits), over 18 years old, diagnosed with at least one behavioral and one medical condition, and seen for a behavioral health visit 12 months before and after implementation of the BH e-Suite.
Measures and Data Collection
Process of care measures included the proportion of patients screened for PHQ-9 and GAD-7. Intermediate clinical outcome measures included PHQ-9 and GAD-7 scores. These measures were extracted for eligible patients from the 6 intervention and 6 control clinic EHRs 12 months before and after implementation.
Patient experience-of-care measures included previously validated patient-reported measures of care coordination (Picker Institute Scale, scores range 1 to 3 and 1 to 5),16 satisfaction with care (scores range 1 to 5), integration of care (Primary Care Assessment Tool, scores range 1 to 3),17 and care communication (Mercer Connecting, Assessing, Responding, and Empowering [CARE] Tool, scores range 1 to 5; see Appendix).18 These measures were collected by surveying a sample of eligible patients among the intervention clinics at baseline and again at 6 months postimplementation. Surveys were administered in the waiting room by either a research assistant or by a front desk staff person and placed in a secure lockbox after collection to ensure anonymity. An alert within the EHR informed clinic administration that the patient was eligible for a survey. At baseline, 337 patients responded to surveys and postimplementation, 302 patients responded to surveys.
Measures of BH e-Suite acceptability included domains from the Technology Acceptance Model (TAM),19 a 16-item survey completed by BHCs at baseline and postimplementation that measured the following: (1) Perceived usefulness, including the degree to which the EHR aligned with BHCs' current workflow and tasks and also improved job performance, ability to manage patient care (eg, assess and diagnose patients), organize patient information, and coordinate care with integrated care team members; (2) perceived ease of use, including the perceived burden, time-efficiency, learnability, and interface qualities (ability to navigate and enter information); and (3) percent usability, including the frequency at which the BHCs found themselves abandoning the BH e-Suite, and the extent to which they used all the available BH e-Suite features.
Factors Influencing Implementation
Practice capacity for change was measured using the validated 23-item Adaptive Reserve questionnaire,20⇓–22 which was collected among intervention clinics via a practice member survey at baseline. Practice member surveys were distributed at monthly all-staff clinic meetings at baseline and had a greater than 75% response rate across practices. Qualitative methods (observations and interviews) were used to assess factors influencing implementation of the BH e-Suite. Observations were conducted by a field team experienced in qualitative research, primary care delivery, informatics, and human factors research. This team conducted a 2-to-4-day site visit at the 6 intervention clinics over a 6-month period, November 2013-April 2014, before implementation of BH e-Suite. The length of visit depended on practice size, and each visit involved intensive observation of the integrated care team's EHR use, including observing individual work areas, team work areas, and encounters with patients. In addition, we shadowed primary care clinicians, BHCs (ie, licensed clinical psychologists and social workers), and other key members of the clinical care team, and sat in on their visits with patients when permitted. Among all 6 participating clinics, we accumulated a total of 184 single-spaced pages of field notes from approximately 150 person-hours of observing all aspects of clinical care.
Semistructured interviews were conducted at clinic site visits with 2 to 4 practice members representing different roles in the practice (eg, BHC, primary care providers, medical assistants) to understand their approach to integrating care, the workflows and tasks involved in integration, and how they used the EHR for documentation and information sharing. By observing and interviewing different roles in the practices, we gained a robust understanding of how care for behavioral health conditions was delivered at baseline and the environment in which the BH e-Suite would be implemented. We also conducted 6-month postimplementation interviews with BHCs to ascertain the level of implementation, experience with the tool, and changes in clinical practice connected to tool use. Interviews followed a semistructured guide23 that assessed integrated care workflows and tasks, experience with the current EHR, practice structure, and organizational culture. We conducted 29 interviews (11 BHCs, 4 medical assistants, 7 mental health clinicians [MHCs], and 7 primary care clinicians), each lasting between 45 minutes and 1 hour. In intervention practices, MHCs provide care for patients with more severe and persistent mental health issues, while BHCs provide care to patients with mild to moderate mental health problems over a shorter period of time.
Data Analysis
Descriptive statistics (tabulations, percentages, means, and standard deviations) were used to describe EHR-derived process of care and intermediate clinical outcome measures, stratified by case and control clinics at baseline. We used SAS PROC MIXED24 to model 2-level, mixed effects (random intercept) regression to account for the clustered sampling design (patients clustered within clinics). Linear regression models were designed to evaluate the interactive relationship between pre- and postimplementation and intervention/control in regard to patient PHQ-9 and GAD-7 scores (dependent variable) across intervention and control clinics. Practice member and patient surveys were descriptively analyzed by examining response frequency distributions. Among intervention clinics, t tests were computed to examine the mean change after implementation in patient-reported outcome measures (eg, care coordination, care communication with primary care physician, integration of care, and satisfaction with care). When specific survey items were not answered, respondents' items were excluded from analyses. Missing data ranged from 1% to 12% among patient-reported constructs.
Our multidisciplinary team used a 4-step approach to analyze qualitative data.25 First, we collectively read field notes and interview data collected from each practice to develop an understanding of each practice, as a case, with particular attention to factors influencing implementation of the tool, BHC workflows, tasks, and tool ease-of-use. Through this process, we created a codebook to label text. We used these codes in group analyses until we reached stability; code definitions were standardized and analysts were trained to code data reliably. Individuals completed the data analysis, meeting as a group to compare work and come to consensus when analytic differences emerged.
Second, we analyzed findings across clinics to identify common themes. We grouped emerging findings into categories of themes by using an immersion-crystallization approach.26 Third, we connected our findings to the existing literature.25 We conducted a second in-depth comparative analysis by using the Consolidated Framework for Implementation Research (CFIR)27 constructs to understand differences in implementation across intervention clinics. After considering all 39 CFIR constructs, 7 relevant CFIR constructs that mapped to 3 domains (inner setting, process characteristics and individual characteristics) were identified and an in-depth analysis was conducted to identify how these factors influenced implementation of the BH e-Suite.
Qualitative and quantitative analyses were first conducted independently to ensure unbiased interpretation. After completing analyses, qualitative and quantitative findings were integrated at the practice level by using data-triangulation techniques.25 The Institutional Review Board at Oregon Health & Science University approved this study protocol (No. 9366).
Results
The distribution of patient characteristics was similar between intervention and control clinics (Table 2).
Patient Sample Description of Intervention and Control Clinics at Baseline
Intervention clinics implemented the BH e-Suite to varying degrees: two practices used all the features and functions of the BH e-Suite EHR tool (“fully implemented”); 2 practices implemented some, but not all, of the features of the tool (“partially implemented”); and 2 practices did not use the tool at all (“did not implement”).
Clinical Care and Patient Perception of Care
Overall, there was a significant increase between intervention and control clinics in process of care, specifically the proportion of patients screened with PHQ-9 (Table 3). There was no change in intermediate clinical outcome measures (ie, PHQ-9 and GAD-7 scores). When examined by degree of implementation, significant but small differences were observed in outcome measures between intervention and control clinics. Table 3 also displays the change in patient-reported experience of care postimplementation. Patient perceptions of integration-of-care improved overall and among clinics that fully implemented the tool. Patient perceptions of communication with primary care providers significantly improved only among clinics that partially implemented the tool.
Clinical Care and Patient Experience of Care Outcomes across Degree of Implementation
Acceptability of BH e-Suite
Clinicians working in clinics that fully or partially implemented the BH e-Suite reported that the tool was acceptable and easy to use. They reported the tool added 1 to 2 minutes to the initial visit but saved a significant amount of time during follow-up visits by automatically populating the history of the presenting illness and patient instructions at subsequent visits. As a BHC from clinic 5 who fully implemented the tool reported:
“[Tool adoption was] actually really easy because we were a small, very small core group of people when we started, and as we brought on new hires the system adopted it. Our IT people adopted it, our Epic people adopted it and it is not like you have a choice. This is just what we're doing. We're using this. As we've hired new folks, this has just been what they have been introduced to and it is been great.”
BHC reported finding the tool beneficial, too:
“I would say the biggest gain for me has been in follow-up visits so I am not having to copy paste from previous visits. It is really streamlined the process for follow-up in terms of mental status examination and history of the presenting illness. Those parts have helped immensely” (Clinic 2).
Clinics that fully or partially implemented the tool found it easy to use (mean Technology Acceptance Model score, 3.75 and 3.5, respectively). Among clinics that partially implemented the BH e-Suite, clinicians reported that they would use the tool more often if they were given more training in its use. For instance, a newly hired BHC at Clinic 1 who only used the tool for documenting encounters, explained that she was just learning to use Epic and picking up the BH e-Suite as well as learning to practice in a new setting was difficult: “I just stopped using it [all] because… it just seemed like a lot… it just takes time and practice, but some days my schedule is just really full, and I just have to go quickly.” All clinics that implemented the tool to some degree reported an increase in perceived usability from preintervention (mean, 1.87) to postintervention (mean, 3.78) (data not shown).
Factors Influencing Implementation of the BH e-Suite
Clinics that fully implemented the BH e-Suite had higher adaptive reserve scores, indicating greater capacity for change (Table 4). They also had leadership buy-in and support, more resources dedicated to implementation, processes and workflows developed in advance of implementation, staff of all levels engaged in implementation, and BHCs who were formally trained in use of the BH e-Suite. BHCs noted that the tool simplified EHR use, and they welcomed it as a way to help address their health IT needs. The lead BHCs championed tool implementation and supported staff use of the tool by creating step-by-step guides, allocating time and space for training and fostering collaboration among BHCs and Epic specialists. Clinics that partially implemented had less adaptive capacity and needed more time, training, and support to use the tool. They also experienced important disruptive events (turnover in BHC staff), which was a barrier to implementation. Clinics that did not implement the BH e-Suite had the lowest adaptive capacity and also lacked leadership buy-in and support for use of the BH e-Suite. Importantly, although these clinics agreed to participate in the study, saying they would have an embedded BHC on staff, they were unable to fulfill this promise and, instead, continued to refer patients to a colocated MHC who had different information and documentation needs; MHCs at these clinics used a different EHR tool to meet their unique documentation needs.
Implementation Degree across Practices and Consolidated Framework for Implementation Research Domains
Discussion
Policy reforms have been implemented across the United States in support of the integration of behavioral health services into primary care clinics,28 and there are an increasing number of initiatives aimed at integrating care across diverse settings.29⇓–31 This momentum to change practice and policy is crucial because integrated care is specifically designed to identify and address patient medical and behavioral health care needs together and to improve quality, patient experience, and reduce costs.4,6,32⇓⇓–35
Systems that have integrated, or are planning to integrate, behavioral health and medical care must consider an investment in health IT. Documentation requirements for BHCs are different from other members on the integrated team. Therefore, clinics must think about workflows, tasks, and health IT functionality to effectively use the EHR to deliver high quality integrated primary care. Our study showed that a user-centered suite of EHR tools tailored for BHCs was feasible to use and acceptable to embedded BHCs practicing in integrated CHCs. Importantly, use of the BH e-Suite increased the perceived productivity of BHCs and improved process of care. At the outset of this study, we were sensitive to the unintended effects of having BHCs use the EHR during patient visits, as this might negatively impact the therapeutic relationship, patient experience of care, and intermediate clinical outcomes. With regard to patient experiences of care, we found the opposite to be true; use of the BH e-Suite improved patient perceptions of communication with their clinicians and BHCs, as well as their perceptions of receiving integrated care. Through in-depth observation, interviews, and the intentional engagement of users in the design process, we successfully developed a tool that was acceptable to patients and BHCs and avoided a common error of overcomplicating documentation for short BHC visits. We believe that this careful design work may have contributed to the BH e-Suite's ability to create care efficiencies and patient connection, without compromising the therapeutic relationship.
Our study has important lessons for clinics planning to use health IT tools for integration. In order for clinics to invest the time to implement the BH e-Suite, these tools must fit their needs. Clinics that had a “colocated” approach to care (patients internally referred to specialty MHCs for longer-term care instead of to an integrated BHC) did not find the BH e-Suite fitting. In fact, these clinics had a competing set of tools, specifically designed by and for specialty MHCs; not surprisingly, these clinics preferred the other tools. This suggests that motivation for implementation and perceived usefulness of a tool are aligned and must be carefully considered during health IT implementation.
Two study clinics with high motivation and adaptive capacity implemented these tools on their own and subsequently demonstrated improvement in patient experience. Two clinics that had midlevel adaptive capacity were only able to partially implement the BH e-Suite, despite recognizing its utility. These latter 2 clinics likely represent the norm among primary care clinics in the United States, which suggests the need for evidence-based implementation support strategies, such as practice facilitation or expert consultation, to assist with tool implementation. This finding has implications for clinics and researchers considering a health IT implementation; regardless of the improvements using a new health IT tool may bring, successful implementation may require external assistance.
Limitations
Our results should be considered in light of several limitations. First, the sample size of clinics was small, and generalizability of the study findings is limited. However, even with the small number of clinics, there was considerable variability in the integration and implementation approach, and we found that use of the BH e-Suite significantly enhanced patient experiences of care. Second, although we recognized the social and technical aspects of the change process when introducing a new tool within primary care,36 this study did not have the resources to provide implementation support to clinics, and we did not test how to best implement the BH e-Suite. Our qualitative findings provide formative insights into the factors that influence implementation, but more work is needed to study the effectiveness as well as the wider dissemination and implementation of this tool among CHCs. Third, some may consider it a limitation that we developed this tool in Epic, as this EHR system may not be affordable for all clinics. However, the partnership between researchers, OCHIN Epic developers, and CHCs made development of this tool possible. OCHIN serves over 500 CHCs across the nation, and they all now have access to the BH e-Suite. We have published details about the functionality of these tool elsewhere, allowing for this suite of tools and features to be developed by others23 and streamlining behavioral health integration.
Integrated primary care clinics need EHR tools that support the delivery of whole-person care. EHRs are seldom designed with behavioral health in mind; to date, few EHR-based tools have been developed and tested to support the work of BHCs and integrated teams. The BH e-Suite may fill this need, as it was acceptable to BHCs, feasible to implement by motivated clinics with some training, and has the promise to improve outcomes. More research is needed to assess the effectiveness of this tool in improving process and/or patient outcomes that are not just limited to depression symptoms but also to manage chronic diseases (such as type 2 diabetes) where integrated care has been shown to have an impact. Furthermore, future research should investigate how to disseminate and implement this tool widely among integrated primary care clinics.
Acknowledgments
We would like to express our deep gratitude with the clinics that participated in this pilot study. Without their participation, this study would have been impossible. In addition, we would like to recognize the input of Benjamin Miller, PhD, Timothy Burdick, MD, MS, and Rodger Kessler, PhD, all whom shared their expertise throughout the course of this study. We would also like to thank Amanda Delzer Hill for providing editorial assistance.
Appendix
Patient-Reported Measures
Notes
This article was externally peer reviewed.
Funding: The National Institute of Mental Health (R34 MH100371) funded this study.
Conflict of interest: none declared.
To see this article online, please go to: http://jabfm.org/content/31/5/712.full.
- Received for publication February 5, 2018.
- Revision received May 21, 2018.
- Accepted for publication May 25, 2018.
