Abstract
Purpose: In an age of value-based payment, primary care providers are increasingly scrutinized on performance metrics that assess quality of care, including the outcomes of their patient population in key areas such as diabetes control. Although such measures often adjust for patient clinical risk factors or clinical complexity, most do not account for the social complexity of patient populations, despite research demonstrating the strong association between social factors and health.
Methods: Using patient electronic health record data from 2 large community health center networks serving safety net patients, we assessed the effect of both clinical and social risk factors on poor glucose control among diabetics. Logistic regression results were used to estimate the impact of adjusting for both clinical and social complexity on provider performance metrics. Clinical complexity was measured at the patient-level using the Charlson Comorbidity Index. Social complexity was measured at the community-level using the Social Deprivation Index.
Results: Clinical complexity alone was not consistently associated with poor diabetes control (ie, HbA1c > 9%) in diabetic patients with HbA1c testing during the study period. However, increasing social complexity was significantly associated with higher rates of poor diabetic control in both cohorts. After adding adjustment for social complexity down to the national median score, our models suggest that approximately 25% of providers would have 1 to 2% improvement in the assessment of their diabetes control measures, with 45% showing a 2 to 5% improvement, and 5% showing more than a 5% improvement.
Conclusions: Providers caring for patients with greater social risk factors may benefit from having their performance metrics adjusted for the social complexity of their patient populations.
- Blood Glucose
- Chronic Disease
- Community Health Centers
- Comorbidity
- Diabetes Mellitus, Disease Management
- Electronic Health Records
- Glycated Hemoglobin A
- Logistic Models
- Patient Care
- Primary Health Care
- Risk Factors
- Social Determinants of Health
Introduction
In an age of value-based payment, primary care providers are increasingly scrutinized on performance metrics that purport to measure quality of care, including the health outcomes of their patient population and the extent to which their patients use services that increase costs, such as visits to the emergency department.1⇓–3 However, outcomes on many of these measures are influenced not only by provider actions, but also by patient behavior and comorbidity as well as social factors that impact their health. Indeed, a large body of research suggests that social determinants of health (SDH)—“the conditions in which people are born, grow, live, work, and age”—play a powerful role in shaping health and contribute as much, or more, to population health than health care services.4,5 Patients with more social risk factors (eg, lower income, limited education, and residing in highly deprived neighborhoods) have increased health care needs,6 face greater obstacles in following medical recommendations, and are more vulnerable to poor health outcomes.7,8 Recent evidence suggests that performance metrics may depend on the SDH of a system's patient population, as much or more than patients' clinical complexities or individual provider performance.9⇓⇓⇓–13 As a result, providers who serve populations with more social risk factors—or greater social complexity—may seem to have lower performance metrics, relative to those serving less socially complex populations.14,15
Despite a growing body of evidence suggesting that adverse SDH can negatively impact the Triple Aim of improved health outcomes, better quality of care, and lower health care expenditures, few performance measures account for social complexity.16,17 Instead, the complexity of a patient or a panel of patients is largely determined by the number and type of clinical diagnoses and medical care utilization.18 This is explained in part by data limitations. While commonly used complexity indices accounting for clinical comorbidity can be computed from diagnosis codes,19 information on patients' SDH has traditionally been absent from most clinical and claims datasets.16,17 This is beginning to change and several recent national initiatives have highlighted the need for documenting a set of patient-reported SDH domains in electronic health records (EHRs).20⇓⇓–23 However, there are numerous barriers to integrating SDH screening into clinical settings and ensuring standardized documentation in EHRs.24,25 As a result, despite recent increases in EHR documentation of patient-reported social risks, it will take time before patient-level SDH information is collected consistently across large populations of patients.26,27
Patient addresses, however, are routinely collected in EHRs and can be linked to publicly available datasets (eg, US Census, American Community Survey) that describe the social, economic, and environmental characteristics of the neighborhoods and communities where patients live. These area-level data—which some refer to as “community vital signs” —could be used as a proxy for patient-level SDH information, thus providing a readily available opportunity to explore the impact of social complexity on provider performance metrics.20,28 Using EHR data from 2 large networks of community health centers (CHCs), we worked closely with patient, provider, and health system stakeholders to assess the effect of patient-level clinical complexity and community-level social complexity on diabetes control, a prominent focus of performance measurement initiatives for over a decade.15,29,30 Our objectives were to 1) understand the relative contributions of clinical and social complexity to the probability of poor diabetes control, and 2) evaluate the association between patient clinical and social complexity and provider performance metrics. The results may have considerable implications for policy and payment, especially given increasing calls to shift toward models of value-based care.31
Methods
Design and Setting
This retrospective cohort study utilized EHR data extracted from OCHIN (not an acronym) and Health Choice Network (HCN), 2 large CHC networks serving safety net patients with high levels of social complexity. OCHIN provides a centralized instance of Epic EHR to over 600 CHCs, serving over 2 million patients, across 18 states.32,33 HCN provides a centralized EHR for CHCs serving over 2 million patients across 19 states. Both are members of the National Patient-Centered Clinical Research Network (PCORnet)—OCHIN is part of the Accelerating Data Value Across a National Community Health Center (ADVANCE) network and HCN is part of OneFlorida—and thus have research-ready EHR data formatted using the PCORnet common data model. The study was conducted in partnership between the OCHIN and the University of Florida and was approved by the Western Institutional Review Board and the University of Florida Institutional Review Board.
Study Sample
The study sample included patients aged 18 to 75 years established in primary care clinics in the OCHIN or HCN network in 2015. Established patients were defined as those with an office visit in 2015 to a clinic and a prior visit (any year) to the same clinic. Among established patients, those with at least 1 outpatient encounter diagnosis of diabetes who received hemoglobin A1c (HbA1c) testing in 2015 were included in the analyses. Primary care providers with at least 20 established patients in the OCHIN or HCN network were included in the sample.
Variables
The primary dependent variable was diabetes control status; patients with HbA1c values greater than 9% were considered poorly controlled. The threshold for diabetes control was chosen to align with the 2014 Physician Quality Reporting System Measure Specifications Manual for Claims and Registry Reporting of Individual Measures.34 If a patient had multiple HbA1c tests, the most recent result was used.
Independent variables included measures of patient-level clinical complexity and community-level social complexity. Patient-level clinical complexity was assessed using the Charlson Comorbidity Index (CCI), a validated measure that predicts the risk of mortality and resource utilization for patients with a range of comorbid conditions.35 The CCI has been used extensively to adjust for clinical complexity in health care utilization models.11,36,37 Because mental health conditions are strongly associated with health outcomes38 but were not included in the original CCI, we supplemented the score by adding an ordinal variable summarizing mental and behavioral health complexity (based on ICD-9 and ICD-10 diagnosis codes for depression, substance use disorders, and psychosis/bipolar disorders; see Appendix 1). Patient data from 2015 and all available data in years prior1 were utilized to calculate the CCI.
To assess community-level social complexity, we utilized the social deprivation index score (SDI), a weighted composite measure of 7 indicators (percent living in poverty, percent with less than 12 years of education, percent single parent households, percent living in rented housing unit, percent living in overcrowded housing unit, percent of households without a car, and percent non-employed adults under 65 years of age) developed using factor analysis. The SDI is calculated as a percentile ranking based on all Zip Code Tabulation Areas (ZCTA) and census tracts in the country. It has previously been shown to be associated with population-level health outcomes.39,40 We used a version of the index that utilizes 2011 to 2015 American Community Survey 5-year estimates. Each patient in the sample was assigned an SDI score at the lowest level of geography possible, using available information on patient address. HCN patients were geocoded at the ZCTA-level. OCHIN patients were geocoded at the census tract level where possible, and at the ZCTA level if their census tract could not be determined.
Analysis
All analyses were conducted in parallel by OCHIN and University of Florida (for HCN) analysts, who utilized shared syntax, but did not share patient-level data. Logistic regression models were computed to assess the odds of poor glucose control (HbA1c > 9%). Independent variables included the CCI (categorized as 0 to 1, 2 to 3, 4 to 5, and 6+), an additional variable to assess the number of mental/behavioral health diagnoses (categorized as 0, 1, 1+; see Appendix 1), and SDI score (continuous variable with 1 unit increments equivalent to 10 National percentile ranks). Models were conducted in a stepped-wise fashion, with the first controlling for CCI, the second for CCI and mental/behavioral health diagnoses, and the third for CCI, mental/behavioral health diagnoses, and SDI score. Additional patient covariates were limited to those used by Centers for Medicare and Medicaid Services (CMS) in assessing provider quality metrics, which included age and gender.
Following the CMS procedure for adjustment of Performance Metrics41 for panel clinical complexity, we adjusted quality metrics for providers with more than 20 patients in the outcome. The denominator was determined from the formula: Padj = (Pobs/Pexp) × Ppop where Padj is the percent of a provider's diabetic panel with poor glucose adjusted for model covariates, obtained by multiplying Ppop (the observed percentage in the entire sample) by the ratio of Pobs (the observed panel percentage) to Pexp (the percentage expected based on the providers patient panel). Pexp (the percentage expected based on the providers patient panel) was obtained by summing the predicted probabilities for their patients from the logistic regression model and multiplying the sum by 100.
Results
Sample Cohort Characteristics
Table 1 provides sociodemographic and both social and clinical complexity characteristics for the study sample. The sample cohorts for diabetes outcomes included 63,906 OCHIN patients, 9422 HCN patients, 898 OCHIN providers and 114 HCN providers. As might be expected among CHC patients, both the OCHIN and HCN cohorts had median SDI score that were higher than the national median (79 for OCHIN, 80 for HCN, compared with the national median of 50). The percentages of patients with HbA1c > 9% (21.9% for OCHIN patients and 25.2% for HCN patients) were lower than those reported by the National Committee for Quality Assurance for 2015 (45.4% for Medicaid patients, 27.4% for Medicare Health Maintenance Organization (HMO) patients, and 26.5% for Medicare Preferred Provider Organization (PPO) patients).
Diabetes Cohort Description
Association of Clinical and Social Complexity with Glucose Control in Diabetics
The results of the final model including controls for age, gender, CCI, mental/behavioral health diagnoses, and SDI score are presented in Table 2. As the addition of SDI had little effect on CCI and mental/behavioral health variables, we only reported the results of the final model. Full model results are available in Appendix 2. In the OCHIN cohort, CCI scores of 4 or higher were associated with a greater probability of poor glucose control. Conversely, the number of mental and behavioral health diagnoses were inversely associated with the probability of poor glucose control, such that those with 1 or more than 1 mental/behavioral health condition(s) had significantly lower odds of poor glucose control (see Table 2). In the smaller HCN cohort neither the CCI scores nor the number of mental and behavioral health diagnoses were significantly associated with poor glucose control after adjustment for age, gender, and SDI.
Association of Clinical and Social Complexity with Odds of Poor Glucose Control in Diabetic Patients
Although there was not a significant relationship between CCI and diabetes control across the 2 cohorts, increasing social complexity (as assessed by neighborhood-level SDI score) did have a significant association with higher rates of poor diabetic control in both the OCHIN and HCN cohorts. Even after adjusting for age, gender, CCI, and mental/behavioral health diagnoses, for each 10-point increase in the SDI, the odds of poorly controlled diabetes increased by 5% in the OCHIN cohort and 3% in the HCN cohort.
Association of Clinical and Social Complexity on Provider Quality Metrics
We also evaluated how adjusting primary care providers' patient panels for clinical and social complexity impacted the assessment of provider quality metrics for glucose control. After adjusting for CCI, our estimates suggest that 95.3% of OCHIN providers and 87.7% of HCN providers would improve their performance assessment (ie, the percent of diabetics in poor control) by 1 or more percentage points (Table 3). Adding an adjustment for mental/behavioral health diagnoses to the CCI did not result in additional improvement in provider quality metrics. However, adjusting for SDI in addition to CCI and mental/behavioral health diagnoses improved the assessed rates of glucose control by an additional one percentage point or more for 15.5% of OCHIN providers and 7.1% of HCN providers. Finally, because the median SDI of both safety net cohorts is higher than would be found in the general population, we determined the predicted adjustment for safety net providers if the SDI score for their patient panels was set to the national median of 50. Notably, we found that 75% of providers across the OCHIN and HCN safety net cohorts would expect to see an improvement in their performance assessment of 1 percentage point or more if their panel had an SDI equivalent to the national median (25% would see a 1–2% improvement, 45% would see a 2–5% improvement, and 5% would see more than a 5% improvement in their assessed performance metrics).
Estimated Effect of Adjustment for Clinical and Social Complexity on Provider Quality Metrics
Discussion
To our knowledge, this is the first study to examine the combined impact of social and clinical complexity on the odds of poor diabetes control and to assess the potential implications for provider quality metrics. Our findings suggest that community-level social complexity, as assessed through census or ZCTA-level SDI score, had a significant positive association with the odds of poor glucose control in both the OCHIN and HCN cohorts. Across provider panels, including adjustment for social complexity and clinical complexity improved the assessment of diabetes performance metrics more than adjusting for clinical complexity alone. These findings support existing evidence of the influence of community-level social risk factors on diabetes metrics,6,39,42⇓⇓–45 and are consistent with previous studies that have found an association between provider quality measures and neighborhood-level characteristics of their patient populations.14,46,47
The effect of clinical complexity was less consistent across the 2 cohorts. Within the OCHIN cohort, a CCI of over 4 was associated with greater odds of poor diabetes control. Although a CCI of 6 or more was also associated with greater odds of poor diabetes control, the odd ratio was slightly smaller than for a CCI of 4 to 5. Interestingly, having 1 or more mental/behavioral health diagnoses was associated with reduced odds of poor control. Previous research has found a similar association between higher levels of clinical complexity and better measures of diabetes control.48⇓–50 A potential explanation for this trend is that patients with more comorbidities use health care more frequently and thus may be more closely monitored.51 Given that those with mental/behavioral health conditions are likely to have more frequent touch points with the health care system overall, a similar explanation could also underlie the finding that those with more than 1 mental/behavioral health diagnoses had reduced odds of poor diabetes control relative to those with no mental/behavioral health diagnoses.
Within the HCN cohort, measures of clinical complexity and mental/behavior health diagnoses were not significant. A potential reason for this could be the relatively small sample size (there were only 9422 HCN patients, compared with 63,906 OCHIN patients), which is also reflected in the number of patients in the higher CCI categories (724 for HCN compared with 10,500 for OCHIN). It could also be due to other unobserved differences between these 2 cohorts that are beyond those scope of the present analysis. Future studies should investigate further these inconsistencies and explore the association between clinical and social complexity and diabetes control in additional cohorts of patients.
Our findings have significant clinical, health systems, and policy implications. First, these findings suggest that providers caring for populations with greater social complexity may benefit from having their performance metrics and reimbursements adjusted. As the United States moves increasingly toward value-based purchasing models of reimbursement, further research and refinement of a combined measure of social complexity may contribute to more effective and equitable value-based models that incentivize the care of socially complex patients. The use of value-based models unadjusted for social complexity may financially penalize providers that serve vulnerable populations, ultimately resulting in provider reluctance to serve these patients and fewer care options among those individuals that need the most care. However, even with appropriate adjustment for social complexity, providers caring for socially complex populations may also need additional infrastructure and supports in place to address the needs of the patients they serve. Thus, any social risk adjustment approach should be carefully implemented to ensure that the need for funding to support infrastructure among safety net providers is not minimized.
Perhaps more importantly, these findings reinforce the vital importance of understanding and addressing SDH as a core component of strategies to reduce disparities and improve health outcomes across the US. Indeed, the association between increasing SDI and poorer diabetic control lends further support to the importance of primary care-public health integration efforts to address patient- and neighborhood-level social conditions to improve health care quality and utilization. Moreover, these findings underscore the need for continued patient-level social risk screening both to identify patients with unmet needs and to inform decisions about individual patient care.52
Limitations
Several limitations must be considered when interpreting our results. First, in the absence of robust patient-level SDH data, we utilized aggregate community-level data about where patients live as a measure of social complexity. As more patient-level social risk screening data becomes available, future research should examine the concordance between measures of community- and patient-level social risk factors and seek to understand the appropriate uses of both sources of information.53 Such research could help to disentangle whether community-level data are a valid a proxy for information on patient-reported social risks, as well as enhance our understanding of the relative impact of “place”—or the neighborhood and community environments in which people live—on both individual and population health, independent of individual-level risk factors.54
Second, our study examined already socially vulnerable cohorts, so the effect size of social complexity adjustment may underestimate the true impact of SDH on diabetes control. Future research in this area should expand to include more heterogeneous populations, such as national or nonsafety net samples, to examine the impact of social complexity across a broader spectrum of patients. Similarly, there are limitations of the sample cohort for unbiased assessment of the impact of social complexity on provider performance. The CMS formula multiplies the ratio of observed/expected performance by the sample mean to get the adjusted performance; because our study cohorts have higher degrees of social complexity than the national population, adjusting to the sample mean is adjusting to a mean that is likely higher than a national sample.
Conclusions
In the absence of wide-spread patient-reported social determinants of health data, we used an area-based measure of social deprivation to understand the impact of social complexity on provider performance metrics. Even after controlling for clinical complexity, our findings showed a positive association between higher levels of community-level social deprivation and the odds of poor diabetes control. As payment models shift toward value-based pay, it will be important to adjust for social complexity when assessing differences in provider metric scores.
Appendix
Mental and Behavioral Health Condition Coding
Association of Clinical and Social Complexity with Glucose Control Outcomes in Diabetic Patients: Results of Stepwise Regression Models
Notes
This article was externally peer reviewed.
Conflicting and Competing Interests: MC: Cornell University has filed a patent for the use of the enhanced comorbidity index to predict future costs. No other authors have conflicting or competing interests.
Funding: The research reported in this publication was conducted using the National Patient-Centered Clinical Research Network (PCORnet®). PCORnet® has been developed with funding from the Patient-Centered Outcomes Research Institute (PCORI). The study was funded by PCORI through PCORI Award (HSD-1603-34987).
↵1 Patient data may extend as far back as 10/25/2005, but differs from patient to patient depending upon data availability.
To see this article online, please go to: http://jabfm.org/content/33/4/600.full.
- Received for publication October 11, 2019.
- Revision received February 14, 2020.
- Accepted for publication February 19, 2020.
