A Randomized Trial to Train Vulnerable Primary Care Patients to Use a Patient Portal

Study Setting

The San Francisco Health Network provides primary care to over 63,000 patients/year as the city's only public health care delivery system. The majority of patients are nonwhite and on Medicaid/uninsured.²³ Launched in January 2015, the online patient portal (called MYSFHEALTH) allows patients to access their visit summaries, lab results, and health education materials online. To register, patients initiate signup in person at their clinic and receive an email to activate their account on their own. At the time of the study, 35% of primary care patients had initiated signup for MYSFHEALTH.

MYSFHEALTH Training Content

Informed by documented usability barriers to portal use in our setting,¹⁷ we worked in close consultation with the library at Zuckerberg San Francisco General Hospital as well as with 2 advisory boards to iterate both the content and format of our training curriculum. Our patient advisory board was comprised of 10 members who were primary care patients or caregivers, and our project advisory board included 9 experts from health technology research, adult learning and literacy, and clinical administration/operations. Over the course of 4 in-person meetings, we created a final portal training curriculum with simple instructions and 11 how-to videos for accessing MYSFHEALTH (patient story, getting started, signing up, signing in, creating a username, creating a password, accessing the homepage, accessing a visit summary, reviewing lab results, using the online health library, and message from a health care provider—all videos available on request). Participants could watch all videos sequentially, or skip to relevant topics as preferred. Our iterations simplified the content to better match the digital and health literacy levels of the population. In particular, these videos used audio (with captions) for explaining the portal functionality as well as screenshots of how to access each feature. When completed, we put the training content onto an online learning platform, LearnerWeb,²⁴ which allowed us to track individual access of the videos during the study. At the conclusion of the study, we moved all video content to the Network's Web site for public access.

Trial Recruitment

From June to October 2016, we recruited participants from 2 primary care clinics, 1 based on the campus of Zuckerberg San Francisco General Hospital and 1 community-based clinic. Through an electronic query of the electronic health record (EHR), we generated a list of patients who had clinic visits before the study (April to July 2016) meeting the following criteria: 1) English speaking (as the portal was only available in English in our setting), 2) age 18 years or older, and 3) diagnosed with a chronic condition (as previous studies have shown that portal use is highest among those managing a chronic illness²⁵). Providers reviewed the lists and excluded individuals with cognitive or visual impairment, severe mental health conditions, or other barriers to enrollment. Through phone screening, we further excluded individuals without email addresses (as this was necessary for portal registration) and those who already self reported using MYSFHEALTH.

Randomization

We randomized patients during an in-person session to receive: 1) an in-person tutorial with a trained research assistant versus 2) a link to access the online tutorial on their own. During this in-person enrollment session, all participants received an informational pamphlet that was disseminated within the general clinic population outlining key features of the patient portal, and were guided through the steps of signing up for a LearnerWeb account to access the training materials. For participants randomized to the in-person training arm, a trained research assistant prompted participants to log into the learning platform and guided them in accessing the training materials. The staff member provided further explanation or clarification if participants had questions about the training material. Participants in the take-home arm were given an article handout with a link to the training materials and an outline of the steps for accessing the training curriculum. The research assistants delivering the intervention were blinded to the randomization allocation until after the consent process was complete.

The trial is registered at clinicaltrials.gov as NCT03354000.

Access of Training Curriculum

Using LearnerWeb, we measured whether participants accessed the training videos, categorized as yes/no and number of videos watched (from 0 to 11).

Portal Signup and Use

Portal outcomes were assessed via EHR chart review as 1) initiating signup for portal access (yes/no), and 2) logging into the portal after the sign-up process was complete (yes/no and total number of logins). Our primary outcome was the binary assessment of portal log-on in the follow-up period of 3 to 6 months post-training (mean, 112 days; range, 82 to 192 days), with an estimated sample size of 100 to detect detect a 25% difference in portal use (10% vs 35%) with 80% power. Because portal enrollment is tied to an in-person sign-up process, we examined portal use data after the participant's next visit whenever possible, using 3 to 6 months as a time frame in which most patients with chronic diseases were generally scheduled for a follow-up appointment.

Clinic Comparison Group

To obtain comparison outcomes in a third usual care cohort, we performed an additional, nonrandomized EHR data pull of all patients who had visited the 2 primary care clinics during the recruitment time period (April to July 2016) and followed them through the same follow-up period (July to December 2016). We used the same portal use outcome ascertainment in this group.

Survey Measures

We also collected patient survey measures at baseline (in-person) and follow-up (via phone). Participants could receive in-person support from the research assistant at any time to complete their written baseline survey. Demographics included age, gender, race/ethnicity, and highest education completed. We assessed health literacy status using a screener about confidence in filling out medical forms independently.²⁶ We categorized participants noting any lack of confidence filling out medical forms as having limited health literacy based on previous work.^10,11,17,27 We asked participants how well they spoke English and categorized less than “very well” as limited English proficiency. Participants also reported how often they used the Internet/email (daily, weekly, or less), as well as their specific chronic condition(s).

To assess portal interest/attitudes and skills, we also asked a series of self-reported items at both baseline and followup. Interest and attitude items included 1) usefulness of Internet for making health decisions (5-point Likert from not at all to very useful), 2) importance of getting medical information electronically (3-point Likert from not at all to very important),²⁸ 3) confidence in safeguards for online medical records (3-point Likert from not at all to very confident),²⁸ 4) interest in using the MYSFHEALTH portal Web site to see their medical record (5-point Likert from no interest to high interest), and 5) interest in using specific potential portal features (4-point Likerts from not at all to very interested²⁸). Next, we assessed their self-reported skills by asking about 1) self-rated skills to use a Web site to manage health care (5-point Likert strongly disagree to strongly agree), 2) confidence in logging into MYSFHEALTH without help (scored 1 to 10), 3) confidence in using MYSFHEALTH to improve their health (scored 1 to 10), and 4) self-reported eHealth literacy using 4 items of a validated scale (such as “I know how to use the Internet to answer my health questions”²⁹).

Next, we assessed patient perceptions of their health care that we hypothesized could be impacted by portal use, either because of increased convenience or access to specific health care information. This included the Patient Assessment of Chronic Illness Care³⁰ and a modified version of the Self-Efficacy for Managing Chronic Disease Scale³¹ (adjusted from 11th to 4th grade Flesch-Kincaid readability level).

The follow-up surveys also included an open-ended question about the reason(s) for portal nonuse among those without any log-ins documented in the EHR.

Participants received $25 gift cards to a major retailer for completing the baseline and follow-up portions of the study.

Analysis

We summarized participant baseline characteristics by trial arm. Next, we examined the portal use outcomes using χ² tests by study arm. We completed intent-to-treat analyses among all participants, as well as a per-protocol examination comparing participants exposed versus not exposed to the online training materials.

Because there were no statistical differences or consistent patterns by study arm, we collapsed all participants for the remaining secondary statistical analyses. We first compared the overall portal sign-up and use in the trial to the total rates in usual care during the same time period, using 2-sample tests of proportions. We also examined changes over time in self-reported survey measures, using paired t-tests for continuous variables and McNemar's test for categorical assessments. In addition, we looked at unadjusted differences in portal use in the follow-up period by participant characteristics using χ² tests. We performed all analyses using Stata 14.2 (College Station, TX).

Finally, to analyze participant-reported reasons for portal nonuse at follow-up, 2 members of the analytic team categorized the reasons into distinct categories, meeting in person to establish consensus.