Root of Some Health Disparities May be Buried in Technology

UCSF Health examines its algorithms and more to address health inequities related to patient no-show processes, telehealth, and vaccinations.

As the coronavirus pandemic revealed significant disparities in health equity, executives at the University of California San Francisco turned to an interesting place to address this issue: technology. And what they discovered was fascinating. Bias built into algorithms, predictive models, and processes could be the root of some problems.

As other organizations seek to address this issue, "Plan to evaluate all healthcare initiatives or tools used to drive healthcare with the equity lens," said Sara Murray, associate chief medical information officer at UCSF Health.

At The Quest for Health Equity, a recent virtual event organized by the health IT association, Workgroup for Electronic Data Interchange (WEDI), Murray described how work that began before the pandemic to evaluate equity at UCSF Health was able to influence the healthcare organization's response to COVID-19.

Two initiatives underway at the pandemic's start helped to set the stage for UCSF Health's equity push. One reached fruition just as the pandemic started, and the other took shape last April.

In January 2020, Murray, along with co-authors Robert M. Wachter, MD and Russell J. Cucina, MD, ABIM, MS, published "Discrimination By Artificial Intelligence In A Commercial Electronic Health Record – A Case Study" in the blog for Health Affairs.

Predictive Algorithms Can Get Blamed for Ensuing Discrimination

Predictive models can be guilty of explicit discrimination when personal characteristics such as ethnicity, financial class, religion, and body mass index are used to predict which patients are more likely to be no-shows for health-related appointments.

Certainly, no-shows are a major source of waste in U.S. healthcare. "We lose about a quarter of a million appointment slots annually, with really profound revenue impacts," Murray told the WEDI audience.

"If used for overbooking, [these characteristics] could result in healthcare resources being systematically diverted from individuals who are already marginalized," the blog stated.

To minimize the financial impact of no-shows, UCSF Health had been considering using an algorithm provided by its EHR vendor, Epic, to overbook appointments on a random basis, using some of the personal characteristics mentioned above, as well as other variables.

Yet overbooking also can lead to problems. "Say both patients show up," Murray said. "Now they're slotted into the same usually short time slot, and they're seen by an overworked and rushed provider. One might argue the appointment quality could decrease."

So UCSF Health decided to build an overbooking algorithm to simply include features of the appointment itself, such as time of day, day of week, and average lead time, as well as the patient's prior history of no-shows.

"By excluding all these sensitive features [such as ethnicity, financial class, religion, and body mass index], we were able to improve the model and basically perform identical to Epic," Murray said.

UCSF Health's data science team "has spent a fair amount of time evaluating commercial tools such as predictive models or artificial intelligence for trustworthiness," Murray said.

UCSF Prioritizes Patient-Positive Interventions

Ultimately, UCSF Health decided it would be best to use the no-show algorithm only for patient-positive interventions—"things we could do to help the patient make it to the appointment—reminders, outreach, sending them a Lyft," Murray said. "We weren't going to use it for booking. We piloted this in 12 departments, and had a modest improvement [a mean reduction of 9%] in no-shows."

Since this work took place, Epic has also made improvements in their overbooking model as well, Murray noted.

"We are not arguing that demographic features should never be represented in a model, as they can be critical predictors of health and access to healthcare," states the Health Affairs blog post. "The question is whether it is tolerable for demographic bias to be represented in a model, not just explicitly (as in the race/ethnicity input) but also implicitly (as in the prior no-show input), if that model may lead to action that negatively affects an individual patient."

Overbooking intervention "risks withdrawing resources from vulnerable patients" and is "ethically problematic," the blog post states. Instead, it recommends "patient-positive" interventions that may increase the likelihood a patient keeps a scheduled appointment, such as flexible appointment times, telehealth visits, or even assistance with transportation or childcare.

As Telehealth Use Surges, UCSF Health Addresses More Equity Issues

Speaking of telehealth, UCSF Health has also measured equity of telehealth utilization. In April 2020, the health system, like many others, saw a dramatic overall increase in telehealth utilization—a 16-fold increase in UCSF Health's case.

"We built health equity analytics in a dashboard that unfortunately illustrates disparities that we were concerned might exist," Murray said. "Telehealth is not being equitably used among non-English speaking patients, or among black patients in comparison with white patients, and there are some disparities related to age as well."

UCSF Health is taking action on these telehealth disparities. "This data is driving action," Murray said. "Our organization has said this is a key area of focus. And work is underway to establish the best interventions to start to close some of these access disparities."

COVID-19 Vaccination Process Highlights Additional Work Underway to Reduce Disparities

The current push to administer COVID-19 vaccinations is yet another focus currently at UCSF Health. Using their own employees as an initial group to study, "we noticed that disparities existed at every step of the process," said Sana Sweis, MS, director of analytics strategy at UCSF Health.

The largest disparity was gaps in rates of individuals scheduling their initial vaccination appointments, with persons of American Indian, Alaskan native, and Black ancestry trailing the Caucasian group.

A related disparity during vaccinations: individuals who failed to register for a patient portal account because they did not own a computer, or lacked necessary technical knowledge to use it properly, Sweis said.

"We targeted interventions that included phone call outreach, with a targeted focus, as well as education and assistance," she said. "In addition, we've set up walk-in clinics where patients can still come in to get vaccinated even if they do not have a patient portal account."

At the 60-day mark in the immunization process, UCSF Health executives noticed an increase in scheduling rates among all race ethnicity groups, but the largest increase was among American Indian, Alaska native, and Black groups.

"This illustrates the impact of the interventions on the initial process," Sweis said.

EU Uses Regulatory Oversight to Address Inequities

In places beyond the United States, regulatory oversight is being considered as a solution to address inequities. For example, the European Union has laid down seven requirements for lawful, ethical, and robust trustworthy AI. These requirements are:

• Human agency and oversight
• Technical robustness and safety
• Privacy and data governance
• Transparency
• Diversity, non-discrimination, and fairness
• Societal and environmental well-being
• Accountability

Because much of this technology is not subject to regulation in this country, Murray said it is important for U.S. providers to evaluate technology for bias right now.