How to Measure the Value and ROI of AI Tools in Clinical Care

Allina Health has determined that use of AI scribes reduces clinician burnout and helps clinicians maintain their full-time equivalent performance, which is generating an ROI that pays for the cost of the technology.

In 2026, health systems should make gains on determining the value and ROI of AI tools that impact clinical care and clinicians, according to the chief digital and information officer at Allina Health.

Assessing the value and ROI of AI tools used in clinical care has been elusive for health systems, hospitals, and medical groups. For example, AI scribes have helped clinicians save time on generating clinical notes, but it has been hard to pinpoint the value of a doctor getting home in time for dinner with their family or a doctor not having to spend time on charting in the evening.

Allina Health has been able to determine that having clinicians use an AI scribe pays for the cost of using the technology, according to Dave Ingham, DO, MBA, chief digital and information officer at the health system.

Allina Health has determined that clinicians who use an AI scribe maintain their hours providing direct patient care, which is also known as full-time equivalent (FTE) performance. Clinicians at the health system who do not use an AI scribe tend to have a higher burnout rate, which is associated with lower FTE performance.

"When we first implemented an AI scribe, the doctors and advanced practice providers who were using the tool kept their FTE about level, whereas, the clinicians who were not using the tool saw a significant decrease in FTE," Ingham says. "This is a measure of the number of patient-facing hours the clinicians are performing."

In comparing 2023 and 2024 data, non-AI scribe users year-over-year saw 0.25 fewer patients per day compared to AI scribe users. In other words, AI scribe users remained steady in the number of patients they saw each day, while on average the non-AI scribe users reduced the number of patients they saw each day.

The impact of reduced clinician FTE is "massive," according to Ingham.

"There is a cost for the health system because we are still paying the overhead for a provider when they reduce their FTE, including benefits and insurance costs," Ingham says. "When a clinician reduces their FTE, they generate less revenue. We also run into issues with patient access, which burdens other clinicians or prompts us to hire more clinicians."

Allina Health is making progress in assessing the value and ROI of other AI tools the health system has adopted that impact clinical care, Ingham explains.

"We have leaned into having finance specialists playing a part in the valuation and vetting team for AI tools," Ingham says. "For assessing our AI tools, we have a team that includes finance, human resources staff, and others to make sure that we are crisp on the expected value of AI tools. Once we have implemented an AI tool, we work to make sure that we measure the impact and whether we achieve what we expected."

One of the primary expectations from implementing AI tools in clinical care is the ability to redeploy staff, according to Ingham.

"If we expected to redeploy people after implementation of an AI tool, we measure whether we achieved that goal," Ingham says. "We also monitor staffing over time. For example, we want to see whether we added people back six months after an AI tool was implemented."

Dave Ingham, DO, MBA, is chief digital and information officer at Allina Health. Photo courtesy of Allina Health.

Adoption of AI Tools in Clinical Care

In addition to an AI scribe, Allina Health has adopted several AI tools that impact clinical care and clinicians.

"We break out AI into three categories," Ingham says. "First, there is patient-facing AI, and we are in an exploratory phase in that area. Second, we have AI tools in back-office areas and clinical operations, where we are moving aggressively. Third, there are AI tools in clinical care, where the technology is helping physicians make decisions and helping physicians be more efficient."

The health system has adopted a generative AI tool that is embedded in the EHR. Clinicians can query the AI tool to pull out information from patient charts. Clinicians can also ask this AI model to summarize information in the EHR.

"The generative AI tool in our EHR reads an entire patient's chart," Ingham says. "For example, a primary care physician can ask the generative AI tool when a patient's last colonoscopy was done, which can be a needle-in-the-haystack task if a clinician is looking in the EHR."

Allina Health is using an AI tool developed by Ferrum Health that has algorithms to read radiology scans.

"One algorithm reads all head imaging, including MRIs and CT scans," Ingham says. "These algorithms support radiologists and help them to be more efficient. The AI tools triage scans and flag the scans that are most emergent."

The health system is using an AI tool called GI Genius that helps gastroenterologists perform colonoscopies.

When a patient sees a gastroenterologist for a colonoscopy, the provider goes all the way to the end of the colon, then they slowly come back looking for polyps that they can take out. One of the challenges of performing colonoscopies is that the polyp detection rate tends to deteriorate through the course of the day.

"GI Genius helps gastroenterologists spot polyps, and the providers using this AI tool have polyp detection rates that stay steady throughout the day," Ingham says. "There is also less variation between gastroenterologists for the polyps they find."

In the clinical operations area, Allina Health has a partnership with Qventus, which includes an AI tool that helps conduct pre-surgery screening. Before the adoption of this AI tool, registered nurses would call the patient a couple of days before a surgical procedure to go through a checklist, including whether the patient has any new illness, any new allergies, and new medications. The goal of the screening is to make sure the patient is ready for surgery and there are no surprises on the day of surgery.

"These pre-surgery calls are a repetitive task, and they are not very complex most of the time," Ingham says. "The Qventus AI tool calls the patient, runs through the checklist, and if there are no surprises the AI tool documents the call. If there are surprises or complexities, an RN connects with the patient."