Blum says the uncertainty around ROI and appropriate metrics in AI provides a good reason for a slow, incremental approach to implementation.
"For instance, with a collapsed lung, you can do fairly straightforward measurements before and after the use of the algorithm to see both how accurate the algorithm is in its recognition, how many times it is finding things more quickly, and how many times it is alerting providers to true findings versus false positives or false negatives," he says. "Then, you can look at how much sooner on average those findings are getting communicated to the providers and treatments are getting put in place than what the historical data looks like. There is a fair amount of historical data from emergency departments and ICUs on how long it takes x-rays to be interpreted after they're shot. Those are more straightforward examples."
Obviously, improving the speed and accuracy of diagnostics is critical in the care of every individual patient. When AI can extrapolate those findings to a patient population, the potential for improving care outcomes and costs savings becomes enormous. Blum says that day is not yet here, but within three years or sooner.
"When you look at improving diagnostic accuracy, complex decision-making for populations is going to require more sophisticated and larger looks at outcomes and process measures along the way in order to determine how accurate they are," he says.
Because providers will want a better understanding of ROI before they invest in these algorithms, Blum says vendors will have to do a better job providing evidence supporting their claims.
"Each time someone comes out with a new algorithm, it will have to come with ‘Here's how it was validated,' so you're sure it works. That validation will show that not only does it work well, but here is the measure that shows how much it improved care from how things were done previously," he says. "The Food and Drug Administration is very interested in this. They will be playing a role in how these algorithms are developed, how they are validated, and when they pass a threshold that they need to be regulated or don't need to be regulated, and when they do pass that threshold how they are going to be validated and can demonstrate that they're effective in improving care."
John Commins is a content specialist and online news editor for HealthLeaders, a Simplify Compliance brand.