Skip to main content

Patient Safety Indicators Ripe for Improvement

News  |  By Tinker Ready  
   January 05, 2017

Several research teams have raised issues about the limitations of claims data to measure PSIs. Now changes are in the works.

New data promises to add fuel to the ongoing discussion over the accuracy of billing data used to calculate patient safety indicators (PSI).

A group of Ohio State University researchers has found that hospital cases "flagged with a clinically validated PSIs" are linked to greater length of stay and higher rates of 30-day unplanned readmissions, and mortality.

The study, published in the American Journal of Medical Quality, "demonstrates a strong association between clinically validated PSIs and patient outcomes."

The Ohio State researchers note that by going beyond claims data to look at patients' medical records, they controlled for "the poor validity of the current PSI algorithm."

Several research teams have raised issues about the limitations of claims data to measure PSIs. And changes are in the works.

Some of the PSIs have been refined since the Ohio State study as part of an ongoing analysis of safety measures by the federal Agency of Healthcare Research and Quality. And, an upcoming report from the National Quality Forum, which works with AHRQ on refining measures, calls for a closer look at the use of claims, or administrative data, to measure PSIs.

Darrell Gray, an OSU gastroenterologist and lead author of the study, said his team saw a gap in the literature linking validated PSIs to other key outcomes, such as readmissions. Their recent research looked at outcome for 1,874 validated cases flagged for PSIs.

The researchers found that patients flagged with validated PSIs were more likely to be readmitted. For example, 22.2% of patient flagged for pressure ulcers were readmitted, compared to 19.8 of those not flagged.


Readmissions Penalties Work. Here's Proof.


The team included OSU researcher Jennifer Hefner and built off her earlier research at Ohio State. Published in August under the title, "Navigating the Ocean with a Broken Compass," the findings involved a retrospective analysis of all flagged PSI for the year 2014.

Hundreds of medical charts were compared to the administrative data to measure the accuracy of the PSI designation. The August study concluded that improvements are needed in the quality of data and algorithms data being used to identify the PSIs.

The PSIs were developed to be a barometer of how well a hospital was doing with preventable complications, said Gray. Now, the role of PSI has expanded to being tied to the assessment of a hospitals performance.

"It's important that we know truly what they are measuring and that what we are measuring truly represents the performance and quality of the hospitals that [are] reporting it," he said.

PSIs Matter to Patients, Too
He said readmission rates, mortality and length of stays are issues that "not only clinicians or hospital administrators care about. These are things patients care about. They want to know how likely they are to go into a hospital one day, be discharged and have to come back again. Or, how long am I going to have to stay for a particular diagnosis? It's on everybody's mind."

Researchers at John Hopkins Medicine have also challenged the accuracy of PSIs and measures for hospital acquired conditions. In a recent meta-analysis, they argued that PSIs should be valid in at least 80% of cases. They found that one out of 21 measures they looked at met that 80% validity threshold. Most other PSIs were valid in between 60%–80% percent of cases.

Lead author Bradford Winters, MD, said he doesn't think PSIs are faulty. "But, we need to optimize how we measure them. I don't think we are quite there yet," he said.

Still, Winters and his team believe that the measures should not be used for programs that levy penalties on providers with low PSI scores until a "transparent" review is held on the accuracy of the measures.

Since Winters' team completed its research, the National Quality Forum has been moving in the direction of addressing some of the PSI shortfalls. NQF has been engaged in a "very vigorous discussion" of the issues, he said.

In a draft of its upcoming annual "Patient Safety 2016" report, the National Quality Forum notes the need for an effort to improve the accuracy of administrative data: "The accuracy of the data used to calculate a measure is a primary consideration when determining its validity. This can be of particular concern when measures are specified using administrative data which were not originally collected to assess quality."

The report endorses measure that may "focus efforts on improving the quality of claims data, and in doing so, increase the validity of measures across the NQF portfolio."

Tinker Ready is a contributing writer at HealthLeaders Media.


Get the latest on healthcare leadership in your inbox.