We should have sympathy for embattled healthcare providers who must meet certain quality measures to comply with Meaningful Use and other incentive payment rules. They, and the entities that were supposed to vet the rules on quality measures to reflect accurate, useable electronic health records, have encountered some head-spinning problems.
It's got to be a tough job, like herding cats, to get consensus in a time of rapid, massive health system change.
But there have been delays and errors, "rampant inconsistencies" in results, large development cost overruns, and a highly critical 76-page Government Accountability Office report. Now, many frustrated providers are asking whether the measures, and all the work they have to do to collect them, will really improve quality.
The problem, in a nutshell, is that many of these measures, such as those evaluating stroke care or prevention of thromboembolism, were endorsed in settings that used hand-written notes in medical records. Now those notes must be adapted for use by EHR systems. The process is complicated and technical. And in understatement, the transition has been bumpy.
Think of it as trying to print a photo of a friend you took with a first-generation low-res digital camera. Your friend is barely recognizable, awash in large boxy pixels. According to many providers' remarks in recent weeks, the true clinical picture —for example, did or didn't the patient get a particular drug, and at what dose?— is lost in translation.
In an interview, Nancy Foster, the American Hospital Association's quality and patient safety vice president, explains the problem in capturing, for example, whether a patient received a pneumococcal vaccine:
"It seems that it would be easy. But while pharmacy data is usually captured electronically (the vaccine was provided), nursing notes are often not (the vaccine was given); they're in text. So you have to search through the text...'I gave Mrs. Jones her pneumococcal vaccine today at 3:45,' which is written out in different ways by different nurses. Imagine the difficulty of doing that in medicalese, the way nurses and doctors write things in charts. And there are medical issues."
Here's a synopsis of what's happened.
Hospitals Unhappy
On January 4, Janet Corrigan, president of the National Quality Forum, received a sharp letter from the AHA's Foster. The NQF has a $10 million annual contract to assure that measures used for EHR and other incentive payments can adapt to an electronic format.
The AHA "strongly supports the move toward automated quality reporting to ease burden and provide real-time information," Foster began.
"To be useable, however, automated quality measurement must be feasible, generate valid and reliable results, and have benefits that outweigh costs," Foster wrote. "Early experience from the hospital field in implementing Stage 1 of meaningful use calls into question whether e-measures will deliver on that promise. NQF has a key role to play in ensuring that they do."
Unfortunately, Foster wrote, these e-specifications never underwent appropriate field or feasibility review. A GAO report subsequently concurred.
Thus, "these measures contain known errors," Foster wrote, adding that pilot testing, specifically quality measures for stroke and venous thromboembolism, "was never conducted."
Yet, "these specifications were finalized by CMS and vendors embedded them into their products using the e-specifications provided."
GAO Finds Serious Problems
In mid-January, the GAO report took both the NQF and the Department of Health and Human Services to task for delays, cost overruns, and for failing to complete critically important endorsement work needed to write incentive payment programs, including the value-based purchasing and EHR Meaningful Use rules.
In an interview with HealthLeaders Media last week, the GAO report's author, Linda Kohn, characterized the problems the agency found as "serious" ones that threaten the credibility of measure based incentive payment programs.
"If there's any question about the measures underneath these programs, these programs themselves will be questioned. The measures have to be sound and workable. That's what the NQF does," she says.
Her report was specific. "For more than half of the projects, NQF did not meet or did not expect to meet the initial time frames approved by HHS."
For example, HHS used 44 measures NQF had converted for the EHR incentive program, but the measures required corrections. "HHS officials also have not tested the retooled measures to assess the feasibility of implementing them in the electronic format; therefore, HHS runs the risk that some of these measures may not work as intended when implemented," Kohn's report said.
"HHS may be unable to ensure that the agency receives the quality measures needed to meet (Affordable Care Act) requirements, including time frames for implementing quality measurement programs," the agency said.
NQF responded to a request for comment saying the subject is "complex terrain," but that they've taken steps to endorse and approve e-specifications. "We agree with AHA that this is a learning process— with so much evolving at once— so we must tread deliberately, but carefully, into implementation."
"NQF agrees with AHA on the need for testing of eMeasures. Testing by measure developers is a critical step to ensure the measures can produce the intended result running on an electronic platform."
Hospitals Question Usability, Point to Inconsistencies
Unfortunately, the problems persist. In a 68-page letter to the Centers for Medicare & Medicaid Services April 30, AHA executive vice president Rick Pollack pleaded for a delay in proposed rules for the Meaningful Use Stage 2 incentive payments because of all these problems.
"'We have heard from many members hospitals/CAHs (critical access hospitals) that all stakeholders – measure developers, the National Quality Forum, CMS, vendors and providers – need much more time to get the process right."
His letter made the following points:
- Hospitals have "experienced many difficulties in implementing automated quality reporting for Stage 1. Despite making tremendous investments to purchase technology and change workflows to input the needed data into the EHR they find that the resulting data are not useable."
- Vendors have needed to program many decisions into EHRs that were not included in the e-specifications" resulting in "rampant inconsistencies between different vendors" as well as inconsistent measure results" when the measures are compared.
- The measures' e-specifications "contain known errors and have never been field tested."
There's more, but you get the gist. I think that now, CMS and the NQF do too.
At the AHA's annual meeting in Washington, D.C. this week, CMS acting administrator Marilyn Tavenner addressed the crowd and promised to be flexible:
"We (CMS) obviously have a lot of work to do around meaningful use next stage," she said. "You have my commitment to be flexible, to listen, and try to move at a pace that works for you."
It surprises me that after so many smart people worked so long and spent so much time and money to get this process down, we've reached this mind-boggling point. But for anyone who works in Washington or in healthcare, it may not be surprising at all.