The public has been led to believe that hospital quality measures on CMS’s Hospital Compare website offer a simple but reliable way to compare the quality of care offered by different hospitals, but recent research casts doubt on that reliability.

Hospital readmission rates in particular have gained substantial attention from policymakers and healthcare providers because of their high frequency and significant costs. Researchers at the Smith Center for Outcomes Research in Cardiology at Beth Israel Deaconess Medical Center (BIDMC) in Boston say condition-specific readmissions measures may not accurately or fairly reflect hospital quality.

The study found significant differences in hospitals’ performance when readmissions were assessed for non-Medicare patients and for conditions other than those currently reported. When these additional factors are taken into account, half of hospitals would be subject to a change in their financial penalty status, the researchers say.

“Medicare metrics alone may not be the final word on hospital quality for readmissions,” senior author Robert W. Yeh, MD, MSc, director of the Smith Center for Outcomes Research in Cardiology at BIDMC, said in a statement announcing the study results.

“As it currently exists, the Medicare public reporting system offers an incomplete picture,” Yeh said.

“Significant attention has been given to hospitals’ overall performance as determined by the public reporting of a small number of specific conditions and patient populations,” he added. “It’s a little bit like issuing a final grade based on a few homework assignments and not a full semester’s worth of work.”

Excess readmission ratios, which examine hospital readmissions for heart failure, heart attack, and pneumonia among Medicare beneficiaries, are used to assess quality and determine hospital payments. Yeh and his colleagues wanted to know if hospitals that reported high readmission rates for fee-for-service Medicare patients also would have high readmission rates for patients with other payers, including Medicaid and private insurance.

They also wanted to find out if hospitals with high readmission rates for the three specified conditions — heart failure, pneumonia, and heart attack — had similarly high readmission rates for other unreported conditions, says first author Neel Butala, MD, MBA, an investigator at the Smith Center at BIDMC and cardiology fellow at Massachusetts General Hospital.

The researchers reviewed data from the Healthcare Cost and Utilization Project’s Nationwide Readmission Database, focusing on more than 2,100 hospital observations in 2013 and 2014 to investigate whether 30-day readmission measures for heart failure, myocardial infarction, and pneumonia among Medicare patients reflect hospital performance on readmissions more broadly in an all-payer national sample.

They found that 29% of hospitals currently being penalized for readmissions would no longer incur a penalty if unreported conditions were used as the basis of the calculations.

The difference was even greater when examining non-Medicare readmission rates: 40% of penalized hospitals would no longer be issued penalties if performance was based on readmission rates for non-Medicare patients hospitalized for heart failure, pneumonia, or heart attack.

“This tells us that similar hospital or patient characteristics may influence readmissions more than similar disease conditions and suggests that efforts to prevent readmissions may be more successful by targeting hospitalwide processes rather than condition-specific processes,” Butala says.

An abstract of the study is available online at: https://bit.ly/2J2lKHp.