Hospital Stars in CMS Ratings Aren’t Necessarily the Best

December 5th, 2016

WASHINGTON, DC — “Star” ratings released earlier this year by the Centers for Medicare & Medicaid Services (CMS) are confusing and potentially misleading, according to a blog published by the journal Health Affairs.

Under the new overall star rating system implemented in July 2016, almost all hospitals are now rated up to five stars, depending on performance on a series of quality metrics. The goal, CMS says, is to “help patients and families learn about hospital quality, compare facilities side by side, and ask important questions about care quality.”

It does not appear to have that result, however.

“Many well-known hospitals that are highly rated by other ranking systems are absent from the list of 102 hospitals that received five stars. Many names on the list may not be familiar. Do these five-star hospitals represent the nation’s best in terms of quality?” asked Atul Grover, MD, executive vice president of the Association of American Medical Colleges, and Susan Xu, an AAMC research associate.

Grover and Xu maintained, “A closer look at the quality performance of five-star hospitals brings into question the CMS results.”

While the star rating methodology intends to rate all hospitals based on 64 quality measures from seven quality domains, according to the review, CMS has assigned 66% of the overall star rating to three domains: mortality, readmission, and patient safety. Less weight is given to domains related to processes, such as efficient use of medical imaging or timeliness of care.

“Nonetheless, CMS calculated and published star ratings for hospitals that had sufficient data to report on as few as three quality domains, including some hospitals that only had data from one clinical outcome domain,” the authors pointed out. “The fewer the clinical outcome domains a hospital reports, the less that hospital’s overall star rating is actually tied to performance on patient outcomes.”

In fact, Grover and Xu noted, 40% of the 102 hospitals receiving a five-star rating did not have the minimum data necessary to report on either mortality or readmissions.

“Of those, 20 performed at only the national average on patient safety. Are all the shining stars here an accurate representation of quality?” the authors inquired, adding, “This inconsistent value matrix leads to a wide performance divide among five-star hospitals.”

They explained that, when a hospital has insufficient data to report on one or more quality domains, the “weights” of those missing domains are reallocated to the domains for which there is sufficient data. In other words, Grover and Xu stated, for a hospital with performance at or above the national average on just three available domains, those ratings might be projected onto the four domains with missing values. That potentially gives hospitals reporting only a few domains an advantage “by allowing it to earn a five-star rating while appearing no different than other hospitals that reported all seven quality domains.”

At a disadvantage, according to the blog post, were major teaching hospitals — a group that includes many of the nation’s “most renowned” hospitals. Because they provide comprehensive services, 80% of major teaching hospitals reported on all seven quality domains.

“To receive ratings with more stars, these hospitals have to meet a higher standard than hospitals with fewer reported quality domains because of their narrower service areas and less diverse patient populations,” Grover and Xu wrote. “Not only do major teaching hospitals need to achieve performance better than the national average in more quality domains, but their overall star ratings will also be heavily tied to the outcomes of their clinical services (e.g., mortality) instead of processes of care delivery (e.g., efficient use of medical imaging). While all improvement efforts can be challenging, we believe that it takes more to improve clinical outcomes — for example, saving a patient’s life — than to improve delivery process, such as reducing the use of imaging.”

The authors concluded, “To provide meaningful information for patients, families, and caregivers about hospital quality, a star ratings system has to make sense. At a minimum, quality performance among hospitals with the same star rating should be consistent. And higher star ratings should reflect better actual quality performance. Unfortunately, the CMS star ratings in their current form fail to meet this basic test and will do more harm than good to patients.”