Relias Media - Continuing Medical Education Publishing

The trusted source for

healthcare information and

CONTINUING EDUCATION.

  • Sign In
  • Sign Out
  • MyAHC
    • Home
      • Home
      • Newsletters
      • Blogs
      • Archives
      • CME/CE Map
      • Shop
    • Emergency
      • All Products
      • Publications
      • Study Guides
      • Webinars
      • Group Sales
    • Hospital
      • All Products
      • Publications
      • Study Guides
      • Webinars
      • Group Sales
    • Clinical
      • All Products
      • Publications
      • Study Guides
      • Webinars
      • Group Sales
    • All Access
      • Subscribe Now
      • My Subscription
    • My Account
      • My Subscriptions
      • My Content
      • My Orders
      • My CME/CE
      • My Transcript
    Home » Public quality reporting: a plea for consistency

    Public quality reporting: a plea for consistency

    May 1, 2014
    No Comments
    Reprints
    Facebook Twitter Linkedin Share Share

    Related Articles

    Study: Public reporting gaining 'long overdue respect' for quality managers

    Study: Public’s interest in quality data is growing

    Group aims for consistency in quality measurement

    Related Products

    How to Improve the Quality of Case Management Departments through Staffing, Part 2

    Watch Out for False Claims Act Violations

    Evidence of economic burden of disparate care for minorities continues to grow

    Keywords

    quality management

    Hospital Management

    Public quality reporting: a plea for consistency

    AAMC sets out guiding principles

    Every few months, another big headline is splashed across the mainstream media, touting the top 100 hospitals, or the best cancer doctors, or your city’s number-one neurologist. But what is missing from these stories is the fact that often, a facility or physician can rank in different places on different lists — fabulous on one, middling on another.

    Why? Because they don’t use the same metrics or data sets. Even when they do, the outcomes may still differ.

    One famous study in Health Affairs in 2008 compared five websites that reported local hospital rankings of five diagnoses.1 "The sites assessed different measures of structure, process, and outcomes and did not use consistent patient definitions or reporting periods," the report noted. "Consequently, they failed to agree on hospital rankings within any diagnosis, even when using the same metric (such as mortality). In their current state, rating services appear likely to confuse, rather than inform, consumers."

    Using the data

    Certainly what is out there isn’t being used, or at least isn’t being used to change minds. One study, released online in March, found that the Centers for Medicare & Medicaid Services’ (CMS) own comparative data on Hospital Compare didn’t have robust enough data to show enough difference to sway consumers’ opinion of hospitals.2

    And a Cochrane review of multiple public quality reporting programs in 2011 found no evidence that these programs changed consumer behavior, improved care, or changed provider behavior3 — all three things which proponents of these programs say they think would be key benefits of giving the public access to key quality metrics.

    It is not that the wider healthcare community doesn’t think that transparency is a good idea, says Jennifer Faerberg, MHSA, the director of clinical transformation at the Association of American Medical Colleges (AAMC). The issue is whether what is out there is valid, and whether it meets the needs of the public by educating them, or whether it just makes things more muddled by presenting incomplete, irrelevant, or incorrect information.

    "There has been a lot of variability and confusion across different reports," she says. As a result, members were concerned, so the AAMC pulled together an expert panel to come up with a framework by which the public and providers and even hospital board members can evaluate the various report cards, lists, and grading sites out there.

    They came up with three domain areas — the purpose of the report, the transparency of the process by which the data were compiled, and the validity of the measures used. Under each of those domains is a list of qualities to be met.

    Start with the purpose of the report. If it is supposed to be about patient safety, then all of the metrics used should relate closely to patient safety or be an accepted proxy. If a measure is something just tangentially linked to patient safety and not supported by evidence, then the measure selection is not supporting the stated purpose, Faerberg says. The purpose of the report should shine through in every metric chosen.

    In the measure area, the AAMC principles state that all measures should be endorsed by the National Quality Forum (NQF) or an equally respected body. "This allows for rigorous review of the measure and methodology," she says. "Without that, you can’t be sure that it is an appropriate measure for quality. If a site is using NQF-endorsed measures, it says something. If it isn’t, that is a red flag that says this measure may not be a good indicator of quality of care."

    The importance of transparency

    Transparency is also important. An organization providing a list of the best hospitals should be willing to share the documentation of how they came up with their scores, Faerberg notes. Being able to replicate scores is vital to scientific validity. If you want the physicians — scientists all — to trust what you say about them, you have to be willing to let your data stand the test of scientific rigor.

    Risk adjustment falls under the transparency and validity domain. "What is in place now is pretty minimal for what is available for outcomes-based measures," she says. "The NQF has a report out for comment now that includes changes to guidelines on risk adjustment that will include socio-demographic factors. Getting a robust and transparent risk adjustment is critical because each patient is unique, with their own set of characteristics."

    "These principles represent the ideal," she says. "No one is there yet. But healthcare is very complex, and to distill it down to a letter grade or numeric rating is very difficult. It doesn’t present a full picture and by its very nature must limit the information it can convey. This tool can help users to evaluate what they are seeing on those sites."

    Engagement

    In addition, Faerberg says she hopes it will lead to engagement with those who create the lists and reports. Some have already been in contact with the AAMC and have expressed a willingness to make changes along the lines suggested by the expert panel — something she finds very gratifying.

    "We support the idea of transparency," Faerberg says. "The issue we have had is that there is no standardization. We want to be sure that sites are using the valid and tested metrics, so that you won’t have a situation where you look on one site and you’re wonderful and another and you’re terrible."

    Every question answered seems to spawn two new ones. What Faerberg does know is that the destination is somewhere far down the road. And she thinks there will be other changes in the next few years. Some of them will be spurred by the AAMC Guiding Principles.

    "I feel encouraged that some of the reporters of this information are willing to engage with stakeholders, including providers," she says. "I don’t know what a report will look like in five years. But there is a willingness by some to get us all to a better place."

    For more information on this topic, contact Jennifer Faerberg, MHSA, Director of Clinical Transformation, Association of American Medical Colleges, Washington, DC. Email: jfaerberg@aamc.org.

    References

    1. Rothberg MB, Morsi E, Benjamin EM, Pekow PS, Lindenauer PK. Choosing The Best Hospital: The Limitations Of Public Quality Reporting. Health Aff November 2008 vol. 27 no. 6 1680-1687
    2. Safavi KC, Dai F, Gilbertsen TA, Schonberger RB. Variation in Surgical Quality Measure Adherence within Hospital Referral Regions: Do Publicly Reported Surgical Quality Measures Distinguish among Hospitals That Patients Are Likely to Compare? Health Serv Res. 2014 Mar 11. doi: 10.1111/1475-6773.12164
    3. Ketelaar NA, Faber MJ, Flottorp S, et al. Public release of performance data in changing the behaviour of healthcare consumers, professionals or organisations. Cochrane Database Syst Rev. 2011 Nov 9;(11):CD004538. doi

    Post a comment to this article

    Report Abusive Comment

    www.reliasmedia.com

    Hospital Peer Review

    View PDF
    Hospital Peer Review 2014-05-01
    May 1, 2014

    Table Of Contents

    When it comes to healthcare quality, should titles matter?

    Public quality reporting: a plea for consistency

    Why QI plus IP is more than alphabet soup

    CDC updates hospital infection data

    CDC updates surgical site infection guidelines

    Only a 50% adherence rate to infection control in ICUs

    How hot is too hot for patients?

    Same strains still mean new shots

    Transition to ICD-10 code sets delayed

    Begin Test
    Buy this Issue/Course

    Shop Now: Search Products

    • Subscription Publications
    • Books & Study Guides
    • Webinars
    • Group & Site
      Licenses
    • State CME/CE
      Requirements

    Webinars And Events

    View All Events
    • Home
      • Home
      • Newsletters
      • Blogs
      • Archives
      • CME/CE Map
      • Shop
    • Emergency
      • All Products
      • Publications
      • Study Guides
      • Webinars
      • Group Sales
    • Hospital
      • All Products
      • Publications
      • Study Guides
      • Webinars
      • Group Sales
    • Clinical
      • All Products
      • Publications
      • Study Guides
      • Webinars
      • Group Sales
    • All Access
      • Subscribe Now
      • My Subscription
    • My Account
      • My Subscriptions
      • My Content
      • My Orders
      • My CME/CE
      • My Transcript
    • Help
    • Search
    • About Us
    • Sign In
    • Register
    Relias Media - Continuing Medical Education Publishing

    The trusted source for

    healthcare information and

    CONTINUING EDUCATION.

    Customer Service

    customerservice@reliasmedia.com

    U.S. and Canada: 1-800-688-2421

    International +1-404-262-5476

    Accounts Receivable

    1-800-370-9210
    ReliasMedia_AR@reliasmedia.com

    Mailing Address

    • 1010 Sync St., Suite 100
      Morrisville, NC 27560-5468
      USA

    © 2021 Relias. All rights reserved.

    Do Not Sell My Personal Information  Privacy Policy  Terms of Use  Contact Us  Reprints  Group Sales

    For DSR inquiries or complaints, please reach out to Wes Vaux, Data Privacy Officer, DPO@relias.com

    Design, CMS, Hosting & Web Development :: ePublishing