How does your hospital really measure up?
How does your hospital really measure up?
Standardization of data reporting needed, experts say
Many organizations currently rank hospitals using publicly reported quality data. But this information can be confusing and even conflicting, since each entity is measuring different aspects of quality, using different methodologies for different purposes.
As more quality data become available to hospitals and the public, there is a need for someone to "grade the graders," says Jerod M. Loeb, PhD, JCAHO's executive vice president for research. "This whole world of measurement and public reporting has been growing explosively of late," he notes. "And one of the problems is, where do you look for credible information? How do you assess the credibility of what you are seeing?"
Quality at your hospital may be very highly ranked on one listing, but on the bottom of another list. "It really is kind of silly when a hospital is rated number one in cardiac care by rating service A, number 10 by rating service B, and is not even on the list of rating service C," says Loeb.
It's true that different entities need to portray the same data in different ways. "We, for example, as an accreditor, have certain needs to fulfill," says Loeb. "We want to take data and turn it into information and allow it to be used by various stakeholders in various ways." For this reason, it's acceptable to use different methodologies as long as they are based on the same data and also, that the methodology is transparent, says Loeb.
Some of the hospital rankings are based on subjective opinions such as surveys or reputation, as opposed to hard data, he adds.
In a 2005 report, Rewarding Provider Performance: Aligning Incentives in Medicare, the Washington, DC-based Institute of Medicine recommended that provider achievement be reported "in ways that are both meaningful and understandable to consumers." But with so many different methodologies being used, this is hardly the case, say experts in the quality field.
"It seems to me we have too may people reporting numbers and not enough understanding of what the numbers, percentages, or stars really mean," says Patti Muller Smith, RN, EdD, CPHQ, a Shawnee, OK-based consultant working with hospitals on performance improvement and regulatory compliance. "There will be some effort to standardize public reporting, but that may be a while in coming."
What's being measured?
Before sharing results on how your hospital did on a given ranking, know exactly what the particular reporting source is measuring, such as cost-effectiveness or compliance with evidence-based treatment practices for specific diagnoses. "The second thing that is important to understand is what group of hospitals and providers are included in the sample from which the information or statistics have been gathered," says Muller-Smith.
For example, CMS is looking at payment issues with the goal of achieving high quality, cost-effective care for the population it covers at a reasonable cost. These data drive efforts to ensure that patients are receiving the appropriate services, in the appropriate setting, and in a timely manner, to achieve the best possible outcome, says Muller-Smith. "As a major payer for health care, CMS basically sets the reimbursement levels and other payers follow their lead," she says.
Many other reporting companies look at similar information but focus on patient mortality and morbidity as well as cost-effectiveness. "The quality professional must be sure that their internal data gathering matches up with the benchmark data they are looking at from an outside organization," advises Muller-Smith.
Since most health care consumers aren't familiar with DRG determination or reimbursement practices, measures looking at cost-efficiency and appropriate levels of care are geared toward providers, not patients, says Muller-Smith. "I think mortality, morbidity, infection rates, and patient safety will be prime indicators from the public's perspective," she says.
Use results as 'road map'
The Leapgrop Group recently started its own "Top Hospitals" list, based on results from its Hospital Quality and Safety Survey, a national rating system of more than 1,200 hospitals based on answers to questions about performance on key patient safety activities.
According to Catherine Eikel, Leapfrog's Hospital Rewards program director, unlike other performance designations, Leapfrog's list focuses entirely on the National Quality Forum's Best Practices for Safer Healthcare. Top-rated hospitals demonstrate excellence in patient safety activities, including staffing ICUs with intensivists and treating certain high-risk procedures.
Use the survey results as a "road map" for strategic planning around patient safety initiatives, advises Eikel. "Hospital-based quality professionals can use our survey as a tool as they develop their patient safety initiatives, to see where their facility may have room to improve quality and patient safety," she says.
Consumers increasingly are accessing the information, according to Eikel. "We see this through the increasing number of employers and health plans that use Leapfrog's performance information in their member education strategies and the increasing number of organizations that use Leapfrog's performance information in consumer decision support tools," she says.
HealthGrades' annual Hospital Quality in America Study ranks more than 5,000 hospitals using a risk-adjustment methodology focused on outcomes. According to its researchers, there is a 69% greater chance of mortality for Medicare patients at low-rated hospitals compared with highly rated hospitals.
This "quality chasm" between the best and poorest-performing hospitals has grown by approximately 5% since last year's study, even as overall mortality rates have improved by nearly 8%, according to the report.
"Every hospital has strengths and weaknesses and should leverage their strengths to improve their weaker areas," says Samantha Collier, vice president of medical affairs for HealthGrades. "It is imperative to understand where you are relative to the large gap between the top and bottom performing groups, and what you need to do to move towards the top. This can only be done with regular performance benchmarking and feedback to relevant stakeholders with a call to action."
Solucient's "100 Top Hospitals: Benchmarks for Success" study measures organizationwide performance and is meant to be a tool for boards and executives to improve performance across the whole organization. A balanced scorecard approach helps leaders improve clinical performance, operations, growth in service to the community, financial performance, and hospital systems, says Jean Chenoweth, senior vice president for performance improvement and Top 100 Programs for Solucient.
"Executives, board members, and quality professionals involved in cross-functional improvement teams should make sure that hospital systems, as well as caregiver performance, are considered as possible causes for poor performance," she says. "That's what makes a difference."
In this way, the best possible solutions can be identified to make permanent changes, says Chenoweth. According to a recent Governance Institute/Solucient study, more than 80% of hospitals boards have set a goal of improving performance on core measures. "Boards are paying attention to core measures."
Reporting to consumers
But with all the data available to the public, are health care consumers getting the message? "That's the million dollar question," says Chenoweth. Although Solucient's study is geared toward health care professionals, not consumers, they do help hospitals to convey the meaning of the scores to their community.
"That is what all national agencies are all struggling with today — AHA, NQF and the federal government and all the various quality alliances: How do we get people to understand what different report cards are for and what they mean?" says Chenoweth. "Now we're entering a new phase in which I think all publishers of report cards must accept greater responsibility for communicating what the report card means."
The greatest source of confusion is that different studies measure different things, says Chenoweth. "When data are published about hospitals, you have to ask: What are they trying to measure?" she says. "For example, core measures are designed to be a minimum standard which reflects use of evidence-based treatment for a few specific disease entities. In essence, core measures address whether hospitals are paying attention to evidence-based medicine."
The 100 Top Hospitals program uses nine performance measures to help hospital boards assess comparative performance of their organization. "We do not provide specific data to consumers. That is very different from JCAHO, HealthGrades, US News, Hospital Compare, and Leapfrog," says Chenoweth. "Solucient's hospitalwide scorecard measures what the organization does, not just the nurse or doctor who touches the patient."
However, new results show that although Solucient and JCAHO are measuring different things, the highest-scoring hospitals on both measurement sets were very highly correlated, according to Chenoweth.
"We are very pleased about that, because what we are measuring with the 100 Top study is whether the organization as a whole is functioning well, and thus, whether leadership is having an impact," says Chenoweth. "Our hypothesis, as a result of finding the strong correlation between these two, is that a high score on a hospitalwide scorecard reflects a strong internal infrastructure that is responsive to leadership priorities for improvement — in this case, a priority for improvement of clinical performance as gauged by core measures."
The results indicate that higher-ranked hospitals can change more quickly than other organizations because of infrastructures that allow faster improvement, she explains. "The direct link between evidence-based care and goals set by hospital leadership says that actions of quality improvement teams make a real difference that touches real patients," says Chenowith. "It says that the infrastructure which quality improvement professionals are a part of is important. What they do matters."
[For more information, contact:
Jean Chenoweth, Senior VP, Performance Improvement and 100 Top Hospitals Programs, Solucient, 5400 Data Ct., Ann Arbor, MI 48103. Telephone: (734) 669-7941. Fax: (734) 769-6544. E-mail: [email protected]
Patti Muller-Smith, RN, EdD, CPHQ, Administrative Consulting Services, Box 3368, Shawnee, OK 74802. Telephone: (405) 878-0118. E-mail: [email protected].]
Many organizations currently rank hospitals using publicly reported quality data. But this information can be confusing and even conflicting, since each entity is measuring different aspects of quality, using different methodologies for different purposes.Subscribe Now for Access
You have reached your article limit for the month. We hope you found our articles both enjoyable and insightful. For information on new subscriptions, product trials, alternative billing arrangements or group and site discounts please call 800-688-2421. We look forward to having you as a long-term member of the Relias Media community.