Study raises concerns on public reporting of data
Study raises concerns on public reporting of data
Does public reporting lead to selection bias?
A vehicle designed to boost quality — the public reporting of outcomes — may in some cases be doing just the opposite, according to the authors of a new report.
The study — the first ever to compare data from a state that requires public reporting of angioplasty results (New York) with data from a state without public reporting (Michigan) — reveals some striking differences, although it can’t show a direct cause-and-effect relationship between public reporting and physicians’ selection of patients.
The study, which was published in the June 7, 2005, Journal of the American College of Cardiology,1 was performed by a University of Michigan (U-M) Cardiovascular Center team and colleagues in the Blue Cross Blue Shield of Michigan Cardiovascular Consortium, in collaboration with researchers from the State University of New York at Stony Brook.
Overall, the New York patients who received angioplasties to open clogged heart arteries were far less likely than Michigan patients to have underlying or acute conditions that raised their risk of dying before they left the hospital. And Michigan angioplasty patients were twice as likely as New York patients to die in the hospital.
But when the researchers corrected for the fact that Michigan patients were much sicker on average than the New York patients before their angioplasties, they found that there was no difference in overall death risk between the two states. Because the states have similar heart disease patterns, the team concluded that the difference in patient selection might be due to the presence of public reporting in New York.
Case selection bias
"Anecdotally, there has always been a concern that [required public reporting] may result in case selection bias," notes Mauro Moscucci, MD, FACC, an associate professor of internal medicine in the U-M Medical School in Ann Arbor, who led the study.
"Anecdotally, in talking to physicians in New York state, they were always concerned about numbers being made available to the public, to the point where it may affect their decision-making process. That prompted us to look at the data," he explains.
They did so, Moscucci adds, by merging U-M’s Michigan data with the New York state registry through the help of David Share, MD, clinical director for the Blues’ Center for Health Care Quality and Evaluative Studies.
The study is based on data from about 80,000 angioplasty patients: 11,374 treated at eight Michigan hospitals, and 69,048 treated at 34 New York hospitals. It is based on New York data from the calendar years 1998 and 1999 that were published in January 2003.
The Michigan data are from the same time period. In all, 32% of patients in the study were women, and the average age was 63. New York patients were slightly older than Michigan patients and more likely to have high blood pressure.
But the Michigan angioplasty patients had a significantly higher incidence of kidney problems, diabetes, lung disease, vascular problems beyond their hearts, and congestive heart failure.
They also were much more likely to have had at least one previous angioplasty, suggesting a history of significant coronary artery disease.
The Michigan patients also were more likely than New York patients to receive blood thinners or nitroglycerin just before their angioplasty; these drugs are recommended in some high-risk patients to prevent blood clots and blood vessel spasms during and after the procedure.
The Michigan patients were much more likely to have their angioplasty performed soon after being treated for an emergency condition: heart attack, cardiac arrest, or cardiogenic shock — a dangerous and often fatal condition in which the heart fails to pump enough blood to the body.
"What really comes to the eye in the data was that there were not only substantial differences [in mortality] — twice as high in Michigan as in New York state — but when you look at comorbidities, they were also much higher in Michigan," adds Moscucci. "The data set suggests that case makes a difference, and those higher rates may be due to case risk."
As Michigan did not have public reporting at that time, the researchers concluded the differences might be due to case selection bias and a tendency in New York state to not intervene.
Moscucci adds that in a recent survey of interventional cardiologists in New York state published in the Archives of Internal Medicine,2 "when asked if they believed having mortality statistics available to the public affected their decision-making process not to intervene, 80% said they were affected."
He notes that the majority (70% to 80%) also said public reporting may result in not performing angioplasties in some high-risk cases.
Moscucci is quick to point out that he is not opposed to public reporting.
"I even promote the reporting of outcome data in Michigan," he asserts. "With aggregate data, your reporting is much more robust."
The key, he says, is how you risk adjust the data. "We report a model in our paper," Moscucci notes. "When you look at the aggregate data, risk adjustment works pretty well."
Public reporting, he adds, can increase accountability and lead to quality improvement.
"If I didn’t like my numbers, I would probably try to do what I could to improve those numbers," says Moscucci. However, he cautions, quality managers need to be aware that public reporting may have a potentially unwanted effect.
"At the physician level, it may be a potential problem as you’re dealing with a smaller data set," he explains.
References
- Moscucci M, Eagle K, Share D, et al. Public reporting and case selection for percutaneous coronary interventions. J Am Coll Cardiol 2005; 45(11):1,759-1,765.
- Narins C, Dozier AM, Ling FS, et al. The influence of public reporting of outcome data on medical decision making by physicians. Arch of Intern Med 2005; 165:83-87.
Need More Information?
For more information, contact:
- Mauro Moscucci, MD, FACC, Associate Professor of Internal Medicine, University of Michigan Medical School, Ann Arbor. E-mail: [email protected].
Subscribe Now for Access
You have reached your article limit for the month. We hope you found our articles both enjoyable and insightful. For information on new subscriptions, product trials, alternative billing arrangements or group and site discounts please call 800-688-2421. We look forward to having you as a long-term member of the Relias Media community.