SHEA: Infection rate disclosures could be bad medicine for IC programs

Quality, credibility could suffer unless risk-adjusted data used carefully

Increasing demands for public disclosure of nosocomial infection rates to improve quality and protect patients may actually have the opposite effect by undermining the efficacy of the nation's infection control programs, hospital epidemiologists are warning.

The growing interest in infection rate data displayed by state and federal lawmakers, accrediting agencies, and health care business leaders was a recurring topic recently in Orlando at the annual conference of the Society for Healthcare Epidemiology of America (SHEA). While generally conceding that appropriate comparisons of thoroughly risk-adjusted data could be useful in benchmarking and establishing standards of care among clinicians, SHEA attendees also noted that the push for disclosure could undercut infection control programs for a wide variety of reasons.

The process could lead to "toxic" data that are poorly risk-adjusted and inappropriate for comparisons, in effect penalizing hospitals that have the best surveillance programs or the sickest patients, SHEA speakers and attendees noted. Infection rate data perceived as "good" could be used inappropriately by health care marketing forces to gain a business edge. By the same token, demands for infection rate data that may be publicly disclosed could create pressures and incentives to underreport infections or cut back surveillance personnel and programs, some warned.

Barry Farr, MD, MSc, hospital epidemiologist at the University of Virginia Health Sciences Center in Charlottesville, appeared to be expressing the sentiments of many SHEA attendees when he asked at the end of one session, "Could we all unite and just say 'no'?"

That does not appear to be the case, with the current momentum toward rate reporting including such disparate and powerful players as patient advocates affiliated with Ralph Nader, and the ever-expanding ORYX program of the Joint Commission on Accreditation of Healthcare Organizations. Concerning the latter, some of the performance measurement systems collecting nosocomial infection data in the ORYX system show little regard for risk adjustment and include data inappropriate for comparisons, reported Robert Gaynes, MD, director of the Centers for Disease Control and Prevention's National Nosocomial Infections Surveillance (NNIS) system. (See related story, p. 68.)

At danger of being lost in the rush toward disclosure is the question of whether reporting rates will actually lower infections by increasing the infection control vigilance and compliance by front-line caregivers, Gaynes reminded.

"Do the patient-care personnel - the people giving the care, putting in the lines, managing the patients on ventilators - perceive value in these nosocomial infection rates?" he asked SHEA attendees. "If they do, they will rely on that data for their decisions and will therefore alter their behavior. If they do not perceive value in these rates, their behavior simply will not change."

In that light, Gaynes warned the whole process could prove to be clinically detrimental and damage the credibility of infection control programs if it leads to collecting and reporting rates that have been shown to have little comparative value (i.e., an overall nosocomial infection rate divided by the total number of patient days).

"[Then] you may have actually wasted your limited resources," he said. "It's actually detrimental to improving the quality of care. In addition to wasting resources, you may have damaged credibility if in fact you change to more useful, comparable rates. From our experiences in the NNIS system, even the most useful, comparative rates still take some time for acceptance by patient care personnel. . . . Remember, the essential purpose to this is to truly reduce infection rates within a health care institution - not just to give somebody in your PR department something to do."

Is disclosure inevitable?

However, Sidney Wolfe, MD, director of Public Citizen Health Research Group in Washington, DC, told SHEA attendees that disclosure of nosocomial infection rates is just a matter of time.

"It is inevitable that data - including infection data - will be made public," said Wolfe, who co-founded Public Citizen Health Research Group with Nader in 1971. "Hopefully, it will be adjusted properly so it is more useful rather than less useful. . . . We have been very cautious in getting or using data unless it has some [risk] adjustment."

Projecting that hospital infection rates will begin to be disclosed by "the millennium [or] sooner," Wolfe cited a long list of failed resistance against the medical data disclosures sought by Public Citizen. The group has successfully obtained and disclosed disciplinary actions by state medical boards, hospital comparative data on cesarean section rates, inappropriate drug prescriptions, and hospitals violating "patient dumping" laws. A successful effort to disclose the costs of pharmaceutical drugs in Virginia led to a favorable ruling by the U.S. Supreme Court in 1976, he noted.

"The Supreme Court decision said, essentially, information is not in and of itself harmful," Wolfe said. "The public will be better served if only they have the right to know. The idea had been put forth that the information is dangerous and the people who see these [pharmaceutical] prices won't understand. The Supreme Court did not agree with that and that case was important in terms of the right to get information."

Wolfe argued that disclosure of medical quality information is the only way to ensure patients are protected, citing cases where corrective action was only taken after release of, for example, data on surgeons with inordinately high patient mortality rates or hospitals with conspicuously high C-section rates.

"If you don't get rid of the bad apples, it has a demoralizing effect, to say nothing of the effect that it has on patients," said Wolfe, noting that his group recently published a four-volume, 4,000-page book titled 16,668 Questionable Doctors.

However well-intentioned, such efforts may prove detrimental when addressing such a complex patient outcome as nosocomial infection, countered William Scheckler, MD, one of the original founders of SHEA who joined Wolfe in a debate on the issue.

"Bad data is a toxic substance," said Scheckler, hospital epidemiologist at St. Marys Medical Center in Madison, WI. "The information available may be misinformation, no matter what the Supreme Court says."

For example, severity of patient illness must be factored into any infection rate to make comparisons meaningful, he noted.

"Just reporting a clean-operation wound infection rate doesn't mean anything in a comparison of hospital A to hospital B," he reminded. "That is lesson No. 1."

Yet, risk adjustments for severity of patient illness remain an imperfect science, particularly if the rates are to be used to generate comparative data between hospitals and other health care systems, he added.

"We're better at it than we were five years ago, but we still aren't there yet," Scheckler said. "There is currently available no generic system that adjusts rates for severity on inpatients, much less outpatients and other categories."

SHEA will soon publish a report on the Project to Monitor Indicators (PMI), done in conjunction with the Joint Commission, that assesses the current state of programs to collect and compare infection rates and other outcomes measures, he added.

"The bottom line in this [PMI study] is that using several systems purporting to measure the same outcome in the same group of hospitals can move the same hospital from one end of the scale to the other," he told SHEA attendees. "I would suggest to you that this is the major problem with the systems that are now in place."

Infection comprises three factors

Addressing the same topic in a different SHEA session, Walter Hierholzer Jr., MD, epidemiologist at Yale-New Haven (CT) Hospital, underscored that nosocomial infection represents a complex, multifactorial event that requires sophisticated risk adjustment to account for all variables.

"We recognize that the risk of infection is the intersection of three factors - the various factors the [infecting] agent has, the various factors the host has, and then the environment in which they both dwell, which may tip the advantage in one direction or another," he said. "Infection therefore is truly a multifactorial event, and is an outcome that we measure that has all sorts of causes working in concert or in opposition. Risk adjustment stratification has to deal with this."

By the same token, it is becoming increasingly difficult to prevent nosocomial infections in severely ill patients admitted to intensive care units, and that must be accounted for in any reporting of data in fairness to the hospitals treating such cases, he added.

"It is obvious mortality is soon to come [in these patients], and yet our ICUs are flooded with these cases," he said. "To prevent nosocomial infection in that group is extremely [difficult]. I think we need to deal with that entire concept ethically and intervention-wise."

Beyond risk-adjustment concerns, there is the issue of how infection rates will be used, Scheckler added, noting that health systems may be tempted to market data that make a facility look good and suppress data that render a bad impression.

"Marketing health systems based on low comparative complication rates is marketing based on fiction, whether it is public or not," Scheckler noted, adding that if the goal is truly to improve quality of care, data should be shared among clinicians in the medical journals, and "not on press releases and advertising."

Look harder, find more

In addition, other SHEA participants noted that hospitals that have aggressive infection surveillance and the full complement of microbiology lab tools will certainly detect more infections than hospitals with fewer resources.

"I think until we have definitions and surveillance methodologies that everyone uses, and it is mandated that we all do it, that even if we stratify the rates [they] aren't going to be accurate," said Loreen Herwaldt, MD, hospital epidemiologist at the University of Iowa Hospital and Clinics in Iowa City. "My personal opinion, with the state of infection control the way it is, is that patients need to know what the hospital is doing. Are they looking for infections? Do they ever identify an outbreak? If they do, what do they do? I think that is much more helpful to patients than infection rates."

Michael Decker, MD, MPH, associate professor of preventive medicine at Vanderbilt University School of Medicine in Nashville, TN, reminded SHEA attendees of the controversy that ensued when the Health Care Financing Administration released national hospital patient mortality data that were not always properly risk-adjusted.

"My impression was that HCFA was making a very earnest effort to adjust the data as best they could, but those efforts were hampered by the fact that they just never had adequate data," he said.

In summing up the Wolfe-Scheckler debate as moderator, Decker referred back to the HCFA controversy with mortality data in light of the obvious concern in the SHEA audience of a similar scenario unfolding for infection rates.

"I think the audience's message very strongly has been when you are dealing with something as slippery as infections that are easy to not discover - unlike dead bodies, which are hard to hide - that the system has counterincentives," Decker said. "That is to say, a system that forces release of this data punishes those who have done their job the best and rewards those who have done their job poorly. It is very important to make sure that we don't act in a way that would [encourage that], thus causing our vice presidents to fire our nurses so we don't find the infections either."

Selected references

1. Gaynes RP, Emori TG. "Surveillance of Nosocomial Infections." In: Bennett JV, Brachman PS, eds. Hospital Infections, 4th edition. San Francisco: Lippincott-Raven; 1998.

2. Gaynes RP. Surveillance of nosocomial infections: A fundamental ingredient for quality. Infect Control Hosp Epidemiol 1997; 18:475-478.

3. Archibald LK, Gaynes RP. Hospital-acquired infections in the United States: The importance of interhospital comparisons. Infect Dis Clinics North Am 1997; 11:245-255.

4. Iezzoni LI. The risks of risk adjustment. JAMA 1997; 278:1,600-1,607.

5. Gaynes RP, Solomon S. Improving hospital-acquired infection rates: The CDC experience. J Qual Improv 1996; 22:457-467.

6. Localio AR, Hamory BH, Sharp TJ, et al. Comparing hospital mortality in adult patients with pneumonia: A case study of statistical methods in a managed care program. Ann Int Med 1995; 122:125-132.