Compliance with protocols may improve outcomes

First large-scale safety study of CABG procedures

What has been called the first large-scale study to examine patient safety issues for isolated coronary artery bypass graft (CABG) showed that hospitals with the highest compliance with three recommended protocols had notably lower risk-adjusted mortality rates than those hospitals whose compliance ranked in the bottom 20%. Representatives of Premier Healthcare Informatics in San Diego and Aurora Health Care in Milwaukee presented the results of the study, titled "Metrics and Measurement in Patient Safety," at the Sixth Annual National Patient Safety Foundation (NPSF) Congress in Boston on May 5.

The year-long study, which included 134 hospitals and more than 40,000 patients, examined the effectiveness of care using widely accepted protocols (i.e., use of aspirin, beta blockers, and an internal mammary grafting procedure) and the impact on patient safety and costs. The findings showed that the variance between the top- and bottom-quintile (i.e., 20% increments) hospitals in terms of compliance with these protocols was as high as 22%. For example, the top quintile used beta blockers 97.9% of the time, compared to 75.1% for the bottom quintile.

Saving 1,300 lives per year

The top quintile also had a risk-adjusted mortality rate of 2.4%, compared to 2.8% for the bottom quintile, a 14% difference. Given that some 350,000 CABG procedures are performed annually in the United States with a mortality rate of 2.66%, according to a Journal of the American Medical Association study published in January 2004, a 14% improvement in mortality outcomes would translate into 1,300 saved lives a year.

In addition, approximately 4% of all CABG patients were flagged with one or more patient safety indicators (PSIs), which indicate that the patient is at risk for a preventable adverse event. The excess variable cost per case for PSI patients was $15,620. No correlation was found between hospital volume of CABG procedures and observed PSIs, although hospitals that did fewer than 200 procedures had much wider variances in performance.

The NPSF study was developed using data from Premier’s Perspective, a web-based clinical performance measurement system that provides patient-level detail to identify opportunities in clinical quality and efficiency.

"PSIs have been studied before — someone did not just grab these out of the clouds — but the validity and true applicability to measure and monitor patient safety outcomes indicators has not been definitively validated, and that’s what we were trying to do," says Kathryn Leonhardt, MD, MPH, associate medical director, care management, for Aurora Health Care.

"Our biggest challenge in patient safety is building a repository of solid evidence that validates how patient safety indicators and interventions directly impact clinical outcomes," Leonhardt adds. "This study is important because it examines patient safety across a broad range of hospitals. While its results are not conclusive, they do support the protocols recommended by the Agency for Healthcare Research and Quality [AHRQ], JCAHO, and others. In addition, the study suggests that effective care is not only safer but also more cost-effective, which is consistent with the limited literature available on this topic."

Aurora and Premier have been working together since 1995, says Leonhardt, with Premier providing informatics services to process its data for process and quality improvement.

"As the movement to focus on patient safety grew, my role became that of orchestrating and facilitating those efforts all within the system," she says. Aurora Health Care is a not-for-profit provider with 14 hospitals and 120 clinics to serve 80 communities throughout eastern Wisconsin.

Using data to drive patient safety efforts

Premier, she notes, recently developed a program through which they utilize AHRQ’s 20 patient safety indicators. "As this conference approached, I started talking with Premier about their sizable database, and if, from our perspective as a local care provider, we could use it to drive patient safety efforts. CABG is a high-volume, high-risk, high-visibility quality issue, involving many regularity requirements on outcomes and processes of care," Leonhardt explains.

So, Leonhardt asked Premier to draw the required information from data it had already collected for its database, and determine how useful it would be. "These PSIs are fairly new and have not necessarily been utilized across the board as measurements of safety," she notes.

Leonhardt says the results of the study indicate that the PSIs do have validity and applicability. "It’s not definitive, however," she says. "For that you need multiple studies. It was impressive on the large scale, but much harder on the small scale — the individual hospital level." She explains that when you drill down in the data from, say, 40,000 cases to 1,000 cases, "you lose some of the forest for the trees." What she recommends is using the larger numbers as benchmarks, norms, and target goals. "In other words, if for all 40,000 total the norm is X percent, and the top performers were X minus ten percent, how do we compare?" She adds that the data are all APC/DRG-adjusted, "So we should be dealing with apples and apples."

In summary, she says the study showed that these reports give a snapshot of some of the safety indicators that are measured. She cautions, however, that it "should not be a definitive statement of hospital safety outcomes, but rather give you overall clinical conditions around those safety indicators."

Need More Information?

For more information, contact:

• Kathryn Leonhardt, MD, MPH, Associate Medical Director, care management, Aurora Health Care. Telephone: (414) 647-3029. E-mail:

For more information about Premier Healthcare Informatics, visit