Do you use benchmarks without benchmarking?

Clinical Value Compass can organize your work

Pop quiz: What’s the difference between benchmarking and benchmarks?

Answer: Benchmarking is the systematic process for finding the best of the best. Benchmarks are statistical measures.

Do your homework with Julie Mohr, MSPH, research associate in Dartmouth Medical School’s Center for the Evaluative Clinical Sciences in Hanover, NH, and you’ll find out that you really do need to understand the difference. Not only that, but you need to make sure you aren’t using either benchmarking or benchmarks in isolation. If you don’t have the benchmarks, you can’t evaluate the merits of different practices. If you don’t do the benchmarking process, you won’t have any idea how those numbers happened.

"It seems obvious that you can’t do benchmarking without the benchmarks," Mohr says. "But you’d be surprised how many people do a simple comparison of data without understanding the underlying reasons for the numbers. You have to use benchmarking and benchmarks together to get the insights you need to implement change."

If you define and systematically follow a benchmarking process, you can expect the following results, Mohr says:

o tension for change;

o awareness of current capability vs. best-known capability;

o movement from a position of inertia to positive action.

So how do you get started? Mohr and her colleagues at Dartmouth have the answer for that one too: the Clinical Value Compass model, developed to help groups organize their quality improvement work. "We had people coming to us for help with benchmarking, but nobody knew exactly what to ask for," Mohr says. "The Clinical Value Compass gives straightforward answers. It’s a set of simple tools that anyone can implement, and there are no tricks involved."

The idea is to break down the process into manageable components that will yield answers about only the most critical indicators of success for whatever process you want to improve. Find out more about those few areas that are of greatest interest to your group, and you can focus attention toward real change.

If you try to measure everything or try to do it without a systematic approach, you’ll miss the value of successful benchmarking, Mohr says. The basic steps in the Dartmouth approach are to identify the outcomes, analyze the process, generate change ideas, and then do a pilot test of a change. "Benchmarking is the best way to generate those change ideas," Mohr says. "If you don’t have the information to back up your ideas, you won’t be able to encourage the culture of change you need to make real improvement."

Start with the compass, which has at its four points: functional health status, costs, satisfaction, and clinical outcomes. The compass provides a guide to help you determine what results to look for in the benchmarking process. The first step is to clearly define the target population and select a related set of outcomes and cost measures based on the four compass points. If you’re looking at total costs, for example, you might start with total charges paid by purchasers, but you should also look at areas such as indirect social costs, time lost from work, and workers’ compensation to find ways to minimize the total costs of illness.

The Dartmouth group makes it easy with a benchmarking worksheet that takes you through the process. The five basic steps are:

1. Identify measures. Using the Clinical Value Compass as a guide, get your improvement team to reach consensus on two or three measures for each compass point. Make sure these benchmarks are important across your team and that valid data on them are likely to be available.

2. Determine resources needed to find the best of the best. The key here is to make sure you can find both internal and external data that are reliable. You’ll need to identify in-house experts who can provide information and also direct you to external experts. Also, create a short bibliography of the best articles on the topic in the literature. The Internet can be your best friend during this step, Mohr says.

3. Design data collection method and gather data. Make a time line for collecting and analyzing the data and reviewing the literature, and identify who is responsible for completing these tasks.

4. Measure best against own performance to determine gap. Look at your internal results, national average results, and data from "the best of the best." Identify the "tension for change" by discovering how big your gap in performance is, Mohr says.

5. Identify the best practices that produce best-in-class results. Identify potential benchmarking partners and establish a collaborative learning relationship with them. Have some good information about your own processes before you get on the phone, Mohr says.

"The basic idea behind benchmarking is simple," Mohr says. "You want to ask yourself what you need to benchmark, how you do the process, how other people do the process, and who’s doing it best?"


Tips For Successful Benchmarking

Here are some of the lessons learned through improvement work at the Center for Evaluative Clinical Sciences at Dartmouth Medical School in Hanover, NH, according to Julie Mohr:

- Remember that learning about best practices is the aim.

- Understand your own processes.

- Data aren’t perfect, and you can’t collect them just once. You must redo them as processes change.

- Benchmarking is an ongoing process.

- You must be specific in your selection of benchmarks.

- You need strong planning and top management support to generate change.

For more information on the Clinical Value Compass model, contact Julie Mohr, Center for the Evaluative Clinical Sciences, Department of Community and Family Medicine, Dartmouth Medical School, 7251 Strasenburgh Hall, Room 304, Hanover, NH 03755-3863. E-mail: Julie.JMohr@Dartmouth.edu.

A series of articles on the model appeared in The Joint Commission Journal on Quality Improvement beginning in April 1996. You can download the articles from the Best Practice Network Web site at www.best4health.org.