Are your patients sicker? Try using risk adjustment
Yet be aware of limitations of adjustment methods
For performance assessment to be meaningful and fair, comparisons among physician practices must account for significant differences among patients such as comorbities and age.
Lisa Iezzoni, MD, MSc, an expert on risk adjustment and professor of medicine at Harvard Medical School and Beth Israel Deaconness Medical Center in Boston, spoke to Patient Satisfaction & Outcomes Management about some common questions and concerns surrounding risk adjustment.
Question: When physicians are presented with comparative outcomes data, such as mortality rates following bypass surgery, they often respond by saying, "But my patients are sicker." Should they feel more trusting of data when told that they were severity-adjusted?
Answer: The first thing you have to acknowledge is that some doctors will have sicker patients, but other doctors won’t. Not all doctors will have patients who are sicker than average. It’s important to respect the concerns that a doctor may have sicker patients, but you have to prove it.
If you’re going to be looking at outcomes such as mortality rates, absolutely you need to adjust for risk. It’s just not meaningful to leave risk out of the equation. But doctors have to be prepared that they may not have sicker patients and realize that sometimes data contain surprises.
Question: Your research indicates that different severity-measurement methods can produce significantly different results. Does that mean that severity adjustment doesn’t work?
Answer: No, that’s not what it means. What it means is that even severity-adjusted data are simply a screen. They’re a first-cut pass at what’s going on. It should not be the only way that you look at the data.
For example, if you find even after severity adjustment that a doctor has higher mortality rates, then you have to go back to the data and see if there are other things that could explain those rates such as unadjusted risk factors.
In a report-card comparison of obstetrical morbidity in obstetrical care, researchers found one of the big teaching hospitals in Pennsylvania had much higher than expected rates even after risk adjustment. This hospital knew that this was going to be published in newspapers all across the state. The risk-adjustment method had not accounted for cocaine abuse and other drug abuse.
Obviously, women who are drug abusing are likely to have worse outcomes than other women. Once they were able to show that, that gave a better understanding for what was going on.
The bottom line, when I talk to anybody about risk adjustment, is that it is literally impossible to adjust for everything. Before immediately leaping to the conclusion that people with worse risk-adjusted outcomes have worst quality, you have to ask, "Were there risk factors that were not considered in the risk adjustment?"
Question: Should all performance measures for physicians be risk-adjusted? For example, some physicians have pointed out that patient compliance may differ based on cultural issues or educational level. Should those aspects be included in risk adjustment for preventive care such as mammograms or Pap smears?
Answer: This is a very important question. If the outcome measure that’s being looked at for physicians is satisfaction with care, there’s good research to show that patients with behavioral or mental health problems are more likely to be dissatisfied with their care. Should you hold physicians responsible for having a patient population that happens to have more mental health problems? Yet getting access to that information is very difficult.
Poor people have different attitudes than rich people. Even [determining the influence of] educational level is hard. There are some risk factors you would love to be able to adjust for, but it simply won’t be possible.
If you have several medical groups that are looking at satisfaction and one has poor satisfaction but [the data are] not adjusted for education or cultural issues, use it to begin a dialogue. How do we do better for poorly educated patients? How do we address cultural competency issues?
Question: Should physicians use some form of risk adjustment even if they are monitoring care for internal quality improvement?
Answer: In a practice, we have access to a lot more data than a health plan does. We probably would have a lot more access to the sensitive information than the people who are using the data externally.
For internal purposes, it probably is important to do some risk adjustment, but there is going to be a cost of risk adjustment. You need not only to have the data, but you need analysts on board who are capable of evaluating it.
Everyone wants to do sophisticated risk adjustment and analysis. But there simply aren’t enough trained staff who understand all the issues and are able to do that. It’s really going to turn on how the data are going to be used internally, whether it’s simply to start a dialogue. We hear a lot about the need to prove quality in health care, about new performance indicators and report cards. Why is there little debate about methods of risk adjustment?
There is a very strong recognition of the need to do risk adjustment. One of the problems is data availability. People can’t begin to debate risk-adjustment methods until they have the data to talk about it.
Who will design the methods?
Who is going to develop these methods? There are a lot of commercial organizations out there developing these methods, and they’re proprietary and treated like black boxes, which makes me very nervous.
You couldn’t have figured out that the risk adjustment in Philadelphia didn’t adjust for drug abusers if you didn’t know what was in the black box. You have to talk about federal and foundation funding to develop risk-adjustment methods.
Question: What questions should physicians ask about risk-adjustment methods?
Answer: Physicians should ask to see the internal logic of risk adjustment. The first questions should be, "What are you adjusting for? Tell me clinically what are you adjusting for? Tell me what the hypotheses are for the relationship between the risk factors and the outcomes and the evidence that supports those hypotheses."
People talk all the time about evidence-based medicine, but you also need evidence for risk-adjustment methods.
If you’re going to risk adjust for outcomes and you find the issues people are concerned about for risk adjustment but you do not have the data to adjust for it, then that’s a warning sign. You should state the data don’t account for all the patient attributes that the literature say are important.
In summary, you first need to feel that the appropriate clinical factors are in the method. Then you need to verify the validity of data to support risk adjustment. Finally, you need to check the statistical performance of the model.