Measure effectiveness with internal audits

At a recent industry roundtable, government and private sector compliance experts were divided on how best to measure the effectiveness of hospital-based compliance plans.

"I don’t think there is anything more difficult than assessing the effectiveness of compliance programs," says Chris Idecker, a partner with Ernst & Young in Atlanta, and a participant in the Government-Industry Roundtable in Washington, DC, sponsored by the Health and Human Services Office of the Inspector General and the Health Care Financing Administration. Idecker says that while it is relatively easy to assess the "inputs" to a compliance program it is extraordinarily difficult to measure the "output" of those resources. "I think it is almost nonsensical," he argues.

For example, while most experts acknowledge the value of conducting audits as a way to gauge effectiveness, there’s sharp disagreement about precisely what type of audit is best suited to the task.

Typically, providers use three types of audits: baseline or initial audits; proactive audits, which are usually based on risk areas identified by the OIG; and issue-based audits, which are triggered when providers discover a problem and attempt to determine its magnitude.

Idecker, for one, doesn’t believe baseline audits always establish "a basis for demonstrating improvement and effectiveness. Baseline audits suggest that you can identify several major problems and correct them. But my experience is the exact opposite," asserts Idecker, who until recently was chief compliance officer for Medaphis Corporation in Atlanta. "When your compliance program is effective, you start finding more problems and more issues to deal with. That is the hallmark of an effective plan."

Idecker says it’s possible for providers to educate staff about certain key aspects of compliance such as coding and then test those skills to measure improvement. "But the problem is that you are dealing with something that you don’t know about," he explains. "Non-compliance can be the result of fraud or abuse or systematic errors but you don’t know about them; otherwise you would stop them. Can you imagine demonstrating effectiveness by saying, We used to have 10% fraud and now it is only 5%’?"

Joe Murphy, Executive Vice President of Compliance Systems Legal Group in Haddonfield, NJ, says certain measures of effectiveness, like billing errors, can be used as a baseline. But he concurs with Idecker that this approach does not address fraud or many types of willful misconduct.

Several compliance officers also point out that when providers perform retrospective audits, they must decide how far back to extend the review. Because that’s a difficult decision, many opt instead to focus their resources on establishing new programs and performing audits once those programs have been implemented.

According to Murphy, one of the techniques that can measure the overall impact of a program’s effectiveness is what he calls "true compliance audits."

"These are not bean counting exercises," says Murphy. Instead, those with an understanding of compliance and the institution are given responsibility to review files and conduct direct interviews with staff, much as a government investigator would. "In my experience, there is no substitute for that approach," says Murphy. "There is no type of broad-based system that will match the return that you get from a true compliance audit."

(In the next issue of Compliance Hotline, we’ll explore the use of surveys to assess the effectiveness of compliance programs.)