How to create and monitor your measures of success
Consider each step in the process
Unless your organization is lucky enough to be found 100% compliant after your next survey by the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) or after completing the periodic performance review, you’ll have to address noncompliant areas for which you must set measures of success (MOS) and then gather data to evaluate improvement.
"Many organizations run into trouble because they create MOS that are not meaningful or accurate," says Michelle H. Pelling, MBA, RN, president of the Newberg, OR-based health care consulting firm The ProPell Group.
To avoid problems, consider following these steps to create useful and effective MOS:
1. Develop selective measures of success.
"One of the most important things to remember is that measurement is counting," says Pelling. "It is the quantification of anything capable of being quantified."
The first step is determining the question to be answered. "Remember that you cannot measure everything," Pelling says. "You need to be selective and relate what you are measuring to the question you are trying to answer."
Do not develop broadly defined measures or try to measure too much at one time; instead, choose specific, focused measures, Pelling recommends.
"You are trying to answer one question with each measure — not two or three," says Pelling. Any measure that includes the words "and," "or," "if," "after," "until," or "when" could cause you to collect information about two or more elements, or to be so vague or ambiguous that it becomes impossible to gather meaningful data, she explains.
At Willis-Knighton Health System in Shreveport, LA, each requirement for improvement was assigned to either a team, committee, or department for follow-up. After review, the responsible parties set their own MOS based on several factors, including history of the issue, resource availability, and time frames, says Lisa Maxey, MBA, RHIA, the organization’s JCAHO coordinator.
2. Begin collecting data.
Once you have developed an MOS, you must begin collecting data. Pelling gives the following example of a measure of success: Are the initial nursing assessments complete in the sample of records reviewed?
For this measure, you would need to define the terms "complete," "nursing assessment," and "initial." You also must determine when in the process you can get these data, such as 24 hours after admission, and who can give them to you — for example, nursing staff, says Pelling.
Next, determine how you can collect the data with minimum effort and chance of error, such as retrospectively or concurrently at the point of care. In addition, determine what else you need to capture data for future analysis, reference, and trackability. For example, if sections are incomplete, you may need to know which sections are incomplete and what specifically is missing from them, Pelling says.
During a March 2004 survey at Willis-Knighton, surveyors required the organization to implement improvements in pain assessment/reassessment. "The issue was, not only was the patient assessed, but also, were they reassessed?" says Maxey. When the pain management committee developed an audit tool to measure improvement in this area, it first had to define the relevant areas and create definitions for which data were "not applicable," she adds.
When are pain reassessments necessary?
"The most difficult part was identifying the appropriate departments to determine which were performing pain assessments/reassessments and which were not," says Maxey. "For example, we explained to our departments that if a work conditioning-type patient is seen for a urine drug screen test, those records would not need to be part of the reassessment audit. Similarly, a patient in one of our physician’s offices may not need to be reassessed if they are only scheduled to receive a routine B-12 injection."
To educate staff about the requirements, inservices were given on pain assessment, and an on-line educational tool was developed. "It really was primarily the outpatient areas that were hard to get our arms around because of the limited amount of time the patient spends on campus, as opposed to patients who are seen over longer periods of time or inpatients where assessments are done during each shift," says Maxey.
Policies and procedures were revised to include all the inpatient and outpatient areas. "We then had to educate the staff that we now require them to perform chart audits and to report their findings to the committee, so we are getting the data from the actual department," says Maxey.
During collection of the data, a weighted value is assigned, since department size and chart volume can differ greatly across the system. The different statistics from each department then are merged in order to come up with an average. "Those statistics are reported to the pain management committee, as well as to the JCAHO preparation team," Maxey says.
3. Design your data collection system to minimize bias.
"Bias results in data that is not representative of the natural state of the process," says Pelling. Common causes of biased data include a misunderstanding of how to collect and record data, inaccurate measuring instruments, changes in the process during data collection, subtle disregard for the facts, poor choice of data collection period, poor sampling techniques, and lost data.
To minimize bias results in your data collection, Pelling recommends doing the following:
- Consider potential sources of bias before data collection.
- Conduct a small trial of data-collection forms to see if they are easy to use.
- Provide thorough training and explanations for how to complete the forms, and address all concerns.
- Audit the data-collection process by examining data as they come in. Verify that you are getting complete information by observing data collectors or cross-checking numbers from other sources.
4. Monitor your success.
Although organizations are given four months to submit MOS data to JCAHO, it’s better to monitor compliance more often, Maxey recommends. "Our goal is to monitor our compliance each month, rather than waiting until the four-month period is over and it is time to submit this information to JCAHO," she says.
When the organization submitted evidence of standards compliance, the organization had to enter a percentage into JCAHO’s extranet site for the stated goals for each requirement for improvement, Maxey says. "If, at the end of the four-month period, we are not in compliance with our stated goals, we will have to suffer the consequences from JCAHO," she says. "That’s why we are measuring and reporting on a monthly basis, following the adage of trust but verify.’"
Initially, some units failed to report MOS data, because not all areas were aware that they had to gather and present data, Maxey says. "More education was done for these areas," she says. "Also, some areas, such as the work conditioning area, should not have been reporting." To address this, the pain management committee went back to the drawing board to further define the areas that should be required to assess/reassess pain and report those statistics.
Various teams and committees are responsible for follow-up on requirements for improvement, and they report data directly to the Joint Commission preparation team, which was made a subcommittee of the performance improvement and medical quality council, notes Maxey. "The JCAHO prep team is held responsible by upper management and medical staff to report the data," she says. "If something is not going as it should, we have to determine quickly how to correct the issue."
It took more time than expected to submit data showing evidence of standards compliance to JCAHO’s extranet, says Maxey. "That’s a problem," she says. "It took us quite a long time to organize the information and then to submit it. I don’t think that the JCAHO realizes how much time it takes to get all this information entered."
To decrease data errors and time expenditures, the organization’s JCAHO prep team devised a standardized reporting form containing prompts for information required for follow-up on the extranet site. The standardized format ensured that each team submitted complete information and that the information was approved by the JCAHO prep team and then correctly entered. "If all information is standardized, allow eight to 10 minutes for each requirement for improvement," says Maxey.
[For more information on measures of success, contact:
• Lisa Maxey, MBA, RHIA, JCAHO Coordinator, Willis-Knighton Health System, P.O. Box 32600, Shreveport, LA 71103. Telephone: (318) 212-4156. E-mail: [email protected].
• Michelle H. Pelling, MBA, RN, President, The ProPell Group, P.O. Box 910, Newberg, OR 97132. Telephone: (503) 538-5030. E-mail: [email protected]. Web: www.propellgroup.com.]
Unless your organization is lucky enough to be found 100% compliant after your next survey by the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) or after completing the periodic performance review, youll have to address noncompliant areas for which you must set measures of success (MOS) and then gather data to evaluate improvement.
Subscribe Now for Access
You have reached your article limit for the month. We hope you found our articles both enjoyable and insightful. For information on new subscriptions, product trials, alternative billing arrangements or group and site discounts please call 800-688-2421. We look forward to having you as a long-term member of the Relias Media community.