‘Homegrown’ satisfaction surveys aren’t enough

How do you make data useful to MCOs?

The theory behind patient satisfaction surveys is simple: Ask questions to determine the level of approval or disapproval of services. Doing it correctly is a bit more complicated, and the use of a flawed survey could have deleterious affects.

What many agencies fail to understand, experts say, is that collecting patient satisfaction data requires more than coming up with a few questions and tallying the results. Failure to track variations of positive responses, for example, can lead to skewed data; or an agency’s inability to sort information from various angles can render their information useless to managed care organizations.

"There are a number of agencies who believe that if you write a questionnaire and get answers you get data," says Melvin Hall, PhD, chief executive officer of Press, Ganey Associates, a patient satisfaction survey consulting firm in South Bend, IN. "There’s probably only one thing worse than having no data, and that’s having bad data, because you make management decisions based on bad data. Your decisions are only going to be as good as your data."

Your agency may be conducting patient satisfaction surveys of recently admitted patients in an attempt to gauge their perception of care being delivered and then following up with another survey after the patient was discharged. The results are likely used to find ways to improve the delivery of services and to prove to payers that your agency is a provider of quality care.

But as all areas of health care move into the data-intensive era of managed care, home care agencies are quickly becoming aware that their homemade surveys with "yes" or "no," non- standardized questions don’t cut it in an industry that wants to look at data from various angles.

For example, it’s not uncommon for managed care organizations to ask agencies to break down their data according to specific patient demographics, such as age, sex or gender; by diagnosis; and by type of care, such as nursing, home visit set-ups, or patient education. It’s not uncommon for a payer to ask that the results be separated from other payers so they can see the quality of services their enrollees are receiving.

National figures provide better picture

Coupled with the task of implementing proper methodology, the task can be daunting. Experts recommend seeking outside assistance, particularly in standardizing questions so they are comparable to national statistics, which provide a reliable yardstick for comparing results.

As Hall points out, while a patient satisfaction survey can provide information that can be used to identify areas that need improvement, being able to compare results to national figures will offer a clearer picture of what needs to be improved and give managed care organizations a better indication of quality. Consulting firms that help organizations construct surveys should provide national figures for standardized questions.

"We know generally that the nursing questions will score the highest," Hall says. "And issues of setting up the home care visit will score the lowest. One could fall into a dangerous trap if they deduce that your nurses are doing well and your home health office people are poorly. For example, if on a 100-point scale, nurses score an 85 and setting up home care visits gets a 75, the manager might pat the nurses on the back and kick the home health service workers in the pants. However, if you have a national database, you’ll know the overall mean for nursing is 89 and the overall mean for setting up visits is 72. Then you’ll want to pat your office people on the back and give a boot to the nurses."

In addition, home care agencies generally enjoy high satisfaction, and poorly conceived surveys may not measure the varying levels of high satisfaction, says Jerry Seibert, president of Parkside Associates, a patient satisfaction survey consulting firm in Park Ridge, IL.

"In our pilot testing we had people either evaluate something on an ‘excellent’ to ‘very poor’ scale, or ‘yes’ or ‘no,’" Seibert says. "We found that in home care that the patient was usually so highly satisfied that the response categories didn’t work. Responses were all very high.

"In home care we’re not really differentiating from satisfied and dissatisfied as much as we are differentiating between those who are satisfied and very satisfied. We changed the response categories to excellent, very good, good, fair, and poor. That allowed us to discriminate between the levels of satisfaction." (See sample survey, inserted in this issue.)

Parkside Associates also discovered the four key characteristics of quality that were most important to patients, and questions that help to develop a clear picture of each:

The care process.

Did staff help the patient feel at ease? Did they show concern for the patient as a person, not just as a case? Did they spend enough time with patients? Were staff dependable?

"There’s not a lot of clinical stuff in there," Seibert says. "It’s probably not how a home care agency would define the care process."

Degree of patient involvement in education.

Were the patient and family involved in the decision about the care? Did they clearly understand the goals? Were they encouraged to ask questions? Did they get a clear explanation of medication? Did they get a clear explanation of financial issues?

Orientation to home care.

Did the patient and family know their rights and responsibilities? Did they know how to lodge a complaint? Were they told what to expect, and whom to call if they needed medical attention? Were home safety and diet issues reviewed?

Medical outcomes.

Did the patient’s condition improve as much as he or she thought it would?

These questions should be answered on a some sort of scale, rather than with "yes" or "no." Affirmative or negative cannot measure varying degrees of quality, the two experts say. Examples of scaled responses include "very good" to "very poor"; "always" to "never"; and 1 to 5.

"For example," Hall says, "you can ask, ‘Did the nurse show up on time? Yes or No.’ Well, how do you answer that when the nurse shows up on time 70% of the time? Is that a yes or a no?

Both Hall and Seibert agree that patients need to be surveyed during their admission and after discharge. That’s over and above the Joint Commission for Accreditation of Healthcare Organizations’ standard of surveying during admission only.

However, Seibert recommends surveying a week after admission, while Hall recommends surveying between two weeks and one month after admission, contending that patients will be more familiar with their care after a couple weeks than after just one week.

Surveys also should be sent regularly to ensure that responses reflect a variety of points of care. "Measure often enough to capture people at both ends of the spectrum and throughout," says Hall.

"There are all types of home visits," says Seibert. "If it’s a visit for a new mother, there may be only one, so you might have to survey right after. You can almost come up with a sampling strategy for each kind of patient. But if you just randomly select patients, you will get a different mix of patients at different points in the home care treatment, and that will be more representative of your patient population."