The Quality Cost Connection: Get actionable results from surveys
Get actionable results from surveys
Part 2 of 2
By Patrice Spath, RHIT
Brown-Spath & Associates
Forest Grove, OR
One of the very best methods to obtain feedback on performance is through the use of survey instruments that give respondents an opportunity to speak directly and frankly. However, formulating a series of questions to obtain answers about important topics is not a simple task. Constructing a questionnaire that will elicit accurate information from most respondents is more complicated than it seems. Many factors must be considered during the process of creating a questionnaire. For example, will most respondents interpret every question in the same way? If not, the survey results might not provide the information you hoped for. Maximum effort is needed during the developmental stage, because once a questionnaire has been distributed, problems are costly or impossible to correct. The time spent in developing the questionnaire will be repaid by collection of relevant, better quality data. In the second part of this 2-part series, pilot testing, survey administration, and results reporting are described.
Once a draft of the questionnaire has been prepared, try it out on a small number of respondents. A debriefing session with these pretest volunteers is helpful in identifying and correcting any problems.
Pretest instrument
No matter how well you've thought out the survey instrument there still are likely to be a few minor problems that must be identified and corrected. This is the function of the pilot test. Prepare a small number of surveys as mock-ups, and recruit volunteers to respond to the questionnaire as though they were members of the sample population. Immediately after completing the instrument, ask the volunteers to identify any flaws or errors in the survey. Find out if the instructions and items were as clear as they possibly could be. Did the respondents know whether they were to make choices, indicate ranking, circle answers, etc., and how this was to be done? Did the respondent know when to turn the page and how to respond each time a new type of item, scale, or topic was used? Even if the number of pretest volunteers is quite small, this step is crucial. It is almost certain that some problems will be identified.
After problems identified through pretesting are corrected, prepare a final copy of the survey for reproduction. Carefully check the final product before distribution. Errors that don't get caught at this step can be costly, especially if you have to discard some of the survey results because of problematic questions or typographical errors.
Administer survey
Ideally, the survey instrument is administered to everyone at the same time. For example, the instrument is administered at a department staff meeting or in a focus group session with former patients. However, usually this is impossible. It may be necessary to distribute questionnaires to individuals who complete and return them either by hand, intra-company mail, or U.S. mail. Although this administration technique does increase privacy and anonymity for respondents, it also leads to decreased return rates. When individuals must be relied on to voluntarily return the survey, you must do everything possible to boost the return rate. There are several in which this can be done.
First, the importance of the survey must be emphasised. This can be emphasized in three ways: (a) in the cover letter; (b) in informal conversation when the questionnaire is being handed to the potential respondent; and (c) by explaining that some authority (e.g., management) supports the survey and why. The second aspect to be emphasized is the confidentiality and privacy of individual responses. Again, this should be stated in the cover letter and verbally. Explain that the survey results will only be reported in the aggregate, not by individual respondent. If possible, questionnaires to be returned by mail should be sent to an address other than the organization's. If employees are expected to return questionnaires at work, provide a sealed box so that respondents can drop their completed questionnaires into it. It should be specified verbally and on the questionnaire form that no names or identifying marks are to be put on the questionnaires or response forms.
Invest as much time and effort as possible in personal contact with the potential respondents, explaining the objectives of the survey verbally and asking for and answering questions. People participating in workplace surveys should be promised a summary of the results. Where possible, survey forms should be given directly to potential respondents, and they should be thanked for their participation. In short, everything possible should be done to maximize personal contact. Respondents should be asked when their responses could be expected, and an attempt should be made to obtain verbal commitments to a specific time frame. These investments of time and energy will pay off in terms of higher return rates.
If you are conducting a one-time survey, set a deadline for receipt of the responses. This date should be included on the questionnaire and mentioned verbally to the respondents. The date given should be at least one week earlier than the actual deadline. Shortly before the stated deadline, all respondents should be reminded of the date by means of a letter or memorandum. An offer can be made to provide another copy of the questionnaire if the first was misplaced.
Tabulate results
The aim here is to present the data so that people can understand and make interpretations from the information generated. Sample tabulations of responses for each item, using percentages (not just raw numbers), generally will suffice. This can be indicated on a "doctored" copy of the questionnaire, with percentages filled in where the check marks would go.
Many results may be ignored later, but it is important to begin by tabulating everything. An example of a tabulation of nurses' answer to a question related to their satisfaction with the continuing education offered by the hospital is found in Figure 1.
The next step is cross-tabulation for items that have some important relation to one another. For example, to determine whether nurses in one unit are less satisfied than those in another, one would cross-tabulate unit by satisfaction, as is shown in the second example in Figure 1. The items in parentheses are the numbers of respondents from given units who gave specific satisfaction-level responses. For example, of the 40 nurse respondents in the intensive care unit, 60% (or 24) were completely satisfied. Of the 10 nurse respondents in the emergency department, 40% (or four people) were completely satisfied. One may not know if a trend such as this is statistically significant but might decide later that it is worth testing. Effective tabulation of survey results leads directly to data interpretation, the final output of all survey work.
Prepare report
The exact form of the final report will depend on how it will be used and other circumstances. If the data will be used to work on problems, with small groups involved at different levels, the report should avoid inferences and conclusions. It should contain data grouped by unit, department, or division.
If senior leaders will be using the report to develop action plans, then more summary, charts, and recommendations usually are desirable. When preparing the final report, consider who will use it and the purpose for which it will be used.
One of the very best methods to obtain feedback on performance is through the use of survey instruments that give respondents an opportunity to speak directly and frankly.Subscribe Now for Access
You have reached your article limit for the month. We hope you found our articles both enjoyable and insightful. For information on new subscriptions, product trials, alternative billing arrangements or group and site discounts please call 800-688-2421. We look forward to having you as a long-term member of the Relias Media community.