Don’t waste your measurement resources

Gathering long-term data can be time-consuming

By Patrice Spath, ART

Consultant in Health Care Quality and Resource Management

Forest Grove, OR

Health care providers have traditionally focused their outcome measurement activities on immediate and intermediate outcomes. But managed care and more difficult cases make it necessary to go beyond those measures.

Immediate outcomes emerge during or immediately after an intervention. Intermediate outcomes occur at the completion of an episode of care and can usually be associated with the health care activities during that episode. Capturing data regarding these outcomes is relatively easy because much of the information is contained within the internal records of one provider organization.

The ultimate results of treatment for a particular disease or condition — long-term outcomes — are becoming the focus of many outcomes management initiatives. Recently, interest in measuring patients’ long-term outcomes has grown because of the increasing incidence of incurable yet treatable chronic diseases. As providers assume greater financial liability for patients with chronic diseases, measuring the impact of various interventions is important. However, collection of long-term outcome data is resource-intensive because information is scattered among many provider sites and may require extraordinary data-gathering efforts.

Traditional long-term health care outcome measures generally fall into three categories:

• physical outcomes (biological, physiological, and functional components);

• service outcomes (satisfaction, timeliness, and access issues);

• financial outcomes (costs associated with health care processes).

Because quality of life is a personal perception, the results of these traditional long-term outcome measures must be supplemented by patient-reported outcomes. For this reason, measures of patients’ general health status and quality of life are also being used by health care providers to evaluate long-term outcomes. Data for these patient-reported outcome measures may be gathered with self-administered questionnaires or through clinician-patient contacts. Collection of this information requires a collaborative effort by caregivers throughout the continuum of care. Organizations must carefully select the measurement methods used to evaluate long-term outcomes; otherwise, they risk wasting resources.

Functional and health status measures

Most outcome measures inadequately account for the impact of patient function limitations or their perceptions of their health status. For this reason, many measurement instruments that look exclusively at functional and health status variables have been designed. Functional status measurement systems provide an assessment of the patient’s physical, mental, and social functioning. These "observable" behaviors have been shown to be predictive factors when determining patients’ post-treatment outcomes, including mortality. Health status measures use patients’ perspectives of their well-being to predict outcomes. An important predictor of a patient’s future health status is often prior health status.

Functional and health status measures can be broad-spectrum, generic (e.g., independent of the patient’s diagnosis) instruments. Other measures are specific to a disease (such as a quality-of-life scale for cancer), specific to a particular type of person (patient satisfaction scales), or to an age group. Specific measurement instruments are generally used to measure change following treatment. Generic instruments were originally designed for use by social science researchers in population-based studies. However, health care providers use these instruments to supplement the findings of their disease-specific measures.

The task of choosing worthwhile functional and health status measurement instruments is not easy. It is estimated that more than 100 measures of functional and health status now exist. Start the selection process by determining exactly what is required of the measurement. For example, will it be used to evaluate a program of care or to study the response of an individual patient? What type of person will be assessed (diagnosis, age group, level of disability)? What time frame will the assessment cover? How broad-ranging must the assessment be, and how much detailed information is required? For example, would a single rating of a patient’s pain level suffice or is a more extensive description of the type as well as intensity of the pain needed?

Consider the time element for completing the assessment. A patient in severe pain is likely to be uninterested in answering a lengthy questionnaire. Functional and health status measurement instruments vary in the number of questions asked and the time it takes patients to answer each question. Surveys with a few single-item measures of health and health-related concepts may be more appropriate for sicker patient populations. However, single-item measures are generally less precise, reliable, and valid than multi-item scales.

(Use the checklist on p. 69 to help define your long-term functional and health status outcomes measurement project.) The questions on this check sheet should be answered by those who will use the data. This may be a task group formed for a specific outcomes management project or the medical staff/hospital committee that will analyze the results.

Once the project’s purpose and scope are clearly defined, the users must determine what outcome measurement methods are available and which will meet their needs. This phase of the project begins with a literature search. For example, if the orthopedic surgeons wish to measure patients’ functional status post-joint replacement, they should be made aware of the various measurement instruments designed for this purpose. Some of the instruments will be generic and some will be disease-specific. Listed at the end of this article are three resources for quality of life measurement instruments. The further reading list includes many different measurement instruments and the articles that have been published about each instrument.

Medline searches, using the key words "quality of life" and the disease/condition being studied, also will yield a list of published articles on functional and health status measures.

Once the measurement instruments relevant to the topic have been identified, the project task group must select one more measurement methods. Their answers to the questions on the checklist will help narrow the selection decisions. In addition, articles and user manuals published by the measurement instrument developers should be reviewed to determine answers to the following questions:

• Is the measurement method appropriate for our intended use?

• Is the measurement method broad enough for the application, neither asking too many nor too few questions?

• How feasible is the measurement instrument to administer and how long does it take to complete?

• Is it clear how the measurement method is scored? Is scoring software available or must scoring be done by hand?

• What degree of change can be detected by the measurement method? Is this adequate for our purpose?

• How strong is the available evidence for reliability and validity? Has the instrument been tested for face validity, content validity, criterion or construct validity, predictive validity, and attributable validity?

Care on front end improves data’s value

Measuring long-term functional and health status is an important component of today’s outcomes management initiatives. However, gathering this data can be expensive in terms of human and information system resources. If the wrong measurement instrument is chosen, these resources may well be wasted. With numerous research efforts being directed toward the development of reliable and valid measurement instruments, it is foolhardy for a nonacademic health care organization to start from scratch. Clinicians should choose one of the already well-designed and validated measurement instruments.

Too often organizations pick long-term outcome measurement instruments that seem to be "popular" without an appreciation of the purpose of the tool. For example, the Alzheimer’s Disease Assessment Scale (Rosen, Mohs, and Davis) is a brief screening test that measures the overall severity of a patient’s dementia. It has proven useful as a measure of severity and as a general screening device. However, it was never intended to be used for other purposes (i.e., predicting patients’ future health care resource use or risk of mortality). A clear understanding of the purpose of the measurement project is essential to selecting the right method. If the purpose is known and the right measurement instrument(s) selected, caregivers are more likely to learn what they want to know about patients’ long-term outcomes without wasting valuable measurement resources.

Resources

Berzon RA. Quality of life bibliography and indexes: 1993 update. Quality of Life Research 1995; 4:53-73.

Spiker B, Molinek FR, et al. Quality of life bibliography and indexes. Medical Care 1990; 18 (suppl) DS1-71.

Spiker B, Simpson RL, et al. Quality of life bibliography and indexes: 1992 update. J Clin Res and Drug Development 1993; 7:203-242.

Related reading

DeLuca JM, Cagan RE. "Controlling for Patient Variables in Medical Effectiveness/Outcomes Management Studies." In: Spath P, ed. Medical Effectiveness and Outcomes Management: Issues, Methods, and Case Studies. Chicago: American Hospital Publishing; 1996.

Iezzoni LI, ed. Risk Adjustment for Measuring Health Care Outcomes. Ann Arbor, MI: Health Administration Press; 1994.

McDowell I, Newell C. Measuring Health: A Guide to Rating Scales and Questionnaires. 2nd ed. New York City: Oxford University Press; 1996.

Reiser SJ. The era of the patient: Using the experiences of illness in shaping the missions of health care. JAMA 1993; 269(8):1,012-1,017.

Rosen WG, Mohs RC, Davis KL. A new rating scale for Alzheimer’s disease. Am J Psychiatry 1984; 141:1,356-1,364.

[Editor’s note: Patrice Spath is a consultant specializing in health care quality issues. She can be reached at P.O. Box 721, Forest Grove, OR 97116. Telephone: (503) 357-9185. E-mail: hc_quality.msn.com]