Abstract & Commentary: Does the Use of Protocols in the ICU Interfere with Learning?
Abstract & Commentary
Does the Use of Protocols in the ICU Interfere with Learning?
By David J. Pierson, MD, Editor, Professor Emeritus, Pulmonary and Critical Care Medicine, University of Washington, Seattle, is Editor for Critical Care Alert.
SYNOPSIS: This study of first-time takers of the American Board of Internal Medicine's critical care certifying examination found no relationship between performance on questions related to mechanical ventilation and the intensity of exposure to mechanical ventilation protocols during fellowship training.
SOURCE: Prasad M, et al. Clinical protocols and trainee knowledge about mechanical ventilation. JAMA 2011;306:935-941.
Prasad and associates conducted a retrospective cohort study of associations between internal medicine trainee exposure to mechanical ventilation protocols and their performance on questions related to this topic on the critical care board-certifying examination. These same investigators had previously conducted a survey on the availability of mechanical ventilation protocols in the ICUs of U.S. teaching hospitals.1 In this study, the authors used data from that survey to assess critical care fellows' exposure to protocols in the ICUs in which they trained, matched with performance data on questions pertaining to mechanical ventilation on the American Board of Internal Medicine's critical care medicine certifying examinations administered in 2008 and 2009. Only the examination results from intensivists from training programs that participated in the previous survey were included, and only data from first-time test takers were used.
Of the 88 critical care medicine training programs whose graduates took the certifying examination, 27 (31%) had no protocols for ventilator weaning, sedation management in ventilated patients, or lung-protective ventilation for managing acute lung injury. Nineteen programs (22%) had 1 such protocol, 24 (27%) had 2, and 18 (20%) had 3 protocols for at least 3 years. Programs were designated as providing high-intensity protocol exposure if they had 2 or more protocols in place for at least 3 years (42 programs, 48%), or low-intensity protocol exposure if they had 1 or 0 protocols (46 programs, 52%).
Of the 778 examinees for the 2 years, 553 (71%) were included in the study by virtue of having trained at a program responding to the previous survey. These examinees were statistically indistinguishable from the others with respect to performance on the examination. Overall, 91% of the examinees in the study passed, and no differences were detected in performance on mechanical ventilation-related questions with respect to protocol exposure during training. Multivariable analysis for potential confounders, such as examination year, exam score, and country of birth and residency training, showed no differences associated with mechanical ventilation knowledge. Thus, the investigators were unable to demonstrate any evidence of a detrimental effect of protocol use on trainee learning with respect to mechanical ventilation.
COMMENTARY
The barriers between evidence and actual practice are numerous and complex, and protocols are one acknowledged means for overcoming them.2,3 Protocols have been shown to decrease practice variation in numerous health care settings and to be associated with improved patient outcomes for several aspects of critical care, including mechanical ventilation. Many protocols empower non-physicians (such as nurses and respiratory therapists) to assess patients for manifestations of illness and responses to interventions, and to initiate, adjust, or discontinue therapy within boundaries established by the institution for the protocol. By turning moment-to-moment decision-making over to clinicians other than physicians, it may be hypothesized that such protocols remove a level of involvement for physician trainees and thus interfere with learning. This study tested this hypothesis and found solid evidence to refute it.
Only one previous study has looked at this issue. In 2000, Stoller and colleagues published a study of medicine house-officers' knowledge of respiratory care practices in two academic centers, one in which respiratory care protocols were extensively used and one without such protocols.4 Internal medicine residents were studied at the Cleveland Clinic, where multiple respiratory care protocols had been used for years and were incorporated into trainee education, and also at the University of Nebraska, where there were no such protocols. Residents representing all 3 training years answered multiple-choice questions related to whether various respiratory care modalities should be administered to the patients in 5 clinical vignettes. The trainees at both institutions got about three-fourths of the questions right, with no detectable differences between the training programs except that residents at the Cleveland Clinic scored statistically better on the questions relating to one of the vignettes.
Only about a third of each program's residents were included in the study, and there are likely many differences between the trainees and the programs at the participating institutions, in addition to whether respiratory care protocols are in use. In addition, the Stoller study4 dealt with respiratory care treatments not involving mechanical ventilation. Nonetheless, it represented the first attempt to address concerns about potential adverse effects of protocolized care on trainee learning.
The present study by Prasad et al focused on mechanical ventilation, employed a much more rigorous study design, and examined trainee knowledge much more comprehensively. It makes an important contribution to education in the critical care setting by reassuring all concerned that the use of protocols does not interfere with trainee learning. Most likely, institution-specific, evidence-based protocols that are tailored to local patient populations and practice patterns actually facilitate learning, by permitting trainees to observe best practices and reducing the opportunity for adverse consequences of a more trial-and-error approach.
REFERENCES
- Prasad M, et al. The availability of clinical protocols in US teaching intensive care units. J Crit Care 2010;25:610-619.
- Cabana MD, et al. Why don't physicians follow clinical practice guidelines? A framework for improvement. JAMA 1999;282: 1458-1465.
- Pierson DJ. Translating evidence into practice. Respir Care 2009;54:1386-1401.
- Stoller JK, et al. The impact of a respiratory therapy consult service on house officers' knowledge of respiratory care ordering. Respir Care 2000;45:945-952.
Subscribe Now for Access
You have reached your article limit for the month. We hope you found our articles both enjoyable and insightful. For information on new subscriptions, product trials, alternative billing arrangements or group and site discounts please call 800-688-2421. We look forward to having you as a long-term member of the Relias Media community.