Do standardized order sets really improve care?

Tool is a "necessary evil"

Like all technology, standardized order sets have both pros and cons, says Larry Abramson, DO, MPH, quality director at POH Medical Center in Pontiac, MI. "There is good and there is bad," he says. "My thoughts on order sets are that they are a necessary evil at this point."

At POH Medical Center, order sets are used for many conditions and surgical procedures such as hip arthroplasty, knee arthroplasty, hysterectomies, and gastrointestinal surgeries. "Incorporating best practice, educating and training staff, and efficiently and effectively implementing and adjusting these sets remains a constant challenge," says Abramson.

It might take many years for evidence-based medicine to become part of routine clinical practice, but the use of standardized order sets can speed that process, says Abramson. "If it is done correctly and health care providers are involved in the development, then you have a better chance for the evidence-based practice to become reality," he says.

If you view standardized orders as the equivalent of a checklist, and apply critical thinking to the use of the order set, then they can definitely bring you closer to evidence-based practice. "But the caveat is you have to keep them current. Otherwise, they can come out of date rapidly and give you a problem," Abramson says. Order sets must allow customization to the individual patient, he adds.

Another potential problem is that order sets are frequently designed for only physicians' use. "There are a whole lot of individuals beyond the physicians that are providing care, however. Having their participation in development of order sets is essential, so that everyone is on the same page," he says. Involve nurses, pharmacists, physical and occupational therapists, patient educators, radiology technicians, registered dieticians, and other members of the care team.

When collecting data for core measure requirements, order sets can make it easier to show that a test was ordered. But to show that the test was actually done, you must go back into the record and collect the data manually or use electronic abstraction in the case of an electronic medical record.

For example, for the data element on whether the proper intervention was ordered to prevent deep venous thrombosis, the question, "Was it ordered?" is answered by the order set, but the answer to the question "Was it done?" is found in the nursing record. "So in terms of data collection, finding the order is a lot easier. But you still have to go back into the record to find out if it was actually executed," says Abramson.

The way an order set is written can leave you in noncompliance with core measure requirements such as stopping antibiotics within 24 hours. "So you may have to do some wordsmithing," says Abramson.

External agencies frequently change data definitions, which makes meeting evidence-based guidelines more difficult in terms of public reporting, explains Abramson. For example, an antibiotic may be approved as an acceptable drug to use as prophylaxis for bowel surgery, but not before a certain date. So the drug is incorporated into a hospital's evidence-based guidelines, but if it's used before that date, the provider will appear noncompliant. "They can't change the data dictionaries fast enough for some emerging guidelines. So that, indeed, causes a bit of a problem on the reporting side," says Abramson.

There is a tendency to think of order sets as merely a kind of word processor, but they are a lot more complex than that, says Abramson. "For example, changes on an order set have to flow through the pharmacy workgroup and nursing service to ensure that those changes are reflected on the medication administration record. It's not just a simple change once it's in an order set," he says. "So it speeds up the process initially, but slows things down on the maintenance side."

Quality professionals have tried to build choices and prompts into the hospital's post-operative order sets to drive the care as close to evidence-based practice as possible with high reliability, says Abramson. "It remains a daunting task, as the algorithms that drive the care must remain flexible enough to customize care for an individual patient requiring variation for valid clinical reasons or based on patient personal choice," he says.

Whenever a system is automated, there is danger in creating over-reliance on the infallibility of an "expert" system, says Abramson. "The caveat in these order sets remains that unless we ensure that it does not supplant or negate critical thinking on the part of clinicians, rather than improving care we may cause harm," he says.

Still, Abramson firmly believes that standardized order sets are essential to reducing variance. "But we must never lose sight of the importance of recognizing that these are tools, not actual practice," he says.

Perform due diligence

If you choose to purchase order sets from outside companies, you still need to perform due diligence with your staff to ensure it fits with the way you do the work and that staff accept the content as consistent with their clinical opinion, says Abramson.

"If you just accept somebody else's order sets and don't do any due diligence with them, you can end up with something that isn't as good as it should be, and can be potentially dangerous," says Abramson.

When order sets originate outside of a facility, whether purchased from a vendor or communally shared through inter-facility cooperation, each individual order within the set must be scrutinized as to either the value added to the process to conform to standards of care, standards of practice, patient safety, and promoting of desired clinical outcomes, he stresses.

"Order sets impact the practice of these health care team members, and ultimately the quality and safety of care provided to patients," says Abramson. Performing due diligence requires multidisciplinary teams that will provide or have their service impacted as a result of a required action within a given order set.

For example, pharmacists should review medication orders to ensure proper dosing, modifications necessary for formulary compliance, and the absence of medication interactions. "This avoids potential reproduction error, as has been demonstrated when a published protocol has a printing error — such as a chemotherapy protocol published a few years ago that had a misplaced decimal point leading to a significant dosing error on a cancer ward at a leading hospital," says Abramson.

Performing due diligence in this manner serves as a "failsafe mechanism," says Abramson. "Think of it as a failure mode effects analysis testing the capability of the system to accurately and reliably deliver the desired care by those charged with that responsibility," he says.

Order sets have helped significantly with compliance for cardiac and pneumonia measures, says Sue Smith, RN, BA, quality information analyst at Presbyterian Hospital Matthews (NC). "Those are the more time-sensitive diagnoses with time frames for antibiotics with pneumonia and aspirin and beta blockers for [acute myocardial infarctions]," she says. The sets also have improved compliance with ordering of appropriate antibiotics. "We no longer have physicians ordering whatever they feel is the most appropriate. They have the evidence-based ones listed on the order set for them to choose from," says Smith.

For the Surgical Care Improvement Project, post-operative order sets are used that are specific for physician groups, such as gastrointestinal surgeons. "We are trying to get to the point where we can have the appropriate antibiotics ordered preoperatively," says Smith.

As a 102-bed community hospital, hospitalists take care of the majority of inpatients. "We've had good buy-in from those physicians," says Smith. "We also have a set that that group uses for anyone with a cardiology admission diagnosis."

The organization was having a difficult time complying with the requirement to stop antibiotics within 24 hours post-operatively. "Some physicians were just writing to continue until a patient was discharged, without documentation of an acute infection, and that is one thing we are trying to get away from," says Smith. Since this was added to the post-operative order sheet, compliance has improved significantly. "We also got pharmacy to buy into the discontinuation so they are flagging those post-operative antibiotics. It all comes down to communication," says Smith.

A physician champion is the individual who handles issues with a physician being noncompliant, and also educates physicians before an order set is created that will impact their practice. "We want to be sure we have their buy-in, so our physician goes to their practices and speaks to them about that," says Smith. "This is a peer-to-peer communication coming from another physician. Sometimes nurse-to-physician communication is not regarded as highly."

[For more information, contact:

Larry Abramson, DO, MPH, Quality Director, POH Medical Center, 50 N. Perry Street, Pontiac, MI 48342. Phone: (248) 338.5510. Fax: (248) 338-5667. E-mail:

Sue Smith, RN, BA, Quality Information Analyst, Presbyterian Hospital Matthews. 1500 Matthews Township Parkway, Matthews, NC 28105. Phone: (704) 384-6241. Fax: (704) 316-7194. E-mail:]