Lean thinking can be well-suited for IRB office
Right time, right place, right fix
The philosophy of Lean thinking and processes has branched from manufacturing and business into healthcare over the years, and now some IRB offices are finding that these types of continuous improvement processes work well when used to create greater efficiencies in the human research protection world.
“Lean is about finding efficient ways to operate our organization,” says Ross Hickey, JD, CIP, CPIA, assistant provost for research integrity in the Office of Research Integrity and Outreach (ORIO) and director of the Maine Regulatory Training and Ethics Center at the University of Southern Maine in Portland.
Technology, including electronic IRB submission and record systems, have made it easier to follow Lean principles in recent years, says Liz Tioupine, CIP, senior system and process specialist in the human research protection program at the University of California, San Francisco (UCSF). Tioupine and Hickey spoke about Lean thinking and IRBs at the PRIM&R Advancing Ethical Research conference in Boston in November 2015.
“Having an electronic review system provides so much functionality and data, including time stamps and a robust workflow engine with tracking,” Tioupine says.
A main rationale for focusing on continuous quality improvement is the evolving nature of the IRB world: IRB office employees come and go; new researchers appear, and regulations and requirements annually are modified and changed.
“When you demonstrate a form and it worked four years ago, now those researchers are gone, and it may not work,” Hickey says. “So you have to go back to it.”
Another reason for Lean thinking is that IRB offices increasingly need to focus on efficiency and cutting waste, he adds.
“You look at the overall process and say, ‘Where’s the waste?’” Hickey explains. “Cutting waste is a key concept in Lean, but make sure what you’ve done isn’t just push waste upstream or downstream to people who will touch those projects either before or after.”
To think in terms of Lean, one has to think organizationally, he says.
“Organizations are like Jell-O: if you push one way someplace, it will squirm somewhere else,” Hickey says.
Any IRB that has engaged in process improvement projects has probably used some Lean strategies — if they knew it or not. The key is for the project to use robust data, to be continuous, and to be thoughtful in considering the effect of changes, Hickey and Tioupine suggest.
“Lean is about trying to get it right the first time,” Tioupine says. “It’s a data-driven improvement process.”
Hickey and Tioupine offer the following tips on improving processes through Lean:
• Drill down in data. Collecting turnaround time on IRB review submissions is a standard metric for IRBs. What an IRB director needs is to know how to interpret the data, Hickey says.
“You need to collect data to show over a period of a year whether or not most of the waste was during the time it was in our office or in the time it was in our customers’ hands,” he says. “The problem is that metrics can hide what the real waste is in a process.”
For example, if an IRB collects submission to decision turnaround time metrics and learns that the average turnaround time is longer than desired, then this suggests IRB processes need to be revamped. But if one were to dig deeper into data and measure time between various steps in the process, an IRB might learn that the submission is only sitting in the IRB office for a day or two and the real time drag is when the protocol is sent back to the researcher for requested revisions, Hickey explains.
“So is the problem really that your office is inefficient, or is it that researchers are not — for whatever reason — returning their protocols in a timely way?” he says. “We tried to find the metrics of each step of the process, looking at how long it was in our office and how much time it was with the researcher.”
This data drilling process resulted in the discovery that certain departments tended to slow down the process, he adds.
“So we came up with solutions to help those departments and improve their turnaround time,” Hickey says. “The data suggested the problem was in all of the back-and-forth: They miss some requested changes and amendments, and you have to send it back to them.”
Further investigation showed that some graduate students in the departments with slower turnaround times were confused about the IRB process, he adds.
“Their faculty advisors might not have been as familiar with the IRB process as they should have been,” Hickey says.
The solution was to show faculty members about which kind of projects take longer when they don’t have all of the necessary information and to provide IRB submission education to graduate students before they submitted studies to the IRB, he says.
“You want to take the right action at the right time and the right amount,” Hickey says. “You can overwhelm stakeholders if you give too much information or if you give them information too early or too late.”
• Make value-added changes. UCSF’s human research protection program studied the time to approval rate from 2010-2013 and found it was 45 days. “There were 45 days of lead time, and all of that time is pre-review corrections,” Tioupine says.
When applying Lean principles, they found that the pre-review correction time wasn’t value-added, she says.
“It’s not value-added to send back submissions for pre-review corrections because it was not changing the outcome of the committee review,” Tioupine says. “It didn’t lead to more approvals — we only approve 2% of studies at the meeting.”
The IRB’s culture is to respect all members’ opinions, and this led to addressing any and all issues raised by members. As a result, the IRB might find issues with submissions that were not directly tied to the criteria for approval, she explains.
“We’re working toward a culture of approval, of good enough,” Tioupine says. “It doesn’t have to be perfect because perfect is not a criteria for approval.”
Also, an analysis of pre-review corrections found that often these corrections involved design consultation and advice that investigators needed because the submissions were so undeveloped and poorly prepared that the IRB office had to help them prepare the application, she says.
“There was great variability between the types of corrections they were requesting,” Tioupine says.
A Lean analysis also identified quality issues, including the following:
- Poorly structured questions: There were certain questions that had a much higher incidence of needing corrections or requests for additional information, she explains.
That kind of finding can be used to revise the application, making it more user friendly, and to provide better guidance to investigators so they will complete it correctly the first time, she adds.
- Submission quality: A high percentage of submissions needed extensive work on the front end, Tioupine says.
“We’d had a rigorous screening process for a number of years,” Tioupine says. “We’d trained researchers to prepare something halfway, knowing we’d help them fix it. This might have been a self-fulfilling prophecy: The more effort we took in correcting things, the less they worried about it, thinking we’d tell them what to do.”
From an IRB workflow perspective, this was a major problem: IRB staff spent so much time working with certain investigators on their poorly-prepared submissions that investigators who had well-prepared applications were being short-changed, she adds.
One solution is to create submission standards and return submissions that fail to meet those standards. For example, these standards could require submissions to be complete and to have achieved scientific or feasibility approval.
“If something is missing, we send it back as incomplete,” Tioupine says.
After making changes, the percentage of submissions that went straight to the agenda rose from 15% to 40%, and the percentage of necessary corrections declined from 85% to 40%. After the intervention, 15% of submissions were incomplete and 5% did not meet submission standards, she says.
With 20% of submissions still not meeting minimum submission standards, more process improvements were needed.
“We received feedback from principal investigators and study staff, and we said, ‘What can we do to help you get it back faster?’” Tioupine recalls. “They said, ‘Ask for fewer changes: If you ask for three changes, I can do that right now; if you send a list with 22 changes, then I can’t do that now and will probably have to do it over several days in little time chunks.’”
Addressing this feedback, the solution was to ask only for big corrections, saving most for post-review, she adds.
The general idea of Lean processes from an IRB’s perspective is to create an effective, efficient review of protocols, protecting human subjects without creating undue burden for researchers, Hickey says.
“We have to think about not just what we do in our office, but how our actions impact others systemwide,” he adds. “That’s critical in Lean because if you’re focused on your one small spot in the stream, you’ll never get to the ultimate goal.”
The philosophy of Lean thinking and processes has branched from manufacturing and business into healthcare over the years, and now some IRB offices are finding that these types of continuous improvement processes work well when used to create greater efficiencies in the human research protection world.
Subscribe Now for Access
You have reached your article limit for the month. We hope you found our articles both enjoyable and insightful. For information on new subscriptions, product trials, alternative billing arrangements or group and site discounts please call 800-688-2421. We look forward to having you as a long-term member of the Relias Media community.