Try These Strategies for Improving Review Consistency
Toyota lean methodology shows the way
Inconsistent reviews cause problems for IRBs and investigators. They can elicit complaints of unfairness. They might lead to workflow inefficiencies. Worse, they could be detrimental to human research protection quality.
One IRB office created simple rules and a process to improve workflow efficiency. Using Toyota lean methodology, the office standardized interpretation of regulations and institutional policies. (For more information on lean methodology, visit: http://bit.ly/2FgAYJy.)
The Seattle Children’s Research Institute has used lean methodology to improve its organizational processes since the early 2000s. The IRB employs the same strategy to increase consistency, improve workflow and efficiency, and to ensure high-quality study reviews.
“We try to follow a system of daily improvement,” says Tia Mynes, MS, CIP, human subjects protection analyst at Seattle Children’s Research Institute.
The IRB used reliable methods as a path to standardize work, says Nicholas Lew, JD, CIP, human subjects protection analyst at Seattle Children’s Research Institute.
“It’s the most effective way of doing something over time,” Lew says. “It boils down to figuring out what is the most effective, efficient method, and that’s how you implement it.”
For example, in the continuing review process, an IRB will check whether the study’s enrollment is accurate and whether it matches up with previous years’ enrollment, Lew says.
“If the current year’s report says the study is enrolling subjects, but the last year’s report said the study had completed enrollment, then you go back to the study team and ask them about it,” he explains. “The reliable method is to check out discrepancies.”
Simple and Reliable
Mynes and Lew offer the following strategies to improving IRB review consistency:
• Establish simple rules. The IRB refers to simple rules as strategy to document how its IRB makes decisions about each study so these thought processes could inform the next study with similar issues.
“The simple rules are for regulatory issues where there could be different interpretations, depending on how you read the regulations,” Mynes says.
“It’s nice to go back and find a written record, a centralized area where you know you can find documents instead of caucusing with people about what happened a year or two ago,” Lew says.
“One topic that came up recently involved the secondary use of data and specimens initially collected for research purposes and how do we handle that as an institution,” Mynes says. “Do we want to see consent forms to see that subjects consented to their samples being used in research? Or do we let the investigators request waivers to use those samples without really checking to see if the subjects said they could be used for secondary research?”
This is a topic that comes up at times, and IRBs struggle with it, she notes.
“We documented how we want to handle that in our simple rules,” Mynes says. “Our simple rules are basically a Word document where we have different categories of things, including an index, table of contents, and we make a regulatory interpretation.”
Then the IRB notes the decision about this issue, the background on the dilemma, and provides an example.
When a similar situation occurs, the IRB can go to these notes to see how it was handled previously.
“We try to be consistent in our language so it can be easily searched,” Mynes says. “We try to have a table of contents that’s descriptive so you can do a word search.”
Another topic documented through simple rules involves recording conversations, Lew notes.
“In simple rules, we’ve outlined what the law requires and we documented specific examples of what has been permissible with recording conversations in studies,” he says. “We reference a study with an audio recording, so when we see another study with a recording, we can see what has been allowed previously and compare it to the current study.”
• Use reliable methods. Reliable methods can be very detailed steps or more of an outline. They cover many different topics related to what IRBs do daily, Lew says.
“Reliable methods has helped new analysts by giving them a written reference point about information they are looking for, and it tells them how to review items that come into our office,” Lew says.
“Previously, we had standard operating procedures,” he adds. “Those give a general idea of what to look for, but they don’t get into much of the details, and having the reliable method gives us the details.”
Before using simple rules and reliable methods, new IRB staff would ask experienced staff with help in every new situation, he notes.
“Having this helps them find a central spot for answers, and we have more focused information on what to do,” Lew says.
It’s a way to impart institutional knowledge without relying on longtime staff members’ recollections, he says.
Improve Review Consistency
The IRB holds reliable methods meetings each week. This also is an institutionwide process.
“It’s a chance for people to bring questions about the process,” Mynes says. “On top of that, every morning, we get together as an IRB office, huddle, and use a whiteboard to show progress.”
The meetings focus on how much existing and new work is being handled in the IRB office. People can troubleshoot problems that arise. They also help each other with consistency issues.
“We discuss different topics, like processing continuing review,” Mynes says. “It helps us make sure we’re all doing things the same way.”
IRB staff and members follow the simple rules strategy to ensure consistency. Then they’ll bring questions about these to the reliable methods meetings, she adds.
Here is an outline of the steps they take to improve consistency of reviews:
• Step 1: IRB professionals use reliable methods when reviewing each new submission.
They ensure the study has funding and all conflicts of interest documentation. They check the submission for the following items:
- Does the protocol have all of the necessary sections?
- Does it meet criteria for full board review, expedited review, or is it exempt? If necessary, compare the study to the checklist for expedited reviews.
- Does the study have an unusual issue, and has that issue been handled by the IRB previously?
- If this issue was handled in a previous case, what does that previous study’s simple rules document say was discussed, and how was it resolved?
• Step 2: The IRB staff is in charge of pre-reviews, working on protocols, and getting them to the point where they can be reviewed and approved by the board.
“Then we make a decision about the study — and, since I brought up the issue, I document it in the simple rules,” Mynes says.
For example, Mynes might document the case with: “This was the situation for this type of study, and this is how we came to the conclusion.”
If the study needs to be reviewed and expedited, Mynes can handle it. But if it must go to the full board, she will provide background about why this study doesn’t fit the expedited category.
In some cases, IRB staff will need to contact the IRB chair about a logistics issue in a study. For instance, there might be a statement in the protocol that indicates someone will approach patients at bedside to ask them about enrollment, Mynes says.
“We’d ask the chair if there were any concerns about how they were doing this,” she explains. “Our IRB chair works at the hospital and could say whether it could work the way they’re describing it.”
These strategies of simple rules, reliable methods, and taking steps to ensure review consistency will work only as long as the IRB regularly audits and updates its documents, Mynes and Lew say.
“In simple rules and reliable methods, we have a place where we can look up information to see what we allowed before,” Lew says. “It helps to keep the institutional knowledge as much as you can, but it has to be documented and an organizational practice to do it.”
One IRB office created simple rules and a process to improve workflow efficiency. Using Toyota lean methodology, the office standardized interpretation of regulations and institutional policies.
Subscribe Now for Access
You have reached your article limit for the month. We hope you found our articles both enjoyable and insightful. For information on new subscriptions, product trials, alternative billing arrangements or group and site discounts please call 800-688-2421. We look forward to having you as a long-term member of the Relias Media community.