Researcher: Measure the Quality and Efficacy of IRBs
Consortia could report IRB individual quality data, benchmarks
While it is intuitive that IRB oversight effectively protects human research subjects from ethical breaches and other risks, there is surprisingly little data on the quality and performance of review boards.
“Despite the importance of IRBs in protecting human subjects participating in research and the well-known benefits of performance measurements, there has been no systematic assessment of the quality and performance of IRBs,” the author of a recent commentary states.1
Although some argue that there are few metrics to measure the quality of IRB functions, the author states that “The IRB, with its well-defined missions, functions, structure, and procedures, should be readily amenable to performance measurements.”
Indeed, the lack of such assessments almost half a century after IRBs were established is striking, observes author Min-Fu Tsan, PhD, of the McGuire Research Institute in Richmond, VA. He argues that IRBs are bound by key tenets of the Common Rule, but focus on them somewhat haphazardly.
Tsan cites a prior study that found that protocol reviews by IRBs primarily discussed informed consent documents — failing to adequately address risk minimization, the risk-benefit ratio, equitable subject selection, data monitoring, privacy and confidentiality, and protection of vulnerable populations.2 Tsan proposes that IRBs and institutions form consortia for sharing agreed-upon quality measures that could be aggregated and reported back as benchmarks.
“Individual IRB data would be kept strictly confidential and only aggregate group data [would be] analyzed and published,” he states in the paper. “Each year, each IRB would be provided with its own data along with the group average so that individual IRBs can identify areas of vulnerability and carry out quality improvement.”
‘Complex Reasons’
IRB Advisor asked Tsan to comment further on this issue in the following interview, which has been edited for length and clarity.
IRB Advisor: What are some of the reasons for the lack of assessment of the quality and performance of IRBs?
Tsan: There are complex reasons for this. First, the IRB is a very important and powerful committee. Most members and chairs of the IRBs believe that they are doing a good job and there is no need for them to be scrutinized. So, there is no incentive for them to assess the quality and performance of their IRBs.
Second, some people believe that since the primary purpose of IRBs is to protect the rights and welfare of human subjects participating in research, if you are not measuring human subject protections, there is no point of measuring the quality and performance of IRBs. Third, conducting ethics reviews of human research protocols is a major function of IRBs. Many people equate IRB quality to the quality of IRB reviews. However, neither human subject protections nor the quality of IRB ethics reviews are well-defined. They are not readily quantifiable and measurable.
Finally, research institutions and IRBs also are concerned about the possibility of their IRB performance measurement data becoming public, which may adversely impact their ability to compete for external research funding, including participation in clinical trials. It was a combination of these reasons that led to the situation we are in today.
IRB Advisor: Can you cite some possible performance measures that may address this problem?
Tsan: I believe that the quality of IRBs should be judged based on how well and efficiently they conduct ethics reviews of human research protocols and how well they provide continued oversight of approved research.
The Common Rule, in implementing the ethical principles of the Belmont Report, has clearly defined what IRBs must do in order to approve human research protocols and conduct continuing reviews of the approved research. One could also measure outcomes, such as the lapse rates of IRB continuing review approval, and how often investigators continued research activities during a lapse. How often do investigators initiate research activities prior to IRB approval? How often are subjects enrolled into research protocols in violation of the inclusion and exclusion criteria? These are just some examples. I believe that there are no real barriers to implement IRB performance measurements.
IRB Advisor: What would be some of the benefits to human research protections if IRB performance could be assessed along the lines you propose?
Tsan: The primary purpose of performance measurements is for quality improvement. Systematic assessments of the quality and performance of IRBs will likely lead to improvement of IRB qualities. It is reasonable to expect that high-quality IRBs are likely to lead to better quality of IRB ethics reviews and better human subject protections, even though we do not yet know how to measure the quality of IRB reviews.
With the scheduled implementation of the recently revised Common Rule in January 2019, collecting IRB performance measurement data prior to and after the implementation will allow the opportunity to determine the impact of the revised rule. [IRBs can] evaluate whether the rule achieves its goal of providing more regulatory flexibility to investigators while improving protections to human subjects. Data on IRB performance will also guide future directions and changes in human subject protection policies and regulations.
REFERENCES
1. Tsan MF, et al. Measuring the Quality and Performance of Institutional Review Boards. J Empir Res Hum Res Ethics 2018 Oct 8:1556264618804686. doi: 10.1177/1556264618804686. [Epub ahead of print]
2. Lidz CW, Appelbaum PS, Arnold R, How closely do institutional review boards follow the Common Rule? Academic Medicine 2012;87:969-974.
While it is intuitive that IRB oversight effectively protects human research subjects from ethical breaches and other risks, there is surprisingly little data on the quality and performance of review boards.
Subscribe Now for Access
You have reached your article limit for the month. We hope you found our articles both enjoyable and insightful. For information on new subscriptions, product trials, alternative billing arrangements or group and site discounts please call 800-688-2421. We look forward to having you as a long-term member of the Relias Media community.