Human research protection programs (HRPPs) are required to assess their quality, efficiency, and effectiveness if they’re seeking accreditation. HRPPs can do this through ongoing quality assurance/quality improvement (QA/QI) programs. But sometimes the quality improvement program also needs to be assessed for quality.
This was what one research institution decided. “Our assistant vice president asked that every program in research integrity and compliance — all 13 of us — conduct formal risk assessments,” says Julie Moore, JD, MS, PA, associate director of research integrity and compliance at the University of South Florida in Tampa.
“Each of the programs did a soup-to-nuts risk assessment, writing formal reports describing the processes they went to and their determinations about their levels of risk,” Moore says.
They also described processes that needed to be changed.
“I found the risk assessment process to be very valuable and thought it might be something other people could utilize in their programs — whether they are IRBs or QA/QI programs,” says Moore, who published a poster on the risk assessment project at PRIM&R’s 2016 Advancing Ethical Research Conference, held Nov. 13-16, 2016, in Anaheim, CA.
Moore researched risk assessments for QA/QI and found very little guidance. She and staff made a list of everything the office does and categorized items into bigger buckets of activity.
“We take a look at each category of activity that we do and look at related processes and procedures,” she explains. “Then we try to determine where there might be risk that we haven’t addressed well.”
For example, in the QA/QI program, activities can be divided into site audits and internal monitoring, as they relate to the IRB and HRPP. There are full routine audits and informed consent-only audits. IRB audits can be for cause or routine. They also can be audits of site records and electronic IRB system audits.
“We can go into our electronic IRB system and make sure all the approval letters contain the correct determination,” Moore says.
“For-cause audits of studies are selected either by the board, chair, or IRB administrator, and the process for conducting those is straightforward,” Moore says. “We looked at the most common findings for those audits and we tried to assess the sort of areas of risk on that end of it and how we could do a better job of educating study teams.”
Assessing the Assessors
Other changes could include revising HRPP policy to address areas of noncompliance.
“We also looked at all IRB record audits in previous years and identified the most common findings,” Moore says.
It might be a little confusing to think of conducting a risk assessment of a program — like QA/QI — that, by its nature, conducts risk assessments. But Moore and colleagues thought of the risk assessment of the QA/QI program in terms of an overall look at the program’s activities.
“Our intention was to do a detailed risk assessment of every program in research integrity and compliance,” Moore says. “But in the process of doing that, many of our program managers realized the formal process of conducting a research process is applicable to everyday activities done by each of our programs.”
Each of the 13 divisions had six months to perform their risk assessments. The end result — the risk assessment reports — were six to 10 pages.
“We have found them to be very helpful, for example, in justifying additional support that’s needed for one of our programs,” Moore says. “The risk assessment has served a dual purpose in that way: It’s not only forced us to look critically at our processes, but it’s also highlighted gaps.”
The risk assessment itself provides necessary evidence to support asking for more resources. “We can go up to senior leadership to say, ‘We did this gap analysis and have this area where there is a gap, and we need additional resources to close that loophole.’”
The risk assessment has been turned into a narrative report template. It starts with the executive summary, which is where the program staff can describe the program’s activities in broad terms, Moore explains.
There is space to list the risk assessment’s findings. Each is assigned a risk and impact score, with activities rated from low to medium to high risk.
“There’s an executive summary in that graph, followed by a narrative of risk assessment,” Moore says. “We break down each area and talk about the activities conducted in that area and the standard operating procedures we follow.”
They can provide justification for the impact scores and recommendations for addressing any areas of risk, highlighted in each area.
The most positive outcome of the risk assessment was the office’s ability to hire a full-time employee who is dedicated to monitoring the IRB, Moore says.
The goal now is to use the risk assessment annually, she says.
