The trusted source for
healthcare information and
As increasing numbers of IRBs consider establishing an auditing and monitoring program, there is at least one trailblazer from which they can learn, and that is the University of Pittsburgh. The university’s research conduct and compliance office, which oversees the IRB, established in 1997 an extensive auditing program for protocols submitted to the IRB. "At the time, it was very unique to an academic medical center to have this type of program," says Kelly Dornin-Koss, RN, BSN, CCRC, education and compliance coordinator.
Initially, the program was developed in the Office for Research for Health Sciences. Then in August 1999, the research conduct and compliance office was created to separate compliance activities from the office of research, which is responsible for generating funds for research, she reports.
Dornin-Koss is in charge of an office of four employees who conduct all of the audits of human subjects research. One of the four staff members is in charge of education and computer-based training, and another is an auditor solely for an area Veterans Affairs (VA) medical center with whom the university is sharing office space and resources, she says. "Most of the investigators who do research in the VA also are faculty at Pitt, so it serves two purposes," Dornin-Koss says.
Since the program began, the standard operating procedures (SOPs) and audit forms have been revised and expanded numerous times. The auditors attended a lecture on the subject by the Greater Pittsburgh Chapter of Clinical Research Professionals and used their sample format in the development of SOPs, Dornin-Koss says. "These were created over time, and I did audits before developing the SOPs," she explains.
As the audit and informed consent process SOPs evolved, it became clear that the process of educating investigators and others also needed to expand and improve. "I went to the Public Responsibility for Medicine in Research meeting this past fall, and one speaker said the investigators should be aware of your SOPs," Dornin-Koss recalls. "We had them internally, but I didn’t think about making them available to investigators until he said that, and so we put them on the web site, and it’s been helpful."
Here is an overview of how the auditing program works:
• Selecting protocols. Auditors query the IRB database for protocols that have received federal or internal funding, an active approval status, IRB approval within the past five years, and have an expected subject enrollment of 10 or more people. Once these protocols are identified, auditors focus on protocols with moderate to high risk, Dornin-Koss says. "For high risk, we’ll look at it with 10 or fewer subjects, but we might choose a minimal risk protocol if there are 300-400 subjects enrolled," she explains. Other criteria considered during audit protocol selection are:
• Involvement of a gene transfer intervention.
• Requirement of periodic review by federal regulations (e.g., research protocols involving approval by the radioactive drug research committee).
• Research terminated by the IRB due to failure by the investigator to submit a subject for continuation of IRB approval.
• Involvement of multiple study sites. Occasionally, the IRB executive committee will instruct the office to conduct 100% audits based on a particular criteria. "A directive from the IRB executive committee last year told us to do audits on all gene transfer protocols, and we had about 10-15 of these," Dornin-Koss says. "Basically, these are all high risk, and they don’t enroll very many subjects."
• Auditing schedule and table. The program’s goal is to conduct one audit per week, but this will be difficult to attain because the audits are very detailed and time-consuming, she explains. In preparing an audit table, coordinators will assess eligibility criteria, screening procedures, study procedures, and follow-up procedures. "We develop audit tables that are specific to each protocol, and we are trying to find a way to do audits that are not so detailed," Dornin-Koss says. "Some of these protocols have 30-40 inclusion/exclusion criteria, and we look at every one of these and every follow-up procedure."
For example, some inclusion criteria may include:
"We look at every admission from 1997 forward, if the protocol was in place since 1997," she explains. "We take into consideration if there have been modifications."
• Selection and notification. One goal is to have every step of the audit in writing. Researchers are hand-delivered the audit notification that they must sign, Dornin-Koss says. Audits typically cover 20%-25% of the enrollment of a protocol that has fewer than 100 subjects; for those with 100 or more subjects, about 10% will be selected, she says. "If it’s a for-cause audit, then we do a 100% record review and a 100% informed consent review," Dornin-Koss adds. "We review the entire IRB file and look at when the protocol initially was submitted and look at the timeliness of the investigator’s responses to the IRB and if there are any lapses." When investigators are notified of the audit, they are asked to meet with the auditing coordinator within the next two weeks.
• Pre-audit interview. "We like to sit down and have a face-to-face conversation with the investigator or research coordinator or whoever they bring to this meeting," Dornin-Koss says.
At this meeting, which may last 30-60 minutes, auditing coordinators ask who is responsible for which protocol-related activity and how investigators handled subject recruitment. Other questions include:
• Audit report. Using auditing forms, coordinators document basic information about protocols, including:
"We ask for a list of all subjects enrolled to get unique identifiers and dates enrolled, and we look for any subjects who were enrolled during a lapse of IRB approval," Dornin-Koss explains.
The audit form mirrors the information that the auditors entered into the database after the audit, and includes inclusion/exclusion criteria, study procedures, adverse events, and the last section of the audit report is a description of the research records and study documentation. Altogether, the audit report could be anywhere from 10-50 pages in length, Dornin-Koss says.
• Identifying problems. "What we have found is that there is a certain level of background noise in the audits, and you’re not going to find everything just perfect," Dornin-Koss says. When problems are discovered, they are documented in the auditing reports and reviewed by the IRB executive committee, which makes the decision about how these will be handled, she reports. "They’ll ask the investigator to respond to the audit and list a corrective plan of action that they’re going to do, and it depends on how many mistakes we find throughout the audit," Dornin-Koss explains. "If we find several mistakes, the IRB executive committee might suspend enrollment."
The executive committee, consisting of the IRB chair, four IRB vice chairs, the director of research conduct and compliance, and legal counsel, also decides which problems should be reported to federal agencies, she adds.
• Reviewing consent forms. Auditors verify that the documentation of informed consent is performed according to federal policy and according to the university’s policies. Files are audited for the following documentation:
Auditors also will assess informed consent documents for these items:
Since the auditing began, there has been considerable improvement in the area of informed consent documentation, Dornin-Koss says. "When we first were doing audits, we’d see extemporaneous modification of the consent form, and that’s something we haven’t seen now for a couple of years," she reports. For instance, in previous years, an investigator might have crossed out the part that reads, "You have four visits," and written in, "You have three visits," and had the subject initial the change. "That’s wrong because every modification to the protocol has to have prior approval from the IRB, and the only exception is to avoid an immediate hazardous research incident to subject," Dornin-Koss explains.
• Other audit goals. "What we like to see is if the investigator has copies of all IRB correspondence and copies of correspondence with the sponsor," she says. "We like to see if they have a status data safety monitoring board; we’d like to see the minutes of that," Dornin-Koss adds. "We like to see a section on training, and our university has implemented requirements, so we have our own computer-based training module that everyone involved in human subjects research is required to complete." So every protocol submitted to the IRB must be checked to see if the investigator has completed the education.
Protocols that use drugs or medical devices must have records of receiving drugs and dispensing drugs and devices. This accountability should be a standard practice, Dornin-Koss says. "We have an investigational drug service as well, and they do audits also," she adds.
• Education and training. None of the auditing forms are copyrighted, and they can be viewed by anyone who is interested in seeing how the university’s auditing program works at the web site: www.rcco.pitt.edu. For example, here is a list of the links to audit SOP forms:
Then there are further links to the informed consent process SOPs: