Take a risk management view of data compliance, cleaning
Take a risk management view of data compliance, cleaning
FDA regs making more stringent data demands
Often overshadowed by other pressing clinical research needs, the detailed, difficult process of data management is drawing closer scrutiny by the Food and Drug Administration, researchers are finding.
"What we do with data is more complex, regulatory oversight is more complex, and people are looking at data quality in a different way now," says Kit Howard, MS, CCDM, CRCP, principal and owner of Kestrel Consultants of Ann Arbor, MI. Kestrel Consultants works with pharmaceutical, biotech, medical device companies, and academic medical centers on topics of data management, standardization, and quality. Howard has spoken at various conferences on the topic, including MAGI Clinical Research Conferences.
"The FDA is beginning to pay a great deal more attention to data quality in a way the agency hasn't before," Howard adds. "The FDA is talking about applying risk management to data cleaning."
When sites apply risk management principles to data cleaning, they might clarify the risks with these questions, Howard suggests:
- What are the risks of not cleaning absolutely everything?
- How do you know what the risks are?
- How do you manage the risks?
- What are the chances that not cleaning something would lead you to different conclusions about the efficacy of the trial or safety of the subject?
It's very time-consuming and challenging for trial sites to meet all of the requirements of the FDA's 21 CFR, part 11, which covers electronic records and electronic signatures, notes William R. Clarke, PhD, professor of biostatistics, University of Iowa in Iowa City, IA. Clarke helped establish in 1989 and formerly directed the Clinical Trials Statistical & Data Management Center at the University of Iowa.
"The regulations are very complex and set a very high bar," Clarke says. "The 21 CFR, part 11, is about the quality of data."
Research sites have to keep a record of every action from the time data are collected to the time of analysis, Clarke says.
Despite the FDA's focus on data management, the research world might not be doing enough to ensure data are free of errors, says Mohammad H. Rahbar, PhD, professor of epidemiology and biostatistics, the University of Texas School of Public Health and director of the biostatistics/epidemiology/research design of the center for clinical and translational sciences in Houston, TX.
For instance, the Consolidated Standards of Reporting Trials (CONSORT) Statement that was developed in 2001 provides important standards for research publication, but it does not include requiring researchers to report the error rate in their data, Rahbar says.
"They assume there are no errors in the database," he adds. "Sometimes they don't do the quality checks they need to do to assess the error rate, and if it's not checked then there might be errors that are unreported."
What's needed is a mandatory requirement that research studies cannot be published until the error rate is reported, he says.
Research sites that handle data management properly might spend as much as 70% of their time in data cleaning and quality checks, Rahbar says.
Studies that involve research monitoring usually do a good job of data monitoring, but single sites are more problematic, he notes.
"We are very serious about data management," Rahbar says. "For example, yesterday for a stroke study, I told my data core staff of the regulatory requirements that have to be met and how we have to be compliant 100%."
The sponsor perspective
From sponsors' perspective, electronic data capture (EDC) provides a clear resolution process, resolves clean data issues quickly, and makes it possible for sponsors to view data at all points during a trial, says Linda Beneze, chief executive officer of KIKA Clinical Solutions of Boston, MA. KIKA sells EDC solutions to sponsors and others involved in clinical research and has released an updated version of Veracity, a collaborative platform for clinical trial research.
"This allows sponsors to make decisions about their drug or device faster," Beneze adds.
For sponsors and others involved in clinical research to get the most out of their data they need an electronic system that captures information carefully and accurately, but also contains flexibility, she says.
The goal of any EDC and data management is to collect data in a format that will lead to an investigational product being approved by regulators, Beneze notes.
"There are a million things going on in a clinical trial, including IRB review, recruitment," she explains. "And people get so busy with all the different activities involved in setting up a clinical trial that sometimes data management can be an afterthought."
Researchers think they should start a trial, enroll patients, and then use data management to fix any problems that occur. But this makes data management more difficult, Beneze says.
It's more efficient and simpler to define what the data export will look like at the beginning, before the trial starts, she says.
"I insist my project managers ask that data management and biostats be present at the study's kick-off meeting, so these managers can give input on defining data," she adds.
The vendor also trains people who will work with CR investigators and staff or meets with them directly to show them the correct ways to input data for the trial.
"We show them how to enter a form and how to use the system," Beneze says.
The electronic data system can be programmed from the beginning to provide suitable edit checks, and it's up to the user to determine how strong these checks will be, she says.
"The idea is to get the right balance with regard to edit checks," Beneze says. "When we're working with a first-time EDC user, we have to guide them a lot more and say, 'Based on this protocol, do you really want this to be a hard edit check?'"
In a hard edit check, the user won't be able to continue to input data until this question is answered. In a soft edit check, the system will have a pop-up note that says the information needs to be completed, she explains.
Electronic data management edit checks also can result in queries that are triggered a little too quickly.
It is possible now with electronic data to be overzealous in cleaning data, Howard notes.
"We've gotten to the point because of electronic trials that we can query absolutely everything, and we often do," Howard says. "This is leading to overburden on everyone involved from a time and cost perspective."
The goal has to be to focus on the value of data cleaning efforts.
"Some people have done a statistical analysis on uncleaned data bases and compared results with a cleaned data base and have seen no differences in outcomes," Howard says. "But that's just one piece of the story because there also are concerns about individual patient safety monitoring."
For example, there could be one incident of anaphylactic shock in a study enrolling 10,000 people. If that one incident was not recorded properly, then the omission would not have much of an impact on the conclusions of the safety of the trial, she explains.
"But from the perspective of protecting the rights and safety of individual patients, it matters very much," she says.
Plus, the FDA wants to know about that one incident, she adds.
The key is looking at data management from the risk management perspective, which is a whole new approach for the clinical research industry, Howard notes.
"This new approach is starting to evolve with respect to cleaning data, and this should be leading to quality data," she says. "But we need to determine what it means and how much is good enough?"
One direction this evolution is heading involves setting definitions for clinical research data.
"We have standardized coding for outcomes rather than open fields," Clarke says. "The National Institutes of Health (NIH) is working on standard nomenclature or panels of instruments to be used across their studies, which would help them assess comparability across studies."
The University of Iowa has its own standard coding, built simply over time and with experience, he says.
"What we've been doing with multiple studies that are around a clinical problem is build up a dictionary of terms and coding and standard assessments that we do in these studies," he adds. "That's evolved over 20 years."
The NIH initiative will expand this type of effort to achieve a future in which everyone will use common terms and common assessment, Clarke says.
Research sites do not need to develop standardized data definitions to be compliant with FDA regulations, but they likely will need to make sure their standard operating procedures (SOPs) are up to date.
"We're an academic unit, so even though we had SOPs we weren't very good about keeping them up, and they weren't very comprehensive," Clarke says. "We had to develop a complete and comprehensive set of SOPs for what we do."
There are dozens of these SOPs, and each one contains detailed sections. Titles include the following:
- FDA reporting requirements;
- Investigational product inventory management;
- Documentation and records retention;
- Clinical data management;
- Protected health information;
- System setup/installation;
- Software development;
- Software validation;
- Application change control;
- System security measures;
- Data collection and data handling (electronic data entry, data validation);
- Data backup, recovery, and contingency plans;
- Retention and protection of electronic records;
- Quality assurance audits;
- FDA inspections;
- Biostatistics project documentation and validation;
- Biostatistical project management;
- Generation and quality evaluation of the analysis database;
- Biostatistics deliverables based on study statistical analysis plan;
"We incorporated [input from] quality assurance personnel to make sure we're following our rules and to make sure our rules are current," Clarke says.
The institution started the process of becoming compliant with the 21 CFR, part 11 rules several years ago, but it took at least two years to become minimally compliant, Clarke says.
"We are continually improving our procedures and methods to make sure we are staying compliant," he adds.
Often overshadowed by other pressing clinical research needs, the detailed, difficult process of data management is drawing closer scrutiny by the Food and Drug Administration, researchers are finding.Subscribe Now for Access
You have reached your article limit for the month. We hope you found our articles both enjoyable and insightful. For information on new subscriptions, product trials, alternative billing arrangements or group and site discounts please call 800-688-2421. We look forward to having you as a long-term member of the Relias Media community.