Benefits of large databases in comparing outcomes
Benefits of large databases in comparing outcomes
'We wanted to be ahead of the curve'
Researchers began six years ago to determine whether use of a large primary care electronic medical record database would yield valid results regarding research questions. They found in a new study that using a new analytical technique does work well in making such databases useful.
"We decided to test this question because we anticipated that there would be a growing number of large databases that would develop," says Richard Tannen, MD, a professor of medicine at the University of Pennsylvania School of Medicine in Philadelphia, PA.
"We wanted to be ahead of the curve, understanding the value of databases long before they were broadly available," Tannen adds.
As researchers try to use these databases, there needs to be some proof that the answers are valid, Tannen explains.
"There really are two primary issues that could impair validity of results," he says.
These are as follows:
- Quality of data: "There is an issue with the quality of the data and database which covers a myriad of issues," Tannen says.
- Confounding bias: "There would be the problem with any kind of study using observational data, and it's a big problem with non-randomized, controlled trials, the potential that there would be unmeasured confounding bias influencing the results," Tannen says.
There are plenty of striking examples of confounding bias, he notes.
Perhaps the most notorious example is the estrogen/progesterone studies of treatment for post-menopausal women, Tannen says.
"Data that existed said if you treat women that way you reduce the incidence of heart attacks," Tannen says. "But when randomized, controlled trials finally were done, they found no evidence of a decrease in heart attacks, and, initially, they showed evidence that the treatment made heart attacks more prevalent."
When investigators went back and reviewed data more carefully they found that the increase in heart attacks affected women over age 70, and the study had a large number of women over age 70, Tannen says.
"The analysis of that study in my judgment should have been done much more carefully and not frightened everyone in terms of that particular adverse effect," Tannen says. "Nonetheless, the study clearly showed there was no reduction in heart attack risk with women treated with estrogens/progesterones."
Presumably, the observational studies found a reduction in heart attack risk because of some unmeasured bias, and this has been subject of a great deal of debate, Tannen says.
As the United States shifts to electronic health records with the potential or producing very large databases, it is necessary to find methods for eliminating confounding bias.
This is what Tannen and co-investigators sought to do with their study using a large British medical database.
"We tested something that was never tested before in a rigorous way," Tannen says. "We couldn't do it with standard observational studies."
So investigators decided to use the large database and try to replicate the design of previously-conducted, randomly-controlled trials, he explains.
"We replicated everything we could replicate aside from randomization and compared the results to the results that occurred in a randomized controlled trial," Tannen says. "We set out to determine whether or not using such databases and using standard analytical methods out there would give valid results."
The researchers found what they were looking for when studying statin drug treatment and cardiovascular risks.
"When we did that study we found indeed that the replicated study or database study showed a decrease in heart attacks like was found in the randomized, controlled trial," Tannen explains.
"It showed an apparent decrease in death like found in the randomized, controlled trial," he adds. "But in striking contrast to the randomized, controlled trial, the rate of coronary revascularization — angioplasty or cardiac bypass surgery — was dramatically higher in the group treated with statins versus the group not treated with statins."
A mysterious outcome
This was an unexpected and inexplicable result and investigators spend months puzzling it out with clinical epidemiologists and biostatisticians.
"We knew by looking at our data that the individuals that got treated with statins in our database study had a higher rate of revascularization before they entered the study than the group that didn't get treated with statins," Tannen says. "Even when we did standard statistical measures to try to correct for it, it didn't correct the results."
They spent several more months working on this dilemma.
"Then one day what should have been patently obvious to us popped up, and it led to development of a totally new way of analyzing observational study data," Tannen says.
The study wasn't adjusted for events that occurred before a person entered the study, he says.
"We went back in time prior to the study, which you can do with the database," he explains. "We knew how many events occurred before the study started and how many occurred after the study started."
This approach led to the finding that the event rate of coronary artery bypass surgery was not lower or higher than it was for the group treated with statins, Tannen says.
"It previously was twice as high," he says. "So this approach did not replicate, but was closer to the results found in the randomized, controlled trial."
The findings led researchers to develop the prior event rate ratio (PERR) adjustment to be used to correct confounding bias in large databases.
"When we applied that statistical technique to adjust the data/results we obtained during the study, it in fact modified the myocardial infarction results from both of those studies from ones that were totally different from the randomized, controlled trials," Tannen says.
PERR also produced better results for the incidence of coronary revascularization, he adds.
"It went from dramatically different to similar, and it tended to make even the results with stroke, some of which more divergent, more similar to the rate of randomized, controlled trials," Tannen says.
"The analysis with the PERR technique was significantly different from standard statistical results and similar to what was found in randomized, controlled trials," he adds. "We believe in a preliminary fashion that we have found a way of potentially overcoming the biggest problem that's out there with the results of observational studies."
Also, investigators have found that despite some of the shortcomings of using electronic medical record databases, these will work well as a data source for these kinds of studies, Tannen says.
"We've validated the fact that you could get apparently valid answers using this data source and using the new analytic technique," he adds. "That's a pretty exciting, a ground-breaking discovery."
Researchers began six years ago to determine whether use of a large primary care electronic medical record database would yield valid results regarding research questions. They found in a new study that using a new analytical technique does work well in making such databases useful.Subscribe Now for Access
You have reached your article limit for the month. We hope you found our articles both enjoyable and insightful. For information on new subscriptions, product trials, alternative billing arrangements or group and site discounts please call 800-688-2421. We look forward to having you as a long-term member of the Relias Media community.