The trusted source for
healthcare information and
Quantitative tests needed to weed out poor products
Many respirators worn by health care workers evidently are constructed so poorly that they fit few people properly, say researchers at the Center for Disease Control and Prevention’s National Institute of Occupational Safety and Health (NIOSH).
In two separate studies — one published in 19961, the other still awaiting publication — laboratory researchers in NIOSH’s Respiratory Disease Studies Division found that many N95 half-mask filtering facepiece respirators perform poorly when tested on a panel of health care workers.
In a study published in 1998, a panel of 25 test subjects who were experienced in donning respirators tried on all 21 of the N95 respirator models available at that time. Three of the models failed to pass a subsequent fit test on any of the test subjects. In addition, 17 of the 21 respirator models tested had acceptable fit tests for fewer than half of the panel members.
"If you look at all 21 models, on average, only about 45% of the people successfully passed the fit test," says Chris Coffee, PhD, a senior research chemist at the laboratory research branch of NIOSH’s Division of Respiratory Disease Studies and lead researcher for the 1998 study.
The data from that study were originally published without naming the manufacturers. After a period of debate, NIOSH staffers made the decision to release the names of the manufacturers, which are published here for the first time. ( See Chart)
A second, still-unpublished study by Coffee and others that tests the generation of N95s now on the market shows the same poor level of performance, says Coffee. "Based on what we’re writing up now, the fitting characteristics [of currently available N95s] have not increased dramatically," he says. The new findings should be published sometime early this summer.
Many experts at NIOSH say they worry that the poor-fitting respirators pose a threat to health care workers. "The worker is the one who has the potential to lose here," says Paul Jensen, PhD, chief of the laboratory research branch. "There are respirators out there that truly don’t fit, but they’re still being sold. So instead of having superior-fitting respirators, we have respirators that rely on the employer to make sure they fit properly."
"When we look at current models, and we do the standard tests, we see very high failure rates," adds Donald Campbell, PhD, a research scientist in the same branch as Coffee. "We have to wonder: How can they sell such things?"
Campbell, Jensen, Coffee, and others interviewed for this story all say they’d like respirator manufacturers to be held more accountable for the quality of their products — perhaps by putting a fit-testing component back into the certification process.
"NIOSH in general would like to pursue a fit-test standard" for manufacturers, says Richard Metzler, acting director of NIOSH’s newly created National Personal Protective Technology Laboratory. "Our hope is to put [a fit-test] back into the proposed standard that we use in our respirator certification program."
That would have the effect of putting more responsibility for the fit onto the manufacturer, says Jensen. "We’d like to have it more balanced, so that workers are provided with a good respirator, and fit-testing becomes a double-check instead of the primary means of assurance," he says.
In the meantime, Metzler, Jensen, Coffee, and others at NIOSH say it’s especially important for employers to make sure their employees are adequately protected — and not just by fit-testing, but by using a quantitative fit test, not a qualitative one, they add.
Quantitative fit tests compare two measurements: the number of particles outside the respirator and the number inside. Qualitative tests, by comparison, rely solely on subjective reports of smelling or tasting a test agent introduced into a test chamber.
Data from studies by Coffee and others have shown that quantitative fit tests are much more reliable than the qualitative tests in widespread use throughout the health care industry.2 Manufacturing industries have moved much more swiftly than the health care industry in adopting quantitative fit-testing.
Coffee’s research shows that qualitative fit-tests are prone to produce false negatives; that is, the tests often show a poor fit for respirators that actually do fit well. Using the Bitrex test, a subjective fit test, Coffee found that in 41% of the tests, wearers flunked the test, even though they were receiving adequate protection. In addition, in 9% of cases, wearers passed the test but actually received inadequate protection.
Coffee’s forthcoming publication is expected to recommend that fit-testing programs in hospitals and other health care facilities switch from the qualitative to the quantitative method of fit-testing. That would have the effect of formalizing a longstanding preference at NIOSH for quantitative fit testing.
Given the widespread deficiencies of available N95 respirators, that’s the right thing to do, adds Jensen. "For now, we have to make sure the employer is doing fit-testing and doing it right," he explains. "So far, [research] has shown that quantitative fit tests are much more predictive of fit than quantitative ones."
Error rate compounded with re-testing
Other recently published work from the agency reinforces the importance of testing accurately and getting a good reading the first time out. In a computer simulation, Campbell and other researchers found that subjects who flunked two fit-tests and were switched to a second and then a third respirator had diminishing chances of getting a good fit, because the rate of error is compounded with re-testing, says Campbell, lead researcher for that study.3
Until 1995, the NIOSH certification process for respirators included a fit-testing component. That part of the certification process was thrown out because the test substance, a vapor called isoamylacetate, did not work well with particulate respirators, which had to be modified with a different, vapor-resistant material before tests using the substance could be performed.
Plus, one NIOSH staffer adds, some inside the agency figured that "if there were crappy masks out there, people simply wouldn’t buy them — or at least, they wouldn’t buy a second one."
Now, the mood at NIOSH has swung back, with many experts calling for more protection for the consumer. But a change in certification standards is probably a long way off, researchers concede. One reason is simply the enormous workload facing the short-staffed agency.
In addition, a logistical hump looms in the road: the question of whether the test panel used to assess fit is still relevant. Respirator manufacturers and NIOSH experts both agree that the panel may need to be updated, because subjects are chosen on the basis of facial types prevalent in 1972, the year scientists at Los Alamos National Laboratory in New Mexico took anthropometric measurements of face types considered typical of the U.S. health care worker population at that time. Since then, the composition of the health care work force has changed. Now, for example, there are more women and more foreign-born people working in health care.
In addition, some in the respirator manufacturing industry dispute whether the six exercises used during the fit test are relevant to tasks performed by health care workers on the job.
NIOSH researchers are preparing to undertake new anthropometric measurements, but that will take time, they say. Regarding the industry’s complaints about the relevance of the exercises, Coffee says he’s not convinced there’s a problem. "The idea is to stress the face seal," he notes. "I don’t know if the movements [in real life] are any more or less stressful, since there are no data one way or the other. We need to do a simulated workplace test, and then a fit test, and compare."
While workers wait on NIOSH for more information, it probably makes sense for the health care industry to develop its own system of certification for respirators, much as the fire-fighting industry has already done, says Coffee. "You set up your own program, basically, and leave NIOSH out of it," he says. "That way, you make sure that only good-fitting respirators get that seal of approval."
The bottom line, he adds, "is to start with a respirator that fits the majority of people — say, between 85 and 90% — and then use a [quantitative] fit test to find those without a good fit."
1. Laboratory performance evaluation of N95 filtering Facepiece respirators. MMWR 1998; 48:1045-1049.
2. CC Coffee, Z Zhuang, DL Campbell. Evaluation of the Bitrex qualitative fit test method using N95 filtering facepiece respirators. J Int Soc for Resp Protection 1998; Spring-Winter:48-55.
3. DL Campbell, CC Coffee, SW Lenhart. Respiratory protection as a function of respirator fitting characteristics and fit-test accuracy. AIHA J 2001; 62:36-44.