Attend any large fire service conference and undoubtedly you will stroll through an exhibit hall with the latest and greatest from fire service equipment manufacturers. Among the exhibitors, you’ll find manufacturers of personal protective equipment as well as the suppliers of the many fabrics and components that go into their construction.
As is the practice within this and other industries, manufacturers tout their specific advantages through a variety of demonstrations, display products, and literature. A substantial proportion of this information includes data illustrating or comparing the product or material/components performance against different standards or competitive products.
These claims are based on the data provided by the manufacturer or supplier and the accuracy and relevance of this information is often pivotal to judging the particular product or material/component in question.
While many times decisions come down to preferences based on price, product design and hopefully the trial use or field testing, consideration of data and test results figures into the selection process for many organizations. The problems that ensue are about the integrity of the data and the ability to both interpret and compare these data to make informed decisions.
What many fire departments do not know is that these data may not be the same data used when certifying a product to a specific standard. While this may seem to be a trivial difference, the fact is that some data can be drastically different depending on where the testing is performed and even under whose control the testing occurs.
Available upon request
Standards such as NFPA 1971 for turnout clothing establish a series of tests that measure the performance of the product or the materials used in their construction against specific performance requirements. In order to be certified, the product must meet all these minimum requirements.
For some, it may be deemed adequate that just meeting certification is enough. Yet, for some there can be a certain level of interest for knowing by how much a given product or its constituent parts exceed the minimum requirements.
Herein lies the challenge for discerning product differences.
What many departments do not know is that manufacturers are required to provide all certification data that demonstrates the compliance to an NFPA standard upon request. The actual certification reports are quite voluminous and can be hundreds of pages given the complexity of all the different parts that make up the clothing and its respective associated data.
This is not what end users want. Instead, it would be much more practical to provide this information in a simplified tabular form specific to a given product that is based on the certification data for that product instead of test results that may come from other sources.
Data variances
Why? Well, by having one source for this information — the certification organization — some of the observed variances and data from different laboratories can be eliminated and thus provide a more consistent comparison of product performance.
There is this tendency to look at a given test result and believe that the reported number is the true value of the products performance. The truth is that test results can vary much more than many of us would expect.
Take the example of a very common test that is applied to turnout gear — thermal protective performance (TPP). This test judges the overall insulation of the three layer — outer shell, moisture barrier and thermal barrier — used to construct firefighter protective clothing against the conditions of a flash fire.
It is the benchmark test for discerning level of thermal insulation performance provided by specific clothing products. The minimum requirement set within the governing NFPA 1971 standard is 35. Therefore any value of 35 or greater is considered acceptable.
Yet, as fire service industry has recognized for the last 15 years, there are trade-offs between high TPP values and the overall burden that the clothing places on the wearer terms of heat stress and other physiological effects. Therefore, the TPP value for a particular clothing system is pivotal in the choice of a particular product.
Margin of uncertainty
The shortcoming of relying on the exact number of making a determination is that every reported value comes with a level of uncertainty. In scientific terms, this principle is referred to as measurement uncertainty.
Essentially, this means that every number that you see from manufacturer or supplier really has a plus or minus associated with it. For instance, a particular set of turnout clothing for which material composite had TPP value of 40, and the measurement uncertainty for this test was 10 percent, then the actual value of TPP for the clothing could be plus or minus four, or a range of 36 to 44.
Measurement uncertainty becomes important because in making comparisons from one product to another we tend to look at the differences in the test data. In the case of a clothing product with the TPP reported at 40 and another clothing product that has the TPP value of 42, there could be the inclination to judge the higher value as the better system.
Yet the range of data that would be expected for both clothing products would overlap considerably. A more statistically based conclusion would be that these systems are actually very similar and could be judged equivalent.
Data made easy
A more significant difference might be where the actual reported values are farther apart such as a system with the TPP of 40 versus a system having a TPP rating of 50. Applying a 10 percent rule, the range of TPP for the first system is 36 to 44, while the range for the second system is 45 to 55. These data do not overlap.
Thinking of data as having a certain level of uncertainty tends to make the differences in product performance seem less significant and has a large impact on product selection.
Also, if this approach is coupled with having consistent data from a limited number of sources that include only the certification organizations, then the ability to make decisions between products becomes more accurate.
Although NFPA standards require manufacturers provide certification data upon request, we advocate that additional requirements within the standards are needed to present these data in a more readable and easily interpreted fashion.
We expect some resistance to the proposal because not all product manufacturers control their data, particularly when it comes from multiple material and component suppliers. In addition, not all manufacturers will want these data shared because they fear that end users will overly scrutinize these data and not account for measurement uncertainty.
Yet, it is our firm belief that data that is consistent and properly presented can only help the industry and provide a fairer comparison of product performance than what is offered only in product literature.
