The term “measurement uncertainty” is a common expression that is applied in the laboratory testing business. It means that for any reported laboratory result, there is expected variation in that value.
As such, laboratories endeavor to identify and control the sources of variation that lead to possible differences between the measured or reported value and the “actual” or true result.
However, even when the causes of testing errors are identified, they cannot always be completely eliminated. Furthermore, there are often inherent variations in many measurements and the more complicated the procedure, the greater the amount of variation that can be expected.
Most individuals tend to forget that most measurements have some uncertainty. Even when you go to the doctor and are provided with medical lab results, there is a certain plus or minus to most numerical readings.
The fact is that all measurements have a degree of variation, and confidence in understanding whether the results are accurate can only come about when there is some way of refereeing the reported values.
Sometimes, for example, a doctor will tell you to have a test redone. If the results come back the same, the first test is confirmed. If the second test comes back with different results, then there is the uncertainty whether the first or the second test is correct.
Then there is always the case when you get a series of results and end up with one that is marginal. Does this mean that things have turned for the worse, or it is just a fluke result?
Impact on PPE
In many cases, it’s probably just the variability of the test. These same principles apply to protective clothing testing and certification, and they can have a huge impact on the PPE item selections your department makes.
We are often amazed by the exacting claims made both by manufacturers and users in the promotion and selection of personal protective equipment when it comes to reporting test results.
The two most frequently used test results for selecting protective clothing are the thermal protective performance (TPP) and total heat loss (THL) values. These tests results are provided for the three-layer composites (outer shell, moisture barrier, and thermal barrier), which are used in the primary construction of protective garments are entirely fabric dependant, having nothing to do with design or construction, .
TPP results are an indication of thermal insulation for protecting firefighters from high heat exposures, particularly under extreme conditions. THL values assess the ability of the clothing to permit the release of heat for alleviating the stress effects of clothing on individual firefighters.
The two results are diametrically opposed, where increasing TPP often comes at the expense of THL and vice versa. As many protective clothing manufacturers offer hundreds of different composites that combine multiple outer shells, moisture barriers, and thermal barriers, end users pay close attention to the TPP and THL measurements to help them as they navigate through the myriad of choices.
The TPP and THL measurements, like any others that are made in testing clothing, are subject to variability. This means that for any given reported number, say a rating of 40 cal/cm2 for the TPP result on a particular composite, there is an actual range of TPP values that exists for that exact composite.
After all, in general the reported result is usually the average of several measurements, which are rarely identical. For convenience, averages are used as a common statistical approach for reporting the value of a given property for a specific test.
NFPA standard
In NFPA 1971, the permissible variation of TPP values measured by a laboratory is 8 percent. This means that the 40 cal/cm2 of our example composite TPP can actually be anywhere from 36.8 to 43.2 cal/cm2 and still be considered a “good” test.
Note that the lowest TPP allowed by NFPA 1971 is 35 cal/cm2. For our example, the 36.8 cal/cm2 value comes fairly close to this minimally acceptable requirement.
Similar measurement precision also applies to THL measurements, though generally the variation is much wider. The test method on which THL is based includes information from an interlaboratory study where the in-lab and between-lab variation was measured.
This study found that the variation increased with increasing THL values. Thus, more variation in the results is expected for the higher THL values. The reported variation within the laboratory is between 8 percent and 10 percent, while the variation between different laboratories is between 12 percent and 14 percent.
If you applied this variation to a THL value of 280 watts per square meter and used the highest reported percentage of variation, the results within one laboratory could vary from 252 to 308 W/m2, while the results between different laboratories would range between 241 to 319 W/m2, nearly an overall difference of 80 W/m2 for the exact same three layer composite.
One question that arises most often is why is there so much variation? Other than just plain errors in equipment and technique, there are a number of reasons why one can expect difference in test results.
In general, these can be attributed to variations in the material samples that are tested, particularly when material layers are taken from different rolls of fabric. Even within the same roll of fabric, variations in the weight and/or loft can occur.
Small variations in the moisture barrier film have a tremendous impact on THL measurements. Differences in the batting or insulation layers of thermal barriers and how these layers are quilted together may also impact TPP results.
Both composite tests are influenced by how the three layers are put together to form a composite sample. Sometimes variations exist in the equipment and procedures used for the measurement.
Small differences
Even though certain parameters are set by the test and subject to calibration such as the heat exposure level for TPP, there can be small differences from test to test. If a particular equipment setting is off just a little bit, variations in results occur.
Likewise, most test procedures attempt to indicate detailed step-by-step products in the conduct of the test. Still, there are nuances in how the test is performed by different operators that contribute to the variation of results.
It is important to further acknowledge that there are variations in the test results reported to end users that may only reinforce the problem of testing certainty. Different sources can choose to report the data differently.
For example, in the testing of the numerous composites for both TPP and THL for certification purposes, each composite is tested using three replicate samples for the initial qualification.
Those results are the average of three replicates. Subsequent to that testing, manufacturers are required to test only one replicate annually if none of the materials layers have changed in any way. In addition, some manufacturers may choose to have their testing done somewhere else other than the certification test laboratory.
This means that there are now different potential sources for test results — initial testing, annual testing, and other source testing.
So, which number gets reported? The answer to that it is up to the manufacturer. Years ago, the NFPA considered requiring putting the test result on the label itself, but that practice was rejected as no agreement could be achieved for which number should be reported.
So where does this leave departments and end users who want to consider all of the different composites and make sense of the data in comparing one material system against another?
Must be judicious
Measurement uncertainty means that those responsible for selection must be judicious for how they determine differences between composites. For example, are TPP differences of 5 cal/cm2 significant? How about a difference of 25 W/m2 for total heat loss testing?
Based on the known variation of these tests, those differences may be within the “scatter” of the data. But even so, how does someone make an informed decision?
Our suggestion is that you know the source of the data. Generally speaking, if data you are comparing is all from the same laboratory and was based on tests performed at the same time, it is more reliable in being compared. Some manufacturers strongly suggest that in order to receive comparable data, the original certification value, which is based on the largest number of composite samples, should be the number reported.
On the other hand, if you are comparing test data from Lab A that was conducted last week to results produced by Lab B ten years ago, that confidence in being able to compare data lessens.
If your department or organization wants to compare products, try to get information about the testing itself – who performed the test, when was it done, was it based on an average of several samples, etc.
That way you will be able to make a more informed decision. In addition, we recommend you consider what differences in results are truly significant in getting better protection or stress release. For TPP, we suggest the minimum guidelines of 8 percent and for THL 12 percent to 15 percent.
However, for THL there is the benefit of the recommendations produced by the International Association of Fire Fighters, which in their study to support the use of THL testing for structural firefighting protective clothing, found that people could perceive differences in garment THL of 20 to 25 W/m2 while there was a 90 percent confidence level that a difference of 40 W/m2 for THL would produce a beneficial reduction in the rate of core temperature rise.
From this article, we want you to understand that there can be variability in test numbers you use to judge the performance of clothing. Sometimes, because of variability, small differences between test systems are meaningless, especially if you do know how and when the data was generated.
On the other hand, it is important to realize that data from a single source that is performed at the same time and done so in a reproducible manner can and does provide a reasonable basis for making a sensible comparison for ranking clothing systems.
Post Note — Please honor and remember the firefighters and others involved in responding to the horrific events of 9/11 and recognize the sacrifices made by those who risked lives on that day and during its aftermath.
