Last month, guidance was provided to conducting a needs assessment and designing the field test as a means for selecting PPE. This month, the additional parts of the field test program are recommended for applying a systematic approach in making a defensible choice of your department’s PPE.
According to NFPA 1851, Standard on Selection, Care and Maintenance of Structural and Proximity Fire Fighting Protective Ensembles, 2008 edition, “Test participants should be selected based on a cross section of personnel, willingness to participate, objectivity and level of operational activity.”
This means you should select participants for your field test that include experienced members who know what to look for in proper functioning bunker gear. These participants should also have an open mind about all of the products being tested — you don’t want a participant in an objective field test who has already decided that one product is the only clear choice before beginning the evaluation.
Make sure that the participants are involved in all areas of the department’s operational activity and that they are willing and able to commit time to the process of testing the gear and completing the field test evaluation forms in a timely manner.
Kicking it off
The initial step in implementing the field test should be a kick off meeting. In this meeting the safety officer will provide the participants with all of the field test details to ensure they know what the objective is and how the test will be conducted. The risk assessment, field test schedule, participant requirements and an explanation of products being tested should be covered in this meeting.
Prior to this meeting, have the manufacturers of the products that will be tested measure the participants and then come in to fit the gear and instruct participants on how to properly wear the gear. This is the time to let the manufacturers know what will be expected of them and to provide rules that prevent undue influence in their effects on the field test (the products should speak for themselves).
The participants should only wear and retain one set of the test gear at a time to minimize confusion and the chance that all gear is not equally tested. Distribute the gear for the initial round of evaluations at the kick off meeting.
Evaluation forms
In accordance with the field test schedule, at preset intervals, the safety officer should provide participants with evaluation forms and have them completed. These forms must be filed for future analysis. This can be done in person or electronically.
The next gear in the rotation should not be provided until the evaluation forms are completed for the current gear being tested. Evaluation forms should be filled out throughout the test — NFPA 1851 recommends that each participant complete three evaluation forms per ensemble or product.
It is important that the safety officer establish a process to report events that fall outside the normal reporting procedures. This may include the need to repair or replace field test gear, any service-related issues associated with the gear or any critical information that comes to light as a result of the field test that may not be included on the evaluation forms. Once documented, it is added to the data used to compile the final report.
It is best to have the same participants throughout the test. If for some reason one participant needs to drop out of the test, it is better not to replace that participant. Also, discard any evaluation forms submitted by that participant unless you have a complete cycle of evaluations from that participant for all gear being tested.
If, based on a limited number of participants (less than four), you need to substitute field test participants, do so only at the beginning of a new cycle. Be sure that any substitute participants are of the same size, weight and height as the original participant to ensure proper fit.
After the test
The safety officer should collect and retain all field test gear once the test is completed so that the selection committee can examine all samples prior to developing a formal recommendation.
Once the official testing in completed, it’s time to compile the results and develop a formal recommendation. Prior to writing a formal recommendation, be sure to look at all of the information and results the field test has generated.
If the evaluation forms provide quantifiable scoring data, then it will be easier to calculate the results for each category tested. But do not limit the report to just the information generated by the evaluation forms.
The safety officer and PPE committee should also perform a thorough inspection of all sample gear used in the field test. Any findings associated with wear, component failure or workmanship should be included as part of the final review and resulting recommendations.
In addition, the safety officer should include any information pertaining to the field test that relates to service, delivery, repair or accuracy to specification in the final recommendation.
Making the recommendation
The most effective recommendations begin with an overview of the requirements that were developed based on the initial risk assessment. This sets the stage by letting readers know exactly what the gear committee was looking for and why.
This section is then followed by results of the field test by category tested. In most cases, departments tabulate all scores and then provide an average product score for each question on the evaluation form. Any participant comments are listed by manufacturer at the end of each section of the report. The overall organization of the report will be based on how the field test was conducted and how the evaluation forms were designed.
The final section should be a formal recommendation of a specific product that includes the rational of why that product was selected. This recommendation should form the basis for the development of a formal bid for the product tested. If your bid specification does not align with the results of your field test, then you are probably not going to get the product that best meets your department’s needs.
Field testing offers the best way for your department to assess the different brands available and to determine how well that gear will live up to the intended and expected performance. A properly designed field test that is quantitative, balanced and transparent allows your department to provide documentation for obtaining the gear that best meets the department’s needs.
