Most state and federal training regulations require that your members show competence in their firefighting skills but don’t define how to measure that competency. If someone goes to 100 calls and uses their equipment appropriately, does that count as competence? If a member goes to 100 hours of training, does that prove competence? What about basic versus advanced competency? There are a lot of questions around assuring competency, but the answers will shape your training program.
Competency can be defined as the ability of a member to demonstrate a defined level of knowledge, skill and ability to do a task at the time of measurement. Unfortunately this does not mean that they will be able to do the skill at a later time, but the hope is that if a member demonstrates competency on Monday then they can do it on Tuesday.
If there is going to be transfer from competency to practice, the measurement has to be reliable, valid, authentic and applicable to practice. If this sounds like a complex process, it is because it is and in some cases it may take a psychometrician and a practice analysis to prove that someone is truly competent. It may be easier to identify what is not a measure of competency than what is.
Common measure
Measuring the amount of ‘seat time’ or sheer hours of training may be the most common measure in departments, but does not measure or assure competency. We tend to measure training hours and attendance as it is easy but unless there is an associated examination or demonstration of competency that is documented, there is no way to tell if the member is competent or has learned the material.
We had all sat in a class, even for a full day session, and still realized at the end we did not learn anything. I always think of a group of firefighters who were required by contract to attend an EMT course. A majority of this group did show up for the class every day for the 120 hours as required, but they sat in class and read a newspaper, listened to their iPod and even slept. I feel bad for the instructor who had to deal with the students but it demonstrates that you have to require a demonstration of competency rather than attendance.
If you are going to measure competency the best bet is to use a standardized program and evaluation such as the Firefighter I curriculum and testing. Some departments have been reluctant to require formal programs for a variety of reasons but there is a standard because it has already been established and tested.
This does not mean you cannot go beyond the FFI program or adapt the training afterward but it does help to establish a basic level of competence. The fortunate thing is that the examination process for these formal programs are developed and validated and could be used for both initial and ongoing competency.
Modules can be used for smaller training initiatives with minimal work. The competency assessment process has already been done for you and will help make sure all of your members have a base level of competency.
If you want to develop your own competency assessment program, it can be done but you will have to do a lot of prep work. Everything will have to be meticulously documented. It all starts with documentation of what made the person competent to develop this whole program in the first place.
Valid tool
Next you need to make sure the tool is valid, such that it measures what it is supposed to and the competency transfers to practice. After the tool is valid, it must be reliable in that it measures the competency the same way every time and for every candidate.
Lastly you need to assure that re tool not only measures knowledge but also skills and abilities. This means just randomly making up a multiple-choice exam does not count as a true measure of assessment in the fire service.
It may sound like a lot of work to set up your own competency assessment program but it may be worth it. For example, you may be able to set up a program where a fire officer can document the skills shown on the fireground by a member to demonstrate competency and then have them ‘test out’ by taking a knowledge test and not have to attend further training.
In that case, your members may have responded to a structure fire wearing SCBA, done a search and rescue, and then extinguished the fire, therefore demonstrating competency in fireground operations. The member may then be able to take a knowledge test and skip the fireground basics drill and spend some well-deserved time with their family.
No matter how you chose to measure competency, the key is that you measure your members’ knowledge, skills and ability. This will assure you are providing a safe, professional service that meets state and federal requirements. Additionally your public can then feel competent that your department will do the right thing when called upon.