Driver training simulation: Measuring a return on your investment
FDNY conducted a study on the effectiveness of its driver training simulation, yielding findings departments across the country can use
By Robert Raheb
FDNY EMS Lt. (ret), Emergency Response Specialist — FAAC, Inc.
The metropolitan New York City area encompasses more than 300 square miles and has more than 8.2 million residents within its five boroughs. The Fire Department of New York City Emergency Medical Service (FDNY EMS) has the responsibility of responding to more than 1.2 million calls for assistance each year.
To meet that demand, EMS averages 359 municipal ambulance tours daily and is augmented by hospital-based ambulances throughout the city, all of whom must maintain a state certification of EMT or Paramedic and are required to drive a department vehicle.
Unfortunately, while in the performance of these duties, they are sometimes involved in collisions themselves, ranging from a minor incident, such as a mirror being struck, to the more catastrophic intersection collision resulting in debilitating injuries and loss of services or even loss of life.
In an average year, EMS units are involved in roughly two collisions per day of various severity. Prior to 2004, intersection collisions were a significant portion of the collision total, even though all members were required to attend and successfully pass a 32-hour EVOC program.
Eventually the program was increased to 52.5 hours, or seven days of lecture and 18 range exercises on the dedicated cone course. Even with the increase, the EVOC program had no discernable method of effectively teaching collision avoidance in the intersection other than lecture.
Historically, EVOC has trained more than 300 new recruits per year since 1980 in a program that contained didactic and practical skills sessions regarding the handling of emergency vehicles.
During that period, collision rates remained steady, though several changes had occurred within the curriculum and the service. EVOC staff with the guidance of Chief J.P. Martin set out to first identify the type of employee that was most likely to have a collision.
At the end of 2003, after much study and comparison, we took delivery of our first computerized driver training simulator to complement the existing training program.
The driver training simulator enabled instructors to teach and reinforce specific driving behaviors he or she wants that driver to incorporate. Students practiced emergency response driving, roadway command principles and procedures, multi-tasking, and defensive maneuvers.
Through the city's internship program we interviewed and selected one summer intern who would work closely with the team and reported to the Lieutenant. The team consisted of one intern that retrieved, collated, and cataloged the data, a Lieutenant who managed the project, and a Division Chief who oversaw the project. The team performed a retrospective study starting with the year 2000 of all data based on the following:
- Citywide Performance Indicators
- number of assignments
- number of ambulance tours
- response times and segment responses
- Safety Battalion Citywide Accident Databases
- total collisions
- types of collisions (intersection vs. other)
- collision date and location
- length of service of the employee
- original date of appointment (ODA)
- age of the driver
The team examined 6,181 collisions that occurred from 2000-2007 and divided the information into two, four-year categories:
- 2000 – 2003 pre-simulator training
- 2004 – 2007 post simulator training
Over an eight-year period, the total number of assignments (runs) increased by 16 percent and the number of units increased by 8 percent. The total number of ambulance collisions slightly increased by 5 percent for the eight-year period (Figure 1), but the number of intersection collisions declined by 50 percent in the second half of the eight-year study (Figure 2).
Intersection collisions declined from 40 percent to 20 percent of the overall collision rate between the first and second four-year period. From 2000 – 2003 (without simulation training) the rate of intersection collisions hovered around 40 percent of the total number of collisions.
From 2004 – 2007 (with inclusion of simulation training into the curriculum) the rate of intersection collisions declined an average of 16 percent of the total collision rate and declined 15 percent per 1,000 runs. By 2007, intersection collisions had declined to 11 percent of the total collision rate — a drop of nearly 75 percent from the first year.
The team also compared collision rates among drivers; comparing employees with less than five years on the job (YOJ) to all other employees. The team identified that intersection collisions were three times more likely for employees with less than five years on the job and that the total collision rate was nearly twice as high for the new drivers when compared to all other drivers.
Other study factors included student ages and the amount of driving experience they had prior to coming to EMS. A majority ranged from 18 – 25, and the rest were under 41 years. Eighty-nine percent of the students had never driven an emergency vehicle before coming to EMS. Twenty-six percent failed the program on the first attempt, requiring them to attend a remedial program that was longer in duration and smaller in class size (Figure 3).
The study concluded that 30 percent of the students attending the program averaged between 0 – 5000 miles driven per year, while 31 percent drove 0 – 500 miles per year, 21 percent drove 501 – 1000 miles per year, and 48 percent drove 1001 – 5000 miles per year (Figure 4).
New drivers were 34 percent more likely to be involved in a non-intersection moving vehicle collision (MVC). Intersection MVCs were three times higher for new drivers. New drivers were responsible for half of the intersection collisions, but were only 35 percent of all EMS drivers.
In October 2003, FDNY EMS EVOC took possession of a driver training simulator system made by FAAC, Incorporated. The high fidelity of the system enabled students to suspend their disbelief of driving in an artificial landscape and become immersed in the training.
The simulator was made from the cab body of a Ford F350 to resemble the fleet's ambulances and other various vehicles. It was equipped with three front screens and two screens mounted behind the cab to represent the landscape in the two physical mirrors attached to the cab doors (photos 1, 2). Motion devices underneath the seat replicated real vehicle movements and added to the realistic experience for students.
Using scenario development software from FAAC, the EVOC program developed a curriculum that enabled the student to build upon his or her training, starting with vehicle acclimation and vehicle dynamics and culminating with multi-tasking scenarios that included high risk, low frequency situations.
Some of the training content delivered included:
- vehicle dynamics and depth perception
- map reading and radio communications
- roadway command
- intersection analysis
- vehicle placement at the scene
By introducing students to the unique hazards associated with operating an emergency vehicle in a variety of situations, the program was able to provide real-life experience in a controlled atmosphere that was repeatable, recordable, and immediately available for playback, critique, and instruction.
The student was able to apply a real-world application to theory and concepts taught in the lecture portion. A large benefit to the simulator was the time-saving factor — all the information given in the lecture could take up to eight hours to disseminate, whereas the practical application, critique, instruction, and having the student re-drive the scenario could take up to 15 minutes.
Perhaps the simulator's largest benefit is that students can crash; this may seem counter-productive, but the truth is all the students learned more from their failures than from their successes in the simulator (figures 5, 6).
Simulation training enabled the students to immerse themselves into a virtual world and apply the theoretical concepts learned in the classroom. The decision process is performed in real-time and with real-life situations and protocols; when the student is involved in the 'collision' they must report to the 'dispatcher' the same information that would be required in the field.
The instructor also elicits information from the student using a Socratic approach as to why the collision occurred, what were the events leading up to the collision, and the resulting short- and long-term consequences. This brings a more tangible understanding to the student's actions.
Students of the 21st century no longer depend on traditional methods of instruction, such as books and lectures. Instead, most depend on electronic media as the most efficient delivery system of information. When educators are faced with the challenge of providing simultaneous multiple inputs of information, electronic media are the optimal routes that can stimulate many of the senses and enhance the learning process.
Several studies have identified that driver behavior has a larger emotional than analytical component, and that 80 percent of the driving experience is composed of attitude. In other independent studies documented in medical journals, it was concluded that the frontal lobe of the brain does not fully mature until the age of 25 years. The frontal lobe is responsible for the control of:
The factors in Table 1 are the reasons that simulation training helps in teaching the new driver — we were able to stimulate and enhance the learning process.
In the FDNY study, we identified that more than half of the student population fell into this category. By being able to connect with them on a more personal level while maintaining an educational experience, students felt they had more of an investment in their careers and control over their outcome while driving.
Similar departments have had significant results by including simulation into their driver training programs.
Across the country, studies on the effectiveness of simulator usage are beginning to trickle in and the results are consistently positive. In fact, in many departments, simulation training for fire apparatus operators has become the benchmark.
Couple that with the generational trending of new hires' expectations for their learning, and driver training simulation is worth a serious look.
1. US Census Bureau, 2006 estimate
2. FDNY Vital Statistics; www.nyc.gov/fdny
About the author
Robert Raheb has been in the EMS field for 31 years and currently is the emergency response subject matter expert for FAAC, Incorporated. As a Firefighter in California, he became a paramedic working in NYC for 27 years and a NYS Instructor Coordinator for 21 years. Introduced to simulation training in 2003, Rob Raheb discovered he had an intuitive skill creating effective simulator training curricula. Realizing the benefits and potential training abilities this high-tech tool held, simulation training has added a new and exciting dimension to his vehicle training program and those benefits were obtained with a 38 percent reduction in intersection collisions within the first year and a steady decline every year since. Robert can be reached at firstname.lastname@example.org.