Every RMS and CAD system has them. Hidden within your dataset are oddball records ready to skew your averages and throw-off your accuracy. As a response analyst it’s your job to identify and dismiss these records before they wreck your reports. Does the number 1,440 mean anything to you? If you’ve worked with raw CAD data you’ll recognize there are 1,440 minutes in a day. The reason you will know this figure is that you’ve struggled to determine why a handful of your response time records show 1,434, 1,436 or 1,436 minutes rather than 4, 5 & 6 minutes.
Infrequently CAD operators stamp the “Time on Scene” before stamping the “Time of the Alarm”. When these time stamps are swapped the number of response time minutes is incorrectly calculated. In fact, they will be nearly 24-hours off.
If there are only a handful of records it may not seem like much of a problem. However, averages are calculated by adding-together the number of minutes each response takes to reach the scene. The number of responses is then divides that total. If your oddball data values totally misrepresent what’s being measured they will have a big impact on the total number of minutes as well as the calculated average.
Oddball data doesn’t have to be too high a value to skew results. Try searching your data for response times equal to zero minutes.
Zero-minute response times are appropriate when the emergency comes to the responding apparatus. Fire station drive-in emergencies or emergencies encountered by apparatus on the street are examples of appropriate zero-minute response times. But these events represent a very small number of responses in any CAD run. Usually CAD data includes many responses with zero response time minutes. So how did these extra zeros get there?
Some CAD systems record responses that never get dispatched. For example, a call is received and the apparatus assignment is keyed into the computer, but additional information causes the dispatcher to abort the response before a formal dispatch is made. Another more frequent example occurs when apparatus is cancelled before signing on the air or cancelled enroute. An obsolete incident reporting standard called for the “Time of Alarm” and “Time on the Scene” times to match when a dispatched vehicle never reached the scene.
Since any average report can be easily affected by odd-ball data a rule should apply. All odd-ball times need to be identified and eliminated before running average reports. This is where a proper search becomes essential. Consider the following searches:
Example #1 - All oddball records allowed
Incident Alarm Date is greater than or equal to 1/1/2004
AND
Incident Alarm Date is less than or equal to 12/31/2004
Example #2 - All oddball records excluded
Incident Alarm Date is greater than or equal to 1/1/2004
AND
Incident Alarm Date is less than or equal to 12/31/2004
AND
Response Time is greater than zero
AND
Response Time is less than 15
Notice the search used in Example #2 not only eliminates oddball records, but it also eliminates those very few legitimate zero-response time records. So why should it be used?
If normal response times are between 3 and 6 minutes, eliminating a few responses that were off by 3 to 6 minutes is not statistically significant. There’s very little impact on the total number of response time minutes.
Also, Example #2 anticipates some type of response when calculating an average response time. So it’s not statistically unfair to eliminate responses where there was no response.
Newer Reports Safeguard Data Integrity
Average reports are useful, but fewer agencies now rely on them as performance indicators. Let’s look at the major types of fire department measurements:
Average or Mean
Sum of values divided by the number of records. Is very sensitive to oddball values.
Median
Records ordered by value with the value “in the middle” taken as the median value. Is not sensitive to oddball values.
Fractiles
Splitting data into value groups with a count provided for each value group. (Example: Count by response time minute) Is not sensitive to oddball values.
Compliance
Pass / Fail testing of each value with a goal or standard. The percentage of “pass” records represents a compliance percentage. Is not sensitive to oddball values.
Averages are valuable to help you identify trends outside expected results. These reports work not only for response time, but for average duration, average dollar loss and other measurements as well. Whenever average reports are used more elaborate searches should also be used to eliminate data that tends to skew average results.
When representing your fire department’s performance it’s best to stick with fractiles and compliance reports. These reports not only give a more accurate picture they are now the type of measurements required to meet standards such as NFPA 1710 and those required by accreditation and Standards of Cover studies.
Locating and Dismissing Oddball Data
The fastest way to locate oddball data in NFIRS 5 Alive is to sort the value column up and down. For example, if you are doing a response time report sort the Response Time column up and down by pressing on the title of the Response Time column for each sort. By ordering records in this fashion all oddball responses will “float” to the top.
If a value field does not appear in the list view simply select the field for display in the field selection column. If you find the field you require is not on the field selection list you can add it to the list by pressing the “Search” button and going to “Quick Search”. At the bottom of the list of fields press the button to edit the field list. Now simply select the field you wish to add by clicking on it. When you return to the list of fields you will see the new field ready for selection. When displaying records in the field selection column you can initiate a sort by pressing the up and down arrows in the upper left corner of the screen. These arrows will sort records by the value of the field you selected.
Remember, searches can be built and stored in NFIRS 5 Alive. Create and store your searches in the Quick Search area. This will save a lot of time when working with more complex search criteria.
For additional information about analyzing fire department operations visit: www.nfirs5.com