Highway Safety: Improved Monitoring and Oversight of Traffic
Safety Data Program Are Needed (04-NOV-04, GAO-05-24).
Auto crashes kill or injure millions of people each year.
Information about where and why such crashes occur is important
in reducing this toll, both for identifying particular hazards
and for planning safety efforts at the state and federal levels.
Differences in the quality of state traffic data from state to
state, however, affect the usability of data for these purposes.
The National Highway Traffic Safety Administration (NHTSA)
administers a grant program to help states improve the safety
data systems that collect and analyze crash data from police and
sheriff's offices and other agencies, and the Congress is
considering whether to reauthorize and expand the program. The
Senate Appropriations Committee directed GAO to study state
systems and the grant program. Accordingly, GAO examined (1) the
quality of state crash information, (2) the activities states
undertook to improve their traffic records systems and any
progress made, and (3) NHTSA's oversight of the grant program.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-05-24
ACCNO: A13372
TITLE: Highway Safety: Improved Monitoring and Oversight of
Traffic Safety Data Program Are Needed
DATE: 11/04/2004
SUBJECT: Comparative analysis
Data collection
Data integrity
Evaluation criteria
Federal aid for highways
Federal/state relations
Grant monitoring
Grants to states
Highway safety
Information systems
Traffic accidents
NHTSA 411 Grant Program
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-05-24
United States Government Accountability Office
GAO Report to Congressional Committees
November 2004
HIGHWAY SAFETY
Improved Monitoring and Oversight of Traffic Safety Data Program Are Needed
On page 45, figure 9 was incorrectly displayed in a previous Web version
of this report. This figure has now been corrected. This error will not
appear in any printed copies of this report.
a
GAO-05-24
[IMG]
November 2004
HIGHWAY SAFETY
Improved Monitoring and Oversight of Traffic Safety Data Program Are Needed
What GAO Found
States vary considerably in the extent to which their traffic safety data
systems meet recommended criteria used by NHTSA to assess the quality of
crash information. These criteria relate to whether the information is
timely, consistent, complete, and accurate, as well as to whether it is
available to users and integrated with other relevant information, such as
that in the driver history files. GAO reviewed systems in 9 states and
found, for example, that some states entered crash information into their
systems in a matter of weeks, while others took a year or more. While some
systems were better than others, all had opportunities for improvement.
States reported carrying out a range of activities to improve their
traffic safety data systems with the grants they received from NHTSA.
Relatively little is known about the extent to which these activities
improved the systems, largely because the documents submitted to NHTSA
contained little or no information about what the activities accomplished.
The states GAO reviewed used their grant funds for a variety of projects
and showed varying degrees of progress. These efforts included completing
strategic plans, hiring consultants, and buying equipment to facilitate
data collection.
NHTSA officials said their oversight of the grant program complied with
the statutory requirements, but for two main reasons, it does not provide
a useful picture of what states were accomplishing. First, the agency did
not provide adequate guidance to ensure that states provided accurate and
complete data on what they were accomplishing with their grants. Second,
it did not have an effective process for monitoring progress. The agency
has begun to take some actions to strengthen oversight of all its grant
programs. If the Congress decides to reauthorize the program, however,
additional steps are needed to provide effective oversight of this
particular program. GAO also noted that in proposing legislation to
reauthorize the program, one requirement was omitted that may be helpful
in assessing progress-the requirement for states to have an up-to-date
assessment of their traffic data systems.
Source: American Association of State Highway and Transportation
Officials.
Example of police car computers used to speed crash information into state
traffic data systems.
United States Government Accountability Office
Contents
Letter
Results in Brief
Background
Quality of Data Systems Varied Greatly, with All States Examined
Showing a Need for Improvement States Carried Out Various Activities Using
411 Grant Funds, but Little Is Known about Progress NHTSA's Limited
Oversight of the 411 Grant Program Contributed to
Incomplete Knowledge of How Funds Were Used Conclusions Matter for
Congressional Consideration Recommendations for Executive Action Agency
Comments and Our Evaluation
1 3 6
10
20
26 31 32 32 32
Appendixes
Appendix I: Appendix II:
Appendix III: Appendix IV: Objectives, Scope, and Methodology
Additional Analysis of Data Quality in NHTSA's State Data System
Variations in Reporting Thresholds Impact the Usefulness of Data in
the State Data System Variations in Reporting Alcohol and Drug Data
Researchers' Use of Another Database Omits Data on Nonfatal
Crashes
Examples of Federal and Other Efforts at Improving Traffic Safety Data
GAO Contacts and Staff Acknowledgments
GAO Contacts
Staff Acknowledgments
34
37
37 39
45
47
48 48 48
Tables Table 1: NHTSA Recommended Criteria for Assessing Quality of
Crash Information 11
Table 2: Extent of Data Integration with Crash Files in the 9
Case-study States 17
Table 3: Status of States in the 411 Grant Program, 1st and 4th
Year 19
Table 4: Examples of Reported Activities Drawn from Available
Documents of Participating States 21
Contents
Table 5: Comparison of MMUCC Guidelines and Crash Information Provided to
NHTSA by 5 States Regarding Alcohol-and Drug-Impaired Driving 41
Figures Figure 1: Figure 2:
Figure 3:
Figure 4: Figure 5: Figure 6: Figure 7: Figure 8: Figure 9:
Model State Traffic Safety Data System 7
Year of State Traffic Safety Data That Were Used to
Develop 2005 State Highway Safety Plans 12
Extent to Which Vehicle Identification Numbers Were
Included in Data Reported by 17 States in the SDS
Program 14
Examples of Reported Activities Drawn from Available
Documents of Participating States 22
Types of Expenditures under the 411 Grant Program in 8
Case-study States 24
State Criteria for Filing a Police Crash Report for
Property-Damage-Only Crashes 38
Extent to Which States Collected Information about
Uninjured Passengers 39
Percentage of Alcohol Test Results That Were Coded as
Missing for 1998 and 2002 44
Percentage of Alcohol Test Results That Were Coded as
Unknown for 1998 and 2002 45
Contents
Abbreviations
CODES Crash Outcome Data Evaluation System
DOT Department of Transportation
FARS Fatality Analysis Reporting System
FHWA Federal Highway Administration
FMCSA Federal Motor Carrier Safety Administration
HSIS Highway Safety Information System
MMUCC Model Minimum Uniform Crash Criteria
NCSA National Center for Statistics and Analysis
NHTSA National Highway Traffic Safety Administration
SDS State Data System
TEA-21 Transportation Equity Act for the 21st Century
TraCS Traffic and Criminal Software program
TSIMS Transportation Safety Information Management System
VIN vehicle identification number
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
A
United States Government Accountability Office Washington, D.C. 20548
November 4, 2004
The Honorable Richard C. Shelby
Chairman
The Honorable Patty Murray
Ranking Minority Member
Subcommittee on Transportation/Treasury and General Government Committee
on Appropriations United States Senate
The Honorable Ernest J. Istook
Chairman
The Honorable John W. Olver
Ranking Minority Member
Subcommittee on Transportation and Treasury, and Independent Agencies
Committee on Appropriations House of Representatives
Automobile crashes exact an enormous personal and economic toll on this
country. In 2003, 42,643 people died in automobile crashes in the United
States, and nearly 2.9 million more were seriously injured. In 2000, the
most recent year for which cost estimates are available, the economic cost
of fatalities and injuries from crashes totaled almost $231 billion.1
Reducing this toll requires making informed decisions about safety
problems. Traffic safety data, which are compiled from police accident
reports completed at the scene of crashes and assembled largely at the
state level, are key to making these decisions. The states and the federal
government use data from state-level crash data systems to make many
roadway-related spending and policy decisions, ranging from deciding to
fix particular roadways to launching major national safety campaigns, such
as preventing alcohol-impaired driving or increasing seat belt use. There
is known variability in the quality of information in state traffic data
systems, and these differences impact the usefulness of data for these
purposes.
When the Congress was considering reauthorizing various transit and
highway programs earlier this year, both the House and Senate proposed
bills that would have expanded the section "411 grant program," which was
1The cost estimate was reported by the National Highway Traffic Safety
Administration.
initially authorized in the Transportation Equity Act for the 21st Century
(TEA-21).2 The 411 grant program was designed specifically to help states
improve their traffic safety data systems and provided states with $36
million over 6 years.3 The House and Senate proposals would have
authorized a similar grant program with a budget authority of up to $270
million over 6 years.4 However, an 8-month extension of TEA-21 was passed
on September 30, 2004, extending current highway and transit programs
through May 2005 when both bills may be reintroduced.
The Senate Appropriations Committee Report5 accompanying the Department of
Transportation Appropriations bill for fiscal year 2004 (S. 1589) directed
us to conduct a survey of state traffic safety data systems. The committee
also asked us to report on the extent to which the 411 grant program has
led to improvements in these systems. The grant program is overseen by the
Department of Transportation's (DOT) National Highway Traffic Safety
Administration (NHTSA), which has established six criteria for assessing
such systems when states request guidance from NHTSA. Four criteria relate
to the information itself (timeliness, consistency, completeness, and
accuracy); two relate to the use of the information (accessibility to
users and links to other related data). Accordingly, this report examines
(1) the quality of state crash information; (2) the activities states
undertook using 411 grant funds to improve their traffic safety data
systems, and the progress they made using the grant funds; and (3) NHTSA's
oversight of the grant program, including what changes in oversight, if
any, might help encourage states to improve their traffic data systems and
ensure accountability under a reauthorized program.
2Public Law 105-178 was enacted in June 1998. The name "411 grant program"
stems from the authorizing section of the law.
3The 411 grant program was in place from fiscal year 1998 through fiscal
year 2004. However, grants for the program were distributed to states only
through 2002, when the grant program funds were fully disbursed. The 411
grant was originally authorized for $32 million. Surplus funds applied
from a grant program that provides funds to states for alcohol-impaired
driving measures raised the total authorized 411 funds for all 6 years to
$36 million.
4The reauthorized program was included in the S. 1072: Safe, Accountable,
Flexible, and Efficient Transportation Equity Act of 2004 (SAFETEA) and
H.R. 3550: Transportation Equity Act -Legacy for Users of 2003 (TEA-LU).
5S.R. 108-146, page 76.
To provide information on the quality of state crash data and state
efforts to improve these data, we conducted site visits, analyzed
available traffic safety data, and reviewed grant documentation. Using
several criteria, we selected 9 states to visit for detailed reviews and
assessed the status of their data systems on the basis of NHTSA's six
quality criteria for crash information. Eight of these 9 states had
participated in the 411 grant program.6 To identify variations in data
structure and quality, we also analyzed crash data for 17 states that
currently participate in NHTSA's State Data System (SDS) program.7
Finally, we reviewed the grant documentation submitted by the 48 states
that participated in the 411 grant program, including grant applications,
traffic records assessments, strategic plans, progress reports, and
highway safety plan annual evaluation reports. To provide information
about NHTSA's oversight of the program, we interviewed NHTSA officials
responsible for oversight and administration and reviewed NHTSA guidance
and policy. We conducted our review from January 2004 through October 2004
in accordance with generally accepted government auditing standards. See
appendix I for more details regarding our objectives, scope, and
methodology. Because an examination of data quality was one of the
objectives of this report, we also conducted an assessment of data
reliability. A more complete discussion of data reliability can be found
in appendix II.
Results in Brief The 9 state traffic safety data systems we reviewed
varied widely in the degree to which they met NHTSA's six recommended
quality criteria for crash information.8 None of the state data systems we
reviewed appeared
6We visited California, Iowa, Kentucky, Louisiana, Maine, Maryland,
Tennessee, Texas, and Utah. Texas did not participate in the grant
program.
7The State Data System program is a database of state census crash data
managed by NHTSA and the National Center for Statistics and Analysis.
State participation is voluntary, with 27 states currently participating.
Using data from this program, NHTSA produces its most frequently requested
publication, the periodic Crash Data Report (most recently updated through
1999), and other important traffic safety publications. Our analysis
included data from 1998 through the most recent year available for
California, Florida, Illinois, Indiana, Kansas, Kentucky, Maryland,
Michigan, Missouri, New Mexico, North Carolina, Ohio, Pennsylvania, South
Carolina, Texas, Utah, and Virginia. Five of these 17 states were included
in our case-study visits: California, Kentucky, Maryland, Texas, and Utah.
8NHTSA's six recommended quality criteria are used by NHTSA to assess
state traffic safety data systems. The assessments identify needed data
system improvements, and it is up to states to decide if and how to
address the findings and where to focus their efforts.
to meet all of the criteria, which affected the usefulness and reliability
of their data. For example, while NHTSA's timeliness criteria call for
data to be available to users preferably within 90 days, the states we
visited had data available from within 1 month to 18 months. Delayed
access to crash data diminishes the ability to identify current and
emerging roadway hazards or other safety problems and to carry out
effective planning efforts, such as the development of annual state
highway safety plans. Likewise, while 4 states completed or checked crash
report data for accuracy by linking the data to information in driver or
vehicle licensing files, 3 states had no accuracy checks at all. Since
crash data are used mainly by states for highway safety planning,
allocating resources, and measuring efforts toward safety goals, the
information states collected varied-reflecting individual state needs. In
addition, some states did not collect information such as vehicle
identification numbers (VIN), which are particularly useful in identifying
automobile safety concerns. Variations such as these can affect DOT's
ability to make the state-to-state comparisons that are necessary to
evaluate past safety problems and develop future policy. For example, a
recent DOT-funded national analysis of vehicle braking performance was
based on data from only 5 states, because only these states had the
required information for the analysis.
States met the requirements of the grant and also undertook a range of
activities using 411 grant funds. However, our review showed that little
is known about the progress resulting from these activities. Entering
2002, the final year for which funding for the 411 grant program was
available, 43 of the 44 states that received grants reported that the
program's basic requirements-an assessment of the current data system, a
strategic plan for making improvements, and a state-level committee to
coordinate the effort-were in place.9 Both the assessments and the
strategic plans appeared generally helpful in establishing baselines and
priorities for improvement. Beyond meeting these basic requirements,
states were given broad flexibility in implementing activities to improve
their traffic data systems. Although the funding available under the 411
grant program was small in comparison with other federal safety grants,
states reported pursuing a variety of activities, such as contracting with
companies to help decrease crash report backlogs, redesigning data forms
to better adhere to recommended guidelines, and attending traffic records
conferences. However, the documents that states filed with NHTSA
concerning these
9A total of 48 states participated in the 411 grant program; however, by
2002, 4 states had discontinued.
activities provided little or no information on how far these efforts
progressed or what they accomplished. In the states we visited, officials
were able to provide examples of how their efforts had improved data
systems. Some of these states spent nearly their entire grant on a single
project, such as creating a crash database, while others spread the money
among multiple activities. In states where the coordinating committee had
broad representation and the ability to commit financial resources,
projects more fully addressed the needs of a broad cross-section of users.
NHTSA conducted limited oversight of the 411 grant program. While the
grant program's statutory requirements were not explicit about the scope
of oversight that NHTSA should have undertaken, NHTSA did not conduct
adequate oversight among states participating in the program, which
resulted in a poor and uneven picture of what the states were doing-or
accomplishing-with their grants. Two key oversight elements were missing.
First, NHTSA's regulatory requirements and guidelines required states to
submit progress reports under the 411 grant program, but were unclear
about what information states should report to document activities and
progress made improving traffic data systems. As a result, states were not
consistent in the scope and detail of the information they reported.
Second, NHTSA did not have an effective process for monitoring state
progress. For example, NHTSA was not able to provide us with complete
grant documentation (grant applications with progress reports, strategic
plans, and traffic data systems assessments) for about half of the states
that had participated in the grant program. Hence, NHTSA has limited
knowledge of the extent to which states improved their traffic data
systems through reported activities and whether states expended grant
monies for intended activities. We found, for example, that 2 of the 8
states we visited were not accurately reporting on 411-funded activities.
NHTSA has planned actions to correct some oversight shortcomings across
its many grant programs, but it is too soon to determine the extent to
which these actions will help ensure accountability if this grant program
is renewed. These actions also may not be specific enough to address the
weaknesses in this one program. In addition, proposed reauthorization
bills that included a follow-on program for the 411 grant program were
considered by the Congress in 2004, and these bills included requirements
related to documenting the use of grant funds and demonstrating measurable
progress that could result in clearer expectations for state reporting and
NHTSA oversight.
This report contains one matter for consideration by the Congress
concerning a requirement that state traffic safety data systems be
assessed
at least once every 5 years. States were to have an assessment of their
traffic data systems within 5 years of participating in the original 411
grant program-this was not included in reauthorization proposals. The
Congress may wish to add this requirement as it considers the legislation.
We also recommend that, if the Congress reauthorizes the traffic safety
incentive grant program, the Secretary of Transportation direct NHTSA to
ensure better accountability and management of grant documentation and
improved monitoring and oversight of 411 grant funds.
Background Traffic safety data are the primary source of knowledge about
crashes and how they are related to the traffic safety environment, human
behavior, and vehicle performance. Most states have developed traffic
safety data systems and manage these data through the initial reporting of
crashes by law enforcement officers through data entry and analysis.
Figure 1, which is based on NHTSA's Traffic Records Highway Safety Program
Advisory10 depicts a model state traffic safety data system, including the
collection and submission of data, the processing of these data into state
safety data systems, and the potential uses for quality crash information.
These data are often not housed in a single file or on just one computer
system; however, users should have access to crash information in a useful
form and of sufficient quality to support the intended use.
10NHTSA's Traffic Records Highway Safety Program Advisory establishes
criteria to guide state development and use of highway safety data.
Figure 1: Model State Traffic Safety Data System
Source: GAO analysis based on NHTSA's Traffic Records Safety Program
Advisory and Nova Development Corporation (clip art).
aStates are required to include highway safety plans in their applications
for federal funding of their highway safety programs. These plans describe
the projects and activities that states plan to implement to reach goals
identified in their performance plans.
At the state level, state agencies use traffic safety data to make highway
safety planning decisions and to evaluate the effectiveness of programs,
among other uses. In those states where quality crash data on a range of
crashes are not available, officials use federal data such as that from
NHTSA's Fatality Analysis Reporting System (FARS) to make programming
decisions.11 FARS data, while useful for some purposes, are limited
because they only include information about fatal crashes, thus preventing
decision
11FARS contains data derived from a census of fatal traffic crashes within
the United States (including Washington, D.C., and Puerto Rico). Fatal
crashes are those that involve a motor vehicle traveling on a traffic way
open to the public and result in the death of a person within 30 days of
the crash.
making based on a range of crash severity or the entirety of a state's
crash situation. At the federal level, NHTSA provides guidelines,
recommendations, and technical assistance to help states improve their
crash data systems and is responsible for overseeing state highway safety
programs.12 Under TEA-21, NHTSA awarded $935.6 million in highway safety
incentive grants to improve safety. In 2003, NHTSA made improving traffic
safety data one of the agency's highest priorities.
Since the early 1980s, NHTSA has been obtaining crash data files from
states, which in turn have been deriving the data from police crash
reports. These statewide crash data files are referred to as the SDS
program. Participation by states is voluntary, with 27 states currently
participating. These data include some of the basic information for the
analyses and data collection programs that support the NHTSA mission of
identifying and monitoring traffic safety problems.
One of NHTSA's grant programs was specifically aimed at improving traffic
safety data. Administered through its 10 regional offices around the
country, the program provided about $36 million to states for improving
their crash data systems. This grant program was authorized under TEA-21
and was known as the "411 grant program" after the relevant section of the
U.S. Code.13 NHTSA administers a number of other grant programs besides
the 411 grant program; however, it was the only incentive grant program
that was specifically directed at improving state traffic safety data
systems.14 The grant program required states to establish a foundation for
improving their traffic safety data systems by first completing three
activities:
o Establish a coordinating committee of stakeholders to help guide and
make decisions about traffic safety data: The committee would ideally
include stakeholders from agencies that manage the various data files
(e.g., representatives from the state department of transportation
responsible for roadway information, and from the state department of
12There are other agencies and associations involved in the improvement of
traffic safety data, see appendix III for examples.
13Title 23 U.S.C. Chapter 4.
14NHTSA's section 402 grant allows states to use some of their 402 funding
to support their state or local safety records systems.
motor vehicles responsible for the management of vehicle licensing
information).
o Conduct an assessment of the current system: The assessment would
evaluate a state's system by identifying strengths and weaknesses and
providing a baseline from which the state could develop its strategic plan
to address data system needs.
o Develop a strategic plan that prioritizes traffic safety data system
needs and identifies goals: The strategic plan is to provide the "map"
specifying which activities should be implemented in order to achieve
these goals. As with the assessment, the focal point for developing the
strategic plan, if a state did not already have one, would be the
coordinating committee.
The level of funding available to a state was dependent on whether states
had already put these requirements in place. Additionally, states were
required to contribute matching funds of between 25 and 75 percent,
depending on the year of the grant.15 Three types of grants were awarded:
o A state received a start-up grant if it had none of the three
requirements in place. This was a one-time grant of $25,000.
o A state received an initiation grant if it had established a
coordinating committee, had completed or updated an assessment within the
previous 5 years, and had begun to develop a strategic plan. This grant
was a one-time grant of $125,000, if funds were available.
o A state received an implementation grant if it had all three
requirements in place and was positioned to make specific improvements as
indicated in its strategic plan. This grant was at least $250,000 in the
first year and $225,000 in subsequent years, if funds were available.16
15In the 1st and 2nd fiscal years, the federal share of the costs shall
not exceed 75 percent. In the 3rd and 4th fiscal years, the federal share
of the costs shall not exceed 50 percent. In the 5th and 6th fiscal years,
the federal share of the costs shall not exceed 25 percent.
16Although states were required to meet basic requirements, they were not
required to submit individual projects for approval.
The Congress has extended TEA-21 until May 2005, and new House and Senate
bills will likely be introduced during the next congressional session. The
most recent House and Senate bills under consideration,17 which were not
passed in the 2004 session, included proposals to reauthorize the 411
grant program in a similar, but not identical, form to the original
program. The proposals included funding up to $270 million, which is over
six times the original funding amount. They also included (1) additional
requirements for documentation from states describing how grant funds
would be used to address needs and goals in state strategic plans and (2)
a requirement that states demonstrate measurable progress toward achieving
their goals. The proposals, however, did not include one of the original
program requirements-that states have an assessment of their traffic
safety data systems that is no more than 5 years old when they applied for
the grant.
Quality of Data Systems Varied Greatly, with All States Examined Showing a
Need for Improvement
The 9 states we examined in detail varied considerably in the extent to
which their traffic safety data systems met NHTSA's recommended criteria
for the quality of crash information. NHTSA's six criteria (shown in table
1 below, along with an explanation of each criterion's significance)
appear in the agency's Traffic Records Highway Safety Program Advisory,
the guide used by NHTSA when it carries out traffic records assessments at
the request of state officials. These assessments are a technical
assistance tool offered to state officials to document state traffic
safety data activities, note strengths and accomplishments, and offer
suggestions for improvement. In addition, NHTSA released the report
Initiatives to Address Improvement of Traffic Safety Data in July 2004,
which emphasized these data quality criteria and provided recommendations
to states. We examined all six criteria for the 9 case-study states, and
our review of 17 states that participated in NHTSA's SDS program provided
additional information for three of these six criteria.18 None of the 9
states in our case-study review met all six criteria, and most had
opportunities for improvement in many of the criteria. The sections below
discuss each criterion.
17H.R. 3550 and S. 1072.
18We looked at data from 17 states participating in NHTSA's SDS program to
provide additional information about three criteria: consistency,
completeness, and accuracy.
Table 1: NHTSA Recommended Criteria for Assessing Quality of Crash Information
Criteria Significance
Timeliness Crash information should be available for analytical purposes
within Timely crash data allow for the use of up-to-date information to a
useful time frame for identifying crash problems within a state- identify
safety problems, for policy making, and for resource preferably within 90
days of a crash. allocation, among other uses. Consistency Crash
information should be consistent among reporting Uniform data within a
state allow for the timely merging of data sets jurisdictions within a
state. It should also be consistent with and the identification of traffic
safety problems as they arise. In nationally accepted and published
guidelines and standards, such addition, states benefit by being able to
compare their results as the Model Minimum Uniform Crash Criteria.
nationally and with one another to identify traffic safety problems
and manage and monitor progress toward fixing them. Finally, consistent
state standards for determining which crashes to report allow for national
comparisons.
Completeness
Data should be collected for all reportable crashes in the state and
Adherence to state reporting requirements permits evaluation of the
on all appropriate crash variables. effectiveness of countermeasures
initiated by the state. Complete data also generate a picture of safety
performance useful for states to qualify for highway safety incentive
funding.
Accuracy Quality control methods should be employed to ensure accurate
Comprehensive information is necessary to understand what and reliable
crash information for both individual crashes and makes a difference and
what has a direct impact on reducing aggregate crash information. deaths,
injuries, injury severity, and costs. Accessibility Crash information
should be readily and easily accessible to the Accessible data enable the
use of data to identify safety problems principal users of such data. This
applies both to direct access of and for resource allocation, the quick
evaluation of recent traffic crash information from the appropriate crash
databases and to safety initiatives, the use of data for reporting
requirements, and the standard reports generated from crash data. ability
to respond to inquiries and requests from state legislative
and executive branches, among others.
Data integration
Crash information should be capable of linking to other information Links
make it possible to evaluate the relationship between specific sources.
Such linking could be accomplished through the use of roadway, crash,
vehicle, and human factors at the time of a crash. common identifiers or
probabilistic data-matching methods. They also permit these factors to be
linked to health outcome data
to determine their association with specific medical and financial
consequences, which facilitates choosing safety priorities that have the
most impact on reducing death and disability.
Source: GAO analysis based on NHTSA's Traffic Records Highway Safety
Program Advisory.
Timeliness: In Some States, Data processing time frames ranged widely in
the 9 states we visited. Three Available Data Were Several of the 9 states
met NHTSA's 90-day timeliness criterion for having useful Years Old data
available. Data processing times for the 9 states ranged from less than
1 month in 2 states to 18 months or more in 2 others. For example, to
develop their 2005 highway safety plans during 2004, 4 of the 9 states
used
data from 2000, 2001, or 2002, and the remaining 5 states used 2003 data.
(See fig. 2.)
Figure 2: Year of State Traffic Safety Data That Were Used to Develop 2005
State Highway Safety Plans
1 state
1 state
2 statesa
5 statesb
Source: GAO presentation of information provided by the case-study states.
aOne state received an extension on the due date of its safety plan to
allow for additional processing of 2002 data. Without this extension,
planning would have been done using 2001 crash information.
bOne state used preliminary 2003 data because data entry of location
information had not been completed.
For 6 of the 9 states, three factors accounted for their not meeting the
timeliness criterion: slow data entry, data integration delays, and
lengthy data edits. As a result, the state safety plans are unable to take
recent crash trends into account in these states. Generally, those states
submitting data electronically from local law enforcement agencies to the
state traffic safety data system had much faster entry of crash
information into
centralized databases.19 In contrast, states that processed reports
manually by keying in information from paper forms at the state level had
longer data entry time frames. The availability of data was also sometimes
delayed by inefficient data completion processes.20 In states where this
is not done automatically, crash data and location information are often
manually entered into the traffic safety data system. In addition, checks
for accuracy also delayed data availability. For example, 1 of the states
that had to use data from 2000 to develop its highway safety plan, had
used electronic methods to enter more recent data, but detailed edit
checks delayed the data's release considerably.
Consistency: Data Were Not Consistent Across States, Even for Basic
Information
Seven of the 9 states we visited had crash forms that could be used to
collect data across all jurisdictions within the state, helping to ensure
that data collected within the state are consistent. However, no state had
forms that met all of the consistency criteria recommended in the Model
Minimum Uniform Crash Criteria (MMUCC) guidelines that were developed
collaboratively by state and federal authorities. These guidelines provide
a recommended minimum set of data elements to be collected for each crash,
including a definition, attributes, and the rationale for collecting each
element. While variation in the crash data collected by states can be
attributed to varying information needs, guidelines help to improve the
reliability of information collected and also assist in state-tostate
comparisons and national analyses. The variation between states can be
seen among the 17 states we analyzed that contribute to NHTSA's SDS
program.21 For example, the MMUCC guidelines recommend reporting on
whether alcohol was a factor in the crash by indicating the presence or
19Law enforcement officers in several states use computer programs to aid
in the collection of crash information. One such program available to
states is the Traffic and Criminal Software (TraCS) program, which was
developed with federal assistance in Iowa. The TraCS program allows
officers to enter crash information into local databases before it is
transmitted to the central state traffic records database. Initial data
entry can be completed either using computers in officer vehicles or using
paper forms that are later keyed into the local computer system once an
officer returns to his or her agency. In addition to facilitating the
entry of data directly into the state database via electronic submission,
the TraCS program maintains a local database of crash information and
provides the local law enforcement agency with tools to do simple analysis
of this information.
20Data completion may involve pulling in additional information from other
state agencies, such as inputting the crash location from state department
of transportation files.
21Appendix II contains additional analysis related to the consistency and
completeness of data for these 17 states.
absence of an alcohol test, the type of test administered, and the test
results. However, several of the states collected information on impaired
driving without specifying the presence of an alcohol test, the test type,
or the test result; thereby making it difficult to determine whether
alcohol-use contributed to the crash. In addition, the states were not
uniform in collecting and reporting the VIN, another element recommended
in the MMUCC. A VIN is a unique alphanumeric identifier that is applied to
each vehicle by the manufacturer. The VIN allows for an effective
evaluation of vehicle design characteristics such as occupant protection
systems. As figure 3 shows, data about VINs were not available for all 17
states in crashes for any year between 1998 and 2002. For example,
although every state had submitted crash data for 1998 and 1999, crash
data for 6 of the 17 states did not include VINs.
Figure 3: Extent to Which Vehicle Identification Numbers Were Included in
Data Reported by 17 States in the SDS Program
Data available without VIN Data availabe with VIN N/A Data not available
for this year Source: NHTSA/NCSA.
The lack of consistency limits the use of state crash data for most
nationwide analyses. For example, in a recent National Center for
Statistics and Analysis (NCSA)22 Research Note on child safety campaigns,
only 3 states met the criteria to be included in the analysis, not nearly
enough data to statistically represent the nation. The criteria necessary
for inclusion in the report included collecting data on all vehicle
occupants, including uninjured occupants, and VINs for the relevant years
(1995-2001). If state systems matched the MMUCC, they would include this
information. Similarly, only 5 states qualified for use in a NCSA analysis
of braking performance as part of the New Car Assessment Program because
only these states collected VINs and had the necessary variables for the
years involved in the study.
There is evidence that as states redesign their crash forms, they are
following the MMUCC guidelines more closely. Remaining differences from
the suggested guidelines often reflect the needs of individual states.
Among the 9 states we visited, 5 had redesigned their crash forms since
1997. All 5 used the guidelines as a baseline, although each of them
tailored the form to a degree. One state, for example, collected no data
about the use of seat belts for uninjured passengers, while another chose
to collect additional state-specific attributes, such as describing snow
conditions (e.g., blowing or drifting). Among the remaining 4 states we
visited, 2 states are currently using the MMUCC guidelines to redesign
their forms.
Completeness: Gaps Existed in the Completeness of Reporting
One factor affecting the degree of completeness is state reporting
thresholds-that is, standards that local jurisdictions use to determine
whether crash data should be reported for a particular crash. These
thresholds include such things as the presence of fatalities or injuries
or the extent of property damage. Although all 9 of the states we visited
had reporting thresholds that included fatalities and injuries, the
thresholds for property damage varied widely. For example, some states set
the property damage threshold at $1,000, while 1 state did not require
reporting of property-damage-only crashes. In addition, it was not
possible to determine the extent that all reportable crashes had been
included in the traffic safety data system. Officer discretion may play a
role. For example, capturing complete documentation of a crash event is
often a low priority when traffic safety data are not perceived as
relevant to the work of the law
22NCSA is an office within NHTSA and is responsible for providing a wide
range of analytical and statistical support to NHTSA and the highway
safety community at-large.
enforcement officer or other public safety provider. In 1 state, for
example, the police department of a major metropolitan area only reported
crashes involving severe injuries or fatalities, although the state's
reporting threshold included damage of $1,000 or more.
Variation in thresholds among states is not the only factor that affects
the completeness of crash data. For the crash information that does make
it into the state database, there are often gaps in the data, as we
learned from evaluating the records of 17 states participating in NHTSA's
SDS program. For 5 of these states, we analyzed data coded "unknown" and
"missing" for 24 data elements. The percentage of data coded as unknown or
missing was frequent for several key data elements, such as the VIN; the
results of alcohol or drug testing; and the use of seat belts, child car
seats, and other restraint devices. For example, the percentage of data
coded as unknown or missing for the use of seat belts and other restraints
ranged between 1.5 and 54.8 percent for 4 of the 5 states. Such data can
be inherently difficult to collect.23 For example, when officers arrive at
the scene of a crash, drivers and passengers may already be outside their
vehicles, making it impossible to know if they were wearing seat belts.
Asked if they were wearing a seat belt, those involved in the crash may
not tell the truth, especially if the state has a law mandating seat belt
use.
Accuracy: States Varied Greatly in the Extent of Checks for Data Accuracy
Six of the 9 states we visited made use of quality control methods to help
ensure that individual reports were accurate when they were submitted to
the traffic safety data system. Of these 6 states, for example, 4 linked
crash reports to other traffic safety data, including driver or vehicle
files, to verify or populate information on crash reporting forms. Table 2
contains examples of other tools and checks that the states used to help
ensure accuracy.
23In addition to data collection difficulties, the data entry policies of
the states varied. For example, some states code occupant protection
system use correctly for injured passengers, but as "unknown" for
uninjured persons. Similarly, the practice by some states of coding the
lack of an alcohol test as ".00" rather than "missing" can lead to
difficulty in obtaining proper information.
Table 2: Examples of Tools and Processes That Were Used to Ensure the
Accuracy of Individual Crash Reports in 6 Case-study States
On-scene data verification tools Postcrash reporting accuracy checks
o Drop-down menus to complete crash o Automatic validity checks
included in reports electronic data submission
o Scannable bar codes on driver licenses or o Links to driver, vehicle,
or roadway files to vehicle registrations validate or populate crash
reports using
o Wireless connections to vehicle or driver common identifiers, such as
license plate files numbers, driver names, or location
o Global positioning system location identifiers equipment
Source: GAO analysis of information provided by states.
Four of the 9 states did quality checks at the aggregate level-that is,
when crash reports are analyzed in batches to identify abnormalities in
reporting that may not be apparent looking at individual reports. Of these
4 states, for example, 1 had staff analyze the reports to identify invalid
entries and data miscodings, while another conducted edit checks each year
to check for invalid vehicle types or other problems. Such aggregate-level
analysis can be useful to identify systematic problems in data collection
that may lead to erroneous investigation or false conclusions, such as
when officers report one type of collision as another. For instance,
officers in 1 state were found to be characterizing some car-into-tree
crashes as head-on collisions. Once identified, such data collection
problems can often be resolved through officer training.
To test data accuracy, we analyzed crash data submitted by the 17 states
to NHTSA and found relatively few instances of data that had been coded as
"invalid"-generally 3 percent or less. Data classified as invalid were
most often for elements more likely to be transposed or miscopied, such as
VINs. However, because we could not observe crash-scene reporting and did
not examine or verify information on source documents (such as police
accident reports), we cannot assume that the other 97 percent of data were
accurately reported and entered correctly. Invalid data entries are a good
starting point for measuring the accuracy of a data system, but they are
only one indication of the accuracy of state traffic safety data.
Accessibility: Crash Data All 9 states produced crash information
summaries, although some were Were Accessible to Users in based on data
that were several years old-a factor that limited their Varying Ways
usefulness. In addition, 8 states provided law enforcement agencies or
other authorized users with access to crash information within 6 months of
crashes. Such access was often via the Internet, and data analysis tools
were typically limited to a certain number of preestablished data reports.
Thus, any in-depth analysis was limited to the tools available online.
Three states had analysts available to provide information or complete
data queries upon request. In another state, which had the capability to
conduct data collection electronically, local law enforcement agencies had
access to analysis tools to use with their own data.
If users wanted direct access to completed data for more detailed
analysis, they often had to wait somewhat longer, given the need for
additional data entry or the completion of accuracy checks. In 1 state,
for example, there was a 2-to 3-month delay due to the transfer of
preliminary crash data from the state police database to the state
department of transportation where location information was added to
complete the data.
Data Integration: Only 1 State Integrated Traffic Safety Information with
All Databases
Only 1 of the 9 states integrated the full array of potential
databases-that is, linked the crash file with all five of the files
typically or potentially available in various state agencies: driver,
vehicle, roadway, citation/conviction, and medical outcome. All 9 of the
states we visited integrated crash information with roadway files to some
degree, but only a few integrated these data with driver or vehicle
licensing files, or with the conviction files housed in state court
systems. (See table 3.) In addition, 7 of the 9 states participated in
NHTSA's Crash Outcome Data Evaluation System (CODES) program,24 which
links crash data with medical information such as emergency and hospital
discharge data, trauma registries, and death certificates.
24The CODES program is funded by NHTSA and links existing statewide
traffic safety data with injury outcome, hospital discharge, and other
injury-related data. The linked data are used to support highway safety
decision making at the local, state, and national levels to reduce deaths,
nonfatal injuries, and health care costs resulting from motor vehicle
crashes.
Table 3: Extent of Data Integration with Crash Files in the 9 Case-study States
Type of file Reported state links with crash data
Driver Three states reported having direct links between the crash
database and
licensing driver records. Another state reported that the state department
of public safety pulled relevant information directly from crash reports,
including insurance information.
Vehicle licensing Two states reported links with the vehicle file. Each of
these transfers was done on at least a weekly basis.
Roadway All 9 of the states reported integrating crash data with roadway
information, although in 3 states, the integration was done by the state
department of transportation with the creation of a new database involving
additional data entry or file manipulation.
Citation/ Two states reported linking citation information with conviction
data from
Conviction the state court databases. Another state noted that 20 percent
of citation information was submitted to the department of public safety.
Medical Seven states were involved in the CODES program, which links crash
outcome data with medical files, such as emergency medical service,
hospital in/outpatient, and death certificate information. Six of these
states carried out data integration using probabilistic linkages, and the
7th was preparing to do so.
Source: GAO analysis of information provided by the case-study states.
Technological challenges and the lack of coordination among state agencies
often posed hurdles to the integration of state data. In 1 state, for
example, crash files were sent from the central traffic records database
kept by the state department of safety to the state department of
transportation for manual entry of location information from the roadway
file. Once the state department of transportation completed these records,
however, there was no mechanism to export that information back into the
central database. Also, in some states data integration was limited
because data were not processed with integration in mind. In 1 state, for
example, state department of transportation officials noted that the new
crash system had been developed for state police use, and that efforts
were still under way to develop an interface to bring crash data into the
department's system. In contrast, a state official in another state noted
that the housing of several agencies involved in the traffic safety data
system-including those responsible for the driver, vehicle, and roadway
files-in the state department of transportation had facilitated the direct
sharing of information and the full integration of data.
NHTSA Continues to Emphasize Quality Criteria
In support of these quality criteria and improved traffic safety data
systems, NHTSA released a report in July 2004 detailing steps that could
be taken by federal and state stakeholders to improve traffic safety data.
The report, Initiatives to Address Improvement of Traffic Safety Data, was
issued by NHTSA and drafted by an Integrated Project Team that included
representatives from NHTSA, the Bureau of Transportation Statistics, the
Federal Highway Administration, and the Federal Motor Carrier Safety
Administration. The report articulates the direction and steps needed for
traffic safety data to be improved and made more useful to data users. It
makes a number of recommendations under five areas, including improving
coordination and leadership, improving data quality and availability,
encouraging states to move to electronic data capture and processing,
creating greater uniformity in data elements, and facilitating data use
and access. Along with these recommendations, the report also outlines
initiatives that NHTSA and other stakeholders should implement. For
example, under the area of data quality and availability, the report
indicates that states-under the guidance of their coordinating
committees-should encourage compliance by law enforcement with state
regulations for obtaining blood-alcohol concentration and drug use
information and should also strive to capture exact crash locations (using
latitude and longitude measures) in their traffic safety data systems.
States Carried Out Various Activities Using 411 Grant Funds, but Little Is
Known about Progress
States reported carrying out a range of activities with funding made
available under the 411 grant program. However, relatively little is known
about the extent to which they made progress in improving their traffic
safety data systems for the years of the grant. When applying for
follow-on grants, states were required to report to NHTSA's regional
offices on the progress they were making in improving their traffic safety
data systems during the prior year. However, the required documents filed
with NHTSA yielded little or no information on what states had achieved.
We were able to discern from the 8 states we reviewed in detail that those
states had indeed used their grants for a variety of projects and showed
varying degrees of progress.25 Regardless of whether states concentrated
their grant funds on one project or funded a number of activities, the
level of progress was influenced by the effectiveness of state
coordinating committees.
25Texas, our 9th case-study state, did not participate in the grant
program.
Most States Received Grants for 4 Years and Initiated State Data
Improvements
Forty-eight states applied for and received grant awards under the 411
grant program. As table 4 shows, most states (29) began their
participation at the implementation grant level-that is, most of them
already had the three basic requirements in place, including a
coordinating committee, an assessment of their data system, and a
strategic plan for improvement. Those states receiving start-up or
initiation grants were expected to put the three requirements in place
before beginning specific data-related improvement projects. By the 4th
year of the grant, 44 states were still participating, and all but 1 was
at the implementation grant level. The 4 states that were no longer
participating by the 4th year reported that they discontinued
participation mainly because they could not meet grant requirements.
Table 4: Status of States in the 411 Grant Program, 1st and 4th Year
1st year of grant (1999) 4th year of grant (2002) Number of
Number of states Size of grants
Type of grant
statesa Size of grants
Start-up 7 $25,000 0 N/A
Initiation 11 63,100 1 $124,524
Implementation 29 126,260 43 224,151
Source: GAO analysis of information provided by NHTSA.
Note: The program was not funded for the final 2 years of TEA-21.
aA total of 48 states participated in the grant program between 1999 and
2002; 47 began in 1999, and 1 state began in 2000.
All three basic program requirements were useful to states to initiate or
develop improvements in their traffic safety data systems. By meeting
these grant requirements, states were able to "jump start" their efforts
and raise the importance of improving state traffic safety data systems.
The assessments, which were required to be conducted within 5 years of the
initial grant application, provided benchmarks and status reports to NHTSA
and state officials and included information on how well state systems
fared in regard to NHTSA's six recommended quality criteria. Officials
with whom we spoke generally agreed that these assessments were excellent
tools for systematically identifying needed state improvements. Similarly,
strategic plans generally appeared to be based on the state assessment
findings and helped states identify and prioritize their future efforts.
The establishment of the traffic records coordinating
committees to guide these efforts was also key to initiating improvements,
since traffic safety data systems involve many departments and their
cooperation is essential in developing and implementing improvements to a
state traffic safety data system.
Progress Reports Were Limited and Difficult to Assess
Documentation of state progress was limited and of little use in assessing
the effect of traffic safety data improvement efforts. To qualify for
grants beyond the first year, each state had to (1) certify that it had an
active coordinating committee and (2) provide documentation of its efforts
through updated strategic plans, separate progress reports, or highway
safety annual evaluation reports. We reviewed these documents when
available and found that they contained a variety of activities, ranging
from completing the basic requirements (such as conducting assessments and
developing strategic plans) to identifying specific projects (such as
outsourcing data entry services and redesigning crash forms). Figure 4
lists examples of these types of reported activities.
Figure 4: Examples of Reported Activities Drawn from Available Documents
of Participating States
Source: GAO presentation of information provided by NHTSA.
The grant documentation NHTSA received provided few details on the quality
of the state efforts.26 For example, although states certified the
existence of a coordinating committee, they were not required to report on
what the committee did or how well it functioned. Also, while states for
the most part identified efforts to improve their data systems, we found
it
26Documentation included grant applications and progress reports,
strategic plans, and traffic records assessments.
difficult to assess their progress because the reports lacked sufficient
detail. For example:
o One state reported using grant funds on alcohol testing devices to
collect more alcohol impairment data on drivers. However, the progress
reports did not indicate who received these devices and how data
collection was improved.
o One state used funds to hire data entry staff to reduce the backlog of
old crash reports. However, the state provided no indication of whether
the increase in staff had reduced the backlog and how any reduction in the
backlog could be sustained in the longer term.
o One state reported using funds on multimillion dollar information
technology projects, but it is unclear how the grant funds were used in
these projects.
Case-study States Conducted a Variety of Activities Ranging from One
Specific Project to a Variety of Activities
Our visits to 8 of the states that participated in the 411 grant program
yielded additional information and documentation about their grant
activities, the nature of their efforts, and the extent of progress made.
These states expended funds on a variety of activities, ranging from
completing the basic requirements of assessments and strategic plans to
implementing specific projects. As figure 5 shows, in the aggregate, these
activities translated into two main types of expenditures-equipment, such
as computer hardware and software, and consultant services, such as
technical assistance in designing new data systems.
Figure 5: Types of Expenditures under the 411 Grant Program in 8
Case-study States
1%
Subgrants
1%
Travel
3%
Training forums
Grant requirements
Other
Operating costs
Equipment (hardware/software)
Consultant/Professional services
Source: GAO analysis of 411 grant expenses reported by 8 case-study
states.
The 8 states either concentrated funding on one large project or used
funding on a variety of activities, including data entry, salaries,
training, and travel. Four of the 8 states focused on a single project
related to improving their data systems mainly by enhancing electronic
reporting. One state reengineered its files to better integrate them with
other data systems; 1 piloted an electronic crash data collection tool;
and the remaining 2 created new electronic data systems, which were
upgrades from their previous manual systems. These states also improved
the tools used by law enforcement officers to input data into their crash
systems, such as software for mapping and graphing traffic crashes or
laptop computers for patrol cars so that law enforcement officers could
collect and transmit crash data electronically to statewide repositories.
The remaining 4 states used funding on multiple activities, such as
obtaining technical support, adding capability for more data entry, or
attending conferences. Some also conducted pilot projects. For example, 1
state created a project that enabled electronic uploads of traffic
citation
data from local agencies to the state department of motor vehicles.
According to state officials, this project helped considerably with both
timeliness and completeness in the uploading of conviction information to
driver files. In another example, the state used funding to pilot a
project to capture data about crashes electronically.
States made improvements under both the single- and multiple-project
approaches. One state that focused on a single project, for example,
developed a new statewide electronic crash system that officials said had
improved data timeliness and completeness. Similarly, of the states that
spread funding among multiple activities, 1 state used funding for a data
project on driver convictions-paying for traffic records staff's salaries
and hiring consultants to map crashes to identify roadway issues. As a
result, the quality and completeness of crash data improved overall,
according to a state official.
One factor that affected state progress was the relative effectiveness of
the state's coordinating committee. In those states, where the state
coordinating committee did not actively engage all stakeholders or where
its level of authority was limited, projects did not fully address system
needs. For example, 1 state established a coordinating committee that
included few stakeholders outside the state police, and this committee
decided to concentrate funding on a new electronic crash data system. The
new system, acknowledged by many stakeholders as improving the timeliness
and completeness of crash data, resulted in a useful resource allocation
and crash-reporting tool for the state police to allocate resources and
report on crashes. According to officials at the state department of
transportation, however, improvements in the crash information did not
effectively serve to facilitate the state's use of crash data to identify
unsafe roadways because the state department of transportation was not
fully engaged in the coordinating committee's process.
Similarly, in another state, the coordinating committee lacked the
authority needed to fully implement its efforts. The coordinating
committee created two subcommittees-a technical committee and an executive
committee. While the executive committee was made up of higher level
managers from various agencies, the coordinating committee did not have
the legislative authority to compel agencies to participate in the process
or to even use the newly created statewide crash data system. To date, the
state does not have all key stakeholders participating in the process and
is continuing to have difficulty persuading the largest municipality in
the state to use the newly developed statewide electronic reporting
system. As a result, the
municipality continues to lag behind other communities in having its crash
information entered into the state crash system. In contrast, another
state's coordinating committee had the authority to approve or reject
proposals for data system improvements as well as funding. This state was
able to complete several agreed-upon projects, including implementing an
electronic driver citation program, which improved the completeness and
timeliness of the state crash data.
NHTSA's Limited Oversight of the 411 Grant Program Contributed to
Incomplete Knowledge of How Funds Were Used
NHTSA did not adopt adequate regulations or guidelines to ensure states
receiving 411 grants submitted accurate and complete information on
progress they were making to improve their traffic safety data systems. In
addition, the agency did not have an effective process for monitoring
progress and ensuring that grant monies were being spent as intended. We
found some examples where states did not report their progress accurately.
NHTSA, while beginning to take some actions to strengthen program
oversight, must be more proactive in developing an effective means of
holding states accountable under this program.
Regulatory Requirements and Guidance for 411 Program Activities Were Not
Specific
In our previous discussion about activities being carried out under the
grant program, we described how state documentation of progress often
contained too little detail to determine anything about the progress being
made as a result of activities being funded with program grants. Reasons
for this lack of information, in our view, were NHTSA's limited regulatory
requirements and inconsistent guidance about what information states
should submit.
Regulations for the 411 grant program required states to submit an updated
strategic plan or a progress report, but did not specify how progress
should be reported. Further, NHTSA's regulations required states to report
on progress as part of their 411 grant application, which in effect meant
that states did not have to report specifically on 411 activities after
fiscal year 2001. According to NHTSA regulations, states were to include
information on progress through their highway safety plans and annual
evaluation reports after fiscal year 2001, which are part of the reporting
for all of NHTSA's highway safety grants. However, our analysis of these
documents found that they lacked the detail needed to adequately assess
state activities undertaken with 411 funds. Further, while NHTSA officials
told us they also informally obtained information about progress after
fiscal year 2001, the available information about what the activities
actually
accomplished was limited. Limitations in the information regarding states
activities were particularly significant given that states spent most of
their grant funds after fiscal year 2001.
NHTSA regional offices supplemented the regulatory requirements with their
own guidance to states, but the guidance varied greatly from region to
region. Some of the regional offices said that their contact with states
about these requirements was informal, and that their primary contact with
states (1) was over the telephone or by e-mail and (2) was generally in
regards to technical assistance, such as training or referring states to
existing guidelines. Other regional office staff said they had additional
contact with states through participation in meetings of state
coordinating committees, where they were able to provide additional
assistance. However, we found this participation occurred most often for
states in proximity to NHTSA regional offices. Few regional offices
provided written guidance to states with specific direction on what to
include in their progress reports. For the regions that did so, the
requested information included documentation indicating how states
intended to use the current year grant funds, a list of projects
implemented in the past fiscal year, a brief description of activities
completed, an account of problems encountered, and the status of allocated
funds.
Without consistent and clear requirements and guidance on the content of
progress reports, states were left to their own devices. We found that
even in regions where NHTSA officials outlined the information that should
be included in the progress reports, states did not necessarily provide
the level of information needed for NHTSA to adequately track state
progress. For example, in 1 region, states were to provide NHTSA with
documentation that included a list of projects and a description of
progress made. However, 1 state in that region did not provide the list of
completed projects; it only provided a brief description of projects
completed during 1 of the 4 years of the grant.
We also found a wide variation in how states reported their activities.
For example:
o Some states provided brief descriptions of the activities completed or
under way, while others did not.
o States that provided brief descriptions of their activities did not
always include the same information. For example, some states indicated
how they were intending to use the current grant funds but did not list
projects implemented in the past year. Some states did not indicate the
status of their allocated funds for ongoing activities.
o None of the states in our review indicated problems that were
encountered in implementing projects or activities.
Monitoring of State Progress and Activities Was Lacking
Under the 411 grant program, NHTSA's oversight process for monitoring
state progress and ensuring that funds were spent in line with program
intent was limited. In fact, NHTSA was unable to provide copies of many of
the documents that states were required to submit to qualify for the 411
grant program. We requested these documents beginning in February 2004,
and NHTSA was only able to provide us with complete documentation for half
of the states participating in the program.27
When we visited 8 states that participated in the program, we were able to
compare expenditure reports obtained from the states with activities that
were reported to NHTSA. We found instances in which documentation of state
reported activities provided by NHTSA did not match information provided
directly to us by the states.
o In documentation submitted to NHTSA, 1 state reported using grant funds
on alcohol breath test devices. However, documents available at the state
level indicate that nearly all of the funds were expended on a single
project to redevelop a crash data system. Officials we spoke with also
indicated that the money had gone for redeveloping the data system.
o In a report to NHTSA, 1 state we visited had reported undertaking four
projects, but we found that two of them were actually funded by a
different federal grant.
The degree to which NHTSA monitored state 411-funded activities was
difficult to determine. NHTSA officials told us that they were not
required to review state 411-funded activities in detail. A few regional
office officials
27We received complete documentation (grant applications, state
assessments, and strategic plans) for 24 of the 48 states that
participated in the program from 1999 through 2002. Our discussions with
NHTSA staff showed there was some confusion between NHTSA headquarters
staff and the regional office staff regarding where the 411 grant program
documents were being held and who was responsible for managing them.
told us that they verified state reported activities by linking them to
objectives identified in state strategic plans; however, no documentation
of these reviews was provided.
Recent Steps Were Announced for Improving Oversight, but Impact on the 411
Program Is Unclear
NHTSA has taken several steps to improve its oversight and assist states
in improving their traffic safety data systems; however, more efforts are
needed. As we were completing our work, NHTSA released a report,
Initiatives to Address Improvement of Traffic Safety Data, that provides
the status of data systems in five areas, including coordination and
leadership, improving data quality and availability, encouraging states to
move to electronic capture and processing, creating greater uniformity in
data elements, and facilitating data use and access. It also provides
recommendations and initiatives in support of NHTSA's efforts to improve
state traffic safety data systems. Although the report outlines (1) steps
to be taken, (2) stakeholder responsibilities for each recommendation, and
(3) the general outcomes expected, the extent to which actions will occur
as a result of the report is unclear. The report is limited to a
description of conditions and needs for traffic safety data improvements
and does not include an implementation plan with milestones or timelines.
The report acknowledges that due to limited funding, NHTSA will focus
primarily on recommendations that are feasible given current resources.
According to NHTSA, the report was issued as a fact-finding status report
and, therefore, no timelines or milestones were included. However,
beginning October 2004, a newly created National Traffic Records
Coordinating Committee is developing an implementation plan for the goals
identified in the report.
NHTSA also recently enhanced its oversight tools for all safety grants. It
has mandated management reviews every 3 years and also expanded its
existing regional planning documents for the areas of occupant protection
and impaired driving, with three additional areas, including traffic
safety data.28 The first of these regional action plans aimed at data
improvements are being initiated fiscal year 2005 and include goals,
objectives, and milestones. Mandating management reviews that encompass
the broad array of grant programs every 3 years is an improvement over the
inconsistent application of these reviews in the past. Also, by
establishing
28Regional action plans identify, among other things, program goals,
performance measures, and specific tasks and strategies for the upcoming
year. The plan for traffic safety data systems will also include one or
two vital improvements needed in each state's traffic records system, such
as improving information on blood-alcohol concentration testing.
traffic safety data improvements as part of the regional action plans,
NHTSA will have more uniform tracking of state data improvements and also
better information on state progress. While these newly initiated efforts
are positive steps to improving oversight, it is too soon to tell how
effective they will be for monitoring and ensuring accountability under
the 411 grant program, should the Congress chose to reauthorize it.
Language in Reauthorization Bills Also Enhances Oversight, but Omits One
Key Step
NHTSA's oversight of the 411 grant program may be strengthened under
reauthorized legislation. Proposed reauthorization bills that were
considered by the Congress in 2004 included additional requirements that
states (1) demonstrate measurable progress toward achieving goals in their
strategic plans and (2) specify how they will use grant funds. These
additional provisions would be important steps in addressing the too-vague
reporting requirements of the current program and would be helpful in
addressing congressional and other inquiries about what the program is
accomplishing.
As the previous proposed bills were drafted, however, they omitted one
requirement that will be important in tracking state progress-the
requirement for a state to have an assessment of its traffic safety data
system no more than 5 years prior to participating in the 411 grant
program. Assessments are used mainly to establish the status of state
efforts, but state and NHTSA officials suggest that updated assessments
could also help in tracking state progress. During our review, we found
some assessments submitted by states that were nearly 10 years old. We
also found that assessments based on recent information reflected the
dynamic and often-changing reality of state systems. For example, 1 of our
casestudy states had recently conducted an assessment in 2002. When we
compared the information we had collected during our site visit, we found
much of the information from our visit reflected in the assessment.
Updating these assessments at least every 5 years would allow NHTSA to
track state progress. According to NHTSA officials, these assessments were
valuable starting points in helping states take stock of the strengths and
weaknesses of their entire systems. Updated assessments would take into
account changes made as a result of the new 411 grant program and other
efforts to improve the system since previous assessments were conducted.
Conclusions The states and the federal government base significant
roadway-related spending and policy decisions on traffic safety data,
ranging from deciding to repair particular roadways to launching major
safety campaigns. The quality of such decisions is tied to the quality of
these data. Our review indicates that there were opportunities for states
to improve crash data. However, because NHTSA exercised limited oversight
over the 411 grant program, it is difficult to say what the program as a
whole specifically accomplished or whether there was a general improvement
in the quality of these data over the program's duration. Nevertheless,
information we obtained from the 8 states we visited suggests the premise
that the 411 program did help states improve their traffic safety data
systems. Based on our work in these 8 states, we believe that states
undertook important improvements in their data systems with the federal
grant funds. The potential reauthorization of the grant program and
NHTSA's recent study of state safety data provide an opportunity to
include assurances that states use these grants on effective and worthy
projects. Furthermore, the reauthorization may provide greater funding
and, therefore, greater opportunity for states to improve their traffic
safety data systems. However, a larger program would come with a greater
expectation regarding what states will accomplish as well as with a need
to effectively track the progress states are making.
NHTSA's inability to provide key grant documentation and its deficiencies
in monitoring state progress with 411 grant funds could be minimized if
NHTSA (1) better managed grant documents, (2) had clearer requirements and
guidance for the grant program, and (3) had an effective oversight process
in place to monitor activities and progress. Requiring more specific
information on the improvements states are making in their data systems
would begin to address the problems we identified with regard to
inadequate reporting on the program. If the program is reauthorized, NHTSA
should develop an oversight process that does a better job of (1) tracking
state activities to their strategic plans and assessments, (2) providing
information about progress made in improving safety data, and (3) ensuring
that NHTSA can adequately manage the documentation it is requiring. In
addition, if NHTSA develops a plan to implement the recommendations in its
recent Integrated Project Team report on traffic safety data systems, it
could incorporate these recommendations through improved oversight
efforts.
Finally, one requirement present in the earlier program-up-to-date
assessments of state traffic safety data systems-was not included in
recent proposals to reauthorize the 411 grant program. These assessments
proved a valuable tool to states in developing and updating their
strategic plans and activities for the 411 grant program. They also
provide NHTSA with valuable information, including the current status of
state traffic safety data systems organized by NHTSA's own recommended
quality criteria.
Matter for In considering the reauthorization of the traffic safety
incentive grant
program, the Congress should consider including the requirement
thatCongressional states have their traffic safety data system assessed or
an update of the Consideration assessment conducted at least every 5
years.
Recommendations for Executive Action
If the Congress reauthorizes the traffic safety data incentive grant
during the next session, we recommend that the Secretary of Transportation
direct the Administrator, National Highway Traffic Safety Administration,
to do the following:
o Ensure better accountability and better reporting for the grant program
by outlining a process for regional offices to manage and archive grant
documents.
o Establish a formal process for monitoring and overseeing 411-funded
state activities. Specifically, the process should provide guidance for
submitting consistent and complete annual reporting on progress for as
long as funds are being expended. These progress reports should, at a
minimum, include the status of allocated funds, documentation indicating
how states intend to use the current year grant funds, a list of projects
implemented in the past fiscal year, brief descriptions of activities
completed, and any problems encountered.
o Establish a formal process for ensuring that assessments, strategic
plans, and progress reports contain the level of detail needed to
adequately assess progress and are appropriately linked to each other.
Agency Comments and We provided a draft of this report to the Department
of Transportation for its review and comment. Generally, the department
agreed with the
Our Evaluation recommendations in this report. Department officials
provided a number of technical comments and clarifications, which we
incorporated as
appropriate to ensure the accuracy of our report. These officials raised
two additional points that bear further comment. First, officials voiced
concern regarding the use of data quality criteria from NHTSA's Traffic
Records Highway Safety Program Advisory to review the quality of data or
the performance of states. The department emphasized that these criteria
are voluntary and states are not required to meet them; therefore, states
should not be judged against them. We acknowledge that these criteria are
voluntary and clarified the report to emphasize this point more fully.
However, we used the criteria as a framework for providing information on
the status of state systems and view this analysis as appropriate since
these criteria are used by NHTSA in conducting assessments of state
traffic safety data systems. Second, department officials noted that their
oversight of the 411 grant program was in accordance with the statutory
requirements. Although we recognize that there were minimal requirements
for the 411 grant program specifically, we believe the department should
carry out more extensive oversight activities so that NHTSA can monitor
the progress states are making to improve their traffic safety data
systems and better ensure that states are spending the grant monies as
intended.
We will send copies of this report to the interested congressional
committees, the Secretary of Transportation, and other interested parties.
We will make copies available to others upon request. In addition, the
report will be available at no charge on GAO's Web site at
http://www.gao.gov.
If you or your staff have any questions about this report, please call me
at (202) 512-6570. Key contributors to this report are listed in appendix
IV.
Katherine Siggerud Director, Physical Infrastructure Issues
Appendix I
Objectives, Scope, and Methodology
The objectives in this report were to identify (1) the quality of state
crash information; (2) the activities states undertook using 411 grant
funds to improve their traffic safety data systems, and progress made
using the data improvement grants; and (3) the National Highway Traffic
Safety Administration's (NHTSA) oversight of the grant program, including
what changes in oversight, if any, might help encourage states to improve
traffic safety data systems and ensure accountability under a reauthorized
program. To address these objectives, we conducted case-study visits to 9
states, analyzed state crash data, interviewed key experts, reviewed 411
grant program documentation, and interviewed NHTSA officials regarding
their oversight and guidance to states in improving their traffic safety
data systems.
To provide information on the quality of state crash data and state
efforts to improve these data, we conducted site visits to 9 states,
including California, Iowa, Kentucky, Louisiana, Maine, Maryland,
Tennessee, Texas, and Utah. The case-study states were chosen on the basis
of a variety of criteria, including population, fatality rates,
participation in the 411 grant program, the level of funding received
through the program, and participation in the State Data System (SDS)
program and the Crash Outcome Data Evaluation System (CODES). We adopted a
case-study methodology for two reasons. First, we were unable to determine
the status of state systems from our review of 411 documents. Second,
while the results of the case studies cannot be projected to the universe
of states, the case studies were useful in illustrating the uniqueness and
variation of state traffic safety data systems and the challenges states
face in improving them. During our case-study visits, we met and discussed
the status of state traffic data systems with a variety of traffic safety
data officials.1 These discussions included gathering information on
NHTSA's criteria, state objectives, and the progress made with 411 grant
funds.2 In addition to these case-study visits, we analyzed data for 17
states that currently participate in NHTSA's SDS program to identify
variations in data structure and quality. We selected a number of elements
to assess the quality of data as they related to completeness,
consistency, and accuracy for 5 of the 17
1These officials, in general, included representatives from the state
traffic coordinating committee, the governor's highway safety office, the
department of public safety, the department of transportation, the
department of motor vehicles, the department of health, and stakeholders
from the medical or injury prevention sector.
2NHTSA's recommended criteria includes timeliness, completeness,
consistency, accuracy, accessibility, and data integration.
Appendix I
Objectives, Scope, and Methodology
states that were part of the SDS program and also part of our case-study
visits. We based the analysis on data and computer programs provided by
NHTSA. We reviewed the programs for errors and determined that they were
sufficiently accurate for our purposes. (See app. II.) Finally, we
interviewed key experts who use traffic safety data, including
consultants, highway safety organizations, and researchers.
In order to describe the activities that states undertook to improve their
traffic safety data systems and the progress made under the data
improvement grant, we reviewed 411 grant documentation for all 48
participating states, including 8 of our 9 case-study states.3 Our review
included examining required documents states submitted to NHTSA, including
their assessments, strategic plans, and grant applications and progress
reports. We obtained these documents from NHTSA regional offices. For the
case-study states, we also obtained additional documentation, including
411 grant expenditure information, in order to (1) describe state
activities and progress made and (2) compare actual expenditures with the
activities states reported to NHTSA.
To review NHTSA's oversight of the 411 grant program, we interviewed NHTSA
officials responsible for oversight and administration of the program. Our
interviews were conducted with NHTSA program staff at headquarters and in
all 10 NHTSA regional offices. We also discussed program oversight with
state officials in 8 of our 9 case-study states. We reviewed NHTSA
guidance and policy, including regulations for the 411 grant program and
rules issued by NHTSA for the program. We also reviewed previous House and
Senate bills that were introduced reauthorizing the 411 grant program.4
Finally, in order to understand NHTSA's broader role in oversight, we
spoke with NHTSA staff and reviewed NHTSA's response to our
recommendations that it improve its oversight.
We conducted our review from January 2004 through October 2004 in
accordance with generally accepted government auditing standards. Because
an examination of data quality was one of the objectives of this
3Texas did not participate in the 411 grant program.
4On September 30, 2004, while we were completing our review, current
highway and transit programs in the Transportation Equity Act for the 21st
Century (TEA-21) were extended to May 2005.
Appendix I
Objectives, Scope, and Methodology
report, we also conducted an assessment of data reliability. Appendix II
contains a more complete discussion of data reliability.
Appendix II
Additional Analysis of Data Quality in NHTSA's State Data System
As part of our work, we examined data quality for 17 states that
participate in NHTSA's SDS program. The body of our report presents
several examples of the kinds of limitations we found; this appendix
contains additional examples. The examples discussed below relate to two
of NHTSA's quality criteria-data consistency and data completeness.
Variations in Reporting Thresholds Impact the Usefulness of Data in the
State Data System
The extent to which a state captures information about various data
elements has much to do with the standards or thresholds it sets for what
should be reported in crash reports. NHTSA's Model Minimum Uniform Crash
Criteria (MMUCC) recommends that every state have reporting thresholds
that include all crashes involving death, personal injury, or property
damage of $1,000 or more; that reports be computerized statewide; and that
information be reported for all persons (injured and uninjured) involved
in the crash.
We found these thresholds differed from state to state. Two thresholds, in
particular, create variation in the data: (1) criteria for whether a crash
report must be filed and (2) criteria for whether to report information
about uninjured occupants.
Determining Which Crashes The states varied greatly in their policies on
when a police report must be
Require a Crash Report filed. Fourteen of the 17 states set a property
damage threshold, but the threshold varied from less than $500 to as much
as $1,000 (see fig. 6). Among the other 3 states, 1 left the reporting of
property-damage-only crashes to the officer's discretion, and 2 stipulated
that no report is to be filed unless at least one vehicle has to be towed
from the scene. Thus, a crash involving $900 of damage to an untowed
vehicle would be reported in some states but not in others.
Appendix II
Additional Analysis of Data Quality in
NHTSA's State Data System
Figure 6: State Criteria for Filing a Police Crash Report for
Property-Damage-Only Crashes
Number of states 10
8
8
6
4
2
0 <$500 $500
$750 $1,000
Officer discretion Minimum one vehicle towed
Source: GAO presentation of NHTSA/NCSA data.
Reporting Information about Uninjured Passengers
Similarly, some states did not collect information about uninjured
passengers involved in crashes. (See fig. 7.) While all 17 states
collected information about uninjured drivers (such as whether he or she
was wearing a seat belt), 5 did not collect such information about
uninjured passengers. Such information could potentially be important, for
example, in assessing the degree to which seat belt use helped prevent
injuries from occurring. Even for states that collected information about
uninjured passengers, the information may be incomplete. NHTSA officials
said they thought that in these states, some officers left seat belt
information blank or coded it as "unknown," either because reporting
officers did not know the information or because collecting it was too
time-consuming.
Appendix II
Additional Analysis of Data Quality in
NHTSA's State Data System
Figure 7: Extent to Which States Collected Information about Uninjured
Passengers
States reporting
Variations in Reporting
Alcohol and Drug Data
Alcohol and drug data also showed state-to-state differences, both in
consistency and completeness. Alcohol and drug data are important in
addressing a major safety issue-impaired driving. In 2000, crashes in
which drivers had blood-alcohol levels above .08 (.08 recently became the
threshold for being legally impaired in all 50 states) accounted for an
estimated 2 million crashes that killed nearly 14,000 people and injured
nearly 470,000 others. Alcohol-related crashes in the United States that
year cost an estimated $114.3 billion.1
To assess the quality of these data in the SDS program, we selected 5
states for detailed review. The states, chosen because they were also
visited as part of our case studies, were California, Kentucky, Maryland,
Texas, and Utah-although they are not identified by name in the results
below. We looked at the degree to which they conform to guidelines
recommended in the MMUCC with regard to the consistency and completeness
of their data.
1Estimated costs include $51.0 billion in monetary costs and an estimated
$63.2 billion in quality of life losses.
States not reporting
Source: GAO presentation of NHTSA/NCSA data.
Appendix II
Additional Analysis of Data Quality in
NHTSA's State Data System
Consistency of Traffic Safety Data
Information collected about alcohol-and drug-impaired driving varied from
state to state and was not consistent with MMUCC guidelines. Table 5
provides examples of this variation by comparing crash information
submitted by states with the recommended guidelines. The table shows
MMUCC's recommended guidelines for four elements-two elements each for
alcohol and drugs. One element relates to whether the officer suspects
alcohol or drug use, and the other to an actual test for alcohol or drugs.
All 5 states collected some type of information on suspected alcohol or
drug use, but each state differed from the others to some degree. Three
states, for example, collected this information as part of a broader
element that includes suspected alcohol and drug use as one attribute in a
list of causes that might have contributed to the crash. For alcohol and
drug testing, 1 state did not report such testing at all, and the 4 others
differed both from each other and from MMUCC guidelines.
Appendix II
Additional Analysis of Data Quality in
NHTSA's State Data System
Table 5: Comparison of MMUCC Guidelines and Crash Information Provided to
NHTSA by 5 States Regarding Alcohol- and Drug-Impaired Driving
Crash information that 5 states collected (variable name and definition)
Recommended
MMUCC element
(variable name
and definition) State A State B State C State D State E
Law Enforcement Suspected Driver/Pedestrian Contributing
Driver/Pedestrian
Suspects Drinking: Drinking: Condition: Factor 2:
Alcohol Use:
Indicates whether Indicates if Indicates the Describes the
the driver condition of
Driver or drinking of the each driver's
impaired the vehicle was driver/pedestrian at actions as
nonmotorist ability of suspected of the time of the the second
the driver, crash.
involved in the contributing
pedestrian, or drinking. factor
bicyclist.
crash suspected Attributes include in the crash.
"had
by law Attributes Attributes been drinking" and
include "not include "had
enforcement to stated" "yes" and been using drugs." Attributes
and "had been "no." include
have used alcohol. "under the
drinking, under
influence." influence of
alcohol" and
"under the
influence of
drugs."
Contributing Circumstance:
Describes first actions taken by the driver that contributed to the crash.
Attributes include "Had been drinking" and "Under the influence of drugs."
Alcohol Test:
Indication of the presence of alcohol by test, test type (blood, breath,
etc.), and test results.
None Alcohol/Drug, Alcohol Test Results, and Alcohol Test Type:
Alcohol/Drug: Indicates if the driver was tested for alcohol, drugs, or
both.
Alcohol Test Results: Indicates the results of the alcohol/drug test.
Alcohol Test Type: Indicates the alcohol test type administered to the
driver.
Alcohol/Drug Use and Alcohol Test Results:
Alcohol/Drug Use: Indicates the presence and the contribution of
controlled substances.
Alcohol Test Results: Indicates the results of the alcohol test. Coded for
drivers, pedestrians, and bicyclists.
Alcohol/Drug Analysis Test and Alcohol/Drug Test Results:
Alcohol/Drug Analysis Test: Indicates the type of specimen taken for an
alcohol and/or drug analysis test. Coded for drivers, pedestrians, and
bicyclists.
Alcohol/Drug Test Results: Describes the results of alcohol and/or drug
test. Coded for drivers, pedestrians, and bicyclists.
Alcohol Test Results and Alcohol Test Type:
Alcohol Test Results: Indicates the results of an alcohol test. Coded for
drivers, pedestrians, and bicyclists.
Alcohol Test Type: Describes how the alcohol test was administered. Coded
for drivers, pedestrians, and bicyclists.
Appendix II
Additional Analysis of Data Quality in
NHTSA's State Data System
(Continued From Previous Page)
Crash information that 5 states collected (variable name and definition)
Recommended
MMUCC element
(variable name
and definition) State A State B State C State D State E
Law Enforcement Human Factors Driver/Pedestrian Contributing
Driver/Pedestrian 1 -
Suspects Drug 3: Condition: Factor 2:
Condition:
Use:
Indicates up Indicates the Describes the
Identifies a physical to three condition of
Driver or condition of factors by each driver's
the driver, humans driver/pedestrian at actions as
nonmotorist pedestrian, that the time of the the second
or bicyclist contributed to crash.
involved in the that the crash. contributing
may have been a factor
crash suspected factor Attributes include in the crash.
in the crash. "had
by law Attributes been drinking" and
include "if "had
enforcement to the officer Attributes
Attributes include suspects been using drugs." include
"under
have used drugs. the drug "under the
influence of drugs" involvement in
and "other physical the crash." influence of
impairment." alcohol" and
"under the
influence of
drugs."
Contributing Circumstance:
Describes first actions taken by the driver who contributed to the crash.
Attributes include "had been drinking" and "under the influence of drugs."
Drug Test:
Indication of the presence of drug test, test type, and test results.
Excludes drugs administered postcrash.
None Alcohol/Drug and Alcohol Test Results:
Alcohol/Drug: Indicates if the driver was tested for alcohol, drugs, or
both.
Alcohol Test Results: Indicates the results of the alcohol/drug test.
Alcohol/Drug Use: Alcohol/Drug Analysis Test
Indicates the presence and Alcohol/Drug
and contribution of Test Results:
controlled substances. Alcohol/Drug Analysis Test: Indicates the type of
specimen taken for an alcohol and/or drug analysis test. Coded for
drivers, pedestrians, and bicyclists.
Alcohol/Drug Test Results: Describes the results of alcohol and/or drug
test. Coded for drivers, pedestrians, and bicyclists.
Alcohol Test Results and Alcohol Test Type:
Alcohol Test Type: Indicates the results of a drug scan. Coded for
drivers, pedestrians and bicyclists.
Alcohol Test Results: Includes positive and negative results of drug scan.
Source: GAO presentation of NHTSA/NCSA data.
Note: State A uploads alcohol test result data from the Fatality Analysis
Reporting System at a later date.
Appendix II
Additional Analysis of Data Quality in
NHTSA's State Data System
Completeness of Traffic Safety Data
To determine the completeness of state data files regarding impaired
driving, we looked at alcohol test result data that were coded as
"missing" or "unknown." Figures 8 and 9 show the results for the first and
last years we reviewed. The percentage of data recorded as missing varied
from 0 percent to more than 12 percent, while the percentage of data
recorded as unknown varied from 0 percent to more than 6 percent.2 In
addition, the 2 states with the most data in these two categories were
almost mirror images of each other: that is, state D showed no data as
missing but had the highest percentage of data classified as unknown,
while state E showed virtually no data as unknown but had the highest
percentage of data classified as missing. These variations could reflect
differences in how states classify and record information. For example,
NHTSA officials said some states may code an alcohol test result that
comes back indicating no alcohol in the driver's blood stream as missing
or unknown, rather than "negative" or ".00."
2While these percentages seem low, the actual number of crashes
represented is sizable. For all 4 states, the number of total crashes
represented ranged from 97,000 to 599,000. The number of crashes with
missing or unknown data for alcohol test results ranged from in the
hundreds to in the thousands.
Appendix II
Additional Analysis of Data Quality in
NHTSA's State Data System
Figure 8: Percentage of Alcohol Test Results That Were Coded as Missing
for 1998 and 2002
Percentage of alcohol test results coded as missing
14
12.3% 12
10
8
6
4
2
0 State A State B State C State D State E
1998
2002
Source: GAO presentation of NHTSA/NCSA data.
Note: State A did not provide alcohol test results to NHTSA during the
period under investigation.
Appendix II
Additional Analysis of Data Quality in
NHTSA's State Data System
Figure 9: Percentage of Alcohol Test Results That Were Coded as Unknown
for 1998 and 2002
Percentage of alcohol test results coded as unknown 14
12
10 8 6.7% 6
4
2
0
Because the alcohol and drug data in SDS are subject to so many problems
with completeness and consistency, many researchers and state policy
makers use alcohol and drug data from the Fatality Analysis Reporting
System (FARS) database instead. This database, which is also administered
by NHTSA, contains information on crashes involving fatalities that occur
within 30 days of the crash. FARS is generally seen as a reliable data
source, with quality control measures and personnel that do as much
follow-up as possible to fill in data gaps by contacting hospitals,
medical offices, and coroners' offices to obtain accurate and complete
information. However, FARS contains information only on fatal
crashes-about 1 percent of all crashes. Thus, while the FARS data may be
more complete and consistent for those crashes that are included, the vast
majority of
State A State B State C State D State E 1998 2002
Source: GAO presentation of NHTSA/NCSA data.
Note: State A did not provide alcohol test results to NHTSA during the period
under investigation.
Researchers' Use of Another Database Omits Data on Nonfatal Crashes
Appendix II
Additional Analysis of Data Quality in
NHTSA's State Data System
alcohol-and drug-related crashes are not included. Further, NHTSA imputes
some of the alcohol information because even with follow-up there are
often gaps in data.3
3Imputation is a statistical inference method used to estimate alcohol
rates.
Appendix III
Examples of Federal and Other Efforts at Improving Traffic Safety Data
The Commercial Vehicle Analysis and Reporting Systems is a cooperative
effort between NHTSA and FMCSA to improve collection of bus and truck
data. Its aim is to improve the national data system for all crashes
involving commercial motor vehicles and to develop a national analytical
data system similar to the Fatality Analysis Reporting System for
commercial vehicles.
The Highway Safety Information System (HSIS) is a 9-state database that
contains crash, roadway inventory, and traffic volume data. Under contract
with FHWA, the University of North Carolina Highway Safety Research Center
and LENDIS Corporation operate the system. The HSIS uses state highway
data for the study of highway safety. The system is able to analyze a
large number of safety problems, ranging from more basic "problem
identification" issues to identify the size and extent of a safety problem
to modeling efforts that attempt to predict future accidents from roadway
characteristics and traffic factors.
The Transportation Safety Information Management System (TSIMS) is a joint
application development project sponsored by AASHTO to enable states to
link crash data with associated driver, vehicle, injury, commercial
carrier, and roadway characteristics. TSIMS is an enterprise safety data
warehouse that will extend and enhance the safety analysis capabilities of
current state crash records information systems by integrating crash data
with other safety-related information currently maintained by each state.
ATSIP aims to improve traffic safety data systems by (1) providing a forum
on these systems for state and local system managers, including the
collectors and users of traffic safety data; (2) developing, improving,
and evaluating these systems; (3) encouraging the use of improved
techniques and innovative procedures in the collection, storage, and uses
of traffic safety data; and (4) serving as a forum for the discussion of
traffic safety data programs.
Sources: AASHTO, ATSIP, and FHWA.
Appendix IV
GAO Contacts and Staff Acknowledgments
GAO Contacts Katherine Siggerud (202) 512-6570 Randall Williamson (206)
287-4860
Staff In addition to those individuals named above, Nora Grip, Brandon
Haller, Molly Laster, Dominic Nadarski, Beverly Ross, Sharon Silas, Stan
Acknowledgments Stenersen, and Stacey Thompson made key contributions to
this report.
GAO's Mission The Government Accountability Office, the audit, evaluation
and investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of The fastest and easiest way to obtain copies of GAO
documents at no cost
is through GAO's Web site (www.gao.gov). Each weekday, GAO postsGAO
Reports and newly released reports, testimony, and correspondence on its
Web site. To Testimony have GAO e-mail you a list of newly posted products
every afternoon, go to
www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out
to the Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are discounted 25
percent. Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
To Report Fraud, Contact:
Waste, and Abuse in Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: [email protected] Programs Automated answering system: (800)
424-5454 or (202) 512-7470
Congressional Gloria Jarmon, Managing Director, [email protected] (202)
512-4400 U.S. Government Accountability Office, 441 G Street NW, Room 7125
Relations Washington, D.C. 20548
Public Affairs Susan Becker, Acting Manager, [email protected] (202)
512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
*** End of document. ***