Public Transportation: Opportunities Exist to Improve the
Communication and Transparency of Changes Made to the New Starts
Program (28-JUN-05, GAO-05-674).
The Transportation Equity Act for the 21st Century (TEA-21) and
subsequent legislation authorized about $13.5 billion in
guaranteed funding for the Federal Transit Administration's (FTA)
New Starts program, which is used to select fixed guideway
transit projects, such as rail and trolley projects, and to award
full funding grant agreements (FFGA). GAO assessed the New Starts
process for the fiscal year 2006 cycle. GAO identified (1) the
number of projects that were evaluated, rated, and proposed for
new FFGAs and the proposed funding commitments in the
administration's budget request; (2) changes FTA has made to the
New Starts application, evaluation, rating, and oversight
processes since the fiscal year 2001 evaluation cycle and how
these changes have been communicated to project sponsors; and (3)
how FTA developed the measures used to evaluate and rate projects
from the criteria outlined in TEA-21 and how those measures are
used in the rating process.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-05-674
ACCNO: A28218
TITLE: Public Transportation: Opportunities Exist to Improve the
Communication and Transparency of Changes Made to the New Starts
Program
DATE: 06/28/2005
SUBJECT: Evaluation criteria
Evaluation methods
Federal funds
Federal grants
Mass transit funding
Program evaluation
Program management
Transportation
Federal aid for transportation
Federal aid programs
Policies and procedures
Transparency
FTA New Starts Program
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-05-674
* Report to Congressional Committees
* June 2005
* PUBLIC TRANSPORTATION
* Opportunities Exist to Improve the Communication and Transparency
of Changes Made to the New Starts Program
* Contents
* Results in Brief
* Background
* FTA Identified Four New Projects for FFGAs and Requested $1.5
Billion for Fiscal Year 2006
* FTA Evaluated and Rated 27 Projects and Proposed 4 for FFGAs
in Fiscal Year 2006
* Administration's Proposed Fiscal Year 2006 Budget Requests
Similar Amount of Funding to Previous Years
* FTA Has Implemented a Number of Changes to the New Starts
Program, Some Without Project Sponsors' Input
* FTA Has Made 16 Changes Intended to Improve the Application,
Evaluation, Rating, and Project Development Oversight
Processes since the Fiscal Year 2001 Evaluation Cycle
* In Spring 2005 FTA Introduced New Cost-Effectiveness
Practice and Technical Changes to the
Cost-Effectiveness Calculation
* FTA Uses Multiple Methods to Communicate Information about
Changes
* FTA Has Not Consistently Used the Public Notice or
Rulemaking Process to Introduce Changes or Solicit Industry
Comment on Proposed Changes
* New Starts Measures Have Evolved Over Time, but Not All Measures
Count Toward a Project's Rating
* Congress Directed FTA to Use Multiple Criteria in Evaluating
and Rating Projects
* FTA Used Industry Input to Develop Measures for All Criteria
Identified in TEA-21
* FTA Does Not Use All Project Justification Criteria in
Determining Ratings Because of Perceived Weaknesses in the
Measures
* Project Sponsors and Others Identified Strengths and
Weaknesses of the Measures and Suggested Improvements
* Conclusions
* Recommendations for Executive Action
* Agency Comments and Our Evaluation
* Scope and Methodology
* Projects with Existing FFGAs and Projects in Preliminary Engineering
and Final Design in the Fiscal Year 2006 Cycle
* Status of Previous GAO Recommendations for Improving the New Starts
Evaluation Process
* Key New Starts Provisions Contained in House and Senate
Reauthorization Bills
* FTA's Project Justification Measures for Evaluating and Rating New
Starts Projects
* GAO Contacts and Staff Acknowledgments
* Related GAO Products
United States Government Accountability Office
Report to Congressional Committees
GAO
June 2005
PUBLIC TRANSPORTATION
Opportunities Exist to Improve the Communication and Transparency of Changes
Made to the New Starts Program
a
PUBLIC TRANSPORTATION
Opportunities Exist to Improve the Communication and Transparency of
Changes Made to the New Starts Program
What GAO Found
For the fiscal year 2006 evaluation cycle, FTA evaluated and rated 27
projects and identified 4 projects that were expected to be ready for new
FFGAs before the end of fiscal year 2006 and an additional 6 projects that
may be eligible for other funding outside of FFGAs. The administration's
fiscal year 2006 budget proposal requests $1.5 billion for the New Starts
program, a request similar to that of the past 2 years.
FTA has made 16 changes to the New Starts application, evaluation, rating,
and oversight processes since the fiscal year 2001 cycle that were
primarily intended to make the process more rigorous and systematic. Seven
of the 16 changes underwent rulemaking, including providing formal notice
to the transit industry and soliciting comment, while 9 changes did not.
FTA officials said that these nine changes are consistent with the
existing regulations governing the New Starts process or relate to the
project development oversight process rather than the evaluation and
rating process and, therefore, in their opinion, do not need to undergo
formal rulemaking. By not consistently soliciting public opinion, however,
FTA is missing an opportunity to obtain stakeholder buy-in, increase the
transparency of the New Starts process, and lessen potential difficulties
project sponsors face in implementing the changes.
Many of the measures FTA uses to evaluate and rate New Starts projects
have evolved over time, with industry input, through formal rulemaking and
informal efforts, such as workshops and reports. Although both TEA-21 and
FTA's New Starts program regulations emphasize the importance of using a
multiple-measure approach for evaluating projects, FTA assigns weight to
all three financial criteria but only two of the five project
justification criteria in developing a project's rating. FTA officials
said that they do not use the other three project justification
criteria-which are specified in TEA-21- because the measures fail to
distinguish among projects. Project sponsors we interviewed offered
suggestions for improving all of the project justification measures, and
FTA has efforts underway to improve some of the measures.
Example of a New Starts Project with an Existing FFGA - Bay Area Rapid
Transit Extension to San Francisco International Airport
Source: GAO.
United States Government Accountability Office
Contents
Letter 1
Results in Brief 3 Background 5 FTA Identified Four New Projects for FFGAs
and Requested $1.5
Billion for Fiscal Year 2006 10 FTA Has Implemented a Number of Changes to
the New Starts
Program, Some Without Project Sponsors' Input 16 New Starts Measures Have
Evolved Over Time, but Not All Measures
Count Toward a Project's Rating 29 Conclusions 42 Recommendations for
Executive Action 43 Agency Comments and Our Evaluation 44
Appendixes
Appendix I: Appendix II: Appendix III: Appendix IV: Appendix V: Appendix
VI:
Scope and Methodology 46 Projects with Existing FFGAs and Projects in
Preliminary Engineering and Final Design in the Fiscal Year 2006 Cycle 49
Status of Previous GAO Recommendations for Improving the New Starts
Evaluation Process 52 Key New Starts Provisions Contained in House and
Senate Reauthorization Bills 54 FTA's Project Justification Measures for
Evaluating and Rating New Starts Projects 55 GAO Contacts and Staff
Acknowledgments 56
Related GAO Products
Table 1:
Tables
Table 2:
Table 3: Table 4: Table 5:
FTA's Criteria for Assigning Overall Project Ratings 9 Changes to the New
Starts Application, Evaluation, Rating, and Project Development Oversight
Processes since the Fiscal Year 2001 Evaluation Cycle 18 Strengths,
Weaknesses, and Other Concerns about New Starts Measures, as Identified by
Project Sponsors 39 Suggestions for Improving New Starts Measures, as
Identified by Project Sponsors 42 Projects Contacted for Our Review 47
Contents
Figures Figure 1: Figure 2: New Starts Planning and Development 7 8
Process New Starts Project Evaluation
Criteria
Figure 3: Distribution of New Starts Projects in
Preliminary
Engineering and Final Design for Fiscal
Year 2000 to 2006
Evaluation Cycles 12
Figure 4: Total New Starts Funding for Fiscal Year
2006 Equals $1.5
Billion 15
Figure 5: Criteria and Measures for Evaluating
Projects, as
Outlined in TEA-21 and FTA's New Starts
Program
Regulations 32
Figure 6: Weights Used to Determine Project
Justification and
Financial Summary Ratings for New Starts 35
Projects
Abbreviations
APTA American Public Transportation Association
BRT bus rapid transit
FFGA full funding grant agreement
FTA Federal Transit Administration
ISTEA Intermodal Surface Transportation Efficiency Act of 1991
LRT light rail transit
MOS minimum operable segment
TEA-21 Transportation Equity Act for the 21st Century
TSUB Transportation System User Benefits
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
A
United States Government Accountability Office Washington, D.C. 20548
June 28, 2005
The Honorable Richard C. Shelby Chairman The Honorable Paul S. Sarbanes
Ranking Minority Member Committee on Banking, Housing, and Urban Affairs
United States Senate
The Honorable Don Young Chairman The Honorable James L. Oberstar Ranking
Democratic Member Committee on Transportation and Infrastructure House of
Representatives
Much of the federal government's share of new capital investment in mass
transportation since the early 1970s has come through the Federal Transit
Administration's (FTA) New Starts program, which awards full funding grant
agreements (FFGA) for fixed guideway rail, bus rapid transit, and ferry
projects across the country.1 An FFGA establishes the terms and conditions
for federal participation in a project, including the maximum amount of
federal funds available for the project, which by statute cannot exceed 80
percent of its net cost. Since fiscal year 1998, the New Starts program
has provided state and local agencies with more than $8.8 billion2 to help
design and construct transit projects throughout the country.
Under the Transportation Equity Act for the 21st Century (TEA-21)3 and
subsequent amendments and other legislation, Congress authorized
approximately $13.5 billion in New Starts commitment authority through
2005.4 Even though the level of program funding is higher than it has ever
been, demand for these funds has also been extremely high. For that
1Fixed guideway systems use and occupy a separate right-of-way for the
exclusive use of public transportation services. They include fixed rail,
exclusive lanes for buses and other high-occupancy vehicles, and other
systems.
2This is the amount appropriated through fiscal year 2005, according to
FTA.
3Pub. L. No. 105-178, 112 Stat. 107 (1998).
4New Starts commitment authority is the amount of funding Congress has
authorized FTA to commit to New Starts projects for a given authorization
period.
reason, TEA-21 directed FTA to prioritize projects for funding by
evaluating, rating, and recommending potential projects on the basis of
specific financial and project justification criteria-including mobility
improvements, cost-effectiveness, environmental benefits, and operating
efficiencies. In applying these criteria, TEA-21 directed FTA to consider
a number of additional factors, including land use and congestion relief.
Furthermore, TEA-21 required FTA to issue regulations for the evaluation
and rating process, which FTA did in December 2000.
TEA-21 also requires us to report each year on FTA's processes and
procedures for evaluating, rating, and recommending New Starts projects
for federal funding and on the implementation of these processes and
procedures.5 This report discusses (1) the number of projects that were
evaluated, rated, and proposed for new FFGAs for the fiscal year 2006
evaluation cycle6 and the proposed funding commitments for New Starts in
the administration's fiscal year 2006 budget request; (2) changes that FTA
has made to the New Starts application, evaluation, rating, and oversight
processes since TEA-21 and how those changes have been communicated to
project sponsors; and (3) how FTA developed the measures it uses to
evaluate and rate projects from the criteria identified in TEA-21 and how
the agency uses these measures in the evaluation process. To address these
objectives, we reviewed the administration's fiscal year 2006 budget
request; the legislation and regulations governing the New Starts process;
and FTA's annual New Starts reports, reporting instructions, and other
program guidance and documentation. We also interviewed FTA officials and
representatives from the American Public Transportation Association, the
New Starts Working Group,7 and the projects that were rated in the fiscal
year 2006 evaluation cycle.8 In addition, we attended FTA's meetings with
project sponsors-the New Starts Roundtables-in April and May
5See the list of related GAO products at the end of this report.
6The fiscal year 2006 evaluation cycle began in May 2004, with the
issuance of the New Starts reporting instructions. Applications were due
in August 2004, and FTA's evaluation of the applications was conducted in
the fall of 2004. The annual report was published in February 2005 and
included funding recommendations for fiscal year 2006.
7The New Starts Working Group is an organization of New Starts project
sponsors, metropolitan planning organizations, and private industry
transit firms who advocate on behalf of the New Starts program and
specific projects.
8FTA rated 27 projects in the fiscal year 2006 cycle, but one of these
projects subsequently withdrew from the New Starts process; therefore, we
interviewed sponsors from 26 projects.
Results in Brief
2005. We conducted our work from November 2004 through May 2005 in
accordance with generally accepted government auditing standards. (See
app. I for more information about our scope and methodology.)
For the fiscal year 2006 evaluation cycle, FTA evaluated and rated 27
projects and identified 4 that were expected to be ready for new FFGAs
before the end of fiscal year 2006. In addition, FTA identified 6 other
projects as potentially being eligible for funding outside of FFGAs. The
administration is requesting a total of $1.5 billion for the New Starts
program (an amount similar to that requested for the last 2 years). This
amount includes $634.6 million for projects with existing FFGAs and $590
million for projects proposed for new FFGAs. The total number of projects
evaluated and rated has declined slightly every year since TEA-21, from 42
projects in the fiscal year 2000 evaluation cycle to 27 projects in the
most recent cycle, while the number of recommended and highly recommended
projects has varied from year to year.
FTA has implemented 16 changes to the New Starts application, evaluation,
rating, and project development oversight processes since the fiscal year
2001 evaluation cycle, using a variety of communication methods, but has
not consistently sought industry input before implementing the changes. By
not soliciting public opinion, FTA is missing an opportunity to obtain
stakeholder buy-in, increase the transparency of the New Starts process,
and lessen potential difficulties project sponsors face in implementing
the changes. Although the impetus for each change varied, FTA officials
stated that, in general, all of the changes were intended to make the
process more rigorous, systematic, and transparent. For example, the
requirement for project sponsors to develop a plan for evaluating the
impacts of the project and the accuracy of ridership projections will
identify lessons learned and hold transit agencies accountable for
results. Some project sponsors told us a few of the changes have helped to
improve the program, while others expressed a variety of concerns about
the effect of some changes. For example, seven project sponsors stated
that the "make the case" document helped them focus on the key benefits of
the projects and three said it helped them "sell" the project to local
decisionmakers or the public. Ten project sponsors noted, however, that
FTA did not provide clear guidance on how to develop the "make the case"
document and eight said that they had to produce multiple iterations of
the document. FTA communicates information about changes to project
sponsors through a variety of different methods, including regulations,
reporting instructions, and the agency's Web site. Project sponsors we
interviewed had varying opinions on the effectiveness of these
communication methods but overall found that the Web site was ineffective
because it was difficult to navigate and information was not consolidated
in one location. Seven of the changes that FTA has made to the New Starts
process since the fiscal year 2001 evaluation cycle underwent rulemaking,
including providing formal notice to the transit industry and soliciting
public comment, while nine changes did not. The Freedom of Information Act
requires federal agencies to publish in the Federal Register notice of
changes to programs, and the Administrative Procedure Act sets out the
rulemaking process required to make changes to agencies' rules and
procedures. FTA officials said that the changes they have made are
consistent with the existing regulations governing the New Starts process
and that some of the changes relate to the project development oversight
process rather than the evaluation and rating process; therefore, in their
opinion, the regulations do not need to be amended. Of the nine changes
that did not undergo rulemaking, six changes-including the
administration's new cost-effectiveness funding recommendation practice
that would generally target funding recommendations to projects that
achieve at least a medium cost-effectiveness rating-were made without FTA
providing any avenues for public review and comment prior to their
implementation. When formal rulemaking is not necessary, there are less
formal options available for soliciting public comment. For example, in
March 2005, FTA solicited public comment on three recent technical changes
to the rating process.
Many of the measures that FTA uses to evaluate projects predate TEA-21 and
have evolved over time through an iterative process-involving FTA,
industry, and Congress-through rulemaking, outreach sessions, and reports.
The measures reflect congressional direction to evaluate projects against
a variety of criteria, including mobility improvements, environmental
benefits, operating efficiencies, and cost-effectiveness, and to consider
other issues, such as land use, in the evaluation of projects. FTA's
regulations governing the New Starts program likewise emphasize a
multiple-measure approach. Using a series of these measures, FTA evaluates
projects against statutorily identified criteria, including three local
financial commitment criteria and five project justification criteria.
Each local financial commitment criterion is counted toward a project's
overall rating. However, only two of the project justification criteria
typically count toward a project's overall rating, despite statements in
the New Starts program regulations that all of the criteria will be used.
Specifically, FTA currently assigns a weight of 50 percent each to the
cost-effectiveness and land use criteria; the other three project
justification criteria are not assigned weights. FTA sought industry input
on whether there should be a weighting system in the last rulemaking
process; however, according to FTA officials, the agency did not receive
input that was useful to inform its policy. Consequently, the regulations
are silent on the weights that should be assigned to each criterion and do
not prohibit FTA from treating various criteria as more important than
others. FTA officials said that the measures for the other three project
justification criteria-mobility improvements, environmental benefits, and
operating efficiencies-do not meaningfully distinguish among projects, and
aspects of the mobility improvements criterion are already captured in the
measure for cost-effectiveness. Although FTA has made improvements to the
measures for land use and cost-effectiveness, it has not yet been able to
identify measures for the other three project justification criteria that
make meaningful distinctions among projects for the purpose of rating and
funding decisions. FTA officials told us that they will initiate a formal
rulemaking process after the program is reauthorized, at which point all
New Starts policies and procedures will be reevaluated.
This report makes recommendations to the Secretary of Transportation to
ensure that FTA's New Starts regulations reflect its weighting policy, to
improve the measures used to evaluate projects so that all criteria named
in statute can be used to develop a project's rating, to publish future
changes to the New Starts program in the Federal Register, and to solicit
industry comment on changes, through rulemaking or some other process, as
appropriate. To ensure that transit agencies have clear information on the
New Starts program, we are also recommending that FTA consolidate
guidance, reporting instructions, and other New Starts program information
in one location on its Web site. The Department of Transportation,
including FTA, reviewed a draft of this report. FTA officials generally
agreed with the report's findings, conclusions, and recommendations.
TEA-21 authorized a total of $36 billion in "guaranteed" funding for a
Background
variety of transit programs, including financial assistance to states and
localities to develop, operate, and maintain transit systems.9 Under one
of these programs, New Starts, FTA identifies and selects fixed-guideway
transit projects for funding-including heavy, light, and commuter rail;
ferry; and certain bus projects (such as bus rapid transit). FTA generally
9"Guaranteed" funds are subject to a procedural mechanism designed to
ensure that a minimum amount of funding is made available each year over
the life of the project.
Page 5 GAO-05-674 Public Transportation
funds New Starts projects through FFGAs, which establish the terms and
conditions for federal participation in a New Starts project and also
define a project's scope, including the length of the system and the
number of stations; its schedule, including the date when the system is
expected to open for service; and its cost.
To obtain an FFGA, a project must progress through a local or regional
review of alternatives and meet a number of federal requirements,
including providing information for the New Starts evaluation and rating
process (see fig. 1). As required by TEA-21, New Starts projects must
emerge from a regional, multimodal transportation planning process. The
first two phases of the New Starts process-systems planning and
alternatives analysis-address this requirement. The systems planning phase
identifies the transportation needs of a region, while the alternatives
analysis phase provides information on the benefits, costs, and impacts of
different corridor-level options, such as rail lines or bus routes. The
alternatives analysis phase results in the selection of a locally
preferred alternative-which is intended to be the New Starts project that
FTA evaluates for funding, as required by statute. After a locally
preferred alternative is selected, project sponsors submit a request to
FTA for entry into the preliminary engineering phase.10 Following
completion of preliminary engineering and federal environmental
requirements, the project may be approved by FTA to advance into final
design,11 after which the project may be approved by FTA for an FFGA and
proceed to construction, as provided for in statute. FTA oversees grantee
management of projects from the preliminary engineering phase through
construction and evaluates the projects for advancement into each phase of
the process, as well as annually for the New Starts report to Congress. We
have recognized the New Starts program as a good model that the federal
government could use for approving other transportation projects.
10During the preliminary engineering phase, project sponsors refine the
design of the proposal, taking into consideration all reasonable design
alternatives, which results in estimates of costs, benefits, and impacts
(e.g., financial or environmental). According to FTA officials, to gain
approval for entry into preliminary engineering, a project must (1) have
been identified through the alternatives analysis process, (2) be included
in the region's long-term transportation plan, (3) meet the statutorily
defined project justification and financial criteria, and (4) demonstrate
that the sponsors have the technical capability to manage the project
during preliminary engineering. Some federal New Starts funding is
available to projects for preliminary engineering activities.
11Final design is the last phase of project development before
construction and may include right-of-way acquisition, utility relocation,
and the preparation of final construction plans and cost estimates.
Figure 1: New Starts Planning and Development Process
Source: FTA.
To help inform administration and congressional decisions about which
projects should receive federal funds, FTA assigns ratings based on a
variety of financial and project justification criteria, as defined by its
program regulations, and then assigns an overall rating. These criteria
are identified in TEA-21 and reflect a broad range of benefits and effects
of the proposed project, such as cost-effectiveness, as well as the
ability of the project sponsor to fund the project and finance the
continued operation of its transit system (see fig. 2). Projects are rated
at several points during the New Starts process-as part of the evaluation
for entry into preliminary engineering and final design, and yearly for
inclusion in the New Starts annual report.
Figure 2: New Starts Project Evaluation Criteria
Source: FTA.
aOther factors can include environmental justice and equity issues,
economic development initiatives, innovative financing, etc.
FTA assigns the proposed project a rating of high, medium-high, medium,
low-medium, or low for each criterion, then assigns a summary rating for
local financial commitment and project justification. Finally, FTA
develops an overall project rating of "highly recommended," "recommended,"
"not recommended," or "not rated." (See table 1 for the criteria FTA uses
to evaluate projects.) The exceptions to this process are statutorily
"exempt" projects, which are those that request less than $25 million in
New Starts funding. These projects are not required to submit project
justification information-although FTA encourages them to do so-and do not
receive ratings from FTA; thus, the number of projects in preliminary
engineering or final design may be greater than the number of projects
evaluated and rated by FTA.
Table 1: FTA's Criteria for Assigning Overall Project Ratings
Overall rating category Criteria
Highly recommended Requires at least a medium-high for both
the financial and project justification
summary ratings
Recommended Requires at least a medium for both the financial and
project justification summary ratings
Not recommended Assigned to projects not rated at least medium for both
the financial and project justification summary ratings
Not rated Indicates that insufficient information was
submitted or that FTA has serious concerns
about the information submitted because
the underlying travel forecasting assumptions used by the
project sponsor may have inaccurately represented the
benefits of the project
Source: FTA.
As required by statute, the administration uses the FTA evaluation and
rating process, along with the stage of development of New Starts
projects, to decide which projects to recommend to Congress for funding.12
Although many projects receive an overall rating of "recommended" or
"highly recommended," only a few are proposed for FFGAs in a given fiscal
year. FTA proposes "recommended" or "highly recommended" projects for
FFGAs when it believes that the projects will be able to meet certain
conditions during the fiscal year that the proposals are made. These
conditions include the following:
o All non-New Starts funding must be committed and available for the
project.
12The administration's funding recommendations are made in the President's
budget and are included in FTA's annual New Starts report to Congress,
which is released each February in conjunction with the President's
budget.
Page 9 GAO-05-674 Public Transportation
FTA Identified Four New Projects for FFGAs and Requested $1.5 Billion for
Fiscal Year 2006
o The project must be in the final design phase and have progressed to
the point where uncertainties about costs, benefits, and impacts
(e.g., environmental or financial) are minimized.
o The project must meet FTA's tests for readiness and technical
capacity, which confirm there are no cost, project scope, or local
financial commitment issues remaining.
Of the 34 projects in preliminary engineering or final design for the
fiscal year 2006 cycle, 27 were evaluated and rated and 7 were statutorily
exempt from the rating process. Four projects were recommended for funding
with the expectation that they would be ready for new FFGAs before the end
of fiscal year 2006, and an additional 6 projects were identified as
potentially receiving a recommendation for New Starts funding outside of
FFGAs. The administration's fiscal year 2006 budget proposal requests a
total of $1.5 billion for the New Starts program, an amount similar to
requests for the past 2 fiscal years. (See app. II for the
administration's 2006 budget proposal and FTA's project ratings.)
FTA Evaluated and Rated 27 Projects and Proposed 4 for FFGAs in Fiscal
Year 2006
FTA's Annual Report on New Starts: Proposed Allocations of Funds for
Fiscal Year 2006 ("annual report") listed a total of 34 projects in
preliminary engineering and final design, and FTA evaluated and rated 27
of them.13 Seven were statutorily exempt from being rated because they
requested less than $25 million in New Starts funding. Of the 27 projects
that were rated, 2 were highly recommended, 12 were recommended, 8 were
not recommended, and 5 were designated "not rated." In its annual report,
FTA said that projects were designated as "not rated" because they did not
submit the required information or because of FTA's continuing concerns
about the reliability of the transportation benefits forecast for these
projects. According to FTA, a principal source of these concerns was
inconsistent assumptions used in defining the baseline alternative and the
proposed New Starts project, making it difficult to isolate the impacts of
the proposed project. In some cases, the local travel demand models were
inconsistent with FTA guidance and good planning practice. FTA is
13FTA does not evaluate and rate projects with existing FFGAs, that are in
alternatives analysis, or that are statutorily exempt because they are
requesting less than $25 million in New Starts funding.
Page 10 GAO-05-674 Public Transportation
currently working with the sponsors of these projects to improve the
forecasts.
As shown in figure 3, the combined number of recommended and highly
recommended projects declined sharply from 27 in the fiscal year 2003
evaluation cycle to 14 in the fiscal year 2004 evaluation cycle, while the
combined number of not recommended and not rated projects rose from 6 to
18. As we previously reported, this was primarily due to difficulties
project sponsors encountered when implementing the new cost-effectiveness
measure-the Transportation System User Benefits (TSUB) measure14-and
adjusting to FTA's preference policy that favors projects that seek a
federal New Starts share of no more than 60 percent of the total project
costs.15 According to FTA, the information that was provided by project
sponsors in their reports on TSUB highlighted previously unknown problems
with many local models that forecast travel demand. FTA first incorporated
both of these changes beginning with the fiscal year 2004 evaluation
cycle. Project ratings generally improved (i.e., there were more projects
with recommended ratings and fewer with not recommended ratings) in the
fiscal year 2005 evaluation cycle as project sponsors improved their
financial plans, grew more comfortable with the new cost-effectiveness
measure, and made corrections and improvements to their models that
forecast travel demand, according to FTA.16 While the combined number of
not rated and not recommended projects was approximately the same in the
fiscal year 2005 and 2006 evaluation cycles, the number of projects
receiving ratings of at least recommended has decreased slightly again.
Unlike in previous years, however, there was no obviously identifiable
reason for this change, except for the fact that some projects moved out
of the ratings pool and into construction, according to FTA.
14FTA uses the incremental cost per hour of TSUB for assessing a project's
cost-effectiveness.
15For more information about the problems project sponsors encountered in
implementing these changes, see GAO, Mass Transit: FTA Needs to Provide
Clear Information and Additional Guidance on the New Starts Ratings
Process, GAO-03-701 (Washington, D.C.: June 23, 2003).
16See GAO, Mass Transit: FTA Needs to Better Define and Assess Impact of
Certain Policies on New Starts Program, GAO-04-748 (Washington, D.C.: June
25, 2004).
Figure 3: Distribution of New Starts Projects in Preliminary Engineering
and Final Design for Fiscal Year 2000 to 2006 Evaluation Cycles
Number 60 50
40
30
20
10
0 2000 2001 2002 2003 2004 2005 2006 Fiscal year
Highly recommended Recommended Not recommended Not rated Exempt Source:
GAO analysis of FTA data.
Note: The pool of projects evaluated and rated each year changes, as some
projects from the previous year receive FFGAs and begin construction,
while new projects enter preliminary engineering and are evaluated and
rated for the first time.
The fiscal year 2006 rating cycle saw the smallest total number of
projects in preliminary engineering and final design during the TEA-21
period. The number of projects evaluated and rated has decreased slightly
every year, from 42 projects in the fiscal year 2000 rating cycle to 27
projects in the most recent cycle.17 During this same time frame, the
number of exempt projects grew steadily through the fiscal year 2004 cycle
but has been declining ever since. FTA officials suggested that these
trends could be the result of a combination of factors. First, many of the
projects that TEA-21 authorized have worked their way through the New
Starts process and obtained FFGAs, and thus are not subject to the annual
evaluation and rating process. At the same time, new projects entered
preliminary engineering each year and were rated for the first time. FTA
officials speculated that additional projects would be included in
reauthorization legislation and would enter the New Starts pipeline over
the course of the reauthorization period. Second, FTA has increased the
level of scrutiny it applies to projects attempting to advance from
alternatives analysis to preliminary engineering to help ensure that only
the strongest projects enter the New Starts pipeline. Third, FTA has
worked with project sponsors to reduce the number of inactive projects in
the New Starts pipeline. Fourth, since the time TEA-21 was enacted, some
state and local transportation agencies have been feeling the impact of
local budget constraints, making it more difficult to secure local funding
for proposed New Starts projects. This could be exacerbated by FTA's
policy of favoring projects that seek a federal New Starts share of no
more than 60 percent of the total project costs. As we reported in 2004,
several project sponsors told us that FTA's push for a lower federal New
Starts share would likely affect their decision to advance future transit
projects.18 Therefore, we recommended that FTA examine the impact of the
preference policy on projects in the evaluation process. FTA is
considering how to conduct such an examination.
FTA's evaluation process informed the administration's recommendation to
fund four projects that are expected to be ready for new FFGAs before the
end of fiscal year 2006, including Charlotte, South Corridor Light Rail
Transit; New York, Long Island Rail Road East Side Access; Phoenix,
Central Phoenix/East Valley Light Rail Transit Corridor; and Pittsburgh,
17While the total number of projects in preliminary engineering and final
design has fluctuated from year to year, this includes the statutorily
exempt projects. Once these projects are excluded from the total, the
number declines steadily each year.
18GAO-04-748.
Page 13 GAO-05-674 Public Transportation
Administration's Proposed Fiscal Year 2006 Budget Requests Similar Amount of
Funding to Previous Years
North Shore Light Rail Transit Connector. The total capital cost of these
four projects is estimated to be $9.9 billion, of which the total federal
New Starts share is expected to be $3.6 billion. FTA executed an FFGA for
the Phoenix project in January 2005 and for the Charlotte project in May
2005. The other two projects are expected to be ready for FFGAs before the
end of fiscal year 2006. According to FTA, the remaining projects that
received overall ratings of recommended or highly recommended do not yet
pass FTA's readiness tests for FFGAs.
The administration also proposed reserving $158.6 million in New Starts
funding for final design and early construction activities for as many as
six "other projects," including San Diego, Mid-Coast Light Rail Transit
Extension; Denver, West Corridor Light Rail Transit; New York, Second
Avenue Subway; Washington County (Oregon), Wilsonville to Beaverton
Commuter Rail; Dallas, Northwest-Southeast Light Rail; and Salt Lake City,
Weber County to Salt Lake City Commuter Rail. These six projects were in
or nearing final design, received overall highly recommended or
recommended ratings, and had cost-effectiveness ratings above "low."
According to FTA officials, no other projects met these criteria. The
annual report did not specify amounts for particular projects to ensure
that the project is moving forward as anticipated prior to making specific
funding recommendations to Congress, according to FTA officials, because
projects may encounter unexpected financial or other obstacles that slow
their progress. For example, FTA told us that the sponsor for one of the
projects is considering a significant expansion of the project scope,
which would put it back in preliminary engineering and render it
ineligible for the funds FTA proposed reserving for the "other projects"
for fiscal year 2006. Reserving funds for these projects without
specifying a particular amount for any given project will allow the
administration to make "real time" funding recommendations when Congress
is making appropriations decisions. FTA does not anticipate that all of
the six projects will be recommended for funding in fiscal year 2006.
The administration's fiscal year 2006 budget proposal requests that $1.5
billion be made available for the New Starts program, an amount similar to
that requested in the last two fiscal years. Figure 4 illustrates the
specific budget allocations the administration has proposed for fiscal
year 2006, including the following:
o $634.6 million would be allocated among the 16 projects with existing
grant agreements,
o $590 million would be allocated to four projects expected to be
proposed for new FFGAs,
o $158.6 million would be allocated to as many as six "other" projects
to continue project development, and
o $122.5 million is reserved to be allocated among projects in
preliminary engineering, at the discretion of Congress and as provided
in law.19
Figure 4: Total New Starts Funding for Fiscal Year 2006 Equals $1.5
Billion
1%
Ferry capital projects, AK and HI ($10 million)
1%
Oversight activities ($15 million)
Preliminary engineering ($122 million)
Other projects ($159 million)
Anticipated FFGAs ($590 million)
Existing FFGAs ($635 million)
Source: GAO analysis of FTA data.
Note: FTA is authorized to use up to 1 percent of amounts made available
for the New Starts program for project management oversight activities.
TEA-21 requires that specified amounts of New Starts funds be set aside
annually for projects in Alaska and Hawaii, for new fixed guideway systems
and
19TEA-21 limits the amount of New Starts funding that can be used for
purposes other than final design and construction to not more than 8
percent of funds appropriated.
Page 15 GAO-05-674 Public Transportation
FTA Has Implemented a Number of Changes to the New Starts Program, Some
Without Project Sponsors' Input
extensions to existing systems that are ferry boats or ferry terminal
facilities or that are approaches to ferry terminal facilities.
FTA had approximately $2.2 billion in commitment authority-that is, the
amount of funding authorized for New Starts projects-remaining as of May
2005. TEA-21 and subsequent amendments and legislation provided FTA the
authority to make about $13.5 billion in funding commitments for New
Starts projects. Surface transportation programs, including the New Starts
program, were scheduled to expire in September 2003, but have subsequently
been extended through June 2005.20 According to FTA officials, the
commitment authority for fiscal year 2006 and beyond will be addressed in
the next surface transportation authorization legislation. However, FTA
officials told us that there will not be sufficient commitment authority
remaining to execute all four proposed FFGAs until additional commitment
authority is provided through congressional authorization.
FTA has made 16 changes to the New Starts application, evaluation, rating,
and project development oversight processes since the fiscal year 2001
evaluation cycle-the first full evaluation and rating cycle after the
enactment of TEA-21. FTA has used a variety of written and electronic
methods to communicate information about these changes, although it
primarily relies on reporting instructions, roundtables, and workshops.
For nine of these changes, FTA did not publish information about the
change in the Federal Register or institute a rulemaking process and for
six of these nine changes did not provide any avenues for public review
and comment prior to implementing the changes. FTA has said that all of
the changes it has made are consistent with the evaluation framework
outlined in the existing regulations governing the New Starts process.
Some project sponsors we interviewed thought certain changes helped to
improve the process; however, a considerable number of project sponsors
expressed concern that they did not have an opportunity to comment on many
of the changes before they were implemented and have experienced
considerable challenges while attempting to incorporate some of the
changes. For example, some changes have required project sponsors to
devote additional agency resources, as well as time.
20The Surface Transportation Extension Act of 2005 (P.L. 109-14) extended
the programs until June 30, 2005.
Page 16 GAO-05-674 Public Transportation
FTA Has Made 16 Changes Intended to Improve the Application, Evaluation,
Rating, and Project Development Oversight Processes since the Fiscal Year
2001 Evaluation Cycle
We identified 16 changes that FTA made to the New Starts application,
evaluation, rating, and project development oversight processes since the
fiscal year 2001 evaluation cycle.21 These changes range from requiring
that projects undergo a risk assessment to instituting new practices for
funding recommendations. (See table 2 for complete list of changes.) For
example, FTA made two significant changes to the evaluation and rating
process for the fiscal year 2004 evaluation cycle. First, FTA implemented
the TSUB measure as a variable in the calculation of cost-effectiveness
and mobility improvements.22 The new measure is intended to calculate the
change in the amount of travel time and costs that people incur for taking
a trip. This is a more comprehensive measure than the old "cost per new
rider" measure because it takes into account a broader set of benefits to
transit riders, including new and existing transit riders. Second, in
response to language contained in appropriations committee reports, FTA
instituted a preference policy favoring projects that seek a federal New
Starts share of no more than 60 percent of the total project cost. As
shown in table 2, FTA implemented changes for a variety of reasons,
including simplifying the New Starts process and focusing more on results
and performance. Although the impetus for each change varied, FTA
officials stated that, in general, all the changes were intended to make
the process more rigorous, systematic, and transparent. For example, the
requirement for project sponsors to develop a plan for evaluating the
impacts of the project and the accuracy of travel forecasts-that is, a
Before and After study-will identify lessons learned and hold transit
agencies accountable for results. In our previous work we have commented
on a number of these changes and made some recommendations for improving
the New Starts process. (See app. III for a list of recommendations from
previous reports.)
21We compiled a list of changes that FTA has made to the application,
evaluation, rating, and project development oversight processes from
TEA-21, FTA regulations, annual reports and reporting instructions, and
previous GAO reports. We verified this list of changes with FTA officials
and project sponsors. We chose the fiscal year 2000 evaluation cycle as
our baseline because this was the first cycle to reflect some TEA-21
changes. TEA-21 formalized many pre-existing FTA practices, so we did not
include those in our review. FTA has also made other minor modifications
to the New Starts process, which we did not include in our review.
22This change was included in the formal rulemaking process initiated
after TEA-21 and was published as part of the New Starts program
regulations in December 2000.
Table 2: Changes to the New Starts Application, Evaluation, Rating, and
Project Development Oversight Processes since the Fiscal Year 2001
Evaluation Cycle
Evaluation Published Formal
in
Brief Method and cycle Reason Federal opportunity
description of date effective for Register? for
Change change introduced change comment?
Additional Several Transportation Fiscal Required Yes Yes
subfactors additional Equity Act for year (FY) by
added to "statutory the 21st 2003 TEA-21
the land considerations" Century
use were added to (TEA-21)/New
criterion the land use
criterion,
including the
cost of sprawl,
infrastructure Starts
cost savings regulations,
due to compact June 1998/Dec.
land use, and 2000
population
density and
current transit
ridership in a
corridor.
Baseline FTA eliminated New Starts FY Simplify New Yes Yes
alternative the requirement regulations, 2003 Starts
introduced for the separate Dec. 2000 process and
no-build and address
transportation project
system management sponsor
alternatives, a concerns
and instead about undue
requires that the reporting
proposed New burden
Start be
evaluated against
a single
"baseline
alternative"
(i.e., the best
that can be done
without the New
Starts
investment).
Pedestrian Pedestrian New Starts FY Formalize Yes Yes
mobility mobility had been regulations, 2003 agency Yes Yes
formally a component of Dec. 2000 New FY practice
incorporated FTA's land use Starts 2003 Address
into land evaluation, as regulations, industry
use described in FTA Dec. 2000 concerns that
criterion guidance issued FTA was
"Employment each year. undervaluing
near However, the destination
stations" regulations trips
measure formally
added to incorporated an
mobility element for
improvements pedestrian
criterion mobility into the
land use
criterion. FTA
added a new
factor to
calculate
destinations for
jobs within a
half-mile radius
of boarding
points on the new
system,
complementing the
existing factor
that measures
low-income
households within
a half-mile
radius of
boarding points.
(Continued From Previous Page)
Brief Method and Evaluation Published Formal
description of date cycle Reason for in opportunity
change introduced effective change Federal for
Change Register? comment?
Decision FTA instituted New Starts FY 2003 Emphasize Yes Yes
rule for a decision both
financial rule that regulations, capital and
required Dec.
projects to
summary receive at 2000 operating
rating least a plans
"medium"
rating for
both capital
and
operating
plans to get
an
overall
"recommended"
rating.
Before Project FY 2003 Yes Yes
and After sponsors New Starts Overall
study seeking a full regulations, government
required funding grant Dec.
b agreement
(FFGA) for 2000 focus on
their New
Starts
project must performance
submit to FTA and
a
plan for the results
collection and
analysis of
information
leading to the
identification
of the impacts
of the project
and the
accuracy of
the
forecasts that
were prepared
during project
planning and
development.
Transportation FTA revised its New Starts FY Address Yes Yes
cost- 2004
System User effectiveness and regulations, industry's
mobility Dec.
Benefits improvements 2000 concerns about
(TSUB) criteria by
measure adopting the TSUB cost per new
measure
introduced that includes rider index
benefits for
both new and
existing transit
system riders.
Project The Reporting FY Other No No
justification cost-effectiveness instructions, 2004 criteria's No No
criteria and land use c June 2003 FY measures do
weights criteria were each Annual report 2004 not
changed 60% weighted 50%. In for FY 2004 meaningfully
preference the previous evaluation distinguish
policy rating cycle, FTA cycle among projects
instituted attempted to use FY 2004
all five project Appropriations
justification Conference
criteria in the Report
rating process.
FTA instituted a
preference policy
in its ratings
process favoring
current and future
projects that do
not request more
than a 60% federal
share. To achieve
this, FTA changed
its criterion
related to capital
finance plans to
give projects
seeking a federal
share greater than
60% a "low"
financial rating.
(Continued From Previous Page)
Evaluation Published Formal
in
Brief Method and cycle Reason for Federal opportunity
description of
Change change date effective change Register? for
introduced comment?
Risk The Risk New Starts FY 2005 Implemented No No
Assessment is an and for
assessments FTA management FTA project
introduced b and oversight construction management
tool intended to roundtables,
d
identify the April/May oversight
issues that 2003
could
affect schedule or cost, as
well as the probability that
a
cost
estimate
will be met.
"Make the case" FTA requires Reporting FY 2005 To have a No No
all project short
document sponsors to instructions, document that
submit a c
three-
required b page June 2003 describes the
narrative
that
justifies
why the New individual
Starts merits
project
is the best of a project
possible from
alternative the
and why it community's
is
needed. perspective
FTA To allow
Travel forecasts encouraged Reporting FY 2006 technical No No
(but did not
requested in require) instructions, assistance to
grantees to c be
submit
advance of their travel April 2004 provided in
forecasts by time
June
application 30, 2004, in for the
advance of project to
the
deadline Aug. 20 be rated in
deadline for the
the rest
of the upcoming
information. annual
report
Cost- Generally, the New Starts FY 2007 Address No No
administration
effectiveness will target its Workshope and concerns
practiceb funding FTA "Dear raised by the
recommendations DOT
to
projects that Colleague" Inspector
achieve a cost- letter, f
effectiveness March 2005 General,
rating of
medium or Congress, and
higher to be
recommended for Office of
funding.
Previously, the Management
and
administration Budget
would
recommend projects for
funding that had a low-
medium cost-effectiveness
rating, if they met all other
criteria.
FTA updated To be
Standard cost the useful New Starts FY 2007 consistent No Yes
life
categories assumptions Workshope and with useful
for various life
required categories of FTA "Dear estimates
assets and is
requiring Colleague"
project letter, f
sponsors to
report costs March 2005
in standard
categories.
Cost- FTA adjusted New Starts FY 2007 Cost- No Yes
cost-
effectiveness effectiveness Workshope and effectiveness
rating
rating breakpoints FTA "Dear breakpoints
for inflation were
breakpoints using the Colleague" first
Gross letter, f established
Domestic
adjusted for Price Index. March 2005 in 2002
inflation
(Continued From Previous Page)
Evaluation Published Formal
in
Brief Method and cycle Reason for Federal opportunity
description date effective change Register? for
Change of change introduced comment?
2030 Project
planning sponsors will New Starts FY 2007 To maintain No Yes
be
horizon permitted to Workshope consistency
permitted use either a and with
2025 or 2030 FTA "Dear what local
planning Colleague" metropolitan
horizon for letter, f
estimating
the
project's March 2005 planning
costs and
benefits
to maintain organizations
consistency are
with
the horizon using
year used by
the
local
metropolitan
planning
organization.
Sources: GAO and FTA.
aThe transportation system management alternative is equivalent to the New
Starts baseline alternative for most New Starts projects.
bFTA officials identified these changes as involving project development
oversight that do not affect project ratings.
cFTA uses reporting instructions to provide project sponsors with
technical and procedural assistance on the application and reporting of
New Starts criteria.
dFTA uses New Starts roundtables to share information and experiences with
project sponsors, identify and discuss common issues, and generate
suggestions for improving the planning and development process.
eThe American Public Transportation Association typically hosts an annual
legislative conference at which FTA conducts an information session on the
New Starts program.
fThe Dear Colleague letter is periodically sent from the FTA Administrator
to project sponsors regarding issues associated with the New Starts
program.
Some project sponsors we interviewed thought that certain changes helped
to improve the process. For instance, seven project sponsors stated that
the "make the case" document helped them focus on the key benefits of the
projects and three said it helped them "sell" the project to local
decisionmakers or the public. In addition, many project sponsors
acknowledged that the TSUB measure is more comprehensive than the old
"cost per new rider" measure. However, project sponsors expressed a
variety of concerns about the effects the changes had on the steps
required to complete the application process and about FTA's
implementation of the changes. For instance, for the "make the case"
document, 15 project sponsors stated that FTA did not create clear
expectations, 10 noted that FTA prepared no written guidance, and 5
indicated that FTA did not provide specific examples or templates.23 A few
of these project sponsors also stated that they did not feel like they
understood what FTA wanted for the "make the case" document, and eight
said that they had to produce multiple iterations of the document. Twelve
project sponsors also mentioned they were not clear about how FTA was
using the document. In addition, when asked what effect the implementation
of the TSUB measure had on their project, 13 of 26 project sponsors said
it made the application process more expensive, and 20 of 26 said that
this measure required them to spend significantly more time to complete
the application. Similar comments were made by project sponsors about
other changes, including the risk assessment and Before and After study
requirements.24 For example, when asked what effect the risk assessment
requirement has had on their project, 4 of the 26 project sponsors said it
resulted in a more rigorous or systematic evaluation and rating process, 7
said it made the process more expensive, and 5 said it delayed their
project. Three project sponsors noted that each of the individual
requirements add to the overall workload and eventually result in the
application process becoming a full-time project. However, some project
sponsors observed that as they become more familiar with each change and
as FTA issues more guidance, such as for using FTA's software to calculate
the TSUB value, the change has become less burdensome.
23In response to project sponsor requests, in June 2005, FTA e-mailed two
examples of the "make the case" document to those project sponsors who had
registered for the 2005 New Starts roundtables.
24Both the TSUB and the Before and After study requirements were included
in the formal rulemaking process, and were implemented as part of the New
Starts program regulations in December 2000.
Page 22 GAO-05-674 Public Transportation
In Spring 2005 FTA Introduced New Cost-Effectiveness Practice and
Technical Changes to the Cost-Effectiveness Calculation
In March 2005, FTA announced a new practice for the New Starts program
whereby the administration will generally target funding recommendations
to projects able to achieve at least a medium or higher cost-effectiveness
rating. FTA announced this change through a "Dear Colleague" letter sent
to FTA grantees, including current New Starts project sponsors. The letter
was also posted on the home page of FTA's Web site. The administration's
previous policy had been to recommend projects for funding that received
at least a medium-low cost-effectiveness rating, provided that they met
all other criteria and project readiness requirements. In the letter to
project sponsors, FTA explained that the impetus for change was the
concerns that GAO, the Office of Management and Budget, the Department of
Transportation's Inspector General, and Congress raised about recommending
projects for funding that had medium-low cost-effectiveness ratings.25 In
the letter, and in subsequent communication to project sponsors, FTA also
stated that the practice applies to a recommendation for funding and not
to the way the project's overall rating is determined. For example, a
project can still receive an overall recommended project rating with a
medium-low cost-effectiveness rating and can advance from preliminary
engineering to final design with this rating; however, as a general rule,
the project would not be recommended for funding by the administration.
FTA has said that this new practice will help it to further prioritize and
distinguish among projects for federal funding, which is important given
current fiscal challenges and the resulting need to maximize the benefit
of every federal dollar invested in transportation.
According to FTA officials, the four New Starts projects with anticipated
FFGAs for the fiscal year 2006 cycle (see app. II) will not be affected by
the new practice. However, FTA has said that the six New Starts projects
that are categorized as "other projects" in the fiscal year 2006 annual
report- and therefore eligible for a portion of the $158.6 million in New
Starts funds FTA reserved-will be subject to the new cost-effectiveness
funding recommendation practice and will continue to be subject to this
practice when they apply for an FFGA. According to FTA, two of the six
"other projects" currently do not meet this standard, and FTA is working
with them to improve their cost-effectiveness rating. Six project sponsors
we
25For example, see GAO-04-748 and "The Rating and Evaluation of New Starts
Transit Systems," Statement of the Honorable Kenneth M. Mead, Inspector
General, U.S. Department of Transportation, before the Committee on
Appropriations, Subcommittee on Transportation, Treasury and Independent
Agencies, U.S. House of Representatives, April 28, 2004.
Page 23 GAO-05-674 Public Transportation
interviewed expressed concern as to whether they would be able to make the
necessary modifications to their projects in order to earn a higher
cost-effectiveness rating by the next evaluation cycle-which begins in
August 2005. Also, seven project sponsors commented that FTA puts too much
emphasis on the cost-effectiveness measure, and four indicated that the
new cost-effectiveness practice increases this emphasis. FTA officials
stated that FTA and the administration consider more than just
cost-effectiveness in making funding recommendations, noting that every
project that received a not recommended rating in the last 2 years did so
because of its poor financial summary rating.
In addition to the new cost-effectiveness practice in the March "Dear
Colleague" letter, FTA also proposed five technical changes to the way
cost-effectiveness is calculated and asked for industry comment on these
proposed changes by April 1, 2005. All comments were posted on the
Department of Transportation's online docket26 and were available for
public review. The proposed changes included (1) adjusting the
cost-effectiveness rating breakpoints (i.e., low, low-medium, medium,
medium-high, high) for inflation, and possibly applying a regional index
in an effort to address cost differences across the country; (2)
permitting project sponsors to utilize either a 2025 or 2030 planning
horizon to be consistent with metropolitan planning organizations'
regional planning processes; (3) permitting standardized costs and the
proposed adjustments to useful life estimates, which clarify and lengthen
these estimates for a number of assets; (4) permitting modal constants for
new guideway modes as a means of enabling travel models to estimate the
effect of improvements to transit service quality beyond the time and cost
measures already accounted for in the travel models (such as comfort and
reliability); and (5) excluding some soft costs (e.g., administrative
expenses) from the calculation of annualized capital costs for the purpose
of calculating cost-effectiveness. FTA officials told us that they wanted
to obtain industry comment on the proposed changes and to use this
feedback to help decide which proposed changes to adopt. Officials from
FTA also said the deadline for industry comments was driven by the
reporting deadlines for the New Starts annual evaluations and the need to
promptly release reporting instructions for the News Starts program. On
the basis of FTA's review of the proposed changes
26The Department of Transportation publishes and stores on-line
information about proposed and final regulations, copies of public
comments on proposed rules, and related information on its Docket
Management System. The department uses this docketed material when making
regulatory and adjudicatory decisions, and makes docketed material
available for review by interested parties.
Page 24 GAO-05-674 Public Transportation
FTA Uses Multiple Methods to Communicate Information about Changes
and project sponsor responses, FTA announced in an April 29th "Dear
Colleague" letter its decision to incorporate three of the five proposed
changes for the fiscal year 2007 cycle-adjusting rating breakpoints for
inflation, permitting the 2030 design year forecast, and permitting the
use of the standardized cost categories' useful life assumptions to
calculate annualized capital costs for the purpose of calculating
cost-effectiveness. FTA stated that further research is needed on the
other two proposed changes. FTA incorporated the three technical changes
into its fiscal year 2007 reporting instructions, which were released to
project sponsors on May 3, 2005.
FTA communicates information about changes to project sponsors through a
variety of different methods, including annual reporting instructions,
"Dear Colleague" letters to project sponsors, conversations with project
sponsors, roundtables and workshops with FTA officials and project
sponsors, and the FTA Web site. FTA officials indicated that they most
heavily rely on regulations, reporting instructions, and roundtables and
workshops to communicate information about changes to the New Starts
application, evaluation, rating, and project development oversight
processes. In particular, FTA officials told us that they use the
workshops as a two-way communication vehicle, during which they can
explain the New Starts application and evaluation process to project
sponsors. The workshops are usually held two to four times a year in
conjunction with a transit industry conference. For example, during the
American Public Transportation Association (APTA) legislative conference
in early March 2005, FTA officials held a workshop to discuss the New
Starts process as well as the administration's new cost-effectiveness
funding recommendation practice. The agency also holds two New Starts
roundtables each year to explain the application and evaluation process
and allow project sponsors the opportunity to have an open discussion with
FTA officials as well as share information on best practices. FTA used
this year's New Starts roundtables in New York and San Francisco to, among
other things, respond to questions about the recently introduced changes.
FTA's annual reporting instructions also describe changes to the
application and evaluation process, in addition to providing guidance to
project sponsors on preparing their New Starts submittal. For instance, in
the reporting instructions for the New Starts report for the fiscal year
2005 evaluation cycle, FTA introduced the change to the weighting system
used to calculate the project justification rating.
FTA Has Not Consistently Used the Public Notice or Rulemaking Process to
Introduce Changes or Solicit Industry Comment on Proposed Changes
Project sponsors told us that they generally learn about changes through
one or more of FTA's communication methods, although they have varying
views on the effectiveness of the different methods. Most project sponsors
we interviewed said that they typically view the roundtables and workshops
and conversations with FTA officials as generally or very effective
methods for learning about changes. However, some concerns were also
raised about these communication methods. First, three project sponsors
stated that they feel compelled to attend all of FTA's workshops and
roundtables in order to keep up with the changes to the New Starts process
because these meetings are the primary methods FTA uses to introduce
changes. Others noted that they have received inconsistent information
about changes depending on the source. As a result, 18 of 26 project
sponsors stated they prefer to obtain information about the changes
directly from FTA officials-and 11 of these project sponsors stated that
they prefer to receive the information in writing for documentation
purposes. Eleven of the 26 project sponsors we interviewed said that FTA's
Web site was generally or very ineffective as a method for communicating
information about changes. Project sponsors cited a variety of challenges
in using the Web site as a source of information about changes to the New
Starts program. For example, some project sponsors stated the Web site was
difficult to navigate and had a poor search engine, while others stated
they had difficulty finding information about the New Starts program and
that in some cases the information they retrieved from the Web site was
not up to date. Several project sponsors indicated that maintaining a
central repository for all information related to the New Starts program
on FTA's Web site would be helpful.
Of the 16 changes that FTA has made to the New Starts application,
evaluation, rating, and project development oversight processes since the
fiscal year 2001 evaluation cycle, seven underwent rulemaking, including
providing formal notice to the transit industry and soliciting public
comment, while nine changes did not (see table 2). The Freedom of
Information Act requires federal agencies to publish in the Federal
Register notice of changes to programs and the Administrative Procedure
Act sets out the rulemaking process-which would include notifying the
public and soliciting and considering comments, among other
things-required to make changes to agencies' rules and procedures. FTA
last undertook rulemaking for the New Starts program in 1999 at the
direction of TEA-21 and issued regulations in December 2000.27 Officials
from FTA told us that they have not amended their regulations to
incorporate the nine changes made to the New Starts process since that
time because, in their opinion, the changes are within the framework of
the current regulations. FTA officials also said that some of the
changes-such as the risk assessment requirement-involve project
development oversight and do not affect the evaluation and rating process;
therefore, in their view, the changes do not need to be included in
program regulations. However, by making changes outside of the regulatory
process, FTA has missed an opportunity to obtain formal public input on
the proposed changes, which would increase the transparency of the
agency's decision-making process and ensure that the views of project
sponsors and other interested parties are considered. There are a range of
options available to FTA for obtaining industry input, from the more
formal rulemaking process to less formal ways of soliciting comment. In
those instances where FTA determines that a formal rulemaking process is
unnecessary, it could provide project sponsors an informal opportunity to
review and comment on any substantive changes proposed for the New Starts
program, as FTA recently did when it solicited public comment on proposed
technical changes to the rating process.
Of the nine changes that were not published in the Federal Register and
did not undergo rulemaking, six were made without providing other avenues
for public review and comment prior to their implementation. For example,
FTA did not seek industry input before implementing the risk assessment
requirement, project justification criteria weights, and the "make the
case" changes to the document. The limited opportunities to review and
comment on proposed changes have resulted in implementation problems,
according to some project sponsors. In addition, FTA did not provide
project sponsors the opportunity to comment on the administration's
recently announced cost-effectiveness funding recommendation practice.
Many project sponsors expressed concern about the way FTA has developed
and implemented changes to the New Starts process. For example, 14 of the
26 project sponsors with whom we spoke said that they have generally not
been given an opportunity to offer input into prospective changes before
they are implemented. Many project sponsors noted that the March "Dear
Colleague" letter was a step in the right direction in that it was one of
the first times since the last rulemaking that FTA had sought
2765 Fed. Reg. 76864 (Dec. 7, 2000). The Federal Transit Act of 1998,
within TEA-21, required FTA to publish regulations on the manner in which
proposed projects will be evaluated and rated.
Page 27 GAO-05-674 Public Transportation
their input on proposed changes. However, five project sponsors commented
that FTA did not give them sufficient time to review and comment on the
proposed changes. In addition, FTA did not publish these changes in the
Federal Register and provide at least 30 days' notice, the minimum time
typically required when changes are subject to the Administrative
Procedure Act. Congressional committees have also expressed concerns about
the transparency by which FTA makes changes to the evaluation and rating
process, as shown by the bills to reauthorize surface transportation
programs-introduced in early 2005-that offer a more formalized approach
for providing notice to and soliciting comment from project sponsors.28 In
particular, the House reauthorization bill would require that FTA provide
notice and an opportunity for comment at least 60 days before issuing any
"nonregulatory substantive changes."29 The Senate proposal states that FTA
should issue periodic descriptions of the rating criteria and allow for
public comment.30 (See app. IV for information about other changes
proposed by the House and Senate.)
Project sponsors offered a number of suggestions for improving FTA's
communication about changes to the New Starts process. For example, nine
project sponsors told us that FTA should provide them with an opportunity
to offer input into a change before it is made. Thirteen project sponsors
mentioned that FTA should provide them with more lead time when the agency
requires them to implement a change. For example, a number of project
sponsors expressed concern that the recent technical changes to the
cost-effectiveness calculation announced in April 2005 were made too close
to when New Starts information is due-June for travel forecasts and "make
the case" documents and August for the remaining application materials.31
In addition, eight project sponsors expressed that FTA should develop a
standard schedule for when it announces and implements changes to the New
Starts process. For example, one project sponsor suggested that FTA
announce all changes at its New Starts roundtable a year before the
changes are to be implemented.
28At the time this report went to print, the House and Senate were in
conference and no conference report was available. 29H.R. 3, Sec. 3031,
109th Cong. (2005).
30151 Cong. Rec. S5668 (daily ed. May 20, 2005). 31FTA cited the delay in
the implementation of the TSUB measure as an example of FTA providing
additional time for project sponsors to comply with a change.
New Starts Measures Have Evolved Over Time, but Not All Measures Count Toward
a Project's Rating
Our previous body of work on organizational transformation32 indicates
that communication with stakeholders should be a top priority for any
agency, is most effective when done early and often in any process, and is
central to forming the partnerships that are needed to develop and
implement an organization's strategies. An effective communication
strategy should facilitate an honest two-way exchange with, and allow for
feedback from, stakeholders. This communication is central to forming the
effective internal and external partnerships that are vital to a program's
success. For the past 2 years, FTA officials have announced at the
roundtables that they are planning to develop and implement a strategy for
improving communication among FTA offices and between FTA and project
sponsors. FTA officials have told us that the strategy could include
providing a single FTA point of contact for project sponsors and improving
the distribution of policy, guidance, and procedures, perhaps using tools
such as a listserve or webinar, as it did when it sought comments from
project sponsors on the recent cost-effectiveness changes.33 FTA has
implemented new communications tools, such as the webinar, but has not
developed a comprehensive communications strategy.
Many of the measures FTA uses to evaluate and rate New Starts projects
predate TEA-21 and have evolved over time. In developing these measures,
FTA has historically sought industry input by way of formal rulemaking as
well as outreach sessions, workshops, and reports. Although both TEA-21
and FTA's current New Starts program regulations emphasize the importance
of using a multiple-measure approach for evaluating projects, FTA assigns
a 50 percent weight to both the cost-effectiveness and the land use
criteria when developing the project justification summary rating. The
other three project justification criteria are not weighted, although the
mobility improvements criterion is used as a "tie-breaker" when the
average of the cost-effectiveness and land use ratings falls equally
between
32See GAO, Highlights of a GAO Forum: Mergers and Transformation: Lessons
Learned for a Department of Homeland Security and Other Federal Agencies,
GAO-03-293SP (Washington, D.C.: Nov. 14, 2002).
33A webinar is a seminar or workshop that is conducted electronically over
the World Wide Web. A listserve is an electronic mailing list that allows
subscribers to send and receive information on a particular topic.
Page 29 GAO-05-674 Public Transportation
Congress Directed FTA to Use Multiple Criteria in Evaluating and Rating
Projects
two ratings (e.g., between "medium" and "medium-high").34 According to FTA
officials, FTA does not use these criteria because the underlying measures
have weaknesses that diminish their use in distinguishing among projects.
Project sponsors we interviewed offered suggestions for improving all of
the project justification measures.
Through the Intermodal Surface Transportation Efficiency Act of 1991
(ISTEA) and TEA-21, Congress has directed FTA to use multiple criteria in
evaluating and rating New Starts projects. In particular, TEA-21
identifies a series of financial and project justification criteria that
reflect a broad range of benefits and effects of the proposed projects.
The financial criteria include the share of non-New Starts funding and the
capital and operating finance plans. The project justification criteria
identified by TEA-21 include mobility improvements, cost-effectiveness,
operating efficiencies, and environmental benefits. TEA-21 also identifies
several additional statutory "considerations" to the evaluation process,
including land use issues. TEA21 directs FTA to "evaluate and rate the
project as `highly recommended,' `recommended,' or `not recommended,'
based on...the project justification criteria," among other things.35 FTA
has recognized and acknowledged the congressional intent for New Starts
projects to be evaluated using multiple measures. For instance, in a
January 2005 report to Congress, FTA states that it is "clear that
Congress intends FTA to evaluate projects based on more than
cost-effectiveness criteria" and that the "statutory framework is
consistent with the concept that a wide range of benefits should be
considered in evaluating projects."36
FTA also has endorsed the concept of using multiple criteria to evaluate
and rate New Starts projects and FTA's New Starts regulations set forth a
multiple measure evaluation process. The New Starts regulations state that
"FTA will combine the ratings for each of the financial rating factors and
project justification criteria into overall `finance' and `justification'
ratings.... These ratings will then be combined into the single, overall
34Specifically, when mobility improvements are rated "low," the summary
rating will "round down" to the lower of the two ratings; for all other
mobility improvement ratings, the rating is "rounded up" to establish the
summary project justification rating.
3549U.S.C. 5309 (e)(6).
36U.S. Department of Transportation, Federal Transit Administration,
"Report to Congress on Evaluating New Starts Projects," January 3, 2005,
p. 6.
Page 30 GAO-05-674 Public Transportation
project ratings."37 Further, in response to a comment on the
cost-effectiveness measure in the proposed New Starts regulations, FTA
stated the following:
"It is important to note that the measure for cost-effectiveness is not
intended to be a single, stand-alone indicator of the merits of a proposed
new starts project. It is but one part of the multiple measure method that
FTA uses to evaluate project justification under the statutory criteria.
While cost-effectiveness is an important consideration, so are mobility
improvements, environmental benefits, and the other factors described both
in TEA-21 and elsewhere in this rule."38
FTA has repeated the importance of using a multiple measure approach in
many reports and other documents since then, including its annual reports
to Congress. The New Starts regulations identify the measures FTA will use
to evaluate projects from the project justification criteria outlined in
TEA-
21.39 Figure 5 shows the project justification criteria identified by
TEA-21 and the associated measures identified by FTA in the New Starts
regulations. (See app. V for more detail on these measures.)
3765 Fed. Reg. 76875 (Dec. 7, 2000). 3865 Fed. Reg. 76873 (Dec. 7, 2000).
39For the financial rating, the criteria also serve as the measures.
Page 31 GAO-05-674 Public Transportation
Figure 5: Criteria and Measures for Evaluating Projects, as Outlined in TEA-21
and FTA's New Starts Program Regulations
Source: FTA.
aOther factors can include environmental justice and equity issues,
economic development initiatives, innovative financing, etc.
bTSUB = Transportation System User Benefits.
FTA Used Industry Input to The measures FTA uses to evaluate and rate New
Starts projects have evolved over time, beginning years before TEA-21,
through an iterative
Develop Measures for All
Criteria Identified in TEA-21 process involving Congress and the transit
industry. FTA (then known as UMTA) introduced the first system for rating
New Starts projects in 1984. At that time, projects were rated on
cost-effectiveness (cost per new rider) and local financial commitment.
Through ISTEA in 1991, Congress added mobility improvements, environmental
benefits, and operating efficiencies to the list of criteria FTA should
use to evaluate projects and also added land use policies as an additional
factor for consideration.
FTA circulated a policy paper in 1994, asking for public comment on its
proposed measures and procedures for assessing projects to address the
requirements laid out in ISTEA. The agency received 31 responses from
transit operators, metropolitan planning organizations, state departments
of transportation, and other interested parties. FTA used these responses
in finalizing its criteria and measures in a notice published in 1996 in
the Federal Register. In the 1994 policy paper, FTA also solicited
comments on the appropriateness of using a multiple-measure method for
evaluating projects. Respondents generally agreed that this was
appropriate, although they were split on how (or whether) the various
criteria should be weighted. FTA formally adopted the multiple-measure
approach in 1996.
TEA-21 required FTA to issue regulations for the evaluation and rating
process, and FTA used that rulemaking opportunity to revise several of the
project justification measures. Before and during the rulemaking process,
FTA conducted outreach sessions around the country, soliciting comments on
its processes and procedures for managing the New Starts program. FTA
issued a Notice of Proposed Rulemaking in 1999 and received comments from
41 individuals and organizations. In response to the comments, FTA made
several changes to the project justification measures. For example, many
of these commenters objected to the "cost per new rider" measure for
cost-effectiveness, saying that the focus on new riders ignores benefits
provided to other riders, which may bias the measure against cities with
"mature" transit systems. In response, FTA replaced it with the
incremental cost per hour of TSUB measure (to capture benefits to both new
and existing riders). The agency also added a mobility improvements
measure for employment near stations to complement the existing low-income
households near stations measure in response to industry comments that a
system that is located near low-income households is of little use to
residents unless it can also provide access to employment and other
activity centers.
Since issuing the regulations, FTA has continued efforts to fine-tune and
improve some of the project justification measures. For example, FTA
convened a panel of experts from the Urban Land Institute to discuss the
land use measures and is seeking industry input on possible changes. FTA's
Strategic Business Plan for Fiscal Year 2005 includes deliverables to (1)
develop a land use measure that is more quantifiable, offers a better
basis for distinguishing among projects, and provides grantees information
with
FTA Does Not Use All Project Justification Criteria in Determining Ratings
Because of Perceived Weaknesses in the Measures
which to improve their projects and (2) develop a methodology for
measuring the congestion relief benefits of New Start projects.40 Both are
scheduled to be completed by summer 2005. FTA officials told us that the
fiscal year 2005 program plan included research funds for these efforts.
In addition, as discussed previously, FTA recently instituted technical
modifications to the cost-effectiveness measure.
Despite the requirement to use multiple measures, FTA does not use three
of the five project justification criteria in calculating a project's
rating. Specifically, FTA currently assigns a weight of 50 percent each to
the cost-effectiveness and land use criteria; the other three project
justification criteria are not assigned weights (see fig. 6). In its
annual report for fiscal year 2006, FTA stated that it assigns individual
ratings for the other three project justification criteria and reports
them in the annual report, but these ratings "are not considered in the
determination of an overall project justification rating."41 FTA does,
however, use all three financial commitment criteria in developing a
project's rating.
40The current mobility improvements and cost-effectiveness measure (i.e.,
TSUB) does not include highway congestion relief benefits.
41The rating for the mobility improvements may be used as a tie-breaker
between cost-effectiveness and land use ratings-that is, a low mobility
improvements rating will round down the summary rating and a mobility
improvements rating above a low will cause the summary rating to be
rounded up. However, FTA officials told us that ties have been rare in the
last several years. The "other" factors will also be considered if there
is a compelling reason to do so; however, an FTA official told us that to
the best of his knowledge these other factors have never changed a
project's rating.
Figure 6: Weights Used to Determine Project Justification and Financial
Summary Ratings for New Starts Projects
Source: GAO presentation of FTA data.
Note: Numbers do not add to 100 due to rounding.
FTA officials told us that they do not use the mobility improvements,
environmental benefits, and operating efficiencies criteria in determining
the project justification summary rating because the measures do not, as
currently structured, provide meaningful distinctions among competing New
Starts projects. Many project sponsors we interviewed had similar views,
noting that individual projects are too small to have much impact on the
whole region or the whole transit system. For example, one of the
environmental benefits measures requires project sponsors to measure the
project's impact on the annual number of tons of emissions forecast for
the region for various pollutants. FTA officials also said that they do
not assign weight to the mobility improvements measures in determining the
project justification rating because they believe that the employment and
low-income household measures do not meaningfully distinguish among
projects, and the user benefits are already captured in the
cost-effectiveness measure. Therefore, to count the user benefits as part
of the mobility improvements would result in a double-counting of the
benefits.42 In contrast, FTA officials told us that the cost-effectiveness
and land use measures help to make meaningful distinctions among projects.
For example, according to FTA, existing transit supportive plans and
policies demonstrate an area's commitment to transit projects and are a
strong indicator of a project's future success.
Because FTA officials believe that these project justification measures do
not provide meaningful distinctions among projects and are not factored
into a project's rating, FTA does not stringently evaluate the projects on
the associated criteria. For instance, according to FTA's New Starts
reporting instructions, every project must submit data on operating
efficiencies and automatically receives a score of medium on that
criterion. Similarly, a project that is in a nonattainment area for a
pollutant and that demonstrates a projected decrease in that pollutant
gets a high environmental benefits rating, while a project in an
attainment area that demonstrates a decrease gets a medium environmental
benefits rating.43
The New Starts regulations do not specify the weights that should be
assigned to each project justification criterion. FTA sought industry
input on whether there should be a weighting system in the last rulemaking
process; however, according to FTA officials, the agency did not receive
input that was useful to inform its policy. Consequently, the regulations
are silent on the weights that should be assigned to each criterion and do
not prohibit FTA from treating various criteria as more important than
other criteria. FTA instituted its current weighting system with the
fiscal year 2004 evaluation cycle after determining that the operating
efficiencies, environmental benefits, and mobility improvements measures
do not effectively distinguish among projects and that there is overlap
among the
42For more information on double-counting the benefits of transportation
investments, see GAO, Highway and Transit Investments: Options for
Improving Information on Projects' Benefits and Costs and Increasing
Accountability for Results, GAO-05-172 (Washington, D.C.: Jan. 24, 2005).
43The U.S. Environmental Protection Agency designates each region as in
attainment, nonattainment, or maintenance-reflecting current compliance
with the National Ambient Air Quality Standards under the Clean Air
Act-for transportation-related pollutants including ozone, carbon
monoxide, particulate matter, and nitrogen oxides. Geographic areas that
have levels of a pollutant above those allowed by the standard are called
nonattainment areas. Areas that did not meet the standard for a pollutant
in the past but have reached attainment and met certain procedural
requirements are known as maintenance areas.
mobility and cost-effectiveness measures. Because the regulations do not
set forth the weights that should be assigned to each criterion, FTA did
not amend the regulations when it instituted the current weighting system.
Although FTA does not use all of the project justification criteria
identified in TEA-21 to calculate project ratings, project sponsors must
submit information for all five project justification criteria and FTA
publishes this information for each project in the New Starts annual
report. FTA officials offered several reasons for requiring project
sponsors to continue to provide this information. First, TEA-21 requires
FTA to consider these criteria when evaluating projects. Even though not
all of the project justification criteria count toward the rating, FTA
officials told us that they review and consider all five criteria and,
therefore, are operating within the evaluation framework established in
TEA-21 and the New Starts regulations. Second, FTA officials said that
they believe that mobility improvements, environmental benefits, and
operating efficiencies are important and should be a part of the
evaluation process, even though FTA does not yet have measures for these
criteria that help make distinctions among projects for the purpose of
rating and funding decisions. No measures are specified in TEA-21, which
only describes the criteria FTA should use to evaluate and rate projects,
so FTA has the flexibility to revise the measures as needed. FTA officials
stated that they continue to examine and pursue options to improve the
measures and that FTA has committed approximately $500,000 of its fiscal
year 2005 research funds to continue its work to improve the New Starts
measures. The officials also said that they would wait to change these
measures until legislation governing the New Starts program is
reauthorized. They said that they will have to institute a formal
rulemaking process at that time and would use that opportunity to solicit
public comment on the New Starts evaluation and rating process. Third, FTA
officials told us that information on these three criteria is presented in
the annual report and may be useful to Congress and local decisionmakers.
A number of project sponsors we spoke to expressed frustration that they
must prepare and submit information for all the measures for the five
project justification criteria even though some of this information does
not contribute to the projects' ratings.
Project Sponsors and Others Identified Strengths and Weaknesses of the
Measures and Suggested Improvements
Project sponsors we interviewed, as well as FTA, the Department of
Transportation's Inspector General, and other industry experts have
identified various strengths and weaknesses of the project justification
measures used to evaluate the New Starts projects. For example, all of
these sources acknowledge that FTA's measure of cost-effectiveness does
not capture benefits that accrue to highway users as more people switch to
the improved transit system and highway congestion decreases. According to
the department's Inspector General, the omission of highway travel time
savings means that the benefits from proposed projects that convey
significant travel time savings for motorists are not recognized in the
selection process. FTA noted that current local models used to estimate
future travel demand for New Starts are incapable of estimating reliable
highway travel time savings as a result of the New Starts project.
According to the department's Inspector General, this limitation is due to
unreliable local data on highway speeds.44 FTA is working with the Federal
Highway Administration to study ways to remedy this problem. Table 3 shows
the strengths, weaknesses, and other concerns most commonly mentioned by
the New Starts project sponsors we interviewed.
44"The Rating and Evaluation of New Starts Transit Systems," Statement of
the Honorable Kenneth M. Mead, Inspector General, U.S. Department of
Transportation, before the Committee on Appropriations, Subcommittee on
Transportation, Treasury and Independent Agencies, U.S. House of
Representatives, April 28, 2004.
Page 38 GAO-05-674 Public Transportation
Table 3: Strengths, Weaknesses, and Other Concerns about New Starts Measures, as
Identified by Project Sponsors
Criterion Measure Strengths Weaknesses Other concernsa
o Captures o Does not o Unclear what it means
Mobility Transportation benefits to new and include or
highway
improvements System User existing riders (3) benefits how it is used (6)
(4)
Benefits o Favors o Difficult to explain
(TSUB) longer (5)
projects
(3)
per project o
Difficult/time-consuming
to
passenger mile calculate (4)
o Does not
Employment o Measures the include
near potential transit projected
or
stations market (4) planned
employment
(3)
o Does not
capture
benefits to
the whole
system or
corridor
(3)
o Measures the o Does not
Low-income potential transit measure
"choice"
households market (7) riders
near (e.g.,
commuters);
stations o Measures urban does not
revitalization, reflect
purpose of
all
equity, or social transit
justice (3) systems (8)
o Does not
really
measure
mobility
(5)
o
Environmental Change in o Directly Individual
regional addresses pollution project has
a small
benefits pollutant reduction (4) impact on
emissions the region
(13)
o
Change in No consensus on Individual
regional strengths project has
a small
impact on
energy the region
(10)
consumption o This is
not within
the transit
agency's
control (3)
o
Environmental o Helps determine Individual o Designation should
where transit project has not
a small
Protection is most needed (3) or no matter-all areas should
Agency's impact on
the
region's
air quality designation be trying to improve air
(6)
designation o This is quality (3)
not within
the transit
agency's
control (6)
o
Operating System o Understandable, Individual
operating simple, project has
small
efficiencies cost per straightforward, impact on
passenger reasonable systemwide
mile (6) efficiency
(5)
o
Incomplete
by itself;
needs
context (3)
o
Incremental o Sensitive o Complicated to
Cost- cost Captures/quantifies to calculate
project variations
in
effectiveness per hour of benefits in one underlying and explain (7)
TSUB number for assumptions
(5)
comparison (9) o Does not o Thresholds/caps seem
include
highway
o Reasonable, benefits arbitrary (4)
understandable (5)
(3) o Does not
account for
other
o Improvement over local goals
old or benefits
(5)
measure (3) o Does not
include
project's
impact on
land use b
(3)
(Continued From Previous Page)
Criterion Measure Strengths Weaknesses Other
concernsa
Land use Existing o Reflects current or o Focuses on
land use future present, but
ridership (5) project is built
for the future
o Demonstrates (5)
project's
benefits/need (3) o Subjective (3)
Transit o Emphasizes link o Qualitative,
supportive between land difficult to
plans and use and transit (8) document/measure
policies (5)
Performance o Measures actual o Qualitative,
and public local subjective, vague
impact of support for transit (8)
policies c (3)
o Shows opportunity
for
expansion/development
(3)
Source: GAO.
Note: Numbers in parentheses indicate response frequency. This table only
includes responses that were mentioned by at least 3 of the 26 project
sponsors we interviewed.
aProject sponsors raised these other concerns during our discussions on
the measures' strengths and weaknesses.
bFor example, one project sponsor said that the measure does not account
for local land use policies and the resulting development potential in the
area, such as the impact of that city's 20-year zoning plans.
cThis measure includes performance of land use policies and potential
impact of transit project on regional land use.
In addition, in a recent report on the benefits and costs of
transportation improvements, we identified challenges in measuring the
benefits and costs of transit investments-some of which are relevant to
the measures used by FTA to evaluate New Starts projects.45 For example,
desirable changes in land use are indirect benefits of a transportation
investment, which are difficult to forecast and quantify. We also reported
that social benefits such as reductions in environmental costs-including
reduced emissions-were difficult to quantify and value. Additionally, we
reported that there is great variation in the models local transportation
planning agencies use to develop travel forecasts (which underlie many of
the New Starts measures), producing significant variation in forecast
quality and limiting the ability to assess quality against the general
state of practice. Some experts have also found that travel demand models
tend to predict unreasonably bad conditions in the absence of a proposed
highway or transit investment. In particular, travel forecasting does not
contend well with land-use changes or effects on nearby roads or other
transportation alternatives that result from transportation improvements
or growing congestion. Before conditions get as bad as they are
forecasted, people
45GAO-05-172.
make other changes, such as residence or employment changes, to avoid the
excessive travel costs.46
Our previous work has stated that agencies successful in measuring
performance had performance measures that, among other things, demonstrate
results and cover multiple priorities. Specifically, successful measures
(1) are aligned with agencywide goals and mission and are clearly
communicated throughout the organization; (2) are clearly stated, with a
name and definition that are consistent with the methodology used to
calculate the measures; (3) have a measurable target; (4) are objective;
(5) are reliable; (6) cover core program activities; (7) have limited
overlap with other measures; (8) provide balance in ensuring that various
priorities are covered; and (9) address governmentwide priorities, such as
quality and cost of service. For example, measures that are not objective
may result in performance assessments that are systematically over- or
understated, and a lack of balance could create skewed incentives when
measures overemphasize some goals.47 While these successful attributes
were developed specifically for performance measures, they also could be
useful in determining how to improve other types of measures, such as
those FTA uses to evaluate and rate New Starts projects.
The sponsors of the 26 projects we interviewed had many suggestions for
improving the project justification measures. These suggestions ranged
from adding measures to the mobility improvements, environmental benefits,
and land use criteria to changing the way in which operating efficiencies
are calculated. The most commonly cited suggestions are listed in table 4,
but we did not determine whether the suggestions were appropriate or
feasible. FTA officials told us that they received limited suggestions for
specific measures or methodologies for mobility improvements,
environmental benefits, and operating efficiencies during the rulemaking
process.
46FTA introduced the Summit software in the fiscal year 2004 rating cycle
to improve the accuracy of local travel models used to support New Starts
projects. According to FTA, Summit has provided a means to identify and
diagnose travel forecasting problems related to assumptions regarding fare
and service policies, regional transportation networks, land use, and
economic conditions. The software has also helped ensure that local
forecasts are utilizing comprehensive and up-to-date data on travel
behavior and local transportation systems.
47See GAO, Tax Administration: IRS Needs to Further Refine Its Tax Filing
Season Performance Measures, GAO-03-143 (Washington, D.C.: Nov. 22, 2002).
Table 4: Suggestions for Improving New Starts Measures, as Identified by Project
Sponsors
Area Suggestions
o Add measures, such as population density,
Mobility improvements residential density, number of high-income
households
served, projected or planned employment near
stations, retail and stadiums near stations,
transit-
oriented development, accessibility, or number of
new riders (10)
Environmental benefits o Add measures or replace existing measures with
movement toward a sustainable transit system,
effect of project on automobile congestion, number
of cars taken off the road, noise quality,
preservation of open space, or other measures
identified in the Environmental Impact Statement
(4)
Operating efficiencies o Measure operating cost per passenger or per
passenger hour instead of per passenger mile (3)
Cost-effectiveness o Add measures, such as cost per new rider, in
addition to the current measure of incremental cost
per hour of Transportation System User Benefits (5)
o Adjust thresholds for inflationa (3)
Land use o No consensus on suggestions
Source: GAO.
Notes: Numbers in parentheses indicate response frequency. This table only
includes responses that were mentioned by at least 3 of the 26 project
sponsors we interviewed.
aFTA announced it was adjusting the cost-effectiveness thresholds for
inflation on April 29, 2005, after we had completed our interviews.
TEA-21 and FTA's New Starts regulations clearly state that New Starts
Conclusions
projects should be evaluated and rated using multiple criteria. Further,
the statute and regulations identify the project justification criteria
that should be used in the evaluation process, including mobility
improvements, environmental benefits, operating efficiencies,
cost-effectiveness, and land use, as well as the financial commitment
criteria that should be used in the evaluation process. The regulations
also identify the measures that will be used to operationalize the
criteria. Although FTA uses all three financial criteria, in practice, FTA
uses only two of the project justification criteria- cost-effectiveness
and land use-to calculate a project's overall rating. FTA's reliance on
two of the five project justification criteria, coupled with the recent
cost-effectiveness practice for funding-which further emphasizes one
criterion-is drifting away from the multiple-measure evaluation and rating
process outlined in TEA-21 and the current New Starts regulations.
According to FTA officials, FTA does not assign weights to environmental
benefits, operating efficiencies, and mobility improvements because of
weaknesses in the measures and the overlap of some measures that could
result in double-counting of benefits. FTA has made notable progress in
improving the measures for cost-effectiveness and land use, including
seeking advice from experts and the transit industry and conducting pilot
tests since the enactment of TEA-21.
However, given that FTA has been statutorily directed to also use
environmental benefits, operating efficiencies, and mobility improvements
in evaluating and rating projects, it is imperative that FTA either pay
additional attention to improving these three criteria so that they can be
more explicitly used in the evaluation process or explicitly demonstrate
the linkages between these criteria and the measures used.
FTA has made a number of changes intended to enhance the rigor of the
program over the past 6 years, and some of these changes, such as the
Before and After study, risk assessment, and cost-effectiveness practice
could help FTA hold project sponsors accountable for results and maximize
the benefits of each dollar invested. However, FTA could improve the
transparency of changes made to the New Starts program by giving project
sponsors an opportunity to review and comment on any substantive changes
before they are implemented. The Freedom of Information and Administrative
Procedure Acts establish formal processes for notifying the public and
making changes to federal programs, including soliciting comments about
proposed changes. If FTA officials determine that a formal rulemaking
process is unnecessary, FTA could still provide project sponsors an
opportunity to review and comment on any substantive changes proposed for
the New Starts program, potentially avoiding some of the implementation
problems that have occurred in the past. In addition, the review and
comment period could help FTA improve the proposed changes as well as gain
industry buy-in and support for changes-elements that are critical for
success.
FTA could also strengthen its communication efforts by improving its Web
site so that project sponsors view it as a viable source for obtaining
information about changes to the New Starts program. The Web site could
provide a central forum for comprehensive, up-to-date information on the
New Starts program and could also be useful for publicizing FTA's
activities to improve the application, evaluation, rating, and oversight
processes. Much of this information is already available on FTA's Web
site. However, the information that remains is scattered in different
locations and many project sponsors told us that it was difficult to
locate needed information. Making the information easier to find could
help reduce confusion about the New Starts process.
To ensure that the New Starts regulations reflect FTA's current evaluation
Recommendations for
and rating process, and to ensure that FTA's New Starts evaluation process
and policies are objective, transparent, and follow the intent of federal
Page 43 GAO-05-674 Public Transportation
statute, we recommend that the Secretary of Transportation direct the
Administrator, FTA, to take the following four actions:
o ensure that the agency's regulations governing the New Starts
evaluation and rating process reflect FTA's current weighting
practices for the criteria when the regulations are revised;
o improve the measures for evaluating New Starts projects so that all
five project justification criteria can be used in determining a
project's overall rating, or provide a crosswalk in the agency's New
Starts regulations showing clear linkages between the criteria
outlined in statute and the criteria and measures used in the rating
process;
o publish future changes to the New Starts program in the Federal
Register and subject future changes to the New Starts program to the
rulemaking process or, at a minimum, a 30-day informal review and
comment period, as appropriate. As part of this process, the agency
should develop criteria for determining which changes should be
subject to the rulemaking process, as outlined in federal statute, or
to an informal review and comment period. At a minimum, these criteria
could include changes that impose new reporting requirements or new
analysis on project sponsors, changes to the principles used to
recommend projects for funding, and changes to the weights assigned to
the criteria used to evaluate and rate projects. The criteria should
be communicated to Congress and to project sponsors and others in the
transit community; and
o consolidate information and guidance related to the New Starts program
in one location on the agency's Web site and regularly review this
information to ensure it is up to date and easy to access.
We provided a draft of this report to the Department of Transportation for
Agency Comments and
review and comment. Officials from the Department and FTA indicated that
they generally agreed with the report's findings, conclusions, and
recommendations. FTA officials also provided technical clarifications,
which we incorporated as appropriate.
We are sending copies of this report to congressional committees with
responsibilities for transit issues; the Secretary of Transportation; the
Page 44 GAO-05-674 Public Transportation
Administrator, Federal Transit Administration; and the Director, Office of
Management and Budget. We also will make copies available to others upon
request. In addition, this report will be available at no charge on GAO's
Web site at http://www.gao.gov.
If you or your staff have any questions on matters discussed in this
report, please contact me at [email protected]. Contact points for our
Offices of Congressional Relations and Public Affairs may be found on the
last page of this report. GAO contacts and key contributors to this report
are listed in appendix VI.
Katherine Siggerud Director, Physical Infrastructure Appendix I
Scope and Methodology
To address our objectives, we reviewed the administration's fiscal year
2006 budget request, the Federal Transit Administration's (FTA) annual New
Starts report, FTA's New Starts regulations, FTA's 2004 and 2005 reporting
instructions for the New Starts program, federal statutes pertaining to
the new Starts program, and previous GAO reports. We also interviewed FTA
officials and representatives from the American Public Transportation
Association, New Starts Working Group, and the Association of Metropolitan
Planning Organizations. In addition, we attended FTA's New Starts
roundtables with project sponsors in New York and San Francisco in April
and May 2005, respectively.
We also conducted semistructured interviews with project sponsors from the
26 projects that were evaluated and rated in the fiscal year 2006
evaluation cycle (see table 5). These interviews were designed to gain
project sponsors' perspectives on a number of topics, including the manner
in which FTA communicates changes to the New Starts application,
evaluation, rating, and project development oversight processes; the
impact of the changes that FTA has made to the application, evaluation,
rating, and project development oversight processes since the fiscal year
2001 evaluation cycle-the first full evaluation and rating cycle after the
enactment of the Transportation Equity Act for the 21st Century (TEA-21);
and the measures FTA uses to evaluate projects. To verify the clarity,
length of time of administration, and understandability of the interview
questions, as well as to determine whether respondents had sufficient
knowledge and information to answer the questions, we pretested the
questions with three project sponsors. We made changes to the content and
format of the final set of interview questions as a result of these
pretests. After conducting the interviews with sponsors from all 26
projects, we used a content analysis to systematically determine project
sponsors' views on key interview questions and identify common themes in
project sponsors' responses. Two analysts reached consensus on the coding
of the responses, and a third reviewer was consulted in case of
disagreements, to ensure that the codes were reliable.
Appendix I Scope and Methodology
Table 5: Projects Contacted for Our Review
Location Project
Final design or proposed for full funding grant agreement
Charlotte South Corridor LRT
Las Vegas Resort Corridor Downtown Extension
New York City Long Island Rail Road East Side Access
Phoenix Central Phoenix/East Valley LRT
Pittsburgh North Shore LRT Connector
Raleigh-Durham Regional Rail System
Washington County Wilsonville to Beaverton Commuter Rail
Preliminary engineering
Boston Silver Line Phase III
Dallas Northwest/Southeast Light Rail MOS
Denver West Corridor LRT
Fort Collins Mason Transportation Corridor
Hartford New Britain-Hartford Busway
Los Angeles Mid-City/Exposition Corridor LRT
Miami North Corridor Metrorail Extension
Minneapolis-Big Lake Northstar Corridor Rail
New Orleans Desire Streetcar Line
New York City Second Ave. Subway MOS
Norfolk Norfolk LRT
Northern VA Dulles Corridor Metrorail Extension to Wiehle Ave.
Orange County Centerline LRT
Philadelphia Schuylkill Valley Metrorail
Portland South Corridor I-205/Portland Mall LRT
Salt Lake City Weber County to Salt Lake City Commuter Rail
San Diego Mid-Coast LRT Extension
San Francisco New Central Subway
Santa Clara County Silicon Valley Rapid Transit Corridor
Sources: GAO and FTA.
Note: LRT = Light Rail Transit; MOS = Minimum Operable Segment
To ensure the reliability of information presented in this report, we
interviewed FTA officials about FTA's policies and procedures for
compiling the New Starts annual reports, including FTA's data collection
and verification practices for New Starts information. Specifically, we
asked them whether their policies and procedures had changed
Appendix I Scope and Methodology
significantly since we reviewed them for our 2004 report on New Starts.1
FTA officials told us that there were no significant changes in their data
collection and verification policies and procedures for New Starts
information. Therefore, we concluded that the FTA information presented is
sufficiently reliable for the purposes of this report.
We conducted our work from November 2004 through May 2005 in accordance
with generally accepted government auditing standards, including standards
for data reliability.
1GAO, Mass Transit: FTA Needs to Better Define and Assess Impact of
Certain Policies on New Starts Program, GAO-04-748 (Washington, D.C.: June
25, 2004).
Page 48 GAO-05-674 Public Transportation
Appendix II
Projects with Existing FFGAs and Projects in Preliminary Engineering and Final
Design in the Fiscal Year 2006 Cycle
Dollars in millions
Total New Starts
funding
Fiscal year 2006 scheduled by FFGA or
State/ Overall project recommended requested by project
territory Location/project funding sponsors
rating/status
Projects with existing full funding
grant agreements (FFGAs)
CA Los Angeles-Metro Gold Line East Side FFGA $80.0 $490.7
Extension
CA San Diego-Mission Valley East Light FFGA 7.7 330.0
Rail
Transit (LRT) Extension
CA San Diego-Oceanside-Escondido Rail FFGA 12.2 152.1
Corridor
CA San Francisco-Bay Area Rapid Transit FFGA 81.9 750.0
Extension to San Francisco Airport
CO Denver-Southeast Corridor LRT FFGA 80.0 525.0
IL Chicago-Douglas Branch FFGA 45.2 320.1
Reconstruction
IL Chicago-North Central Corridor FFGA 20.6 135.3
Commuter
Rail
IL Chicago-Ravenswood Line Extension FFGA 40.0 245.5
IL Chicago-South West Corridor Commuter FFGA 7.3 103.0
Rail
IL Chicago-Union Pacific West Line FFGA 14.3 80.8
Extension
MD Baltimore-Central LRT FFGA 12.4 120.0
Double-Track
NJ Northern New FFGA 100.0 500.0
Jersey-Hudson-Bergen
Minimum Operable Segment (MOS)-2
OH Cleveland-Euclid Corridor FFGA 24.8 82.2
Transportation
Project
OR Portland-Interstate MAX LRT FFGA 18.1 257.5
Extension
PR San Juan-Tren Urbano FFGA 10.2 307.4
WA Seattle-Central Link Initial FFGA 80.0 500.0
Segment
Subtotal 634.6 4,899.6
Projects recommended for FFGAs
AZ Phoenix-Central Phoenix/ East Recommended 90.0 587.2
Valley LRT
NC Charlotte-South Corridor LRT Recommended 55.0 192.9
NY New York-Long Island Rail Road Highly recommended 390.0 2,632.0
East Side
Access
PA Pittsburgh-North Shore LRT Recommended 55.0 217.7
Connector
Subtotal 590.0 3,629.8
Appendix II Projects with Existing FFGAs and Projects in Preliminary
Engineering and Final Design in the Fiscal Year 2006 Cycle
(Continued From Previous Page)
Dollars in millions
Total New Starts
funding
Fiscal year 2006 scheduled by
FFGA or
State/ Overall project recommended requested by
project
territory rating/status funding sponsors
Location/project
Projects in final design
MO Kansas City-Southtown Bus Rapid Transit Exempt 12.3
(BRT)
NC Raleigh-Durham-Regional Rail System Not rated 416.1
NV Las Vegas-Resort Corridor Downtown Not recommended 159.7
Extension
OR Washington County-Wilsonville to Recommended 51.8
Beaverton Commuter Raila
TN Nashville-East Corridor Commuter Exempt 23.5
Rail
Subtotal 663.4
Projects in preliminary engineering
AK Wasilla-South Wasilla Track Realignment Exempt N/A
CA Los Angeles-Exposition Corridor LRT Not rated 276.0
CA Orange County-CenterLine LRT Not rated 482.8
CA San Diego-Mid-Coast LRT Extensiona Recommended 65.8
CA San Francisco-Central Subway Recommended 762.2
Projects in preliminary engineering
CA Santa Clara County-Silicon Valley Rapid Not recommended 973.0
Transit Corridor
CO Denver-West Corridor LRTa Recommended 249.0
CO Fort Collins-Mason Transportation Corridor Not recommended 33.0
CT Hartford-New Britain-Hartford Busway Not recommended 168.5
DE Wilmington-Wilmington to Newark Exempt 24.9
Commuter Rail Improvements
FL Miami-North Corridor Metrorail Extension Not rated 421.3
FL Tampa Bay-Tampa Bay Regional Railb Not recommended 727.7
LA New Orleans-Desire Streetcar Line Not recommended 68.7
MA Boston-Silver Line Phase III Recommended 468.3
MN Minneapolis-Big Lake-Northstar Corridor Not recommended 132.5
Rail
NY New York-Second Avenue Subway MOSa Highly recommended 1,300.0
OR Portland-South Corridor I-205/Portland Recommended 296.2
Mall
LRT
PA Harrisburg-CORRIDORone Rail MOS Exempt 24.9
PA Philadelphia-Schuylkill Valley MetroRail Not recommended 2,071.1
Appendix II Projects with Existing FFGAs and Projects in Preliminary
Engineering and Final Design in the Fiscal Year 2006 Cycle
(Continued From Previous Page)
Dollars in millions
Total New Starts funding
Fiscal year 2006 scheduled
by FFGA
or
State/ Overall recommended requested
project by
project
territory Location/project rating/status funding sponsors
RI Providence-South County Exempt 24.9
Commuter Rail
TX Dallas-Northwest/Southeast Recommended 700.0
Light Rail
MOSa
TX El Paso-Sun Metro Area Rapid Exempt N/A
Transit
Starter Line
UT Salt Lake City-Weber County to Recommended 466.8
Salt Lake
City Commuter Raila
VA Norfolk-Norfolk LRT Not rated 100.7
VA Northern Virginia-Dulles Recommended 760.5
Corridor Metrorail
Project - Extension to Wiehle Avenue
Subtotal 10,598.8
Total 1,224.6 19,791.6
Source: FTA. Note: Totals may not add due to rounding. aProjects proposed
for potential federal funding commitments outside of full funding grant
agreements. bThis project has since withdrawn from the New Starts
application process.
Appendix III
Status of Previous GAO Recommendations for Improving the New Starts Evaluation
Process
Report Recommendation Status
GAO/T-RCED-00-104 The Federal Transit FTA officials told us that
Administration (FTA) their recently implemented
should prioritize the cost-effectiveness practice
projects it rates as was partly in response to
highly recommended or this recommendation. FTA will
recommended and ready to generally not recommend
receive New Starts funds. funding for a project that
does not achieve at least a
"medium" cost-effectiveness
rating.
GAO-01-987 FTA should make In fiscal year 2003, FTA
GAO-03-701 commitment authority released $157 million in
allocated to projects for commitment authority reserved
which the federal funding for a Los Angeles Mid-City
commitments have been subway project that had been
withdrawn available for suspended for more than 3
all projects competing years. FTA disagreed with
for New Starts funding this recommendation, arguing
(i.e., "release" the that its preference policy is
funding). FTA should not a legally binding
amend its regulations requirement and therefore
governing the level of should not be reflected in
federal funding share for the New Starts regulations.
projects to reflect its However, the agency did
policy of favoring change the way it
projects that request characterized the policy in
less than 60 percent New its fiscal year 2006 New
Starts funding. Starts annual report. This
document states that FTA
"generally" will not
recommend funding for
projects that request more
than a 60 percent New Starts
share of funding.
GAO-03-701 FTA should issue additional guidance to transit agencies
describing FTA's expectations regarding the local travel forecasting
models and the specific type of data FTA requires to calculate the
Transportation System User Benefits measure.
FTA completed several actions and has other ongoing efforts to address the
recommendation. (1) FTA provided additional guidance in its reporting
instructions, clarifying the practices that must be followed in preparing
the Transportation System User Benefits measure. (2) FTA provided
additional guidance in July 2003, on how it would evaluate the measures in
the fiscal year 2005 rating process. (3) FTA held several special
workshops for transit agencies on Transportation System User Benefits,
travel forecasting, and development of transit alternatives. (4) FTA
initiated a proactive outreach to project sponsors to help them prepare
and evaluate the measure, identifying weaknesses in their proposed measure
and suggesting improvements. (5) FTA has a number of ongoing planned
improvements to its guidance, including developing a users guide to the
Summit software, developing case studies and exemplary practices, and
updating its guidance on travel forecasting for New Starts projects.
GAO-04-748 FTA should clearly explain the basis on which it decides which
projects will be recommended for funding outside of full funding grant
agreements, and what projects must do to qualify for such a
recommendation. These explanations should be included in FTA's annual New
Starts report and other published New Starts guidance.
In its fiscal year 2006 New Starts report, FTA described its criteria for
recommending projects for funding outside of full funding grant
agreements. These criteria include projects that were in or nearing final
design, received overall highly recommended or recommended ratings, and
had cost-effectiveness ratings above a "low."
Appendix III Status of Previous GAO Recommendations for Improving the New
Starts Evaluation Process
(Continued From Previous Page)
Report Recommendation Status
GAO-04-748 FTA should examine the impact of its preference FTA has not
implemented this recommendation. policy on projects currently in the
evaluation process, as well as projects in the early planning stages, and
examine whether its policy results in maximizing New Starts funds and
local participation.
Source: GAO.
Appendix IV
Key New Starts Provisions Contained in House and Senate Reauthorization Bills
Provision Senate proposal House proposal
Streamline the o Allows the Secretary of o Establishes a "Small
New Starts Transportation discretion to Starts" program for projects
evaluation develop a streamlined requesting between $25 million
process for evaluation process for and $75 million in New Starts
projects under projects requesting less funding, with a total project
$75 million than $75 million in New cost of less than $200
Starts funds. o Eliminates million. These projects would
the "exempt" classification be evaluated through a
for projects requesting less streamlined rating process. o
than $25 million in New Maintains the "exempt"
Starts funding and allows classification for projects
FTA to analyze and rate all requesting less than $25
projects through a million in New Starts funding.
streamlined process.
Expand the o Allows nonfixed-guideway transit projects (e.g., bus o
Expands New Starts funding eligibility to include definition of eligible
rapid transit operating in nonexclusive lanes) nonfixed-guideway projects
with a majority of fixed-projects requesting less than $75 million to be
eligible for New guideway components seeking between $25 million
Starts funding. and $75 million as part of its "Small Starts" initiative.
Change the rating o Revises the current rating system (that uses "highly
o Maintains the current rating system.
categories recommended," "recommended," and "not recommended" ratings) to
five levels of ratings: "high," "medium-high," "medium," "medium-low," and
"low."
Maintain a maximum o Maintains the maximum New Starts share at 80 o
Maintains the maximum New Starts share at 80 federal New Starts percent
for individual projects (in contrast to the percent for individual
projects. share at 80 percent administration's reauthorization proposal,
which would o Specifically prohibits FTA from requiring a nonfederal
lower the maximum New Starts share to 50 percent). share that is more than
20 percent of the project's cost.
Formalize o Requires FTA to issue periodic descriptions of the o
Requires FTA to provide notice and an opportunity for communication review
and evaluation process and criteria and allow comment at least 60 days
before issuing any about changes for public comment. nonregulatory
substantive changes.
Sources: H.R. 3, 109th Cong. (2005), and 151 Cong. Rec. S5667-S5669 (daily ed.
May 20, 2005).
Appendix V
FTA's Project Justification Measures for Evaluating and Rating New Starts
Projects
Criterion Measure Definition
Mobility improvements Transportation System User Annual Transportation
Benefits per project User Benefits (user
passenger mile (normalized expenditure savings
travel time savings) between New Starts
baseline and build
alternatives) per
forecasted project
passenger mile, divided
by total Transportation
System User Benefits per
passenger mile
Low-income households Estimated number of
served low-income households
(i.e., households below
the poverty level)
located within 1/2 mile
of boarding points
(transit stations) on the
proposed New Starts
project
Employment near stations Number of jobs within 1/2
mile of the New Starts
project's proposed
transit stations
Environmental Change in criteria Annual number of tons of
benefits pollutant/precursor emissions forecast for
emissions the region-comparing
conditions under the New
Starts investment to the
New Starts baseline
alternative-for carbon
monoxide, particulate
matter, nitrogen oxides,
volatile organic
compounds, and carbon
dioxide
Change in regional energy Net impact on energy
consumption savings as a result of
changes in automobile and
commercial travel in the
region, offset in part by
the energy requirements
for operation of the
proposed transit
investment, measured in
British Thermal Units
Current Environmental Protection Agency regional air quality designation
U.S. Environmental Protection Agency's current air quality designation for
the region, reflecting current compliance with the National Ambient Air
Quality Standards, reported as attainment, nonattainment, or maintenance
for transportation-related pollutants including ozone, carbon monoxide,
particulate matter, and nitrogen oxides
Operating Operating cost per Change in systemwide
efficiencies passenger mile operating cost per passenger
mile in the forecast year,
comparing the New Starts
build alternative to the
baseline alternative
Cost-effectiveness Incremental cost of Annualized capital and
Transportation System operating costs divided by
User Benefits Transportation System User
Benefits (annual user
expenditure savings) of the
New Starts project as
compared to the baseline
Land use Existing land use Includes corridor and
station area development,
character (i.e.,
residential, commercial,
mixed-use),
pedestrian facilities, and
parking supply
Transit supportive plans Includes growth management,
and policies transit supportive
corridor policies,
supportive zoning
regulations near
transit stations, and tools
to implement land use
policies
Performance and impact of Includes performance of land
policies use policies and potential
impact of transit project on
regional land use
Source: FTA.
Appendix VI
GAO Contacts and Staff Acknowledgments
Katherine Siggerud, (202) 512-2834, [email protected]
GAO Contacts
Nikki Clowers, (202) 512-4010, [email protected]
In addition to the individuals named above, other key contributors to this
Staff
report were Bert Japikse, Jessica Lucas-Judy, Sara Ann Moessbauer, and
Acknowledgments Gary Stofko.
Related GAO Products
Highway and Transit Investments: Options for Improving Information on
Projects' Benefits and Costs and Increasing Accountability for Results.
GAO-05-172. Washington, D.C.: January 24, 2005.
Mass Transit: FTA Needs to Better Define and Assess Impact of Certain
Policies on New Starts Program. GAO-04-748 . Washington, D.C.: June 25,
2004.
Mass Transit: FTA Needs to Provide Clear Information and Additional
Guidance on the New Starts Ratings Process. GAO-03-701. Washington, D.C.:
June 23, 2003.
Mass Transit: Status of New Starts Program and Potential for Bus Rapid
Transit Projects. GAO-02-840T. Washington, D.C.: June 20, 2002.
Mass Transit: FTA's New Starts Commitments for Fiscal Year 2003. GAO-
02-603 . Washington, D.C.: April 30, 2002.
Mass Transit: FTA Could Relieve New Starts Program Funding Constraints.
GAO-01-987. Washington, D.C.: August 15, 2001.
Mass Transit: Implementation of FTA's New Starts Evaluation Process and FY
2001 Funding Proposals. GAO/RCED-00-149. Washington, D.C.: April 28, 2000.
Mass Transit: Challenges in Evaluating, Overseeing, and Funding Major
Transit Projects. GAO/T-RCED-00-104. Washington, D.C.: March 8, 2000.
Mass Transit: "Mobility Improvements" Is One of Many Factors Used to
Evaluate Mass Transit Projects. GAO/RCED-00-6R. Washington, D.C.: October
15, 1999.
Mass Transit: Status of New Starts Transit Projects With Full Funding
Grant Agreements. GAO/RCED-99-240. Washington, D.C.: August 19, 1999.
Mass Transit: FTA's Progress in Developing and Implementing a New Starts
Evaluation Process. GAO/RCED-99-113. Washington, D.C.: April 26, 1999.
GAO's Mission
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site ( www.gao.gov ). Each weekday, GAO posts GAO
Reports and newly released reports, testimony, and correspondence on its
Web site. To
have GAO e-mail you a list of newly posted products every afternoon, go to
www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
Contact:
To Report Fraud, Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: [email protected]
Federal Programs Automated answering system: (800) 424-5454 or (202)
512-7470
Gloria Jarmon, Managing Director, [email protected] (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125 Relations
Washington, D.C. 20548
Paul Anderson, Managing Director, [email protected] (202) 512-4800
Public Affairs
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
*** End of document. ***