Highlights of an Expert Panel: The Benefits and Costs of Highway
and Transit Investments (06-MAY-05, GAO-05-423SP).
The nation's economy and its citizens' quality of life depend on
our transportation system. While all government levels have made
significant investments in transportation, projections of future
passenger and freight travel indicate that considerable
investment will be needed to maintain the system. However, this
comes amid growing concern about the size of the federal budget
deficit and increasing demands on state and local government
revenue. As a result, careful decisions will need to be made to
ensure that transportation investments maximize the benefits of
each dollar invested. The House Appropriations Committee report
accompanying the fiscal year 2004 Departments of Transportation
and Treasury and Independent Agencies Appropriations Bill,
required GAO to review the benefits and costs of various
transportation modes. (See GAO-05-172.) As part of this study,
GAO convened an expert panel that included some of the leading
transportation economists and practitioners from throughout the
nation. The panel discussed the benefits and costs of highway and
transit investments.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-05-423SP
ACCNO: A23797
TITLE: Highlights of an Expert Panel: The Benefits and Costs of
Highway and Transit Investments
DATE: 05/06/2005
SUBJECT: Cost analysis
Cost effectiveness analysis
Decision making
Economic analysis
Federal aid for highways
Highway planning
Highway research
Mass transit funding
Projections
Public roads or highways
Data collection
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-05-423SP
* Highlights of An Expert Panel: The Benefits and Costs of Highway and
Transit Investments
* Contents
* Abbreviations
* Message from the Comptroller General of the United States
* Introduction: Expert Panel on the Bene.ts and Costs of Transit and
Highway Investments
* Highlights of the Expert Panel Discussion
* Appendix I: Scope and Methodology
* Appendix II: Overview of Expert Panel and Opening Remarks
* Appendix III: How Should We Think About Transportation Bene. ts and
Costs?
* Appendix IV: How Are Bene. ts and Costs of Transit and Highway
Investments Best Measured?
* Appendix V: How Could Bene. t-Cost Analysis Be Improved?
* Appendix VI: How Can Bene. t-Cost Analysis Be Most Useful In
Investment Decisions?
* Appendix VII: Panelists' Closing Remarks
* Appendix VIII: Selected Biblography and Related GAO Products
* Appendix IX: GAO Contacts and Acknowledgements
* Endnotes
HIGHLIGHTS OF AN EXPERT PANEL
The Benefits and Costs of Highway and Transit Investments
What Participants Said
GAO asked expert panel participants to discuss how to conceptualize,
measure, improve, and use information about the benefits and costs of
highway and transit investments. The expert panel was not designed to
reach a consensus on these issues, but several themes emerged from the
panel's discussion, including the following:
o Benefit-cost analysis can be a useful tool to inform transportation
investment decisions.
o Requiring benefit-cost analysis can be useful if it is fully
integrated into the decision making process and not seen as a
compliance checklist.
o Transportation investments seldom are compared across modes.
o Better analytic tools are needed to evaluate land use and
distributional impacts of investments.
o Quality of state and local transportation data needs to be improved so
that travel models can accurately predict patterns, trends, and needs.
United States Government Accountability Office
Contents
Abbreviations 5
Message from the Comptroller General of the United States 7
Introduction: Expert Panel on the Benefits and Costs of Transit and
Highway Investments 9
Highlights of the Expert Panel Discussion 11
Appendix I: Scope and Methodology 14
Appendix II: Overview of Expert Panel and Opening Remarks 19
Appendix III: How Should We Think About Transportation Benefits and Costs?
24
Opening Comments by Dr. Lewis and Dr. Wachs 24
Panel Discussion 28
Summary of Panel Responses to Audience Questions 36
Appendix IV: How Are Benefits and Costs of Transit and Highway Investments
Best Measured? 38
Opening Comments by Dr. Small and Dr. Pickrell 38
Panel Discussion 42
Summary of Panel Responses to Audience Questions 58
Appendix V: How Could Benefit-Cost Analysis Be Improved? 59
Opening Comments by Dr. Forkenbrock and Dr. Kirby 59
Panel Discussion 63
Summary of Panel Responses to Audience Questions 72
Appendix VI: How Can Benefit-Cost Analysis Be Most Useful In Investment
Decisions? 73
Opening Comments by Dr. Meyer and Dr. Gomez-Ibanez 73
Panel Discussion 78
Summary of Panel Responses to Audience Questions 83
Appendix VII: Panelists' Closing Remarks 85
Appendix VIII: Selected Biblography and Related GAO Products 88
Appendix IX: GAO Contacts and Acknowledgements 90
GAO Contacts 90
Acknowledgments 90
Endnotes 91
Abbreviations
AASHTO American Association of State Highway and Transportation Officials
CAAA Clean Air Act Amendments
CMAQ Congestion Mitigation and Air Quality
DOT Department of Transportation
EIS Environmental Impact Statement
EPA Environmental Protection Agency
FHWA Federal Highway Administration
FTA Federal Transit Administration
GDP Gross Domestic Product
GPS Global Positioning System
HOT High Occupancy Toll
HOV High Occupancy Vehicle
ISTEA Intermodal Surface Transportation Efficiency Act
MPO Metropolitan Planning Organization
NAS National Academy of Sciences
NCHRP National Cooperative Highway Research Program
NEPA National Environmental Policy Act
OMB Office of Management and Budget
ROI Return on Investment
TANF Temporary Assistance for Needy Families
TCRP Transit Cooperative Research Program
TEA-21 Transportation Equity Act for the 21st Century
TRB Transportation Research Board
Message from the Comptroller General of the United States
In speeches and presentations over the past several years, I have called
attention to our large and growing long-term fiscal challenge and the
risks it poses to our nation's future. Simply put, our nation's fiscal
policy is on an unsustainable course. As long-term budget simulations by
GAO and others show, we face a large and growing structural deficit over
the long term due primarily to known demographic trends and rising health
costs. These trends are compounded by the presence of near-term deficits
arising from new discretionary and mandatory spending as well as lower
revenues as a share of the economy.
Continuing on this unsustainable fiscal path will gradually erode and may
suddenly damage our economy, standard of living, and ultimately our
national security. Given the size of our projected deficit, we will not be
able to grow our way out of this problem-tough choices will be required.
We need nothing less than a fundamental reexamination of all major
existing spending and tax policies and priorities. While prompted by
fiscal necessity, such a fundamental review of major program and policy
areas also serves the vital function of updating the federal government's
programs and priorities to meet current and future challenges. Many
current federal programs and policies were designed decades ago to respond
to trends and challenges that existed at the time of their creation. The
transportation sector is one of many areas where emerging challenges
necessitate difficult decisions about investments and priorities.
I want to thank the distinguished experts who participated in our panel on
the benefits and costs of highway and transit investments for their
willingness to share their knowledge and time to examine issues with
immediate importance and serious concern. Their insights about maximizing
the benefits of federal dollars invested in transportation will be of
value to Congress and the transportation community. I look forward to
working with the panelists and others on this and other issues of mutual
interest and concern in the future.
David M. Walker Comptroller General of the United States
Introduction: Expert Panel on the Benefits and Costs of Transit and Highway
Investments
he nation's economic vitality and its citizens' quality of life depend
substantially on the soundness, security, and availability of its
transportation system. The transportation system provides people with
access to goods, services, recreation, and jobs; provides businesses with
access to materials, markets, and people; and promotes the movement of
personnel and material to meet national defense needs. Given the
importance of the transportation system, all levels of government have
made significant investments in the system. However, future decisions
about investments in the transportation system are set to collide with new
realities and emerging trends. In particular:
o Securing funding for transportation investments is becoming
increasingly difficult. Federal transportation grant
programs-including the nation's highways and transit programs-are
funded by the Highway Trust Fund. Revenues to the Highway Trust Fund
are drawn from fuel taxes and user fees. The purchasing power of these
revenues is declining, and future fuel tax revenues could be further
eroded by the increasing fuel efficiency of vehicles. Many experts
question whether the current financing scheme for transportation is
ultimately sustainable. As a result, decision makers are increasingly
looking more to the general fund to finance transportation programs,
and state and local governments are increasingly relying on property
and sales taxes to fund transportation improvements. Attempts to
secure funding for transportation from these other sources come amid
growing concerns about the size of federal budget deficits and future
Social Security and Medicare commitments that will consume a greater
share of the nation's resources. Moreover, transportation faces
increasing competition from education, Medicaid, and other public uses
for state and local government revenues.1
o The gap between travel needs and the transportation system's condition
and capacity is growing. Increasing passenger and freight travel has
generated substantial congestion throughout the national
transportation system. Travel projections indicate that considerable
investment will be needed to prevent congestion from overwhelming the
system, while maintaining system safety and condition. The Department
of Transportation (DOT) estimated that the nation's highway and
transit systems will require $90.7 billion in annual capital
investment through 2020 to maintain their current level of conditions
and performance, while up to $127.5 billion in annual investment would
be required to improve conditions and performance.2
o The transportation investment decision-making process is increasingly
complex. Transportation investment decisions are inextricably linked
with land use, economic, environmental, and energy policy concerns,
among other things. Therefore, when making investment decisions,
decision makers must consider a number of factors as well as the
diverse, and sometimes conflicting, interests of numerous
stakeholders. For example, the Intermodal Surface Transportation
Efficiency Act of 1991 (ISTEA)3 and the Transportation Equity Act for
the 21st Century (TEA-21)4 require that a number of factors-including
safety,
environmental impacts, system connectivity, and accessibility-be considered in
investment decisions and
that state and local transportation agencies involve numerous stakeholders in
the decision-making process.
ISTEA and TEA-21 also gave local and state transportation agencies greater
discretion in planning for and
selecting transportation investments that meet local needs and priorities. As a
result, the transportation
investment decision-making process has been broadened to a wider range of
viewpoints and interests.
Moreover, new security imperatives in a world after September 11, 2001,
present additional challenges for
the transportation system that must be considered in the decision-making
process.
These trends raise questions about how to make transportation investment
decisions in an increasingly fiscally constrained and complex environment.
GAO and other federal agencies-including the Office of Management and
Budget (OMB) and DOT-have identified benefit-cost analysis as a useful
tool for integrating the social, environmental, economic, and other
effects of investment alternatives and for helping decision makers
identify projects with the greatest net benefits. In addition, the
systematic process of benefit-cost analysis helps decision makers organize
and evaluate information about, and determine trade-offs between,
alternatives.
The use of benefit-cost analysis in transportation investment decision
making was the subject of an expert panel that GAO convened on June 28,
2004, to discuss four key issues-how to conceptualize, measure, improve,
and use information about benefits and costs of highway and transit
investments. We convened the expert panel, in collaboration with the
National Academy of Sciences, as part of a larger study of the benefits
and costs of transit and highway investments. The panel included top
transportation economists and practicioners from throughout the country,
including David J. Forkenbrock, Jose A. Gomez-Ibanez, Ronald F. Kirby,
David L. Lewis, Michael D. Meyer, Donald Pickerell, Kenneth A. Small,
Brian D. Taylor, and Martin Wachs. (See app. I for the methodology we used
in convening the panel and a profile of each panelist.) We included the
major themes that emerged from the panel in our January report to the
House and Senate Appropriations Committees.5 However, the panel produced
many additional important insights that were beyond the focus of our
January 2005 report. Given the importance of these issues to decision
makers and the transportation community, we decided to publish a separate
report devoted exclusively to the results of the panel. We conducted the
work to prepare this report from February through April 2005 according to
generally accepted government auditing standards.
Following is a summary of the discussion among the panel participants. The
summary reflects the major themes that surfaced at the panel, and we used
boldface type in the report to highlight points that these experts
emphasized. Appendixes II to VII contain an edited transcript of the
panel's discussion as well as subsequent comments received from the
panelists based on a draft of this report. The views expressed by the
panelists do not necessarily represent the views of GAO or the National
Academy of Sciences. Appendix VIII contains a select bibliography and a
list of related GAO products.
This report will be posted on our Web site at www.gao.gov. For additional
information on our work related to transportation decision making, please
contact Katherine Siggerud on (202) 512 2834 or at [email protected]. Key
contributors to this report are listed in appendix IX.
Highlights of the Expert Panel Discussion
everal themes emerged as the panelists responded to the four major issues
that we presented for discussion-(1) conceptualizing, (2) measuring, (3)
improving, and (4) using information about the benefits and costs of
highway and transit investments. Although the expert panel was not
designed to reach a consensus on these issues or specific questions that
we presented, a number of themes emerged from the panel's discussion, as
shown in table 1.
* Benefi t-cost analysis can be a useful tool to inform
transportation investment decisions. Benefit-cost analysis can
provide important information to transportation decision makers
about transportation investments and a structure for discussing
benefits and costs of alternative investments with public and
private stakeholders. In particular, it provides an analytic
framework that decision makers can use to consider a range of
factors in a systematic manner and clarifies what is and is not
known about the impacts of a transportation project. However,
efforts to increase the use of benefit-cost analysis need to be
tempered with the knowledge that the results of benefit-cost
analysis represent only one factor of many that are considered in
investment decision making. Factors such as federal funding, public
* comment, limitations imposed by existing infrastructure, and
political considerations also influence investment decisions.6
o Transportation investments are not often compared across modes.
Alternatives in other modes are seldom systematically analyzed to
determine how efficiently and effectively they could meet the
transportation need. The highly compartmentalized structure and
funding of federal highway and transit programs work against an
advantage of benefit-cost analysis-the ability to evaluate how well
alternative investments meet transportation problems. Separations
between federal programs and funds give state, regional, and local
agencies little incentive to systematically compare the trade-offs
between investing in different transportation alternatives to meet
passenger and freight travel needs because funding can be tied to
certain programs or types of projects. In addition, the modal
structure of federal programs gives rise to advocacy for specific
modes or investments.
o Requiring benefit-cost analysis can be useful if it is fully
integrated into the decision-making process and not seen as a
compliance checklist. Since systematic analyses of the benefits and
costs of highway and transit investments are not often conducted
voluntarily,7 requiring a benefit-cost analysis for new highway and
transit investments could be useful. However, past experience with
federal benefit-cost analysis requirements shows that they can either
be treated in a pro forma way or "gamed" by the affected agencies.
This experience indicates that mandates alone are not sufficient. Both
incentives to conduct analysis and enforcement mechanisms would be
needed to ensure that the analytic requirement is fully integrated
into the decision-making process, thereby ensuring meaningful
compliance. Experts noted that lessons can be learned from other
federal analytic requirements. One expert also noted that the National
Environmental Policy Act of 1969 (NEPA)8 requirements are not
typically manipulated because such manipulation could result in a
lawsuit.
o Better analytic tools are needed to evaluate land use and
distributional impacts of transportation investments. Land use impacts
are often major drivers of investment choices. However, benefit-cost
analysis and other types of economic analysis usually pay limited
attention to land use issues, in part, because land-use issues-as well
as other indirect benefits-are difficult to estimate. The panel also
highlighted the importance of taking into account which groups benefit
from a project and which bear the costs. Although the distribution of
transportation investments' benefits and costs is an important local
concern, it is frequently not considered adequately in the evaluation
of a project's benefits and costs.
o Quality of state and local transportation data needs to be improved so
that travel models can accurately predict patterns, trends, and needs.
Local and state transportation agencies require valid, reliable data
in order to conduct analyses, including benefit-cost analysis. Yet,
experts expressed concerns about the quality of local and state
transportation data. Data quality is a pivotal concern in
transportation modeling, as the available data provide critical input
for travel models. For example, data about traffic flow throughout the
day, rather than at a single time, are crucial to producing valid
representations of travel needs and problems. However, reliable and
complete data are not always available-which can result in forecasting
errors. Collecting the data needed for modeling is growing more
expensive and difficult. For instance, a home survey of travel habits,
which identifies basic transportation needs and travel patterns of a
region and is the foundation of transportation modeling, is now beyond
most local transportation agencies' annual budgets, according to one
expert.
Appendix I: Scope and Methodology
e contracted with the National Academy of Sciences (NAS) to convene a
balanced, diverse panel of experts to discuss the use of benefit-cost
analysis in highway and transit project decision making and gather views
about options to improve the information available to decision makers. The
NAS Transportation Research Board (TRB) identified potential panelists who
were knowledgeable about benefit-cost analysis, transportation policy and
planning, highway and transit use, and transportation decision making. We
worked closely with TRB to select panelists who could adequately respond
to our general and specific questions about conceptualizing, measuring,
improving, and using benefit and cost information in investment decisions.
In keeping with NAS policy, the panelists were invited to provide their
individual views, and the panel was not designed to reach a consensus on
any of the issues that we asked them to discuss. We also asked the
panelists to submit two published articles related to the subject, which
were disseminated to the audience the day of the panel. (See app. VIII for
the list of articles submitted by the panelists.)
The panelists convened at the National Academy of Sciences' Keck Center in
Washington, D.C., on June 28, 2004, after reviewing discussion questions
that we provided in advance. To start the day, the panel moderator, Brian
Taylor of the University of California, Los Angeles, provided an overview
of the issues to be discussed; and during the remainder of the day, the
panelists addressed the questions we had provided for their consideration.
We did not verify the panelists' statements, although we did ask the
panelists, in some instances, to clarify certain details. The views
expressed by the panelists do not necessarily represent the views of GAO
or NAS.
After the expert panel was conducted, we used a content analysis to
systematically analyze a transcript of the panel's discussion in order to
identify each expert's views on key questions, and we used boldface type
in the report to highlight points that they emphasized. We also used the
content analysis to highlight principal themes that emerged from the
panel's discussion. To ensure that we accurately represented the
panelists' comments in our report, we provided each panelist the
opportunity to review and comment on the edited transcript. We
incorporated changes or clarifications provided by the panelists to the
draft report. Finally, we added endnotes to the transcript to define
terminology used by the panelists, where appropriate, and to reference
cited publications, laws, and programs.
The discussion summarized in this report should be interpreted in the
context of two key limitations and qualifications. First, although we were
able to secure the participation of a balanced, highly qualified group of
experts, there are other experts in this field who could not be included
because of the need to limit the size of the panel. Although many points
of view were represented, the panel was not representative of all
potential views. Second, even though GAO, in cooperation with NAS,
conducted preliminary research and heard from national experts in their
fields, a day's conversation cannot represent the current practice in this
vast arena. More thought, discussion, and research must be done to develop
greater agreement on what we really know, what needs to be done, and how
to do it. These two key limitations and qualifications provide contextual
boundaries. Nevertheless, the panel provided a rich dialogue on the
benefits and costs of transit and highway investments, and the panelists
provided insightful comments in responding to the questions posed to the
panel.
Participants in the expert panel included the following:
David J. Forkenbrock is Director of the Public Policy Center, Director of
the Transportation Research Program, Professor in Urban and Regional
Planning, and Professor in Civil and Environmental Engineering at the
University of Iowa. His research and teaching interests include analytic
methods in planning and transportation policy and planning. From 1995
through 1998, Dr. Forkenbrock chaired a National Research
Council-appointed committee to review the Federal Highway Administration's
(FHWA) Cost Allocation Study process. He is a member of the College of
Fellows, American Institute of Certified Planners, and a lifetime National
Associate of the National Academies. He is chairman of the TRB Committee
for Review of Travel Demand Modeling by the Metropolitan Washington
Council of Governments and a member of the TRB Committee for the Study of
the Long-Term Viability of Fuel Taxes for Transportation Finance. In 2004,
he received the first-ever TRB William S. Vickrey Award for Best Paper in
Transportation Economics and Finance for his work on mileage-based road
user charges. He received the Michael J. Brody Award for Excellence in
Faculty Service to the University and the State, from the University of
Iowa in 1996. He earned a Ph.D., from the University of Michigan; a Master
of Urban Planning, from Wayne State University; and a B.A., from the
University of Minnesota.
Jose A. Gomez-Ibanez is Derek C. Bok Professor of Urban Planning and
Public Policy at Harvard University's John F. Kennedy School of Government
and Graduate School of Design. His research interests are primarily in the
areas of transportation policy and urban development and privatization and
regulation of infrastructure. He has served as a consultant for a variety
of public agencies. His recent books include
Regulating Infrastructure: Monopoly, Contracts, and Discretion; Regulation
for Revenue: The Political Economy of Land Use Exactions (with Alan
Altshuler); Going Private: The International Experience with Transport
Privatization (with John R. Meyer); and Essays on Transport Policy and
Economics (ed.).
Ronald F. Kirby is Director of Transportation Planning for the
Metropolitan Washington Area Council of Governments. He began his career
in the United States as a Senior Research Associate with Planning Research
Corporation. He joined the Urban Institute as a Senior Research Associate
and became a Principal Research Associate and Director of Transportation
Studies. He has served on several TRB committees and is currently a member
of the TRB Executive Committee. He has a B.S. and a Ph.D., in applied
mathematics, from the University of Adelaide, South Australia.
David L. Lewis is President and CEO of HLB Decision Economics. His credits
include a range of widely adopted applications in cost-benefit analysis,
productivity measurement, risk analysis, and approaches to establishing
public-private investment partnerships. He has authored three books,
including Policy and Planning as Public Choice: Mass Transit in the United
States (Ashgate Press), 1999. His past positions include
Partner-in-Charge, Division of Economics and U.S. Operations, Hickling
Corporation; Chief Economist, Office of the Auditor General of Canada;
Executive Interchange Program and Principal Analyst, U.S. Congressional
Budget Office, Congress of the United States; and Senior Economist and
Director of the Office of Domestic Forecasting, Electricity Council. He
has a Ph.D. and an M.S., in economics, from the London School of
Economics; and a B.A., in economics, from the University of Maryland.
Michael D. Meyer is Professor of Civil and Environmental Engineering at
the Georgia Institute of Technology. Prior to coming to Georgia Tech in
1988, he was the Director of the Bureau of Transportation Planning and
Development at the Massachusetts Department of Public Works for 5 years.
Prior to his employment at the Massachusetts Department of Public Works,
he was a professor in the civil engineering department of the
Massachusetts Institute of Technology. His research interests include
transportation planning and policy analysis, environmental impact
assessment, analysis of transportation control measures, and intermodal
and transit planning. He is a Professional Engineer in the State of
Georgia, and a member of the American Society of Civil Engineers and the
Institute of Transportation Engineers. He has chaired TRB's Task Force on
Transportation Demand Management, the Public Policy Committee, the
Committee on Education and Training, and the Statewide Multimodal
Transportation Planning Committee. He is a former member of the National
Research Council policy study Panel on Statistical Programs and Practices
of the Bureau of Transportation Statistics. Currently, he is a member of
TRB's Executive Committee and Standing Committee on Statewide Multimodal
Transportation Planning.
Donald Pickrell is DOT's Volpe Center's Chief Economist. Prior to joining
DOT, he taught economics, transportation planning, and government
regulation at Harvard University. While at the Volpe Center, he also was a
lecturer in the Department of Civil Engineering at the Massachusetts
Institute of Technology. He has authored over 100 published papers and
research reports on various topics in transportation policy and planning,
including transportation pricing; transit planning and finance; airline
marketing and competition; travel demand forecasting; infrastructure
investment and finance; and the relationships of travel behavior to land
use, urban air quality, and potential climate change. He received his
undergraduate degree in economics and mathematics from the University of
California at San Diego; and Master's and Ph.D. degrees, in urban
planning, from the University of California at Los Angeles.
Kenneth A. Small is Professor of Economics at the University of California
at Irvine, where he served 3 years as chair of the Department of Economics
and 6 years as Associate Dean of Social Sciences. He previously taught at
Princeton University and was a Research Associate at The Brookings
Institution. He has written numerous books and articles on urban
economics, transportation, public finance, and environmental economics. He
serves on the editorial boards of several professional journals in the
fields of urban and transportation studies and has served as coeditor or
guest editor for four of those boards. In 1999, he received the
Distinguished Member award of the Transport and Public Utilities Group of
the American Economic Association. During 1999 to 2000, he held a Gilbert
White Fellowship at Resources for the Future. He has served on two TRB
policy study committees-the Committee for a Review of the Highway Cost
Allocation Study and the Committee for a Study on Urban Transportation
Congestion Pricing.
Brian D. Taylor (Moderator) is Associate Professor of Urban Planning and
Director of the Institute of Transportation Studies at the University of
California at Los Angeles as well as Vice-Chair of the Urban Planning
Department. His research centers on transportation finance and travel
demographics. He has examined the politics of transportation finance,
including the influence of finance on the development of metropolitan
freeway systems and the effect of public transit subsidy programs on
system performance and social equity. His research on the demographics of
travel behavior has emphasized access-deprived populations, including
women, racial-ethnic minorities, the disabled, and the poor. He also has
explored relationships between transportation and urban form, with a focus
on commuting and employment access for low-wage workers. Prior to coming
to the University of California at Los Angeles in 1994, he was Assistant
Professor in the Department of City and Regional Planning at the
University of North Carolina at Chapel Hill. Prior to that, he was a
Transportation Analyst with the Metropolitan Transportation Commission in
Oakland, California.
Martin Wachs is Professor of Civil and Environmental Engineering and City
and Regional Planning, and Director of the Institute of Transportation
Studies at the University of California at Berkeley. He was formerly
Professor of Urban Planning and Director of the Institute of
Transportation Studies at the University of California at Los Angeles
where he served three terms as Head of the Urban Planning Program. Dr.
Wachs' research interests include methods for evaluating alternative
transportation projects; relationships among land use, transportation, and
air quality; and fare and subsidy policies in urban transportation. Most
recently, he chaired the TRB policy study Committee for the Study on Urban
Transportation Congestion Pricing. He is past Chairman of the TRB
Executive Committee. Dr. Wachs holds a Ph.D., in transportation planning,
from Northwestern University.
Appendix II: Overview of Expert Panel and Opening Remarks
is an Associate Professor of Urban Planning and Director of the Institute
of Transportation Studies at the University of California at Los Angeles
as well as Vice-Chair of the Urban Planning Department.
DR. TAYLOR (Moderator): To examine the use of benefit-cost analysis in
transportation decision making, the panel will examine four issues-
conceptualizing the benefits and costs of transit and highway investments;
measuring benefits and costs; improving benefit-cost analysis as an
evaluation tool; and using benefit-cost analysis to inform public
decisions. Two panelists will offer comments to begin the discussion of
each issue.
A key question is implied in these four major issues that GAO asked the
panel to discuss: Why does benefit-cost analysis play a relatively limited
role in transportation decision making? I see three possible answers:
o it's the wrong analytical tool- we have other, better tools for this
purpose;
o it's the right tool, but often improperly applied; or
* it's the right tool, but hard to apply - in other words, we need
better data, a more sophisticated application of this tool, or a
more formal incorporation of this tool into decision making.
* Regardless of what may explain the relatively limited role of
benefit-cost analysis in transportation decision making, there
are at least four unstated premises to this issue that warrant
reflection:
o first, that transportation investments sometimes are misguided;
o second, that improper evaluations or failure to conduct evaluations
have played a role in misguided investments;
o third, that improved analyses can better inform transportation
decision making (or, put another way, if decision makers have better
information, it will be harder for them to make bad decisions);
o and fourth, that better informed decision making can reduce the number
of misguided transportation investments.
Collectively, these premises suggest links between information,
evaluation, and decision making that are far from settled in my view. The
panel faces an important question:
Will public officials actually find better, more transparent
evaluations of the transportation merits of proposed
projects "threatening" to the current, well-established
processes of transportation decision making?
I would contend that benefit and cost comparisons do, in fact, guide all
public investments in transportation, but not in the way that students of
benefit-cost analysis might expect. In practice, such comparisons center
on geopolitical benefits and costs-that is, they concern bargaining by
elected officials over the distribution of limited public resources.
In such a world, transportation benefits and costs are secondary. Thus,
geopolitical benefits and costs trump consideration of transportation
benefits and costs, so that programs and projects become the ends of
public investments, rather than means to transportation ends.
Further, the rise of legislative earmarking,9 which bypasses many
evaluation processes, increases the extent to which concerns over
geopolitical distribution of benefits and costs trump transportation
project analyses.
o ISTEA in 1991 included earmarks for 40 rail transit projects.
o TEA-21 in 1998 increased the number of earmarked rail transit projects
to 191, many of which were in places not normally viewed as ideal
environments for rail investments.
o All three versions of legislation pending in Congress in June 2004 to
reauthorize surface transportation programs contain significant
increases in earmarking over TEA-21.
Why the earmarking? First, earmarking bypasses evaluation processes that
vest bureaucrats with significant authority over transportation
investments. Second, most (though not all) earmarks are capital projects
that provide good "ribbon-cutting" opportunities, and attendant media
attention, for elected officials. Further, projects like new rail transit
lines, highway bypasses, and maintenance facilities generate local
economic benefits that are clear and unambiguous to both public officials
and the people who elect them. But while earmarked projects may be the
products of a careful geopolitical calculus, they may provide few
transportation benefits in relation to their costs. Our goal is to
recognize, and separate,
"Will public offi cials actually fi nd better, more transparent
evaluations of the transportation merits of proposed projects threatening
to the current, well-established processes of transportation decision
making?"
-Dr. Taylor
our consideration of these two effects-political and transportation-in
analyzing the benefits and costs of public investments in transportation.
Elected officials and transportation analysts think about transportation
investments in different ways, and this is the source of conflict about
analytical techniques. Transportation analysts and economists have long
advised us to focus on the transportation effects of public investments,
and not on the expenditure effects of such investments. The former concern
whether and how public investments lower transportation costs-such as by
reducing congestion, increasing safety, reducing emissions, etc.-and the
latter concern the direct effects of spending public dollars to hire
construction workers, pay truck drivers, and so forth. Indeed, most
analysts would argue that transportation investments should be judged,
first and foremost, on how they reduce transportation costs, rather than
on their local expenditure effects. Such transportation benefits make it
cheaper to produce current goods and services, make new forms of goods and
services possible, and benefit the economy by lowering transportation
costs for system users and society at large, as described in table 2.
For transportation analysts, the redistributive effects of expenditures
are largely a zero sum game. Although transportation expenditures can
generate significant local economic activity, much of it is simply
redistributed from other taxpayers and places that lost out in the
geographic competition for subsidy dollars. From this point of view,
policy makers are simply missing the point when they focus almost
exclusively on the local expenditure effects of transportation investment
decisions.
Despite such admonitions from analysts, however, many elected officials
and other policy makers view the transportation effects of public
investments as abstract, arcane, and arbitrary. While a new freeway ramp
metering project might smooth traffic flows, which in turn lower
production costs for a particular set of firms, which in turn increase
sales, which in turn add to total employment, such effects are difficult
to unambiguously link to the highway investment. In contrast, the
consequences of the public expenditures on transportation projects in a
given congressional district are clear and unambiguous-dollars get spent,
projects get built, people get hired. New highways and transit investments
are dramatic and highly visible and generate economic activity, especially
during construction. That much of this activity is simply shifted from
taxpayers in other jurisdictions is almost beside the point to most
elected officials.
For most elected officials responsible for transportation taxation and
spending, the overriding concern is with the equity of transportation
funding among states, districts, and jurisdictions. Concerns over who pays
and who receives are paramount. This concern ensures a political focus on
the expenditure effects of transportation investments and makes it all but
impossible for elected officials to consider the transportation effects of
investments. From the perspective of most public officials, it's the
transportation analysts and economists who miss the point by focusing on
transportation effects and tools like benefit-cost analysis in making
investment decisions. A Member of Congress from a western state, for
example, may find a study showing that rail transit investments in a
densely developed, older east coast city are likely to yield far greater
transportation benefits than those in his/her city all but irrelevant to
debates over the equitable geographic distribution of federal
transportation funds.
These divergent views pose several related questions:
o How do public officials view the benefits and costs of transportation
benefit-cost analyses? Are transparent evaluations of transportation
benefits seen as conflicting with and a direct threat to the
geopolitical logic of political bargaining? If so, does this conflict
explain why many benefit-cost analyses are conducted after the fact to
gather evidence to support decisions, and why many analyses are of an
already preferred alternative and some straw men? In my experience as
a metropolitan planner during the 1980s, it was evident that
alternatives were selected very carefully to ensure that they would
not be too effective in competing with the clearly preferred
alternative.
* If public officials perceive benefit-cost analyses as shifting
decision- making power and authority to analysts, does this help
to explain some of the criticisms leveled against the technique?
For example, unpopular benefit-cost analyses frequently are
dismissed for excluding factors that are difficult to measure.
While such criticism may be well founded, the results often are
not very sensitive to the excluded
* factors. But, this frequently is lost in "attack and defend"
debates over unpopular analysis results.
o How can we cope with deep conflicts over what constitutes good
transportation systems and good cities? While most transportation
analysts see lowering transportation costs (both for travelers and
shippers, and for society at large) as a principal objective, many
transportation activists and environmental advocates view declining
transportation costs as a problem. This is a vexing, often unspoken
issue that underlies many debates over benefit-cost analyses.
o Can evaluations focus more on clearly defined problems and less on
solutions to poorly defined problems? Analysts rarely are asked to
generate and evaluate alternative approaches to address a
transportation problem in the current project-focused political
climate. Instead, they usually are asked to evaluate/compare poorly
defined solutions-rail transit, increased highway capacity, high
occupancy vehicle (HOV) lanes, or bus rapid transit-to poorly defined
problems.10
Appendix III: How Should We Think About Transportation Benefits and Costs?
Opening Comments by Dr. Lewis and
Dr. Wachs
DR. LEWIS: There are two key points about conceptualizing benefits and
costs.
o Analysts and economists need to help decision makers look at their
choices, including highway versus transit choices, on a level playing
field-something we palpably lack today.
o We should reinvent benefit-cost analysis so that it facilitates
decision by discussion. Benefit-cost analysis needs to shift from a
study presented in a report and delivered by remote experts who stand
aloof from the decision-making process to a facilitated analysis
framework
David L. Lewis is in which stakeholders can participate in formulating
values.
President and CEO
of HLB Decision Decision makers have many single choices or combinations
of
Economics. choices, yet we rarely help them to look at their choices on a
level playing field. As Dr. Taylor pointed out, decision makers very
rarely and certainly never systematically ask for-nor do analysts
provide-a comparative analysis of the payoffs associated with
"Benefit-cost analysis needs to shift from a study presented in a report
and delivered by remote experts who stand aloof from the decision-making
process to a facilitated analysis framework in which stakeholders can
participate in formulating values."
-Dr. Lewis
investment alternatives. These may be alternatives in design, scope, or
mode pricing and various other alternatives. These also may be investments
in education, health, or even tax reductions. Nor are alternatives
analyzed in relation to timing-a consideration because there are many good
projects whose time has not come. When Dulles Airport opened in 1963 it
was empty. Today, it is unbelievably crowded. (See fig.1 for passenger
traffic trends at Washington Dulles International Airport.)
Figure 1: Passenger Traffic at Washington Dulles International Airport, 1962
to 2004
Number of passengers in millions
25
20
15
10
5
0
Year Source: GAO presentation of data from the Metropolitan Washington
Airports Authority.
Dulles Airport was empty for the first 25 or 30 years of its life. Does
that mean we were visionary in anticipating the huge crowds that would
ultimately use it? No. We could have used those billions of dollars (in
current prices) in much better ways in the meantime and still have beaten
inflation by a lot in building the facility 20 or 30 years later. There
are alternatives in scope, design, and time and ways to compare
alternatives.
Everybody understands and is generally comfortable with Return on
Investment (ROI) calculations.11 Any and all options and combinations can
be boiled down to their ROI's. Decision makers could be treated to a
clear, honest portrayal of a "risk-adjusted" comparative ROI of widening a
highway versus building a light rail line down the corridor, versus doing
a bit of both, versus doing nothing, versus delaying bits and pieces of
it, etc.
Maximizing ROI is a good, very accessible way for most decision makers and
stakeholders to appreciate how alternatives differ.
1962
1965
1968
1971
1974
1977
1980
1983
1986
1989
1992
1995
1998
2001
Martin Wachs is Professor of Civil and Environmental Engineering and City
and Regional Planning, and Director of the Institute of Transportation
Studies at the University of California at Berkeley.
Second, benefit-cost analysis needs to shift from studies delivered by
remote experts who stand aloof from decision making to reinventing
benefit-cost analysis as a means of facilitating decision by discussion.
We analysts take fundamental values-the value of human life, the value of
reducing environmental emissions and greenhouse gases, the value of a
job-as data. We have sophisticated techniques for measuring how people
feel about things and expressing those feelings in the form of people's
willingness to pay monetary equivalent values by empirically measuring
transactions in the marketplace and through survey data. But these values
are not data. Some modern economists and philosophers, like Amartya Sen,12
argue convincingly that discussion is the melting pot in which values tend
to form. Analysts and economists can help the public, decision makers, and
stakeholders discuss values by using benefit-cost analysis as a powerful
framework and facilitation tool. Welfare economics13 has provided an
incredibly powerful way of thinking that people drink up when it's
presented to them in a digestible format.
The conceptual benefits and costs of transportation solutions to
congestion, development, and mobility and the macroeconomic effects of
these solutions on jobs, income, and the tax base-including the
redistributional or expenditure impacts that Dr. Taylor discussed-can be
laid out to enable people to discuss things in a logical, reasonable way.
This process can help isolate the minority who wish to game the system or
bend the discussion to suit a particular outcome. Economic analysis,
benefit-cost analysis, welfare economics-whatever you want to call it-
brings reason to a debate if it is transformed into a facilitation tool.
DR. WACHS:
Two examples from California show the different poles at which
benefit-cost analysis is being discussed and used.
First, the California Department of Transportation (CALTRANS) provides a
benefit-cost analysis template on its Web site. It assists local planners
and decision makers with problems at the project level-an intersection, a
small corridor, something in a metropolitan area, or a grade crossing.
Project alternatives can be evaluated using CALTRANS's list of benefits
and assumptions about the value of time, of a life, of property damage, of
accidents, etc. It is possible to get an answer about the benefits of
highway widening versus traffic signal timing improvement options. It is
very useful to compare alternatives when you have limited resources and
can approach decision making with an analytical framework that is readily
available on line.
However, enumerating benefits (such as time savings, which often is the
largest benefit category) raises enormous questions and important
assumptions that require answers. Do we believe that non-work and work
travel should have the same value of time? We may not believe that they
should, but we make assumptions and operationalize these assumptions on
the Web site. Do we see the value of time as linear or nonlinear with
respect to the amount of time saved? I do not-I cannot usefully use one
minute saved in the same way that I can use 20 minutes saved. Do rich and
poor people have the same value of time? I think not. Do we believe that
the value of an old person injured or killed in an accident is the same as
the value of a young person? Yet, these important, implied questions are
addressed by assumptions and set aside for the purpose of analysis. At the
narrow scale of an intersection or a mile, this does not do much harm.
The second example from California is that benefit-cost analysis is being
advanced as the appropriate way to debate a major state public policy
issue. The question is whether California should build a high-speed rail
system among San Diego, Los Angeles, the San Francisco Bay Area, and
Sacramento at a cost of about $35 billion and 30 years to build. Voters
will be asked to tax themselves by voting on a statewide proposition. The
high-speed rail debate, couched in benefit-cost analysis terms, is highly
politicized. Proponents state that the project will provide enormous time
and travel time savings for high-speed rail users and car and air
travelers, based upon assumptions about the growth of car and air travel
over the next 20-30 years. Some say it will achieve smart growth by
concentrating new community development for a population growth of 30
million people in these corridors, preserving open space, and reducing
development and preserving agricultural land outside of these corridors.
What does this really mean? Are there really large transfers involved? If
one saves time using one mode, is this somehow a net saving for the state
or a transfer of benefits from one system to another, one set of users to
another, and/or one geographic area to another? We're told that one
enormous benefit will be reduced air pollution and energy consumption. But
Dr. Forkenbrock's article asks whether this is actually a secondary effect
of travel time savings.14
How can we conceptualize the difference between costs that will accrue to
those who use highways, air transportation, and rail 30 years from now
aside from making rather heroic assumptions? Are we not saying it is the
secondary effect that is very important in policy terms-the secondary
effect of the principal effect? Isn't that all an artifact of the
assumptions that we make? I have no difficulty making assumptions when I
am comparing one intersection configuration to another because assumptions
are necessary to get useful outcomes. Here, assumptions are being made
about matters of such enormous ethical, moral, and political consequence
that I am much more uncomfortable saying that benefits exceed costs by a
substantial margin, as proponents say. Their argument is based upon
assumptions that are reasonable if you are a proponent, but not reasonable
if you are an opponent. This argument is entirely about redistribution,
but the benefit-cost analysis is entirely about the benefits to whomsoever
they accrue, while the total cost is borne by every state citizen.
Dr. Lewis said in the paper he shared with us that the benefit-cost
framework enables us to have a debate because it facilitates dialogue.15
But at what level? It does so at the intersection or corridor level where
you might have bus rapid transit versus standard bus transit. But does the
benefit-cost framework confound or clarify the dialogue about an
enormously important statewide project? Can we focus on making
benefit-cost analysis more transparent and useful so that it plays a
positive role in such debates? Currently, benefit-cost analysis appears to
be limiting rather than enhancing this debate.
Panel Discussion
DR. SMALL: Dr. Wachs's last remark raises the question of why high- speed
rail proponents have couched their arguments in benefit-cost analysis
terms, particularly if the analysis is overshadowed by politics or other
considerations. If we can decide why other people are using benefit-cost
analysis, perhaps we can figure out how we might structure it to make
better decisions.
DR. MEYER: Conceptualizing benefits and costs depends on scale. At the
intersection or perhaps corridor levels, benefit-cost analysis is a very
important force in decisions. The benefits and costs are quite clear, but
the congressional earmarking level is off this scale. My state and
metropolitan government experience has indicated that geographic
distribution and political considerations increase as the scale increases.
We need to be careful in presenting benefit-cost analysis as the primary
tool for determining trade-offs, because it is only one piece of
information in decision making at a higher scale. In fact, cost
effectiveness may be preferable to benefit-cost analysis.
DR. LEWIS: I disagree. The question of scale is a red herring because you
can find benefit-cost analysis-not necessarily great applications of
it-conducted at huge scales. The Three Gorges Dam, which changed the lives
of millions of people along China's Yangtze River, became the framework
for a great deal of political and divisive discussion. It was not a
paragon of benefit-cost analysis and should have been done much better,
according to many critics. But it did not cause controversy because of
anything inherent in benefit-cost analysis theory or practice.
Benefit-cost analysis is something we offer to help people to think
through problems at any scale. If we lose political reality in doing that,
it is because we are not looking at the options, alternatives, and complex
policy arrangements that people want to include in benefit-cost analysis.
Nothing about benefit-cost analysis limits the scope or range of choices
that we look at-including comparing a rail system from Los Angeles to San
Francisco to options that would reconcile the losses or redistributional
problems that people far from the Los Angeles/San Francisco corridor might
perceive.
DR. TAYLOR: Does the definition of potential benefits change or evolve at
higher scales?
DR. LEWIS: What changes are functions of the scope, policy dimensions, and
range of choice. The problem with California's high-speed rail analysis is
that it is not compared to a baseline of not having that investment. Since
many other things could happen to those transportation dollars, this
should be explicit.
DR. KIRBY: Dr. Lewis's notion of benefit-cost analysis as an instrument of
discursive democracy in the paper he shared with us is a truly valuable
concept and a very constructive way for us to bring these techniques into
decision making. Benefit-cost analysis is about more than just informing
discussion-it also is about facilitating a decision. I also liked the
statement that the objective of discursive democracy is to reach consensus
without minority dissent, but I feel less optimism about that. In many
major and even smaller projects, we have minority dissent that does not
preclude a decision to move forward. Lawsuits also may affect decisions,
but that is the nature of the process. Good analysis and communication
with the public could resolve a lot of issues through facilitation and
discussion.
A major project to widen Washington's Wilson Bridge is an example of how
information and facilitation can help get people on the same page about a
problem. (See fig. 2 for photographs of the Woodrow Wilson Bridge
Project.) In this instance, opposing views of the problem were resolved by
simply looking at the data. A citizen group was determined that the
solution to congestion was building another bridge to handle all Miami to
Boston truck traffic. This posed the empirical question-what is the
composition of traffic on the Wilson Bridge? Data from our consultants,
models, surveys, and counts indicated that trucks were only 15 percent of
bridge traffic. Only 4 percent of trucks were coming into the region and
exiting at the other end of the region. The other 11 percent were trucks
doing business here, with one origin or destination in the region-often at
a local grocery store. The citizen group reacted with disbelief and
decided to do its own traffic count. At the next meeting, they agreed with
our data and our conclusion that this was primarily local traffic and very
much a local issue.
Figure 2: The Woodrow Wilson Bridge Project
Source:Woodrow Wilson Bridge project. The Woodrow Wilson Bridge Project is
a 7-1/2 mile corridor that begins in Maryland and connects to Virginia by
a bridge over the Potomac River. The project consists of the replacement
of the existing Woodrow Wilson Bridge, among other things. Pictured on the
left: Traffic crosses the existing structure as the new bridge rises in
October 2004. Pictured on the right: One of two V-shaped piers that will
support the draw span of the new bridge is constructed in December 2004.
DR. WACHS: This relates to the scale debate because it occurred at the
scale of an individual bridge and there was something to count. It was
very relevant to the success of the technique in this deliberative,
democratic setting. If the debate were about the benefits of high-speed
rail versus airport expansion over 40 years, could the citizens have done
the same thing?
DR. KIRBY: It definitely would be more difficult-especially in the longer
term. Regarding scale, it is very often the project's nature rather than
scale that determines the degree to which different groups are engaged. A
left turn lane to improve safety can be as contentious as a major highway
if it moves traffic onto a local street. It will bring the neighbors out.
A huge highway interchange project-hundreds of millions of dollars-in our
metropolitan area generated virtually no public comment. This was because
the project had virtually no local impact. Vast amounts of land around the
interchange already were owned, and there were only a few houses that
owners did not mind leaving that needed to be demolished. It is really a
question of the impacts and whom the impacts affect. It just depends on
the situation, as Dr. Meyer said.
However, the idea of discursive democracy is relevant. Objectively
presenting data analysis can help people focus on real issues, as opposed
to arguing about unrealistic issues, even in longer term decisions.
DR. GOMEZ-IBANEZ: I echo Dr. Kirby's comments and make two points. First,
Dr. Small raised one of the most interesting questions-why do proponents
of California's rail transit system justify it by benefit-cost analysis
when it is so hard to do a benefit-cost analysis of something so large and
long-term? It may reflect the appeal of systematic thinking and
rationality in modern Western culture, i.e., if you propose to
"One thing that discredits benefit-cost analysis is that it does not pay
much attention to the land use effects of major transportation capital
investments... transportation planners do not join that debate because
they have such poor tools for forecasting the land use effects and they
find it hard to adapt benefit-cost analysis to that context."
-Dr. Gomez-Ibanez
spend $35 billion, you should have good reasons that you can explain. This
is a very important, powerful leverage. It means that people are open to
the challenge of making explicit their assumptions in thinking that this
$35 billion investment is viable or in the public interest. They must
appeal to something beyond parochial self-interest. Their appeal
essentially must be similar to benefit-cost analysis-benefits to society
will be larger than costs.
Second, I agree with Dr. Kirby that scale does not mean less controversy,
but also with Dr. Wachs that the broader the scale, the more difficult it
is to apply benefit-cost analysis and reach a single number with
confidence. This does not mean that benefit-cost analysis or something
like it is not extremely helpful at a large scale. The California rail
transit program requires listing and thinking systematically about a set
of assumptions.
o Is it true that you would concentrate all the growth in this corridor?
o What does our experience suggest-is rail transit enough or is
high-speed rail better?
o What are the benefits of Smart Growth?16
o Since one benefit usually is reducing air pollution, would the project
result in less driving?
o Would it lead to less local infrastructure?
o What does the literature say?
o Are we going to save $35 billion on cheaper sewers, sidewalks, and
narrower roads or not?
Although you might not be able to reduce the discussion to a single
benefit-cost ratio or internal rate of return, you would force both sides
to be systematic about the assumptions that they are making and focus the
discussion in an extremely helpful way.
One thing that discredits benefit-cost analysis is that it does not pay
much attention to the land use effects of major transportation capital
investments. There may be enormous benefits from density or sprawl,
depending on your side of the argument. But transportation planners do not
join that debate because they have such poor tools for forecasting the
land use effects and they find it hard to adapt benefit-cost analysis to
that context. They understand the travel time savings, but how should we
think about the
land use impacts? Are land use impacts just a reflection of the travel
time savings? If infrastructure is under-priced, should it be? Doing more
land use evaluation of transportation investments rather than straight
transportation evaluation would help because it would force an open
discussion about the assumptions on both sides.
DR. FORKENBROCK: I agree and would build on these points with three
words-visioning, scale, and assumptions. Visioning is the key.
o What do we want our community to become?
o What do we want our region to aspire to?
o What sorts of economic progress are we pursuing through the project
being considered?
In Lewis Carroll's Through The Looking Glass, one character asked another,
how do I get there? The other character said, where do you want to go?
Well, I don't know. Then how do you expect me to tell you how to get
there? This is a difficult problem with benefit-cost analysis, i.e., what
do we want our city to become? What urban form do we aspire to? If we
spend all our time on the "big three measures"-the value/reliability of
travel time, safety, and vehicle operating costs-we can miss the whole
point. We can get a nice benefit-cost ratio for the wrong project. It is
something we have to worry a great deal about.
Getting things right is a function of scale. It is much easier to vision
what will happen with a left turn lane than with Boston's Central Artery
or some other very large project that will have a big impact on urban
form, travel patterns, activity patterns, and quality of life. (See fig. 3
for photographs of Boston's Central Artery Project.) Questions about the
valuation of external costs or the effects on air quality in an area, or
about putting a value on air quality changes versus travel time saved, all
bring us back to scale.
We should spend more time identifying the problem and less time analyzing
the wrong problem, especially as we get into big projects. Once we have
done that, we need to worry about assumptions and attaching values to key
parameters. This follows the major point on visioning. What are we trying
to accomplish? What will be the long-term effects?
DR. LEWIS: I sense at least two broad questions on the table: what is the
nature of benefit-cost analysis and what is the nature of transportation
benefits and costs? They are two different questions-both are very
operational, practical, and important questions.
"Benefit-cost analysis gives us a frame within which to logically
address...effects to which we attach value... and their costs and
benefits."
-Dr. Lewis
Figure 3: Boston's Central Artery/Tunnel Project
Source: Central Artery/ Tunnel Project.
The Central Artery/Tunnel Project replaced the six-lane elevated highway
(Central Artery Highway) that ran through downtown Boston, MA, with an
eight-to-ten lane underground expressway directly beneath the existing
road. The project spans almost 8 miles of highway, about half in tunnels.
There is no reason why the answer to the first question cannot be that
benefit-cost analysis is technically broad enough to frame a discussion
about the kind of community we want, rather than, given the kind of
community we want to be, which option is the most efficient and cost
effective to get us there. Benefit-cost analysis is a framework to think
about big questions. When it is all done, the accuracy of numbers becomes
less relevant than the framework or the fact that a community has been
able to systematically work through enough options to take a course of
action that commands some broader support.
But what is the nature of benefits and costs? We are hearing that in
addition to traditional time, reliability, and safety effects, other
effects have value and benefit-cost analysis is remiss in not dealing with
them. If it is hard to predict the effect of a transportation program or
project on achieving a land use outcome that the community values, then
benefit-cost analysis is a good framework for admitting that the decision
you take or delay has the following risks and rewards and that there are
things we do and do not know. Benefit-cost analysis gives us a frame
within which to logically address any and all of the effects to which we
attach value, both positive and negative, and their costs and benefits.
DR. TAYLOR: What would happen if there is such profound disagreement about
the visioning that Dr. Forkenbrock discussed that there is essentially a
tacit agreement to disagree, and individual projects that collectively may
be at odds with one another are pursued to satisfy different groups? For
example, some may want a compact, transit-oriented city and projects that
make sense on those merits. Others may want single -family detached
dwellings outside of town and to drive cars and projects to pursue that.
DR. GOMEZ-IBANEZ: Where I disagree and think Dr. Small's earlier comment
has power is that proponents of different viewpoints usually see the need
to explain their position-it may be why a compact city is desirable or why
sprawl is desirable-in terms that you can test. This may be whether a more
compact city generates less pollution, has less infrastructure
expenditures per capita, etc. Sprawl proponents say sprawl will result in
lower housing prices and more housing choices. Those are testable
propositions. People feel they should have reasons for their positions,
and benefit-cost analysis offers a way to structure the debate that Dr.
Lewis wants.
DR. KIRBY: Land use impacts are absolutely critical. One reason for
conflict between local transit project proponents and the Federal Transit
Administration (FTA) is that proponents (e.g., the transit agency, local
politicians, and land developers) have land use development goals. But
when they go to FTA, these groups are supposed to explain how they are
saving travel time and cost, which can be expressed in terms of
"generalized time savings." A lot of creative work is done to turn a land
development project into a travel time savings project. This is where a
lot of technical conflict arises. FTA now requires a procedure that tacks
a travel time savings component on the end of a four step modeling
process. However, it has a problem dealing with land development impacts,
as the following examples illustrate.
A new rail station in the Washington region is going into an old,
dilapidated warehouse area where the current land use forecast predicts no
activity. This station will stimulate much new development (evidenced by
the fact the developers are paying one-third of the station's cost). But
we are told that the same land use forecast is needed to evaluate both
alternatives-building the station and not building the station. This is
somewhat problematic.
Building commuter rail lines is another example. A transit user moves to a
distant suburb and takes commuter rail, thereby substantially increasing
the user's transit costs and trip time. It seems unreasonable, but
actually this is a true benefit of the commuter rail project. From the
regional perspective, focusing development along these transit corridors
offers choices and is what we are trying to do. Yet, this does not match
up with FTA's evaluation criteria, which are overly focused on travel time
benefits and used to compare projects around the country. That is a real
glitch in this process and one of many conflicts.
DR. TAYLOR: Panelists seem to be saying that analytical tools do a poor
job of addressing a range of benefits that are central to local concerns
and local decision makers. Is there a fundamental problem emerging in
applying these tools?
DR. KIRBY: Absolutely. Tools that are being mandated from the top are not
always appropriately applied. For example, DOT lists 13 environmental
streamlining projects around the country to be moved expeditiously through
the environmental process to demonstrate that you should not get bogged
down with paperwork. One that got through the process recently was blocked
in court because the Environmental Impact Statement (EIS)17 process did
not address the land use impacts of a circumferential roadway proposed in
the project. Despite the project having gone through the entire federal
review and approval process, some local groups felt that the land
development impacts of this project were adverse. They readily identified
an EIS provision on secondary and cumulative impacts and went to court.
The judge basically said he would defer to the Federal Highway
Administration (FHWA), unless a legal requirement was not addressed. He
concluded that secondary and cumulative impacts were not addressed and
stopped the project. It is a technical process failure when a special
streamlined project goes through formal review and is blocked in court for
what appears to be a very valid reason.
DR. SMALL: I endorse the idea that land use is a major category that is
omitted in benefit-cost analysis. The difficulty is that analysts worry
about not double-counting benefits and distinguishing between transfers of
benefits and real net benefits. Land use involves considerable transfers.
It goes to Dr. Taylor's point that you are in the middle of everybody's
self-interest and their effort to get their piece of the pie when you talk
about land use. So, land use arguments that are couched in terms of
benefits are difficult to distinguish from the transfers of benefits. That
is where analysts could help. While we do not have great tools for doing
this, we understand some things about agglomeration18 and its value as
well as positive and negative externalities from one parcel of land to a
neighboring one.19 Improving benefit-cost analysis to distinguish between
the net benefits and transfers deserves attention.
Summary of Panel Responses to Audience Questions
What is the effect of transit-particularly rail transit-on reducing
congestion and on land use?
o Dr. Lewis said that unlike highway travel times, which slow as more
people use the system, transit travel times remain similar regardless
of the number of people using the system due to transit's fixed
schedules; this creates congestion stability rather than reducing
congestion. He added that the level of benefits created through this
stability varies.
o Dr. Small noted that transit can (a) encourage consolidation of
employment in a downtown area and (b) enable people to live further
out and travel downtown along the transit corridor. In contrast, he
said highways encourage residents and employment to spread out to the
countryside. He added that a model might show that transit would be
good for downtown employment and suburban residences, whereas highways
would tend to encourage suburban employment and residences.20
o Dr. Pickrell added that all rapid transit capacity investments
therefore promote decentralization-at least of residences. He said
that the conventional thinking that transit will create more urban
area density or that highways will promote decentralization is
incorrect.
When comparing highway versus rail benefits for both freight and
passengers, what additional user benefits-beyond travel time savings-and
additional costs do we need to consider?
o Dr. Forkenbrock mentioned a number of issues related to freight
transportation investment, 21 including (1) whether in some
circumstances it can make sense to invest in truck-only lanes on
interstates and major highways, (2) whether states can invest in rail
to reduce the need for investment in facilities that serve trucks, (3)
the concern of the business community that a rail system for freight
would need to have a comparable arrival time reliability as trucking,
when the trucking industry is one of the most competitive industries
in the country while rail is exactly the opposite; and (4) the lack of
data to create a good investment analysis comparing the two modes.
o Dr. Pickrell said that there is a whole separate category of freight
benefits from the construction and use of highway infrastructure. 22
o Dr. Lewis conjectured that highway network improvements could
encourage freight shippers to reorganize their production and
logistics technology, resulting in a gain in productivity and a net
gain in Gross Domestic Product (GDP).23
o Dr. Forkenbrock stressed that trucks do not pay their full cost
responsibility for operating on the road and that if billions of
dollars are to be spent on truck-only lanes, it would be better to
have a proper pricing mechanism upon which to base those decisions.
Appendix IV: How Are Benefits and Costs of Transit and Highway Investments Best
Measured?
Kenneth A. Small is Professor of Economics at the University of California
at Irvine.
Opening Comments by Dr. Small and Dr. Pickrell
DR. SMALL: I would like to raise some problems that are likely to generate
a discussion of techniques for measuring costs and benefits:
o Measuring the value of life.
o Measuring how benefits and time vary with income.
o Identifying the real decision being analyzed.
The first problem is measuring the value of life. Dr. Pickrell gave us a
reading on DOT's guidance on the value of life and time, showing that DOT
backs away from a literal application of benefit-cost methodology
regarding these measurements.24 Although the guidance recognizes and
articulates the value of life concept quite well in the summary memo that
guides DOT staff on its use, the guidance is very, very cautious about
using it. It basically says-don't let anyone know you are valuing life.
Instead, it suggests using what is chiefly a cost-effectiveness
analysis-you analyze all other parts of benefit-cost analysis but leave
out the value of life. Then you rank projects by safety and the cost of
each life saved. Following this guidance poses a difficulty for
benefit-cost analysts because it asks us to treat differently one area
that is considered too hot to handle-or at least too hot to handle
explicitly.
The second problem is that DOT's guidance backs away from a literal
application of benefit-cost methodology with respect to how benefits vary
with income. It is well recognized that the value of time varies with
income. In fact, the guidance directs us to value time as a percentage of
wage rates. Yet the guidance declares that we do not want the value of
time to vary with income in project analyses. This is understandable-we do
not want any variations on the value of life-and this is not unique to the
United States. A little more surprising is that value of time
recommendations do not vary by mode. The rationale for this in the
guidance is that the information is not good enough. This may be true, but
the real rationale may have been the distributional issue again-that is,
not wanting to get into the issue that more poor people take the bus and
their value of time is lower, so we do not have to pay as much attention
to them in terms of valuing time saved via bus travel. While all these
positions have political explanations, they create difficulties in
performing benefit-cost analysis.
The third problem is that understanding the real decision being analyzed
is a critical, continuing problem for benefit-cost analysis. Figuring out
how to use a value of life calculation is a good example. You can say
someone's life is worth $2.5 million if you make that explicit and
recognize that you are not analyzing a decision that will take someone's
life away-you are analyzing a decision that will change the safety level
that people perceive. Usually, you are considering very small changes in
the safety level, for example, going from a 0.001 to a 0.0012 probability
of some adverse event. This is well articulated in DOT's basic guidance.
While this process may suggest a way of successfully incorporating the
value of life into an analysis, it brings up another problem related to
the issue that Dr. Wachs raised when he questioned whether, when analyzing
the amount of time saved, you can use 1 minute versus 20 minutes. The
question is what decision is being analyzed. If you are considering an
infrastructure improvement that will last 20, 30, or 40 years, are you
really giving some identifiable person a minute and asking them how much
do you care about this minute over 40 years? No. You are changing the
environment. Many people will buy and sell houses and change jobs and
alter their daily lives over many years. All those people are going to
face a 1-minute difference in some parameter that affects them, and
analysts will try to identify what effects will occur. This changes the
way of looking at it from whether somebody can use 1 minute versus 20
minutes to the broader context of how decisions will affect the
environment over time.
Donald Pickrell is DOT's Volpe Center's Chief Economist.
I generally advocate that benefit-cost analysis should not over-concretize
the benefits. We all tend to use ourselves as examples. While this can
keep us connected with the real world, it is a little dangerous. We tend
to make decisions more concrete to a particular situation-yours, your
daughter's, etc. In fact, these decisions are likely to be altered and to
affect circumstances in many ways that might not be immediately obvious.
DR. PICKRELL: An invaluable first step in identifying and measuring
benefits and costs is to develop explicit descriptions of the cause and
effect paths by which alternative decisions or investments may lead to
their anticipated effects. This step has hidden value for several reasons:
o It requires decision/investment advocates to articulate not only the
impacts that they expect and alternatives they prefer, but also the
pathways through which they expect those impacts to occur, as Dr.
Gomez-Ibanez said.
o It forces us to think clearly about how and how extensively each
alternative is expected to produce speed, safety, quality, or other
desirable transportation service characteristics.
o It helps clarify the magnitude and timing of specific resource
commitments that would be necessary to produce these service
characteristics.
In this process, it is important to develop quantitative estimates of as
many of each alternative's expected impacts as practical. This is not to
say that whatever cannot be quantified and denominated in dollars should
be ignored, as many critics of the practice occasionally argue. The fact
that we do not know how to monetize or quantify important impacts should
not be a reason to avoid quantifying impacts that we know how to quantify
and whose monetary value we can estimate.
It generally is useful to measure impacts as close to their source as
possible. This ensures the correct attribution of the impact to the
characteristics of the alternative that we expect to cause it. For
example, it is better to measure travel time savings produced by an
investment that extends some transportation service into a relatively
undeveloped area than to measure property value increases that
theoretically will result. We have discussed the importance of clarifying
land use impacts of transportation investments. But, as Dr. Small reminded
us, doing so involves wading into the morass of transfers and multiple
counting of benefits that are best, most reliably measured at their primal
source. Land use impacts represent the consequences of these primal
benefits working their way through the complex urban economy in which we
make most investments.
Measuring impacts at their source will greatly improve prospects for
heeding another deceptively simple sounding, but often violated
guideline-count each impact only once. Measuring impacts as close to their
causal sources as possible and stopping there helps to avoid the most
obvious multiple counting that tends to occur primarily on the benefit
side of the ledger. Similarly, one needs to be extremely circumspect about
including indirect impacts25 of infrastructure investments because most of
these represent multiple counting. While we want to think about how primal
project benefits are transformed into other forms through the urban
economy, we should not count them in a variety of places where it may be
convenient to identify them.
One also needs to be especially careful of including employment and
so-called multiplier induced benefits in benefit-cost analyses.26 Here is
an example of what not to do:
Seven Midwestern states that were purchasing rail branch lines being
abandoned by their operators were surveyed. The survey showed that
employees' wages from shippers that would remain in business along these
lines (as a result of state acquisition and operation of these lines) was
the major benefit category for continued operation of these branch lines.
So the states moved costs of the employees' wages to the benefit side of
the ledger. To make matters worse, regional impact multipliers were
applied to these payments. Not only were costs that were completely
irrelevant to the decision included as benefits, they often were counted
twice or more in evaluating the advantages of doing so.
One needs to be very careful about indirect impacts-they often count
something that we already have counted and included. A related issue is
that measuring impacts in their naturally occurring or generic terms
enormously facilitates comparisons among alternatives with different
designs or technologies-such as highways and rail systems. Generic
measures are those denominated in units-a person's hours of travel or
expected injuries-customarily used to measure consequences of most
transportation projects.
There is considerable hand-wringing in public debate and literature about
the difficulty of evaluating intermodal investments. However,
transportation is quite well equipped to do intermodal comparisons because
of the benefit measures that we customarily collect. There still are
pressing questions about how to value service quality differences in
different modes, but we are making progress in identifying specific,
measurable dimensions of quality such as travel time reliability and its
more esoteric aspects like privacy and security.
A final caution: do not confuse the effects of seemingly related but
ultimately separate decisions in evaluating benefits. The most common
violation is including the costs of proceeding with an alternative
investment as a benefit of making a completely separate decision- although
the first may bear on the subsequent desirability of the second, as shown
in the following example:
Proponents of constructing high-speed rail lines in inter-city travel
corridors through a state argue that the benefits include savings from
avoiding new highway and airport investments that otherwise would be
needed to accommodate growing travel volumes. This contorted logic
suggests that investing heavily to move people in the corridor is the only
alternative. This is incorrect, given the degree to which public
infrastructure investments commonly are mispriced. It suggests that
competing investments have only costs but no benefits. If this logic is
pursued to its extreme and we avoid spending on airport capacity by
building a high-speed rail line, we forego any benefits that expanding
airport capacity might have had without having considered those benefits.
The importance of separating decisions that can and will be made
separately and have logically separable impacts relates to what Dr.
Forkenbrock wanted us to think about-the collective impact of our
infrastructure investment decisions on evolving urban forms. In high-speed
rail corridors, the impacts may extend even to the forms of development in
corridors connecting major urban areas. While it is important to consider
the cumulative impact of separate investments on metropolitan area forms
and their desirability, it is very important to logically separate
decisions to avoid confusing their benefits and costs.
Panel Discussion
DR. TAYLOR: You just cautioned us against focusing too much on secondary
effects, double-counting effects that are too often double-counted, and
using the cost of avoided expenditures in some other mode but not the
associated benefits in an analysis. Are these mistakes the result of
incompetence, error, or a desire to count as many benefits as possible for
a project? Does this speak to motivations of the people doing the
analysis?
DR. PICKRELL: I don't know. The modal structure of most state agencies and
the federal DOT gives rise to more advocacy in evaluating alternative
infrastructure investments than one would wish to see. As Dr. Gomez-Ibanez
said, many participants in debates about major public infrastructure
investments focus on what benefit
"...by compressing everything into a single metric, you lose the ability
to discuss impacts that cannot be quantified."
-Dr. Forkenbrock
cost analysts regard as secondary impacts. These impacts are primary
concerns for noneconomists who, after all, represent the majority of the
world. Including these impacts is often motivated by a desire to refocus
on decision consequences that noneconomists view as the most important,
contentious, or debatable impacts. Often simple advocacy is at work.
Finally, financing for many locally planned/selected infrastructure
projects in which the federal government has a significant share tends to
transform costs into benefits before the eyes of local political
officials. Some focus on indirect benefits, such as job creation, is an
expected consequence of this financing structure.
DR. FORKENBROCK: Regional economic models are one of the main sources of
double-counting-the problem is outsourcing. This often happens when an
operating agency such as a Council of Governments or state transportation
agency subscribes to a very good model like REMI.27 First, the traditional
travel time savings analysis for freight carriers like trucks is added to
the model. The model then generates employment changes, tax revenue, etc.,
that result from economic activity moving into the region by virtue of
lower transportation costs. Lower transportation costs experienced by
trucking are then added to the economic tax revenue and employment. This
is blatant double-counting that probably reflects poor understanding of
how the model actually works, rather than incompetence.
Four more words-behavior, dread, philosophy, and rhetoric-are relevant to
this discussion. Dr. Small's comments about valuing time, etc., have
several important elements that relate to behavior. There is very good
research about how people value their own time. However, when we simplify
behavior, we miss a lot. For example, people have a five times greater
disutility of time when they are waiting for the bus than when they are
riding it and know they are en route to their destination.28 The highest
disutility of time occurs when people get off a bus in a strange
neighborhood and wait for another bus to come. People hate that. It really
is a question of different kinds of disutility.
Ian Savage at Northwestern University wrote an excellent piece that
illustrates dread.29 Dread is willingness to pay to avoid the risk of
losing your life-it is essentially the way we value saving lives. However,
his survey respondents were much more willing to pay to avoid dying from
stomach cancer than from the similar risk of dying in a car crash. People
regard themselves as better drivers than someone who would be involved in
an accident.
Discount rates illustrate the philosophy issue. If your philosophy favors
small government, you will want a high discount rate because benefits
accrue over a long period of time. When Richard Nixon raised the discount
rate to 10 percent, he declared that there was too darned much money going
into the public sector and that we were building too many things. This is
a really good way to reduce public sector investment because many studies
show how sensitive a project using benefit-cost analysis is to the
discount rate. It really matters.
Finally, rhetoric. We cannot measure some things, as Dr. Pickrell said,
such as the impact that a pound of carbon dioxide has on global climate
change.
There has been much debate about the right value to give to such
intangibles. Should we exclude them? The answer is no.
If we want a single measure at the end of the day-net present value,
benefit-cost ratio, or internal rate of return-how can we do this? I would
choose benefit-cost analysis because, as Dr. Lewis said, the purpose of
benefit-cost analysis is to help improve decision making and allow us to
discuss impacts that cannot be quantified but need to be part of the
dialogue. An important problem is that by compressing everything into a
single metric, you lose the ability to discuss impacts that cannot be
quantified.
DR. GOMEZ-IBANEZ: All of us probably would agree that there are indirect
impacts, such as employment or land use impacts, from transportation
projects. If you create a job for someone who is unemployed, most of us
also would agree that that person's wages are not a real social cost
because there is no opportunity cost of putting that person to work.30 The
opportunity cost is not the wages, as it normally is with someone who is
otherwise employed. And there are legitimate land use benefits. The
problem of valuing these indirect impacts centers on mispriced complements
to transportation, especially the possibility that other complementary
infrastructure such as water and sewer needed to develop a project is
under-priced. But zoning made historic land use choices. It is difficult
to argue that markets for transportation complements and substitutes are
perfectly competitive and function smoothly since project developers,
users, and residents often do not pay the full cost of infrastructure that
their projects use.
While each viewpoint about land use ought to be included in the debate
because it has some legitimate role, there are several reasons that we
fight so hard against including them.
o First, land use effects are often taken much farther than the limited
circumstances in which we believe they should be included.
o Second, it is hard to forecast the path between policy and predicted
outcome. Land use transportation models are by far the weakest part of
urban and inter-city transportation demand forecasting.
o Third, most of us believe that if you systematically measured these
indirect benefits, they would be small.
This presents a strategic decision-how to handle areas that are so
confusing and difficult to forecast that they become the refuge for
scoundrels who want to exaggerate a project's benefits. Discussing
indirect land use benefits is their opportunity to do so with the least
chance of being embarrassed by demonstrably true facts. Yet it is a real
problem for us because the public has enough sense to see some legitimacy
to these arguments. We run the risk of discrediting the entire process by
denying them. How can you let these views in and still force the
conversation about them to be reasonably clear sighted and hard nosed?
DR. LEWIS: That goes to my point that we need to consider subjective
probabilistic analysis as a method or framework of analysis.32 One reason
that we analysts get cold shoulders from stakeholders and decision makers
is that they perceive us as pretending to know that which we do not and
cannot know. Weather forecasters caught onto this a long time ago when
they started talking about the probability of rain. People are much
happier working with precipitation or hurricane probabilities than trying
to suspend their disbelief in the forecaster's ability to know what will
occur.
We could take our lead from the biomedical statistical world. The U.S.
Food and Drug Administration and National Academy of Sciences do good work
with panels that bring solid statistical data to the table. Then experts,
stakeholders, and people who work in the field develop another estimation
layer called subjective or Bayesian-what you believe to be true, based
upon your experience.33 For example, the Journal of the American Medical
Association recently published the study of two heart attack suppressive
drugs based on 10 years of double-blind trial data brought to the table by
the National Institutes of Health. One drug costs $1 per vial; one costs
$10 per vial. Which drug is more effective? A substantial amount of
empirical, hard-nosed data indicated that one drug was 1 percent more
effective than the other. However, a facilitated subjective probabilistic
study with paramedics, nurses, cardiologists, and other experts concluded
that the efficacy of the two drugs was indistinguishable. This study
remains the foundation for the American Medical Association's
recommendation on the efficacy of these two drugs.
We can allow benefit-cost analysis to be a more powerful, useful technique
for thinking through a topic based on values and defuse the "expert versus
expert" mistrust in numbers. We can do this by bringing probability
distributions of what we think is the value of a statistical life, a small
time saving, or the likelihood that someone unemployed will have a new job
along with the basis for these statements to the table. In doing so, we
present everything we know in the context of how wrong we might be. People
deal with this very comfortably, even if they do not have any statistical
background.
While Dr. Pickrell's point about going to the root source of benefits to
avoid double-counting is very well put, there is a source of danger in the
land use issue. Empirical evidence suggests that the value of time saved
does not entirely account for property value increases that seem to be the
capitalization of time savings. In suburbs where commercial and
residential developments spring up around transit stations, when you
account for measured increases in property value based on proximity or
time-related benefits, there seems to be a residual that is left over.
People appear willing to pay a premium to live or work in a transit-served
area and the density, diversity, walking distances, etc., that come with
that. Even if they do not use the transit system, they are willing to pay
a premium for office space or to live in these areas. One might call it an
existence benefit or option value as distinct from a user value. There are
land use and property values that are not entirely accounted for by using
the system for time savings.
DR. TAYLOR: If you saw a similar cluster of disparate elements in places
without transit investment, how do you know that the residual is caused by
transit investment time savings and not the agglomeration itself?
DR. LEWIS: You can never know. You could hypothesize that some people are
willing to bid up commercial or residential land prices because of the
direct or indirect effect of the rail station. Then you can study
neighborhoods that are similar except for the transit and measure the
change in land values. You can determine to what extent that increment may
be accounted for by time savings, but you will never know if it was
actually something else. This is why risk and probability are so
important. People may accept a 20 percent chance that this is the
explanation as a basis for accepting that value in a benefit-cost
analysis. We do not know unless we ask them, and this is a fundamental
part of facilitation. Are you willing to take an 80 percent risk that this
value might be a cause and effect link that will produce a certain transit
rate of return and some effects that you like, or an 80 percent chance it
might be wrong? My premise is that people do this in their minds anyway.
Benefit-cost analysis makes it formal and makes the size of the risk
understandable.
If the question is how to distinguish the need for values of time that
vary with income-the answer is that you must allow for it because values
of time do vary. Demand studies and forecast accuracy would be wrong if we
blur over that. The answer is to distinguish between positive
economics-the predictive end of benefit-cost analysis-
and normative economics-the valuation end.34 When you assume that everyone
has the same value of time because we believe it is fairer, you need to be
very explicit about what you are doing and ask whether others agree. If
not, let the market reveal values in the benefit-cost process. Then you
will get a different result for transit that helps poorer people, and
people will deal with that information.
DR. MEYER: Two of GAO's questions-what are the most common problems in
measuring the benefits and costs of transit and highway investments and
which problems pose the greatest obstacles to accurate measurement-may
take us in different directions. Much of our discussion has related to
valuing benefits-that is, the value of time, reliability, and life. My
field experience suggests that although people may raise their eyebrows at
valuing human life, they more or less accept it. The problem is
change-especially over a 25- or 30-year period. How do you predict
reductions in fatal crashes? How do you predict air quality changes such
as tons of X emitted? We mentioned developments in intermodal comparisons,
but that presupposes that our models can look at person hours of travel by
different modes with some confidence over a 25- or 30- year horizon.
While value issues are very important, existing technical tools and models
at the regional, state, and even more at the national level are unable to
predict change. In my transportation safety course, we spend a
considerable amount of time on safety, value of life, etc. When asked what
will be the reduction in fatalities for a city bus in 2025, students
accept the value of a life but say there is no way to answer the question.
In my area, we are examining how to put value and reliability into a time
perspective so that we can measure this when we evaluate the context of
our updated regional transportation plan. However, without the full
microsimulation model35 that can look at freeway system performance it is
very difficult to predict transportation system reliability in 2030.
DR. LEWIS: I think our difficulty in forecasting lies more on the
engineering and science side than the value side of the equation.
An analysis conducted for the National Cooperative Highway Research
Program sought to deconstruct uncertainty in estimates of the economic
benefits of reduced highway congestion. Analysts found that about 80
percent of the uncertainty is associated with potential error in the shape
of the speed-flow relationship (an engineering problem), while the balance
of error lies in uncertainty about the value of time (an economic
problem).
We are presenting the public with our best guess, but I do not think they
want our best guess. I think they want to know the likelihood that
the sun will shine. If we are honest and peer into our models for standard
hours and extract probability information, we can let them know that the
time savings of a proposal has about a 10 percent chance of being greater
than 3 minutes per person. If you want 8 minutes per person, it may have a
5 percent probability of happening. I think we know how wrong we are.
DR. MEYER: I agree and view this as a way to improve benefit-cost
analysis. On the other hand, it often is very difficult to explain to
nontechnical, elected officials that it is within plus or minus 5 percent
of the most likely result.
DR. TAYLOR: Are you getting around uncertainty by presenting probability?
Certainty and probability are different things.
DR. LEWIS: No. I think including probability makes information more useful
to decision makers, given the state of the information at any given time.
The bond insurance industry has been using traditional "four step"
transportation demand studies36 for revenue forecasts as the basis for
determining whether and at what price the project bonds are insurable. It
was something of a surprise to the bond industry to learn that such
forecasts have fully a 50 percent chance of being too high or low. To
obtain forecasts with a 99.5 percent certainty of being met or exceeded,
the four-step model must be quite considerably modified.
The probabilistic presentation is not a way of eradicating uncertainty,
but of making the information more appealing, truthful, and useful. This
is especially the case when it is used to compare a transit project with
widening a highway, major corridor, or other alternatives. You may get
similar results, but my experience is that one option has a much higher
risk of going underwater than the other, even though the risk profile of
the projects is very different. If we get this information to people, it
can be more useful.
DR. WACHS: Dr. Lewis has made the point that benefit-cost analysis is
extremely important to us because it gives us a language for our discourse
and enables deliberative democracy to take place. I return to that
question and ask whether what we have just discussed gives us any
confidence that it helps creates a deliberate discourse and helps us
govern ourselves as a democracy? Or, does it actually distract us? Does
discussing whether or not we are double-counting really help elected
officials and citizens who represent us come to better decisions? This
group is experienced and committed to benefit-cost analysis and would like
to believe that the answer is yes. However, there are critics.
"Does discussing whether or not we are double-counting really help elected
officials and citizens who represent us come to better decisions? ... we
have to...demonstrate that these important questions actually help make
better decisions."
-Dr. Wachs
I have been reading a book called Priceless: On Knowing the Price of
Everything and the Value of Nothing.37 It argues that if you want to
invest enough public resources to prevent humanity from becoming extinct,
then the benefit-cost calculus is irrelevant and not helpful. If a public
decision was made to preserve a species or build a rail line, surely you
can retrospectively conclude that benefits must have exceeded costs. In
Joseph Berechman's benefit-cost analysis of investing in the Appian Way,38
he concluded that if you valued intangibles at a certain level, then it
was worth doing; if you valued them at less, it was not worth doing.
Clearly, their society made a deliberative decision to invest in the
Appian Way without quantitative analysis. The relevant question is whether
the measurement questions that we have been discussing are so difficult to
address that they hinder our ability to have a deliberative, democratic
discussion. If GAO is asking whether we can encourage Congress to be more
attentive to this, we have to carry this discussion one level farther and
demonstrate that these important questions actually help make better
decisions.
DR. GOMEZ-IBANEZ: All evaluation is relative. I agree with your concern
about things becoming a distraction and the duel of the analysts being
irrelevant to most things. What is an alternative? What would you propose?
DR. WACHS: I am not sure. I certainly would not give up analysis,
modeling, prediction, using probabilities, etc. However, I might weaken
the requirement that everything be done in monetary terms. This
requirement may force us to make assumptions that govern what we are doing
and obscure the dialogue. Ordered, analytical approaches are very helpful,
but
it is healthy to keep asking whether our society benefits so much from a
dollar framework for benefits and costs that we should devote all this
effort to trying to get the numbers right.
DR. SMALL: The problem with looking solely at the primary benefits, rather
than the benefits where they finally accrue, is that doing so, although
absolutely correct in analytic terms, conflicts directly with the mandate
we've been discussing-making benefit-cost relevant to decision makers who
are not experts in benefit-cost analysis. Land use is one of the first
things that people consider as a transportation project impact, but one of
the last things on the chain of causation. We cannot afford to ignore it
by just presenting final benefit numbers, saying that benefits were
measured correctly at the source of the primary benefit and therefore we
can ignore land use effects. Benefit-cost analysis could be used in
deliberative discussions so that people actually understand that the
impact they passionately care about is a little further down the chain.
When you actually do the numbers, you can see if there are some real
indirect benefits. I hope that terminology can be developed to convey this
whole difficulty.
"We are very focused on quantifying dollar benefits and do not want to
talk about transfers, distributional issues, or externalities....But...
this is what people out there really care about."
-Dr. Kirby
Regarding employment as another indirect effect, I disagree a bit with Dr.
Gomez-Ibanez. We agree if somebody who is unemployed becomes employed,
that is an extra benefit. However, we may be ignoring other related things
by being too concrete. If a transportation project throws someone out of
work, we count it in a certain way. But, other things may be happening.
The Federal Reserve may be tightening monetary and employment policy
because it is afraid that the economy will grow too quickly. Even
something that looks like an obvious benefit, such as putting an
unemployed person to work, may not be in a broader context where there are
other policy objectives that cause that person to be out of work in the
first place.
DR. KIRBY: Regarding Dr. Wachs's concern about whether monetary terms are
just getting in the way, I think that if we see benefit-cost analysis as
distilling everything to a dollar number, then monetary terms are getting
in the way. There is much work that can illuminate impacts in an
extraordinarily useful way, but we cannot push it too far. We are very
focused on quantifying dollar benefits and do not want to talk about
transfers, distributional issues,39 or externalities-sort of second class
in terms of benefit-cost analysis. But the reality is that this is what
people out there really care about. We can illuminate those issues
enormously as long as we do not overstep what we can really do.
Distributional issues regarding affected populations are what decision
makers discuss. Land development is very important to local jurisdictions
that are watching their tax base and plays through the decision-making
process to the governor. If a transportation investment will take an
attractive new technology into the neighboring jurisdiction, rather than
mine, I care a whole lot. Dr. Lewis' example of people paying a premium to
live near a transit station raises such an issue. Lower income people for
whom we are trying to provide transit are being bid out of those areas and
must move to areas without transit. This issue, which is a distributional
issue, comes to the table in the political discussion. We may not be
comfortable with including it in our benefit-cost analyses, but it is a
very important issue.
The value of time in the context of who uses a toll or high-occupancy toll
(HOT) lane40 is a tremendously important issue
that Dr. Small's paper addresses.41 Most highway expansions proposed in
our region now are toll financed roads because we lack the ability to do
them any other way. Private firms proposing to build these facilities want
investment grade analysis from those who will issue and service the bonds.
They want to know who and how many people will use the road, usage at
various toll levels, and how much will be paid to investors. This moves
travel demand forecasting to a whole new dimension and puts new demands on
modelers. When we could keep drawing down on the federal highway trust
fund to finance transportation projects, it was a different decision. Now,
the project will not go forward unless there is real comfort with
forecasts.
The distributional aspect of HOT lanes is fascinating. At first, these
were seen as Lexus Lanes for rich people who had high values of time-the
assumption was that poor people would not use them and therefore they were
bad. This was a showstopper for quite a while. However, monitoring these
lanes' usage showed that it is not just the rich-people that value
reliability at all income levels use them. Women are disproportionately
represented because they have child care and other responsibilities. Dr.
Small demonstrates that complex values cause people to use these
facilities, but that these values can be analyzed. This analysis will play
into policy and decisions about moving forward with some toll facilities
because who will use them is a critical issue.
There is a tremendous amount that we can contribute by focusing on these
issues that appear to be somewhat less tractable. Who else is going to do
it? The discourse will occur anyway, but in a much less informed
atmosphere if we do not wade in on these issues.
DR. PICKRELL: To be clear, I did not advocate ignoring downstream effects
of transportation infrastructure investments. I advocated clearly
articulating the paths through which primary and secondary benefits that
become the main focus of public debate are generated so that they can be
reliably measured. Keeping these paths conceptually separate allows us to
respond to people who claim that land use consequences of their preferred
investment have been ignored.
To some extent, we get in these binds because we do not do our homework.
Research on taxi cab use that we read as graduate students clearly
demonstrated that they were not used exclusively by the rich. Taxis were
used by elderly people going to medical appointments and a variety of
people making trips that were highly valued. We are our own worst enemies
because we react to criticisms about so-called Lexus Lanes in a way that
does a disservice to what we otherwise, in other circumstances, would
claim that we already knew.
DR. LEWIS: I agree with Dr. Wachs's concern that seeking to monetize
everything could be distracting. As I tried to say in my paper,
benefit-cost analysis has diverged from what might be perceived as useful
because it is not consistent with some realities of the last 50 years. One
such reality is that most people are not satisfied with some of the
restrictive assumptions of conventional benefit-cost analysis.
One convention is the idea that a project can be declared a welfare
improvement if benefits are sufficiently large to enable beneficiaries
potentially to compensate losers and still remain better off. Today, it
matters to people whether such compensation actually is paid. Furthermore,
it matters whether the project and compensation, as a package, garner
community consensus.
The debate about making all transit systems accessible to people in
wheelchairs versus creating a separate transit service for people in
wheelchairs is an excellent example. When the benefits of creating a
separate transit service for those in wheelchairs were found to be greater
than making all transit systems wheelchair accessible, something had gone
wrong with the analysis. Benefit-cost analysis had not stepped into the
situation's reality and facilitated a discussion about the value we put on
accessibility above and beyond monetary considerations.
John Rawls' book, A Theory of Justice,42 emphasizes that there are certain
rights, duties, or obligations that are not necessarily enshrined in the
Constitution, but which we believe we have acquired. These include
environmental rights, disability rights, obligations regarding greenhouse
gas production, etc. They ought to be treated as givens, and benefit-cost
analysis should then ask how we can optimize our world in that context.
Economists need to be flexible to facilitate this discussion, and
benefit-cost analysis needs to morph into a framework in which that
discussion actually occurs. The Americans with Disabilities Act 43 would
have been passed 10 years sooner if full accessibility had been discussed
in that way. There is no reason why a benefit-cost process could not have
led to that result had we not been confined by our view of the analytic
process. Converting benefits and costs into money is important, but not
the Holy Grail.
DR. GOMEZ-IBANEZ: The benefit-cost analysis of universal transit access
compared to separate transit systems fits into Dr. Lewis's deliberative
model. People realized its limitations through discussion and decided to
do something else-that is, make all transit systems accessible to the
disabled. Although a narrow benefit-cost analysis said that the benefit of
making all transit systems accessible to the disabled is not worth the
cost, people responded that this is not strictly a benefit-cost question.
DR. PICKRELL: It provoked a discussion about whether making all transit
systems accessible is a right and the value of enforcing that right.
DR. TAYLOR: Dr. Kirby said that distributional issues are absolutely
central. We heard that benefit-cost analysis sometimes does not deal with
them effectively or effectively enough. Why is this so
"...distributional issues...are at the heart of benefit-cost analysis and
are the central reason why we need it."
-Dr. Small
important? There is one issue where we can make normative judgments about
redistribution-we are going to redistribute wealth from the haves to the
have-nots. Dr. Kirby also described redistribution as a competition where
there are winners and losers. Such a redistribution may not been seen as a
loss to anyone at the regional, state, or national level, but stakes are
enormous at the local level. Can consideration of these stakes-that is,
winners and losers-be incorporated into the analytical process?
DR. FORKENBROCK: Sometimes the opposite may be true, as with regional
economic models. We discuss how jobs will come into the region if we make
a transportation investment, but not where the jobs came from. The rule of
thumb is that if job redistribution occurs in the same jurisdiction that
is funding the project, there is no net gain. But when you use federal
funds, redistribution occurs in the same jurisdiction. A state that funds
redistribution of jobs from one community to another has made a mistake.
The difficulty is what I call the moral imperative-that is, decision
makers know that they have nothing to lose by arguing strongly for their
project-the federal gas tax will not go up or down as a result of a single
project. And if the model does not indicate where the jobs came from, no
other politician is going to tell me that these are jobs off my plate.
There is an advantage to obscuring this, and models do that so well.
DR. SMALL: I have strong views about distributional issues-they are at the
heart of benefit-cost analysis and are the central reason why we need it.
Some decisions are made with unanimity-perhaps everyone's incentives are
aligned. Benefit-cost analysis is needed when real differences in the
outcomes to people must be adjudicated. This usually occurs through the
hurly-burly of politics. But we are agreeing that benefit-cost analysis
can help this adjudication process lead to better results. Technically,
the adjudication process is a way to get potential Pareto improvements44
transformed into real Pareto improvements. This is because, when a project
has positive net benefits, there are still some people who are winners and
some who are losers-although there are more on the winning side. If you
apply this methodology consistently to many different projects and ensure
that it is not always the same group of people that loses, then there is
an increased chance that you will have real improvements for everybody.
These distributional issues are so fundamental for benefit-cost analysis
that in practice we often have to carry them out and make them explicit in
order to move forward with a project.
DR. PICKRELL: I agree that distributional issues are of paramount
importance. Any analytic process is embedded in an inherently political
decision-making process that turns on considerations of who pays and who
benefits. I personally believe that is for the better.
I was adamant about systematically tracing the mechanisms by which public
infrastructure investments are expected to produce direct and indirect
benefits because this naturally leads us to tally the benefits and costs
borne by different participants in the political decision calculus. At the
very least, it helps provide systematically forecasted rather than
speculative or woefully misrepresented information about the impacts'
nature and magnitude. It also produces the advantage that Dr. Small is
describing-we can examine whether the succession of decisions from this
political and analytical process tends to work to the advantage of some
groups or others over time.
DR. KIRBY: I do not disagree, but I am concerned that we tend not to focus
on things that we are less comfortable with analytically or less able to
quantify. As a result, we may be vulnerable to a challenge that we failed
to comprehensively assess a project and a judge is going to stop it. If it
was a good project that does not get built because the process failed to
address a legitimate issue, it is a real criticism of us as practitioners
and something to be very concerned about. You cannot make every project a
winner for everybody-and do not need to. If projects are in a large enough
context to show overall net benefits and that no group is systematically
without benefits, then I think you can proceed.
However, in our very elaborate public involvement process, you may be
unable to take action if anyone loses. The electorate is very well
informed.
People articulate the disbenefits45 to them and this can be enough to
overwhelm the more diffuse beneficiaries. This may be where we are in
transportation. Project after project goes on the shelf-it really is a big
concern. A county transportation director recently said that numerous
studies-but not one single project-were proceeding in his county. High
benefit transportation improvements were not going forward-paralysis by
analysis. If we do not recognize that there are some disadvantaged groups
early in the process, we will not be able to deal with the issues they
raise in an effective manner.
DR. TAYLOR: Perhaps we are not doing a good job of compensating the losers
if we have situations where a concentrated set of opponents can kill
projects for which the benefits overwhelmingly outweigh the costs. Are
there legal or procedural limitations to the ability to compensate losers?
DR. KIRBY: No. But we often fail to recognize that there are those who do
not benefit, never mind thinking about how we can compensate them. These
groups do not feel that their issues have been fully addressed and
sometimes use legal challenges to stop a project. Our failure to address
this is rather critical, as is shown in the following example:
These issues came up in a large project that involved land use questions.
People were concerned about the induced demand46 caused by a major highway
expansion project completed 20 years ago. We went back 20 years and looked
at the forecast. Traffic did exceed the original projections, largely
because the development shifted to that corridor away from other areas
relative to the forecast. There was no question-it was not from the
outside, it was a shift. Another jurisdiction used the study to argue that
a proposed new highway facility would take development away from that
jurisdiction. Based on that earlier study we are now saying that each
jurisdiction must revisit its land use and activity forecast under the
assumption that this proposed new highway facility will be built.
Employment forecasts relative to the project may be reduced for some areas
and come out on the table as part of the political discussion. There is no
way of avoiding that. But, we must put that topic on the table and
recognize that there will be some marginal effect. We cannot expect
everyone to win on everything-otherwise, nobody will get anything. If we
discuss this type of issue explicitly, we will be much better off.
DR. FORKENBROCK: The issue of compensating losers is fascinating, but
extremely elusive. A new highway is going to raise noise levels and people
are upset. But, what if the access value has raised property values?
Looking at many different dimensions, deciding on winners and losers, and
summing it all up could show that overall, the people who are upset lose
little or not at all, but the analysis would be extremely difficult.
DR. LEWIS: I agree with Dr. Kirby-there is evidence to suggest that
making the redistributional effects and the winners/losers more explicit
and transparent is likely to speed decisions more than slow them down.
We did a large discursive benefit-cost exercise47 for building a new
airport runway in a large Canadian city. The proposal had been on
the table for nearly 20 years and modifications had been knocked
down by noise advocates. Although the airport authority was
reluctant, we engaged the community in a very scientific discussion
of measuring noise and the empirical evidence on depreciation. We
even used loss of household as surplus in addition to financial losses
of value. Through an analytical process, we got quite wide assent
that there would be $100 million of property value losses over 30
years due to the additional noise over a specific area. But that was
put in the context of about $4.5 billion in economic benefits. The
benefits lost without the runway were mainly in time savings and
resources, not jobs.
This precipitated some compensation proposals that were integrated into
the solution-noise insulation, etc.-nothing terribly dramatic. There is a
runway there now, for better or worse. I think that the numbers created
political will that did not exist before in the environmental review
office, the litigation domain, and runway supporters. The numbers simply
made the case. I think you can make things real and the truth sort of
bubbles up to the surface.
DR. FORKENBROCK: The problem is that the objector is the same as the
loser.
DR. LEWIS: In airport cases, they often are. When you get to valuable
development around transit stations, perhaps you are gentrifying the
community and what was an affordable housing community is no longer. What
do you do with that? In this case, objectors are rarely the people who
would be displaced. If there is awareness that this could happen, then
simply acknowledge and analyze it. This was not done in the Model Cities
program, otherwise the removal of ghetto residents to even worse, poorer
ghettos might have been anticipated and Model Cities would not have been
the disaster that it was. On the other hand, if people living near a
station now have to walk a little bit further, we might acknowledge that
this is a reality of the market system. Being explicit about it makes all
the difference.
DR. GOMEZ-IBANEZ: I will throw a little cold water on this love fest for
distributional analysis. I do not disagree that keeping track of projects'
distributional consequences is ethically important-the whole Pareto
Principle depends on compensation-as well as politically prudent. If you
ignore them, you're going to fail.
However, I worry that we are not being honest about the technical
difficulty of some of what we are proposing. We are discussing several
kinds of redistributions. One is between different income groups; the
other is between different jurisdictions or locations. Both of them are
really difficult. What you need is some general equilibrium model that
traces out how these direct effects get transferred to other parties. The
classic example has been mentioned-that is, a lot of transportation
improvements' value does not end up with a traveler, but is capitalized
into the land values of an owner's property when the improvement is made.
But that is just one example.
In addition, economists are not very good at tracing the subtle ways in
which things get passed along. If the chain gets too complicated, as when
you make downtown more accessible and try to assess how much goes into
downtown property values or higher wages and profits for downtown workers
and businesses, it is very tough for us to give you
"...the geographic and the population distribution issues...are very, very
difficult, and benefit-cost analysis is only part of the total evaluation
framework."
-Dr. Meyer
an answer with much confidence. Focusing on jurisdictions is at least as
bad and is compounded by confusion about whether growth necessarily
benefits a jurisdiction. Often it is not clear-at least to existing
residents- that job or residential growth is such a benefit. So a probable
benefit to landowners may not be a benefit to all the citizens.
In addition, we are not very sophisticated about the implications of some
of the distributional effects, particularly geographically. On one level,
I agree that the arguments for tracing distribution are pretty strong. On
another level it is garbage in, garbage out. You may end up with some
quite naive and misleading estimates of distributional consequences. The
example of noise from airport runways probably is one of the places where
it is easiest and clearest. Many other things we deal with are more
complicated.
DR. MEYER: I agree with what Dr. Gomez-Ibanez just said because I face
both the geographic and the population distribution issues. They are very,
very difficult, and benefit-cost analysis is only part of the total
evaluation framework. For example, to get the message about real
distribution issues in regional investment, we did not look so much at
benefit-cost analysis as we indicated on a map where the investment was
going. We also indicated how this investment related to the poor and
minority population. This distributional impact got people's attention.
This visual relationship initiated a discussion about equitable
distribution of investment vis-a-vis population location and where the
revenues are coming from. We really should be talking about evaluation and
the role for benefit-cost analysis in this much broader framework.
Summary of Panel Responses to Audience Questions
Could benefits and costs be measured at the national level so that
Congress had a framework for deciding, for example, whether to spend $1
billion on transit or highway projects? If it is possible, is it a good
idea?
o Dr. Forkenbrock stated that the only way to do such an analysis would
be to use major projects undertaken in the past decade as sample
data-and this would be precarious.
o Dr. Pickrell said that the Highway Economics Requirements System
(HERS)48 model attempts to do this for highways by simulating the most
beneficial improvement projects on a large sample of U.S. highway
segments, adding the benefits of these projects, determining the
spending level required to generate those benefits, and expanding to a
national estimate of how much you would have to spend on highways to
achieve various criteria such as maximum benefits. He said that there
is no corresponding model for transit investments or one that
incorporates both potential highway and transit investments.
o Dr. Meyer said that we do not have the analytical tools needed to
provide Congress such a framework.
o Dr. Taylor noted that evaluating transit investments at the national
level is particularly problematic because the consumption of transit
services is so spatially asymmetric: since most transit riders are in
the centers of the largest and oldest U.S. cities, most of the
benefits of transit projects will be in these locations.
Appendix V: How Could Benefit-Cost Analysis Be Improved?
David J. Forkenbrock
is Director of the Public Policy Center, Director of the Transportation
Research Program, Professor in Urban and Regional Planning, and Professor
in Civil and Environmental Engineering at the University of Iowa.
Opening Comments by Dr. Forkenbrock and Dr. Kirby
DR. FORKENBROCK: To stimulate discussion, each of you has my handout,
"Improving Benefit-Cost Analysis."49 Several concerns have developed from
my practical work with state agencies to refine benefit-cost analysis for
major investments and with a team that conducted feasibility analyses of
three of the national interest corridors50 identified by ISTEA. These
problems occur particularly at the metropolitan and state agency level,
rather than the national level.
o One problem is excessive dumbing down of benefit-cost analyses. When a
colleague surveyed state agencies about their current benefit-cost
analysis practices, he found that a series of them do not even
discount their benefit and cost streams51-they just add them up. Many
state agencies and metropolitan planning organizations (MPOs)52 use
computer software that contains a range of assumptions that are not
explicitly chosen and that the user does not understand. A good
example is using national averages for the value of time. Similarly,
some analyses ignored network effects-that is, a project's effects on
the larger area if, for example, traffic is diverted. Safety and modal
shift effects also were ignored.
o A second problem concerns questions of who benefits, who loses. We
have talked today about the Kaldor-Hicks criterion-that is extremely
important.53 We have just finished a National Cooperative Highway
Research Program guidebook on taking into account
distributional effects and incidence that are very important, but very
commonly ignored, in benefit-cost analysis.54
o A third problem is that parameter values are very poorly chosen.55 In
state agency project evaluations, the basis for fatality or injury values
rarely is explicitly considered. Many states use very low values.
One corridor of national significance runs between two states. One state
uses a million dollars per life; the other uses $500,000 per life. My
colleague asked the states surveyed why they used whatever discount rate
they used and found that the basis was not well understood. As a result,
the impacts of different discount rates were very rarely considered. Yet,
the choice of discount rate and value of time can make a bad project into
a very good project or a very good project look infeasible. These
parameter values are incredibly important when evaluating specific
projects. Moreover, time values rarely are tied to local wage rates, and
there is no consideration of how the value of time might be varied.
The "second best" issue56 is almost never considered. This means that if
you are estimating demand on the basis of heavily subsidized prices, you
are not going to get an efficient evaluation. This fact is almost never
taken into account by benefit-cost analysis users at the state and MPO
levels. When I chaired the National Research Council oversight committee
on the National Highway Cost Allocation Study in 1997, we found that
certain vehicles-particularly heavy trucks with fewer axles-have a cost
responsibility that is perhaps 20, 30, 40 times the amount that they pay
in user fees. There will be very different results if you build a road to
serve current traffic levels and forecasted traffic increases than if you
price the road at full cost and see what the demand level is, using that
as the basis for your investment. The "second best" issue is a major and
difficult problem.
It occurs to me that one solution to these problems is the American
Association of State Highway and Transportation Officials' (AASHTO) new
manual on benefit-cost analysis-the Redbook.57
It contains good, useful information on doing benefit-cost analyses in
transportation. Other solutions may be practical courses on benefit-cost
analysis sponsored by FHWA, AASHTO, or some other group. Courses could pay
special attention to using sensitivity analyses58 to test the impacts of
different parameter values and increasing the use of state or
Ronald F. Kirby is Director of Transportation Planning for the
Metropolitan Washington Area Council of Governments.
regional travel demand analyses to estimate network effects. Many times,
travel modeling that is the basis for determining whether investments are
called for is done on a partial basis-the models do not actually look at
the entire travel corridor. State cost allocation studies59 have been
improving over recent years, but they still have many conceptual and
data-driven limitations. And, without knowing cost allocation levels, you
have no idea of exactly how close different vehicles are to paying their
full cost. This information can drive investment analyses and become a
critical element in doing confident benefit-cost analyses.
DR. KIRBY: From the perspective of an MPO that does a lot of number
crunching, modeling and analytical work, what we do does not map very well
onto an ideal benefit-cost framework. Like other MPOs, we have spent the
last few years very focused on air quality and computing regional
emissions to meet Clean Air Act Amendments (CAAA) conformity
requirements.60 These requirements are very stringent-they can block and
limit project development. MPOs are required to meet fixed "emission
budget" levels that are part of a much larger regional air quality
analysis that looks at all kinds of different sources. We must meet a
fixed air quality standard. If we are below it, we pass; if we are above
it, we fail. That is a benefit-cost framework for you. The U.S.
Environmental Protection Agency (EPA) currently mandates that we use the
MOBILE6 model.61 This model requires us to model 28 different vehicle
classes (the previous model had eight different vehicle classes). It puts
vehicles into different weight categories, engine types, etc., and
requires information that goes way beyond the data that we have for our
region.
Air quality modeling has been one of our major preoccupations because it
could be a fatal flaw. If we do emissions calculations incorrectly, we
could be challenged by those who might not like a particular project and
see air quality requirements as a way of slowing things down.
Air quality and transportation investment studies are insulated from each
other-this is a big problem, as Dr. Forkenbrock noted. The requirement
applies to individual projects and involves intensive work at the corridor
level. While it draws on regional travel modeling, the EIS is a separate
undertaking-often employing consultants who take the regional models and
use them at the corridor level. The focus is on a project and corridor-and
the process does not look as comprehensively as it should at regional
implications.
In a case in which citizens requested a study of alternative sites for
a bridge crossing, the project study team initially responded that
this was outside the project study scope. Yet, alternative sites were
within 10 miles of the existing bridge. A month later, the study team
"We have a tendency to focus too much on improving travel models and not
enough on improving the data that drive the models."
-Dr. Kirby
thought better of its answer and agreed to test where traffic would
come from and go to at other crossings. This is the philosophy that
surrounds some corridor studies. However, as scrutiny of these
studies increases, we are being driven to look at things much more
comprehensively. Travel modeling is a big part of MPO work and
much is being asked of modeling work.
We now are being asked to look at pricing issues in addition to air
quality issues. Pricing is a whole different approach to project planning
that puts a different angle on issues and poses very tough questions in
forecasting the impacts of different pricing strategies. For example, we
are doing a major corridor study in which the project is a new managed
roadway-18 miles, 6 lanes, with 50 miles per hour peak period speeds in
2030. Our job is to find out what the toll is supposed to be, see what the
revenues will be, and match this up with additional revenue sources. This
will press our modeling capabilities more than in the past. In addition,
groups who have various views about the outcomes they do and do not want
will scrutinize our modeling procedures. This will be one line of
challenge to the modeling outcome, and we are going into this project
knowing that the technical work will be challenged.
Elected officials and citizens now have a very substantial interest in
knowing everything that is behind modeling. Many things that formerly were
in-house technical issues are now scrutinized by citizens and elected
officials. For example, values of time and how well the model replicates
existing travel patterns, root mean square errors in terms of matching
model results to traffic counts are going to be very important.
This is good, but it requires us to explain many more things than before.
Data availability and quality need to be improved to improve benefit-cost
analysis. In my view, we have a tendency to focus too much on improving
travel models and not enough on improving the data that drive the models.
For example, there is growing interest in time of day modeling-how peak
travel periods are spreading at congested locations on the freeway systems
and how people react to congestion by changing their travel behavior. If
people react to congestion by traveling later or earlier and you increase
capacity, those same people will revert to the time they traveled before,
and congestion will return. Has your project failed to eliminate
congestion?
Modeling travel at different times of day is important, but travel models
are not terribly good at dealing with it or other traffic operational
effects. An obvious question is whether we have good time of day counts.
The answer is that we do not. We have found instances where the published
counts are clearly inconsistent. Although this is an important issue, we
have very few permanent count stations in our region and some are not
operating. It is difficult to get data from state agencies because it is
not a priority for them to collect counts at the level of detail that we
need.
Much more attention has to be focused on data if MPO modeling and analysis
is going to get to this new level of scrutiny. There are obstacles- home
interview surveys are increasingly harder to do, telephone surveys are
difficult, all survey response rates are going down. Those people who have
only cell phones are not included in a random selection of households.
Should we put global positioning system (GPS)63 devices on the vehicles
and track them around to get data in the future? Refreshing our data poses
really tough issues.
We really are in a new ball game-dealing with issues that we have never
been asked to handle before. For example, we need analytical methods to
help us predict how people will respond to various prices on a new lane
or-even more complicated-on one or two tolled lanes on an existing freeway
with lots of entry and exit points along the way. When the private sector
wants to know travel volumes to decide whether or not to finance a
project, we are facing an information challenge that we have not faced
before, namely to get "investment grade" forecasted volumes. Before you
can ascribe benefits and costs, you really need to know what travelers'
responses are going to be to the project. Until you can get a rough handle
on that, all the other issues-discount rates and everything else-are
rather academic.
There are plenty of consumers-the public and elected officials-for the
things that we are talking about today. But, we will need to do a much
more comprehensive job than we have done in the past if we are going to
respond to their interests and meet our planning requirements.
Panel Discussion
DR. TAYLOR: We have heard that there is less and less willingness to put
the significant resources required toward refreshing, updating, and
revising data that are collected in household activity surveys. At the
same time, there is much more detailed interest in your analyses. That
seems somewhat contradictory.
DR. KIRBY: The dilemma is that there is a long lead time in getting and
applying these data. If you want the data now, you should have collected
it 10 years ago and been improving the models over that period of time.
The increased funding from ISTEA and TEA-21 was a big help in improving
our data collection and model development. But a whole new home interview
survey is beyond our annual budget. If reauthorization of TEA-21 helps or
the state agency funds such a survey, it still will be 3 years before we
have results that are cleaned up, calibrated, and in the models.
Currently, we are dependent on what we were able to do over the past 10
years.
The idea of a standard national travel demand model disappeared 20 years
ago. There has not been a strong federal presence in this area-travel
modeling has devolved into independent vendors with different software
packages. As an MPO, we can choose among four or five vendors who tailor
models to meet our needs. But, there is no standard, as Dr. Forkenbrock's
committee concluded.64 If you ask how are we doing or are we doing as well
as anyone else-nobody can answer these questions because nobody knows the
big picture. We hope that DOT will fund TRB to prepare a synthesis of
modeling best practices to see how we can address this problem. We have
had 10 or 15 years of everybody doing their own modeling, and now it is
very difficult to know the current state of the practice.
DR. LEWIS: There are analogous data quality problems in other areas. One
type of solution is suggested by the practice of benefit-cost analysis in
relation to information technology investments. Here, large organizations
want to know if the benefit-a productivity gain-is going to outweigh the
costs. However, the productivity data often are as awful as the counts
that Dr. Kirby describes. So, what do you do about it? Increasingly, we
are saying that the quality of the data is what it is. The important point
is that continuously or periodically measuring productivity becomes a
necessary project cost. We have a baseline and even if we are unable to
measure productivity perfectly, we can measure it the same way each and
every time, starting before the project is implemented. For example, the
U.S. Department of Homeland Security has determined that queue lengths for
visitor programs65 will not only remain stable but decline by using tools
such as biometrics at various points of entry in and out of Canada and
Mexico. We are measuring the extent to which queue lengths are improved by
these interventions. The simple point is to measure it in the same way for
a year before the changes are introduced and thereafter. Track performance
on the basis of data for which one can be reasonably sure there is a
standard of measurement.
We do not have the habit of measuring the after-effects and benefits of
transportation projects. However, doing so has become second nature in the
information technology world because there were so many large-scale
failures that companies are being forced to take risks based on whether or
not benefits are likely to be realized. In a private or public toll road
situation, one could imagine something similar. For example, the private
or public toll authority would promise a certain degree of congestion
relief, and be willing to assume the risks, in exchange for rewards should
the results materialize at a level greater than forecast- that is, the
authority would get to keep the revenues that were above and beyond what
was expected. The point is, if we start measuring outcomes, we will insist
on consistent-not perfect-measurement. We will see whether the benefits
and costs realized match our projections and more importantly, are
worthwhile.
While we do not see much retrospective evaluation of how well we are
doing, we must be able to do it-not so much in relation to ridership and
revenue, but regarding economic benefit. If a project's evaluation could
be extended into the domain of economic outcomes, we might see some better
data and more frequent counts. If benefit realization must be measured,
money will be budgeted to do that over a project's life cycle.
DR. TAYLOR: Do we see less retrospective evaluation in transportation than
other areas? A huge amount of program evaluation goes on in health care
and welfare-very detailed evaluations that anticipate changes, look back,
and make adjustments. Why would there be more evaluation in other
government endeavors than in transportation? We also have talked mostly
about projects today. But huge federal and state transportation programs
also can be evaluated using some of the tools we are discussing. Why a
focus more on projects than programs?
DR. PICKRELL: Perhaps there is the same answer to those questions. My
sense is that there have been relatively infrequent attempts to assess
whether individual projects realized their forecasted costs, utilization,
or revenue-and almost no assessment of their benefits' similarity to
forecasts. However, there has been something of a cottage evaluation
industry at the program level. I am not sure why, except that we did not
do benefit-cost analyses of major programs until relatively recently.
Transit projects in the 1970s and 1980s were planned and constructed under
NEPA's environmental impact assessment process. This process provided most
of the information needed to estimate what should have been a project's
benefits, but it was almost never used to estimate benefits in the format
that we are discussing today.
FHWA has a tall stack of research that attempts to estimate the highway
program's benefits using a wide variety of analytic and econometric
methods. For example, David Aschauer66 and others did work on the
program's macroeconomic benefits. Although somewhat uneven, there is a
wealth of research on the subject-people have not shied away from it.
DR. GOMEZ-IBANEZ: One of my best Ph.D. students wrote a dissertation on
transportation and poverty. The study compared 20 families who had cars
and 20 families who did not-both poor and living in suburbs-and how they
managed. Despite my hope that this student would have a transportation
career, the student went into poverty research
"...we do not evaluate projects-people just want to declare success when
the streetcar starts rolling and be done with it..."
-Dr. Gomez-Ibanez
because that is where all the money is. Unlike transportation, where we do
not evaluate projects-people just want to declare success when the
streetcar starts rolling and be done with it-this student is evaluating
demonstration projects. Perhaps there is less of a sense in the
antipoverty, education, or health communities that they know what works.
However, there is much more willingness to evaluate these projects. In
transportation, there is an ideological battle between environmentalists
and smart growth supporters-each with confidence that truth is on their
side and demonstration is not needed.
DR. TAYLOR: Perhaps controlled experiments are done much more easily and
less expensively with individuals than with metropolitan areas.
Individuals are routinely selected from different groups and the eligible
pool at random and assigned to different treatments. In the metropolitan
areas, we would need to treat one city with a light rail system and an
identical city with an expressway system. It is harder to fashion the
experimental or research paradigm in our context than in theirs. Dr.
Gomez-Ibanez described the analysis as being entirely
different-individuals, households, and smaller neighborhoods rather than
metropolitan areas. Does that reflect institutional bias in the way these
things are administered? Why not analyze the travel behavior of
individuals, households, and firms? It could be like analyzing changes in
poverty in one city versus another.
DR. GOMEZ-IBANEZ: But the policies we are discussing are applied at a
metropolitan level. You treat the entire metropolitan area when you build
or do not build the transit line.
DR. PICKRELL: We were in the waning stages of a very ambitious program to
design and test various innovations in providing transit services-the
Services and Methods Demonstration Program-when I first worked for the
U.S. DOT. Reports produced by that program were absolute models of how to
generate all the information necessary to comprehensively analyze the
value of the investment, innovation, and service that had been performed.
However, the analysis invariably was left undone-it was not synthesized in
the format that we have been advocating.
DR. SMALL: I served on a TRB committee, chaired by Dr. Wachs, that
assessed the Congestion Management and Air Quality Improvement (CMAQ)
program.67 One of our recommendations was to systematically do
post-evaluations of projects, including cost-benefit analyses, because
evaluations being done were very haphazard and infrequent. In addition,
because CMAQ projects are experimental and innovative projects, it is
important to find out about their benefits and what works-especially at
the federal level.
"It is not coincidental that project evaluations do not occur-they would
harm the purposes of many of those who put forward the programs...the
issue is the geopolitics of resource allocation, not whether benefits
exceed costs for a particular project."
-Dr. Wachs
DR. TAYLOR: There may be another significant difference between an
antipoverty program like the Temporary Assistance for Needy Families
(TANF)68 that moves people to paid employment and a building or facility
where a public official can stand and cut a ribbon. The facility serves
important political purposes-people have been hired to work and build it.
It is less abstract than an employment project. On some level, it has
accomplished things that its promoters argued that it would. A more
sophisticated evaluation of whether it returns benefits to the degree that
some other investment might have seems more academic than the fact that
there is concrete in the ground. What benefit is there in going back and
saying that-while this may be popular locally-it is a dog project? There
is much risk and potentially little reward in those analyses.
DR. FORKENBROCK: Alan Altshuler's excellent book on megaprojects69 exhumes
a lot of old projects and points out what might have been wrong with the
forecasting and the process.
DR. LEWIS: In the information technology sector, companies that are
proponents of large-scale projects are forced to measure outcomes and take
risks in terms of their fee in relation to those outcomes. Might we not
visualize a similar incentive structure in transportation? Large
engineering and architectural companies often are proponents of large
transit and highway projects. If they were on the hook financially for
certain economic outcomes, the information and reason that goes into
project evaluation and the information about whether transit or highway
capacity represents the more effective solution-at least at the corridor
level-might improve. I do not know how realistic that might be in the
transportation world. However, it certainly did not seem very realistic in
the information technology world 10 years ago-now it is the state of the
art.
DR. WACHS: It is not coincidental that project evaluations do not
occur-they would harm the purposes of many of those who put forward the
programs. It comes back to the opening point that Dr. Taylor made this
morning-the issue is the geopolitics of resource allocation, not whether
benefits exceed costs for a particular project. I have been attending
symposia, discussions, and conferences for over 40 years where this point
has been made. We have not just discovered it-it has a life of its own.
Our elected officials have not responded to admonitions that more
transportation funds should be spent on evaluation. Their answer clearly
is that the benefits of doing so are smaller than the potential risks.
DR. MEYER: I am not so sure that the reason evaluations are so rarely done
is that benefits are perceived as smaller than potential risks as much as
it is limited money. If I were an MPO or state agency director and someone
gave us money to do before and after evaluations, I probably would do
them. The problem is that there is no money to do evaluations. I do not
agree that evaluations are not done because proponents are afraid to risk
discovering that they did something wrong.
DR. TAYLOR: Then you have to ask yourself, why is it different in health
and education?
DR. MEYER: There is a very clear answer-evaluation is very common to the
social sciences' function and service orientation. Transportation is
infrastructure. Although we now talk about transportation customers and
services, infrastructure programs do not have a tradition of doing before
and after evaluations. It just has not happened.
DR. FORKENBROCK: In some ways, health and transportation have similar
problems. My academic center has both health and transportation policy
programs. In health, epidemiology is sort of a dark science. It is
difficult to decide whether an intervention led to changed health status
in people because some smoke and some do not, some are obese and some are
not.
Our legislature created a transportation program to invest in highways to
promote economic development. Our center tracked each project that was
funded by the program in order to determine whether it had the intended
effect-did it create the number of jobs and produce the value added that
was expected within 3 or 4 years. We ran into the difficulty of economic
cycles.
Someone promised to generate 400 new jobs, but needed better
access to an interstate. So, we invested a couple of million dollars
and poured a road for you. However, you have only increased
employment by 75 people. Did the economy go bad or the industry
go through a cyclical perturbation so that things are not the same
as when we made the forecast? Was the forecast done with very
sanguine assumptions or did the economic cycle change? The
truth probably lies somewhere in between. It is very difficult to
do economic impact forecasts and determine X years later
whether that promised economic impact really occurred.
Changes in people's taste, economic cycles, and a hundred other
things can make that very difficult.
DR. PICKRELL: Something else may be at work here. My contact with public
officials who oversee construction of major transportation infrastructure
projects suggests that their view is that when the ribbon is cut, the
important work is done. In an extremely cynical sense, that is what makes
the local political machine work. If so, the important political part of
the project is finished at exactly the time that its transportation
function begins. I'm not sure I believe this, but offer it as a
hypothesis.
People are not consciously trying to cover up presumed failures-Dr. Meyer
is right. This fear does not underlie the unwillingness to look back. It
is admittedly inconvenient to know that a project failed to meet criteria
on which it was justified, but there always is an explanation that has a
superficial plausibility about it.
DR. GOMEZ-IBANEZ: A less cynical view would be that TANF can be redesigned
every year so it is worth finding out how the program works and adjusting.
But then you have to ask why communities that are building a transit
system are immediately proposing the next extension. Why are these
communities not interested in what happened?
DR. LEWIS: They could be interested if congressional appropriations
committees that put a lot of federal money into New Starts,70 light rail,
heavy rail, and highway projects insisted on performance reports and
measurements of returns. This would be a means of getting a handle on
return on the federal dollar, but not limiting it to the federal share.
One reason this does not happen is no one with a big hammer says that you
have to do it. If we could agree that doing so would be a good thing, this
could be an incentive for these committees to write language insisting on
it and put money into it or insisting that certain funds be designated for
it.
DR. PICKRELL: What Dr. Lewis is describing is essentially how the
cooperative research programs71 work-they are funded by a set-aside from
the program. Moreover, their perspectives tend to be broader than an
individual project. Perhaps we need a companion provision that a small
fraction of project level funding be dedicated to post hoc evaluation, as
the CMAQ report recommended.
DR. MEYER: I would like to ask Dr. Lewis about his discussion of the
importance of applying risk analysis or its principles to improve
benefit-cost analysis. What does this mean in practical terms? How would
it benefit the methodology?
DR. TAYLOR: I would like to ask Dr. Lewis about his comments concerning
looking at values and risks so that stakeholders define what is important.
Is this a slippery slope to a completely value-laden process where
people's perceptions of outcomes become divorced from empirical measures?
DR. LEWIS: I do not suggest that benefit-cost analysis be a broad,
free-for-all conversation. I am advocating that the benefit-cost analysis
discipline be applied both on the quantitative and value sides. The
discipline-the logic and everything we have learned about cause and effect
and measured values from revealed and stated preference studies72-should
be rendered accessible and brought
to the public deliberative process to increase its quality. I am not
suggesting that we ask people to come up with their own subjective values
of time without being constrained by that discipline. I am not suggesting
this in lieu of carefully measured value of time based on a good stated
preference analysis or econometric analysis. Risk analysis is being done
in biomedical research, other scientific research, and to some extent in
transportation.
I was involved in presenting value of time issues to a group that is
discussing a light rail system for their city. I told them that their
city's value of time estimate, based on a stated preference analysis, was
$14.50 an hour during commuting time and 40 percent of that at other
times. We tried to explain the statistical meaning of an expected value
and used other associated statistics to present the rest of the
curve-which inevitably will be a symmetric, bell-shaped distribution
because of assumptions that have gone into the analysis. People saw that
other values were possible. Then we layered in another level of
uncertainty from studies that came from outside the community. We produced
estimates from the Journal of Economic Statistics that present values of
time as low as $3 or $4 an hour during commuting hours, based on the same
kinds of stated preference methods.
Bottom line: these other studies seem to provide evidence that there are
extremes and other possible outcomes that differ from the expected value
of time-the data also appears skewed toward the low end of time values. We
redrew the probability distribution with a skew to the left, showing the
probability that the truth lies more on the downside of the expected value
than the upside. Then, we facilitated a discussion with local MPO planners
where the subjective element would enter. We asked about the reality of
value of time choices among communities on this corridor and how they
would modify what we had. We used elicitation protocols73 that have been
honed rather nicely in the biomedical world of statistical research. These
protocols elicited beliefs from local experts and those with experience in
observing traffic flows and household choices to add to the shape of this
distribution. Remember, a Bayesian probability74 has to start from the
frequency with which something happens-just a counting exercise. Bayesian
is probability. It reflects the degree of belief that an expert or someone
whose beliefs are convincing is right that something will happen with a
certain probability. We tried to allow for a degree of subjective
probability. During this discussion, people learned about the data-a bit
about the central tendency,75 the down sides, probabilities associated
with the down side. We went through the same exercise for statistical
lives, injuries, and the environmental value of a ton of carbon dioxide.
"...one way to improve benefit-cost analysis as a tool in transit is to
actually do it. It is not being done today..."
-Dr. Lewis
In my earlier airport runway noise example, we had property distributions
based on econometric studies of the property value effects of a unit
increase in overhead noise. We also brought in real estate agents and
taught them how to think probabilistically so they could contribute to the
shape of the distribution for that community. We ended up with what we
believe is a scientific, locally informed representation of the
probability range for values of time and life. We can do the same thing on
the quantity side.
The four-step congestion relief model is how uncertain are we about the
cross elasticity of transit demand76 with respect to its generalized cost.
We try to get at the distribution and how evidence from other studies
informs the shape and breadth of this distribution. We then get local
people's beliefs. The value of the elicitation process is its discipline.
For example, if something is 30 percent likely to happen, then it is 70
percent likely not to happen. Simple disciplines like that smoke out some
of the strategic behavior rather effectively. If you break the problem
into its logical parts and attack one variable at a time, you can bring
the discipline to bear on how people behave. Then you put it all into a
Monte Carlo type simulation77 and see what the results seem to suggest.
The valuation question is not asking people to invent a value of time
number based on their anecdotal instincts. It is looking at scientific
evidence and working their beliefs into the context of that evidence. In
doing so, people are learning a great deal about the science that we bring
to the table.
Finally, one way to improve benefit-cost analysis as a tool in transit is
to actually do it. It is not being done today, so I would hate to see us
move on without that getting into the record. FTA's New Starts is many
things, but it is not benefit-cost analysis. This means that neither we
nor the Appropriations Committee will ever be able to compare the rate of
return or the net benefit of building a light rail system along Highway
I-71 to widening it instead. It seems to me that we must be able to
compare different ways of using transportation money. How to improve it?
Let's do it.
Summary of Panel Responses to Audience Questions
How can the quality and value of transportation data be improved to
provide better information for decision makers and benefit-cost analyses?
o Dr. Kirby emphasized that the first step is to thoroughly review the
existing travel modeling process to document the need for better data
and to focus on which data is most important to collect. He said that
analysts are likely to need different kinds of information collected
in different ways, such as GPS and smart cards,78 particularly as old
methods of data collection such as household surveys are becoming more
difficult to perform. He added that some transportation programs focus
all their resources on developing very sophisticated modeling
techniques rather than on collecting the better data that they really
need.
o Dr. Forkenbrock said that there is a need for a better behavioral
understanding of urban trip making. He noted that researchers like
Sandra Rosenbloom79 have shown that women's travel behavior in cities
is more complex than men's travel and involves stops along the way,
while traditional four-step travel modeling generates information on
trips only from one traffic analysis zone to another. He said that an
understanding of the complexity of trips is needed to assess the
benefits of different transportation interventions.
Appendix VI: How Can Benefit-Cost Analysis Be Most Useful In Investment
Decisions?
Michael D. Meyer is Professor of Civil and Environmental Engineering at
the Georgia Institute of Technology.
Opening Comments by Dr. Meyer and Dr. Gomez-Ibanez
DR. MEYER: I thought it would be interesting to share information about
how some other countries use benefit-cost analysis in decision making. I
recently was in Australia, New Zealand, Japan, and Canada with a team to
examine transportation performance measures and their use at the national,
state, and provincial levels. In Australia, we found benefit-cost analysis
being a very important part of the support structure for infrastructure
decisions. As my paper describes, the State of Victoria portrays
benefit-cost analysis as an important part of its decision-making process
and describes it as a risk-based approach. 80 The State of Victoria
defines benefit-cost analysis in a very traditional way-much the same as
we do here.
We found that Australian state officials monitor some performance
indicators on a yearly basis. These indicators relate directly to what
they call a "return on construction expenditure" (RCE) and an "achievement
index." The RCE is essentially a benefit-cost ratio. In Victoria,
benefit-cost ratios are prepared for all road projects. Interestingly,
road officials explain projects with benefit-cost ratios of less than one
as being "political projects." Australian road officials also conduct
before and after studies of the benefit-cost ratios themselves. A sample
of implemented projects is targeted for the collection of after data to
determine what the actual benefit-cost ratio is after approximately 2
years. The ratio of the "after" benefit-cost ratio to the "analysis"
benefit-cost ratio is defined as the achievement index. One of our first
questions was how much time they allow for achieving a steady state in
benefits. Two years seemed to be the common time frame to develop the
post-implementation benefit-cost ratio, although this varied because of
the different scopes of the projects (everything from a left turn lane to
major highway construction).
The Victoria state government has an auditing agency that audits the
analysis and the management activities of all state agencies. This
independent, semiautonomous group actually provides another review of the
quality of the decision support function in the transportation agencies.
Australia also has identified performance indicators for the different
states that are used to compare one state to another.
Victoria officials emphasized that their approach is important to convince
Australian parliamentary decision makers that VICRoads (the state road
agency) has a good technical analysis process in place to justify its
investments. AUSTRoads-a national organization like our AASHTO-recommends
that all Australian states take this approach. VICRoads also is headed in
the direction of putting risk analysis into their cost estimates-an issue
that Dr. Lewis raised earlier. The risk analysis essentially says that the
longer you wait to implement a project, the greater the uncertainty
associated with benefit and cost streams. By doing this, VICRoads is
trying to incorporate some sense of uncertainty into the project
implementation time frame. This is interesting because it is really a
different model than we would usually find in the United States.
Turning to GAO's questions, the first one concerns the most appropriate
roles benefit-cost analysis can play in contributing to transit and
highway investment decisions. Dr. Lewis talked about bringing benefit-cost
to the discussion of projects and I completely agree. It is a very
important piece of information for investment planning and decision-making
processes. It is not the only criterion that one would look at in decision
making-no one here has said that-but it gives decision makers a sense of
what it is they can get for the investment they are considering. I view
benefit-cost analysis as one very important part of an evaluation process
and decision-making framework.
Dr. Jose A. Gomez-Ibanez is Derek C. Bok Professor of Urban Planning and
Public Policy at Harvard University's John
F. Kennedy School of Government and Graduate School of Design.
As we said earlier, benefit-cost analysis can be very important for
looking at distributional calculations, if appropriately done. It can give
a sense of what distribution may mean in either geographic and/or
population group terms. In some cases, this can become an incredibly
important factor in decision making. Benefit-cost ratios can become a very
important element of how decisions are made about projects that are on a
smaller scale.
When should benefit-cost analysis be conducted? Clearly, the answer to
this is before you make the decision! This is a bit facetious, but really
an important point from the perspective of local, regional, and state
government. The evaluation process for these agencies occurs during the
planning process, when the relative values of projects and strategies are
considered and when benefits and costs are defined.
Similar to the Australian experience, we need to do some assessment of
what happens after projects are put in place. VICRoads assesses a 10
percent sample of all projects implemented each year-about 40 projects out
of approximately 400 projects. This provides feedback to the project
analysis process and critically examines the effectiveness of the
information produced that supports the decisions.
Should benefit-cost analysis be required in planning transit and highway
investments? Clearly, the answer is yes. Some analysis framework that
looks at benefits and costs certainly should be required. If correctly
done, benefit-cost analysis is one approach that could satisfy that
requirement.
What is the federal role in improving the quality and use of benefit-cost
analysis? We have talked about providing dollars to improve data quality,
supplying technical guidance and information relating to modeling and the
dissemination of information, and distributing information about best
practices. This is most likely the best federal role.
Do retrospective analyses of the performance of transit and highway
investments have value? Absolutely. Positively yes. No question about it.
DR. GOMEZ-IBANEZ: I am only going to talk about the question of whether
benefit-cost analysis should be required. The implicit question that I
imagined was whether Congress should require benefit-cost analysis or
something like it. One thing that struck me in this session is that Dr.
Pickrell and Dr. Kirby both essentially said that all the ingredients for
benefit-cost analysis are there. Local agencies are required, mainly by
NEPA and the CAAA, to do elaborate analyses and generate much of the
information you would need. We just do not take that extra step of doing
the benefit-cost analysis.
"I recall several lessons from requiring and applying analysis...At its
worst, the analysis requirements ended up being like an arms race where no
one was much safer than they were before-they were just spending a lot
more on weapons."
-Dr. Gomez-Ibanez
One of the interesting questions is whether benefit-cost analysis should
be required--since it might not be so much of a burden-and what the
consequences might be. There are many places we could gather information
about how such a requirement would work. We have made analytic
requirements, either benefit-cost analysis or something similar to it, for
projects and local planning in many contexts and many ways. The 1934 Water
Act mandated that the Army Corps of Engineers (COE) execute only projects
whose benefits exceeded their costs. COE now has been doing benefit-cost
analysis for 70 years. The FTA's predecessor, the Urban Mass Transit
Administration (UMTA) was mandated to use cost effectiveness, which was
somewhat like benefit-cost analysis. . Analytic requirements have also
occurred in other contexts. Planning-Programming-Budgeting (PPB)81 was
required for systematic analysis of all federal government budgets in
1960. The State of Victoria's experience with analysis is not surprising.
The British have required their department of transport to do formal
benefit-cost analyses of all projects for decades. They take economics
much more seriously than Americans. Each Canadian province has the Crown
Corporation Commission do benefit-cost analysis of Crown Corporation major
investments.
I recall several lessons from requiring and applying analysis. One lesson
is that two can play this game. When you require analysis, you put
something at stake-such as when PPB was required or UMTA was first
required to perform cost effectiveness analysis. With these requirements,
agencies initially needed help doing the analysis. For example, there was
considerable confusion in federal departments about how to do PPB. That
put the Bureau of the Budget (BOB)-later the Office of Management and
Budget (OMB)-at an advantage in dealing with departments. After training
these departments, OMB had to be an analytic cop as the departments got
good at PPB. Federal departments learned how to cook the analysis to favor
their preferred alternative, and they got more and more sophisticated
about it. At its worst, the analysis requirements ended up being like an
arms race where no one was much safer than they were before-they were just
spending a lot more on weapons.
An even worse outcome at the national level is that analyses become so
technical that the public drops out. Dr. Lewis' goal of informing local
decisions gets lost. You hire two experts to dispute each other,
newspapers play it up as a dispute, and the public generally reacts by
thinking that if two economists cannot agree on whether the benefit-cost
ratio is four or one-fourth, it must just be all garbage and we will
ignore it.
A second corollary is that benefit-cost analysis never ends up being the
determinant of decisions. Benefit-cost analysis was never the determinant
of what the COE did. Despite PPB, federal departments ended up doing what
they always did-the analysis did not seem to change the outcome. And
UMTA's New Start analysis-its cost effectiveness index- did not stop
earmarking, which I think was its goal.
DR. TAYLOR: They might argue that it has increased earmarking.
DR. GOMEZ-IBANEZ: It may have. But actually, I am going to counterargue.
If you are thinking seriously about requiring benefit-cost analysis or
revising the FTA cost effectiveness analysis measure as a tool, it really
would be worth your while to read about COE's benefit-cost analysis, PPB,
and UMTA's cost effectiveness analysis. This history would show some
success in all these cases. Success is not measured by getting a portfolio
of projects that is as profitable as what VICRoads is claiming. The
Tennessee-Tombigbee82 Waterway and other examples of projects that had
post-analysis requirements certainly belie that.
An analysis requirement made it harder to advance really outrageous
projects. Were it not for that requirement, you would have had more
Tennessee-Tombigbees. In the case of FTA's cost-effectiveness analysis,
projects that have actually moved toward substantial construction funding
are the ones that are not laughable on the cost-effectiveness test. In
that sense, it has done a service.
I would argue for making the benefit-cost analysis simple and
understandable rather than too complex. This is where Dr. Lewis and I
might part company a bit. I was involved in the discussions about
requiring cost-effectiveness analysis in the federal transit program.
Transit capital program earmarking really started later-after President
Reagan was elected and then OMB Director David Stockman was zeroing out
transit rail New Starts. Congress responded by appropriating the money.
DOT reacted, in part, by trying not to spend the money. Congress then
responded by earmarking. Congress was saying that if you are not going to
spend what we have approved, we are going to tell you exactly what to
spend.
At this point, DOT realized it had lost control and that the end result
would not be good for the program's health. In changing the debate to get
projects of higher public value implemented, DOT faced the dilemma that we
face as benefit-cost practitioners-being either precisely wrong or
approximately right. It decided to calculate cost-effectiveness using
dollars per new rider attracted-not a full benefit-cost analysis. Instead,
DOT said if the purpose of New Starts is to get new rail riders, then you
must calculate cost effectiveness in doing so. This approach required
analytic police because everybody now was cooking the forecasts. Still, my
impression was that FTA recognized that this was not the only test-it was
just one of several tests. If you had an embarrassingly low or high dollar
cost per new rider, it put a greater burden on you to argue that your
project ought to be earmarked in Congress. In that sense, I think it
worked.
In this case, DOT did not pretend that this crude cost-effectiveness index
or benefit-cost analysis should be the sole determinant of which projects
the federal government funded. It was recognized as only part of the
debate. However, it did end up shaping the argument in a way that was
useful to help stop or control the most wasteful projects. Over the years,
the value of that requirement eroded. It may have been changed to time
measures-dollars per hour or minutes saved, in response to big cities that
did not want New Starts, but wanted to improve or renovate existing
service. Slowly, its power got lost. Still, it was a useful requirement.
My guess is that requiring a benefit-cost analysis or something like FTA's
New Starts cost-effectiveness measure would help shape the debate in ways
that Dr. Lewis wants and not be a complete waste of time. However,
insisting that benefit-cost analysis determine local or federal decisions
is asking for a lot of trouble.
Panel Discussion
DR. WACHS: I think we can learn something relevant to Dr. Gomez-Ibanez's
statement by looking at the history of NEPA. When NEPA was debated, one
proposal was to require the environmental impact review to be done by an
independent agency, rather than the agency preparing the project. The
genius of the decision to give the agency that prepared the project this
responsibility was that it infused these agencies with an environmental
capacity that they would not have had if this review had been done by an
independent agency. In addition, the decision embedded the environmental
impact requirement in a process that forced democratic debate. The draft
environmental analysis is subject to review, and must be presented at a
hearing for public comments-and in turn, the comments must be responded to
by the agency. I would take Dr. Gomez-Ibanez's recommendation and add that
if transportation agencies had to prepare benefit-cost analyses, they
would become more expert.
Such a requirement probably should be embedded in a public process that
subjects benefit-cost calculations to public comment and review.
DR. GOMEZ-IBANEZ: Makes sense.
"...these large infrastructure investment decisions are inherently
political decisions. They should remain so, although I would advocate a
stronger role for some systematic analysis to inform and organize the
discussion that accompanies that political decision process."
-Dr. Pickrell
DR. SMALL: One question that we did not get to in the last session was
ensuring the objectivity of benefit-cost analysis. I believe that Dr.
Pickrell's article about transit systems and their forecasts83 suggested
peer review-just as a simple requirement that is not very onerous and
shines a light on what is done. That seems to fit right into keeping a
requirement from becoming a total game-playing exercise.
DR. PICKRELL: I now have a different perspective on the peer review
requirement and would not make the recommendation again. It may have been
naive, given the success that project sponsors have had in gaming the
process by loading peer review boards with people selected from a cadre of
supporters. There is a bit of log rolling or back scratching. However,
peer review probably has made things marginally better- not worse.
Regarding a potential requirement for benefit-cost analysis in public
decisions, I would observe that these large infrastructure investment
decisions are inherently political decisions. They should remain so,
although I would advocate a stronger role for some systematic analysis to
inform and organize the discussion that accompanies that political
decision process. In the current funding environment, I believe that local
officials who make these decisions simply are incapable of seeing the
benefits and costs accompanying their decision alternatives from the broad
perspective that is required to accomplish all the objectives we have been
talking about today-avoiding multiple counts of impacts, considering
distribution and downstream impacts, and thinking carefully about which
impacts genuinely add to a project's benefits versus simply shifting
benefits around.
The current federal structure and funding for transportation
infrastructure investment provides no incentive for local political
officials to engage in the systematic, forthright analysis that we are
recommending. Recent integrated funding legislation for transit and
highway programs has helped the situation a bit by moving slightly toward
a unified funding process for transportation. Superficially, it looks as
if we have a unified transportation funding structure at the federal
level. But
there is still pretty extreme compartmentalization of the individual
funding programs under the umbrella authorization legislation.
Firewalls among the funding programs remain relatively strong and tend to
be flexible-where they are at all-in only one direction.84 In this funding
environment, there is simply not much incentive for local officials to
engage in the kind of analysis that most of us seem to be advocating.
My guess is until an infrastructure investment decision imposes an
opportunity cost on the officials making it by drawing down on their
ability to make competing investments, any requirement
"The whole federal transportation program is currently structured
precisely to avoid trade-offs between modes and different sorts of
investments."
-Dr. Taylor
to conduct systematic analysis will be a pro forma activity in somewhat
the same way that the EIS has become-checking off boxes and listing
mitigation measures. Responding to public comments on an EIS can become a
pro forma process as well. Inevitably, it is a process of going down the
list and making sure that you have said something in response to each of
the 43,000 letters that have been elicited from property owners along the
northeast corridor rail line or something like that.
Until the federal funding environment changes, any requirement to perform
analysis will either be treated in a pro forma way, or worse, gamed in the
way that Dr. Gomez-Ibanez was describing. A requirement will create the
need for a policing agency. OMB seems to serve quite enthusiastically and
capably in that capacity. And this does improve the analysis-there is no
question. But I do not think you want the threat of having to negotiate
the completeness and forthrightness of your analysis with OMB being the
enforcement mechanism. I think you want to provide some incentive for
project sponsors to willingly engage in the kind of process that we are
describing.
DR. TAYLOR: I want to go outside the moderator's role to say that I
completely agree with your point. Internalizing these opportunity costs is
really key. The whole federal transportation program is currently
structured precisely to avoid trade-offs between modes and different sorts
of investments. There is a lot of uncertainty in that process and this
creates considerable transaction costs for the people involved.85 As a
result, the policy makers involved in federal transportation funding
decisions negotiate politically on how to divide the pie. Then the federal
funding for different transportation programs is established. Regions and
states that compete for federal money or projects are simply engaging in a
gaming process. There are zero opportunity costs involved in doing so.
Trade-offs can never be internalized, despite enforcement or regulatory
mechanisms, as long as there are those strict divisions of transportation
funding programs. The goal is always to maximize funds that can be
accessed from each funding source.
A real key is involving those who are trying to divide up the pie in
project level decisions-this is where the honesty is going to come in. It
is a pretty fair process when different interests compete for dollars and
it puts a lot more rigor in the analyses. For example, California's
Transportation Development Act essentially ensures that funds in smaller
counties go first to transit. If all reasonable transit needs have been
met, the money can be used for streets and roads. Transit agencies argue
that they need to spend this money, and county road departments challenge
their analyses. This is much different from situations in which entities
in the same categorical area are competing. However, federal funding
programs are designed specifically to avoid just those kinds of choices-I
think because of a lack of trust.
DR. GOMEZ-IBANEZ: I have heard for ages that we would be better off with
consolidated transportation programs to allow trade-offs, etc. Richard
Nixon tried it with categorical grants and he did not get far. I do agree
that it would make the incentives very different. But it is a counsel of
despair to say that, absent consolidation, there is not much we can do.
This does not recognize that there are other allies for different points
of view-particularly at the local level. In almost any local project, you
will find people who are worried that the transit line is not going to the
right section of the metropolitan area or taxpayers are going to have a
white elephant on their hands. My guess is there are natural local allies
on the other side to fuel debate. Even in the transportation agency that
is advocating a project, there may be people who will tell a city council
member that benefits will not exceed the costs of light rail for the city.
There probably were COE people who knew that water systems were scarce
resources and were glad to have the benefit-cost test to protect them from
the worst projects being pressed on the Army.
DR. KIRBY: I agree that discipline has to come from competing interests at
the local level. Absent a complete restructuring of the federal
transportation program-which is unlikely-we must do better with the
process we have. MPO and EIS requirements really bind us, particularly
conformity requirements with regard to model structure and latest data. We
struggle to meet conformity requirements because no one wants to be
responsible for the whole regional plan being disapproved when it goes to
DOT and federal money held up-which is the prospect if we do not do
everything right. Similarly, you do not ignore EIS environmental
requirements or you will be in court and your project will stop. So to
some degree, those things work.
Although the conformity rule is very well specified-we know what we have
to do and we had better do it-other requirements are not. In particular,
MPOs need much clearer, better specified descriptions of what FTA is
requiring them to do. It is difficult to find a piece of paper that
describes FTA's requirements for technical procedures and the requirements
seem to change from time to time. This is a process that has not been
fully specified and is difficult to follow in practice. There could be the
need for a more prescriptive requirement from Congress through DOT.
There is great potential to really improve our processes and the value of
what we are doing. Knowing clearly what you must do from the start would
provide discipline and head off a lot of problems. And local level
competition plays a role. You are only going to get so much funding, and
you do not want to put all your effort into a project
that is not going to make it through the approval process. Local interests
recognize this and will propose working on projects that have the best
chances of getting through the process.
DR. LEWIS: There are two ways to interpret congressional interest in
requesting this study.86 Congress can be asking what can or should be done
to improve the role of benefit-cost analysis in improving decision making.
If that is the question, I buy into what Dr. Gomez-Ibanez and Dr. Kirby
have just said. However, the question may be what can Congress do to help
improve things-what Dr. Gomez-Ibanez and Dr. Kirby are saying suggests
actions to introduce more local level resource trade-off analysis.
One thing that Congress-particularly the appropriators-can do is to
encourage both FHWA and FTA to look at more than how to spend fixed
allocations. I think that appropriators want to know if a $1.1 billion New
Starts budget is well spent compared to alternatives, such as highway
expenditures, no further expenditures, or noncapital projects.
One thing that we have not dealt with today, although Dr. Forkenbrock
touched on it, is the role of benefit-cost analysis in looking at
noncapital solutions. If appropriators want to encourage more
comprehensive appraisal of how New Start projects and highway proposals
line up against transit alternatives, I would hope that that could be a
requirement-notwithstanding the fact that an enforcement modality is never
the best incentive. Then I would agree that the apparatus and ingredients
for doing that are in place-it is the incentive for doing it that is
lacking.
Summary of Panel Responses to Audience Questions
Do the current federal transportation financing and funds distribution
structures suboptimize the federal dollar, since the rate of return for
the federal transportation dollar keeps going down?
o Dr. Lewis said that while he would not counsel using benefit-cost
analysis as a wedge to restructure the entire financial and budget
framework, some small steps would help, including (1) Congress
legislatively declaring its interest in an economically rational
allocation and information to address this allocation and (2) Congress
insisting that modal agencies conduct benefit-cost analysis from a
multimodal perspective. He added that motivation for efficiency can
also come from outside the federal financing framework. As an example,
he said that in one location, there was interest in building a light
rail system because federal dollars were available, but that
industrial and commercial executives lobbied to use a benefit-cost
framework first to determine whether a light rail system or widening
highways made more sense.
o Dr. Meyer said that what is optimal to one person may not be to
another. As an example, he said that building an interstate highway
network benefited some states more than others, but that all states
agreed to build that national network. When the interstate system was
complete, many state governments wanted to deal with their own
problems rather than sacrificing so that others could get the national
network in place. While this may be suboptimal to some from a national
perspective, others would say that states have provided their
contribution and now need to spend money on their states' problems.
o Dr. Forkenbrock added that the argument for states to get their money
back for use on local priorities is gaining strength and that many
people are arguing against having a federal program at all, bringing
into question FHWA's future.
o Dr. Taylor added that a recent dissertation showed that the federal
highway program generally was redistributing funds from those with
less financial capacity to those with more financial capacity.
What practical advice would you give on structuring an ex post analysis87
of a transportation project when the analysis at the beginning of the
project was not necessarily done for comparison purposes? The Senate
reauthorization bill88 requires some type of New Starts ex post analysis
that focuses on ridership, and FTA is trying something similar. Are there
other benefits that are more appropriate to analyze?
o Dr. Forkenbrock recommended looking at where ridership comes from, as
rail ridership often is gained at the expense of bus ridership.
o Dr. Pickrell said that he thought either substantial travel time
savings or ridership increases were the primary benefits to measure.
He added that ex post evaluation is better than anticipatory
evaluation because in anticipatory evaluation you have to simulate the
results of two possible decisions; whereas in ex post evaluation at
least you know the results of one decision-that of building the
project-and have to simulate only what would have happened if the
project had not been built.
o Dr. Small said that one need not be deterred if no benefit-cost study
was done before the project was implemented, as changes during a
project's construction often make it difficult to compare studies done
before and after a project in any case.89
Would clearly specified standards for conducting benefit-cost analyses
help increase the impacts of these analyses on policy decisions?
o Dr. Kirby said that adding requirements to the process would improve
decisions. He also said that Dr. Lewis' previous comment about a
business community's interest in a benefit-cost analysis of transit
lines showed that involving more people can create a constituency for
good analysis, making it more likely that tough questions will be
asked and that the focus will be on projects that are good for the
community. He cautioned that bad decisions can result when the agency
building a project works closely with the agency funding the project.
A larger constituency asks tough questions and focuses on projects
that are good for the community.
o Dr. Meyer agreed that a constituency for analysis was important and
said that adding requirements would increase the constituency for good
analysis.
Appendix VII: Panelists' Closing Remarks
DR. TAYLOR: What remains on our schedule is for me to offer a 5minute
summary of what has been 6-1/2 hours of discussion. I have 17 pages of
notes and 444 points that were made. Instead, I will ask each panelist to
repeat what he thought was the most important thing for people to carry
away from all of this.
DR. PICKRELL: I have been encouraging you to think about how to revamp the
funding structure for transportation infrastructure programs to encourage
investors-public agencies-to get benefits and costs on the correct side of
the ledger. If you could do that one thing, you will have accomplished
more than I will have in my entire career.
DR. SMALL: One of the strengths of benefit-cost analysis is that it offers
a unifying principle or consistent measure that can cut across many of the
questions that we have discussed today-secondary benefits and so on. If
you consistently keep in mind the key principle that the benefits we are
trying to measure are people's willingness to pay for things and the costs
are what people would be willing to pay to avoid, you can answer many
conundrums about double counting and externalities.
DR. KIRBY: A major feature of the federal transportation program is the
process you go through to use federal funds, for example, the
environmental impact statement and planning process. Many people value
that process very highly, and I would consider that to be one reason for
the federal government to be funding transportation. If you can strengthen
the benefit-cost requirement in that process through something analogous
to the present conformity rule and environmental impact statement process
and get more people involved in looking at these projects, you will get
better projects and have a stronger rationale for federal involvement in
transportation.
DR. GOMEZ-IBANEZ: The reading that most struck me was Dr. Lewis's piece
about requiring benefit-cost analysis-not in the false hope that it might
be the answer and determine everything, but with the idea that it would
stimulate a better debate and wiser decisions. Since Dr. Kirby says that
we require everything short of a benefit-cost analysis already, it seems a
small step to require that of federally-funded projects.
DR. LEWIS: The main point I want to leave with you is the importance of
comparing major capital projects across modes. I think we have the tools,
the apparatus, and-even with huge imperfections-the models and the data.
What we lack is a demand from Congress and from the broader constituency
for good analysis for information about returns on investment among
transit versus highway alternatives. Benefit-cost analysis makes this
possible.
DR. FORKENBROCK: I agree. I would extend it to my great concern from
watching MPOs, state transportation agencies, and other transportation
agencies that many people who do benefit-cost analyses are not doing them
very well. Perhaps one reason is that they have never really been given
the opportunity to learn the rudiments about costs. There is this
tremendous gap between the state of the art in benefit-cost analysis and
the state of the practice. Anything that GAO can do to help MPOs, state
DOTs, and other agencies to do better benefit-cost analyses would be a
huge step forward. If benefit-cost analyses are not being done correctly,
you run the risk of creating misinformation or-even worse-disinformation
for decision makers. That leads to bad investment decisions.
DR. MEYER: Much of our discussion has revolved around the question of how
to enhance the role of benefit-cost analysis in decision making- and we
all agree that decision making is inherently a political process. I think
you do this through a requirement that makes benefit-cost analysis part of
the process for looking at federally supported transportation investments.
This is not to say that decision makers will necessarily do what
benefit-cost analysis indicates, but it brings more information to the
table that will become much more useful for that decision-making process
over time.
DR. WACHS: I have three points in closing.
* First, benefit-cost analysis and the political nature of our
decision making about large public works projects, including
transportation projects, is a constant struggle to find the right
balance between subjective judgments that our elected officials can
make very effectively and being informed by the rational analysis
that benefit-cost analysis contributes to the process. We will
probably not be very effective if one of those two strains
completely dominates the other. What we are trying to find is a
balance. We are trying to inform the political decision making with
good information. To me, that requires attention to the process in
which the benefit-cost analysis is embedded as well as to the
tools, techniques, and data. I would be really careful to ask
questions about public hearings, about process, about the order in
which analysis and debate take place and about feedback
* from the analysis to the political process. It is possible to have
a very good benefit-cost analysis that is completely politicized,
and the goal would be to have an informed political process in
which the analysis is reasonably objective. That is a process
design problem more than it is dealing with the subtleties of
benefit-cost analysis itself.
o Second, I heard that the quality of data, data collection and
analysis, and tools and techniques of benefit-cost analysis are very,
very important.
o Third, this is a very homogenous group. It might be useful to have
other constituencies address the same questions that we did-
environmentalists, shippers, state highway officials who were not
terribly well represented in this group. You might find some other
answers that would complement our perspectives.
DR. TAYLOR: I am hearing some consensus that the decision making process
can only be improved by incorporating a federal requirement for
benefit-cost analysis. However, it is important to incorporate such a
requirement into a deliberative process. The debate we had about the idea
of a checklist and the danger of developing requirements that become like
a bureaucratic checklist is really important. A bureaucratic checklist
probably is the worst alternative we could envision. Any federal
requirement needs to be crafted to ensure that benefit-cost analysis
informs decision making as opposed to complying with requirements.
Appendix VIII: Selected Biblography and Related GAO Products
Related Articles Suggested by Panelists
Brian Taylor
Hill, M.C., B. Taylor, A. Weinstein, and M. Wachs, "Assessing the Need for
Highways," Transportation Quarterly, Spring, 2000, pp. 93-100.
Taylor, B. and K. Samples, "Jobs, Jobs, Jobs: Political Perceptions,
Economic Reality, and Capital Bias in U.S. Transit Subsidy Policy," Public
Works and Management, Vol. 6, April 2002, pp. 250-263.
David Forkenbrock
Forkenbrock, D.J. "Some Technical Issues in Benefit-Cost Analysis of
Potential Transportation Investments," July 2004.
Forkenbrock, D.J. and Norman S.J. Foster, "Economic Benefits of a Corridor
Highway Investment," Transportation Research-A, Vol. 24A, No. 4, 1990, pp.
303-312.
Michael Meyer
T. Partridge & Associates Ltd, "Translink Transportation Evaluation
Guidelines," October 11, 1999.
Kenneth Small
Small, K.A., "Project Evaluation" in Essays in Transportation Economics
and Policy-A Handbook in Honor of John E. Meyer
(Jose Gomez-lbanez, W. B. Tye, and C. Winston, eds.), Brookings
Institution Press, Washington. D.C: 1999.
Brownstone D. and K.A. Small, "Valuing Time and Reliability: Assessing the
Evidence from Road Pricing Demonstrations." Abstract, October 9, 2003.
David Lewis
Lewis, D., "Making Cost-Benefit Analysis More Useful in Policy Making" in
The Reform of Cost-Benefit Analysis by David Lewis, forthcoming from
Ashgate Press.
Lewis, D., "The Case for Increasing Transit Capacity to Mitigate
Congestion," presented at Conference on Traffic Congestion: Issues and
Opportunities, UCLA Extension Public Policy Program and Institute of
Transportation Studies. Washington, D.C.: June 26-27, 2003.
Don Pickrell
Mohring, H., "Maximizing, Measuring, and Not Double Counting
Transportation Improvement Benefits: A Primer on Closed- and Open-Economy
Cost-Benefit Analysis," Transportation Research-B, Vol. 27B, No.6, 1993,
pp. 413-424.
U.S.
DOT, Office of the Secretary, "Treatment of Value of Life and
Injuries in Preparing Economic Evaluations," January 8, 1993 and
revised January 29, 2002.
U.S.
DOT, Office of the Secretary, "Departmental Guidance for the
Valuation of Travel Time in Economic Analysis," April 9, 1997 and
revised February 11, 2003.
Cohen, J.P. and C.C. Coughlin, "Congestion at Airports: The Economics of
Airport Expansions," The Federal Reserve Bank of St. Louis, May/ June
2003.
Related GAO Products
Federal-Aid Highways: FHWA Needs a Comprehensive Approach to Improving
Project Oversight, GAO-05-173 (Washington, D.C.: Jan. 31, 2005)
Highway and Transit Investments: Options for Improving Information on
Projects' Benefits and Costs and Increasing Accountability for Results,
GAO-05-172 (Washington, D.C.: Jan. 24, 2005)
Federal-Aid Highways: Trends, Effect on State Spending, and Options for
Future Program Design, GAO-04-802 (Washington, D.C.: Aug. 31, 2004)
Surface Transportation: Many Factors affect Investment Decisions,
GAO-04-744 (Washington, D.C.: June 30, 2004)
Mass Transit: FTA Needs to Better Define and Assess Imp[act of Certain
Policies on New Starts Program, GAO-04-748 (Washington, D.C.: June 25,
2004)
Freight Transportation: Strategies Needed to Address Planning and
Financing Limitations, GAO-04-165 (Washington, D.C.: Dec. 19, 2003).
Appendix IX: GAO Contacts and Acknowledgements
GAO Contacts
o Katherine Siggerud (202) 512-2834
o Nikki Clowers (202) 512-4010
Acknowledgments
In addition to those named above, Jay Cherlow, Libby Halperin, Sara Ann
Moessbauer, Andrew Von Ah, and Alwynne Wilbur made key contributions to
this report.
Endnotes
1National Association of State Budget Officers. 2003 State Expenditure
Report, 2004 (Washington, D.C.).
2U.S. Department of Transportation, 2002 Status of the Nation's Highways,
Bridges, and Transit: Conditions and Performance, Report to Congress
(Washington, D.C.: 2002). Estimates are in 2000 dollars.
3ISTEA (P.L. 102-240) authorized federal transportation programs for
highways, highway safety, and transit for fiscal years 1992 to 1997.
4TEA-21 (P.L. 105-178) authorized federal surface transportation programs
for highways, highway safety, and transit for fiscal years 1998 to 2003.
The act has since been extended.
5GAO, Highway and Transit Investments: Options for Improving Information
on Projects' Benefits and Costs and Increasing Accountability for Results,
GAO-05-172 (Washington, D.C.: Jan. 24, 2005).
6GAO, Surface Transportation: Many Factors Affect Investment Decisions,
GAO-04-744 (Washington, D.C.: June 30, 2004).
7For more information about the use of benefit-cost analysis in
transportation investment decision making, see GAO-05-172.
8NEPA (P.L. 91-190) declares that it is national policy to use all
practicable means to create and maintain conditions in which man and
nature can exist in productive harmony, among other purposes. It requires
federal agencies to integrate environmental values into their decisions by
considering the environmental impacts of their proposed actions and
reasonable alternatives to those actions.
9Earmarking refers to dedicating appropriations for a particular purpose.
10HOV lanes promote ridesharing by providing dedicated lanes on a highway
for buses, vanpools, and carpools. Bus Rapid Transit is designed to
provide major improvements in the speed, reliability, and quality of bus
service through barrier-separated busways, high-occupancy vehicle lanes,
or reserved buses or other enhancements on arterial streets. For more
information about bus rapid transit, see GAO, Mass Transit: Bus Rapid
Transit Shows Promise, GAO-01-984 (Washington, D.C.: Sept. 17, 2001).
11ROI is one of several approaches that decision makers use to evaluate
the investment potential of projects or actions. ROI is a ratio that
compares the net benefits of a project to its total costs.
12Professor Sen was awarded the Nobel Prize in Economic Sciences in 1998
for his contributions to welfare economics. See Amartya Sen, "The
Possibility of Social Choice," Lecture Delivered in Stockholm, Sweden, on
December 8, 1998.
13Welfare economics studies how efficiently an economy distributes income
and the consequences that are associated with it. It is concerned with the
welfare of individuals-rather than groups or societies.
14David J. Forkenbrock and Norman S. J. Foster, "Economic Benefits of A
Corridor Highway Investment," Transportation Research-A. No. 4, (1990),
303-312.
15David Lewis, "Making Cost-Benefit Analysis More Useful in Policy
Making." Paper (June 22, 2004) drawn from the manuscript of Dr. Lewis's
forthcoming book from Ashgate Press.
16Smart Growth seeks to accommodate development by focusing on
environmentally sensitive land development with the goals of minimizing
dependence on auto transportation, reducing air pollution, and making
infrastructure investments more efficient.
17NEPA requires federal agencies to integrate environmental values into
their decision making by considering the environmental impacts of their
proposed actions and reasonable alternatives to those actions. To meet
this requirement, agencies prepare a detailed statement known as an EIS
for all major federal actions that could have significant effects on the
environment.
18Agglomeration refers to the tendency of firms to cluster close to each
other, or to residences, presumably due to advantages this gives them in
production.
19Externalities are direct spillover effects on third parties that result
from production and/or consumption of goods and services for which no
appropriate compensation is paid.
20 Dr. Small noted that he owed this answer to a discussion with Anthony
Downs and Katharine Bradbury. Downs is a Senior Fellow at the Brookings
Institution in Washington, D.C. He is the author or co-author of numerous
books, including "Still Stuck in Traffic: Coping with Peak-Hours Traffic
Congestion," Brookings Institution Press. Washington, D. C., 2004.
Katharine Bradbury is a Vice President and Economist at the Federal
Reserve Bank of Boston. Her research focuses on the regional economy and
state and local public finance and income inequality.
21Dr. Forkenbrock mentioned a study by a number of Midwestern states:
Transportation Economics & Management Systems, Inc., in association with
HNTB Corporation, Midwest Regional Rail System Executive Report, September
2004. Prepared for the Illinois, Indiana, Iowa, Michigan, Minnesota,
Missouri, and Wisconsin Departments of Transportation, the Nebraska
Department of Roads, and the Ohio Rail Development Commission.
22Dr. Pickrell recommended the following paper: Herbert Mohring and Harold
F. Williamson, Jr., "Scale and `Industrial Reorganization' Economies of
Transport Improvements," Journal of Transport Economics and Policy 3.
September 1969, pp. 251-271.
23GDP is the total value of all goods and services produced within a
territory during a specified period.
24Guidance from the U.S. Department of Transportation/Office of the
Secretary, "Treatment of Value of Life and Injuries in Preparing Economic
Evaluations," January 8, 1993; Revised on January 29, 2002.
25Indirect impacts include changes in land use and development, changes in
decisions to locate homes and businesses, etc., and changes in operations
that businesses make to take advantage of improved transportation system
speed and reliability. These impacts lead to increased property values,
increased productivity, employment, and economic growth. There also can be
indirect costs, such as reduced land values for regions that might lose
economic activity that is diverted to an area where transportation is
improved.
26Multiplier refers to the cumulatively reinforcing interaction between
consumption and production that amplifies changes in investment,
government spending, or exports. Multipliers are called estimators of the
"ripple effect" in an area.
27REMI Policy Insight is the economic forecasting and policy analysis
model of Regional Economic Models, Inc.
28Disutility of time refers to the value that people place on reducing
time spent in a particular activity. To the extent that travel time is
less pleasant than time spent in other ways, people may be willing to pay
to reduce their travel time. Therefore, travel time can be said to have
disutility. See Jay R. Cherlow, "Measuring Values of Travel Time Savings,"
Journal of Consumer Research Vol. 7 (March 1981).
29Ian Savage, "An Empirical Investigation into the Effect of Psychological
Perceptions on the Willingness-to-Pay to Reduce Risk," Journal of Risk and
Uncertainty 6(1): pages 75-90, 1993.
30Opportunity cost is the value of the best alternative given up when
making a choice.
32Probabilistic analysis allows for the range of all possible outcomes
rather than the single most likely outcome.
33Bayesian logic applies to decision making and inferential statistics
that deal with probability inference- using the knowledge of prior events
to predict future events. The way to quantify a situation with an
uncertain outcome is to determine its probability.
34Positive economics deals with objective, relatively testable statements
that focus on descriptions that do not reflect obvious value judgments and
predictions about economic relationships. Normative economics deals with
subjective statements based on opinion about "what ought to be." 35A
microsimulation model is a tool used to evaluate the impact of changes on
a system. For example, such an approach to travel demand forecasting might
integrate household activities, land use distributions, regional
demographics, and the transportation network to estimate the impacts of
converting a highway intersection into a cloverleaf.
36The Four-Step Urban Transportation Planning Process integrates various
aspects of travel behavior (trip or mode choices) with information on land
use patterns and the transportation network. The four steps include: trip
generation, trip distribution, mode choice, and assignment to specific
parts of the transportation network.
37Frank Ackerman and Lisa Heinzerling, Priceless: On Knowing the Price of
Everything and the Value of Nothing. The New Press, 2004.
38Joseph Berechman, "TransportationcEconomic Aspects of Roman Highway
Development: The Case of Via Appia." Transportation Research Part A, Vol.
37 (2003), pp. 453-478.
39Distributional issues refer to who gets what and how much they get of
benefits or items, such as money, land, transportation facilities,
services, etc.
40HOT lanes are limited access, barrier-separated highway lanes that
provide free or reduced cost access to qualifying high-occupancy vehicles
and also provide access to other paying vehicles that do not meet
passenger occupancy requirements.
41D. Brownstone and K.A. Small, "Valuing Time and Reliability: Assessing
the Evidence from Road Pricing Demonstrations," Working Paper. Oct. 2003.
42John Rawls, A Theory of Justice, 1971, (Revised edition, 1999). Belknap
Press.
43The Americans with Disabilities Act of 1990 (P.L. 101-336) prohibits
discrimination of various sorts against persons with physical or mental
handicaps. It emphasizes employment and outlaws most physical barriers.
44A Pareto improvement is based on the notion that an action improves
efficiency if it is possible for one person to benefit without anyone else
being harmed. A Pareto improvement is possible if the economy has idle
resources or market failures that can be corrected without hurting others.
45Disbenefits also can be described as disadvantages.
46Induced demand is the phenomenon that more of a good/service is consumed
after the supply increases.
47A discursive benefit-cost exercise is the conduct of a benefit-cost
analysis within a formal public process that engages stakeholders in the
analysis process.
48The Federal Highway Administration uses the HERS model to estimate
future investment requirements of the nation's highway system for
Congress. The HERS-ST model is a version of the HERS model that allows
states to simulate future highway conditions and performance levels and
identifies deficiencies using engineering principles. It then simulates
the selection of improvement projects, applying economic criteria to
estimate the most cost-beneficial mix of improvements for system-wide
implementation.
49David Forkenbrock, "Improving Benefit-Cost Analysis" (Washington, D.C.:
2004. Photocopy).
50Section 1105 of ISTEA contained corridor provisions and identified 21
corridors of significance. This identification allowed states to give
funding priority to these corridors, provided federal funding to specific
projects on these corridors, and directed other benefits to these
corridors. When TEA-21 was enacted, there were 29 corridors, which
increased to 44 by 2002.
51Discounting is the process of comparing current and future values that
finds the present worth of a future amount of money.
52MPOs are regional transportation policy bodies made up of
representatives from various government and other organizations. The
Federal-Aid Highway Act of 1970 required that such agencies be developed
in areas with populations of more than 50,000 to carry out cooperative
planning at the metropolitan level. MPOs are responsible for planning,
programming, and coordinating federal highway and transit investments in
urbanized areas.
53The Kaldor-Hicks criterion holds that for a change in policy to be
viewed as beneficial, the gainers should be able to compensate the losers
and still be better off. The criterion does not require that compensation
actually be paid.
54Distributional effects also are described in OMB Circular No. A-94,
Office of Management and Budget- Guidelines and Discount Rates for
Benefit-Cost Analysis of Federal Programs. (Washington D.C.; 1992).
55Parameter values are assumed or estimated values of key components of an
analysis (such as a statistical life saved). These values are used with
observations on key explanatory variables (such as the number of lives
saved) to estimate the value of the variable being studied (such as
benefits).
56In general, the economists' theory of the second best says that the
application of policy rules that are designed to enhance net economic
welfare-such as benefit-cost analysis for public investment decisions-
might not enhance economic welfare if elsewhere in the economy there are
distortions from economic efficiency. In this context, the second best
issue is that the demand for travel on a new road would be much less if
users had to pay the full cost that their additional travel causes than
if, as is true under current road pricing policies, users pay only part of
the cost. Hence, the benefit-cost ratio of an investment decision could be
quite different if the distortions introduced by the fact that individuals
and businesses do not take into account the full benefits and costs of
making their decisions are removed.
57American Association of State Highway and Transportation Officials, A
Manual of User Benefit Analysis for Highways, 2003.
58Sensitivity analyses assess how sensitive outcomes are to changes in
assumptions. The assumptions that deserve the most attention should depend
largely on the major benefits and costs and areas of greatest uncertainty
in the program or process that is analyzed.
59Highway cost allocation studies are used to evaluate the highway-related
costs attributable to different vehicle classes and the user fees they
pay. Comparing user fees and cost responsibility indicates the relative
equity of highway user fees.
60The CAAA were passed in 1990 (P.L. 101-549). The CAAA's conformity
requirements specify that MPOs and the U.S. DOT determine that
transportation plans and programs in areas that do not meet federal air
quality standards set by the National Ambient Air Quality Standards move
toward reducing pollutant emissions to meet these standards.
61MOBILE6 Vehicle Emission Modeling Software is a model for predicting
gram per mile emission of hydrocarbons, carbon monoxide, nitrogen oxides,
carbon dioxide, particulate matter, and other toxics.
63GPS is a system of satellites, computers, and receivers that is able to
determine the latitude and longitude of a receiver on earth.
64TRB's Committee for Review of Travel Demand Modeling conducted by the
Metropolitan Washington Council of Governments (Sept. 8, 2003/ first
report).
65The purpose of the United States Visitor and Immigrant Status Indicator
Technology Program is to improve border management at ports of entry by
capturing more complete arrive/departure data for those who require visas
to enter the United States.
66David A. Aschauer is the Elmer W. Campbell Professor at Bates College.
His primary research field is macroeconomics. He is currently conducting
research on the relationship between fiscal and monetary policy in the
U.S. as well as the relationships between real interest rates and real
exchange rates in an open-economy setting.
67TRB Special Report 264, The Congestion Mitigation and Air Quality
Improvement Program: Assessing 10 Years of Experience, April 2002.
Washington, D.C
68TANF is a block grant program to help move recipients into work. The
Office of Family Assistance in the
U.S. Department of Health and Human Services administers this program.
69Alan Altshuler and David Luberoff, Mega-Projects: The Changing Politics
of Urban Public Investment, Lincoln Institute of Land Policy, 2003.
70FTA's New Starts program helps pay for designing and constructing
certain rail, bus, and trolley projects.
71The National Cooperative Highway Research Program (NCHRP) is
administered by TRB and sponsored by the member departments (individual
state departments of transportation) of AASHTO in cooperation with FHWA.
Support is voluntary and funds are drawn from the states' Federal-Aid
Highway apportionment of State Planning and Research funds. The Transit
Cooperative Research Program (TCRP) functions under the direction of the
Federal Transit Administration, the National Academies (acting through the
Transportation Research Board) and a nonprofit research/education
organization established by the American Public Transportation
Association.
72 Revealed preference studies use data from real consumer behavior to
obtain information about demand, values, etc. Stated preference studies
employ data from a particular kind of consumer survey.
73Elicitation is the process of obtaining knowledge from experts or
persons to be used in decision making or producing a design.
74In the Bayesian approach, the probability of an event is a person's
degree of belief that the event will occur, given all the relevant
information currently known to that person. Thus, the probability is a
function of both the event and the state of information.
75Central tendency is the average outcome of any given phenomenon (travel
time, etc.) as distinct from the range of other outcomes-both above and
below the average-that occur.
76Cross elasticity of demand is the percentage of change in quantity
demanded in response to a 1 percent change in the price of another good.
77A Monte Carlo simulation is a problem-solving and investigation
technique used to approximate the probability of certain outcomes by
running multiple trial runs (simulations) using random variables.
78A smart card is a plastic card with a built-in microprocessor and memory
that is used for identification or financial transactions.
79Sandra Rosenbloom is Professor of Planning and Director of the Roy P.
Drachman Institute for Land and Regional Development Studies at the
University of Arizona. Her work has centered on transportation problems of
poor people, working women with children, and the elderly.
80Michael Meyer, "Paper Distributed to Expert Panel" (Washington, D.C.,
2005. Photocopy).
81PPB, which was grounded in systems analysis, was the process of defining
objectives and designing alternative systems to achieve them.
82The Tennessee-Tombigbee Waterway is a canal located in northeast
Mississippi and west central Alabama.
83Pickrell, Donald H., "A Desire Named Streetcar: Fantasy and Fact in Rail
Transit Planning," Journal of the American Planning Association, 1992, pp.
158-76.
84TEA-21 established firewalls, or new budget categories, that ensured
that highway user fee revenues would be used for transportation programs.
85Transaction costs are the full costs of making an exchange.
86See GAO, Highway and Transit Investments: Options for Improving
Information on Projects' Benefits and Costs and Increasing Accountability
for Results, GAO-05-172 (Washington, D.C.: Jan. 24, 2005) for information
about the request.
87Evaluations can be conducted ex ante (before the intervention is
initiated or outcomes have been produced) or ex post (measures the
outcomes produced by the interventions to date).
88S. 1072, 108th Cong.(2004). This bill would have authorized funds for
federal-aid highways, highway safety programs, and transit programs and
for other purposes.
89As an example, Dr. Small said that several authors pulled together three
different studies of the Coquihalla Highway in British Columbia-one study
was done before it opened, one was done part way through construction, and
one was done when it was near completion. The numbers are phenomenally
different, but there is not an obvious bias in this case. Many things
changed along the way, including the scope of the project.
*** End of document. ***