Air Pollution: Limitations of EPA's Motor Vehicle Emissions Model and
Plans to Address Them (Letter Report, 09/15/97, GAO/RCED-97-210).
Pursuant to a congressional request, GAO reviewed the Environmental
Protection Agency's (EPA) MOBILE series of complex computer models to
estimate motor vehicle emissions, focusing on the model's major
limitations and EPA's process for improving the current and future
versions of the model.
GAO noted that: (1) EPA and a group of stakeholders have identified 14
major limitations in the current MOBILE model; (2) some vehicle
emissions-producing activities are not accounted for in the current
model, and other emissions-producing activities may not be adequately
represented on the basis of the most recent information; (3) according
to EPA, much of this information has become available since MOBILE5a was
released; (4) these limitations cause the model to underestimate vehicle
emissions in some cases and overestimate them in others; (5) other
studies indicate that some activities are inadequately represented in
the model; (6) another study indicates that carbon monoxide and
hydrocarbon emissions from higher mileage vehicles may be significantly
less than the model's estimates; (7) EPA plans to address most of these
limitations in its next revision to the MOBILE model, however, according
to agency officials, three of the limitations will probably not be
addressed until later because of a combination of factors; (8) according
to agency officials, these include the negligible impact on emissions
inventory predictions, a relatively low priority ascribed by EPA and
stakeholders, the cost and length of time required for these studies
relative to the schedule for release of MOBILE6, and the emergence of
new technologies that will make the improvements more feasible or cost
effective in a few years; (9) EPA officials pointed out that they have
updated the estimating capabilities of the MOBILE model 10 times since
it was first introduced in 1978; (10) irrespective of these limitations,
there are specific actions, most of which were recommended by the
Science Advisory Board in its 1989 resolution, that, when followed, can
enhance a model's estimating capabilities; (11) among other things,
these actions involve documenting the implicit and explicit assumptions
that are the basis of the formulas contained in the model, obtaining
external stakeholders' input during the model's development, and having
the model peer reviewed before it is used; (12) EPA officials
acknowledged that, primarily because of resource limitations, until
recently such actions have been delayed or forgone; (13) however, EPA is
developing the next model, MOBILE6, with significantly increased
openness and input from other stakeholders; and (14) EPA also plans to
carry out the actions recommended by the Science Advisory Board, such as
peer review, as part of its program for developing MOBILE6, due to be
issued in late 1998.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: RCED-97-210
TITLE: Air Pollution: Limitations of EPA's Motor Vehicle Emissions
Model and Plans to Address Them
DATE: 09/15/97
SUBJECT: Computer modeling
Motor vehicle pollution control
Air pollution control
Pollution monitoring
Projections
Statistical methods
IDENTIFIER: EPA MOBILE Model
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. Major **
** divisions and subdivisions of the text, such as Chapters, **
** Sections, and Appendixes, are identified by double and **
** single lines. The numbers on the right end of these lines **
** indicate the position of each of the subsections in the **
** document outline. These numbers do NOT correspond with the **
** page numbers of the printed product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
Cover
================================================================ COVER
Report to the Chairman, Subcommittee on Oversight and Investigations,
Committee on Commerce, House of Representatives
September 1997
AIR POLLUTION - LIMITATIONS OF
EPA'S MOTOR VEHICLE EMISSIONS
MODEL AND PLANS TO ADDRESS THEM
GAO/RCED-97-210
Air Pollution
(160369)
Abbreviations
=============================================================== ABBREV
API - American Petroleum Institute
AQIRP - Air Quality Improvement Research Program
CAA - Clean Air Act
CARB - California Air Resources Board
CO - carbon monoxide
CRC - Coordinating Research Council
EPA - Environmental Protection Agency
FACA - Federal Advisory Committee Act
FTP - Federal Test Procedure
GAO - General Accounting Office
HC - hydrocarbons
ISTEA - Intermodal Surface Transportation Efficiency Act
I&M - inspection and maintenance
MOBILE -
MOBILE4.1 -
MOBILE5 -
MOBILE5a -
MOBILE6 -
MOBILE7 -
OMS - Office of Mobile Sources
ORD - Office of Research and Development
NOx - nitrogen oxides
OBD - on-board diagnostics
RVP - reid vapor pressure
SAB - Science Advisory Board
SIP - state implementation plan
Letter
=============================================================== LETTER
B-277440
September 15, 1997
The Honorable Joe Barton
Chairman, Subcommittee on Oversight
and Investigations
Committee on Commerce
House of Representatives
Dear Mr. Chairman:
To reduce and control air pollution, the Clean Air Act requires that
the Environmental Protection Agency (EPA) establish national ambient
air quality standards and that the states develop strategies for
reaching and maintaining these standards. In order to evaluate the
states' strategies for reducing the atmospheric concentrations of
three pollutants emitted by motor vehicles--carbon monoxide,
hydrocarbons, and nitrogen
oxides\1 --EPA has developed a series of complex computer models,
known as the MOBILE series, to estimate motor vehicle emissions. The
current version is MOBILE5a. Both EPA and the states rely on this
model to estimate future emissions after various control strategies
are employed. Consequently, to the extent that the model erroneously
estimates emissions, EPA could approve control strategies that may
not be sufficient to attain the air quality standards, or,
conversely, EPA could require that the states implement additional,
often expensive control measures that may go beyond what is needed.\2
In view of the pivotal role that the MOBILE model plays in the
states' efforts to improve air quality, you asked us to describe the
major limitations in EPA's MOBILE model and EPA's process for
improving the current and future versions of the model.
--------------------
\1 Carbon monoxide (CO) impairs lung and heart functioning, and
hydrocarbons (HC) and nitrogen oxides (NOx) combine in sunlight to
form ozone, or smog, which has been linked to a variety of health
problems, ranging from eye, nose, and throat irritation to
bronchitis, emphysema, and other serious lung diseases.
\2 The states must obtain sufficient reductions in emissions or risk
Clean Air Act sanctions that could result in losing millions of
dollars in federal highway funds and jeopardizing industrial growth.
Motor vehicles are one of several categories of sources contributing
to air pollution.
RESULTS IN BRIEF
------------------------------------------------------------ Letter :1
EPA and a group of stakeholders have identified 14 major limitations
in the current MOBILE model.\3 Some vehicle emissions-producing
activities are not accounted for in the current model, and other
emissions-producing activities may not be adequately represented on
the basis of the most recent information. According to EPA, much of
this information has become available since MOBILE5a was released.
These limitations cause the model to underestimate vehicle emissions
in some cases and overestimate them in others. For example, the
impact of road grade--such as when a car climbs a hill--is not a part
of the current model, although some studies have indicated that both
the increased load on the engine from climbing a hill and the
decreased load that accompanies engine deceleration significantly
increase vehicle emissions. Similarly, other studies indicate that
some activities--such as when someone drives at high speeds,
accelerates quickly, runs the air conditioner, or uses a higher
mileage vehicle\4 --are inadequately represented in the model. For
example, some testing indicates that nitrogen oxide emissions from
running the air conditioner may be from 30 to 75 percent greater than
estimated by the MOBILE model. Conversely, another study indicates
that carbon monoxide and hydrocarbon emissions from higher mileage
vehicles may be significantly less than the model's estimates. EPA
plans to address most of these limitations in its next revision to
the MOBILE model; however, according to agency officials, three of
the limitations, including the impact of road grade, will probably
not be addressed until later because of a combination of factors.
According to agency officials, these include the negligible impact on
emissions inventory predictions, a relatively low priority ascribed
by EPA and stakeholders, the cost and length of time required for
these studies relative to the schedule for release of MOBILE6, and
the emergence of new technologies that will make the improvements
more feasible or cost effective in a few years. EPA officials
pointed out that they have updated the estimating capabilities of the
MOBILE model 10 times since it was first introduced in 1978.
Irrespective of these limitations, there are specific actions--most
of which were recommended by the Science Advisory Board in its 1989
resolution\5 --that, when followed, can enhance a model's estimating
capabilities. Among other things, these actions involve documenting
the implicit and explicit assumptions that are the basis of the
formulas contained in the model, obtaining external stakeholders'
input during the model's development, and having the model peer
reviewed before it is used. EPA officials acknowledged that,
primarily because of resource limitations, until recently such
actions have been delayed or forgone. However, EPA is developing the
next model, MOBILE6, with significantly increased openness and input
from other stakeholders. EPA also plans to carry out the actions
recommended by the Science Advisory Board, such as peer review, as
part of its program for developing MOBILE6, due to be issued in late
1998.
--------------------
\3 EPA and a group of stakeholders working with the agency on
modeling issues identified a number of limitations in MOBILE5a and
selected 14 of these for potential revision in MOBILE6.
\4 EPA's current model assumes that the emissions systems of vehicles
with 50,000 or more odometer miles deteriorate much more rapidly than
the systems of other vehicles.
\5 EPA's Science Advisory Board recommended many of these processes
in its January 1989 resolution on models, including one process--peer
review--that has been required by EPA policy since January 1993. The
Board is a legislatively established body of independent experts who
provide advice to the EPA Administrator on scientific and engineering
issues.
BACKGROUND
------------------------------------------------------------ Letter :2
The current MOBILE model, MOBILE5a,\6 also known as the EPA mobile
source emissions factor model, is a computer program\7 that estimates
the emissions of carbon monoxide, hydrocarbons, and nitrogen oxides
for eight different types of gasoline-fueled and diesel highway motor
vehicles.\8 The model consists of an integrated collection of
mathematical equations and assumptions about the emissions from
vehicles manufactured from 1960 to 2020; generally, the cars produced
in the 25 most recent model years are assumed to be in operation in
any given calendar year.\9 The first MOBILE model was made available
for use in 1978; since that time, major updates and improvements to
the model have been made as more has become known about the
complexity of the factors affecting vehicle emissions, as measurement
devices have improved, and as more data have been collected.
According to agency officials, these improvements have resulted in
the refinement of emissions estimates for evaporative emissions (such
as occur when the fuel tank and fuel system heat up on a hot summer
day); for the uncorrected in-use deterioration (wear and tear) that
results from poor vehicle maintenance or tampering; and for other
factors.
In its simplest form, EPA's MOBILE model allows the model user to
produce a number--an estimated quantity of emissions for the three
pollutants of concern--by multiplying the estimated emissions per
mile for an average urban trip times the estimated number of trip
miles traveled in an area. Over the years, however, researchers have
learned that vehicle emissions are highly complex. For example, EPA
and others have indications today that as much as half of all
hydrocarbon emissions from motor vehicles are evaporative emissions,
under certain conditions. To compensate for the complexities of
these and other emissions-producing activities, EPA has periodically
adjusted its basic formula--through the use of revised "correction
factors"--to approximate vehicle exhaust emissions in a range of
situations. In essence, the correction factor is a multiplier added
to the basic formula (miles traveled times emissions rate per mile)
to adjust the model's output to more closely reflect actual
emissions. Except for California,\10 EPA supplies the baseline
emissions rates and correction factors for other model
users--primarily state and local agencies--that typically supply
their own estimates of the number of vehicle miles traveled,
according to agency officials, as well as many other local area
parameters, such as the average ambient temperature, vehicle
classifications, and types of fuels sold.
The MOBILE model exists because precise information about the
emissions behavior of the approximately 200 million vehicles in use
in the United States is not known, yet the need exists to estimate
the impact of motor vehicles on air quality. For the states, the
MOBILE model is a tool for constructing emissions inventories,\11
creating control strategies, producing state implementation plans
(SIP),\12 and--subsequently--
demonstrating control strategy effectiveness to EPA and others. For
example, the states are allowed to vary a number of control strategy
features, including the types of fuels used, the type of inspection
and maintenance (I&M) testing network,\13 the frequency of I&M
testing, the ages and types of vehicles to be inspected, the
stringency of the tailpipe test, the number and percent of inspected
vehicles that may receive a waiver,\14 and a host of other factors.
The states may choose among a number of control options as long as
the state's control strategy achieves at least as many reductions as
required by the Clean Air Act.\15
For EPA, the MOBILE model is a tool for evaluating the adequacy of a
state's emissions inventory estimate, motor vehicle control
strategies, and implementation plans. In essence, the model's
estimates provide EPA regulators with critical information that is
used to evaluate the adequacy of a state's program and the relative
benefits of various policies to control motor vehicle emissions.
Additionally, the model's estimates can affect state policy decisions
on issues such as the content and volatility of fuels, and some
decisions on highway improvement projects. For example, the 1991
Intermodal Surface Transportation Efficiency Act (ISTEA) required,
among other things, that state transportation improvement programs in
certain nonattainment areas\16 conform with the applicable state
implementation plan developed under the Clean Air Act. Although the
model's original purpose was to support the development of mobile
source emissions inventories, over the years its role and influence
have been expanded considerably. Today, its estimates have a
substantial influence not only on state and local programs but also
on the automobile and oil industries, environmental and trade
organizations, the public, and others. According to estimates
derived partly from the MOBILE model, motor vehicles produce about 90
percent of the carbon monoxide, 50 percent of the hydrocarbons, and
30 percent of the nitrogen oxides emitted annually in major urban
areas.
--------------------
\6 MOBILE5a, issued in March 1993, is the current version of the EPA
MOBILE model that the states are required to use; EPA issued an
update in September 1996, known as MOBILE5b, as an option for the
states to use in calculating selected emissions reduction credits.
Unless otherwise indicated, the limitations discussed in this report
relate to both MOBILE5a and MOBILE5b.
\7 To facilitate computer programming, EPA created two other
models--one for fuels, another for inspection and maintenance
measures--whose outputs are inputs to MOBILE5a. Unless otherwise
specified, references to the MOBILE model hereinafter in this report
will include these input models.
\8 The eight classes of vehicles are light duty gasoline-powered
vehicles (passenger cars); light duty gasoline-powered trucks up to
6,000 pounds gross weight, and from 6,000 to 8,500 pounds gross
weight; heavy duty gasoline-powered trucks; light duty diesel
vehicles; light duty diesel trucks; heavy duty diesel trucks; and
motorcycles.
\9 Vehicles more than 25 years old are lumped into one category
called "25 years and older."
\10 With EPA's approval, California uses its own emissions factor
model to estimate vehicle emissions.
\11 National emissions are estimated annually by EPA on the basis of
statistical information about each of more than 450 source
categories, including the emissions factors and control efficiencies
for each source category. These aggregated emissions show emissions
trends at national and regional levels for man-made pollutants.
\12 These plans explain in detail how, and by how much, state and
local areas will reduce or control emissions from both mobile and
stationary sources.
\13 Testing networks may be test-only--where inspectors are
prohibited from making repairs; test-and-repair--where
inspectors/mechanics are allowed to make repairs; or hybrid--where
both types of facilities are allowed.
\14 The act requires that vehicle owners spend at least $450 toward
emissions-related repairs before a waiver may be granted.
\15 The states must achieve percentage reductions in HC and, for CO
and NOx, must submit plans for attaining the national ambient air
quality standards by the dates specified in the act relative to the
seriousness of their pollution problems.
\16 Areas that have not yet reached attainment with the national
ambient air quality standards are classified by EPA as nonattainment
areas, generally ranging in seriousness from marginal to extreme.
The more serious the classification, the more stringent the control
measures. ISTEA refers specifically to areas that are in
nonattainment for ozone or carbon monoxide.
LIMITATIONS IN THE CURRENT
MODEL
------------------------------------------------------------ Letter :3
EPA officials are examining 14 areas in the current MOBILE model in
which major limitations exist. According to agency officials, it is
their plan for each new version of the MOBILE model to reflect the
most recent testing, data collection, and research that are
available. They pointed out that EPA has updated the estimating
capabilities of its MOBILE source emissions model 10 times since the
model was first introduced in 1978. Table 1 briefly summarizes the
areas in which major limitations exist, as well as EPA's plans to
address these limitations in its next revision to the model, MOBILE6,
due to be issued in late 1998. (Additional information on these 14
areas is provided in app. I.)
Table 1
Summary of Major Limitations in the
MOBILE5a Model
Planned to be
addressed in
Area in MOBILE model Brief description of limitation\a MOBILE6
-------------------- ------------------------------------------ --------------
1. Emission Higher speeds produce increasingly larger Yes
estimates for emissions of CO, HC, and NOx;
higher speeds, technological limits of previous
especially speeds dynamometers\c meant most data were
in excess of 65 collected at speeds below 57 mph;
mph.\b increases in speeds above 65 mph have
left data gaps in current model that are
not adequately represented by existing
correction factors; ongoing work will be
used to revise these estimates.
2. Representation of Rapid acceleration and deceleration can Yes
emissions from produce large increases in emissions of
rapid acceleration CO and HC; technological limits of
and deceleration, previous dynamometers meant most data
including were collected at or below acceleration
aggressive driving rates of 3.3 mph/second; one EPA study
behaviors.\b indicated that about one-third of trips
had acceleration rates of more than 7
mph/second; concerns exist as to whether
these increased emissions are adequately
represented by existing correction
factors; ongoing work will be used to
revise these estimates.
3. Representation of Until vehicle engines reach normal Yes
emissions operating temperatures, emissions
immediately after typically bypass emissions control
engine start-up, equipment for 3 to 5 minutes, making cold
known as cold start starts one of the single largest
emissions.\b emissions-producing activities; recent
studies indicate that cold starts occur
more frequently than estimated in the
Federal Test Procedure, thus increasing
emissions; however, newer technology
vehicles now reach normal operating
temperatures more quickly, thereby
decreasing emissions; concerns exist as
to whether these activities are
adequately represented by the current
MOBILE model and whether they should be
separately calculated on the basis of
activity levels as opposed to
representing one portion of EPA's typical
driving cycle; EPA has already announced
plans to separate cold starts from other
driving cycle activities.
4. Representation of Air conditioners place an additional load Yes
emissions from air on a vehicle's engine and cause a vehicle
conditioner to produce more NOx emissions than when
usage.\b the air conditioner is not used; although
EPA had added 10 percent more load to the
dynamometer to simulate the effects of
air conditioners on emissions, testing on
a few vehicles indicates that the
emissions of NOx may be from 30 to 75
percent greater at some speeds than
currently represented by the MOBILE
model\e; the results of recent air
conditioner emissions studies are planned
to be reflected in the model's next
revisions.
5. Representation of Increased road grade can result in No
emissions from road substantial emissions increases,
grade, such as when especially in CO and NOx; due to the
a car climbs a difficulty of testing for this driving
hill.\b activity and the difficulty of plotting
road grades for millions of miles of
highways, this activity has not been
represented in EPA's model to date; due
to the expense and time required for road
grade testing and mapping, EPA plans to
add road grade emissions estimates in
several years when it develops MOBILE7.
6. Representation of High emitting vehicles account for a Yes
high emitting disproportionate amount of vehicle
vehicles\d in the emissions; thus, the estimate of their
MOBILE model's relative proportion in the overall
supporting vehicle population is critical to
database. accurate inventory emissions estimations;
a California study found high emitting
vehicles underrepresented in its own
model by nearly 5 times; concerns exist
as to whether these high emitting
vehicles are appropriately represented in
the current EPA model; ongoing I&M lane
testing in three states will be used to
review these estimates.
7. Representation of Lower volatility fuels produce less Yes
emissions from emissions, especially HC and CO
lower polluting emissions; over the years, low volatility
fuels, especially fuels have been produced that are lower
fuels with lower than the lowest level EPA estimated in
volatility. 1989 when it placed a lower limit
restriction in the MOBILE model, below
which low volatility fuels received no
additional emissions reductions credits;
additionally, one study found that the
model overestimates the CO emissions
benefits of oxygenated fuels; EPA plans
to eliminate the lower limit restriction,
review correction factors for lower
volatility fuels, and review CO estimates
for oxygenated fuels.
8. Representation of On the basis of a small sample, EPA added Yes
emissions system new correction factors in 1992 on the
deterioration for assumption that emissions control systems
vehicles with perform much worse after 50,000 miles;
50,000 or more some testing indicates that this
odometer miles. adjustment now causes the MOBILE model to
overestimate the emissions from fuel-
injected vehicles with 50,000 or more
odometer miles.; EPA plans to lower this
adjustment; ongoing work will determine
the extent of change.
9. Emissions Vehicle condition is one of the most Yes
estimates and important factors contributing to excess
assumptions for emissions; more recent and more complete
vehicle inspection data are needed about many facets of I&M
and maintenance programs, including the impact of newer
(I&M) programs. on-board computer technology, better
emissions control system warranties, the
durability of equipment, and alternative
ways to complement inspection programs
and improve repair effectiveness; for
MOBILE6, EPA plans to add an I&M
flexibility module covering most of these
areas so that individual states' studies
and ongoing EPA studies of I&M testing in
three states can be used to revise these
estimates as study data become available.
10. Estimates and Hydrocarbons evaporate from the fuel tank Yes
assumptions for and fuel lines when the vehicle is
non-tailpipe parked; improved testing over a longer
evaporative time period with more realistic
emissions when the temperature increases is ongoing to
vehicle is not better reflect actual emissions and
operating. refine existing correction factors in the
MOBILE model; however, some vehicles that
leak fuel are now believed by EPA to be
so significant that these will become a
separate emissions category in MOBILE6;
indications are that vehicles with such
fuel leaks--while a small percentage of
the fleet--can exceed the evaporative
emissions of corresponding non-leaking
vehicles by one to two orders of
magnitude.
11. Emissions EPA's supporting data on the in-use No
estimates and emissions of this category of vehicles
assumptions for the are about 20 years old; certification
inspection and standards are higher for these vehicles
maintenance (I&M) than their light duty counterparts, and
of heavy duty they are generally older and driven more
vehicles--those miles annually than their light duty
with a gross counterparts, although improved emissions
vehicle weight of technology has lessened the contributions
8,501 pounds or of individual vehicles; some studies are
more. under way, but agency officials question
whether sufficient I&M data will be
obtained in time to change emissions
estimates for heavy duty vehicles.
12. Data Many model assumptions, such as emissions Yes
characterizing control system deterioration, are closely
vehicle fleet. tied to accumulated mileage; the MOBILE
model's data on mileage accumulation has
been superseded by more recent data
indicating that passenger cars are driven
about 2,000 miles, or 10 percent more,
than currently estimated in the MOBILE
model; also, heavy duty vehicles are
believed to constitute a significantly
larger percentage of the overall vehicle
fleet than currently estimated in the
MOBILE model; EPA plans to revise these
estimates at the national level and notes
that this is an area where model users
often supply their own local data.
13. Greater Roadway classification (freeway, arterial, Yes
distinctions in feeder/collector street, along with their
roadway associated level of service) has a
classifications. substantial impact on emissions; the
current model was designed for estimating
areawide emissions over an entire trip
and has not provided for assessing
emissions' impacts on a smaller scale,
yet today's users need a model that
provides different estimates for
different roadways; for MOBILE6, EPA
plans to provide separate model emissions
estimates for at least three roadway
classifications.
14. Quantifying the Traditionally, when estimates are made No
uncertainty of the from limited data, the magnitude of
model's estimates. uncertainty, or range, associated with
the estimate is provided; however, this
has not been done for any versions of the
MOBILE model. Several studies suggest
that the current model's uncertainty may
be large; however, many factors,
including cost, will likely prevent EPA
from reporting the uncertainty of its
MOBILE6 estimates, although the agency
plans to move in this direction.
--------------------------------------------------------------------------------
Note: Representatives from EPA and from the Federal Advisory
Committee Act (FACA) workgroup are continuing to work on other less
significant limitations in MOBILE5a that are not addressed above.
From an original list of 90 items, an EPA-sponsored FACA subcommittee
has identified 47 high-priority items for improvement in the current
model; however, many of these items involved overlapping issues, and
in some instances, two or more of these items have been consolidated
to form the 14 categories of limitations discussed above.
Additionally, some of these items deal with suggested procedural
changes in the process that EPA should follow in making MOBILE model
revisions. Procedural changes in EPA's modeling process are
discussed in the next section of this report.
\a In some instances, the suggested emissions impact is based on a
few studies of vehicle activities that EPA is currently trying to
corroborate.
\b While many model assumptions are interconnected, these limitations
relate primarily to limitations in the Federal Test Procedure (FTP)
for certifying new cars, which was revised in October 1996 to better
estimate emissions from high speeds, aggressive driving, cold engine
starts, and air conditioner use; these changes in the revised FTP
have not been reflected in MOBILE5a (issued in March 1993), nor in
MOBILE5b (an optional version that was issued September 1996); the
relationship of FTP assumptions to EPA's MOBILE model estimates is
discussed more fully in appendix I.
\c A dynamometer is a treadmill-like device that simulates the load
placed on a vehicle by electronically controlling inertial forces
while the vehicle's wheels are driven at various pre-determined
speeds, including acceleration and deceleration.
\d High emitters are generally defined as vehicles with emissions
more than twice the applicable standard for the particular make and
model of vehicle, and are often described on the basis of the amount
of excess emissions they produce. For CO and HC, EPA classifies such
excess emitters as high, very high, and super emitters; for NOx,
vehicles are classified as either normal or high.
\e These tests compared actual air conditioner emissions with the
FTP's estimate of such emissions by adding 10 percent additional load
to the dynamometer; MOBILE5a employed essentially the same correction
factor, thereby estimating air conditioner emissions in the same
manner and amount as the FTP.
While acknowledging that some vehicle emissions-producing activities
are not accounted for in the current model and that other
emissions-producing activities are not adequately represented in the
current model on the basis of the most recent information, EPA
officials said that it is important to note that EPA has conducted
and/or partially funded some of the studies that have led to the new
data that now question the old estimates and assumptions.
Additionally, they said that EPA has work under way to address most
of these limitations. For example, since its formation in September
1995, an EPA-sponsored Federal Advisory Committee Act (FACA)
subcommittee workgroup has identified 47 high-priority items for
improvement in the current model, and EPA and workgroup
representatives are examining these limitations. EPA is also in the
process of developing new procedures for improving models in general,
which are discussed below.
EPA'S PROCESS FOR IMPROVING THE
MOBILE MODEL
------------------------------------------------------------ Letter :4
Several model experts told us that it is the nature of models such as
MOBILE5a to have limitations and to be in a continuous improvement
mode. Agency officials agreed with this assessment, noting that the
current model is better than any previous versions and reflects
consistent growth in the quality and quantity of information
available on very complex issues. Additionally, they pointed out
that--through the FACA workgroup process--the revisions to the next
MOBILE model, MOBILE6, have been undertaken with significantly
increased openness and input from other government agencies,
academia, the automobile and oil industries, environmental groups,
and others. Our contacts with representatives of these groups
confirmed this increased level of external stakeholders' involvement
in preparation for MOBILE6. Several commended EPA's efforts in
recent years to reach out to persons outside of the agency, and some
noted that the outreach effort had given them a much greater
appreciation of the model.
While acknowledging that, historically, there have been few firm
criteria on the processes that should be followed when creating or
revising a model such as the MOBILE model, the executive director of
EPA's Science Advisory Board (SAB) told us that the agency has a
project under way to develop agencywide procedures for improving
models. According to the project director, the Office of Research
and Development (ORD) is planning a workshop in December 1997 to
discuss the status of modeling across several media and other
modeling issues, including the need for better agencywide modeling
procedures. The project director also said that the Science Advisory
Board's January 1989 resolution on models was one of the best
documents available on the processes for creating and improving
models in general. The SAB executive director and the ORD project
director told us that in their opinion, there are specific
actions--most of which were recommended by the SAB in its 1989
resolution--that, when followed, can enhance a model's predictive
capabilities. Among other things, these actions include the
following:
-- Obtaining external stakeholders' input to ensure that the
model's assumptions and formulas receive critical review by
those not involved in the model's development.
-- Documenting the implicit and explicit assumptions so that others
can evaluate the basis of the formulas embedded in the model.
-- Performing sensitivity analyses over key parameters to identify
the most sensitive parameters and to establish the areas most in
need of further research.
-- Verifying the adequacy of the model's mathematical code.
-- Testing the model's predictions with laboratory and field data
to confirm that the model generates results consistent with its
underlying theory.
-- Conducting peer review to enhance the quality, credibility, and
acceptability of the model's applications.
The SAB executive director pointed out that because of continuing
concerns with the quasi-regulatory use of agency models, EPA issued
agencywide guidance in 1994 specifically calling for the peer review
of such models. This directive was a follow-on to EPA's January 1993
agencywide policy requiring peer review of the scientific and
technical work products used to support agency decisions. In
September 1996\17
and March 1997,\18 we reported and testified on the uneven
implementation of EPA's peer review policy, including that the MOBILE
model had not been peer reviewed. EPA agreed with our
recommendations for educating staff and managers about the merits of
and procedures for conducting peer review and for ensuring that all
relevant products are considered for peer review. The agency has set
in motion a three-pronged approach to improve the implementation of
peer review agencywide, including peer review of the next MOBILE
model. According to the ORD project director, it is too early in
EPA's study to predict whether the agency may recommend that EPA
require its offices to follow the other actions listed above when
creating or revising models.
According to Office of Mobile Sources (OMS) officials, they plan to
carry out all six of the above activities as part of their
improvement process for MOBILE6 and noted that some of these
activities are already well under way, such as involving external
stakeholders. For example, OMS held its first stakeholder meeting in
June 1994, established a FACA mobile modeling workgroup in July 1995,
and has held five meetings since that time to obtain external views
by those not involved in the model's development, according to agency
officials. OMS officials acknowledged that some of the key formulas
in the current model have not been properly documented,\19 that
full-scale sensitivity analyses have not been performed since May
1990 (when they were performed for MOBILE4.1),\20 that fewer
resources have resulted in fewer confirming data, and that the MOBILE
model has not been peer reviewed. However, they said they have
efforts under way or planned to address these and other modeling
needs. For example, one of the recommendations of the FACA mobile
modeling workgroup--made up of representatives from EPA, state and
local agencies, industry, environmental groups, and academia--is that
EPA more fully document the model's assumptions. Additionally, OMS
plans to perform sensitivity analyses for the next version of the
model and to have the studies supporting key changes for MOBILE6 peer
reviewed. Also, OMS officials explained that as changes are proposed
for each area of major limitations in the model, they plan to have
the entire area peer reviewed.\21
Agency officials explained that declining modeling resources have
affected the pace of model improvements over the years, particularly
their ability to confirm the model's estimates with large numbers of
vehicle tests. For example, a study of the emissions characteristics
of 100 passenger cars for both exhaust and evaporative emissions
could cost from $1.4 to $1.6 million, according to the agency's
current estimates, and still not address the emissions impacts of
road grade, air conditioning, or most fuel studies. Studies of heavy
duty trucks and other larger vehicles would cost considerably more.
As an illustration of the magnitude of the task compared with
available resources, we obtained EPA's estimate of mobile modeling
needs in response to the mobile source requirements envisioned for
the 1990 Clean Air Act amendments. This June 1990 analysis indicated
that the Office of Mobile Sources would need about $60 million for
the modeling improvements known at that time. Since then, because of
higher priority needs, OMS has been allocated only $21.8 million,
cumulatively, for modeling improvements, although the research needs
have increased. However, many more groups have become involved in
non-EPA-funded vehicle emissions studies than in the past, allowing
EPA to benefit from their studies and observations. Additionally, in
some instances, researchers have sought EPA's input on study
protocols beforehand and have shared the data collected with EPA
afterwards. While concerned about resources, OMS officials explained
that making model improvements is an ongoing, continuous process--and
one that will continue after MOBILE6 is issued in 1998. They pointed
out that their goal is for each new version of the MOBILE model to
reflect the latest testing, data collection, and research. While
still not able to quantify the improvements, they said that in their
opinion, each new version of the MOBILE model is better than its
predecessor.
--------------------
\17 Peer Review: EPA's Implementation Remains Uneven
(GAO/RCED-96-236, Sept. 24, 1996).
\18 Peer Review: EPA's Implementation Remains Uneven
(GAO/T-RCED-97-95, Mar. 11, 1997), testimony before the House
Subcommittee on Energy and Environment, Committee on Science.
\19 Reports by Sierra Research, Systems Applications International,
and the Department of Transportation's Federal Highway
Administration, among others, found that some of the key formulas in
the MOBILE model are not properly documented, making it difficult for
researchers to know what data or assumptions were used in developing
the formulas.
\20 This was the last sensitivity analysis EPA published; agency
officials indicated that limited sensitivity analyses were performed
over selected parameters--such as key speeds, temperatures, and
fuels--in MOBILE5 (Dec. 1992) and MOBILE5a (Mar. 1993) but that
these analyses were limited and not published.
\21 OMS officials explained that they are still early in the peer
review process and have not yet developed a peer review plan for the
MOBILE model.
AGENCY COMMENTS
------------------------------------------------------------ Letter :5
We provided copies of a draft of this report to the Environmental
Protection Agency for its review and comment. We obtained comments
from EPA officials, including the Director of the EPA Office of
Mobile Sources. EPA agreed with the overall message of the report
but expressed concerns with imprecise language and suggested several
changes to clarify information in the report. For example, EPA
suggested that in lieu of describing some limitations associated with
its use of the Federal Test Procedure as a basis for estimating
emissions as "FTP assumptions," since the FTP is a specific, codified
test cycle, these would be better described as "FTP parameters." EPA
also suggested that we provide specific citations for four studies
referred to in appendix I. We made the language changes suggested by
EPA, including adding the citations.
EPA was also concerned that the report did not clearly distinguish
between limitations that may result in only trivial emissions impacts
and those that could be significant. Where it was possible to do so,
we believe the estimated emissions impact had already been quantified
or qualitatively described in the report. Additionally, as noted in
the section on the uncertainty limitation, one of the 14 limitations
of the MOBILE model is that it does not currently have information
about, nor estimates of, the uncertainty associated with its
emissions estimates. However, we agree that researchers viewed some
limitations as having a more significant impact on emissions than
others, and we have provided this view in the report. Appendix II
contains the agency's overall written comments.
---------------------------------------------------------- Letter :5.1
We conducted our review from October 1996 through August 1997 in
accordance with generally accepted government auditing standards. A
detailed discussion of our scope and methodology is provided in
appendix III.
As arranged with your office, unless you publicly announce its
contents earlier, we plan no further distribution of this report
until 15 days after the date of this letter. At that time, we will
send copies to the Administrator of EPA and other interested parties.
We will also make copies available to others upon request.
Please call me at (202) 512-9692 if you or your staff have any
questions. Major contributors to this report are listed in appendix
IV.
Sincerely yours,
Lawrence J. Dyckman
Associate Director, Environmental
Protection Issues
MAJOR LIMITATIONS IN MOBILE5A
=========================================================== Appendix I
According to agency officials and model experts we contacted, it is
the nature of models to have limitations and to be in a continuous
improvement mode. As a result, the Environmental Protection Agency
(EPA) has periodically updated the estimating capabilities of its
mobile source emissions model to reflect new information as data have
become available; MOBILE5a reflects the 10th major revision since the
model was first introduced in 1978. The following sections provide
additional information on 14 areas in which major limitations exist
in EPA's current MOBILE model, MOBILE5a.
FIVE LIMITATIONS RELATED TO
USE OF THE FEDERAL TEST
PROCEDURE AS A BASIS FOR
INVENTORY MODELING
------------------------------------------------------- Appendix I:0.1
The underlying basis for EPA's original model, and all subsequent
versions, has been the Federal Test Procedure (FTP),\1 a laboratory
dynamometer\2 test used to certify new cars against new-car emissions
standards. The FTP is roughly based on a typical urban area trip,
complete with starts and stops, covering 7.5 miles in the Los Angeles
urban area in the late 1960s. Such a trip is known as a driving
cycle, which can be approximated on a dynamometer.\3 Primarily
because of the limitations in past dynamometers,\4 the FTP driving
cycle parameters stipulate, among other things, that vehicles average
19.6 miles per hour (mph) over the 7.5 mile trip, do not exceed 57
mph, accelerate gradually (not to exceed 3.3 mph/second), and travel
on a flat surface. Additionally, EPA added 10 percent to the FTP
dynamometer load in an attempt to simulate the effects of air
conditioner usage. However, five of the major limitations in the
current MOBILE model relate to FTP parameters. These five are (1)
emissions from road grade, (2) emissions from air conditioner usage,
(3) emissions at higher speeds, (4) emissions from aggressive
driving, and (5) emissions immediately after engine start-up (cold-
start). Agency officials have long recognized that some of these
original FTP parameters were not representative of actual driving
conditions and, to compensate for these limitations, have added
correction factors to the MOBILE model to estimate what emissions
would be for speeds in excess of 57 mph, for rapid acceleration
beyond 3.3 mph/second, and for other scenarios, such as different
temperatures or different fuels. For example, the impact of
temperature on emissions can be substantial. Consequently, while FTP
testing has been performed between 68 and 86 degrees Fahrenheit,
EPA's MOBILE model used a correction factor to estimate that 1995
exhaust emissions of hydrocarbons would be 3 times greater at 25
degrees than at the FTP temperatures.
However, MOBILE5a does not account for the impact of road grade--such
as when a car climbs a hill--although some studies have indicated
that both the increased load on the engine from climbing a hill and
the decreased load that accompanies engine deceleration significantly
increase vehicle emissions. According to agency officials, it is not
expected that MOBILE6 will have adjustments for road grade, although
such adjustments are being planned for MOBILE7. In addition to being
uncertain about the amount of emissions related to road grade, EPA
officials explained that obtaining the basic data from instrumented
cars and chase cars to make such adjustments would be expensive at
this time and that because of the cost and length of time required
for these studies, the impact of road grade will probably not be
addressed until MOBILE7. Additionally, an equally important
consideration is that once these basic data on the effects of road
grade on emissions are obtained, state and local agencies would have
to plot road grades for millions of miles of roadways in their
jurisdictions, also a costly and time-consuming activity. However,
as global positioning technology\5 for vehicles becomes less costly
and more widely available and used, it is envisioned that the impacts
of road grade emissions will be modeled in the future. According to
EPA officials, the amount of data needed to estimate the impact of
road grade is still years away. They also noted that time would be
needed to develop consistent guidance on how state and local agencies
should go about collecting these data.
With respect to the representation of other emissions-producing
activities not represented by the FTP, EPA officials have made
periodic adjustments but recognize that the current model's
correction factor adjustments may not reflect the most up-to-date
information. For example, as noted in the EPA-FACA materials
disseminated in March 1997, the 10-percent additional load intended
to simulate the effect of air conditioner usage "is obsolete." More
recent information indicates that nitrogen oxide emissions may be
from 30 to 75 percent greater at some speeds than the current model's
estimates when the air conditioner is used.
Additionally, increases in speeds above 65 mph have left data gaps in
the current model that are not adequately represented by existing
correction factors. Others have recognized that the MOBILE model's
estimates are inextricably tied to the FTP's parameters, and some
studies have questioned the representativeness of key assumptions as
they relate to the FTP. For example, the California Air Resources
Board (CARB) commissioned Sierra Research in 1993 to develop an
improved driving cycle--known as the Unified Driving Cycle--by using
an instrumented "chase car" to better characterize typical urban
driving patterns. Among other things, this driving cycle allows cars
to travel up to 67.2 mph (versus 57 mph for the FTP), allows for
acceleration at a rate of up to 6.9 mph/second (versus a maximum of
3.3 mph/second for the FTP), and uses an average speed of 24.6 mph
(versus 19.2 for the FTP). Several model experts believe these
parameters more closely approximate actual driving conditions today.
According to a 1993 CARB study, the FTP may underestimate
hydrocarbon, carbon monoxide, and nitrogen oxide emissions by 27, 68,
and 17 percent, respectively.
A 1993 study sponsored by EPA's FTP improvement project found that
more than one-third of the trips studied had acceleration rates of
more than 7 mph/second--more than double the FTP's maximum of 3.3
mph/second rate. Similarly, another 1993 EPA-sponsored study of
instrumented vehicles in the Baltimore area found that 18 percent of
total driving time in the area was composed of higher speeds and
sharper accelerations than those represented on the FTP test. Also,
a 1995 National Research Council report noted that aggressive driving
with many accelerations resulted in hydrocarbon and carbon monoxide
emissions being 14 and 15 times higher, respectively, than the
emissions from average driving over the same 7-mile trip.\6 According
to the 1995 National Research Council report, "Virtually all motor
vehicle testing has been based on a limited set of driving test
cycles that inadequately represent current urban driving conditions."
However, one model expert told us that it took more than 1 year to
evaluate one component of the model and that collecting vehicle
emissions data on large data sets is very costly. EPA officials
pointed out that there is not a consistent definition of what
constitutes aggressive driving, that aggressive driving happens only
over a portion of the trip and is highly variable among drivers, and
that the above observations are not representative of average driving
patterns.
The Congress has also recognized that the FTP may not reflect actual
driving conditions. Concerned about the gap between emissions as
measured by the FTP and actual, real-world emissions, in 1990 the
Congress added Section 206(h) to the Clean Air Act, which required
EPA to review and revise the FTP within 18 months "to insure that
vehicles are tested under circumstances which reflect the actual
current driving conditions under which motor vehicles are used."
EPA's October 1996 final rule on FTP revisions\7 addressed four
emissions-producing activities that, according to the rule's
preamble, are not adequately represented in the current FTP. These
emissions-producing activities include (1) aggressive driving
behavior (such as high acceleration rates and high speeds), (2) rapid
speed fluctuations (such as quick deceleration\8 ), (3) emissions
immediately after engine start-up, a period when--because of the fact
that engines are designed to operate at higher
temperatures--emissions typically bypass emissions controls for an
estimated 3 to 5 minutes until the engine reaches normal operating
temperatures, and (4) actual air conditioner usage.
EPA has not yet revised the MOBILE model to reflect the results of
recent studies that have led to these FTP rule revisions but has work
ongoing in all four areas. According to agency officials, although
adjustments had been made to the MOBILE model for most of these
activities prior to issuing the revised FTP rule, given the state of
knowledge today, it appears that these activities may not be
adequately represented in MOBILE5a. EPA officials told us that
incorporating new estimates for these emissions-producing activities
would be a high priority for MOBILE6, due to be issued in late 1998.
For example, in March 1997 agency officials announced their plans to
substantially revise the cold-start segment of the next MOBILE model,
moving--for the first time ever--from an areawide, trip-based model
to a roadway-specific model that also separately accounts for
start-up emissions. Under this revised model, the magnitude of
start-up emissions will not depend on vehicle speed or the driving
cycle. Instead, EPA is proposing to allow model users to model the
emissions impacts of cold starts on the basis of local areas'
estimates of the number of such starts. Additionally, model users
will be able to estimate emissions for three different types of
roadways--freeways, arterials, and local roadways.
--------------------
\1 Since 1991, EPA has used dynamometer test data from thousands of
vehicles tested in IM240 inspection lanes in Hammond, Indiana, to
improve the basic emissions rate estimates in the MOBILE model. The
IM240 test is a 240-second test designed to replicate certain
segments of the longer FTP test and, as such, has the same basic
benefits and limitations as the FTP test.
\2 A dynamometer is a treadmill-like device that simulates the load
placed on a vehicle by electronically controlling inertial forces
while the vehicle's wheels are driven at various pre-determined
speeds, including acceleration and deceleration. The new FTP
requires improved single-roll dynamometers, 48 inches in diameter,
with higher power absorption capacity to provide better replication
of actual emissions (or a system that provides equivalent or superior
results), replacing the smaller (about 9 inch) double-roll types.
\3 EPA's driving cycle, known as the EPA Urban Dynamometer Driving
Schedule, tests a vehicle over several driving scenarios during a
1,877-second, or about a 31-minute dynamometer test. "Cold-start"
emissions are represented by the first 505 seconds of emissions,
after which so-called "hot stabilized" emissions are collected to
represent the period when the vehicle is fully warmed up and both the
engine and catalytic converter have reached typical operating
temperatures (the 1,877-second driving cycle also includes a "hot
start" component).
\4 According to EPA, FTP driving cycle parameters, such as top speed
and acceleration rate, until recently were substantially constrained
from approximating actual driving conditions by the physical
limitations in dynamometer technology.
\5 Global positioning system and on-board vehicle computer technology
allow a driver to electronically determine his location on earth,
among other things. Appropriately instrumented vehicles would use
these technologies to plot and record both grade and location.
\6 Expanding Metropolitan Highways: Implications for Air Quality and
Energy Use, Special Report 245, Transportation Research Board,
National Research Council, National Academy of Sciences, 1995.
\7 Final Regulations for Revisions to the Federal Test Procedure for
Emissions From Motor Vehicles, 61 Fed. Reg. 54852 (Oct. 22, 1996).
\8 Rapid deceleration causes excess emissions because more fuel is
fed into the engine than the engine now demands, resulting in
improperly combusted fuel.
EIGHT LIMITATIONS RELATED TO
OTHER AREAS OF THE MOBILE
MODEL
------------------------------------------------------- Appendix I:0.2
In addition to concerns about the representativeness of the FTP
parameters, the following issues were also identified by model
experts, workgroup participants, and/or stakeholders we contacted.
In each instance, EPA officials agreed that the limitation is an area
of concern and in most cases noted that the agency has ongoing work
to address the issue, which is discussed below.
One concern is the representation of high emitters\9 in EPA's MOBILE
model database, since the data now indicate that this group of
vehicles accounts for a disproportionate amount of an area's overall
emissions and that if this subset of the overall vehicle population
is underrepresented, the impact on the emissions estimates can be
substantial. For example, Sierra Research testified in 1995 that the
worst polluting 22 percent of the vehicles produce about 50 percent
of the emissions, and EPA estimates that, overall, from 10 to 30
percent of the vehicles cause the bulk of the pollution problems.
As noted by the nonfederal co-chair of the FACA modeling workgroup in
a 1993 study, "The general problem of failing to control for
significant factors is compounded by the likelihood that a small
fraction of the vehicle fleet are currently responsible for a large
percentage of vehicle emissions."\10 He told us that he still
believes this to be one of the most significant issues facing EPA
today, primarily because a very small number of vehicles can
potentially be responsible for unusually high levels of pollution.
This was also a significant issue for one of the Session Chairs for
the Coordinating Research Council's (CRC) April 1997 Workshop,\11 who
noted that, in his opinion, this is the single greatest issue that
EPA faces--how to identify and repair high emitters and properly
represent them in the modeling database. The nonfederal co-chair
also noted that if the occurrence of such vehicles is not properly
represented in the model, the model's emissions estimates can be
seriously flawed. He and others have concerns that the existing
database may underrepresent high emitters because, among other
reasons, the owners of such vehicles may avoid surrendering such
vehicles for inspection and maintenance (I&M) and other testing at a
higher rate than the normal population. As noted in a February 1996
study, "individuals with intentionally tampered or poorly maintained
vehicles may be less likely to offer their vehicles for testing."\12
Additionally, a 1993 CARB study of 186 vehicles indicated that high
emitters could represent 16.8 percent of the California fleet, or
nearly 5 times the assumption in the California model.\13 EPA
officials have some concerns with the study, its reliance on remote
sensing devices, and its applicability to other states.
Additionally, EPA officials believe that the larger data sets
provided by their ongoing I&M lane testing in three other states
properly identify most high emitters. However, they agreed that
appropriate representation of high emitters is important to the
model's emissions estimates and noted that this is also a high
priority issue currently being addressed by EPA and one of the
subgroups of the mobile modeling workgroup.
A second concern is the current correction factors for lower
volatility fuels and for oxygenated fuels. For example, a February
1997 report by Sierra Research\14 found, among other things, that
MOBILE5a likely underestimates the impact of low reid vapor
pressure\15 (RVP) fuels on hydrocarbon and carbon monoxide emissions
at temperatures above 75 degrees. The report notes that the
correction factor for low RVP fuels has not changed since February
1989, when limited data on fuels with RVPs lower than 9.0 pounds per
square inch (psi) caused EPA to place a constraint code in the model
precluding users from being able to calculate reductions below this
level. The limited data collected since that time indicate that
reducing fuel RVP from 9.0 psi to 7.0 psi may reduce hydrocarbon and
carbon monoxide exhaust emissions from 18 to 27 percent more than the
model estimates, respectively. The January 1997 Auto/Oil Air Quality
Improvement Research Program (AQIRP) Final Report\16 suggested that
reducing fuel volatility by 1 psi, from 9.0 to 8.0 psi, would reduce
exhaust CO by 9 percent, exhaust HC by 4 percent, and total
evaporative HC by 34 percent (NOx remained unchanged). The RVPs for
most fuels used to be higher than 9.0 psi,\17 but today they can go
lower than 7.0 psi. Similarly, the model currently has no emissions
reduction credits for low sulfur fuels (except, according to agency
officials, for the lower sulfur effect in reformulated gasoline),
although recent studies suggest that lowering the concentration of
sulfur in fuel reduces the emissions of hydrocarbons and nitrogen
oxides.
EPA officials said that they plan to eliminate the constraint code in
the next model, MOBILE6, which will allow users to receive credit for
correction factors for fuels lower than 9.0 psi; however, they noted
that work in this area is still ongoing and that the data on the
emissions benefits of lower RVP fuels, as well as low sulfur fuels,
are limited. In addition, a 1996 National Research Council study
suggested that the model may overestimate the benefits of oxygenated
fuels. For example, the study noted that EPA's MOBILE model
"apparently overpredicts the oxygenated fuel effect by at least a
factor of two" when the model's estimate of carbon monoxide
reductions is compared with observed data. Similarly, a 1997
study\18 of wintertime oxygenated fuels suggested that the observed
oxygenated fuel benefits were much lower than the 20 to 30 percent
estimated by EPA's model. EPA officials agreed that this is also an
area that needs more study, but one which they plan to address in
MOBILE6.
A third concern is MOBILE5a's estimates of emissions system
deterioration for vehicles with more than 50,000 odometer miles.
This concern stems from studies that have questioned the rate and
quantity of the data supporting EPA's significantly higher rate of
emissions system deterioration once vehicles reach 50,000 odometer
miles.\19 For example, prior to MOBILE5, the model assumed that a
vehicle with 100,000 miles emitted about 1.0 grams of hydrocarbons
for each mile driven, or about 4 times the amount a new car would
emit. However, EPA adjusted the deterioration rates for vehicles
with more than 50,000 miles beginning with MOBILE5 (Dec. 1992) so
that the MOBILE model's deterioration formula now calculates that the
same car was emitting about 2.0 grams of hydrocarbons for each mile
driven, or about 8 times the amount a new car would emit. EPA
acknowledges that these adjustments were made on the basis of limited
data and that only recently have 1990-technology vehicles become old
enough to accurately assess their emissions deterioration.
An October 1996 Sierra Research study\20 of 75 vehicles with over
100,000 odometer miles questioned whether EPA had perhaps adjusted
the formula too much, resulting in a model that currently
overestimates the emissions from vehicles with 50,000 or more
odometer miles. Among other things, the study found that EPA's
current model estimated that 80 percent or more\21 of these higher
mileage vehicles (which, on average, had accumulated 123,900 odometer
miles) would be high emitters, whereas the study found that only 32
percent of the vehicles fell into this category. Similarly, an April
1997 study\22 of 227 vehicles (model years 1991 to 1993) with more
than 50,000 odometer miles found no significant changes in emissions
or deterioration as indicated by the current model. OMS officials
said they used all the data that were available to them at the time
(1991-1992) to estimate the deterioration rate of such vehicles and
that researchers since then had had more time and more vehicles to
test than were available to EPA. They also said the model's
correction factor for vehicles with 50,000 or more odometer miles
would likely be lowered in MOBILE6, but they were uncertain at the
time of our audit how much this emissions estimate would be reduced.
A special subgroup of the FACA mobile model workgroup has been
established to address this issue, and their work is still ongoing.
A fourth concern is MOBILE5a's emissions credits and assumptions
about inspection and maintenance programs. According to a 1995
National Research Council report, vehicle condition--whether the
vehicle is well maintained, or has been tampered with or is
malfunctioning--is more important than vehicle age in determining
emissions. Among other issues, there is a need to update the basic
data supporting I&M emissions reduction credits to reflect a growing
population of vehicles in which rates of tampering may be diminishing
since tampering with newer vehicles adversely affects gas mileage and
vehicle performance. Also, vehicle owners often replace older,
carbureted vehicles with newer fuel-injected vehicles.
Additionally, according to agency officials, the current model
provides no additional I&M credits for vehicles equipped with
on-board diagnostics\23 (OBD), a requirement for all 1994 and later
light duty vehicles and trucks. This vehicle computer technology
alerts a car owner when an emissions system malfunctions, permitting
quicker repairs than when such malfunctions are identified through an
I&M testing program, and diagnostic trouble codes assist mechanics in
making better repairs. Additionally, newer vehicles have up to 8
years or 80,000-mile emissions control system warranties for two
components (the on-board computer and catalytic converter), which
should equate to less-polluting vehicles as a result of more durable
emissions control systems and the requirement that manufacturers
cover the costs of certain repairs. The current model does not
provide specific credits for this growing population of OBD-equipped
vehicles designed and believed to have less in-use deterioration than
their predecessors.
Also, more recent and more complete data are needed on the
effectiveness of repairs in an I&M program, including the adequacy
and durability of these repairs, actual participation rates, and
impact of remote sensing efforts. Except for remote sensing, the
current model's estimates for these parameters is based on aging and
limited data.\24 For example, EPA has not performed any tampering
surveys since 1992, and agency officials said that as a result of
this lack of data, the tampering assumptions for MOBILE6 will remain
unchanged from MOBILE5a. However, EPA's goal for MOBILE6 is to
provide users with greater flexibility in designing I&M programs, as
long as the state or local programs' estimated I&M credits can be
substantiated with state or local data. With respect to newer
vehicles equipped with on-board diagnostics, because of the limited
data on the longer-term emissions impact of this technology, the
agency has provided credit for these OBD-equipped vehicles equal to
that provided for operating an enhanced I&M program. EPA officials
said that, in addition to the states' own studies, the agency
currently has I&M effectiveness studies being carried out in three
states, but they were unsure whether sufficient data would be
available in time to further revise the I&M assumptions in MOBILE6.
A fifth concern is the proper representation of diurnal emissions.
Diurnal emissions refer only to hydrocarbons and are a form of
evaporative emissions that occur when a vehicle is parked and the
ambient temperature is fluctuating. For all previous versions of the
MOBILE model, the data supporting these 8- to 24-hour emissions
estimates were collected during a 1-hour period during which
temperatures were forcibly increased over a range of temperatures.
More recent testing over 24-hour and longer periods without
constraining temperature increases to a one-hour period indicates
some differences from the MOBILE5a estimates for such evaporative
emissions. Also, evaporative emissions from vehicles with fuel leaks
are now believed to be so significant that, for MOBILE6, EPA plans to
model these emissions separately from other evaporative emissions.
According to EPA's most recent data, indications are that some
vehicles with fuel leaks--similar to super emitters of
exhaust/tailpipe emissions--can exceed the evaporative emissions of
corresponding vehicles by one to two orders of magnitude. A 1996
automotive industry study\25 of 150 vehicles found that 24-hour
diurnal emissions ranged from 0.6 grams of HC to 777.2 grams of HC,
with vehicles with liquid fuel leaks providing the vast majority of
the emissions. EPA officials explained that while they may develop a
separate category for some vehicles with significant fuel leaks, this
does not necessarily mean there will be a significant overall
increase in evaporative emissions estimates because, until more data
are collected, there is no clear indication that these emissions were
significantly underestimated in prior diurnal estimates. EPA has
testing under way to determine how to better define this category of
vehicles with significant fuel leaks and also plans tests to estimate
their distribution within the current fleet, rate of occurrence as a
function of accumulated mileage, vehicle age, and/or vehicle
technology, such as fuel tank design. Agency officials are uncertain
at this time whether the correction factors for other evaporative
emissions estimates will be revised for MOBILE6.
A sixth concern is the adequacy of the data supporting MOBILE5a's
assumptions about emissions for in-use\26 I&M heavy duty\27 vehicles.
According to EPA's June 1994 workshop on state needs, the in-use
credits for heavy duty gasoline-powered vehicles are based on data
approximately 20 years old, and there has been much change in the
technology and emissions rates of these vehicles since that time.
Still, the certification standards are higher for heavy duty vehicles
than their light duty counterparts, and they are generally older and
driven more miles annually than their light duty counterparts. EPA
officials said that testing heavy duty vehicles is difficult and
quite expensive and agreed that there is a lack of recent data on the
in-use emissions from this category of vehicles once they have been
put in service. While some studies are under way, EPA does not
envision at this time that significant changes in the in-use
emissions rates for heavy duty vehicles will be included in MOBILE6.
A seventh concern is the fleet characterization data in EPA's
database, stemming from a concern that much of the data used for
MOBILE5a are quite old. For example, MOBILE5a's estimates are based
on the assumption that, on average, light duty vehicles are driven
about 14,000 miles annually when new, decreasing to less than 10,000
miles annually after 10 years. More recent data from the U.S.
Department of Transportation indicates that passenger cars are driven
about 2,000 miles more annually than currently estimated by EPA's
MOBILE model, or nearly a 10-percent increase over MOBILE5a.
According to a 1996 report,\28 because of the linkage between
odometer mileage and I&M program assumptions, a small change in
mileage accumulation rates can result in a large impact on emissions
estimations. EPA officials pointed out that the agency's guidance
encourages model users to provide their own accumulated mileage
estimates; thus, they said the default values for accumulated mileage
in MOBILE5a would be a problem only in those cases in which model
users fail to provide their own accumulated mileage estimates.
Additionally, heavy duty gasoline-powered vehicles, which have higher
certification standards than their light duty counterparts, are
believed to comprise a significantly larger percentage of the overall
vehicle fleet than currently estimated by the MOBILE model.
According to OMS, the agency plans to update the fleet
characterization data for MOBILE6, including reflecting the increases
in the heavy-duty vehicle population.
Similarly, another fleet characterization issue involves urban buses.
For example, the current model does not have a separate
classification for urban buses, although this is a growing vehicle
category in many urban areas with unique operating characteristics,
such as very frequent starts and stops. EPA officials explained that
while buses have not been a separate category in MOBILE5a, EPA plans
to expand the current list of vehicle categories from 8 to 20, one of
which will be a separate category for buses.
An eighth concern is the level of distinctions in roadway
classifications. The MOBILE model was originally designed only for
estimating areawide emissions on the basis of assumptions associated
with an entire trip. It was not designed for making decisions for
various roadway classifications, such as transportation improvement
projects for urban interstate, rural arterial, or urban
feeder/collector streets. Several model experts have pointed out
that the same average travel speed--35 mph, for instance--would
indicate smooth traffic flow on a local street but severe congestion
on a freeway. EPA officials pointed out that MOBILE6 will allow
users to separate start emissions from any linkage to the FTP driving
cycle assumptions and will also provide different correction factors
for speed and driving cycle for three different types of
roadways--freeways, arterials, and local roadways. Additionally,
while not planned for MOBILE6, the agency plans to partially fund
ongoing research with the Department of Transportation to develop a
modal emissions model that may one day allow users to model
additional parameters, such as the relative emissions impact of
sequencing traffic signals to enhance traffic flow.
As noted above, an EPA-sponsored FACA mobile model workgroup made up
of representatives from other federal, state, and local government
agencies, academia, the automobile and oil industries, environmental
groups, and others has been assisting EPA in improving the current
model, and much of the research to fill data gaps and update aging
databases was still ongoing at the time of our audit. Agency
officials said that it is their plan for each new version of the
MOBILE model to reflect the most recent testing, data collection, and
research that are available. Except for the impact of road grade on
emissions and revising the in-use credits for heavy duty vehicles,
EPA officials said they plan to address each of the above limitations
in the next revision, MOBILE6. However, as discussed below, the
agency will not be able to quantify the uncertainty associated with
its MOBILE model estimates, primarily because of the complexity and
timing of factors affecting vehicle emissions and the high cost of
vehicle studies.
--------------------
\9 High emitters are generally defined as cars with emissions more
than twice the applicable standard and are often described on the
basis of the amount of excess emissions they produce. For CO and HC,
EPA classifies such excess emitters as high, very high, and super
emitters; for NOx, vehicles are either normal or high.
\10 R. Guensler, "Data Needs For Evolving Motor Vehicle Emission
Modeling Approaches" (Institute of Transportation Studies, Univ. of
California - Davis, Aug. 1993).
\11 Proceedings of the Seventh Coordinating Research Council's
On-Road Vehicle Emissions Workshop, April 9-11, 1997; published June
30, 1997.
\12 "Analysis of Causes of Failure in High Emitting Cars,"
Publication Number 4637, Health and Environmental Sciences
Department, American Petroleum Institute, February 1996.
\13 M. Carlock, "An Analysis of High Emitting Vehicles in the
On-Road Vehicle Fleet," California Air Resources Board, 1993.
\14 This study was sponsored by the American Petroleum Institute
(API).
\15 RVP is a measure of fuel volatility, expressed as pounds per
square inch, with higher pressures resulting in higher volatility
and, therefore, more hydrocarbon emissions.
\16 Auto/Oil Air Quality Improvement Research Program Final Report,
January 1997.
\17 According to a Senior Environmental Scientist, Health and
Environmental Sciences Department, American Petroleum Institute, from
1990 to 1992, average gasoline RVP was about 10 psi.
\18 L. Anderson, R. Jones, P. Wolfe, "Assessing the Effectiveness
of Oxygenated Fuels Use for the Reduction of Ambient Carbon Monoxide"
(Univ. of Colorado at Denver), Seventh CRC On-Road Vehicle Emissions
Workshop, April 9-11, 1997.
\19 According to EPA, 50,000 miles was the regulatory �useful-life�
standard for light-duty vehicles at the time MOBILE5a was developed.
\20 This study was prepared by Sierra Research for API using data
collected under another API-sponsored study, the High Mileage Vehicle
project.
\21 This range results from differences in the model's assumptions
about emissions because of the type of fuel-delivery systems used.
The model estimates that 80 percent of port fuel-injected vehicles
would be high emitters, and that 97 percent of throttle-body
fuel-injected vehicles would be high emitters. Carbureted vehicles
were not a part of this study because this fuel delivery technology
is no longer used on light duty vehicles, according to the study.
\22 D. Berens (Ford Motor Company), H. Haskew (GM Powertrain
Group), R. Ortega (Chrysler Corporation), "FTP Emissions From
1991-1993MY In-Use Vehicles," Seventh CRC On-Road Vehicle Emissions
Workshop, April 9-11, 1997.
\23 All 1994 and later light duty vehicles and light duty trucks must
be equipped with an emissions control diagnostic system capable of
identifying deterioration and/or malfunction of the emissions control
system, referred to in EPA's regulations as on-board diagnostics .
\24 EPA's remote sensing data are limited, but recent, according to
agency officials.
\25 H. Haskew (GM Powertrain Group), "Diurnal Emissions From In-Use
Vehicles," Seventh CRC On-Road Vehicle Emissions Workshop, April
9-11, 1997.
\26 In-use emissions estimates attempt to assess actual vehicle
emissions over time, mileage, vehicle type, and other factors, such
as whether the vehicle is subjected to an I&M program, according to
EPA officials.
\27 Generally defined as vehicles 8,501 pounds or more in gross
vehicle weight.
\28 J. Heiken, B. Austin, and A. Pollack (Environ International
Corporation) and D. Coe, L. Chinkin, and D. Eisinger (Sonoma
Technology, Inc.), "Estimation of Local Fleet Characteristics and
Activity Data for Improved Emission Inventory Development," March 18,
1996.
THE UNCERTAINTY LIMITATION
------------------------------------------------------- Appendix I:0.3
Another significant limitation involves the lack of information about
the range, or magnitude, of the uncertainty\29 associated with the
model's estimates. Uncertainty occurs for several reasons but often
is the result of omissions, or gaps, in the understanding of
emissions-producing activities and of errors in assumptions that are
the result of data gaps or data limitations, as well as mathematical
and statistical variabilities. However, according to the nonfederal
co-chair of the FACA workgroup, information about the uncertainty of
the model's estimates is critical to the ability of policymakers to
make good decisions today and for researchers to address the most
significant areas in route to a better future model. The co-chair
and other model experts told us that the current MOBILE model has no
information about, nor estimates of, the uncertainty associated with
its predictions. According to one researcher, the model "just
produces a number for the user, with no documentation of the
reliability of the output." Similarly, according to a 1994 National
Research Council special report\30 addressing the MOBILE model,
"Uncertainty is pervasive in all three emission modeling
components: vehicle activity, activity-specific emission rates,
and emission rate correction factors. Uncertainty is compounded
in the methodologies used to develop the emission inventory.
That is, vehicle activity uncertainty is combined with emission
rate uncertainty that has already been combined with correction
factor uncertainty."\31
Additionally, the limited work in this area indicates there are
significant uncertainties associated with the current MOBILE model's
estimates. For example, one study\32 found that "the range of
uncertainty is huge" for a change in one variable--average vehicle
speed--of the many variables contained in the MOBILE model.
According to the study, most model users generally believe that
increasing average vehicle speed from 30 mph to 50 mph will reduce
vehicle emissions (because of less congested driving, with more
driving at cruising speeds). The study noted that EPA's MOBILE model
estimates a 24-percent reduction in carbon monoxide emissions by
increasing average vehicle speed from 30 mph to 50 mph. However,
when a 95-percent confidence interval is applied, the change in
emissions can range from a 72-percent decrease in carbon monoxide
emissions to a 75-percent increase in such emissions. Similarly, a
1996 study\33 of EPA's speed correction factors for vehicle exhaust
emissions found substantial uncertainty in EPA's current MOBILE
model. Among other things, the study concluded that the MOBILE model
may significantly underestimate carbon monoxide and hydrocarbon
emissions--"by up to 3 orders of magnitude"--as the model relates to
changes in vehicle speed.
According to Office of Mobile Sources officials, EPA has been unable
to quantify the model's uncertainty primarily because of the cost and
time associated with such quantification, the fact that the on-road
vehicle population is a constantly changing universe of differing
emissions control devices and levels, technological limitations in
measurement devices, and because there is substantial naturally
occurring variability in vehicle emissions (leading to further data
gaps/limitations). Agency officials pointed out that there is
substantial variability across (1) vehicle types (such as model year,
emissions control system, engine type), (2) vehicle operating
conditions (cold start, load, speed), (3) the external environment
(road grade, temperature, humidity, altitude), (4) vehicle fuels
(reformulated, oxygenated, reid vapor pressure), and (5) driver
behavior (quick starts and stops, timing and frequency of trips).
For these reasons, OMS officials told us they do not plan to develop
uncertainty ranges with the next revision to the MOBILE model.
Similar to the global positioning issue for road grade, they said
significant technological advancement may be needed before it becomes
cost-effective to address this issue. For example, future vehicles
may have on-board computers with the ability to instantaneously
record and later report emissions under different operating
scenarios. EPA officials pointed out that it is their plan, at some
point in the future, to report uncertainty ranges for some model
estimates. They said they are currently saving both qualitative and
quantitative descriptors for the data being collected by and for the
agency in order to perform these calculations in the future.
(See figure in printed edition.)Appendix II
--------------------
\29 Traditionally, this has been referred to as the 95-percent
confidence interval, indicating with 95-percent certainty that the
true number is within a specified range.
\30 R. Guensler and D. Sperling, Congestion Pricing and Motor
Vehicle Emissions: An Initial Review, Institute of Transportation
Studies, University of California at Davis, commissioned paper for
the Transportation Research Board, National Research Council,
National Academy of Sciences.
\31 With respect to MOBILE, according to EPA, only two of these
modeling components--emissions rates and correction factors--are
subject to concerns about uncertainty; model users supply their own
vehicle activity data from other sources.
\32 R. Guensler, Vehicle Emission Rates and Average Vehicle
Operating Speeds, Ph. D. dissertation, Institute of Transportation
Studies, University of California at Davis, 1993, excerpted for the
1994 commissioned paper for the Transportation Research Board,
National Research Council, National Academy of Sciences.
\33 M. Kini, Probabilistic Modeling of Exhaust Emissions From Light
Duty Gasoline Vehicles, Graduate Thesis, Department of Civil
Engineering, North Carolina State University, Oct. 1996.
COMMENTS FROM THE ENVIRONMENTAL
PROTECTION AGENCY
=========================================================== Appendix I
(See figure in printed edition.)
OBJECTIVES, SCOPE, AND METHODOLOGY
========================================================= Appendix III
The Chairman, Subcommittee on Oversight and Investigations, House
Committee on Commerce, asked us to (1) describe the major limitations
in the current version of EPA's MOBILE model and (2) describe EPA's
process for improving both current and future versions of the MOBILE
model.
To describe the major limitations in the current model, we obtained
and reviewed the MOBILE5a User's Guide; EPA/OMS model documentation;
the most recent published sensitivity analyses (MOBILE4.1, May 1990);
relevant EPA guidance and memorandums on the MOBILE model; selected
vehicle studies; the results of stakeholders meetings about the
model; and the charter, objectives, minutes, and proceedings of the
EPA-Federal Advisory Committee Act mobile modeling workgroup. We
also reviewed five electronic databases\1 for studies pertaining to
the EPA MOBILE model; attended one mobile sources symposium where
modeling issues were discussed; and attended the March 1997 FACA
public workshop. We also obtained and discussed studies relating to
potential MOBILE model limitations with selected representatives of
state and local agencies, academia, industry, environmental groups,
consulting firms, and other government agencies. Additionally, we
interviewed officials and obtained documents from EPA's Office of
Mobile Sources in Ann Arbor, Michigan; Office of Research and
Development in Research Triangle Park, North Carolina, and Athens,
Georgia; and the EPA Science Advisory Board in Washington, D.C. We
also discussed model limitations with individuals identified to us by
EPA or other representatives, as well as through our own efforts as
noted above.
To describe EPA's process for improving both current and future
versions of the MOBILE model, we obtained and discussed information
from knowledgeable EPA/OMS air quality officials relative to ongoing
activities and documented plans for making model revisions. We also
discussed EPA's past, ongoing, and planned actions with
representatives of academia, industry, environmental groups,
consulting firms, and other government agencies and observed one
process--EPA's open solicitation of input by external stakeholders
not involved in the model's development--at work. We also obtained
documents and discussed EPA's process for improving models in general
with the Science Advisory Board and EPA's Office of Research and
Development.
We conducted our review from October 1996 though August 1997 in
accordance with generally accepted government auditing standards.
--------------------
\1 The electronic databases reviewed included "Enviroline," which
covers more than 5,000 international primary and secondary
environmental publications on all aspects of the environment;
"EiCompendex Plus," the electronic version of The Engineering Index,
which provides worldwide coverage of approximately 2,600 journals and
selected government reports and books on the environment and other
issues; "Pollution Abstracts," a leading resource for references to
environment-related literature on pollution, its sources, and its
control; "Energy Science and Technology," one of the world's largest
sources of literature references on energy and related topics,
including the environment, with coverage of journal articles, report
literature, conference papers, books, patents, dissertations, and
translations; and EPA's Technology Transfer Network, a worldwide
network of electronic bulletin boards providing information and
technology exchange in areas pertaining to air pollution control,
with emphasis on EPA's Mobile Sources Information bulletin board,
which covers information pertaining to mobile source emissions,
regulations, test results, models, and guidance.
MAJOR CONTRIBUTORS TO THIS REPORT
========================================================== Appendix IV
RESOURCES, COMMUNITY, AND ECONOMIC
DEVELOPMENT DIVISION, WASHINGTON,
D.C.
Lawrence J. Dyckman, Associate Director
William F. McGee, Assistant Director
Judy K. Pagano, Senior Operations Research Analyst
James R. Beusse, Evaluator-in-Charge
Hamilton C. Greene, Jr., Staff Evaluator
DeAndrea M. Leach, Staff Evaluator
*** End of document. ***