[Senate Hearing 110-818]
[From the U.S. Government Publishing Office]
S. Hrg. 110-818
OFFLINE AND OFF-BUDGET: THE DISMAL STATE OF INFORMATION TECHNOLOGY
PLANNING IN THE FEDERAL GOVERNMENT
=======================================================================
HEARING
before the
FEDERAL FINANCIAL MANAGEMENT, GOVERNMENT
INFORMATION, FEDERAL SERVICES, AND
INTERNATIONAL SECURITY SUBCOMMITTEE
of the
COMMITTEE ON
HOMELAND SECURITY AND
GOVERNMENTAL AFFAIRS
UNITED STATES SENATE
ONE HUNDRED TENTH CONGRESS
SECOND SESSION
__________
JULY 31, 2008
__________
Available via http://www.gpoaccess.gov/congress/index.html
Printed for the use of the Committee on Homeland Security
and Governmental Affairs
U.S. GOVERNMENT PRINTING OFFICE
44-586 WASHINGTON : 2009
-----------------------------------------------------------------------
For sale by the Superintendent of Documents, U.S. Government Printing
Office Internet: bookstore.gov Phone: toll free (866) 512-1800
Fax: (202) 512-2250 Mail: Stop IDCC, Washington, DC 20402-0001
COMMITTEE ON HOMELAND SECURITY AND GOVERNMENTAL AFFAIRS
JOSEPH I. LIEBERMAN, Connecticut, Chairman
CARL LEVIN, Michigan SUSAN M. COLLINS, Maine
DANIEL K. AKAKA, Hawaii TED STEVENS, Alaska
THOMAS R. CARPER, Delaware GEORGE V. VOINOVICH, Ohio
MARK L. PRYOR, Arkansas NORM COLEMAN, Minnesota
MARY L. LANDRIEU, Louisiana TOM COBURN, Oklahoma
BARACK OBAMA, Illinois PETE V. DOMENICI, New Mexico
CLAIRE McCASKILL, Missouri JOHN WARNER, Virginia
JON TESTER, Montana JOHN E. SUNUNU, New Hampshire
Michael L. Alexander, Staff Director
Brandon L. Milhorn, Minority Staff Director and Chief Counsel
Trina Driessnack Tyrer, Chief Clerk
FEDERAL FINANCIAL MANAGEMENT, GOVERNMENT INFORMATION, FEDERAL SERVICES,
AND INTERNATIONAL SECURITY SUBCOMMITTEE
THOMAS R. CARPER, Delaware, Chairman
CARL LEVIN, Michigan TOM COBURN, Oklahoma
DANIEL K. AKAKA, Hawaii TED STEVENS, Alaska
BARACK OBAMA, Illinois GEORGE V. VOINOVICH, Ohio
CLAIRE McCASKILL, Missouri PETE V. DOMENICI, New Mexico
JON TESTER, Montana JOHN E. SUNUNU, New Hampshire
John Kilvington, Staff Director
Katy French, Minority Staff Director
Monisha Smith, Chief Clerk
C O N T E N T S
------
Opening statement:
Page
Senator Carper............................................... 1
WITNESSES
Thursday, July 31, 2008
Hon. Paul A. Denett, Administrator, Office of Federal Procurement
Policy, U.S. Office of Management and Budget................... 5
Hon. Karen S. Evans, Administrator, Office of Electronic
Government and Information Technology, U.S. Office of
Management and Budget.......................................... 7
David A. Powner, Director, Information Technology Management,
U.S. Government Accountability Office.......................... 8
Alfred Grasso, President and Chief Executive Officer, MITRE
Corporation.................................................... 23
Norm V. Brown, Executive Director, Center for Program
Transformation................................................. 26
Thomas M. Jarrett, Secretary, Department of Technology and
Information, State of Delaware................................. 28
Alphabetical List of Witnesses
Brown, Norm V.:
Testimony.................................................... 26
Prepared statement........................................... 111
Denett, Hon. Paul A.:
Testimony.................................................... 5
Prepared statement........................................... 39
Evans, Hon. Karen S.:
Testimony.................................................... 7
Prepared statement........................................... 45
Grasso, Alfred:
Testimony.................................................... 23
Prepared statement........................................... 100
Jarrett, Thomas M.:
Testimony.................................................... 28
Prepared statement........................................... 120
Powner, David A.:
Testimony.................................................... 8
Prepared statement........................................... 50
APPENDIX
Questions and Responses for the Record from:
Mr. Denett................................................... 124
Ms. Evans.................................................... 128
Charts submitted by Senator Carper............................... 134
OFFLINE AND OFF-BUDGET: THE DISMAL STATE OF INFORMATION TECHNOLOGY
PLANNING IN THE FEDERAL GOVERNMENT
----------
THURSDAY, JULY 31, 2008
U.S. Senate,
Subcommittee on Federal Financial Management,
Government Information, Federal Service,
and International Security,
of the Committee on Homeland Security
and Governmental Affairs,
Washington, DC.
The Subcommittee met, pursuant to notice, at 9:33 a.m., in
room SD-342, Dirksen Senate Office Building, Hon. Thomas R.
Carper, Chairman of the Subcommittee, presiding.
Present: Senator Carper.
OPENING STATEMENT OF SENATOR CARPER
Senator Carper. The Subcommittee will come to order. Our
thanks to our guests and witnesses for being here with us
today. This is the third hearing that our Subcommittee has held
on the issue of poorly planned and poorly performing IT
investments. This hearing will focus once again on the ability
of the Office of Management and Budget to oversee and provide
Congress with visibility into the $70-plus billion that
agencies will spend on information technology this year.
Up until March of this year, Congress had an extremely
limited ability to understand why OMB considered an IT
investment to be poorly planned, and I must commend OMB and in
particular Ms. Evans for finally releasing this data. Although
I believe this is a good start, more complete and accurate
information needs to be shared. I firmly believe that in order
to hold agencies accountable for their investments, Congress
and OMB need to work together as partners. The American
taxpayers demand it.
Information technology investments, if planned and
implemented properly, can increase productivity, improve
efficiency, and reduce an agency's cost, and also enable us to
provide better service to our constituents. However, some of
these projects can be extremely difficult to manage and
mistakes may be made along the way.
In fact, I think it was Richard Nixon who said the only
people who don't make mistakes are the people who don't do
anything. So we know if we try to do something in these complex
areas, we are going to make mistakes. That is to be expected.
The key is, I think, to make sure that we don't continue to
make the same mistake over and over again. However, the
mistakes that we do make, we learn from them.
I experienced this firsthand when I was privileged to serve
as Governor of Delaware. Sometimes we bit off more than we
could chew and the IT project would eventually spiral out of
control. When this happened, sometimes we came to the
conclusion that the best course of action was just to finally
pull the plug. It was a tough decision a lot of times, but it
was often the right thing to do in certain cases.
Unfortunately, many agencies in the Federal Government are
allowed to spend billions of taxpayer dollars on investments
that are duplicative, that lack clear goals, and that are
managed by unqualified individuals. In fact, according to
recently-released GAO data, some $25 billion in IT investments
are poorly planned, poorly performing, or both. Even worse,
some of these projects have been delayed up to a decade and are
costing us billions more than was originally expected. This is
simply unacceptable and it makes me wonder whether it is time
for Congress to pull the plug on some of these failed
investments.
Regrettably, Congress still does not have the information
necessary from OMB to hold agencies accountable and to choose
where we want to invest scarce resources each year. With risky
investments such as IT, it is important to increase
collaboration and visibility, not hinder it. And so far, this
has not always been the case.
Since 1994, Congress through legislation called the Federal
Acquisition Streamlining Act has required agencies to keep
costs, delivery dates, and performance goals of major
acquisition within 90 percent of the originally proposed plan.
OMB then was required to annually report to Congress on agency
progress.
Unfortunately, despite this requirement in law, OMB has, I
am told, only issued three reports in the last 14 years.
Although my staff tells me we may have received a report around
midnight last night. Someone here was staying up late and I
think may have received a fourth report in 14 years in the
midnight hour.
Moreover, agencies are required to create an investment
baseline that takes into account potential risks that could
lead to increased costs, delayed delivery dates, and reduced
performance. Agencies then use this baseline to track whether
an investment is progressing according to plan.
There are obviously some legitimate reasons why an agency
may change the original baseline on a given project, but I am
disappointed to say that some agencies have used rebaselining
to hide the cost overruns or schedule delays from Congress.
We are about to hear an extremely troubling report from the
Government Accountability Office today revealing that almost 50
percent of all Federal agency IT investments are rebaselined.
Even more disturbing, some agencies, such as the Department of
Agriculture, the Department of Commerce, Veterans Affairs, have
rebaselined, I understand, more than five times on a single
investment.
Although agencies are responsible for the excessive
rebaselining, there is one thing in common between all of these
investments. Every baseline and rebaseline was approved by OMB.
Someone, somewhere, in my view, is not fulfilling their
responsibility to ensure the taxpayer dollars are spent only on
those investments that are well thought out and truly needed.
Using the information that was provided to our Subcommittee
in March, we have created a report card for agencies that take
into account several criteria related to the planning and
implementation of IT investments. In fact, we have two report
cards side by side. I say this as a parent whose children have
both now graduated from high school and who is used to getting
report cards. Sometimes my children aren't anxious for us to
see these report cards. But on the report card on the left, we
have a list of agencies who received a passing grade, and on
the right, those who receive a failing grade. If anybody in the
audience is able to read those, I take my hat off to you. We
have copies of the report cards that are provided to our
witnesses and I hope to the press.
As we can clearly see the Federal Government is miserably
failing. In fact, half the 28 agencies received an F. Let me go
ahead and list some agency grades starting with the Department
of Labor, whose score was about 64 percent. The Department of
Treasury, 56 percent. Office of Personnel Management, 55
percent, all the way to the bottom of this chart on the
right,\1\ down to the Department of Defense with 37 percent,
followed by the Department of Agriculture with a 36 percent
score.
---------------------------------------------------------------------------
\1\ The chart referred to appears in the Appendix on page 135.
---------------------------------------------------------------------------
About half of the 28 agencies received an F. In total,
these failing agencies are overseeing $57 billion in IT
investments, and from what we can tell right now, we are
getting too little in return for those enormous investments.
In addition, you might notice on the left, we have a number
of agencies who receive a passing grade. Several received
scores of A-plus. They included HUD, the National Science
Foundation, the Smithsonian Institute--they all got 100
percent--the Social Security Administration, 99 percent,
Department of Energy, 93 percent, all the way down to the
bottom of the chart, coming in at a 69 percent for a D-plus,
and that is the Department of Justice.
However, and while we are encouraged by the passing grades
in the report card on the left, we want to temper our joy by a
couple of things. HUD received their A-plus partly because they
reported that every project was being delivered on cost, on
schedule, and performing as planned. However, as GAO will
testify today, HUD rebaselined at least one project seven
times, possibly to mask spiraling costs, and GAO has
consistently testified that data provided by agencies to OMB is
oftentimes inaccurate or even incomplete.
So we don't really have the complete picture, even for
those agencies that are reportedly doing a better job than
others. That is why I plan on introducing legislation today,
along with Senator Lieberman, Chairman of our full Committee,
and Senator Collins, the Ranking Republican on the full
Committee, which will give Congress and OMB the information we
need to make better decisions about which IT investments should
continue and which should be shut down.
Our bill, called the Information Technology Investment
Oversight Enhancement and Waste Prevention Act, would make
agencies report regularly on significant deviations on cost,
schedule, and performance. But we don't just want better
information. Our bill also helps OMB take a crucial step aimed
at preventing IT investments from drifting toward failure.
Recognizing that agencies may not have the skills necessary to
manage complex IT investments and may have trouble recruiting
qualified managers, our bill would set up a team of experts
from inside and outside of government that agencies may use as
a resource. This team, which my staff and I regard as something
of an IT strike force, would have the skills and background
necessary to make sure that agencies are focused on the right
things, making the right decisions, and spending the money
wisely.
I again want to thank our witnesses for joining us today.
We look forward to hearing from you. We look forward to your
testimony. We look forward to the discussion that will follow
it and hopefully to the better IT performance, for the money
that our taxpayers are investing in those projects today.
Taxpayers expect us to be good stewards of their money. In
fact, they demand it, and I know that everyone here in this
room wants to see that become a reality.
Before I introduce Mr. Denett, Ms. Evans, and Mr. Powner,
none of whom are strangers to this Subcommittee or to this
room, I realize these are tough issues. These are not easily
done. If they were, somebody would have done them a long time
ago. The problems were recognized not just in this
Administration, but in the last Administration, as well. While
we appreciate that the problems are being identified, what is
really important is that we identify them more quickly and that
we fix them. I believe that we need good information here in
the Legislative Branch so that we can be better partners with
the Executive Branch.
We are going to see a new Administration coming in in 6
months. It is going to be President McCain or President Obama.
But these problems and these challenges will still be before
the next Administration and the next Congress.
As Governor, I used to say the reason why we invested money
in these IT projects was to enable us to provide better
constituent service and to do it more cost effectively, and
that is really the case here, too. In some areas, we are doing
a pretty good job. In too many areas, though, we are not. In
too many areas, we are masking mismanagement, misallocation of
resources and cost overruns in ways that make it look like we
are doing a better job on some projects than we truly are. We
have to get beyond that.
With that having been said, on that cheery note, we have a
couple of panels before us. In the interest of time, I am just
going to move right to the first panel for their testimony.
Biographies of our witnesses are provided and will be submitted
for the record.
Our first witness is the Hon. Paul Denett. Mr. Denett
serves as the Administrator of the Office of Federal
Procurement Policy at the Office of Management and Budget.
Previously, Mr. Denett served as counselor to Clay Johnson, who
is now the Deputy Director for Management at OMB. Mr. Denett is
a retired Senior Executive from the Federal service and has
received many prestigious awards, including a Presidential Rank
Award. When did you receive that?
Mr. Denett. I received that in the late 1990s.
Senator Carper. Did you really? Who was President then?
Mr. Denett. President Clinton.
Mr. Denett. Yes, it was.
Senator Carper. All right. Some of us are lost in the
1960s. [Laughter.]
Some days, I would like to be, but not today. [Laughter.]
All right. Our next witness is the Hon. Karen Evans. Ms.
Evans, welcome. She is Administrator of the Office of
Electronic Government and Information Technology at OMB. Ms.
Evans oversees the implementation of IT throughout the Federal
Government, including capital planning and investment control.
She is a 20-year veteran of government service and has
testified before this Subcommittee on a number of occasions. It
is a pleasure to have you join us again today. Thank you.
Our final witness is Dave Powner, Director of Information
Technology Management at the Government Accountability Office.
I am glad that Mr. Powner doesn't charge us for each visit that
he makes and each time he testifies before this Subcommittee,
because we would be owing him a lot. But in the private sector,
he has held several executive level positions. In the
telecommunications industry, Mr. Powner has been instrumental
in helping this Subcommittee provide oversight of risky IT
investments, along with other things. I know he has appeared
before us on several occasions, including as we try to address
our current challenges involving conducting a good Census for
the year 2010.
We are going to lead off and ask Mr. Denett to lead off. We
will ask you to keep your statements to roughly 5 minutes. If
you go a little bit beyond that, that is OK. If you go a lot
beyond that, that is not OK. So we will turn to you now. As you
present your testimony, just keep in mind that your full
statements will be entered into the record.
TESTIMONY OF HON. PAUL A. DENETT,\1\ ADMINISTRATOR, OFFICE OF
FEDERAL PROCUREMENT POLICY, U.S. OFFICE OF MANAGEMENT AND
BUDGET
Mr. Denett. Thank you. Chairman Carper, Ranking Member
Coburn, and Members of the Subcommittee, I appreciate the
opportunity to appear before you today. The Administration
places a high priority on working with agencies to mitigate
cost overruns, schedule delays, and performance shortfalls in
their major acquisitions.
---------------------------------------------------------------------------
\1\ The prepared statement of Mr. Denett appears in the Appendix on
page 39.
---------------------------------------------------------------------------
This morning, my colleague, Karen Evans, will discuss the
initiatives the Administration is pursuing to effectively
manage the government's IT portfolio. I would like to briefly
summarize agencies' progress in implementing performance-based
management for their major non-IT acquisition programs. I have
also prepared written remarks that I ask the Subcommittee to
enter into the record.
Similar to IT, major acquisitions of non-IT capital assets
must be justified in terms of agencies' strategic goals and
reflect sound acquisition and capital planning decisions. The
law requires agencies to apply performance-based management
principles by establishing cost, schedule, and performance
goals and achieving 90 percent of these goals, on average.
OMB guidance for meeting these requirements is set forth in
Circular A-11, which provides guidance to agencies on preparing
their budget submissions, and the Capital Programming Guide,
which is a supplement to A-11. The guide was substantially
revised in 2006 to emphasize the importance of key steps in the
acquisition planning process, such as needs assessment and
alternative analysis.
To assess the use of performance-based management, my
office directed agencies to provide information on their new
and ongoing non-IT major acquisition projects. We look for
several indicators of progress, including the existence of
capital planning and investment control policies, cost schedule
and performance goals for new and ongoing projects, use of
performance-based management systems to monitor progress and
success in meeting goals.
Results were mixed. While many agencies demonstrated
progress, we found that capital planning policies for non-IT
are often not as well established as they are for IT
investments. In addition, performance-based management systems
are not always being used to track cost schedule and
performance. We can and must do better.
A number of steps are being taken to strengthen the
application of performance-based management to non-IT
investments. First, the Chief Acquisition Officers' Council
created a Project Management Working Group to help OMB evaluate
the appropriate application of performance-based management to
different types of major non-IT investments, such as
construction and aircraft.
Second, OMB and GAO are partnering with DOD, DOE, and NASA,
each of whom is on the GAO's High-Risk List, in the development
and implementation of corrective action plans to mitigate risk
in major acquisitions and evaluate success against clear goals
and metrics.
Third, the Office of Federal Procurement Policy and the
Federal Acquisition Institute launched a Federal Acquisition
Certification Program to provide civilian program and project
managers with standardized training on competencies critical to
successful performance-based management. These include
requirements analysis, cost estimating financial management,
risk management, and quality assurance. Program and project
managers that are assigned to major acquisitions will need to
be certified under this program.
Finally, OFPP created a standardized approach for agency
self-assessments of the acquisition function based on a
framework that was created by GAO. Self-assessments of major
acquisition functions will address issues to help agency
managers understand if planning is effective. If a material
weakness is identified, it will be addressed during corrective
action monitoring and reflecting on the agency's statement of
assurance prepared under OMB Circular A-123, which lays out
management's responsibilities for internal controls.
In accordance with the Federal Acquisition Streamlining
Act, OFPP issued a report--I think it was a little before
midnight, but it was late--discussing civilian agency progress
in implementing performance-based management----
Senator Carper. Actually, I am told it was 12:04.
[Laughter.]
But who is keeping time. [Laughter.]
Mr. Denett. OK. For both IT and non-IT investments. We will
be happy to answer any follow-up questions the Subcommittee may
have. Thank you.
Senator Carper. Right on the money. Thank you. Ms. Evans,
welcome.
TESTIMONY OF KAREN S. EVANS,\1\ ADMINISTRATOR, OFFICE OF
ELECTRONIC GOVERNMENT AND INFORMATION TECHNOLOGY, U.S. OFFICE
OF MANAGEMENT AND BUDGET
Ms. Evans. Good morning, Mr. Chairman. My remarks will
focus on the Administration's strategy and continued progress
in performing oversight of agencies' capital investments and
information technology. Specifically, I will address the topic
of OMB's ability to effectively analyze, track, and evaluate
agencies' major capital IT investments.
---------------------------------------------------------------------------
\1\ The prepared statement of Ms. Evans appears in the Appendix on
page 45.
---------------------------------------------------------------------------
The President's budget request for fiscal year 2009
includes approximately $71 billion for IT and associated
support services government-wide. Of the $71 billion in the
President's budget request, $22 billion, or 31 percent,
represents proposed funding for development, modernization, or
enhancement of new or existing information systems,
infrastructure, or services. Projected expenditures on existing
systems, operations, and maintenance is $49 billion, or 69
percent.
As I have discussed in previous testimony on the topic of
OMB's oversight of IT investments, the Clinger-Cohen Act of
1996 established processes for executive agencies to analyze,
track, and evaluate the risks and results of major capital
investments for information systems. The operative means by
which OMB evaluates agency capital asset plans and their
associated budgetary requests is a business case, or Exhibit
300, for individual investments.
Agencies develop and submit business cases with their
annual budget requests to OMB and we, in turn, evaluate each
business case in terms of its ability to support a given
investment proposal, including factors such as alignment with
IT architecture, plan performance, improvement goals, cost-
benefit analysis, and eliminating costly, duplicative, and
outdated systems. OMB also evaluates the capability to manage
the investment as demonstrated in the business case and
planning process, including factors such as having a qualified
project manager, acquisition planning, systems security, risk
management, and the use of earned value management to track and
manage costs and schedule goals.
Last year, I came before this Subcommittee to explain two
specific tools OMB uses to track and review agency IT capital
investments, the Management Watch List and the High-Risk List.
I am pleased to report that we have since expanded upon our
criteria for evaluating investments and improved upon the
transparency of how investments and projects are placed on
either or both lists. My written statement includes the details
regarding these lists.
GAO's report on project rebaselining points out weaknesses
in agencies' policies in terms of specifying all of the
elements of rebaselining according to best practices. We
acknowledge more should be done in this area in the future. In
particular, OMB needs to clarify expectations for when an
original baseline should be established and elaborate upon
process steps for agencies to submit to and receive feedback
back from OMB on rebaseline requests.
OMB, Congress, and the agencies must work collaboratively
to address weaknesses in IT program and project performance. We
are all vested in a common interest, delivering results for the
American people.
How can we best do this going forward? I believe OMB has
the foundational processes in place to perform program and
project investment oversight at the macro level. These core
processes--capital planning, architecture alignment, the
Management Watch List, the High-Risk List, and the E-GOV
scorecard--have been and can continue to be further enhanced
and incrementally improved to provide better visibility into
program performance before a project is in major trouble.
One improvement for OMB would be to leverage the efforts of
the budget formulation and execution line of business to ensure
the use of analytical tools and the collaboration environment
to improve on our own information management capabilities. The
various data sets collected by OMB from the agencies can and
should be better integrated into a more comprehensive knowledge
base.
I have brought a display of an example of how we could
better integrate and expand upon information on the agency's IT
investments. This is one potential snapshot of project
performance at a point in time. We would like to work with
Congress to improve transparency and the ease of use of this
information. With the transparency into departments' and
agencies' performance will come improved accountability and
results which the American people deserve.
Thank you for this opportunity to discuss the
Administration's strategy and progress to date. With this
foundation in place, we can continue to work together to
achieve the outcome we both desire, which is successful
implementation of information technology for program and
mission results. We have accomplished a lot in the last 8 years
and there is much more that we can continue to do in this area.
I would be happy to take questions at the appropriate time.
Senator Carper. Thank you for that testimony. Mr. Powner.
TESTIMONY OF DAVID A. POWNER,\1\ DIRECTOR, INFORMATION
TECHNOLOGY MANAGEMENT, U.S. GOVERNMENT ACCOUNTABILITY OFFICE
Mr. Powner. Chairman Carper, we appreciate the opportunity
to testify this morning on poorly planned and performing IT
projects and the results of our rebaselining review completed
at your request.
---------------------------------------------------------------------------
\1\ The prepared statement of Mr. Powner appears in the Appendix on
page 50.
---------------------------------------------------------------------------
Mr. Chairman, I would like to thank you and Dr. Coburn for
your oversight of the Federal IT budget, which now exceeds $70
billion. This is the third annual hearing, and your oversight,
along with the leadership of Karen Evans, has resulted in a
clear picture of exactly what projects need work and the
specific weaknesses that need to be addressed. That is the good
news. The bad news is that we have nearly $25 billion in IT
investments that are currently at risk of being wasted and that
this figure is likely much higher.
This morning, I have three points to make. First, over 400
IT projects totaling nearly $25 billion are currently not
appropriately planned for or managed.
Second, the number of projects reporting issues with cost
and schedule variances is understated because rebaselining is
occurring excessively without adequate guidance or
transparency.
And third, additional oversight from both OMB and agency
CIOs is needed to address project weaknesses and to bolster
accountability.
Expanding on each of these, first, currently, there are
about 350 projects totaling $23 billion on the Management Watch
List, and nearly 90 projects totaling $5 billion that are being
reported as high-risk projects with shortfalls. Common to both
lists are 26 projects totaling about $3 billion, meaning that
these projects are both poorly planned and performing. For
example, DHS's Secure Border Initiative project continues to be
on both lists.
In addition to the 26 projects on both lists, of particular
concern are the repeat offenders, namely 32 projects that have
been on the Management Watch List since September 2006 and 17
high-risk projects reporting shortfalls in each of the last
four quarters. My written statement highlights each of the
projects on both lists and those that have been on either list
far too long.
Mr. Chairman, having so many projects with longstanding
planning and performance problems is unacceptable and requires
more attention from both OMB and agency CIOs. Key reasons why
projects remain on the Management Watch List are poor cost and
schedule performance, poor security measures, poor privacy
planning, and poor project management, while cost and schedule
variances continues to be the primary performance shortfall
associated with high-risk projects.
Despite cost and schedule performance being a major reason
why so many projects are highlighted here, our rebaselining
report being released today raises significant questions about
whether we are getting an accurate picture of project cost and
schedule performance.
First, I would like to mention that rebaselining is needed
at times to reset realistic cost and schedule targets. However,
this process should not be used to mask cost and schedule
overruns and should be transparent and approved by both OMB and
agency management. Our survey of nearly 200 projects showed
that about half of all IT projects had been rebaselined, and of
those that are rebaselined, about 50 percent are rebaselined at
least twice, and 10 percent are rebaselined four or more times.
Clearly, these projects are operating nowhere near the desired
10 percent cost and schedule threshold.
Because of the amount of rebaselining occurring across the
Federal Government, we reviewed each of the major 24
departments' rebaselining policies to ensure that it included
basic items like specifying when a rebaselining is warranted,
how a new rebaseline is validated, and who approves the new
rebaseline. Agency policies were weak across the board and we
recommended that OMB issue rebaselining guidance and that each
agency develop rebaselining policies that address the
weaknesses we identified.
Although we have more data than we have ever had
historically, we still need better information so that all
projects requiring attention are highlighted. For example, OMB
could publicize high-risk projects with shortfalls as we have
in our written statement. And we still need more transparency
on rebaselining efforts so that we have a true picture of cost
and schedule performance from all agencies.
Put more simply, Mr. Chairman, do we think that of the 472
high-risk projects only 70, or 15 percent, have cost and
schedule issues? No. That number should clearly be higher.
Now turning toward solutions, the Federal Government needs
to focus on addressing root cause problems: Security, project
management, risk management, and cost and schedule performance.
We are encouraged by OMB's efforts to address some of these
weaknesses, like requiring techniques like earned value
management to improve agencies' cost and schedule performance,
but there is much more work ahead. Agency CIOs and OMB from a
government-wide view need to aggressively attack these problem
projects by starting with those that have been on OMB's radar
far too long.
Next, longer-term improvement efforts need to be pursued
and these, at a minimum, need to focus on bolstering the IT
workforce and addressing root causes more aggressively. I look
forward to suggestions from our second panel of experts on
other approaches.
In summary, Mr. Chairman and Dr. Coburn, knowing what to
fix is the first step, and we wouldn't have this information
without your leadership. Thank you. Keeping the current
momentum and energy and focusing more on solutions will be
essential as we transition to a new Administration.
I would be pleased to respond to questions.
Senator Carper. Good. Thank you for that testimony. Thanks
very much for the work that preceded it. On behalf of both
Senator Coburn and myself and our Subcommittee and staff, thank
you for the very good work that you have done.
What I hope we focus on more today--yes, the sun is coming
up. I don't know if it seemed dark to the rest of you, but it
seemed dark to me. I just asked my staff to brighten the
lights. So if you are not awake, wake up now. [Laughter.]
I was wondering, who are some good models on the public or
private sector for the Federal Government? Who out there is
managing IT projects well? Not just in planning and developing
them, but implementing them on time and fairly close to budget
without rebaselining six or seven times on a single project? Do
we have any idea who is doing an especially good job and why?
Mr. Powner. I can tell you from our focus on some of the
high-risk modernization efforts that we look at, even though
some of these organizations received low department grades, if
you look at pockets within the Federal Aviation Administration,
if you look at IRS, recently there have been successes with
delivering more projects on time and within budget. And if you
look at what has resulted, there has been an extreme focus on
improving their processes, executive governance over those
projects, and one other technique that seems to have worked at
some of those agencies is putting in place Centers of
Excellence where project offices can go for help, where there
is best practices and things they can emulate across the board.
They are not perfect by any means, but there are pockets of
success in both of those agencies.
Senator Carper. Thank you for that response. In the
National Governors Association, all 50 governors are members of
that, we had a Center for Best Practices and if they had a
particular State that was doing a good job in early childhood,
the rest of us could learn from them. If we had a particular
State that was doing a good job in reducing recidivism in the
prison population, or if a State was doing an especially good
job in providing better outcomes in our schools, we had the
opportunity through the Center for Best Practices to see who
was doing a good job and learn from them. Do we have that kind
of ability in the Federal Government?
Ms. Evans. Yes, sir. Through the CIO Council, we have a
Best Practices Committee where we do take on topics such as
these specific areas, like who has implemented earned value
management really well and can people learn from that. We need
to do more in those Best Practices Committees, and I would also
like to build off of what Mr. Powner talked about.
What we have been doing from an OMB perspective is really
zeroing in in particular areas. The two areas that Mr. Powner
mentioned are also on the GAO High-Risk List. So there is a lot
of work that we have done in conjunction with the agency, in
conjunction with GAO, so that we could then cross-pollinate the
expectations, the best practices in those areas, and then the
intent is to take that and then share it back across the board
with other agencies.
So we have done that in the area of, with NARA and the
Electronic Archives Initiative. As they go forward, what we
have learned from FBI on their Sentinel project is they have
improved. We have then partnered them up with other agencies as
they are going forward on major investments so that they make
sure that they don't make the same mistakes.
Senator Carper. Mr. Denett.
Mr. Denett. We also have the Chief Acquisition Officers
Council, and this past year to try to respond to the problems
in the program project management area formed a new working
group. It is chaired by Bill McNally, who is the Senior
Procurement Executive of NASA. But all the major departments
send representatives and they are sharing best practices and we
are going to post them on the Federal Acquisition Institute
website.
Senator Carper. All right. Mr. Powner.
Mr. Powner. I would just like to add that I agree with the
comments that were made here, but one important thing that I
think Ms. Evans made, an important comment, is the
implementation of those best practices. Earned value
management, a very good technique when used appropriately. Our
preliminary work on earned value management implementation
across the government is that it is rather immature, and I will
give you an example.
We just completed a review of FAA. It is likely one of the
better agencies when it comes to implementing earned value.
There has been a lot of focus and attention. But I can tell you
there that a quarter of their projects don't use earned value.
Not everyone is trained. And when you start looking at the
data, the earned value data--and we did, we dove deep on a
couple projects--we saw quite a few problems that we pointed
out.
Now, granted, that is probably one of the better Federal
agencies, but when we look at the cost and schedule problem
that we have in the Federal Government coupled with
rebaselining, we have to get much better at using these
techniques like earned value to make good management decisions
to get early warning indications.
Senator Carper. All right. Thank you. The next question I
want to ask, and I have touched on this already, but we started
this hearing off with a little broader context than I think we
had originally planned to discuss, but it is something that I
am troubled by and I just want to come back to it again and
discuss it.
One of the Subcommittee's purposes is, as you know, to
prevent waste, fraud, and abuse. In order to do that, we need
information on how agencies are managing our investments. Mr.
Denett, in 1994, Congress directed your office to provide a
report on agency progress in delivering on their capital
investments, including IT investments. However, as I stated in
my opening statement, in 14 years, OMB has only provided this
report on three separate occasions, now maybe four. Let me just
ask, why is OMB not fulfilling this critically important
statutory requirement?
Mr. Denett. It was a mistake on our part, for sure. When I
became aware of this, we have had the staff work diligently, go
out to departments and pull it in, and I don't have a good
explanation as to why some years were not submitted. However,
there is the annual budget process under A-11 where a lot of
the data that is included in the report is submitted with the
President's budget each year, so that contains some of the
information we sought. But it should have been wrapped up, put
into a formal report, and provided to you each year. We have
now put steps in to make sure that this doesn't happen in
future years.
Senator Carper. What can you do? Who is responsible for
making sure this statutory requirement is met?
Mr. Denett. The Office of Management and Budget, so we are.
Senator Carper. All right. What assurance can you provide
for us that it is going to be met in the future?
Mr. Denett. We have added it to a follow-up list that OMB
has so that the new Administration will have it as a reminder
before them. We have told the staff the importance of it, and
it is not our prerogative to decide what years to submit it and
what not, that we all need it as a tracking tool. This one will
be stronger. We are going to have our own baselines on which
ones are--just as you are having your report card, we are going
to have that kind of method in there, and I believe in the
future, you will get the reports each year.
Senator Carper. All right. I certainly hope so.
Ms. Evans, this is the third hearing during which you have
testified how billions of taxpayer dollars are wasted on bad IT
investments. I want to commend you for all the progress that
you and your team have made over the past several years.
However, I believe that there is still more that needs to be
done. I am sure you do, too. In fact, you said that.
Specifically, in 2006, GAO first recommended that OMB
ensure agencies are accurately and reliably reporting on their
investments. Also, GAO recommended that OMB provide Congress a
single aggregate list for troubled IT investments that can
allow us to track progress and performance government-wide, and
it is now 2 years later.
Just revisit with us, if you will, what specific progress
since 2006 have you and your colleagues made toward ensuring
that agencies are reporting complete and accurate information
to OMB. And second, why doesn't OMB publish the list of high-
risk projects with shortfalls as GAO highlights in their
testimony? I am just asking, don't you feel that Congress would
want to know whether projects are over-budget and behind
schedule?
Ms. Evans. Sir, I actually would like to answer the first
question first----
Senator Carper. Please.
Ms. Evans [continuing]. Which would be what we have done to
increase the transparency in this. And I would say, first off,
with the hearing and everything and the several hearings along
this line, that I was hesitant at first, which I think you
would admit, to releasing all of this information because we
were concerned about some of the effects that it would have.
Senator Carper. What do you mean----
Ms. Evans. I was going to go into this a little bit. And
some of it, you are talking about and we have kind of gone
around it a little bit, which is driving compliance versus
actually achieving the results. And so things happen like the
grades, not that this is not great because OMB has an A, so I
am pleased that we have an A. But it is an A-minus. I would
have preferred an A-plus, but I know why it is a minus. But
what this will do is drive----
Senator Carper. I asked my staff if we were grading on a
curve here. They said, no, we are not.
Ms. Evans. No, but I don't have as many investments as
everyone else, so----
Senator Carper. We are calling them as we see them.
Ms. Evans. But part of the issue is, one, that there is a
part of us that love to see the grades, and so we strive for
this. This is how our society is built. But the other part then
drives compliance, where people will do just enough in order to
get to the next level of the grade and they are focused more on
the grade than actually achieving the result or getting the
project implemented.
And so this is a balance that my office has really been
struggling with, about how much information do we release, how
much shame and embarrassment do you bring upon an agency,
because we are really supposed to be helping them. We are
supposed to be there to help the agency achieve success. We are
supposed to take areas in our knowledge that we see across the
government and go in and help an agency achieve results.
So I agree that all this information should be out there. I
don't necessarily disagree that we could provide better
reporting, not necessarily along the lines of GAO, because I
don't view OMB as an auditor. I view OMB as a facilitator to
help the Executive Branch achieve results.
So I would like us to be able to provide the information in
a way that you would find it useful as an oversight entity the
same way that we find it useful so that we could get to the
solution for the agency and for the taxpayers.
So we have done a lot to put all this out here. We have not
put together the comprehensive list the way that GAO does or
the way that it was written. I brought two examples, which are
also in the written testimony, and if we want later on, I could
go into a couple areas of how it could be done. We put a lot of
effort into this so that we could match and cross-reference and
make sure that all of this is reliable data, and then we could
work with you in a way similar to the way that we have done
with USA Spending and even release some of the raw data so that
as you go forward, you guys could manipulate and do things with
the data, as well.
So I am always very cautious as we go forward on this
because I am always really cautious about driving compliance
versus trying to get to the result.
Senator Carper. All right. A related question. You may have
spoken to this, but is there a reason why OMB hasn't provided
Congress with a single aggregate list that will allow us to
provide proper oversight and track progress? Just try that. I
think you have answered that question, but I want you to take
another shot at it.
Ms. Evans. I was hoping you didn't realize I didn't answer
that part---- [Laughter.]
But I will tell you, the reason why, and this is our
first--the tables that you have are our first attempt at
aggregating the data together. The short answer is that based
on what we had previously to provide the aggregated report, we
believe would cause more confusion than actually bring clarity
to the result. And then the other piece was we had to work a
little bit more with the agencies to get better quality data in
order to establish the relationship to do an aggregate report.
I believe we are there now and what we would like to work with
you on is how to actually provide that information in a way
that would be useful for your use.
Senator Carper. Why not just provide the same information
as GAO?
Ms. Evans. Because I would say that right now, we have
several pieces of that, but we don't go in and do full audits
the same way that GAO does. So some of the information, for
example, like the number of times that a project has been
rebaselined, we would have to modify some of our A-11 guidance.
Right now, we just capture that information as a yes and no,
and I have not captured that information on an ongoing basis so
that I could go down and map all the records and say that this
investment has been rebaselined several times, X, Y, Z, and
this year, those types of things like that.
I have not kept the information at the micro level. The
agencies are supposed to keep it at that level. And what we are
trying to do is make sure that there is better quality as we
start putting it forward. So I would be hesitant to publish the
information because I do not feel that we have the quality
still at that real detailed level because we haven't conducted
the audit as GAO has.
Senator Carper. I am going to ask Mr. Powner to comment on
some of what Ms. Evans has just said.
Mr. Powner. A couple thoughts here. One, first of all, I
think Ms. Evans and OMB, they deserve a lot of credit for all
the information we have on the Management Watch List, which are
poorly planned projects. OK, you want to fix the planning up
front.
But the High-Risk List is not that important. What is
important is Figure 4 on page 19 in my testimony, which shows
those projects that have shortfalls and the reasons for the
shortfalls. So the No. 1 reason why we have shortfalls, and
this is about performance now, it is not just about planning,
but we are expending funds and we have shortfalls. We are not
meeting cost and schedule goals. There are 70 projects. What I
am hearing is I think the Administration--I am going to cut
right to it--they are reluctant to highlight projects with
shortfalls.
Senator Carper. Why do you suppose that is?
Mr. Powner. I think the discussion was along the lines of
embarrassing agencies and that type of thing. You can't fix
problems----
Senator Carper. There is a reluctance to embarrass them?
Mr. Powner. I believe so, but I don't want--I will let OMB
comment further on that. I don't think you can fix shortfalls
without fully disclosing all your problems, so you need to take
those shortfalls--we have 10 percent--we have 70 projects that
aren't meeting the 10 percent threshold. That number is greatly
understated, and I will tell you why. Earned value
implementation is weak at many agencies, so some of that data
is not based on reliable information coming from the agencies,
and then the point about rebaselining, OK.
If we had transparency on rebaselining--that number right
there says that 85 percent of the high-risk projects are
meeting the 10 percent cost and schedule threshold. It is
likely--that 85 percent, you are walking on water if you are a
private, public organization, 85 percent of your projects are
meeting within 10 percent. You are doing very well, extremely
well.
So we need accurate data there, but we need to disclose all
that information in order to fix those shortfalls, so that is
why we, in our testimony, highlighted the important bit of
information here are the high-risk projects with shortfalls so
that we can fix performance. I actually think this is very
consistent with the Administration's President's Management
Agenda and their E-GOV scoring. One of their criteria to get to
green is you operate within 10 percent of costs and schedule
using earned value techniques. So we are all talking the same
thing here and we have the same goal. It is a matter of
disclosing all the weaknesses and then fixing it.
Senator Carper. Ms. Evans.
Ms. Evans. I would like to respond to this.
Senator Carper. Please.
Ms. Evans. First and foremost, as the Administrator of
this, I think it is my responsibility to make sure that I have
done everything that I can for the agencies to make sure that
the guidance, the policy, everything is clear, and what GAO has
highlighted is that there is a systemic problem throughout the
government. So we could be focused on the one-offs and let us
zero in on this one particular project or let us zero in on
this. But the tenets of the President's Management Agenda is
getting good management foundation processes, with these types
of things in place, so that no matter what the project is, no
matter what the investment is, the agency is going to succeed.
So when you start looking at the GAO report and you look at
Table 5, which talks about who is consistent with the best
practices, and going across and looking at that, what you
want--I am looking at, what is the problem across the board,
and right now, what I have gotten from all of this is that I
need to do a better job of clarifying what is a baseline. How
do you do integrated baselines? How do you set certain things
up? What are the best practices, so that the agencies have the
tools to succeed.
If I am going to sit there and highlight one or two
projects and say, Agency A is a real screw-up and they are not
doing PDQ, we are going to drive behavior down underground and
these issues of rebaselining are going to continue on and on
and on because they are going to want to hit the mark of being
at the 90 percent because grades are coming out. So I think it
is my responsibility to do the things that I can in the first
place to make sure that it is clear to the agencies.
Now, once I believe that I have done everything that I can,
that we have put everything in place, that it is totally clear
in the agencies, the bulk of the agencies are producing, then I
have no problems providing all the information, putting the
transparency out there, having accountability, because that now
becomes a different issue and that is a leadership issue within
the agency itself.
But right now, Mr. Powner has made it very clear that it is
an immature process, at best, what we have in the agencies, and
so they should be rewarded for moving forward and being
forthcoming in the information that they have so that we can
address and fix the problems, not that I feel passionately
about this.
Senator Carper. Well, I would be disappointed if you
didn't.
Mr. Powner, do you want to say anything else? Go ahead.
Mr. Powner. Just to piggyback off of what Ms. Evans was
saying there, I think the point, too, about the accountability
at the agency level, agency CIOs are accountable for these
fixes. I mean, we need to highlight them, and it is not just
all in OMB's lap, but the agency CIOs are accountable here.
Senator Carper. Has OMB issued guidance on rebaselining?
Ms. Evans. The policy memos that we have in place are high-
level. They refer back to the Capital Planning Guide. They also
refer back to A-11. But I, in preparation for this hearing,
have gone back all through those, and maybe it is because I
live and eat and breath this, it seems clear to me, but when I
am looking at all the results across the board, I think there
are other things that we can do like putting together a
framework document that would show them how to do certain
things the same way that the CIO Council put together a
framework for project managers and made it very clear what a
tier one project was, what a tier two project was, and those
types of things. I think we need to go down now to another
level and give them another set of tools in order to be able to
improve their performance.
Senator Carper. Mr. Powner, should OMB issue guidance on
rebaselining?
Mr. Powner. Yes. There clearly is some clarification that
needs to occur so that agencies have clear direction going
forward. Our report highlighted very simplistic items that the
policy should cover. For instance, when you establish a new
baseline, how do you validate it? That is very consistent, Ms.
Evans, with the integrated baseline reviews that you require
and those types of things. So it is not inconsistent with
things that OMB is already endorsing.
But the other key part of their policy is it has to be
approved by management. We found that a lot of policies didn't
even require when agencies rebaseline an approval from key
managers. That is all about the transparency thing so that, in
fact, we are getting accurate data on costs and schedule
performance.
Senator Carper. Mr. Denett, does your office collect
information on capital investments?
Mr. Denett. That is through the A-11 process, through the
budget. So we get these 300s and the budget side, the Resource
Management Office culls through those and looks at them to see
which ones are using earned value management and which ones are
behind or doing well on cost and schedule.
Senator Carper. OK. What do you do with that information?
Mr. Denett. The Resource Management Office looks through
them and goes back, challenges them, sometimes sends them back,
asks for more data, asks them what they are doing to fix it,
and it all rolls up into the budget process. But, important
also is the caliber of people we have working with this, and as
I mentioned earlier, we are real pleased that we now have
certification requirements for the managers and people assigned
to this, contracting officers, program managers for the first
time have to meet certain experience levels and they have to
take mandatory training. All of that will also assist in doing
a better job in this area.
Senator Carper. All right. Let me just ask you what
Congress' responsibilities are, Ms. Evans and Mr. Denett. I
think we understand that GAO is our watchdog to help us better
ensure that you are doing your jobs well. What do you see as
the appropriate role for Congress in our oversight
responsibility as the Legislative Branch? We don't pretend to
be experts on these hundreds of different IT projects or
capital investments. At the same time, we have an obligation to
authorize and appropriate monies to fund these projects. If we
don't have information that is timely and that is
understandable, we are not able to do a very good job as
legislators in our oversight capacity. What do you see as our
appropriate role that would enable us to be constructive?
I think part of it is to embarrass. I am a person who
believes in rewarding behavior. I am a big positive reinforcer.
But I think the time comes when people need a swift kick in the
pants and there are times when folks just need to be
embarrassed. Sometimes, I think that is an appropriate role for
us. Sometimes, it is an appropriate role for OMB, and certainly
for GAO.
But let me just ask your thoughts about how the Congress,
including this Subcommittee, can be more constructive. Mr.
Powner, any thoughts?
Mr. Powner. Well, I would agree that you need to shine a
spotlight on the problems. That is very helpful at times and
perhaps it is public embarrassment, but it gets action.
Hopefully, over time with the oversight, we can move down the
road here and focus more on the solutions. I mean, we know all
these projects, we know the agencies, the projects, and the
problems with specific projects. We know what the issues are.
Now we need to put the fixes in place to look at how we
implement things going forward. I think your second panel is a
good start, where we start looking at that. But again, we are
going to probably continue to need to step back to make sure
that you are getting accurate information because I contend
that there is still some information here that is understated.
Senator Carper. We have talked about some of it, but just
be more specific. You say you contend that some of the
information is understated. Just elaborate on that some more. I
just want to stay on this point.
Mr. Powner. Out of 800 major IT projects, reporting 70 that
have performance shortfalls on cost and schedule is an
understatement.
Senator Carper. All right. Fair enough. Ms. Evans, my first
question.
Ms. Evans. If I could point to----
Senator Carper. No, I want you to come back to my question
what----
Ms. Evans. Well, I am. I am going to give you an example.
Senator Carper. We spent plenty of time telling you how you
ought to do your job.
You have a chance to give us some advice on how we can
better do our job so you can do your job better.
Don't pass up this opportunity.
Ms. Evans. I know, and so I do believe that there would be
indicators off of the information that we provide to you that
by having hearings, and I don't disagree that having public
hearings ensures transparency and accountability, and you can
use this set of hearings as an example because there is a
particular area when we first came up here you had no
information. You just had gross numbers of what we were saying
and you kind of had to trust us that we were telling you the
right thing.
So I think that if the agencies come up here with the
mindset along the lines that you are there to achieve the same
goal as ourselves, that then it becomes a partnership. And I
think that the hearings themselves and the way that you have
approached the hearings has allowed for that partnership to
happen, and so that is why we are at the point with all the
information that is out there.
But to your point about--and on one of the charts it has
the Census project, which we can all talk about. By having
those hearings, it does have the agency very focused on it. The
way that the policies are set up, there are indicators on there
that you could see it before they have to do a budget
amendment. There are things that you can do based on pulling
this information out that if you had a particular area that you
were interested in, you can request the more detailed
information. GAO can request the more detailed information from
the agency themselves.
We have set up everything that way so that you can get
those quarterly reports, the monthly reports, everything that
goes behind the high-level numbers, and by talking with the
agencies or having your staff work directly with the agencies
without a hearing--I mean, a lot of the work that we have done
under the GAO High-Risk List in conjunction with GAO and staff
up here, those aren't hearings, but the whole idea of having
meetings with the staff and having to present their goals and
their plans and then us reporting out on it on a quarterly
basis and a 6-month basis really keeps agencies on focus, and
that is not even a hearing. That is just saying what you are
going to do in the next 6 months and then come back up here and
present what you have done.
Senator Carper. All right. Let us talk about solutions. We
have talked a bit about information technology projects that
are making it or not making it, how we can get better
information, more timely information on a regular basis, and
information we can understand. In terms of solutions and
solving these problems and moving forward, a long time ago,
when I was a Naval flight officer, when I was on active duty on
the West Coast, our job was to hunt for Red October. We flew
airplanes, 13-man aircraft that tracked Soviet nuclear
submarines in all the oceans of the world and we had a number
of squads. We flew an airplane called the P-3.
We would all go through a training command to be prepared
to be assigned to our respective squadrons and then we would
get to our squadrons and we were assembled into 13-man crews
and fly out our missions. But our respective squadrons didn't
always do an especially good job and we had some squadrons that
were better than others and we had some crews that were better
than others.
And the admiral that was in charge of our operations came
up with an idea. I think he called it a Tactical Training Team,
and established it in the basic training squadrons in which we
all initially were trained. These Tactical Training Teams would
be deployed to squadrons that weren't doing a particularly good
job throughout the Pacific and Indian Oceans. They would work
with us for a while and then they would go back home or
deployed to assist other squadrons on an as-needed basis. There
was always plenty of work for them to do because we weren't
always uniform--just like these IT projects. You have pockets
of excellence. We had pockets of excellence in the work that we
did.
My staff in working with others has come up with an idea.
It reminds me of the Tactical Training Team idea. Our folks
here call it an IT Strike Force. I would like to think that it
is a constructive and realistic solution, or part of the
solution to what we are trying to do here. I am not sure, but I
hope that it is.
But Ms. Evans, I believe you may have seen a pre-release
draft of the bill that I am going to be introducing later today
with Senators Lieberman and Collins that will allow your office
to create, if you will, a Tactical Training Team, but really an
IT Strike Force. It would be comprised of experts from both
within government and maybe from outside of government, folks
who can help agencies control some of our IT problems before
they become even more unwieldy.
And the hope here is that we won't see a repeat performance
like we have experienced with our Census hand-held devices.
This Strike Team notion would help agencies control small
problems before they become billion-dollar problems, as they
are today.
First of all, is this a realistic proposal in your idea? Is
this an idea that is half-baked? Is this an idea that has
promise but needs to be further thought out? What do you like
about it? What are your concerns about it? I would just start
with asking you for your comments and then I will ask Mr.
Powner and Mr. Denett, as well.
Ms. Evans. Sir, the initial reaction to it is that it is a
solutions-oriented approach. It is trying to get to the goal
and identifying the problems quick enough in a way that we can
prevent a situation like the Census from happening again. So
the initial concept, I believe, is a good concept and is a
solutions-oriented concept.
We do something on a very similar basis, but it is not as
formal as that process would be, now as we identify the
problems, but they are not as quick. We are not getting ahead
of the problem.
One of the efforts that we have underway now, which I think
this particular feature would complement, is what we are
calling the Policy Utilization Assessment effort that we
started this year. So Mr. Powner has talked a lot about the
quality of the data, and it is only as good as the agency
reports to us. So what we have started is an initiative with
GSA that actually looks at our policy in totality and then does
an assessment.
So, for example, in the cyber security area, we have a
policy, total policy going from acquisition all the way through
how you accept it. If I looked at the numbers reported by the
agencies, I would be at 50 percent. But what we have worked on
and what we have developed is a methodology that gives us
statistical certainty. I am really only at 30 percent.
So based on that, using that and then in complement with
what is envisioned, I believe, in this bill, I could use this
to go into an agency and say, OK, you said that you are green.
You have all these things in place. You haven't rebaselined, so
we are going to do an assessment of the policy, not an audit
like GAO, but an assessment, and it would tell me whether they
were at 100 percent, 90 percent, or 60 percent. If they were
below a certain threshold, then this team could go in, analyze
what the issues are, and then help the agency to move forward
to really realize that 100 percent implementation.
Senator Carper. Mr. Powner, this notion, this IT Strike
Team notion, is it half-baked? Is it more than half-baked? One
of the things I think you mentioned in your testimony, we have
these IT projects that we know are problematic. They appear on
a High-Risk List or a Management Watch List, not just once, but
again and again and again, for extended periods of time, for
months and in some cases for years. And maybe that might be an
appropriate place or instance in which to deploy these teams.
Mr. Powner. Yes. I think the idea is a good one from this
point. Agencies need help, and if you look internally to some
agencies, they set up similar--they refer to them as Centers of
Excellence where they can go to for help in establishing
contracts. How do we establish a baseline estimate up front?
How do we define our requirements? How do we manage risk? And
so you put the appropriate processes in place. And if you look
at those things, a lot of those are the root causes why we have
projects on this list, why we rebaseline. We got the
requirements creep. Requirements creep is out of control so we
have rebaselined. That was the No. 1 reason we found.
Senator Carper. Is that right? OK. I am not surprised.
Mr. Powner. No, exactly. So if you put this group together,
I can tell you right out of the gate what some of the areas you
want to focus on: Getting good estimates up front, defining
your requirements well, putting in place a sound risk
management program, and overseeing contractors, and there are
probably some other thoughts, but those are four areas where we
have major weaknesses across the board.
So I think it is a good idea because if you focus on
solutions and areas where they can go for help--but what is
important is that group from a central government point of
view, there will be a lot of pressure on that group because
there is a lot of help that is needed. What is important is to
establish that group and then to have that somewhat replicated
in agencies, too. And we already have some of that. I think
DOD, FAA, IRS, some of the big organizations, you will see
pockets of that occurring.
So it is also important to do that centrally, but to make
sure that it gets replicated and not to lose sight, too, that
it is about the people, also, not just the processes but the
people. We need to bolster our workforce. I think Ms. Evans
comes out with an annual report showing that we need help in
the PM area, with architects, engineers, those types of folks.
One other item that I would like to highlight is if you
look at IRS as an example, where they had some successes, they
used what is called Critical Position Pay Authority, where you
actually can pay above some of the Federal limits to attract
some better folks. So in addition to these processes and strike
forces, we need to continue to bolster our workforce through
some of those existing processes.
Senator Carper. All right. Well, I think I have probably
kept you here long enough. I wish that more of my colleagues
were here with us today. I know Dr. Coburn has a lot on his
plate these days, but is keenly interested in these issues. He
has not gone away. He is going to be around for a while. He has
self-imposed a 12-year time limit if the voters of his State
concur with that, and they might.
I am going to be around for at least another 4 years and my
guess is that we will continue to work together on this
Subcommittee for a while, so our interest in this issue, in
these issues, as well as a broader range of issues that involve
concerns about just how wisely we are spending our resources,
we are going to stay on these, but we are going to stay on this
one, as well.
I think we are making some progress, but God knows we need
to make more. There is plenty of work for us to do here.
Mr. Denett.
Mr. Denett. I agree with what my colleagues have said. I
would like to tack on, we do have an initiative called the
SHINE initiative, because so often----
Senator Carper. The SHINE? Is that an acronym or what is
that?
Mr. Denett. It is just meaning shine, give people a chance
to shine and look good. Too often, they are focused on, when
they get behind schedule and over cost, and that is
appropriate, but we don't often enough praise those that are
within budget and on schedule. So SHINE is an attempt to give
recognition to programs and employees that are doing well. So
we gather them in, give them some praise----
Senator Carper. That is good.
Mr. Denett. I think it would be helpful and would even like
to approach you when we have our next round of those to see if
you would be willing to participate in honoring some of those
special projects.
Senator Carper. I would be pleased to do that. I would be
pleased to do that. What we used to say in the Navy, praise in
public, reprimand in private. We would be pleased to join in
the praise for some of these folks. But every now and then we
like to, around here, we like to reprimand in public, too. It
has a salutary effect. [Laughter.]
All right. I suspect you are going to have some follow-up
questions for the record. I would just ask that you respond to
them promptly, as you always do.
Thank you for your stewardship here, and Mr. Powner, a real
special thanks to you and your folks at GAO who are a big help
to us in this area. Thank you.
All right. Mr. Grasso, Dr. Brown, Dr. Jarrett, we are happy
to see you, Tom Jarrett. Welcome.
Our second panel today begins with Al Grasso. Mr. Grasso is
President and Chief Executive Officer of the MITRE Corporation,
a not-for-profit organization chartered to work in the public
interest. I think you all have been involved, as I understand
it, in doing some work on the Census project and we are very
grateful for the work that you are doing there. At MITRE, Mr.
Grasso is responsible for developing and leading the
corporation's overall strategic and business operations. MITRE
has been involved in helping to oversee multiple complex and
high-risk IT investments in the Federal Government. You have
had plenty to do.
Our next witness is Dr. Norm Brown, and I couldn't help but
notice Dr. Brown sitting in the first row behind the witnesses
on our earlier panel. Dr. Brown did a lot of head nodding one
way or the other, so now we will let you actually lend a voice
to all the body language that you were sharing with our staff
and me earlier.
But he is Executive Director for the Center for Program
Transformation. For over 30 years, I am told, Dr. Brown has
served as a commercial program manager and program management
troubleshooter, and you have also served as the Assistant
Secretary of--did you serve as the Assistant Secretary or in
the office of the Assistant Secretary of the Navy--in the Navy,
good for you--where you worked across all military branches to
bring troubled projects back on track. In addition, I am told
that you represented the military services on the 2000 Defense
Science Board with the mission of solving why risky IT
investments spiral out of control.
And our last witness, I am tempted to say saving the best
for last--he is a home boy--but Tom Jarrett, whom I have been
privileged to know for a long time, Secretary of our Department
of Technology and Information for the State of Delaware. Mr.
Jarrett is no stranger in testifying before this Subcommittee.
Previously, he testified as President--President, that is a job
a lot of my colleagues would like to have--President of the
National Association of State Chief Information Officers on
issues relating to cyber security when Senator Coburn was
Chairman of this Subcommittee. Mr. Jarrett oversees an IT
investment budget of over $200 million and has achieved 90
percent of cost, schedule, and performance goals for the past 7
years as the Chief Information Officer of Delaware.
When I was governor, we made progress on a whole lot of
different fronts overall in our education and welfare system. I
was very pleased with much that we accomplished, such as job
creation and job preservation. One of the areas that we made
some progress, but maybe not enough, is the area of our
government that Mr. Jarrett now leads. He has taken our State
clearly to the next level. I think one of the very proud things
that our current governor can take credit for is the work that
Mr. Jarrett and his folks have done in his department. So,
welcome. In terms of best practices, we think you are one.
Mr. Jarrett. Thank you.
Senator Carper. Mr. Grasso, I am going to recognize you for
your statement, and again, I would ask you to use about 5
minutes. If you go a little over, that is all right. And then
once all of our witnesses have testified, we will come back and
ask some questions. Your entire statement will be made part of
the record, so please proceed.
TESTIMONY OF ALFRED GRASSO,\1\ PRESIDENT AND CHIEF EXECUTIVE
OFFICER, MITRE CORPORATION
Mr. Grasso. Thank you, Chairman Carper. Thank you for
affording me the opportunity to appear before this
Subcommittee. I fear that many of the remarks I had prepared
have already been stated this morning, but I hope to
reemphasize some of those key points.
---------------------------------------------------------------------------
\1\ The prepared statement of Mr. Grasso appears in the Appendix on
page 100.
---------------------------------------------------------------------------
Senator Carper. Around here, we talk about an echo effect.
It is really hard to cut through the media and to actually
deliver a message to constituents, to voters. If you are the
President, you have a big megaphone and you have a lot of echo
effect from your cabinet secretaries and others who work for
the Administration, so you have a good echo effect. But things
have to be said over and over and over again in order for them
to get through, including to people like us up here. So a
little repetition is not bad.
Mr. Grasso. Thank you. Our company's 50 years of
experience, contributions, and accomplishments has given us a
perspective that I believe is highly relevant to the topic of
information technology planning and management. From the early
days of the SAGE air defense system to present-day deployment
of advanced command and control and business modernization
systems, MITRE has been witness to great successes and,
similarly, to great disappointments. We are honored to be asked
to share our lessons and insights with your Subcommittee.
Federal IT programs operate in an environment of rapid
technology evolution, where some system components become
obsolete before the program completes. This pace of technology
change challenges program teams to keep their technical skill
base current. IT systems and business processes are
increasingly interconnected within and across agencies, making
it hard to achieve consensus on vision, operational concepts,
and requirements. The Federal Government's stretched fiscal and
human resources further complicate the situation.
The net effect is the widespread failure of many programs
to deliver on time and on budget, with only a few notable
exceptions where programs are able to overcome these challenges
and succeed. Our experience leads me to comment on several
critical areas and to offer three steps for improvement.
My first comment pertains to governance. Governance relates
to decisions that define expectations, grant power, assign
accountability, or verify performance. Effective governance
comprises consistent management cohesive policies, processes,
and decision rights for a given area of responsibility.
Governance becomes increasingly complicated as programs and
processes cross organizational boundaries and intersect
multiple governing bodies. Authorities and responsibilities
become ambiguous and program managers are disenfranchised. It
is often said that the debate begins in government once the
decision is made.
Successful programs must have unambiguous governance.
Decision making authority----
Senator Carper. Say that again. Successful programs must
have what?
Mr. Grasso. Unambiguous governance.
Senator Carper. OK.
Mr. Grasso. Decision making authority and accountability
that address the implications of intersecting organizations
must be clearly defined at the onset. These authorities must
encompass the areas of budget and finance, investment portfolio
management, business process, and program and project
management.
My second comment pertains to requirements, an equally
important consideration. Requirements, reality, and flux are
often recognized as the root cause of program rebaselining.
Rebaselining is not necessarily a dirty word, but a necessary
part of delivering capabilities that meet users' needs.
Requirements are too often determined in the absence of cost,
schedule, and technology risk consideration, and once
determined, they are very difficult to change.
The biggest difference between successful commercial IT
developments and troubled government IT acquisitions is how
requirements are managed. Successful commercial IT developers
handle requirements with great caution. If a certain
requirement adversely drives cost, performance, or schedule, it
is quickly modified or eliminated. This does not happen in a
typical government IT acquisition. Time to market is a
competitive driver in the commercial marketplace, and I would
submit it is as important, if not more so, in a world where
adversary capabilities change as quickly as the technology
cycle. System requirements must be considered living, but
managed with a controlled process to use regular trade-off
analyses to determine the value of change.
My final comment addressees program management practices.
Successful programs are characterized by a strong government
Program Management Office (PMO), capable of a peer relationship
with the contractor on systems engineering and program
management issues. With a strong and capable PMO, the
government has the capability to make informed decisions and
manage the risk in acquisition programs.
A key function of a strong PMO is best described by the
metaphor of an architect's relationship with the user and the
builder of a building. The architect is the user's agent as
independent of the builder. The architect works to understand
the user's operational needs and translate them into technical
requirements enabling builders to develop the needed
capability. The architect evaluates development feasibility and
performs an independent conceptual design and cost estimate.
These architect functions enable the user to make informed cost
and capability tradeoffs and prioritize requirements. The
architect is accountable to the user to ensure that delivered
capability meets the user's highest priority needs within the
constraints imposed by available technology, funding, and time.
I offer the following recommendations based on our
experience with these issues. First, change the tone and tenor
of oversight to focus equally on programs that have gone from
bad to good and good to great, to reveal best practices which
then can be applied more broadly. No program is without risk.
We should all be more interested in those programs that have
managed the risks well and harvest those results for the
betterment of the larger set of programs.
Second, to navigate the dynamics of uncertainty of today's
environment, IT programs are best structured as a portfolio
with internal planning and management flexibility. Oversight
should focus on the long-term funding envelope and the overall
capabilities to be delivered. This allows flexibility at the
program level to make informed trade-off decisions and to
concentrate on manageably-sized increments that deliver
capabilities in shorter time frames. This approach makes it
easier for programs to demonstrate success or to fail early,
which is valuable if a program is put in place and funded
contingencies. It also puts capabilities in the hands of the
users more quickly. This incremental approach is the norm in
commercial practice.
Third, Congress should continue to support and refine
programs such as the DOD's Highly Qualified Experts program
and, as Dave Powner mentioned earlier, the IRS's Critical Pay
Authority that helps attract and retain critical government
professionals. Additionally, the IRS's pay-for-performance
program has helped motivate performance aligned to outcomes.
These are valuable tools that address the capacity,
capabilities, and incentives needed to manage effective
programs. We encourage the Congress to look to these as models,
streamline their execution, and broaden their application
government-wide.
I request that my prepared statement be included in the
record and I would be pleased to answer questions.
Senator Carper. Your entire statement will be included in
the record. That was an excellent statement, an excellent
summary. Thank you very much.
Mr. Grasso. Thank you.
Senator Carper. Dr. Brown, welcome.
TESTIMONY OF NORM V. BROWN,\1\ EXECUTIVE DIRECTOR, CENTER FOR
PROGRAM TRANSFORMATION
Mr. Brown. Good morning, Chairman Carper. First, let me
congratulate you for holding this hearing since literally
billions of taxpayer dollars are wasted every year in poorly
managed IT. Clearly, you are onto something important.
---------------------------------------------------------------------------
\1\ The prepared statement of Mr. Brown appears in the Appendix on
page 111.
---------------------------------------------------------------------------
Vice Admiral Jerry O. Tuttle, Retired, the former Deputy
Chief of Naval Operations for C4I and an icon for naval
computing and net-centric warfare would counsel, ``Lead,
follow, or get out of the way.'' Thank you, Senators. Thank you
for leading the way. As we say, bravo zulu.
Let us be clear about one thing. Although it is difficult
to effectively manage a large-scale project, on the other hand,
producing a large-scale IT train wreck is easy. The good news
is that wrecks can be avoided by effectively using best
practices.
Today, I would like to briefly offer actions that
government departments and agencies, OMB, and Congress can take
to prevent wrecks. At its core, these actions address rapidly
achievable improvements.
I will begin with a structural observation. Much is
expected of each agency CIO. Many have responsibility without
real authority. Many Federal departments include numerous
essentially independent fiefdoms because Congress has so
arranged it, fiefdoms independently funded by Congress. The
Pentagon rule is, he that has got the gold makes the rules. So,
too, in Federal agencies. Although I don't today have any
solutions to offer, I would be happy to work with your staff.
Next, an observation regarding those IT problems in project
management and oversight. When OMB testified before you last
September, they expressed a recent interest in IT program
execution, and that is a very good thing, but IT programs
simply don't manage themselves.
From my understanding and as we have heard this morning,
very few agencies have much in the way of any real IT program
management and oversight. Earned value is held up as a do-all
silver bullet solution, yet little is done to prevent the easy
gaming and corruption that earned value is vulnerable to, and
associated rebaselining may lack the transparency needed to
ensure effective oversight. Far too much is expected of earned
value. Although earned value is a powerful visibility technique
that supports program management, earned value cannot replace
program management.
Unfortunately, there seems little in the land of government
IT program management that implements the needed essential
techniques of managing risk, requirements, and change, or
integrated baseline reviews. As a solution, I would recommend
that each agency be required to actually have real program
management and oversight, that they focus on implementing the
important critical details with minimum overhead, and that they
identify remaining weaknesses and vulnerabilities.
Transparency is dandy, but it must be converted to
visibility to be useful. A transparent contractor can deliver a
53-foot truck full of boxes of data, but what you really need
is only the bottom-line information. What is needed is true
visibility of IT project health and progress in near real time.
Since earned value seems to be the only principal
visibility technique relied on by OMB and the agencies, I would
propose a more comprehensive visibility product, let us say an
Exhibit 350, to provide real project visibility indicators
monthly, primarily for the program manager, with quarterly
simplified versions for agency and component CIOs, OMB, and
Congress. I will be happy to work with your staff and OMB on
this.
Tracking schedule progress is not easy. One reason for
schedule surprises is that it is pressure to meet schedule
increases. The hard-to-do things are kicked down the road, with
difficult, uncompleted requirements now moved into the future,
a future which was not planned to receive it. As it turns out,
this unplanned future work will now require a successive series
of miracles to be accomplished in order to complete the
development on time and on budget. Don't bet on the miracles
happening.
To motivate agencies to focus on IT project management and
oversight, I would recommend adapting the Nunn-McCurdy
notification process, not that it is a great visibility
technique; it is not. But it, in fact, serves as a powerful
motivator, as something really to be avoided. As little else
can do, it gives a clear focus to the business of cost and
schedule containment.
We have a serious problem regarding people. It is difficult
to reward good talent, to hire good talent, and to train good
talent. We expect CIOs and our IT personnel to do more with
less and then give them less as if to prove the point. OMB
needs to address this people issue as a priority.
Training project personnel to effectively implement
fundamental processes is minimal. While DOD has a certificate
program in IT program management, GSA long ago disbanded their
excellent similar IT Trail Boss program for civilian agencies.
The various communities charged with making IT development work
have received essentially no education, training, or any
certification to do what they are asked to do. No training, no
education, no certification, and no experience--it is a wonder
we do as well as we do, as bad as it is.
Apropos of your remarks earlier this morning about pulling
the plug, a program termination process should be considered.
The former Assistant Secretary of the Army for Acquisition, the
Hon. Claude Bolton, is a strong proponent of terminating
projects if measurable outcomes cannot be achieved within the
agreed-to program cost schedule and performance baselines. The
DOD has no formal decisionmaking process or policies to
terminate programs, and I am not aware of any for other
agencies. I think it wise and prudent to consider including
such a process among ways to improve the government's IT
acquisition process.
Contract incentives and other considerations are important,
and I discuss them further in my formal testimony.
That concludes my summary and I will be happy to take
questions at the appropriate time.
Senator Carper. Again, another excellent testimony. Thank
you for the thought that you put into it. Thank you for your
years of service, too, and your counsel here today.
Mr. Jarrett, you are recognized. Please proceed. Welcome.
TESTIMONY OF THOMAS M. JARRETT,\1\ SECRETARY, DEPARTMENT OF
TECHNOLOGY AND INFORMATION, STATE OF DELAWARE
Mr. Jarrett. Chairman Carper, Ranking Member Coburn, and
Members of the Subcommittee, thank you for the opportunity to
be here today. As Secretary of Delaware's Department of
Technology and Information, I can well appreciate the
complexity, the challenges, and the significant
responsibilities associated with managing information
technology projects in an investments portfolio that cuts
across many agencies, and in Delaware's case, all three
branches of government.
---------------------------------------------------------------------------
\1\ The prepared statement of Mr. Jarrett appears in the Appendix
on page 120.
---------------------------------------------------------------------------
Albeit in a much smaller scale than the Federal Government,
Delaware and other State Governments are faced with similar
concerns regarding IT project management. In fact, Delaware's
Department of Technology and Information was established in
part because of ongoing IT project delays and cost overruns
that Delaware was experiencing. Delaware's centralized IT
structure charges our agency with direct oversight and approval
on nearly $200 million in active IT projects.
As a new agency, we were able to develop and employ new
approaches to IT project management that incorporate many best
practices from private industry and others in the government
sector. While our methods are under constant review for
improvement, we are enjoying some significant progresses. We
have an excellent track record of delivering much-needed IT
solutions that are on time and on budget.
There are no silver bullets, no one or two changes that you
can point to for perfect project management. Instead, there are
many small improvements that we have made that, in the
aggregate, are making the difference for Delaware. I would like
to take a few minutes to give you a high-level view of a
project life cycle using the Delaware model. In addition, I
would like to point out what I believe are the significant
processes and procedures that we have put in place to
successfully manage our portfolio.
Like the Federal model, we require agencies to submit a
business case that addresses the major items we believe help
ensure a project's success. Our model includes the following
major areas: Risk management, processes reengineering,
architectural review, resource and funding availability,
project management oversight, organizational change management,
needs assessment, customization requirements, disaster recovery
levels, and management and executive sponsorship.
Recommended projects are forwarded for the concurrence of
the State CIO before they move ahead in our process. This
recommendation usually includes ongoing funding contingent on
meeting project milestones. Once approved, DTI works with the
agency customer to develop full and complete requirements so a
request for proposals can be released to secure vendor bids in
order to meet the needs of the project. DTI stays involved to
help the agency make the vendor selection and to structure a
contract that ensures the project is delivered on time and on
budget.
I cannot emphasize enough how critical the requirements
gathering process is to the project's ultimate success or
failure. The Delaware model does not allow for requirements
gathering to be conducted solely by the vendor. There must be
an active involvement by the DTI project management team. There
is an old saying, ``The customer doesn't know what they want
until you give them what they ask for.'' Requirement gathering
is critical to the process and helps alleviate scope, time, and
budget creep if it is done correctly.
At the start of a new project, a nationally certified
program or project manager is assigned, as well as a certified
organizational change management team, to run parallel courses
in managing our projects. It is not enough to be certified. We
take extra care in selecting the people to fill these
positions, as they are essential to the success of the project.
We count on these folks to do what we call inflicting
discipline and structure to these projects.
It is important to note that DTI manages Delaware's IT
projects, not the vendors assigned to them. While we demand the
vendor assign a certified and talented project manager, our
State folks manage the project.
Further, I would like to highlight our change management
process. Change management starts with the review of existing
business processes and is focused on preparing the organization
for the cultural changes that are a part of any major project.
According to a recent study conducted by the National
Association of State Chief Information Officers, which
represents State CIOs across the 50 States, 80 percent of major
IT project failures can be directly attributed to a lack of
change management. Too often, the employees who will actually
use the new application are left out of the project process,
and when this takes place, it is almost certain that the
project is doomed to fail or under-perform.
Another major element of our success is how we manage the
overall process. All of Delaware's major IT projects have
executive sponsors. Executive sponsors include high-level
managers from the agencies involved as well as key executives
from DTI and our Office of Management and Budget. The executive
sponsors hold regular monitoring and update meetings and
provide high-level oversight. When difficult project decisions
need to be made, the executive sponsors are aware of the issues
and have the authority to make critical decisions on whether or
not to keep the project moving along.
A real example is our present project involving all of
Delaware's courts. Due to the lack of employee training and
readiness, the executive sponsors recommended that the project
be paused so that the training could take place. In fact,
Delaware's Chief Justice himself invoked a 6-month pause in
recognition of the need for court employees to be prepared if
the project was to succeed.
Besides ongoing project meetings, all projects in the
State's portfolio are reviewed weekly by our Project Management
Office and bimonthly at a workload management meeting where
project managers present their projects' status to the
assembled DTI senior managers. Opportunities for improvement or
needed adjustments are vetted in an open forum with the goal of
keeping the project on track and transparent.
Our agency is committed to project transparency all the way
to our legislature and our governor. We believe in continuous
information exchange and dialogue with our elected
representatives so that they are educated and aware of the
complexity of major IT projects. Providing information up
front, even when it may be painful, is far better than saving
unpleasant surprises for yearly budget sessions. Bad news does
not age well.
I wish I could tell you that everything is humming along
perfectly and that the processes that we have put in place are
the final answer, but I can't, as we have many challenges
similar to those being talked about today. However, we believe
that by tightly managing these challenges, we can deliver
projects on time and on budget. Although our project tracking
system is complex, in our communication with stakeholders, we
use a high-level red/green/yellow scorecard to inform them how
the projects are developing.
All projects have or will have ongoing issues that require
senior management attention. Under the Delaware model, we
believe that providing the facts, good or bad, to all
stakeholders, including the legislature, is the proper policy,
and most importantly, lives up to one of my agency's core
values of integrity. Our vendors are held accountable to
deliver what they said they would, as well, and we call the
facts as we see them.
I thank you for the opportunity to testify today and would
be happy to answer any questions.
Senator Carper. Thank you. Thanks not just for your
testimony, but for the terrific leadership that you provide for
our State and have shared some of the fruits of that leadership
with us today.
I think I would like to ask, first of all, Mr. Grasso and
Dr. Brown a question and ask you to reflect on what we have
heard from Mr. Jarrett today. I have oftentimes described the
50 States as laboratories of democracy and the belief that
somewhere in those 50 States, somebody has come up with a
solution that will help us solve a number of the problems we
face at the Federal level, and the same is true of the private
sector. Somebody has figured out how to solve most of the
problems, not all, but most of the problems that we face as a
Nation and we just have to figure that out and be able to grow
them to size or to scale.
What did you hear from Mr. Jarrett in talking about
Delaware, the way we operate in our State, that might be
applicable to us here at the Federal level? What are some good
lessons learned that you think we could take from his testimony
and apply them to the Federal Government?
Mr. Grasso. I think a strong element that I heard in the
successes that we have seen in Delaware is the investment,
again, that is made in the project management team, in the
strength of that project management team. We heard that the
project manager is certified, but it goes much further than
just simple certification. There is a continuous development
effort that is required.
With a strong project management team, that project
management team can be held accountable to basically continue
to own the technical baseline of the program and not just
basically contract it away such that you just get what you get.
The team is informed throughout the process, is able to
establish that peer relationship which I mentioned in my
testimony with the contractor, and that allows for the
successful acquisition.
Requirements are important. We heard that. The level at
which requirements are stated are very important. All too
often, we detail requirements down to a level that we believe
perhaps causes some of the rebaselining that we see because we
specify things in terms of technology as opposed to in terms of
outcomes. Although he didn't say it, the importance of
requirements and the emphasis placed on requirements, I suspect
they may have gotten that right in terms of the level by which
they establish requirements.
So I think a combination of the strength of the program
management team, the investment made in those people to keep
them highly qualified, and the ability to deal with
requirements at the right level have achieved some of the
successes.
Senator Carper. Thanks for that analysis.
Dr. Brown, what did you hear from Mr. Jarrett's testimony
that you think might apply to us at the Federal level?
Mr. Brown. First, let me echo what Al Grasso has said. I
think the emphasis on requirements development and control and
change control is absolutely fundamental and determines much of
our cost and schedule problems that we see.
And I think there is one other thing that Mr. Jarrett
hasn't fully articulated to you that is very relevant, which is
his process of paying their IT folks. Maybe you could talk more
about that.
Senator Carper. You actually pay them?
Mr. Jarrett. Yes, imagine, they want to get paid.
[Laughter.]
As you know, Mr. Chairman, a number of years ago, the
legislature allowed Delaware, and I still believe the only
State today that has ever done it, to take the entire IT
organization and convert it from a civil service structure to a
non-civil service structure, and along with that, they gave me
the ability to write our own compensation structure. So I think
we are one of the few, if only, State agencies in the country
that actually pays its IT people to the market, and what that
means is that we have very qualified people, and I am happy to
say we also have the highest retention rate of any agency in
the State.
Senator Carper. I used to be State Treasurer. I remember a
time early on when we thought that we hired people, usually
fairly young people out of school, and trained them so that
they could be hired away by the private sector or some other
employer. That sounds like that is not the case anymore, is it?
Mr. Jarrett. That is not the case, not in my department.
Senator Carper. OK, good. Dr. Brown, do you want to add
anything?
Mr. Brown. Sure. One thing that you had talked about that
particularly resonated with me was the concept of the tactical
training teams, and something that I had started at the Navy
Department and later became DOD was the Software Program
Managers Network. We had 10,000 members across the country. We
supported over 200 programs in helping them. We had a stable of
these people that we would call tactical trainers, experts in a
wide range of subject matter experts, over 100 of these people,
and as programs need, they could just call in the tactical
team.
If they needed help on a task activity network to support
earned value or if they needed to better understand how to
identify risks in the program or had a plan for testing or had
a better oversight of what the contractor is doing or
incentives, anything like that, they could call in the team.
That is very consistent with your IT support team, and I think
that is a fundamentally important thing to do.
Senator Carper. All right. Mr. Grasso and Dr. Brown have
been good enough to comment on your testimony. Let me just ask
you if you would do the same with respect to some of what they
said and just reinforce the relevance of their counsel to us.
Mr. Jarrett. Well, I was telling Dr. Brown before we
started, I was so pleased when I had a chance to read and hear
what they both had said because I kind of threw up my arms and
said, gee, we are doing that and have been doing it for some
time and I think it proves out in a couple of areas.
One is the requirements. People get sick about hearing
about that, but in fact, what we have found and where we have
spent all of our time is on the front end of the project
process, not on the back end, which means that if you are going
to get them right, then you have to determine what the
requirements are. What we learned in the very beginning is that
is not unlike--and you need actually qualified and trained
people to be able to do that. That is not as simple as going
out to an agency and saying, ``OK, tell me what it is that you
need,'' because what we have found is that in a lot of cases,
they can't articulate what they need, at least not in a way
that when you are looking to replace a system and do that. So
you have to help them through that process. So we have spent a
lot of time doing that.
The other is in the change management area. I think a lot
of States and a lot of folks are moving into the areas of PMOs,
Project Management Offices. Something that we have spent an
awful lot of time on is, again, in the organizational change
management, and I will give you an example. We are doing a new
financial system in the State. We are spending a lot of our
time not on the technical aspects but in dealing with each of
the agencies so that we can help the users kind of work through
the processes and the changes that this new project or this new
system is going to provide to them. Most times, we don't spend
time doing that.
So I hear both of those areas in the testimony that was
given, and from my perspective, I think it is well thought out
and something that I think the Federal Government could do, as
well.
Senator Carper. Mr. Grasso, did you have a point you wanted
to make?
Mr. Grasso. I might just offer an additional remark.
Senator Carper. Before you do that, in your testimony, Mr.
Jarrett, you said there is an old saying the customer doesn't
know what they want until you give them what they have asked
for. There is a lot of wisdom in that.
Mr. Grasso. In my written statement, there are a couple of
references that I would just point to which emphasize the
requirements point, and one of those references points to the
32 programs at NASA that were evaluated and the up-front
investment and the program overrun result. What you see is when
the definition phase as a percent of the total program is
greater than 10 percent, what you find is that the program
overrun is typically less than 20 percent.
Senator Carper. Say that one more time. I want to make sure
I got it.
Mr. Grasso. The graph refers to 32 NASA programs that have
been evaluated and it takes a look at the definition phase of
those programs as a percent of the total program. And when the
definition phase of the program is 10 percent or greater, what
you find is that the program overrun is 20 percent or less.
Senator Carper. OK. Good.
Mr. Grasso. And that suggests that considerate thought is
being given to the requirements up front and thus less change
is required later on.
Senator Carper. All right. Thank you. My staff has been
good enough to prepare a number of questions, and I am just
going to take a moment and look at those and I suspect we will
use these as a point of departure for some further discussion.
In some ways, the kind of discussion that we are having here is
actually more helpful than not.
I have asked you to sort of reflect on your respective
testimonies here today. Go back with me to the first panel and
some of the things that you heard from the first panel, some of
the things that we discussed. What were some of the things that
struck you that were especially important or maybe something
that you wanted to add something that wasn't said, or maybe an
answer that wasn't given, or at least wasn't given as well as
it might have been? Does anything stand out for you, referring
back to the first panel's testimony?
Mr. Grasso. I would offer one remark because I can't stop
looking at the report card. Having several daughters in high
school, I have encouraged my daughters to take Advanced
Placement classes, whereas I know some of her friends are
taking lesser challenging classes, if you will, and----
Senator Carper. What grades are your daughters going to be
in this fall?
Mr. Grasso. Actually, my oldest is going to the University
of Virginia this year, so she is just graduating, and my middle
daughter is a rising 11th grader.
Senator Carper. OK. We have two boys and our youngest son
just graduated from the Charter School of Wilmington and he is
going off to William and Mary and we know all about Advanced
Placement tests.
We just got our results about a week ago, high fives all
around. [Laughter.]
Mr. Grasso. So you know the amount of work that goes into
preparing----
Senator Carper. Yes, we do.
Mr. Grasso [continuing]. And the level of effort that is
required. The report cards are a great start, but at the same
time, all IT programs aren't borne of the same level of
complexity, the same level of interdependency and challenge. So
as we step back and I look at the report cards, it begs the
question of what are the correlations, if you will, to those
grades.
As I look at some of those organizations, some of those
organizations are more insular than other organizations, if you
will, and there are fewer interdependencies, and as a result,
what you find is the complexity of the IT system is not
necessarily as challenging as an enterprise system that touches
a number of different organizations, crosses boundaries, and
has unclear governance.
So a comment that I will make is a report card as such, I
don't know if those are AP classes or if those are substandard
classes. So an F on AP class may not necessarily give you all
of the details that you need to respond.
Senator Carper. Well, you could not have picked a better
analogy.
Mr. Brown. My view is that those don't represent AP
classes. [Laughter.]
And with regard to what Ms. Evans and Dave Powner were
saying about earned value management and particularly the
problem of rebaselining, DOD deals with that in a very direct
and forthright manner, which is just to track all the costs and
the curve and what you see is a curve that starts going down
and then the rebaseline goes up and goes up and it is just this
downward spiral, a death spiral. And it is very easy to track
what is happening when you see that picture, and that would be
very easy for OMB to provide to you. I don't know if they are
willing to do that, but they should have that information when
they say--I don't know what agencies exactly report to them in
their private conversations, but that should be certainly
public information.
Senator Carper. OK.
Mr. Grasso. The 1987 Defense Science Board had an
interesting conclusion and it said, technology is not our
problem, management is, and they were talking about software
problems, and hopefully----
Senator Carper. Who said that?
Mr. Brown. The 1987 Defense Science Board study on
software.
Senator Carper. OK. Mr. Grasso, the folks at MITRE have
been good to help us in the Bureau of the Census. You tried to
recover from a very bad situation. You may not be intimately
familiar with this, but maybe you are, and to the extent that
you have some familiarity with how the Census Bureau got into
this mess as they prepared for and approached the 2010 Census,
where do you think they went wrong and what lessons can we take
from that experience to make sure that whether it is the 2020
Census or other IT projects, how we don't let this kind of
thing happen again?
As it has turned out, it is an enormously expensive problem
and we are going to not only end up spending more money, I
don't know that we will get a better product. We will probably
not get as good of a product in the end, but we will have spent
a lot more money, a very unhappy situation. But we appreciate
what you all have done to try to minimize the loss and get us
going in the right direction. But what are some lessons
learned?
Mr. Grasso. I think the two lessons have already been
described today. The first is requirements. In the case of
Census, as you step back, there certainly has been a
significant number of requirements changes throughout the
process which has led to changes with the contractor. In the
time period in which this all has occurred, the technology has
evolved, so the solution that was envisioned some time ago
perhaps is not necessarily the best today in terms of the
architectural basis for that solution. So the program was not
able to evolve as quickly as the technology was able to evolve.
The second part of that is as all of this is evolving and
all of these changes are being made, the strength of the
program office, quite frankly, I would say, could have used
some building up in the early stages. That has occurred as a
result of the response to the issues that they have been
confronted with most recently, but quite frankly, I would say
that many of the responsibilities that should have been within
the program office in establishing the technical baseline were
really given to the contractor as opposed to being held within
the program office.
Senator Carper. This is sort of the inverse of the
situation that I think Mr. Jarrett described.
Mr. Grasso. Exactly. And we find that to be the case quite
often. When you don't have a strong program office, a lot gets
transferred to the contractor and you end up with a lot more
dynamics in the program.
Senator Carper. All right. Let me just ask, do you all have
any closing thoughts that you would like to give us? I like to
talk about take-aways, and it is impossible for me to remember
everything that is said here. Even our staff, as smart as they
are, it is impossible to remember everything. But just some of
the key take-aways for us from what you have heard from other
witnesses and maybe some points that you have brought up that
you would like to reemphasize as we prepare to conclude.
Is it Secretary Jarrett? Do they call you Secretary? Mr.
Secretary.
Mr. Jarrett. You may call me anything you wish.
Senator Carper. No, but seriously, aren't you a cabinet
secretary.
Mr. Jarrett. I am.
Senator Carper. Then I shouldn't call you Mr. Jarrett. I
should call you Mr. Secretary. Do you want to take the first
shot at that?
Mr. Jarrett. I guess the only thoughts I had, and it kind
of goes back to your earlier question about comments from OMB,
and that is that reports and all the things that they put out
are good and I guess they are a requirement here, but in fact,
I have always believed that reports are only as good as how you
use them to actually make a difference and to change things.
What I guess I don't see in a lot of ways is how those
reports are being utilized to actually begin to change the
process. It just kind of has become embedded in the reports,
and in fact, again, if you are not using them, then why even
produce them. We spend less time on reports and more on the
performance of the projects themselves.
The second would be I see a lot of red. Well, we actually
end up with red on some of our projects, too. The difference
is, just because it is red doesn't mean it is ready to go and
tank. It just means, at least in our case, that it is something
that needs a lot of extra look-see and effort on it to get it
back to yellow or to green, where it ultimately needs to be.
Projects go in and out of red, green, and yellow all of the
time. It depends on the complexity. So I think maybe red is
nice, but I think there is this perception that red is bad.
Yes, it is not good, but it is not necessarily the end of all
things.
The final thing is that in the issue about information back
to Congress, I was telling some folks earlier, kind of the
mindset which they take, and having come from the private
sector and then Delaware Government for the last 7\1/2\ years,
I am by no means an expert in the dynamics of Washington, but
we took a very simple approach with our legislature which was
we will notify them and show them this process on everything,
painful as it may be on occasion. I have had some painful
discussions with Senators and Representatives one-on-one and
even in groups.
Over time, what it has built, though, is the fact that they
have come to understand the complexities with projects. They
have become educated that it is not as easy as people think it
is and that there are a lot of bumps in the road. So now what
we find is that they have a far better understanding of that
and we have a lot more productive dialogue back and forth as we
work through those particular projects than we did when we
started 7 years ago.
So I would recommend to them to take the opposite approach,
which is to provide that information almost as much as they can
and not be concerned about the fact of whether it shames
somebody or not because I think as you said earlier, I think
sometimes that is not necessarily a bad thing.
Senator Carper. All right. Thank you. Dr. Brown.
Mr. Brown. I have three things that might be take-aways.
One is to actually have Congress require visibility at agencies
and at OMB.
The second is to have Congress require the agencies to
maintain and use effective critical practices.
And the third is for Congress to maintain its involvement.
Senator Carper. All right. Expand just a little bit on the
second one.
Mr. Brown. On maintaining critical practices, that is to
have a real program management and oversight capability at each
agency. What happens is there is so much focus on the Exhibit
300s and the planning of programs that their execution and
management go almost unnoticed, and that is where the troubles
begin. It has been said that the plan doesn't survive the first
battle, and so, too, with the plans for our programs. Things
change and are very dynamic. Requirements change all the time
and that is why, as Mr. Jarrett said, change management is very
important.
Agencies tend to ignore change management. They tend to
ignore a very important thing called risk management to
identify risks. They don't pay a lot of attention to what is
going on in programs in terms of incentivizing the contractor
to do the kinds of things that are fundamental, to examine what
the fundamental capabilities that are needed by government are
and not be absolutely controlled by requirements, to ignore the
kinds of things that allow you to have a real confidence in
your way of understanding what is going on.
Instead of just requiring earned value as your sole
indicator, you could look at real products being developed. You
want to have products being developed all the time, weekly or
monthly deliveries to the government, and to have assurance
that those products are being properly integrated together. I
have probably given you more than you want.
Senator Carper. All right. Just enough. Thank you. Mr.
Grasso.
Mr. Grasso. I guess I would start by saying this is hard. I
have seen good organizations fail at IT projects, and I would
start by simply saying that we all have A teams and B teams. We
need a lot more A teams, which means investment in people and
the processes to do this job right.
The teams need to feel ownership, if you will, in the
programs and need to have the authorities to do the right
things. I once used this analogy. As a homeowner, I may make
some decisions to make some investments in my home. This year,
I am going to put $3,000 into a sprinkler system in my home.
Well, as I proceed to do that, I spring a leak in my roof. If
this were the government, I wouldn't be able to fix that roof
because I don't have the right kind of money to fix that roof
and I would continue to fix my sprinkler system. I don't have
the ownership of that portfolio, if you will, to treat my home
as a whole complex suite of things. And the ability for a
program manager to be able to manage a portfolio as opposed to
an individual activity will allow him to manage some of those
risks and make the necessary shifts as is appropriate.
I mentioned earlier in my testimony about contingencies and
funding contingencies. A risk management plan is absolutely
critical to any program, but a risk management plan must have
actionable alternatives. If the alternatives are not
actionable, then we will just continue to mount on the risks.
So I think the key points that I would make is we must
continue to invest in the people and the organizations. As I
mentioned earlier, the Highly Qualified Experts Program and the
Critical Pay Program in IRS are good examples of investment. It
sounds like Delaware has been able to do that. We must be able
to manage these risks and empower our program managers with
strong teams to make the right decisions and to have
alternatives and choices as they see the technologies change
and some of those risks surface.
Senator Carper. Good. Thank you. Well, in closing, I really
want to commend our staffs for being smart enough to invite you
to come and testify. The way it works here, you have Democrats
and Republicans who submit ideas and come up with ideas and
ultimately we end up with a witness list. But this has been an
especially helpful and beneficial panel of witnesses for us.
One of the things that you are especially good at is
explaining things in terms that even I can almost understand in
most cases, and these are not easily understood concepts. That
is a great gift.
Thanks a lot for making time in your schedules to be here
with us today, for preparing for this hearing, and for giving
us not just a lot of food for thought, but, I think, a pretty
good road map to follow to better ensure that we are getting
our money's worth out of these IT projects that are going on
throughout the government, actually around the world, and for
all the taxpayers who are putting their money into this. Thank
you for them, too. Keep up the good work that you are doing and
much obliged.
With that, you will probably hear from some of us with some
additional questions. We will try to get those out in the next
week or so, but if you could respond promptly, we would be most
grateful.
Thank you so much, and with that, this hearing is
adjourned.
[Whereupon, at 1:41 p.m., the Subcommittee was adjourned.]
A P P E N D I X
----------
[GRAPHIC] [TIFF OMITTED] 44586.001
[GRAPHIC] [TIFF OMITTED] 44586.002
[GRAPHIC] [TIFF OMITTED] 44586.003
[GRAPHIC] [TIFF OMITTED] 44586.004
[GRAPHIC] [TIFF OMITTED] 44586.005
[GRAPHIC] [TIFF OMITTED] 44586.006
[GRAPHIC] [TIFF OMITTED] 44586.007
[GRAPHIC] [TIFF OMITTED] 44586.008
[GRAPHIC] [TIFF OMITTED] 44586.009
[GRAPHIC] [TIFF OMITTED] 44586.010
[GRAPHIC] [TIFF OMITTED] 44586.011
[GRAPHIC] [TIFF OMITTED] 44586.012
[GRAPHIC] [TIFF OMITTED] 44586.013
[GRAPHIC] [TIFF OMITTED] 44586.014
[GRAPHIC] [TIFF OMITTED] 44586.015
[GRAPHIC] [TIFF OMITTED] 44586.016
[GRAPHIC] [TIFF OMITTED] 44586.017
[GRAPHIC] [TIFF OMITTED] 44586.018
[GRAPHIC] [TIFF OMITTED] 44586.019
[GRAPHIC] [TIFF OMITTED] 44586.020
[GRAPHIC] [TIFF OMITTED] 44586.021
[GRAPHIC] [TIFF OMITTED] 44586.022
[GRAPHIC] [TIFF OMITTED] 44586.023
[GRAPHIC] [TIFF OMITTED] 44586.024
[GRAPHIC] [TIFF OMITTED] 44586.025
[GRAPHIC] [TIFF OMITTED] 44586.026
[GRAPHIC] [TIFF OMITTED] 44586.027
[GRAPHIC] [TIFF OMITTED] 44586.028
[GRAPHIC] [TIFF OMITTED] 44586.029
[GRAPHIC] [TIFF OMITTED] 44586.030
[GRAPHIC] [TIFF OMITTED] 44586.031
[GRAPHIC] [TIFF OMITTED] 44586.032
[GRAPHIC] [TIFF OMITTED] 44586.033
[GRAPHIC] [TIFF OMITTED] 44586.034
[GRAPHIC] [TIFF OMITTED] 44586.035
[GRAPHIC] [TIFF OMITTED] 44586.036
[GRAPHIC] [TIFF OMITTED] 44586.037
[GRAPHIC] [TIFF OMITTED] 44586.038
[GRAPHIC] [TIFF OMITTED] 44586.039
[GRAPHIC] [TIFF OMITTED] 44586.040
[GRAPHIC] [TIFF OMITTED] 44586.041
[GRAPHIC] [TIFF OMITTED] 44586.042
[GRAPHIC] [TIFF OMITTED] 44586.043
[GRAPHIC] [TIFF OMITTED] 44586.044
[GRAPHIC] [TIFF OMITTED] 44586.045
[GRAPHIC] [TIFF OMITTED] 44586.046
[GRAPHIC] [TIFF OMITTED] 44586.047
[GRAPHIC] [TIFF OMITTED] 44586.048
[GRAPHIC] [TIFF OMITTED] 44586.049
[GRAPHIC] [TIFF OMITTED] 44586.050
[GRAPHIC] [TIFF OMITTED] 44586.051
[GRAPHIC] [TIFF OMITTED] 44586.052
[GRAPHIC] [TIFF OMITTED] 44586.053
[GRAPHIC] [TIFF OMITTED] 44586.054
[GRAPHIC] [TIFF OMITTED] 44586.055
[GRAPHIC] [TIFF OMITTED] 44586.056
[GRAPHIC] [TIFF OMITTED] 44586.057
[GRAPHIC] [TIFF OMITTED] 44586.058
[GRAPHIC] [TIFF OMITTED] 44586.059
[GRAPHIC] [TIFF OMITTED] 44586.060
[GRAPHIC] [TIFF OMITTED] 44586.061
[GRAPHIC] [TIFF OMITTED] 44586.062
[GRAPHIC] [TIFF OMITTED] 44586.063
[GRAPHIC] [TIFF OMITTED] 44586.064
[GRAPHIC] [TIFF OMITTED] 44586.065
[GRAPHIC] [TIFF OMITTED] 44586.066
[GRAPHIC] [TIFF OMITTED] 44586.067
[GRAPHIC] [TIFF OMITTED] 44586.068
[GRAPHIC] [TIFF OMITTED] 44586.069
[GRAPHIC] [TIFF OMITTED] 44586.070
[GRAPHIC] [TIFF OMITTED] 44586.071
[GRAPHIC] [TIFF OMITTED] 44586.072
[GRAPHIC] [TIFF OMITTED] 44586.073
[GRAPHIC] [TIFF OMITTED] 44586.074
[GRAPHIC] [TIFF OMITTED] 44586.075
[GRAPHIC] [TIFF OMITTED] 44586.076
[GRAPHIC] [TIFF OMITTED] 44586.077
[GRAPHIC] [TIFF OMITTED] 44586.078
[GRAPHIC] [TIFF OMITTED] 44586.079
[GRAPHIC] [TIFF OMITTED] 44586.080
[GRAPHIC] [TIFF OMITTED] 44586.081
[GRAPHIC] [TIFF OMITTED] 44586.082
[GRAPHIC] [TIFF OMITTED] 44586.083
[GRAPHIC] [TIFF OMITTED] 44586.084
[GRAPHIC] [TIFF OMITTED] 44586.085
[GRAPHIC] [TIFF OMITTED] 44586.086
[GRAPHIC] [TIFF OMITTED] 44586.087
[GRAPHIC] [TIFF OMITTED] 44586.088
[GRAPHIC] [TIFF OMITTED] 44586.089
[GRAPHIC] [TIFF OMITTED] 44586.090
[GRAPHIC] [TIFF OMITTED] 44586.091
[GRAPHIC] [TIFF OMITTED] 44586.092
[GRAPHIC] [TIFF OMITTED] 44586.093
[GRAPHIC] [TIFF OMITTED] 44586.094
[GRAPHIC] [TIFF OMITTED] 44586.095
[GRAPHIC] [TIFF OMITTED] 44586.096
[GRAPHIC] [TIFF OMITTED] 44586.097
[GRAPHIC] [TIFF OMITTED] 44586.098