[Senate Hearing 110-818]
[From the U.S. Government Publishing Office]


                                                        S. Hrg. 110-818
 
  OFFLINE AND OFF-BUDGET: THE DISMAL STATE OF INFORMATION TECHNOLOGY 
                   PLANNING IN THE FEDERAL GOVERNMENT

=======================================================================


                                HEARING

                               before the

                FEDERAL FINANCIAL MANAGEMENT, GOVERNMENT
                   INFORMATION, FEDERAL SERVICES, AND
                  INTERNATIONAL SECURITY SUBCOMMITTEE

                                 of the

                              COMMITTEE ON
                         HOMELAND SECURITY AND
                          GOVERNMENTAL AFFAIRS
                          UNITED STATES SENATE


                       ONE HUNDRED TENTH CONGRESS

                             SECOND SESSION

                               __________

                             JULY 31, 2008

                               __________

       Available via http://www.gpoaccess.gov/congress/index.html

       Printed for the use of the Committee on Homeland Security
                        and Governmental Affairs



                  U.S. GOVERNMENT PRINTING OFFICE
44-586                    WASHINGTON : 2009
-----------------------------------------------------------------------
For sale by the Superintendent of Documents, U.S. Government Printing 
Office  Internet: bookstore.gov Phone: toll free (866) 512-1800 
Fax: (202) 512-2250  Mail: Stop IDCC, Washington, DC 20402-0001




        COMMITTEE ON HOMELAND SECURITY AND GOVERNMENTAL AFFAIRS

               JOSEPH I. LIEBERMAN, Connecticut, Chairman
CARL LEVIN, Michigan                 SUSAN M. COLLINS, Maine
DANIEL K. AKAKA, Hawaii              TED STEVENS, Alaska
THOMAS R. CARPER, Delaware           GEORGE V. VOINOVICH, Ohio
MARK L. PRYOR, Arkansas              NORM COLEMAN, Minnesota
MARY L. LANDRIEU, Louisiana          TOM COBURN, Oklahoma
BARACK OBAMA, Illinois               PETE V. DOMENICI, New Mexico
CLAIRE McCASKILL, Missouri           JOHN WARNER, Virginia
JON TESTER, Montana                  JOHN E. SUNUNU, New Hampshire

                  Michael L. Alexander, Staff Director
     Brandon L. Milhorn, Minority Staff Director and Chief Counsel
                  Trina Driessnack Tyrer, Chief Clerk


FEDERAL FINANCIAL MANAGEMENT, GOVERNMENT INFORMATION, FEDERAL SERVICES, 
                AND INTERNATIONAL SECURITY SUBCOMMITTEE

                  THOMAS R. CARPER, Delaware, Chairman
CARL LEVIN, Michigan                 TOM COBURN, Oklahoma
DANIEL K. AKAKA, Hawaii              TED STEVENS, Alaska
BARACK OBAMA, Illinois               GEORGE V. VOINOVICH, Ohio
CLAIRE McCASKILL, Missouri           PETE V. DOMENICI, New Mexico
JON TESTER, Montana                  JOHN E. SUNUNU, New Hampshire

                    John Kilvington, Staff Director
                  Katy French, Minority Staff Director
                       Monisha Smith, Chief Clerk


                            C O N T E N T S

                                 ------                                
Opening statement:
                                                                   Page
    Senator Carper...............................................     1

                               WITNESSES
                        Thursday, July 31, 2008

Hon. Paul A. Denett, Administrator, Office of Federal Procurement 
  Policy, U.S. Office of Management and Budget...................     5
Hon. Karen S. Evans, Administrator, Office of Electronic 
  Government and Information Technology, U.S. Office of 
  Management and Budget..........................................     7
David A. Powner, Director, Information Technology Management, 
  U.S. Government Accountability Office..........................     8
Alfred Grasso, President and Chief Executive Officer, MITRE 
  Corporation....................................................    23
Norm V. Brown, Executive Director, Center for Program 
  Transformation.................................................    26
Thomas M. Jarrett, Secretary, Department of Technology and 
  Information, State of Delaware.................................    28

                     Alphabetical List of Witnesses

Brown, Norm V.:
    Testimony....................................................    26
    Prepared statement...........................................   111
Denett, Hon. Paul A.:
    Testimony....................................................     5
    Prepared statement...........................................    39
Evans, Hon. Karen S.:
    Testimony....................................................     7
    Prepared statement...........................................    45
Grasso, Alfred:
    Testimony....................................................    23
    Prepared statement...........................................   100
Jarrett, Thomas M.:
    Testimony....................................................    28
    Prepared statement...........................................   120
Powner, David A.:
    Testimony....................................................     8
    Prepared statement...........................................    50

                                APPENDIX

Questions and Responses for the Record from:
    Mr. Denett...................................................   124
    Ms. Evans....................................................   128
Charts submitted by Senator Carper...............................   134


  OFFLINE AND OFF-BUDGET: THE DISMAL STATE OF INFORMATION TECHNOLOGY 
                   PLANNING IN THE FEDERAL GOVERNMENT

                              ----------                              


                        THURSDAY, JULY 31, 2008

                                 U.S. Senate,      
        Subcommittee on Federal Financial Management,      
               Government Information, Federal Service,    
                              and International Security,  
                          of the Committee on Homeland Security    
                                        and Governmental Affairs,  
                                                    Washington, DC.
    The Subcommittee met, pursuant to notice, at 9:33 a.m., in 
room SD-342, Dirksen Senate Office Building, Hon. Thomas R. 
Carper, Chairman of the Subcommittee, presiding.
    Present: Senator Carper.

              OPENING STATEMENT OF SENATOR CARPER

    Senator Carper. The Subcommittee will come to order. Our 
thanks to our guests and witnesses for being here with us 
today. This is the third hearing that our Subcommittee has held 
on the issue of poorly planned and poorly performing IT 
investments. This hearing will focus once again on the ability 
of the Office of Management and Budget to oversee and provide 
Congress with visibility into the $70-plus billion that 
agencies will spend on information technology this year.
    Up until March of this year, Congress had an extremely 
limited ability to understand why OMB considered an IT 
investment to be poorly planned, and I must commend OMB and in 
particular Ms. Evans for finally releasing this data. Although 
I believe this is a good start, more complete and accurate 
information needs to be shared. I firmly believe that in order 
to hold agencies accountable for their investments, Congress 
and OMB need to work together as partners. The American 
taxpayers demand it.
    Information technology investments, if planned and 
implemented properly, can increase productivity, improve 
efficiency, and reduce an agency's cost, and also enable us to 
provide better service to our constituents. However, some of 
these projects can be extremely difficult to manage and 
mistakes may be made along the way.
    In fact, I think it was Richard Nixon who said the only 
people who don't make mistakes are the people who don't do 
anything. So we know if we try to do something in these complex 
areas, we are going to make mistakes. That is to be expected. 
The key is, I think, to make sure that we don't continue to 
make the same mistake over and over again. However, the 
mistakes that we do make, we learn from them.
    I experienced this firsthand when I was privileged to serve 
as Governor of Delaware. Sometimes we bit off more than we 
could chew and the IT project would eventually spiral out of 
control. When this happened, sometimes we came to the 
conclusion that the best course of action was just to finally 
pull the plug. It was a tough decision a lot of times, but it 
was often the right thing to do in certain cases.
    Unfortunately, many agencies in the Federal Government are 
allowed to spend billions of taxpayer dollars on investments 
that are duplicative, that lack clear goals, and that are 
managed by unqualified individuals. In fact, according to 
recently-released GAO data, some $25 billion in IT investments 
are poorly planned, poorly performing, or both. Even worse, 
some of these projects have been delayed up to a decade and are 
costing us billions more than was originally expected. This is 
simply unacceptable and it makes me wonder whether it is time 
for Congress to pull the plug on some of these failed 
investments.
    Regrettably, Congress still does not have the information 
necessary from OMB to hold agencies accountable and to choose 
where we want to invest scarce resources each year. With risky 
investments such as IT, it is important to increase 
collaboration and visibility, not hinder it. And so far, this 
has not always been the case.
    Since 1994, Congress through legislation called the Federal 
Acquisition Streamlining Act has required agencies to keep 
costs, delivery dates, and performance goals of major 
acquisition within 90 percent of the originally proposed plan. 
OMB then was required to annually report to Congress on agency 
progress.
    Unfortunately, despite this requirement in law, OMB has, I 
am told, only issued three reports in the last 14 years. 
Although my staff tells me we may have received a report around 
midnight last night. Someone here was staying up late and I 
think may have received a fourth report in 14 years in the 
midnight hour.
    Moreover, agencies are required to create an investment 
baseline that takes into account potential risks that could 
lead to increased costs, delayed delivery dates, and reduced 
performance. Agencies then use this baseline to track whether 
an investment is progressing according to plan.
    There are obviously some legitimate reasons why an agency 
may change the original baseline on a given project, but I am 
disappointed to say that some agencies have used rebaselining 
to hide the cost overruns or schedule delays from Congress.
    We are about to hear an extremely troubling report from the 
Government Accountability Office today revealing that almost 50 
percent of all Federal agency IT investments are rebaselined. 
Even more disturbing, some agencies, such as the Department of 
Agriculture, the Department of Commerce, Veterans Affairs, have 
rebaselined, I understand, more than five times on a single 
investment.
    Although agencies are responsible for the excessive 
rebaselining, there is one thing in common between all of these 
investments. Every baseline and rebaseline was approved by OMB. 
Someone, somewhere, in my view, is not fulfilling their 
responsibility to ensure the taxpayer dollars are spent only on 
those investments that are well thought out and truly needed.
    Using the information that was provided to our Subcommittee 
in March, we have created a report card for agencies that take 
into account several criteria related to the planning and 
implementation of IT investments. In fact, we have two report 
cards side by side. I say this as a parent whose children have 
both now graduated from high school and who is used to getting 
report cards. Sometimes my children aren't anxious for us to 
see these report cards. But on the report card on the left, we 
have a list of agencies who received a passing grade, and on 
the right, those who receive a failing grade. If anybody in the 
audience is able to read those, I take my hat off to you. We 
have copies of the report cards that are provided to our 
witnesses and I hope to the press.
    As we can clearly see the Federal Government is miserably 
failing. In fact, half the 28 agencies received an F. Let me go 
ahead and list some agency grades starting with the Department 
of Labor, whose score was about 64 percent. The Department of 
Treasury, 56 percent. Office of Personnel Management, 55 
percent, all the way to the bottom of this chart on the 
right,\1\ down to the Department of Defense with 37 percent, 
followed by the Department of Agriculture with a 36 percent 
score.
---------------------------------------------------------------------------
    \1\ The chart referred to appears in the Appendix on page 135.
---------------------------------------------------------------------------
    About half of the 28 agencies received an F. In total, 
these failing agencies are overseeing $57 billion in IT 
investments, and from what we can tell right now, we are 
getting too little in return for those enormous investments.
    In addition, you might notice on the left, we have a number 
of agencies who receive a passing grade. Several received 
scores of A-plus. They included HUD, the National Science 
Foundation, the Smithsonian Institute--they all got 100 
percent--the Social Security Administration, 99 percent, 
Department of Energy, 93 percent, all the way down to the 
bottom of the chart, coming in at a 69 percent for a D-plus, 
and that is the Department of Justice.
    However, and while we are encouraged by the passing grades 
in the report card on the left, we want to temper our joy by a 
couple of things. HUD received their A-plus partly because they 
reported that every project was being delivered on cost, on 
schedule, and performing as planned. However, as GAO will 
testify today, HUD rebaselined at least one project seven 
times, possibly to mask spiraling costs, and GAO has 
consistently testified that data provided by agencies to OMB is 
oftentimes inaccurate or even incomplete.
    So we don't really have the complete picture, even for 
those agencies that are reportedly doing a better job than 
others. That is why I plan on introducing legislation today, 
along with Senator Lieberman, Chairman of our full Committee, 
and Senator Collins, the Ranking Republican on the full 
Committee, which will give Congress and OMB the information we 
need to make better decisions about which IT investments should 
continue and which should be shut down.
    Our bill, called the Information Technology Investment 
Oversight Enhancement and Waste Prevention Act, would make 
agencies report regularly on significant deviations on cost, 
schedule, and performance. But we don't just want better 
information. Our bill also helps OMB take a crucial step aimed 
at preventing IT investments from drifting toward failure. 
Recognizing that agencies may not have the skills necessary to 
manage complex IT investments and may have trouble recruiting 
qualified managers, our bill would set up a team of experts 
from inside and outside of government that agencies may use as 
a resource. This team, which my staff and I regard as something 
of an IT strike force, would have the skills and background 
necessary to make sure that agencies are focused on the right 
things, making the right decisions, and spending the money 
wisely.
    I again want to thank our witnesses for joining us today. 
We look forward to hearing from you. We look forward to your 
testimony. We look forward to the discussion that will follow 
it and hopefully to the better IT performance, for the money 
that our taxpayers are investing in those projects today. 
Taxpayers expect us to be good stewards of their money. In 
fact, they demand it, and I know that everyone here in this 
room wants to see that become a reality.
    Before I introduce Mr. Denett, Ms. Evans, and Mr. Powner, 
none of whom are strangers to this Subcommittee or to this 
room, I realize these are tough issues. These are not easily 
done. If they were, somebody would have done them a long time 
ago. The problems were recognized not just in this 
Administration, but in the last Administration, as well. While 
we appreciate that the problems are being identified, what is 
really important is that we identify them more quickly and that 
we fix them. I believe that we need good information here in 
the Legislative Branch so that we can be better partners with 
the Executive Branch.
    We are going to see a new Administration coming in in 6 
months. It is going to be President McCain or President Obama. 
But these problems and these challenges will still be before 
the next Administration and the next Congress.
    As Governor, I used to say the reason why we invested money 
in these IT projects was to enable us to provide better 
constituent service and to do it more cost effectively, and 
that is really the case here, too. In some areas, we are doing 
a pretty good job. In too many areas, though, we are not. In 
too many areas, we are masking mismanagement, misallocation of 
resources and cost overruns in ways that make it look like we 
are doing a better job on some projects than we truly are. We 
have to get beyond that.
    With that having been said, on that cheery note, we have a 
couple of panels before us. In the interest of time, I am just 
going to move right to the first panel for their testimony. 
Biographies of our witnesses are provided and will be submitted 
for the record.
    Our first witness is the Hon. Paul Denett. Mr. Denett 
serves as the Administrator of the Office of Federal 
Procurement Policy at the Office of Management and Budget. 
Previously, Mr. Denett served as counselor to Clay Johnson, who 
is now the Deputy Director for Management at OMB. Mr. Denett is 
a retired Senior Executive from the Federal service and has 
received many prestigious awards, including a Presidential Rank 
Award. When did you receive that?
    Mr. Denett. I received that in the late 1990s.
    Senator Carper. Did you really? Who was President then?
    Mr. Denett. President Clinton.
    Mr. Denett. Yes, it was.
    Senator Carper. All right. Some of us are lost in the 
1960s. [Laughter.]
    Some days, I would like to be, but not today. [Laughter.]
    All right. Our next witness is the Hon. Karen Evans. Ms. 
Evans, welcome. She is Administrator of the Office of 
Electronic Government and Information Technology at OMB. Ms. 
Evans oversees the implementation of IT throughout the Federal 
Government, including capital planning and investment control. 
She is a 20-year veteran of government service and has 
testified before this Subcommittee on a number of occasions. It 
is a pleasure to have you join us again today. Thank you.
    Our final witness is Dave Powner, Director of Information 
Technology Management at the Government Accountability Office. 
I am glad that Mr. Powner doesn't charge us for each visit that 
he makes and each time he testifies before this Subcommittee, 
because we would be owing him a lot. But in the private sector, 
he has held several executive level positions. In the 
telecommunications industry, Mr. Powner has been instrumental 
in helping this Subcommittee provide oversight of risky IT 
investments, along with other things. I know he has appeared 
before us on several occasions, including as we try to address 
our current challenges involving conducting a good Census for 
the year 2010.
    We are going to lead off and ask Mr. Denett to lead off. We 
will ask you to keep your statements to roughly 5 minutes. If 
you go a little bit beyond that, that is OK. If you go a lot 
beyond that, that is not OK. So we will turn to you now. As you 
present your testimony, just keep in mind that your full 
statements will be entered into the record.

 TESTIMONY OF HON. PAUL A. DENETT,\1\ ADMINISTRATOR, OFFICE OF 
   FEDERAL PROCUREMENT POLICY, U.S. OFFICE OF MANAGEMENT AND 
                             BUDGET

    Mr. Denett. Thank you. Chairman Carper, Ranking Member 
Coburn, and Members of the Subcommittee, I appreciate the 
opportunity to appear before you today. The Administration 
places a high priority on working with agencies to mitigate 
cost overruns, schedule delays, and performance shortfalls in 
their major acquisitions.
---------------------------------------------------------------------------
    \1\ The prepared statement of Mr. Denett appears in the Appendix on 
page 39.
---------------------------------------------------------------------------
    This morning, my colleague, Karen Evans, will discuss the 
initiatives the Administration is pursuing to effectively 
manage the government's IT portfolio. I would like to briefly 
summarize agencies' progress in implementing performance-based 
management for their major non-IT acquisition programs. I have 
also prepared written remarks that I ask the Subcommittee to 
enter into the record.
    Similar to IT, major acquisitions of non-IT capital assets 
must be justified in terms of agencies' strategic goals and 
reflect sound acquisition and capital planning decisions. The 
law requires agencies to apply performance-based management 
principles by establishing cost, schedule, and performance 
goals and achieving 90 percent of these goals, on average.
    OMB guidance for meeting these requirements is set forth in 
Circular A-11, which provides guidance to agencies on preparing 
their budget submissions, and the Capital Programming Guide, 
which is a supplement to A-11. The guide was substantially 
revised in 2006 to emphasize the importance of key steps in the 
acquisition planning process, such as needs assessment and 
alternative analysis.
    To assess the use of performance-based management, my 
office directed agencies to provide information on their new 
and ongoing non-IT major acquisition projects. We look for 
several indicators of progress, including the existence of 
capital planning and investment control policies, cost schedule 
and performance goals for new and ongoing projects, use of 
performance-based management systems to monitor progress and 
success in meeting goals.
    Results were mixed. While many agencies demonstrated 
progress, we found that capital planning policies for non-IT 
are often not as well established as they are for IT 
investments. In addition, performance-based management systems 
are not always being used to track cost schedule and 
performance. We can and must do better.
    A number of steps are being taken to strengthen the 
application of performance-based management to non-IT 
investments. First, the Chief Acquisition Officers' Council 
created a Project Management Working Group to help OMB evaluate 
the appropriate application of performance-based management to 
different types of major non-IT investments, such as 
construction and aircraft.
    Second, OMB and GAO are partnering with DOD, DOE, and NASA, 
each of whom is on the GAO's High-Risk List, in the development 
and implementation of corrective action plans to mitigate risk 
in major acquisitions and evaluate success against clear goals 
and metrics.
    Third, the Office of Federal Procurement Policy and the 
Federal Acquisition Institute launched a Federal Acquisition 
Certification Program to provide civilian program and project 
managers with standardized training on competencies critical to 
successful performance-based management. These include 
requirements analysis, cost estimating financial management, 
risk management, and quality assurance. Program and project 
managers that are assigned to major acquisitions will need to 
be certified under this program.
    Finally, OFPP created a standardized approach for agency 
self-assessments of the acquisition function based on a 
framework that was created by GAO. Self-assessments of major 
acquisition functions will address issues to help agency 
managers understand if planning is effective. If a material 
weakness is identified, it will be addressed during corrective 
action monitoring and reflecting on the agency's statement of 
assurance prepared under OMB Circular A-123, which lays out 
management's responsibilities for internal controls.
    In accordance with the Federal Acquisition Streamlining 
Act, OFPP issued a report--I think it was a little before 
midnight, but it was late--discussing civilian agency progress 
in implementing performance-based management----
    Senator Carper. Actually, I am told it was 12:04. 
[Laughter.]
    But who is keeping time. [Laughter.]
    Mr. Denett. OK. For both IT and non-IT investments. We will 
be happy to answer any follow-up questions the Subcommittee may 
have. Thank you.
    Senator Carper. Right on the money. Thank you. Ms. Evans, 
welcome.

   TESTIMONY OF KAREN S. EVANS,\1\ ADMINISTRATOR, OFFICE OF 
 ELECTRONIC GOVERNMENT AND INFORMATION TECHNOLOGY, U.S. OFFICE 
                    OF MANAGEMENT AND BUDGET

    Ms. Evans. Good morning, Mr. Chairman. My remarks will 
focus on the Administration's strategy and continued progress 
in performing oversight of agencies' capital investments and 
information technology. Specifically, I will address the topic 
of OMB's ability to effectively analyze, track, and evaluate 
agencies' major capital IT investments.
---------------------------------------------------------------------------
    \1\ The prepared statement of Ms. Evans appears in the Appendix on 
page 45.
---------------------------------------------------------------------------
    The President's budget request for fiscal year 2009 
includes approximately $71 billion for IT and associated 
support services government-wide. Of the $71 billion in the 
President's budget request, $22 billion, or 31 percent, 
represents proposed funding for development, modernization, or 
enhancement of new or existing information systems, 
infrastructure, or services. Projected expenditures on existing 
systems, operations, and maintenance is $49 billion, or 69 
percent.
    As I have discussed in previous testimony on the topic of 
OMB's oversight of IT investments, the Clinger-Cohen Act of 
1996 established processes for executive agencies to analyze, 
track, and evaluate the risks and results of major capital 
investments for information systems. The operative means by 
which OMB evaluates agency capital asset plans and their 
associated budgetary requests is a business case, or Exhibit 
300, for individual investments.
    Agencies develop and submit business cases with their 
annual budget requests to OMB and we, in turn, evaluate each 
business case in terms of its ability to support a given 
investment proposal, including factors such as alignment with 
IT architecture, plan performance, improvement goals, cost-
benefit analysis, and eliminating costly, duplicative, and 
outdated systems. OMB also evaluates the capability to manage 
the investment as demonstrated in the business case and 
planning process, including factors such as having a qualified 
project manager, acquisition planning, systems security, risk 
management, and the use of earned value management to track and 
manage costs and schedule goals.
    Last year, I came before this Subcommittee to explain two 
specific tools OMB uses to track and review agency IT capital 
investments, the Management Watch List and the High-Risk List. 
I am pleased to report that we have since expanded upon our 
criteria for evaluating investments and improved upon the 
transparency of how investments and projects are placed on 
either or both lists. My written statement includes the details 
regarding these lists.
    GAO's report on project rebaselining points out weaknesses 
in agencies' policies in terms of specifying all of the 
elements of rebaselining according to best practices. We 
acknowledge more should be done in this area in the future. In 
particular, OMB needs to clarify expectations for when an 
original baseline should be established and elaborate upon 
process steps for agencies to submit to and receive feedback 
back from OMB on rebaseline requests.
    OMB, Congress, and the agencies must work collaboratively 
to address weaknesses in IT program and project performance. We 
are all vested in a common interest, delivering results for the 
American people.
    How can we best do this going forward? I believe OMB has 
the foundational processes in place to perform program and 
project investment oversight at the macro level. These core 
processes--capital planning, architecture alignment, the 
Management Watch List, the High-Risk List, and the E-GOV 
scorecard--have been and can continue to be further enhanced 
and incrementally improved to provide better visibility into 
program performance before a project is in major trouble.
    One improvement for OMB would be to leverage the efforts of 
the budget formulation and execution line of business to ensure 
the use of analytical tools and the collaboration environment 
to improve on our own information management capabilities. The 
various data sets collected by OMB from the agencies can and 
should be better integrated into a more comprehensive knowledge 
base.
    I have brought a display of an example of how we could 
better integrate and expand upon information on the agency's IT 
investments. This is one potential snapshot of project 
performance at a point in time. We would like to work with 
Congress to improve transparency and the ease of use of this 
information. With the transparency into departments' and 
agencies' performance will come improved accountability and 
results which the American people deserve.
    Thank you for this opportunity to discuss the 
Administration's strategy and progress to date. With this 
foundation in place, we can continue to work together to 
achieve the outcome we both desire, which is successful 
implementation of information technology for program and 
mission results. We have accomplished a lot in the last 8 years 
and there is much more that we can continue to do in this area.
    I would be happy to take questions at the appropriate time.
    Senator Carper. Thank you for that testimony. Mr. Powner.

    TESTIMONY OF DAVID A. POWNER,\1\ DIRECTOR, INFORMATION 
  TECHNOLOGY MANAGEMENT, U.S. GOVERNMENT ACCOUNTABILITY OFFICE

    Mr. Powner. Chairman Carper, we appreciate the opportunity 
to testify this morning on poorly planned and performing IT 
projects and the results of our rebaselining review completed 
at your request.
---------------------------------------------------------------------------
    \1\ The prepared statement of Mr. Powner appears in the Appendix on 
page 50.
---------------------------------------------------------------------------
    Mr. Chairman, I would like to thank you and Dr. Coburn for 
your oversight of the Federal IT budget, which now exceeds $70 
billion. This is the third annual hearing, and your oversight, 
along with the leadership of Karen Evans, has resulted in a 
clear picture of exactly what projects need work and the 
specific weaknesses that need to be addressed. That is the good 
news. The bad news is that we have nearly $25 billion in IT 
investments that are currently at risk of being wasted and that 
this figure is likely much higher.
    This morning, I have three points to make. First, over 400 
IT projects totaling nearly $25 billion are currently not 
appropriately planned for or managed.
    Second, the number of projects reporting issues with cost 
and schedule variances is understated because rebaselining is 
occurring excessively without adequate guidance or 
transparency.
    And third, additional oversight from both OMB and agency 
CIOs is needed to address project weaknesses and to bolster 
accountability.
    Expanding on each of these, first, currently, there are 
about 350 projects totaling $23 billion on the Management Watch 
List, and nearly 90 projects totaling $5 billion that are being 
reported as high-risk projects with shortfalls. Common to both 
lists are 26 projects totaling about $3 billion, meaning that 
these projects are both poorly planned and performing. For 
example, DHS's Secure Border Initiative project continues to be 
on both lists.
    In addition to the 26 projects on both lists, of particular 
concern are the repeat offenders, namely 32 projects that have 
been on the Management Watch List since September 2006 and 17 
high-risk projects reporting shortfalls in each of the last 
four quarters. My written statement highlights each of the 
projects on both lists and those that have been on either list 
far too long.
    Mr. Chairman, having so many projects with longstanding 
planning and performance problems is unacceptable and requires 
more attention from both OMB and agency CIOs. Key reasons why 
projects remain on the Management Watch List are poor cost and 
schedule performance, poor security measures, poor privacy 
planning, and poor project management, while cost and schedule 
variances continues to be the primary performance shortfall 
associated with high-risk projects.
    Despite cost and schedule performance being a major reason 
why so many projects are highlighted here, our rebaselining 
report being released today raises significant questions about 
whether we are getting an accurate picture of project cost and 
schedule performance.
    First, I would like to mention that rebaselining is needed 
at times to reset realistic cost and schedule targets. However, 
this process should not be used to mask cost and schedule 
overruns and should be transparent and approved by both OMB and 
agency management. Our survey of nearly 200 projects showed 
that about half of all IT projects had been rebaselined, and of 
those that are rebaselined, about 50 percent are rebaselined at 
least twice, and 10 percent are rebaselined four or more times. 
Clearly, these projects are operating nowhere near the desired 
10 percent cost and schedule threshold.
    Because of the amount of rebaselining occurring across the 
Federal Government, we reviewed each of the major 24 
departments' rebaselining policies to ensure that it included 
basic items like specifying when a rebaselining is warranted, 
how a new rebaseline is validated, and who approves the new 
rebaseline. Agency policies were weak across the board and we 
recommended that OMB issue rebaselining guidance and that each 
agency develop rebaselining policies that address the 
weaknesses we identified.
    Although we have more data than we have ever had 
historically, we still need better information so that all 
projects requiring attention are highlighted. For example, OMB 
could publicize high-risk projects with shortfalls as we have 
in our written statement. And we still need more transparency 
on rebaselining efforts so that we have a true picture of cost 
and schedule performance from all agencies.
    Put more simply, Mr. Chairman, do we think that of the 472 
high-risk projects only 70, or 15 percent, have cost and 
schedule issues? No. That number should clearly be higher.
    Now turning toward solutions, the Federal Government needs 
to focus on addressing root cause problems: Security, project 
management, risk management, and cost and schedule performance. 
We are encouraged by OMB's efforts to address some of these 
weaknesses, like requiring techniques like earned value 
management to improve agencies' cost and schedule performance, 
but there is much more work ahead. Agency CIOs and OMB from a 
government-wide view need to aggressively attack these problem 
projects by starting with those that have been on OMB's radar 
far too long.
    Next, longer-term improvement efforts need to be pursued 
and these, at a minimum, need to focus on bolstering the IT 
workforce and addressing root causes more aggressively. I look 
forward to suggestions from our second panel of experts on 
other approaches.
    In summary, Mr. Chairman and Dr. Coburn, knowing what to 
fix is the first step, and we wouldn't have this information 
without your leadership. Thank you. Keeping the current 
momentum and energy and focusing more on solutions will be 
essential as we transition to a new Administration.
    I would be pleased to respond to questions.
    Senator Carper. Good. Thank you for that testimony. Thanks 
very much for the work that preceded it. On behalf of both 
Senator Coburn and myself and our Subcommittee and staff, thank 
you for the very good work that you have done.
    What I hope we focus on more today--yes, the sun is coming 
up. I don't know if it seemed dark to the rest of you, but it 
seemed dark to me. I just asked my staff to brighten the 
lights. So if you are not awake, wake up now. [Laughter.]
    I was wondering, who are some good models on the public or 
private sector for the Federal Government? Who out there is 
managing IT projects well? Not just in planning and developing 
them, but implementing them on time and fairly close to budget 
without rebaselining six or seven times on a single project? Do 
we have any idea who is doing an especially good job and why?
    Mr. Powner. I can tell you from our focus on some of the 
high-risk modernization efforts that we look at, even though 
some of these organizations received low department grades, if 
you look at pockets within the Federal Aviation Administration, 
if you look at IRS, recently there have been successes with 
delivering more projects on time and within budget. And if you 
look at what has resulted, there has been an extreme focus on 
improving their processes, executive governance over those 
projects, and one other technique that seems to have worked at 
some of those agencies is putting in place Centers of 
Excellence where project offices can go for help, where there 
is best practices and things they can emulate across the board. 
They are not perfect by any means, but there are pockets of 
success in both of those agencies.
    Senator Carper. Thank you for that response. In the 
National Governors Association, all 50 governors are members of 
that, we had a Center for Best Practices and if they had a 
particular State that was doing a good job in early childhood, 
the rest of us could learn from them. If we had a particular 
State that was doing a good job in reducing recidivism in the 
prison population, or if a State was doing an especially good 
job in providing better outcomes in our schools, we had the 
opportunity through the Center for Best Practices to see who 
was doing a good job and learn from them. Do we have that kind 
of ability in the Federal Government?
    Ms. Evans. Yes, sir. Through the CIO Council, we have a 
Best Practices Committee where we do take on topics such as 
these specific areas, like who has implemented earned value 
management really well and can people learn from that. We need 
to do more in those Best Practices Committees, and I would also 
like to build off of what Mr. Powner talked about.
    What we have been doing from an OMB perspective is really 
zeroing in in particular areas. The two areas that Mr. Powner 
mentioned are also on the GAO High-Risk List. So there is a lot 
of work that we have done in conjunction with the agency, in 
conjunction with GAO, so that we could then cross-pollinate the 
expectations, the best practices in those areas, and then the 
intent is to take that and then share it back across the board 
with other agencies.
    So we have done that in the area of, with NARA and the 
Electronic Archives Initiative. As they go forward, what we 
have learned from FBI on their Sentinel project is they have 
improved. We have then partnered them up with other agencies as 
they are going forward on major investments so that they make 
sure that they don't make the same mistakes.
    Senator Carper. Mr. Denett.
    Mr. Denett. We also have the Chief Acquisition Officers 
Council, and this past year to try to respond to the problems 
in the program project management area formed a new working 
group. It is chaired by Bill McNally, who is the Senior 
Procurement Executive of NASA. But all the major departments 
send representatives and they are sharing best practices and we 
are going to post them on the Federal Acquisition Institute 
website.
    Senator Carper. All right. Mr. Powner.
    Mr. Powner. I would just like to add that I agree with the 
comments that were made here, but one important thing that I 
think Ms. Evans made, an important comment, is the 
implementation of those best practices. Earned value 
management, a very good technique when used appropriately. Our 
preliminary work on earned value management implementation 
across the government is that it is rather immature, and I will 
give you an example.
    We just completed a review of FAA. It is likely one of the 
better agencies when it comes to implementing earned value. 
There has been a lot of focus and attention. But I can tell you 
there that a quarter of their projects don't use earned value. 
Not everyone is trained. And when you start looking at the 
data, the earned value data--and we did, we dove deep on a 
couple projects--we saw quite a few problems that we pointed 
out.
    Now, granted, that is probably one of the better Federal 
agencies, but when we look at the cost and schedule problem 
that we have in the Federal Government coupled with 
rebaselining, we have to get much better at using these 
techniques like earned value to make good management decisions 
to get early warning indications.
    Senator Carper. All right. Thank you. The next question I 
want to ask, and I have touched on this already, but we started 
this hearing off with a little broader context than I think we 
had originally planned to discuss, but it is something that I 
am troubled by and I just want to come back to it again and 
discuss it.
    One of the Subcommittee's purposes is, as you know, to 
prevent waste, fraud, and abuse. In order to do that, we need 
information on how agencies are managing our investments. Mr. 
Denett, in 1994, Congress directed your office to provide a 
report on agency progress in delivering on their capital 
investments, including IT investments. However, as I stated in 
my opening statement, in 14 years, OMB has only provided this 
report on three separate occasions, now maybe four. Let me just 
ask, why is OMB not fulfilling this critically important 
statutory requirement?
    Mr. Denett. It was a mistake on our part, for sure. When I 
became aware of this, we have had the staff work diligently, go 
out to departments and pull it in, and I don't have a good 
explanation as to why some years were not submitted. However, 
there is the annual budget process under A-11 where a lot of 
the data that is included in the report is submitted with the 
President's budget each year, so that contains some of the 
information we sought. But it should have been wrapped up, put 
into a formal report, and provided to you each year. We have 
now put steps in to make sure that this doesn't happen in 
future years.
    Senator Carper. What can you do? Who is responsible for 
making sure this statutory requirement is met?
    Mr. Denett. The Office of Management and Budget, so we are.
    Senator Carper. All right. What assurance can you provide 
for us that it is going to be met in the future?
    Mr. Denett. We have added it to a follow-up list that OMB 
has so that the new Administration will have it as a reminder 
before them. We have told the staff the importance of it, and 
it is not our prerogative to decide what years to submit it and 
what not, that we all need it as a tracking tool. This one will 
be stronger. We are going to have our own baselines on which 
ones are--just as you are having your report card, we are going 
to have that kind of method in there, and I believe in the 
future, you will get the reports each year.
    Senator Carper. All right. I certainly hope so.
    Ms. Evans, this is the third hearing during which you have 
testified how billions of taxpayer dollars are wasted on bad IT 
investments. I want to commend you for all the progress that 
you and your team have made over the past several years. 
However, I believe that there is still more that needs to be 
done. I am sure you do, too. In fact, you said that.
    Specifically, in 2006, GAO first recommended that OMB 
ensure agencies are accurately and reliably reporting on their 
investments. Also, GAO recommended that OMB provide Congress a 
single aggregate list for troubled IT investments that can 
allow us to track progress and performance government-wide, and 
it is now 2 years later.
    Just revisit with us, if you will, what specific progress 
since 2006 have you and your colleagues made toward ensuring 
that agencies are reporting complete and accurate information 
to OMB. And second, why doesn't OMB publish the list of high-
risk projects with shortfalls as GAO highlights in their 
testimony? I am just asking, don't you feel that Congress would 
want to know whether projects are over-budget and behind 
schedule?
    Ms. Evans. Sir, I actually would like to answer the first 
question first----
    Senator Carper. Please.
    Ms. Evans [continuing]. Which would be what we have done to 
increase the transparency in this. And I would say, first off, 
with the hearing and everything and the several hearings along 
this line, that I was hesitant at first, which I think you 
would admit, to releasing all of this information because we 
were concerned about some of the effects that it would have.
    Senator Carper. What do you mean----
    Ms. Evans. I was going to go into this a little bit. And 
some of it, you are talking about and we have kind of gone 
around it a little bit, which is driving compliance versus 
actually achieving the results. And so things happen like the 
grades, not that this is not great because OMB has an A, so I 
am pleased that we have an A. But it is an A-minus. I would 
have preferred an A-plus, but I know why it is a minus. But 
what this will do is drive----
    Senator Carper. I asked my staff if we were grading on a 
curve here. They said, no, we are not.
    Ms. Evans. No, but I don't have as many investments as 
everyone else, so----
    Senator Carper. We are calling them as we see them.
    Ms. Evans. But part of the issue is, one, that there is a 
part of us that love to see the grades, and so we strive for 
this. This is how our society is built. But the other part then 
drives compliance, where people will do just enough in order to 
get to the next level of the grade and they are focused more on 
the grade than actually achieving the result or getting the 
project implemented.
    And so this is a balance that my office has really been 
struggling with, about how much information do we release, how 
much shame and embarrassment do you bring upon an agency, 
because we are really supposed to be helping them. We are 
supposed to be there to help the agency achieve success. We are 
supposed to take areas in our knowledge that we see across the 
government and go in and help an agency achieve results.
    So I agree that all this information should be out there. I 
don't necessarily disagree that we could provide better 
reporting, not necessarily along the lines of GAO, because I 
don't view OMB as an auditor. I view OMB as a facilitator to 
help the Executive Branch achieve results.
    So I would like us to be able to provide the information in 
a way that you would find it useful as an oversight entity the 
same way that we find it useful so that we could get to the 
solution for the agency and for the taxpayers.
    So we have done a lot to put all this out here. We have not 
put together the comprehensive list the way that GAO does or 
the way that it was written. I brought two examples, which are 
also in the written testimony, and if we want later on, I could 
go into a couple areas of how it could be done. We put a lot of 
effort into this so that we could match and cross-reference and 
make sure that all of this is reliable data, and then we could 
work with you in a way similar to the way that we have done 
with USA Spending and even release some of the raw data so that 
as you go forward, you guys could manipulate and do things with 
the data, as well.
    So I am always very cautious as we go forward on this 
because I am always really cautious about driving compliance 
versus trying to get to the result.
    Senator Carper. All right. A related question. You may have 
spoken to this, but is there a reason why OMB hasn't provided 
Congress with a single aggregate list that will allow us to 
provide proper oversight and track progress? Just try that. I 
think you have answered that question, but I want you to take 
another shot at it.
    Ms. Evans. I was hoping you didn't realize I didn't answer 
that part---- [Laughter.]
    But I will tell you, the reason why, and this is our 
first--the tables that you have are our first attempt at 
aggregating the data together. The short answer is that based 
on what we had previously to provide the aggregated report, we 
believe would cause more confusion than actually bring clarity 
to the result. And then the other piece was we had to work a 
little bit more with the agencies to get better quality data in 
order to establish the relationship to do an aggregate report. 
I believe we are there now and what we would like to work with 
you on is how to actually provide that information in a way 
that would be useful for your use.
    Senator Carper. Why not just provide the same information 
as GAO?
    Ms. Evans. Because I would say that right now, we have 
several pieces of that, but we don't go in and do full audits 
the same way that GAO does. So some of the information, for 
example, like the number of times that a project has been 
rebaselined, we would have to modify some of our A-11 guidance. 
Right now, we just capture that information as a yes and no, 
and I have not captured that information on an ongoing basis so 
that I could go down and map all the records and say that this 
investment has been rebaselined several times, X, Y, Z, and 
this year, those types of things like that.
    I have not kept the information at the micro level. The 
agencies are supposed to keep it at that level. And what we are 
trying to do is make sure that there is better quality as we 
start putting it forward. So I would be hesitant to publish the 
information because I do not feel that we have the quality 
still at that real detailed level because we haven't conducted 
the audit as GAO has.
    Senator Carper. I am going to ask Mr. Powner to comment on 
some of what Ms. Evans has just said.
    Mr. Powner. A couple thoughts here. One, first of all, I 
think Ms. Evans and OMB, they deserve a lot of credit for all 
the information we have on the Management Watch List, which are 
poorly planned projects. OK, you want to fix the planning up 
front.
    But the High-Risk List is not that important. What is 
important is Figure 4 on page 19 in my testimony, which shows 
those projects that have shortfalls and the reasons for the 
shortfalls. So the No. 1 reason why we have shortfalls, and 
this is about performance now, it is not just about planning, 
but we are expending funds and we have shortfalls. We are not 
meeting cost and schedule goals. There are 70 projects. What I 
am hearing is I think the Administration--I am going to cut 
right to it--they are reluctant to highlight projects with 
shortfalls.
    Senator Carper. Why do you suppose that is?
    Mr. Powner. I think the discussion was along the lines of 
embarrassing agencies and that type of thing. You can't fix 
problems----
    Senator Carper. There is a reluctance to embarrass them?
    Mr. Powner. I believe so, but I don't want--I will let OMB 
comment further on that. I don't think you can fix shortfalls 
without fully disclosing all your problems, so you need to take 
those shortfalls--we have 10 percent--we have 70 projects that 
aren't meeting the 10 percent threshold. That number is greatly 
understated, and I will tell you why. Earned value 
implementation is weak at many agencies, so some of that data 
is not based on reliable information coming from the agencies, 
and then the point about rebaselining, OK.
    If we had transparency on rebaselining--that number right 
there says that 85 percent of the high-risk projects are 
meeting the 10 percent cost and schedule threshold. It is 
likely--that 85 percent, you are walking on water if you are a 
private, public organization, 85 percent of your projects are 
meeting within 10 percent. You are doing very well, extremely 
well.
    So we need accurate data there, but we need to disclose all 
that information in order to fix those shortfalls, so that is 
why we, in our testimony, highlighted the important bit of 
information here are the high-risk projects with shortfalls so 
that we can fix performance. I actually think this is very 
consistent with the Administration's President's Management 
Agenda and their E-GOV scoring. One of their criteria to get to 
green is you operate within 10 percent of costs and schedule 
using earned value techniques. So we are all talking the same 
thing here and we have the same goal. It is a matter of 
disclosing all the weaknesses and then fixing it.
    Senator Carper. Ms. Evans.
    Ms. Evans. I would like to respond to this.
    Senator Carper. Please.
    Ms. Evans. First and foremost, as the Administrator of 
this, I think it is my responsibility to make sure that I have 
done everything that I can for the agencies to make sure that 
the guidance, the policy, everything is clear, and what GAO has 
highlighted is that there is a systemic problem throughout the 
government. So we could be focused on the one-offs and let us 
zero in on this one particular project or let us zero in on 
this. But the tenets of the President's Management Agenda is 
getting good management foundation processes, with these types 
of things in place, so that no matter what the project is, no 
matter what the investment is, the agency is going to succeed.
    So when you start looking at the GAO report and you look at 
Table 5, which talks about who is consistent with the best 
practices, and going across and looking at that, what you 
want--I am looking at, what is the problem across the board, 
and right now, what I have gotten from all of this is that I 
need to do a better job of clarifying what is a baseline. How 
do you do integrated baselines? How do you set certain things 
up? What are the best practices, so that the agencies have the 
tools to succeed.
    If I am going to sit there and highlight one or two 
projects and say, Agency A is a real screw-up and they are not 
doing PDQ, we are going to drive behavior down underground and 
these issues of rebaselining are going to continue on and on 
and on because they are going to want to hit the mark of being 
at the 90 percent because grades are coming out. So I think it 
is my responsibility to do the things that I can in the first 
place to make sure that it is clear to the agencies.
    Now, once I believe that I have done everything that I can, 
that we have put everything in place, that it is totally clear 
in the agencies, the bulk of the agencies are producing, then I 
have no problems providing all the information, putting the 
transparency out there, having accountability, because that now 
becomes a different issue and that is a leadership issue within 
the agency itself.
    But right now, Mr. Powner has made it very clear that it is 
an immature process, at best, what we have in the agencies, and 
so they should be rewarded for moving forward and being 
forthcoming in the information that they have so that we can 
address and fix the problems, not that I feel passionately 
about this.
    Senator Carper. Well, I would be disappointed if you 
didn't.
    Mr. Powner, do you want to say anything else? Go ahead.
    Mr. Powner. Just to piggyback off of what Ms. Evans was 
saying there, I think the point, too, about the accountability 
at the agency level, agency CIOs are accountable for these 
fixes. I mean, we need to highlight them, and it is not just 
all in OMB's lap, but the agency CIOs are accountable here.
    Senator Carper. Has OMB issued guidance on rebaselining?
    Ms. Evans. The policy memos that we have in place are high-
level. They refer back to the Capital Planning Guide. They also 
refer back to A-11. But I, in preparation for this hearing, 
have gone back all through those, and maybe it is because I 
live and eat and breath this, it seems clear to me, but when I 
am looking at all the results across the board, I think there 
are other things that we can do like putting together a 
framework document that would show them how to do certain 
things the same way that the CIO Council put together a 
framework for project managers and made it very clear what a 
tier one project was, what a tier two project was, and those 
types of things. I think we need to go down now to another 
level and give them another set of tools in order to be able to 
improve their performance.
    Senator Carper. Mr. Powner, should OMB issue guidance on 
rebaselining?
    Mr. Powner. Yes. There clearly is some clarification that 
needs to occur so that agencies have clear direction going 
forward. Our report highlighted very simplistic items that the 
policy should cover. For instance, when you establish a new 
baseline, how do you validate it? That is very consistent, Ms. 
Evans, with the integrated baseline reviews that you require 
and those types of things. So it is not inconsistent with 
things that OMB is already endorsing.
    But the other key part of their policy is it has to be 
approved by management. We found that a lot of policies didn't 
even require when agencies rebaseline an approval from key 
managers. That is all about the transparency thing so that, in 
fact, we are getting accurate data on costs and schedule 
performance.
    Senator Carper. Mr. Denett, does your office collect 
information on capital investments?
    Mr. Denett. That is through the A-11 process, through the 
budget. So we get these 300s and the budget side, the Resource 
Management Office culls through those and looks at them to see 
which ones are using earned value management and which ones are 
behind or doing well on cost and schedule.
    Senator Carper. OK. What do you do with that information?
    Mr. Denett. The Resource Management Office looks through 
them and goes back, challenges them, sometimes sends them back, 
asks for more data, asks them what they are doing to fix it, 
and it all rolls up into the budget process. But, important 
also is the caliber of people we have working with this, and as 
I mentioned earlier, we are real pleased that we now have 
certification requirements for the managers and people assigned 
to this, contracting officers, program managers for the first 
time have to meet certain experience levels and they have to 
take mandatory training. All of that will also assist in doing 
a better job in this area.
    Senator Carper. All right. Let me just ask you what 
Congress' responsibilities are, Ms. Evans and Mr. Denett. I 
think we understand that GAO is our watchdog to help us better 
ensure that you are doing your jobs well. What do you see as 
the appropriate role for Congress in our oversight 
responsibility as the Legislative Branch? We don't pretend to 
be experts on these hundreds of different IT projects or 
capital investments. At the same time, we have an obligation to 
authorize and appropriate monies to fund these projects. If we 
don't have information that is timely and that is 
understandable, we are not able to do a very good job as 
legislators in our oversight capacity. What do you see as our 
appropriate role that would enable us to be constructive?
    I think part of it is to embarrass. I am a person who 
believes in rewarding behavior. I am a big positive reinforcer. 
But I think the time comes when people need a swift kick in the 
pants and there are times when folks just need to be 
embarrassed. Sometimes, I think that is an appropriate role for 
us. Sometimes, it is an appropriate role for OMB, and certainly 
for GAO.
    But let me just ask your thoughts about how the Congress, 
including this Subcommittee, can be more constructive. Mr. 
Powner, any thoughts?
    Mr. Powner. Well, I would agree that you need to shine a 
spotlight on the problems. That is very helpful at times and 
perhaps it is public embarrassment, but it gets action. 
Hopefully, over time with the oversight, we can move down the 
road here and focus more on the solutions. I mean, we know all 
these projects, we know the agencies, the projects, and the 
problems with specific projects. We know what the issues are. 
Now we need to put the fixes in place to look at how we 
implement things going forward. I think your second panel is a 
good start, where we start looking at that. But again, we are 
going to probably continue to need to step back to make sure 
that you are getting accurate information because I contend 
that there is still some information here that is understated.
    Senator Carper. We have talked about some of it, but just 
be more specific. You say you contend that some of the 
information is understated. Just elaborate on that some more. I 
just want to stay on this point.
    Mr. Powner. Out of 800 major IT projects, reporting 70 that 
have performance shortfalls on cost and schedule is an 
understatement.
    Senator Carper. All right. Fair enough. Ms. Evans, my first 
question.
    Ms. Evans. If I could point to----
    Senator Carper. No, I want you to come back to my question 
what----
    Ms. Evans. Well, I am. I am going to give you an example.
    Senator Carper. We spent plenty of time telling you how you 
ought to do your job.
    You have a chance to give us some advice on how we can 
better do our job so you can do your job better.
    Don't pass up this opportunity.
    Ms. Evans. I know, and so I do believe that there would be 
indicators off of the information that we provide to you that 
by having hearings, and I don't disagree that having public 
hearings ensures transparency and accountability, and you can 
use this set of hearings as an example because there is a 
particular area when we first came up here you had no 
information. You just had gross numbers of what we were saying 
and you kind of had to trust us that we were telling you the 
right thing.
    So I think that if the agencies come up here with the 
mindset along the lines that you are there to achieve the same 
goal as ourselves, that then it becomes a partnership. And I 
think that the hearings themselves and the way that you have 
approached the hearings has allowed for that partnership to 
happen, and so that is why we are at the point with all the 
information that is out there.
    But to your point about--and on one of the charts it has 
the Census project, which we can all talk about. By having 
those hearings, it does have the agency very focused on it. The 
way that the policies are set up, there are indicators on there 
that you could see it before they have to do a budget 
amendment. There are things that you can do based on pulling 
this information out that if you had a particular area that you 
were interested in, you can request the more detailed 
information. GAO can request the more detailed information from 
the agency themselves.
    We have set up everything that way so that you can get 
those quarterly reports, the monthly reports, everything that 
goes behind the high-level numbers, and by talking with the 
agencies or having your staff work directly with the agencies 
without a hearing--I mean, a lot of the work that we have done 
under the GAO High-Risk List in conjunction with GAO and staff 
up here, those aren't hearings, but the whole idea of having 
meetings with the staff and having to present their goals and 
their plans and then us reporting out on it on a quarterly 
basis and a 6-month basis really keeps agencies on focus, and 
that is not even a hearing. That is just saying what you are 
going to do in the next 6 months and then come back up here and 
present what you have done.
    Senator Carper. All right. Let us talk about solutions. We 
have talked a bit about information technology projects that 
are making it or not making it, how we can get better 
information, more timely information on a regular basis, and 
information we can understand. In terms of solutions and 
solving these problems and moving forward, a long time ago, 
when I was a Naval flight officer, when I was on active duty on 
the West Coast, our job was to hunt for Red October. We flew 
airplanes, 13-man aircraft that tracked Soviet nuclear 
submarines in all the oceans of the world and we had a number 
of squads. We flew an airplane called the P-3.
    We would all go through a training command to be prepared 
to be assigned to our respective squadrons and then we would 
get to our squadrons and we were assembled into 13-man crews 
and fly out our missions. But our respective squadrons didn't 
always do an especially good job and we had some squadrons that 
were better than others and we had some crews that were better 
than others.
    And the admiral that was in charge of our operations came 
up with an idea. I think he called it a Tactical Training Team, 
and established it in the basic training squadrons in which we 
all initially were trained. These Tactical Training Teams would 
be deployed to squadrons that weren't doing a particularly good 
job throughout the Pacific and Indian Oceans. They would work 
with us for a while and then they would go back home or 
deployed to assist other squadrons on an as-needed basis. There 
was always plenty of work for them to do because we weren't 
always uniform--just like these IT projects. You have pockets 
of excellence. We had pockets of excellence in the work that we 
did.
    My staff in working with others has come up with an idea. 
It reminds me of the Tactical Training Team idea. Our folks 
here call it an IT Strike Force. I would like to think that it 
is a constructive and realistic solution, or part of the 
solution to what we are trying to do here. I am not sure, but I 
hope that it is.
    But Ms. Evans, I believe you may have seen a pre-release 
draft of the bill that I am going to be introducing later today 
with Senators Lieberman and Collins that will allow your office 
to create, if you will, a Tactical Training Team, but really an 
IT Strike Force. It would be comprised of experts from both 
within government and maybe from outside of government, folks 
who can help agencies control some of our IT problems before 
they become even more unwieldy.
    And the hope here is that we won't see a repeat performance 
like we have experienced with our Census hand-held devices. 
This Strike Team notion would help agencies control small 
problems before they become billion-dollar problems, as they 
are today.
    First of all, is this a realistic proposal in your idea? Is 
this an idea that is half-baked? Is this an idea that has 
promise but needs to be further thought out? What do you like 
about it? What are your concerns about it? I would just start 
with asking you for your comments and then I will ask Mr. 
Powner and Mr. Denett, as well.
    Ms. Evans. Sir, the initial reaction to it is that it is a 
solutions-oriented approach. It is trying to get to the goal 
and identifying the problems quick enough in a way that we can 
prevent a situation like the Census from happening again. So 
the initial concept, I believe, is a good concept and is a 
solutions-oriented concept.
    We do something on a very similar basis, but it is not as 
formal as that process would be, now as we identify the 
problems, but they are not as quick. We are not getting ahead 
of the problem.
    One of the efforts that we have underway now, which I think 
this particular feature would complement, is what we are 
calling the Policy Utilization Assessment effort that we 
started this year. So Mr. Powner has talked a lot about the 
quality of the data, and it is only as good as the agency 
reports to us. So what we have started is an initiative with 
GSA that actually looks at our policy in totality and then does 
an assessment.
    So, for example, in the cyber security area, we have a 
policy, total policy going from acquisition all the way through 
how you accept it. If I looked at the numbers reported by the 
agencies, I would be at 50 percent. But what we have worked on 
and what we have developed is a methodology that gives us 
statistical certainty. I am really only at 30 percent.
    So based on that, using that and then in complement with 
what is envisioned, I believe, in this bill, I could use this 
to go into an agency and say, OK, you said that you are green. 
You have all these things in place. You haven't rebaselined, so 
we are going to do an assessment of the policy, not an audit 
like GAO, but an assessment, and it would tell me whether they 
were at 100 percent, 90 percent, or 60 percent. If they were 
below a certain threshold, then this team could go in, analyze 
what the issues are, and then help the agency to move forward 
to really realize that 100 percent implementation.
    Senator Carper. Mr. Powner, this notion, this IT Strike 
Team notion, is it half-baked? Is it more than half-baked? One 
of the things I think you mentioned in your testimony, we have 
these IT projects that we know are problematic. They appear on 
a High-Risk List or a Management Watch List, not just once, but 
again and again and again, for extended periods of time, for 
months and in some cases for years. And maybe that might be an 
appropriate place or instance in which to deploy these teams.
    Mr. Powner. Yes. I think the idea is a good one from this 
point. Agencies need help, and if you look internally to some 
agencies, they set up similar--they refer to them as Centers of 
Excellence where they can go to for help in establishing 
contracts. How do we establish a baseline estimate up front? 
How do we define our requirements? How do we manage risk? And 
so you put the appropriate processes in place. And if you look 
at those things, a lot of those are the root causes why we have 
projects on this list, why we rebaseline. We got the 
requirements creep. Requirements creep is out of control so we 
have rebaselined. That was the No. 1 reason we found.
    Senator Carper. Is that right? OK. I am not surprised.
    Mr. Powner. No, exactly. So if you put this group together, 
I can tell you right out of the gate what some of the areas you 
want to focus on: Getting good estimates up front, defining 
your requirements well, putting in place a sound risk 
management program, and overseeing contractors, and there are 
probably some other thoughts, but those are four areas where we 
have major weaknesses across the board.
    So I think it is a good idea because if you focus on 
solutions and areas where they can go for help--but what is 
important is that group from a central government point of 
view, there will be a lot of pressure on that group because 
there is a lot of help that is needed. What is important is to 
establish that group and then to have that somewhat replicated 
in agencies, too. And we already have some of that. I think 
DOD, FAA, IRS, some of the big organizations, you will see 
pockets of that occurring.
    So it is also important to do that centrally, but to make 
sure that it gets replicated and not to lose sight, too, that 
it is about the people, also, not just the processes but the 
people. We need to bolster our workforce. I think Ms. Evans 
comes out with an annual report showing that we need help in 
the PM area, with architects, engineers, those types of folks.
    One other item that I would like to highlight is if you 
look at IRS as an example, where they had some successes, they 
used what is called Critical Position Pay Authority, where you 
actually can pay above some of the Federal limits to attract 
some better folks. So in addition to these processes and strike 
forces, we need to continue to bolster our workforce through 
some of those existing processes.
    Senator Carper. All right. Well, I think I have probably 
kept you here long enough. I wish that more of my colleagues 
were here with us today. I know Dr. Coburn has a lot on his 
plate these days, but is keenly interested in these issues. He 
has not gone away. He is going to be around for a while. He has 
self-imposed a 12-year time limit if the voters of his State 
concur with that, and they might.
    I am going to be around for at least another 4 years and my 
guess is that we will continue to work together on this 
Subcommittee for a while, so our interest in this issue, in 
these issues, as well as a broader range of issues that involve 
concerns about just how wisely we are spending our resources, 
we are going to stay on these, but we are going to stay on this 
one, as well.
    I think we are making some progress, but God knows we need 
to make more. There is plenty of work for us to do here.
    Mr. Denett.
    Mr. Denett. I agree with what my colleagues have said. I 
would like to tack on, we do have an initiative called the 
SHINE initiative, because so often----
    Senator Carper. The SHINE? Is that an acronym or what is 
that?
    Mr. Denett. It is just meaning shine, give people a chance 
to shine and look good. Too often, they are focused on, when 
they get behind schedule and over cost, and that is 
appropriate, but we don't often enough praise those that are 
within budget and on schedule. So SHINE is an attempt to give 
recognition to programs and employees that are doing well. So 
we gather them in, give them some praise----
    Senator Carper. That is good.
    Mr. Denett. I think it would be helpful and would even like 
to approach you when we have our next round of those to see if 
you would be willing to participate in honoring some of those 
special projects.
    Senator Carper. I would be pleased to do that. I would be 
pleased to do that. What we used to say in the Navy, praise in 
public, reprimand in private. We would be pleased to join in 
the praise for some of these folks. But every now and then we 
like to, around here, we like to reprimand in public, too. It 
has a salutary effect. [Laughter.]
    All right. I suspect you are going to have some follow-up 
questions for the record. I would just ask that you respond to 
them promptly, as you always do.
    Thank you for your stewardship here, and Mr. Powner, a real 
special thanks to you and your folks at GAO who are a big help 
to us in this area. Thank you.
    All right. Mr. Grasso, Dr. Brown, Dr. Jarrett, we are happy 
to see you, Tom Jarrett. Welcome.
    Our second panel today begins with Al Grasso. Mr. Grasso is 
President and Chief Executive Officer of the MITRE Corporation, 
a not-for-profit organization chartered to work in the public 
interest. I think you all have been involved, as I understand 
it, in doing some work on the Census project and we are very 
grateful for the work that you are doing there. At MITRE, Mr. 
Grasso is responsible for developing and leading the 
corporation's overall strategic and business operations. MITRE 
has been involved in helping to oversee multiple complex and 
high-risk IT investments in the Federal Government. You have 
had plenty to do.
    Our next witness is Dr. Norm Brown, and I couldn't help but 
notice Dr. Brown sitting in the first row behind the witnesses 
on our earlier panel. Dr. Brown did a lot of head nodding one 
way or the other, so now we will let you actually lend a voice 
to all the body language that you were sharing with our staff 
and me earlier.
    But he is Executive Director for the Center for Program 
Transformation. For over 30 years, I am told, Dr. Brown has 
served as a commercial program manager and program management 
troubleshooter, and you have also served as the Assistant 
Secretary of--did you serve as the Assistant Secretary or in 
the office of the Assistant Secretary of the Navy--in the Navy, 
good for you--where you worked across all military branches to 
bring troubled projects back on track. In addition, I am told 
that you represented the military services on the 2000 Defense 
Science Board with the mission of solving why risky IT 
investments spiral out of control.
    And our last witness, I am tempted to say saving the best 
for last--he is a home boy--but Tom Jarrett, whom I have been 
privileged to know for a long time, Secretary of our Department 
of Technology and Information for the State of Delaware. Mr. 
Jarrett is no stranger in testifying before this Subcommittee. 
Previously, he testified as President--President, that is a job 
a lot of my colleagues would like to have--President of the 
National Association of State Chief Information Officers on 
issues relating to cyber security when Senator Coburn was 
Chairman of this Subcommittee. Mr. Jarrett oversees an IT 
investment budget of over $200 million and has achieved 90 
percent of cost, schedule, and performance goals for the past 7 
years as the Chief Information Officer of Delaware.
    When I was governor, we made progress on a whole lot of 
different fronts overall in our education and welfare system. I 
was very pleased with much that we accomplished, such as job 
creation and job preservation. One of the areas that we made 
some progress, but maybe not enough, is the area of our 
government that Mr. Jarrett now leads. He has taken our State 
clearly to the next level. I think one of the very proud things 
that our current governor can take credit for is the work that 
Mr. Jarrett and his folks have done in his department. So, 
welcome. In terms of best practices, we think you are one.
    Mr. Jarrett. Thank you.
    Senator Carper. Mr. Grasso, I am going to recognize you for 
your statement, and again, I would ask you to use about 5 
minutes. If you go a little over, that is all right. And then 
once all of our witnesses have testified, we will come back and 
ask some questions. Your entire statement will be made part of 
the record, so please proceed.

 TESTIMONY OF ALFRED GRASSO,\1\ PRESIDENT AND CHIEF EXECUTIVE 
                   OFFICER, MITRE CORPORATION

    Mr. Grasso. Thank you, Chairman Carper. Thank you for 
affording me the opportunity to appear before this 
Subcommittee. I fear that many of the remarks I had prepared 
have already been stated this morning, but I hope to 
reemphasize some of those key points.
---------------------------------------------------------------------------
    \1\ The prepared statement of Mr. Grasso appears in the Appendix on 
page 100.
---------------------------------------------------------------------------
    Senator Carper. Around here, we talk about an echo effect. 
It is really hard to cut through the media and to actually 
deliver a message to constituents, to voters. If you are the 
President, you have a big megaphone and you have a lot of echo 
effect from your cabinet secretaries and others who work for 
the Administration, so you have a good echo effect. But things 
have to be said over and over and over again in order for them 
to get through, including to people like us up here. So a 
little repetition is not bad.
    Mr. Grasso. Thank you. Our company's 50 years of 
experience, contributions, and accomplishments has given us a 
perspective that I believe is highly relevant to the topic of 
information technology planning and management. From the early 
days of the SAGE air defense system to present-day deployment 
of advanced command and control and business modernization 
systems, MITRE has been witness to great successes and, 
similarly, to great disappointments. We are honored to be asked 
to share our lessons and insights with your Subcommittee.
    Federal IT programs operate in an environment of rapid 
technology evolution, where some system components become 
obsolete before the program completes. This pace of technology 
change challenges program teams to keep their technical skill 
base current. IT systems and business processes are 
increasingly interconnected within and across agencies, making 
it hard to achieve consensus on vision, operational concepts, 
and requirements. The Federal Government's stretched fiscal and 
human resources further complicate the situation.
     The net effect is the widespread failure of many programs 
to deliver on time and on budget, with only a few notable 
exceptions where programs are able to overcome these challenges 
and succeed. Our experience leads me to comment on several 
critical areas and to offer three steps for improvement.
    My first comment pertains to governance. Governance relates 
to decisions that define expectations, grant power, assign 
accountability, or verify performance. Effective governance 
comprises consistent management cohesive policies, processes, 
and decision rights for a given area of responsibility. 
Governance becomes increasingly complicated as programs and 
processes cross organizational boundaries and intersect 
multiple governing bodies. Authorities and responsibilities 
become ambiguous and program managers are disenfranchised. It 
is often said that the debate begins in government once the 
decision is made.
    Successful programs must have unambiguous governance. 
Decision making authority----
    Senator Carper. Say that again. Successful programs must 
have what?
    Mr. Grasso. Unambiguous governance.
    Senator Carper. OK.
    Mr. Grasso. Decision making authority and accountability 
that address the implications of intersecting organizations 
must be clearly defined at the onset. These authorities must 
encompass the areas of budget and finance, investment portfolio 
management, business process, and program and project 
management.
    My second comment pertains to requirements, an equally 
important consideration. Requirements, reality, and flux are 
often recognized as the root cause of program rebaselining. 
Rebaselining is not necessarily a dirty word, but a necessary 
part of delivering capabilities that meet users' needs. 
Requirements are too often determined in the absence of cost, 
schedule, and technology risk consideration, and once 
determined, they are very difficult to change.
    The biggest difference between successful commercial IT 
developments and troubled government IT acquisitions is how 
requirements are managed. Successful commercial IT developers 
handle requirements with great caution. If a certain 
requirement adversely drives cost, performance, or schedule, it 
is quickly modified or eliminated. This does not happen in a 
typical government IT acquisition. Time to market is a 
competitive driver in the commercial marketplace, and I would 
submit it is as important, if not more so, in a world where 
adversary capabilities change as quickly as the technology 
cycle. System requirements must be considered living, but 
managed with a controlled process to use regular trade-off 
analyses to determine the value of change.
    My final comment addressees program management practices. 
Successful programs are characterized by a strong government 
Program Management Office (PMO), capable of a peer relationship 
with the contractor on systems engineering and program 
management issues. With a strong and capable PMO, the 
government has the capability to make informed decisions and 
manage the risk in acquisition programs.
    A key function of a strong PMO is best described by the 
metaphor of an architect's relationship with the user and the 
builder of a building. The architect is the user's agent as 
independent of the builder. The architect works to understand 
the user's operational needs and translate them into technical 
requirements enabling builders to develop the needed 
capability. The architect evaluates development feasibility and 
performs an independent conceptual design and cost estimate. 
These architect functions enable the user to make informed cost 
and capability tradeoffs and prioritize requirements. The 
architect is accountable to the user to ensure that delivered 
capability meets the user's highest priority needs within the 
constraints imposed by available technology, funding, and time.
    I offer the following recommendations based on our 
experience with these issues. First, change the tone and tenor 
of oversight to focus equally on programs that have gone from 
bad to good and good to great, to reveal best practices which 
then can be applied more broadly. No program is without risk. 
We should all be more interested in those programs that have 
managed the risks well and harvest those results for the 
betterment of the larger set of programs.
    Second, to navigate the dynamics of uncertainty of today's 
environment, IT programs are best structured as a portfolio 
with internal planning and management flexibility. Oversight 
should focus on the long-term funding envelope and the overall 
capabilities to be delivered. This allows flexibility at the 
program level to make informed trade-off decisions and to 
concentrate on manageably-sized increments that deliver 
capabilities in shorter time frames. This approach makes it 
easier for programs to demonstrate success or to fail early, 
which is valuable if a program is put in place and funded 
contingencies. It also puts capabilities in the hands of the 
users more quickly. This incremental approach is the norm in 
commercial practice.
    Third, Congress should continue to support and refine 
programs such as the DOD's Highly Qualified Experts program 
and, as Dave Powner mentioned earlier, the IRS's Critical Pay 
Authority that helps attract and retain critical government 
professionals. Additionally, the IRS's pay-for-performance 
program has helped motivate performance aligned to outcomes. 
These are valuable tools that address the capacity, 
capabilities, and incentives needed to manage effective 
programs. We encourage the Congress to look to these as models, 
streamline their execution, and broaden their application 
government-wide.
    I request that my prepared statement be included in the 
record and I would be pleased to answer questions.
    Senator Carper. Your entire statement will be included in 
the record. That was an excellent statement, an excellent 
summary. Thank you very much.
    Mr. Grasso. Thank you.
    Senator Carper. Dr. Brown, welcome.

 TESTIMONY OF NORM V. BROWN,\1\ EXECUTIVE DIRECTOR, CENTER FOR 
                     PROGRAM TRANSFORMATION

    Mr. Brown. Good morning, Chairman Carper. First, let me 
congratulate you for holding this hearing since literally 
billions of taxpayer dollars are wasted every year in poorly 
managed IT. Clearly, you are onto something important.
---------------------------------------------------------------------------
    \1\ The prepared statement of Mr. Brown appears in the Appendix on 
page 111.
---------------------------------------------------------------------------
    Vice Admiral Jerry O. Tuttle, Retired, the former Deputy 
Chief of Naval Operations for C4I and an icon for naval 
computing and net-centric warfare would counsel, ``Lead, 
follow, or get out of the way.'' Thank you, Senators. Thank you 
for leading the way. As we say, bravo zulu.
    Let us be clear about one thing. Although it is difficult 
to effectively manage a large-scale project, on the other hand, 
producing a large-scale IT train wreck is easy. The good news 
is that wrecks can be avoided by effectively using best 
practices.
    Today, I would like to briefly offer actions that 
government departments and agencies, OMB, and Congress can take 
to prevent wrecks. At its core, these actions address rapidly 
achievable improvements.
    I will begin with a structural observation. Much is 
expected of each agency CIO. Many have responsibility without 
real authority. Many Federal departments include numerous 
essentially independent fiefdoms because Congress has so 
arranged it, fiefdoms independently funded by Congress. The 
Pentagon rule is, he that has got the gold makes the rules. So, 
too, in Federal agencies. Although I don't today have any 
solutions to offer, I would be happy to work with your staff.
    Next, an observation regarding those IT problems in project 
management and oversight. When OMB testified before you last 
September, they expressed a recent interest in IT program 
execution, and that is a very good thing, but IT programs 
simply don't manage themselves.
    From my understanding and as we have heard this morning, 
very few agencies have much in the way of any real IT program 
management and oversight. Earned value is held up as a do-all 
silver bullet solution, yet little is done to prevent the easy 
gaming and corruption that earned value is vulnerable to, and 
associated rebaselining may lack the transparency needed to 
ensure effective oversight. Far too much is expected of earned 
value. Although earned value is a powerful visibility technique 
that supports program management, earned value cannot replace 
program management.
    Unfortunately, there seems little in the land of government 
IT program management that implements the needed essential 
techniques of managing risk, requirements, and change, or 
integrated baseline reviews. As a solution, I would recommend 
that each agency be required to actually have real program 
management and oversight, that they focus on implementing the 
important critical details with minimum overhead, and that they 
identify remaining weaknesses and vulnerabilities.
    Transparency is dandy, but it must be converted to 
visibility to be useful. A transparent contractor can deliver a 
53-foot truck full of boxes of data, but what you really need 
is only the bottom-line information. What is needed is true 
visibility of IT project health and progress in near real time.
    Since earned value seems to be the only principal 
visibility technique relied on by OMB and the agencies, I would 
propose a more comprehensive visibility product, let us say an 
Exhibit 350, to provide real project visibility indicators 
monthly, primarily for the program manager, with quarterly 
simplified versions for agency and component CIOs, OMB, and 
Congress. I will be happy to work with your staff and OMB on 
this.
    Tracking schedule progress is not easy. One reason for 
schedule surprises is that it is pressure to meet schedule 
increases. The hard-to-do things are kicked down the road, with 
difficult, uncompleted requirements now moved into the future, 
a future which was not planned to receive it. As it turns out, 
this unplanned future work will now require a successive series 
of miracles to be accomplished in order to complete the 
development on time and on budget. Don't bet on the miracles 
happening.
    To motivate agencies to focus on IT project management and 
oversight, I would recommend adapting the Nunn-McCurdy 
notification process, not that it is a great visibility 
technique; it is not. But it, in fact, serves as a powerful 
motivator, as something really to be avoided. As little else 
can do, it gives a clear focus to the business of cost and 
schedule containment.
    We have a serious problem regarding people. It is difficult 
to reward good talent, to hire good talent, and to train good 
talent. We expect CIOs and our IT personnel to do more with 
less and then give them less as if to prove the point. OMB 
needs to address this people issue as a priority.
    Training project personnel to effectively implement 
fundamental processes is minimal. While DOD has a certificate 
program in IT program management, GSA long ago disbanded their 
excellent similar IT Trail Boss program for civilian agencies. 
The various communities charged with making IT development work 
have received essentially no education, training, or any 
certification to do what they are asked to do. No training, no 
education, no certification, and no experience--it is a wonder 
we do as well as we do, as bad as it is.
    Apropos of your remarks earlier this morning about pulling 
the plug, a program termination process should be considered. 
The former Assistant Secretary of the Army for Acquisition, the 
Hon. Claude Bolton, is a strong proponent of terminating 
projects if measurable outcomes cannot be achieved within the 
agreed-to program cost schedule and performance baselines. The 
DOD has no formal decisionmaking process or policies to 
terminate programs, and I am not aware of any for other 
agencies. I think it wise and prudent to consider including 
such a process among ways to improve the government's IT 
acquisition process.
    Contract incentives and other considerations are important, 
and I discuss them further in my formal testimony.
    That concludes my summary and I will be happy to take 
questions at the appropriate time.
    Senator Carper. Again, another excellent testimony. Thank 
you for the thought that you put into it. Thank you for your 
years of service, too, and your counsel here today.
    Mr. Jarrett, you are recognized. Please proceed. Welcome.

  TESTIMONY OF THOMAS M. JARRETT,\1\ SECRETARY, DEPARTMENT OF 
         TECHNOLOGY AND INFORMATION, STATE OF DELAWARE

    Mr. Jarrett. Chairman Carper, Ranking Member Coburn, and 
Members of the Subcommittee, thank you for the opportunity to 
be here today. As Secretary of Delaware's Department of 
Technology and Information, I can well appreciate the 
complexity, the challenges, and the significant 
responsibilities associated with managing information 
technology projects in an investments portfolio that cuts 
across many agencies, and in Delaware's case, all three 
branches of government.
---------------------------------------------------------------------------
    \1\ The prepared statement of Mr. Jarrett appears in the Appendix 
on page 120.
---------------------------------------------------------------------------
    Albeit in a much smaller scale than the Federal Government, 
Delaware and other State Governments are faced with similar 
concerns regarding IT project management. In fact, Delaware's 
Department of Technology and Information was established in 
part because of ongoing IT project delays and cost overruns 
that Delaware was experiencing. Delaware's centralized IT 
structure charges our agency with direct oversight and approval 
on nearly $200 million in active IT projects.
    As a new agency, we were able to develop and employ new 
approaches to IT project management that incorporate many best 
practices from private industry and others in the government 
sector. While our methods are under constant review for 
improvement, we are enjoying some significant progresses. We 
have an excellent track record of delivering much-needed IT 
solutions that are on time and on budget.
    There are no silver bullets, no one or two changes that you 
can point to for perfect project management. Instead, there are 
many small improvements that we have made that, in the 
aggregate, are making the difference for Delaware. I would like 
to take a few minutes to give you a high-level view of a 
project life cycle using the Delaware model. In addition, I 
would like to point out what I believe are the significant 
processes and procedures that we have put in place to 
successfully manage our portfolio.
    Like the Federal model, we require agencies to submit a 
business case that addresses the major items we believe help 
ensure a project's success. Our model includes the following 
major areas: Risk management, processes reengineering, 
architectural review, resource and funding availability, 
project management oversight, organizational change management, 
needs assessment, customization requirements, disaster recovery 
levels, and management and executive sponsorship.
    Recommended projects are forwarded for the concurrence of 
the State CIO before they move ahead in our process. This 
recommendation usually includes ongoing funding contingent on 
meeting project milestones. Once approved, DTI works with the 
agency customer to develop full and complete requirements so a 
request for proposals can be released to secure vendor bids in 
order to meet the needs of the project. DTI stays involved to 
help the agency make the vendor selection and to structure a 
contract that ensures the project is delivered on time and on 
budget.
    I cannot emphasize enough how critical the requirements 
gathering process is to the project's ultimate success or 
failure. The Delaware model does not allow for requirements 
gathering to be conducted solely by the vendor. There must be 
an active involvement by the DTI project management team. There 
is an old saying, ``The customer doesn't know what they want 
until you give them what they ask for.'' Requirement gathering 
is critical to the process and helps alleviate scope, time, and 
budget creep if it is done correctly.
    At the start of a new project, a nationally certified 
program or project manager is assigned, as well as a certified 
organizational change management team, to run parallel courses 
in managing our projects. It is not enough to be certified. We 
take extra care in selecting the people to fill these 
positions, as they are essential to the success of the project. 
We count on these folks to do what we call inflicting 
discipline and structure to these projects.
    It is important to note that DTI manages Delaware's IT 
projects, not the vendors assigned to them. While we demand the 
vendor assign a certified and talented project manager, our 
State folks manage the project.
    Further, I would like to highlight our change management 
process. Change management starts with the review of existing 
business processes and is focused on preparing the organization 
for the cultural changes that are a part of any major project. 
According to a recent study conducted by the National 
Association of State Chief Information Officers, which 
represents State CIOs across the 50 States, 80 percent of major 
IT project failures can be directly attributed to a lack of 
change management. Too often, the employees who will actually 
use the new application are left out of the project process, 
and when this takes place, it is almost certain that the 
project is doomed to fail or under-perform.
    Another major element of our success is how we manage the 
overall process. All of Delaware's major IT projects have 
executive sponsors. Executive sponsors include high-level 
managers from the agencies involved as well as key executives 
from DTI and our Office of Management and Budget. The executive 
sponsors hold regular monitoring and update meetings and 
provide high-level oversight. When difficult project decisions 
need to be made, the executive sponsors are aware of the issues 
and have the authority to make critical decisions on whether or 
not to keep the project moving along.
    A real example is our present project involving all of 
Delaware's courts. Due to the lack of employee training and 
readiness, the executive sponsors recommended that the project 
be paused so that the training could take place. In fact, 
Delaware's Chief Justice himself invoked a 6-month pause in 
recognition of the need for court employees to be prepared if 
the project was to succeed.
    Besides ongoing project meetings, all projects in the 
State's portfolio are reviewed weekly by our Project Management 
Office and bimonthly at a workload management meeting where 
project managers present their projects' status to the 
assembled DTI senior managers. Opportunities for improvement or 
needed adjustments are vetted in an open forum with the goal of 
keeping the project on track and transparent.
    Our agency is committed to project transparency all the way 
to our legislature and our governor. We believe in continuous 
information exchange and dialogue with our elected 
representatives so that they are educated and aware of the 
complexity of major IT projects. Providing information up 
front, even when it may be painful, is far better than saving 
unpleasant surprises for yearly budget sessions. Bad news does 
not age well.
    I wish I could tell you that everything is humming along 
perfectly and that the processes that we have put in place are 
the final answer, but I can't, as we have many challenges 
similar to those being talked about today. However, we believe 
that by tightly managing these challenges, we can deliver 
projects on time and on budget. Although our project tracking 
system is complex, in our communication with stakeholders, we 
use a high-level red/green/yellow scorecard to inform them how 
the projects are developing.
    All projects have or will have ongoing issues that require 
senior management attention. Under the Delaware model, we 
believe that providing the facts, good or bad, to all 
stakeholders, including the legislature, is the proper policy, 
and most importantly, lives up to one of my agency's core 
values of integrity. Our vendors are held accountable to 
deliver what they said they would, as well, and we call the 
facts as we see them.
    I thank you for the opportunity to testify today and would 
be happy to answer any questions.
    Senator Carper. Thank you. Thanks not just for your 
testimony, but for the terrific leadership that you provide for 
our State and have shared some of the fruits of that leadership 
with us today.
    I think I would like to ask, first of all, Mr. Grasso and 
Dr. Brown a question and ask you to reflect on what we have 
heard from Mr. Jarrett today. I have oftentimes described the 
50 States as laboratories of democracy and the belief that 
somewhere in those 50 States, somebody has come up with a 
solution that will help us solve a number of the problems we 
face at the Federal level, and the same is true of the private 
sector. Somebody has figured out how to solve most of the 
problems, not all, but most of the problems that we face as a 
Nation and we just have to figure that out and be able to grow 
them to size or to scale.
    What did you hear from Mr. Jarrett in talking about 
Delaware, the way we operate in our State, that might be 
applicable to us here at the Federal level? What are some good 
lessons learned that you think we could take from his testimony 
and apply them to the Federal Government?
    Mr. Grasso. I think a strong element that I heard in the 
successes that we have seen in Delaware is the investment, 
again, that is made in the project management team, in the 
strength of that project management team. We heard that the 
project manager is certified, but it goes much further than 
just simple certification. There is a continuous development 
effort that is required.
    With a strong project management team, that project 
management team can be held accountable to basically continue 
to own the technical baseline of the program and not just 
basically contract it away such that you just get what you get. 
The team is informed throughout the process, is able to 
establish that peer relationship which I mentioned in my 
testimony with the contractor, and that allows for the 
successful acquisition.
    Requirements are important. We heard that. The level at 
which requirements are stated are very important. All too 
often, we detail requirements down to a level that we believe 
perhaps causes some of the rebaselining that we see because we 
specify things in terms of technology as opposed to in terms of 
outcomes. Although he didn't say it, the importance of 
requirements and the emphasis placed on requirements, I suspect 
they may have gotten that right in terms of the level by which 
they establish requirements.
    So I think a combination of the strength of the program 
management team, the investment made in those people to keep 
them highly qualified, and the ability to deal with 
requirements at the right level have achieved some of the 
successes.
    Senator Carper. Thanks for that analysis.
    Dr. Brown, what did you hear from Mr. Jarrett's testimony 
that you think might apply to us at the Federal level?
    Mr. Brown. First, let me echo what Al Grasso has said. I 
think the emphasis on requirements development and control and 
change control is absolutely fundamental and determines much of 
our cost and schedule problems that we see.
    And I think there is one other thing that Mr. Jarrett 
hasn't fully articulated to you that is very relevant, which is 
his process of paying their IT folks. Maybe you could talk more 
about that.
    Senator Carper. You actually pay them?
    Mr. Jarrett. Yes, imagine, they want to get paid. 
[Laughter.]
    As you know, Mr. Chairman, a number of years ago, the 
legislature allowed Delaware, and I still believe the only 
State today that has ever done it, to take the entire IT 
organization and convert it from a civil service structure to a 
non-civil service structure, and along with that, they gave me 
the ability to write our own compensation structure. So I think 
we are one of the few, if only, State agencies in the country 
that actually pays its IT people to the market, and what that 
means is that we have very qualified people, and I am happy to 
say we also have the highest retention rate of any agency in 
the State.
    Senator Carper. I used to be State Treasurer. I remember a 
time early on when we thought that we hired people, usually 
fairly young people out of school, and trained them so that 
they could be hired away by the private sector or some other 
employer. That sounds like that is not the case anymore, is it?
    Mr. Jarrett. That is not the case, not in my department.
    Senator Carper. OK, good. Dr. Brown, do you want to add 
anything?
    Mr. Brown. Sure. One thing that you had talked about that 
particularly resonated with me was the concept of the tactical 
training teams, and something that I had started at the Navy 
Department and later became DOD was the Software Program 
Managers Network. We had 10,000 members across the country. We 
supported over 200 programs in helping them. We had a stable of 
these people that we would call tactical trainers, experts in a 
wide range of subject matter experts, over 100 of these people, 
and as programs need, they could just call in the tactical 
team.
    If they needed help on a task activity network to support 
earned value or if they needed to better understand how to 
identify risks in the program or had a plan for testing or had 
a better oversight of what the contractor is doing or 
incentives, anything like that, they could call in the team. 
That is very consistent with your IT support team, and I think 
that is a fundamentally important thing to do.
    Senator Carper. All right. Mr. Grasso and Dr. Brown have 
been good enough to comment on your testimony. Let me just ask 
you if you would do the same with respect to some of what they 
said and just reinforce the relevance of their counsel to us.
    Mr. Jarrett. Well, I was telling Dr. Brown before we 
started, I was so pleased when I had a chance to read and hear 
what they both had said because I kind of threw up my arms and 
said, gee, we are doing that and have been doing it for some 
time and I think it proves out in a couple of areas.
    One is the requirements. People get sick about hearing 
about that, but in fact, what we have found and where we have 
spent all of our time is on the front end of the project 
process, not on the back end, which means that if you are going 
to get them right, then you have to determine what the 
requirements are. What we learned in the very beginning is that 
is not unlike--and you need actually qualified and trained 
people to be able to do that. That is not as simple as going 
out to an agency and saying, ``OK, tell me what it is that you 
need,'' because what we have found is that in a lot of cases, 
they can't articulate what they need, at least not in a way 
that when you are looking to replace a system and do that. So 
you have to help them through that process. So we have spent a 
lot of time doing that.
    The other is in the change management area. I think a lot 
of States and a lot of folks are moving into the areas of PMOs, 
Project Management Offices. Something that we have spent an 
awful lot of time on is, again, in the organizational change 
management, and I will give you an example. We are doing a new 
financial system in the State. We are spending a lot of our 
time not on the technical aspects but in dealing with each of 
the agencies so that we can help the users kind of work through 
the processes and the changes that this new project or this new 
system is going to provide to them. Most times, we don't spend 
time doing that.
    So I hear both of those areas in the testimony that was 
given, and from my perspective, I think it is well thought out 
and something that I think the Federal Government could do, as 
well.
    Senator Carper. Mr. Grasso, did you have a point you wanted 
to make?
    Mr. Grasso. I might just offer an additional remark.
    Senator Carper. Before you do that, in your testimony, Mr. 
Jarrett, you said there is an old saying the customer doesn't 
know what they want until you give them what they have asked 
for. There is a lot of wisdom in that.
    Mr. Grasso. In my written statement, there are a couple of 
references that I would just point to which emphasize the 
requirements point, and one of those references points to the 
32 programs at NASA that were evaluated and the up-front 
investment and the program overrun result. What you see is when 
the definition phase as a percent of the total program is 
greater than 10 percent, what you find is that the program 
overrun is typically less than 20 percent.
    Senator Carper. Say that one more time. I want to make sure 
I got it.
    Mr. Grasso. The graph refers to 32 NASA programs that have 
been evaluated and it takes a look at the definition phase of 
those programs as a percent of the total program. And when the 
definition phase of the program is 10 percent or greater, what 
you find is that the program overrun is 20 percent or less.
    Senator Carper. OK. Good.
    Mr. Grasso. And that suggests that considerate thought is 
being given to the requirements up front and thus less change 
is required later on.
    Senator Carper. All right. Thank you. My staff has been 
good enough to prepare a number of questions, and I am just 
going to take a moment and look at those and I suspect we will 
use these as a point of departure for some further discussion. 
In some ways, the kind of discussion that we are having here is 
actually more helpful than not.
    I have asked you to sort of reflect on your respective 
testimonies here today. Go back with me to the first panel and 
some of the things that you heard from the first panel, some of 
the things that we discussed. What were some of the things that 
struck you that were especially important or maybe something 
that you wanted to add something that wasn't said, or maybe an 
answer that wasn't given, or at least wasn't given as well as 
it might have been? Does anything stand out for you, referring 
back to the first panel's testimony?
    Mr. Grasso. I would offer one remark because I can't stop 
looking at the report card. Having several daughters in high 
school, I have encouraged my daughters to take Advanced 
Placement classes, whereas I know some of her friends are 
taking lesser challenging classes, if you will, and----
    Senator Carper. What grades are your daughters going to be 
in this fall?
    Mr. Grasso. Actually, my oldest is going to the University 
of Virginia this year, so she is just graduating, and my middle 
daughter is a rising 11th grader.
    Senator Carper. OK. We have two boys and our youngest son 
just graduated from the Charter School of Wilmington and he is 
going off to William and Mary and we know all about Advanced 
Placement tests.
    We just got our results about a week ago, high fives all 
around. [Laughter.]
    Mr. Grasso. So you know the amount of work that goes into 
preparing----
    Senator Carper. Yes, we do.
    Mr. Grasso [continuing]. And the level of effort that is 
required. The report cards are a great start, but at the same 
time, all IT programs aren't borne of the same level of 
complexity, the same level of interdependency and challenge. So 
as we step back and I look at the report cards, it begs the 
question of what are the correlations, if you will, to those 
grades.
    As I look at some of those organizations, some of those 
organizations are more insular than other organizations, if you 
will, and there are fewer interdependencies, and as a result, 
what you find is the complexity of the IT system is not 
necessarily as challenging as an enterprise system that touches 
a number of different organizations, crosses boundaries, and 
has unclear governance.
    So a comment that I will make is a report card as such, I 
don't know if those are AP classes or if those are substandard 
classes. So an F on AP class may not necessarily give you all 
of the details that you need to respond.
    Senator Carper. Well, you could not have picked a better 
analogy.
    Mr. Brown. My view is that those don't represent AP 
classes. [Laughter.]
    And with regard to what Ms. Evans and Dave Powner were 
saying about earned value management and particularly the 
problem of rebaselining, DOD deals with that in a very direct 
and forthright manner, which is just to track all the costs and 
the curve and what you see is a curve that starts going down 
and then the rebaseline goes up and goes up and it is just this 
downward spiral, a death spiral. And it is very easy to track 
what is happening when you see that picture, and that would be 
very easy for OMB to provide to you. I don't know if they are 
willing to do that, but they should have that information when 
they say--I don't know what agencies exactly report to them in 
their private conversations, but that should be certainly 
public information.
    Senator Carper. OK.
    Mr. Grasso. The 1987 Defense Science Board had an 
interesting conclusion and it said, technology is not our 
problem, management is, and they were talking about software 
problems, and hopefully----
    Senator Carper. Who said that?
    Mr. Brown. The 1987 Defense Science Board study on 
software.
    Senator Carper. OK. Mr. Grasso, the folks at MITRE have 
been good to help us in the Bureau of the Census. You tried to 
recover from a very bad situation. You may not be intimately 
familiar with this, but maybe you are, and to the extent that 
you have some familiarity with how the Census Bureau got into 
this mess as they prepared for and approached the 2010 Census, 
where do you think they went wrong and what lessons can we take 
from that experience to make sure that whether it is the 2020 
Census or other IT projects, how we don't let this kind of 
thing happen again?
    As it has turned out, it is an enormously expensive problem 
and we are going to not only end up spending more money, I 
don't know that we will get a better product. We will probably 
not get as good of a product in the end, but we will have spent 
a lot more money, a very unhappy situation. But we appreciate 
what you all have done to try to minimize the loss and get us 
going in the right direction. But what are some lessons 
learned?
    Mr. Grasso. I think the two lessons have already been 
described today. The first is requirements. In the case of 
Census, as you step back, there certainly has been a 
significant number of requirements changes throughout the 
process which has led to changes with the contractor. In the 
time period in which this all has occurred, the technology has 
evolved, so the solution that was envisioned some time ago 
perhaps is not necessarily the best today in terms of the 
architectural basis for that solution. So the program was not 
able to evolve as quickly as the technology was able to evolve.
    The second part of that is as all of this is evolving and 
all of these changes are being made, the strength of the 
program office, quite frankly, I would say, could have used 
some building up in the early stages. That has occurred as a 
result of the response to the issues that they have been 
confronted with most recently, but quite frankly, I would say 
that many of the responsibilities that should have been within 
the program office in establishing the technical baseline were 
really given to the contractor as opposed to being held within 
the program office.
    Senator Carper. This is sort of the inverse of the 
situation that I think Mr. Jarrett described.
    Mr. Grasso. Exactly. And we find that to be the case quite 
often. When you don't have a strong program office, a lot gets 
transferred to the contractor and you end up with a lot more 
dynamics in the program.
    Senator Carper. All right. Let me just ask, do you all have 
any closing thoughts that you would like to give us? I like to 
talk about take-aways, and it is impossible for me to remember 
everything that is said here. Even our staff, as smart as they 
are, it is impossible to remember everything. But just some of 
the key take-aways for us from what you have heard from other 
witnesses and maybe some points that you have brought up that 
you would like to reemphasize as we prepare to conclude.
    Is it Secretary Jarrett? Do they call you Secretary? Mr. 
Secretary.
    Mr. Jarrett. You may call me anything you wish.
    Senator Carper. No, but seriously, aren't you a cabinet 
secretary.
    Mr. Jarrett. I am.
    Senator Carper. Then I shouldn't call you Mr. Jarrett. I 
should call you Mr. Secretary. Do you want to take the first 
shot at that?
    Mr. Jarrett. I guess the only thoughts I had, and it kind 
of goes back to your earlier question about comments from OMB, 
and that is that reports and all the things that they put out 
are good and I guess they are a requirement here, but in fact, 
I have always believed that reports are only as good as how you 
use them to actually make a difference and to change things.
    What I guess I don't see in a lot of ways is how those 
reports are being utilized to actually begin to change the 
process. It just kind of has become embedded in the reports, 
and in fact, again, if you are not using them, then why even 
produce them. We spend less time on reports and more on the 
performance of the projects themselves.
    The second would be I see a lot of red. Well, we actually 
end up with red on some of our projects, too. The difference 
is, just because it is red doesn't mean it is ready to go and 
tank. It just means, at least in our case, that it is something 
that needs a lot of extra look-see and effort on it to get it 
back to yellow or to green, where it ultimately needs to be. 
Projects go in and out of red, green, and yellow all of the 
time. It depends on the complexity. So I think maybe red is 
nice, but I think there is this perception that red is bad. 
Yes, it is not good, but it is not necessarily the end of all 
things.
    The final thing is that in the issue about information back 
to Congress, I was telling some folks earlier, kind of the 
mindset which they take, and having come from the private 
sector and then Delaware Government for the last 7\1/2\ years, 
I am by no means an expert in the dynamics of Washington, but 
we took a very simple approach with our legislature which was 
we will notify them and show them this process on everything, 
painful as it may be on occasion. I have had some painful 
discussions with Senators and Representatives one-on-one and 
even in groups.
    Over time, what it has built, though, is the fact that they 
have come to understand the complexities with projects. They 
have become educated that it is not as easy as people think it 
is and that there are a lot of bumps in the road. So now what 
we find is that they have a far better understanding of that 
and we have a lot more productive dialogue back and forth as we 
work through those particular projects than we did when we 
started 7 years ago.
    So I would recommend to them to take the opposite approach, 
which is to provide that information almost as much as they can 
and not be concerned about the fact of whether it shames 
somebody or not because I think as you said earlier, I think 
sometimes that is not necessarily a bad thing.
    Senator Carper. All right. Thank you. Dr. Brown.
    Mr. Brown. I have three things that might be take-aways. 
One is to actually have Congress require visibility at agencies 
and at OMB.
    The second is to have Congress require the agencies to 
maintain and use effective critical practices.
    And the third is for Congress to maintain its involvement.
    Senator Carper. All right. Expand just a little bit on the 
second one.
    Mr. Brown. On maintaining critical practices, that is to 
have a real program management and oversight capability at each 
agency. What happens is there is so much focus on the Exhibit 
300s and the planning of programs that their execution and 
management go almost unnoticed, and that is where the troubles 
begin. It has been said that the plan doesn't survive the first 
battle, and so, too, with the plans for our programs. Things 
change and are very dynamic. Requirements change all the time 
and that is why, as Mr. Jarrett said, change management is very 
important.
    Agencies tend to ignore change management. They tend to 
ignore a very important thing called risk management to 
identify risks. They don't pay a lot of attention to what is 
going on in programs in terms of incentivizing the contractor 
to do the kinds of things that are fundamental, to examine what 
the fundamental capabilities that are needed by government are 
and not be absolutely controlled by requirements, to ignore the 
kinds of things that allow you to have a real confidence in 
your way of understanding what is going on.
    Instead of just requiring earned value as your sole 
indicator, you could look at real products being developed. You 
want to have products being developed all the time, weekly or 
monthly deliveries to the government, and to have assurance 
that those products are being properly integrated together. I 
have probably given you more than you want.
    Senator Carper. All right. Just enough. Thank you. Mr. 
Grasso.
    Mr. Grasso. I guess I would start by saying this is hard. I 
have seen good organizations fail at IT projects, and I would 
start by simply saying that we all have A teams and B teams. We 
need a lot more A teams, which means investment in people and 
the processes to do this job right.
    The teams need to feel ownership, if you will, in the 
programs and need to have the authorities to do the right 
things. I once used this analogy. As a homeowner, I may make 
some decisions to make some investments in my home. This year, 
I am going to put $3,000 into a sprinkler system in my home. 
Well, as I proceed to do that, I spring a leak in my roof. If 
this were the government, I wouldn't be able to fix that roof 
because I don't have the right kind of money to fix that roof 
and I would continue to fix my sprinkler system. I don't have 
the ownership of that portfolio, if you will, to treat my home 
as a whole complex suite of things. And the ability for a 
program manager to be able to manage a portfolio as opposed to 
an individual activity will allow him to manage some of those 
risks and make the necessary shifts as is appropriate.
    I mentioned earlier in my testimony about contingencies and 
funding contingencies. A risk management plan is absolutely 
critical to any program, but a risk management plan must have 
actionable alternatives. If the alternatives are not 
actionable, then we will just continue to mount on the risks.
    So I think the key points that I would make is we must 
continue to invest in the people and the organizations. As I 
mentioned earlier, the Highly Qualified Experts Program and the 
Critical Pay Program in IRS are good examples of investment. It 
sounds like Delaware has been able to do that. We must be able 
to manage these risks and empower our program managers with 
strong teams to make the right decisions and to have 
alternatives and choices as they see the technologies change 
and some of those risks surface.
    Senator Carper. Good. Thank you. Well, in closing, I really 
want to commend our staffs for being smart enough to invite you 
to come and testify. The way it works here, you have Democrats 
and Republicans who submit ideas and come up with ideas and 
ultimately we end up with a witness list. But this has been an 
especially helpful and beneficial panel of witnesses for us.
    One of the things that you are especially good at is 
explaining things in terms that even I can almost understand in 
most cases, and these are not easily understood concepts. That 
is a great gift.
    Thanks a lot for making time in your schedules to be here 
with us today, for preparing for this hearing, and for giving 
us not just a lot of food for thought, but, I think, a pretty 
good road map to follow to better ensure that we are getting 
our money's worth out of these IT projects that are going on 
throughout the government, actually around the world, and for 
all the taxpayers who are putting their money into this. Thank 
you for them, too. Keep up the good work that you are doing and 
much obliged.
    With that, you will probably hear from some of us with some 
additional questions. We will try to get those out in the next 
week or so, but if you could respond promptly, we would be most 
grateful.
    Thank you so much, and with that, this hearing is 
adjourned.
    [Whereupon, at 1:41 p.m., the Subcommittee was adjourned.]


                            A P P E N D I X

                              ----------                              

[GRAPHIC] [TIFF OMITTED] 44586.001

[GRAPHIC] [TIFF OMITTED] 44586.002

[GRAPHIC] [TIFF OMITTED] 44586.003

[GRAPHIC] [TIFF OMITTED] 44586.004

[GRAPHIC] [TIFF OMITTED] 44586.005

[GRAPHIC] [TIFF OMITTED] 44586.006

[GRAPHIC] [TIFF OMITTED] 44586.007

[GRAPHIC] [TIFF OMITTED] 44586.008

[GRAPHIC] [TIFF OMITTED] 44586.009

[GRAPHIC] [TIFF OMITTED] 44586.010

[GRAPHIC] [TIFF OMITTED] 44586.011

[GRAPHIC] [TIFF OMITTED] 44586.012

[GRAPHIC] [TIFF OMITTED] 44586.013

[GRAPHIC] [TIFF OMITTED] 44586.014

[GRAPHIC] [TIFF OMITTED] 44586.015

[GRAPHIC] [TIFF OMITTED] 44586.016

[GRAPHIC] [TIFF OMITTED] 44586.017

[GRAPHIC] [TIFF OMITTED] 44586.018

[GRAPHIC] [TIFF OMITTED] 44586.019

[GRAPHIC] [TIFF OMITTED] 44586.020

[GRAPHIC] [TIFF OMITTED] 44586.021

[GRAPHIC] [TIFF OMITTED] 44586.022

[GRAPHIC] [TIFF OMITTED] 44586.023

[GRAPHIC] [TIFF OMITTED] 44586.024

[GRAPHIC] [TIFF OMITTED] 44586.025

[GRAPHIC] [TIFF OMITTED] 44586.026

[GRAPHIC] [TIFF OMITTED] 44586.027

[GRAPHIC] [TIFF OMITTED] 44586.028

[GRAPHIC] [TIFF OMITTED] 44586.029

[GRAPHIC] [TIFF OMITTED] 44586.030

[GRAPHIC] [TIFF OMITTED] 44586.031

[GRAPHIC] [TIFF OMITTED] 44586.032

[GRAPHIC] [TIFF OMITTED] 44586.033

[GRAPHIC] [TIFF OMITTED] 44586.034

[GRAPHIC] [TIFF OMITTED] 44586.035

[GRAPHIC] [TIFF OMITTED] 44586.036

[GRAPHIC] [TIFF OMITTED] 44586.037

[GRAPHIC] [TIFF OMITTED] 44586.038

[GRAPHIC] [TIFF OMITTED] 44586.039

[GRAPHIC] [TIFF OMITTED] 44586.040

[GRAPHIC] [TIFF OMITTED] 44586.041

[GRAPHIC] [TIFF OMITTED] 44586.042

[GRAPHIC] [TIFF OMITTED] 44586.043

[GRAPHIC] [TIFF OMITTED] 44586.044

[GRAPHIC] [TIFF OMITTED] 44586.045

[GRAPHIC] [TIFF OMITTED] 44586.046

[GRAPHIC] [TIFF OMITTED] 44586.047

[GRAPHIC] [TIFF OMITTED] 44586.048

[GRAPHIC] [TIFF OMITTED] 44586.049

[GRAPHIC] [TIFF OMITTED] 44586.050

[GRAPHIC] [TIFF OMITTED] 44586.051

[GRAPHIC] [TIFF OMITTED] 44586.052

[GRAPHIC] [TIFF OMITTED] 44586.053

[GRAPHIC] [TIFF OMITTED] 44586.054

[GRAPHIC] [TIFF OMITTED] 44586.055

[GRAPHIC] [TIFF OMITTED] 44586.056

[GRAPHIC] [TIFF OMITTED] 44586.057

[GRAPHIC] [TIFF OMITTED] 44586.058

[GRAPHIC] [TIFF OMITTED] 44586.059

[GRAPHIC] [TIFF OMITTED] 44586.060

[GRAPHIC] [TIFF OMITTED] 44586.061

[GRAPHIC] [TIFF OMITTED] 44586.062

[GRAPHIC] [TIFF OMITTED] 44586.063

[GRAPHIC] [TIFF OMITTED] 44586.064

[GRAPHIC] [TIFF OMITTED] 44586.065

[GRAPHIC] [TIFF OMITTED] 44586.066

[GRAPHIC] [TIFF OMITTED] 44586.067

[GRAPHIC] [TIFF OMITTED] 44586.068

[GRAPHIC] [TIFF OMITTED] 44586.069

[GRAPHIC] [TIFF OMITTED] 44586.070

[GRAPHIC] [TIFF OMITTED] 44586.071

[GRAPHIC] [TIFF OMITTED] 44586.072

[GRAPHIC] [TIFF OMITTED] 44586.073

[GRAPHIC] [TIFF OMITTED] 44586.074

[GRAPHIC] [TIFF OMITTED] 44586.075

[GRAPHIC] [TIFF OMITTED] 44586.076

[GRAPHIC] [TIFF OMITTED] 44586.077

[GRAPHIC] [TIFF OMITTED] 44586.078

[GRAPHIC] [TIFF OMITTED] 44586.079

[GRAPHIC] [TIFF OMITTED] 44586.080

[GRAPHIC] [TIFF OMITTED] 44586.081

[GRAPHIC] [TIFF OMITTED] 44586.082

[GRAPHIC] [TIFF OMITTED] 44586.083

[GRAPHIC] [TIFF OMITTED] 44586.084

[GRAPHIC] [TIFF OMITTED] 44586.085

[GRAPHIC] [TIFF OMITTED] 44586.086

[GRAPHIC] [TIFF OMITTED] 44586.087

[GRAPHIC] [TIFF OMITTED] 44586.088

[GRAPHIC] [TIFF OMITTED] 44586.089

[GRAPHIC] [TIFF OMITTED] 44586.090

[GRAPHIC] [TIFF OMITTED] 44586.091

[GRAPHIC] [TIFF OMITTED] 44586.092

[GRAPHIC] [TIFF OMITTED] 44586.093

[GRAPHIC] [TIFF OMITTED] 44586.094

[GRAPHIC] [TIFF OMITTED] 44586.095

[GRAPHIC] [TIFF OMITTED] 44586.096

[GRAPHIC] [TIFF OMITTED] 44586.097

[GRAPHIC] [TIFF OMITTED] 44586.098

                                 
