    [Senate Hearing 117-675]
    [From the U.S. Government Publishing Office]

                                                           S. Hrg. 117-675

                     VA'S ELECTRONIC HEALTH RECORD MODERNIZA-
                      TION: AN UPDATE ON ROLLOUT, COST, AND
                      SCHEDULE

    =======================================================================

                                     HEARING

                                    BEFORE A

                              SUBCOMMITTEE OF THE

                COMMITTEE ON APPROPRIATIONS UNITED STATES SENATE

                        ONE HUNDRED SEVENTEENTH CONGRESS

                                 SECOND SESSION

                                   __________

                                SPECIAL HEARING

                       September 21, 2022--WASHINGTON, DC

                                   __________

             Printed for the use of the Committee on Appropriations

    [GRAPHIC NOT AVAILABLE IN TIFF FORMAT]

           Available via the World Wide Web: https://www.govinfo.gov

                                   __________

                        U.S. GOVERNMENT PUBLISHING OFFICE
    52-328 PDF                  WASHINGTON : 2023

    -----------------------------------------------------------------------------------

                          COMMITTEE ON APPROPRIATIONS

                      PATRICK J. LEAHY, Vermont, Chairman
    PATTY MURRAY, Washington             RICHARD C. SHELBY, Alabama, Vice
    DIANNE FEINSTEIN, California             Chairman
    RICHARD J. DURBIN, Illinois          MITCH McCONNELL, Kentucky
    JACK REED, Rhode Island              SUSAN M. COLLINS, Maine
    JON TESTER, Montana                  LISA MURKOWSKI, Alaska
    JEANNE SHAHEEN, New Hampshire        LINDSEY GRAHAM, South Carolina
    JEFF MERKLEY, Oregon                 ROY BLUNT, Missouri
    CHRISTOPHER A. COONS, Delaware       JERRY MORAN, Kansas
    BRIAN SCHATZ, Hawaii                 JOHN HOEVEN, North Dakota
    TAMMY BALDWIN, Wisconsin             JOHN BOOZMAN, Arkansas
    CHRISTOPHER MURPHY, Connecticut      SHELLEY MOORE CAPITO, West
    JOE MANCHIN, West Virginia               Virginia
    CHRIS VAN HOLLEN, Maryland           JOHN KENNEDY, Louisiana
    MARTIN HEINRICH, New Mexico          CINDY HYDE-SMITH, Mississippi
                                         MIKE BRAUN, Indiana
                                         BILL HAGERTY, Tennessee
                                         MARCO RUBIO, Florida

                       Charles E. Kieffer, Staff Director
                      Bill Duhnke, Minority Staff Director
                                     ------

     Subcommittee on Military Construction, Veterans Affairs, and Related
                                    Agencies

                     MARTIN HEINRICH, New Mexico, Chairman
    BRIAN SCHATZ, Hawaii                 JOHN BOOZMAN, Arkansas, Ranking
    JON TESTER, Montana                  MITCH McCONNELL, Kentucky
    PATTY MURRAY, Washington             LISA MURKOWSKI, Alaska
    JACK REED, Rhode Island              JOHN HOEVEN, North Dakota
    TAMMY BALDWIN, Wisconsin             SUSAN M. COLLINS, Maine
    CHRISTOPHER A. COONS, Delaware       SHELLEY MOORE CAPITO, West
    JOE MANCHIN, West Virginia               Virginia
                                         MARCO RUBIO, Florida
                                         BILL HAGERTY, Tennessee

                               Professional Staff

                               Michelle Dominguez
                                  Joanne Hoff
                                 Jason McMahon

                          Patrick Magnuson (Minority)
                           Jennifer Bastin (Minority)
                            Lucy Gardner (Minority)

                             Administrative Support

                                   Drew Platt

                                C O N T E N T S

                                  ----------
                                                                       Page

                                    PANEL I

    Statement of Hon. Donald M. Remy, Deputy Secretary, Department of
      Veterans Affairs...............................................      1

    Opening Statement of Senator Martin Heinrich.....................      1

    Statement of Senator John Boozman................................      3

    Summary Statement of Hon. Donald M. Remy.........................      4

        Prepared Statement of Donald M. Remy.........................      5

            EHRM: The Plan Going Forward.............................      6

            Deployment Schedule......................................      6

            Capability Enhancements..................................      7

            System Reliability.......................................      8

            New Functional Champion..................................      8

            Budget Overview and Cost Estimate........................      9

            Conclusion...............................................     10

                                  ----------

                                    PANEL II

    Statement of Mr. Mike Sicilia, Executive Vice President,
      Industries, Oracle.............................................     19

        Prepared Statement of Mr. Mike Sicilia.......................     21

            Introduction.............................................     21

            Conclusions..............................................     26

    Statement of Dr. Brian Rieksts, Ph.D., Research Staff Member,
      Cost Analysis and Research Division, Institute for Defense
      Analyses.......................................................     41

        Prepared Statement of Dr. Brian Rieksts......................     42

            Results..................................................     42

            Cost Estimate by Phase...................................     42

            Comparison...............................................     42

            Comparison of VA and IDA Estimates.......................     43

            Path Forward.............................................     43

    Statement of Mr. David Case, Deputy Inspector General, Office of
      Inspector General, Department of Veterans Affairs..............     43

        Prepared Statement of Mr. David Case.........................     44

            Infrastructure Deficiencies and Unreliable Cost Estimates     45

            Deficiencies in Infrastructure Readiness for Deploying
              VA's New Electronic Health Record System (April 2020)..     46

            Deficiencies in Reporting Reliable Physical
              Infrastructure Cost Estimates for the EHRM Program (May
              2021)..................................................     46

            VHA Cost Estimates for Physical Infrastructure Upgrades
              Needed in Support of the EHRM Program Were Not Reliable
                                                                          46
            OEHRM Did Not Include Cost Estimates for Upgrading
              Physical Infrastructure in Reports to Congress.........     47

            Unreliable IT Infrastructure Cost Estimates for the EHRM
              Program (July 2021)....................................     48

            Lack of Readiness Exhibited by No Integrated Master
              Schedule and Ineffective Training......................     49

            The EHRM Program Did Not Fully Meet the Standards for a
              High-Quality, ReliableSchedule (April 2022 Report).....     49

            Training Deficiencies for VA's New EHR System at the
              Mann-Grandstaff VAMC (July 2021 Report)................     50

            New Patient Scheduling System Needs Improvement as VA
              Expands Its Implementation (November 2021 Report)......     51

            Implementation Deficiencies and the Lack of Remediation..     51

            Review of Access to Care and Capabilities during VA's
              Transition to a New Electronic Health Record at the
              Mann-Grandstaff VA Medical Center (April 2020).........     52

            Medication Management Deficiencies after the New EHR Go-
              Live at the Mann- Grandstaff VAMC (March 2022 Report)..     53

            Care Coordination Deficiencies after the New EHR Go-Live
              at the Mann-Grandstaff VAMC (March 2022 Report)........     55

            Ticket Process Concerns and Underlying Factors
              Contributing to Medication Management and Care
              Coordination Deficiencies (March 2022 Report)..........     56

            The New EHR's Unknown Queue Caused Multiple Patient Harm
              Events (July 2022 Report)..............................     58

            Conclusion...............................................     59
              .
            Appendix A. VA Responses to Recommendations: Deficiencies
              in Infra- Structure Readiness for Deploying VA'S New
              EHR System, April 27, 2020.............................     59

            Appendix B. VA Responses to Recommendations: Deficiencies
              in Reporting Reliable Physical Infrastructure Cost
              Estimates for the EHRM Program, May 25, 2021...........     60

            Appendix C. VA Responses to Recommendations: Unreliable
              Information Technology Infrastructure Cost Estimates
              for the EHRM Program, July 7, 2021.....................     60

            Appendix D. VA Responses to Recommendations: The EHRM
              Program Did Not Fully Meet the Standards for a High
              Quality, Reliable Schedule, April 25, 2022.............     61

            Appendix E. VA Responses to Recommendations: Senior Staff
              Gave Inaccurate Information to OIG Reviewers of EHR
              Training, July 14, 2022................................     61

            Appendix F. VA Responses to Recommendations: Training
              Deficiencies with VA'S New EHR System at the Mann-
              Grandstaff VAMC in Spokane, Washington, July 8, 2021...     61

            Appendix G. VA Responses to Recommendations: New Patient
              Scheduling System Needs Improvement as VA Expands its
              Implementation, November 10, 2021......................     62

            Appendix H. VA Responses to Recommendations: Review of
              Access to Care and Capabilities During VA's Transition
              to a New EHR System at the Mann-Grandstaff VAMC, April
              27, 2020...............................................     62

            Appendix I. VA Responses to Recommendations: Medication
              Management Deficiencies After the New EHR Go-Live at
              the Mann-Grandstaff VAMC, March 17, 2022...............     63

            Appendix J. VA Responses to Recommendations: Care
              Coordination Deficiencies After The New EHR Go-Live at
              the Mann-Grandstaff VAMC, March 17, 2022...............     63

            Appendix K. VA Responses to Recommendations: Ticket
              Process Concerns and Underlying Factors Contributing to
              Deficiencies After the New EHR Go-Live at the Mann-
              Grandstaff VAMC, March 17, 2022........................     63

            Appendix L. VA Responses to Recommendations: The New
              EHR'S Unknown Queue Caused Multiple Events of Patient
              Harm, July 14, 2022....................................     63

            Appendix M. VA Responses to Recommendations: Deficits
              With Metrics Following Implementation of the New EHR at
              the Mann-Grandstaff VAMC, June 1, 2022.................     64

            Appendix N. VA, DOD, and FEHRM Responses to
              Recommendations: Joint Audit of the DOD and the VA
              Efforts to Achieve EHR System Interoperability, May 5,
              2022...................................................     64

    Additional Committee Questions...................................     73

    Questions Submitted to Mr. Mike Sicilia..........................     73

        Questions Submitted by:
            Senator Martin Heinrich..................................     73

            Senator John Hoeven......................................     75

    Questions Submitted to Dr. Brian Rieksts.........................     75

        Questions Submitted by Senator Martin Heinrich...............     75

    Questions Submitted to Mr. David Case............................     76

        Questions Submitted by:
            Senator Martin Heinrich..................................     76

            Senator John Hoeven......................................     77

      VA'S ELECTRONIC HEALTH RECORD MODERNIZATION: AN UPDATE ON ROLLOUT,
                               COST, AND SCHEDULE

                                  ----------

                         WEDNESDAY, SEPTEMBER 21, 2022

                                           U.S. Senate,
               Subcommittee of the Committee on Appropriations,
                                                        Washington, DC.
        The subcommittee met at 10:01 a.m. in room SD-124, Dirksen
    Senate Office Building, Hon. Martin Heinrich (chairmain)
    presiding.
        Present: Senators Heinrich, Tester, Boozman, and Hagerty.

                                    PANEL I

                         DEPARTMENT OF VETERANS AFFAIRS

    STATEMENT OF HON. DONALD M. REMY, DEPUTY SECRETARY,
                DEPARTMENT OF VETERANS AFFAIRS

                  opening statement of senator martin heinrich

        Senator Heinrich. Good morning. This hearing of the
    Military Construction, Veterans Affairs, and Related Agencies
    Appropriations Subcommittee is now called to order.
        Today, we will discuss VA's efforts around Electronic
    Health Record Modernization, or EHRM. This is a large but
    important initiative that was promised to result in a single
    health record for an individual, from service entry through a
    veteran's life, with seamless health data sharing.
        Unfortunately, implementation efforts to date have been
    plagued by delays, provider complaints, patient safety
    concerns, and questions about the accuracy of reporting to
    Congress, particularly related to cost.
        Our goal today is to address these issues over two panels.
    With that, I would like to welcome Donald Remy, Deputy
    Secretary of Veterans Affairs, and the accountable official for
    VA's Electronic Health Record Modernization Initiative. He is
    accompanied by Dr. Terry Adirim, the Program Executive Director
    of the EHRM Integration Office; Dr. Elnahal, VA's Under
    Secretary for Health; and Mr. Jon Rychalski--all right, three
    for four is not too bad--Assistant Secretary for Management and
    Chief Financial Officer.
        I want to start just by reiterating the committee's support
    for VA's EHRM objective. This is an extremely important effort
    to solve a decade-long problem, decades-long problem, and is
    essential that VA get it right for the health and safety of
    veterans. In addition, this is a major investment, and VA has a
    responsibility to taxpayers to ensure the system works, and its
    success can be measured.
        In 2018 when VA announced it was signing a sole-source
    contract with Cerner Corporation, now known as Oracle Cerner,
    VA estimated the EHRM Initiative would cost $16 billion over 10
    years. Congress was told that this amount incorporated the full
    cost of deploying and operating the new health record across
    VA's enterprise, including $10 billion for Cerner Corporation
    to provide the record itself.
        Since that initial estimate we learned that VA did not
    include all of the costs required for a successful
    implementation, and did not report those omissions to Congress
    in a timely way. This lack of transparency was disappointing,
    and the committee is glad that VA has made the effort to be
    more forthright under new leadership.
        Based on a recent independent cost estimate completed by
    the Institute for Defense Analyses, at VA's request, the cost
    to the Department could be significantly more than the initial
    estimate. The nearly $50 billion estimate assumes a longer
    deployment timeline, and 15 years of sustainment costs, as well
    as additional related and necessary costs that VA did not
    initially contemplate.
        The components and elements raised by IDA are reasonable
    and appropriate to consider, which is why it is standard
    practice to have a third party complete a life-cycle cost
    estimate before contracts are awarded.
        It is irresponsible that this step was skipped by VA when
    rushing through a sole-source contract in 2018, though I will
    acknowledge that none of today's witnesses, who are responsible
    for EHRM today, were part of that decision. We will discuss
    that estimate more in the second panel, and would appreciate
    VA's view of it as well.
        To date Congress has appropriated $8.2 billion since fiscal
    year 2018 to VA's dedicated account for this effort, though
    more has been spent. Of this money VA has allotted about $4.1
    billion, 40 percent of the contract ceiling, toward the Oracle
    Cerner contract, and the system has been deployed to five sites
    with 166 to go. I understand that there was a lot of upfront
    costs to the system, but would like a clear understanding of
    what that funding has bought us.
        Closures related to COVID had a huge impact on rollout, but
    that is not the only challenge this effort is facing. Of
    significant concern is how the new system has been received by
    providers. This initiative will not succeed without provider
    buy-in. It is not surprising that productivity decreased
    following rollout, but it is not clear whether it has improved
    over time.
        I understand that providers from the first rollout site,
    which deployed nearly 2 years ago, are still raising concerns.
    More alarming, are concerns that the new system is a risk to
    patient safety, and reports of repeated system degradations and
    outages. We know VA and Oracle Cerner have taken steps to
    address these issues, including training processes. And I look
    forward to hearing about how that collaboration is moving
    forward.
        Finally, I would like to discuss VA's deployment schedule.
    Last year VA took a strategic pause to assess the program and
    lessons learned. The last deployments were in June, and VA has
    postponed all planned rollouts until 2023 while still intending
    to meet the goal to be fully deployed by 2028. I am glad VA is
    not rushing deployments until there is more confidence in the
    likelihood of success, but the Department needs to be
    straightforward with Congress about what is reasonable and
    achievable. This effort is too important, and needs to succeed.
        And with that, I would recognize Ranking Member Boozman for
    his opening comments as well.

                       statement of senator john boozman

        Senator Boozman. Thank you Mr. Chairman. And Deputy
    Secretary Remy, thank you for you and your team being here
    today to discuss the VA's ongoing efforts to develop and deploy
    its new electronic medical record.
        Over the last 5 years this subcommittee has appropriated
    nearly $8.5 billion towards this endeavor, and we are
    deliberating on the fiscal year 2023 request of another $1.75
    billion. At the outset we were told that this program would
    cost no more than $16 billion, and would be complete in 10
    years. In the years since VA has deployed the new system at
    only a small handful of sites; and those rollouts have been
    challenging, to say the least.
        This is precisely why we are holding this hearing today. We
    want to get a sense of where the EHRM program is today, what
    the path forward is, and what the true cost is.
        And thank you, Mr. Chairman, very much for facilitating
    this. And again, this is just a very, as so many things are
    that have to do with veterans, this is certainly a very
    bipartisan effort, just to see if we can get things moving in
    the right direction.
        Certainly, the pandemic slowed the deployment and training,
    and we must not forget that, but the repeated system outages,
    persistent patient safety concerns, and lack of productivity at
    the deployment sites indicate problems much deeper than the
    challenge COVID posed. VA took an intentional pause to
    reevaluate the program, and has brought in new leadership, and
    now Cerner has been acquired by Oracle, meaning there is a
    completely new leadership team in place.
        I look forward to hearing how the new team, on all sides,
    will work together to get this right, and get deployments back
    on track. Our second panel will have representatives from
    Oracle, the IG, and from the Institute for Defense Analyses who
    recently finished work on the first Independent Life Cycle Cost
    Estimate for the EHRM program. This estimate is very different
    than the numbers VA has promised, namely, that is based on the
    assumption that VA cannot deploy the system in the 10-year
    window it has promised. Beyond that, the estimate also shows
    the potential for costs significantly higher than the $16
    billion VA number. I look forward to hearing this new
    perspective on the program, its timeline, and the associated
    costs.
        It is always appreciated hearing from the Office of the
    Inspector General, and we value the work that you do; so Mr.
    Case, thank you for being here. Mr. Rieksts, welcome, and we
    look forward to learning more about your analysis. I also want
    to welcome Mr. Sicilia from Oracle. Oracle recently completed
    its acquisition of Cerner, meaning you are now on the hook to
    see the rest of this deployment through successfully.
        I look forward to hearing your plan to do so, and how you
    can leverage your work with other government entities to help
    accelerate the VA deployment.
        And again, thank you very much, Mr. Chairman. And I yield
    back.
        Senator Heinrich. Thank you Senator.
        Deputy Secretary Remy, you are recognized for your opening
    statement. And your full written testimony will be included in
    the record as well.

                    summary statement of hon. donald m. remy

        Mr. Remy. Good morning, Senator. Good morning, Chairman
    Heinrich, Ranking Member Boozman, Senator Tester.
        Chairman Heinrich, you have already introduced the expert
    team I have here with me today, so I won't go through those
    introductions again.
        I simply want to say thank you for your consideration of
    the President's fiscal year 2023 Budget Request, and this
    opportunity to update you on VA's initiative to modernize our
    electronic health record, both so critical to veterans, their
    families, caregivers, and survivors.
        The resources this committee has repeatedly secured for VA
    and the EHRM effort reflect our close partnership, and your
    continued focus on our shared commitment to improving Veteran's
    access, outcomes, and experiences.
        In delivering world-class health care, VA adheres to the
    principles of high reliability organizations. Our fundamental
    goal, as an HRO, is achieving zero harm. Just one vet harmed is
    one vet too many.
        And right now the bottom line is that the Cerner system is
    not delivering for veterans in the ways that it should, not
    even close. It needs major improvements, whether a system
    outage lasts for one minute, one hour, or one day, that outage
    is unacceptable. So on behalf of veterans we serve, and
    providers serving them, we couldn't be more frustrated, that is
    why we are holding Cerner and ourselves accountable. That is
    why we are applying lessons learned from every deployment to
    continue to improve. That is why we have paused on all future
    go-lives until 2023 to get this right. And make no mistake we
    will get this right. We have to, for veterans, for providers.
        But these challenges cannot and will not stop us from
    modernizing the record system, because it is what veterans
    need. Our nearly 40-year-old Legacy system has served us well,
    but it has reached the end of its life cycle. Now now we need
    an EHR system where vets can access their health care records
    in one place, from the first day they put on their uniforms to
    the last day of their lives.
        A system that empowers vets to receive care anywhere,
    whether it is from DOD, VA, or Community providers, without
    having to worry about cumbersome paperwork, or potentially
    harmful gaps in their records. A system that helps providers
    understand injuries that veterans suffered 50 years ago, so
    that they can provide those vets with the best possible care,
    today.
        In other words, getting this new system right is essential
    to delivering a lifetime of world-class health care and
    benefits to our veteran population. So we are continuing to
    drive toward these goals by making sure that future EHR
    deployments reflect what we learn, with each challenge
    informing and better positioning us for the next deployment.
        During the remainder of this calendar year, and beyond, we
    are working with the contractor to ensure stability and
    resiliency of the system. We are using this interim period to
    make improvements like testing at our most complex sites, and
    adding capability enhancements to improve usability for our
    health care personnel. We are also fully engaged with past
    deployment sites, closely assessing their experiences,
    providing support, and applying lessons learned for future
    deployments, while also helping our hard-working, frontline
    users fix issues as they are identified.
        We have added a new functional champion, Dr. David Massaro
    to help lead the EHRM effort, along with Dr. Adirim; and if any
    future sites fail to meet our integrated readiness criteria for
    any reason we will not go live, we simply will not subject our
    veterans to avoidable risk.
        In short, we are continuously improving, continuously
    reviewing past deployments, and continuously holding ourselves,
    and Oracle, accountable to get this right before future
    deployments, not after. Whenever we make a decision at VA we
    ask ourselves one simple question: What does it mean for the
    veterans we serve, for their families, their caregivers, and
    survivors?
        In the case of EHRM the answer is clear, veterans need a
    system supporting our high reliability organization, delivering
    them world-class health care for decades to come, the new
    system hasn't yet achieved those goals, and we won't rest until
    it does. Until it serves veterans as well as they have served
    all of us.
        We look forward to your questions.
        [The statement follows:]
                      Prepared Statement of Donald M. Remy
        Good morning, Chairman Heinrich, Ranking Member Boozman and
    distinguished Members of the subcommittee. Thank you for the
    opportunity to testify today in support of VA's initiative to modernize
    its electronic health record (EHR) system. I am accompanied today by
    VA's experts on this initiative, Dr. Shereef Elnahal, Under Secretary
    for Health; Mr. Jon Rychalski, Assistant Secretary for Management and
    Chief Financial Officer; and Dr. Terry Adirim, Program Executive
    Director, Electronic Health Record Modernization Integration Office
    (EHRM-IO).
        I want to begin by thanking Congress and this subcommittee for your
    continued support and shared commitment to Veterans. The resources you
    have invested in VA's EHRM effort will improve access, outcomes and
    experiences for Veterans. Successful deployment of a modern EHR is
    essential in the delivery of lifetime, world-class health care and
    benefits for Veterans, as well as to set the standard for U.S. health
    care writ large. We will get this right. With a unified, seamless,
    trusted information flow between VA and the Department of Defense
    (DoD), we can further empower Veterans and their families, caregivers
    and survivors to achieve and sustain health and wellness. In addition,
    we can enable care teams to deliver best-in-class access and outcomes
    while enhancing VA's ability to innovate and advance Veteran care and
    services.
        I look forward to further engagement with you and your staffs to
    ensure that we are successful-and I assure you that we remain committed
    to full transparency regarding our deployment efforts. Veterans and
    patient safety are at the center of everything we do. In delivering
    world-class health care to Veterans, VA adheres to the principles of a
    High Reliability Organization and our fundamental goal is to achieve
    zero patient harm. To those important ends, I wanted to provide a
    program update, including what continues to be a sometimes challenging,
    but much more informed deployment and operational plan moving forward.
        Our charge has been clear: create a single, seamless, integrated
    health record for Veterans, starting with their military service days.
    This complete record within a single system allows those who care for
    the Nation's Veterans to proactively prepare for the future and deliver
    the benefits, care and services those Veterans have earned.
        This is one of the most complex clinical and business
    transformation endeavors in the Department's history. But the
    complexity and challenges associated with this effort should not deter
    us from modernizing our technology and processes. This is an
    opportunity for VA to fundamentally transform health care for Veterans
    through standardization of its operations to deliver consistent, high-
    quality care whenever, wherever Veterans seek it.
        Our nearly 40-year-old legacy system has served us well, but it has
    reached the end of its life-cycle-and given its limitations, it needs
    replacing. As Secretary McDonough said, this is a leap forward we can
    and must get right.
        We acknowledge that the first deployment at Mann-Grandstaff VA
    Medical Center (VAMC) in Spokane, Washington was problematic. The
    mistakes identified in the months following the Spokane deployment are
    unacceptable. We are holding ourselves and our vendor accountable to
    get these issues resolved at Mann-Grandstaff and our other deployment
    sites.
        Since the deployment in Spokane nearly 2 years ago, VA has applied
    the lessons learned from that experience to improve future deployments.
    We conducted a Department-wide strategic review that identified patient
    safety and other areas for improvement and used these lessons to change
    our deployment strategies with a focus on reducing risk and improving
    adoption. VA is unequivocally committed to providing safe, effective
    care to Veterans.
        This EHR modernization effort is led by the EHRM Integration Office
    with Dr. Terry Adirim, as the Program Executive Director, responsible
    for integrating efforts across the enterprise-wide, to include the
    Veterans Health Administration (VHA), the Office of Management, and
    other offices. We are excited to have on board as the first Senate-
    confirmed Under Secretary for Health since 2017--Dr. Shereef Elnahal,
    whose leadership of VHA will be critical to the success of this effort.
    In addition, Mr. Jon Rychalski has led VA's Office of Management for
    over 4 years and can address the updated independent cost estimate
    (ICE).
                          ehrm: the plan going forward
        Any implementation of this scale and complexity comes with inherent
    challenges. While we are working diligently to address them, we also
    know change like this can be challenging and, as such, have always
    viewed this process iteratively. We are currently in the initial
    operating capability (IOC) phase. In this phase, we are learning what
    is working, what is not working-and applying the lessons learned moving
    forward.
                              deployment schedule
        Following the 2020 Mann-Grandstaff VAMC deployment and strategic
    review in 2021, VA revised its EHR deployment schedule through the
    first quarter (Q) of fiscal year 2024. We understand, and VA has always
    made clear, that the deployment schedule is subject to change based on
    unforeseen events that may prevent a safe and successful deployment.
    This may include a determination that a site may not be ready for
    deployment due to implementation tasks not being completed on time or
    an assessment by EHRM-IO and Veterans Health Administration (VHA)
    leaders that a timeline adjustment is needed for a specific clinical
    site.
        In preparation for deployments, EHRM-IO employs a detailed
    integrated readiness criteria checklist to assess risk at future sites.
    Additionally, we now use a continuous feedback loop with deployed sites
    to capture improvement opportunities and to drive future changes at
    sites not yet deployed. Pre-deployment activities are underway in
    Veterans Integrated Service Networks (VISN) 10 and 20, as well as
    preparation activities for site deployments in VISNs 12 and 23
    scheduled later in fiscal Year 2023 and in early fiscal Year 2024.
        Demonstrating the value of the new readiness process put in place,
    VA decided to postpone its planned go-live at Boise VAMC, originally
    scheduled for July 23, 2022. This decision was based on concerns that
    the site and Cerner had not completed all the tasks on the site
    deployment readiness checklist. A new launch date for Boise VAMC has
    not been determined. We also shifted Puget Sound VA Health Care System
    (HCS), which includes the American Lake and Seattle VAMCs, from the
    original date of August 2022 to March 2023; and the VA Portland HCS,
    which includes the Portland and Portland-Vancouver VAMCs, from November
    2022 to April 2023. These decisions were based on system stability
    concerns. Moving the deployment of these larger, more complex sites
    allows Oracle Cerner more time to deliver on its commitment to
    stabilize the system and implement our top priority capability
    enhancements.
        We have paused going live at sites until 2023 to get this right. We
    are using our readiness checklist to determine their viability, and as
    always, we will adjust to ensure we are deploying a safe and effective
    EHR system. During the remainder of the calendar year, VA will be
    actively working on updates to the system, which includes testing at
    the Department's most complex facilities, as well as adding new
    capability enhancements. We are also still very much engaged with our
    past deployment sites, closely monitoring and assessing user
    experience, adoption of the new system and lessons learned.
        The full EHR deployment schedule through 2028 is currently under
    development with VHA and VA's Office of Information and Technology
    (OIT) and will be ready in fall 2022.
                            capability enhancements
        VA is committed and working diligently to resolve the challenges
    and issues identified in the strategic review, and by the Office of
    Inspector General (OIG) and the Government Accountability Office (GAO).
    We already have made progress on many of the issues identified. As of
    September 2022, we have closed 20 of the 68 OIG recommendations and are
    working with OIG to close an additional 12. However, some of the
    remaining recommendations are complex that cannot be closed out until
    the IOC phase is complete. Additionally, we are focused on ensuring
    technology stability and system enhancements, as well as on rigorous
    processes to manage budget and expenditures, aligning them with
    schedule, requirements and performance, among many other program
    improvements. Given the lessons learned from recent deployments, we
    also anticipate improving metrics, system stability, user adoption and
    training.
        In terms of capability enhancements, VA currently is focused on
    four priority areas: pharmacy, suicide prevention, research and revenue
    cycle. Some of these enhancements are above the baseline requirements
    in the original contract but are necessary to ensure that our medical
    providers can deliver care safely according to VA policy and to meet
    the unique needs of Veterans. A notable example of progress is the task
    order modification for seven pharmacy capability enhancements, which
    was awarded to Cerner Government Services on July 6, 2022. The
    preliminary timeline for development of all 7 enhancements was 13-36
    months. Oracle Cerner recently indicated that it would deliver the top
    three capability enhancements prioritized by the pharmacy community in
    6-9 months. In the interim, VA has engaged MITRE experts to evaluate
    and provide recommendations to optimize the current pharmacy process to
    reduce burden on our medical personnel.
        Another key concern among clinicians has been the visibility and
    prominence of patient behavioral health record flags. Flags are
    currently configured and available in PowerChart and FirstNet as part
    of core commercially available capabilities. However, these flags can
    be bypassed by clinicians, so we are working to enhance them--in all
    Cerner applications--to prompt clinicians to address them without the
    ability to move forward until appropriate action has been taken. In the
    meantime, staff have been trained on the workflow of accessing the
    alerts via an additional click from within those applications. We are
    in the process of adding additional mental health and patient record
    flags with task order modification award anticipated in the next 1-3
    months. Once awarded, we anticipate having the ability to add three
    capabilities in 2-4 months, and an additional 2 capabilities in 18-24
    months.
        Like other EHR systems, the Cerner EHR system includes a queue to
    capture erroneous orders. The ``unknown queue,'' is not a defect of the
    EHR, but rather how the system is designed. It functions to catch
    orders that cannot be delivered and completed so that they can be
    reviewed by staff for correction.
        The problem with the unknown queue at Mann-Grandstaff VAMC was
    related to a failure of communication, training and processes.
    Unfortunately, responsible Mann- Grandstaff VAMC personnel initially
    were not aware of the unknown queue and how to work with this feature
    when the new Cerner EHR system was deployed. Subsequent actions have
    been taken to ensure that the queue is working optimally, including
    ensuring order locations are configured properly, adjusting workflows,
    identifying staff to monitor the queue who are trained in its use,
    developing tip sheets and additional resources, among others. Almost
    all this work was completed prior to the 2022 deployments.
        VA now has a process in place for facilities to track orders in the
    unknown queue daily and to assign facility staff to correct and
    resubmit the orders in a timely manner.
        The issues discussed in the recent OIG report regarding the unknown
    queue were useful to further enhance its operation. VA has implemented
    corrective actions and reported them to OIG via a memorandum, dated
    July 6, 2022, requesting closure of the recommendations, and is
    diligently working to ensure that all facilities that have already
    deployed or are deploying in the future are adhering to the appropriate
    processes.
        Proper training is an important element of a successful deployment.
    This means providing timely tailored, well-constructed coursework that
    requires active participation. VA has taken several actions to address
    identified training concerns, including:

      --Engaging with independent consultants (McKinsey & Co) to review the
            contents and delivery of the training program, collecting end-
            user feedback and other related data and providing
            recommendations for improvements in the training program based
            on industry best practices;

      --Conducting interviews on content areas of concern with super users
            and Clinical Councils;

      --Working with EHRM-IO to incorporate feedback from listening
            sessions with super users at Puget Sound HCS regarding virtual
            super-user training;

      --Piloting transition of 400-level courses including Sign-On Fair/
            Favorites Fair courses to local sites' super users and/or local
            sites' informatics teams (this is part of the strategy to
            transition ownership of appropriate activities from Cerner to
            VHA for long-term sustainment); and

      --Implementing ongoing training content updates based on lessons
            learned, system changes and feedback from active EHR users.
                               system reliability
        VA continues to actively address concerns regarding system outages
    and degradations and is holding Cerner accountable. Not only are these
    episodes frustrating and disruptive to our medical personnel, but they
    potentially could put Veterans' safety at risk. We are also working
    collaboratively with DoD, the Federal Electronic Health Record
    Modernization Program Office, Cerner and Leidos to ensure stability of
    the Federal network. We are instituting prevention strategies and
    working to recognize problems earlier and improve notification
    procedures. Further, Cerner has committed to upgrading the current
    system and to the introduction of procedures for responding more
    quickly to service disruptions to ensure a better, more reliable user
    experience.
        Cerner has failed to meet the 99.9 percent service uptime Service
    Level Agreement for 7 out of the last 13 months (June 2021 through July
    2022) and the Department has received financial credits for Cerner's
    failure in meeting the contractual level of performance. To further
    hold them accountable, VA sent a second Letter of Concern to Cerner on
    August 5, 2022, reiterating our concerns and directing Cerner to
    provide their technical and operational roadmap to remedy the ongoing
    system instability issues within 30 calendar days.
        These problems put our medical professionals' ability to deliver
    safe and effective care to Veterans at risk. Cerner's failure to
    resolve the system instability issues may result in the use of other
    contractual remedies within the Government's authority.
                            new functional champion
        VHA's involvement with the EHRM program is critical to the success
    of the EHR modernization initiative. The Office of the Functional
    Champion (OFC) is VHA's representative embedded within EHRM-IO and
    engaged across the Department. The OFC will lead functional initiatives
    to support VA's medical personnel, including collaborating daily across
    VA offices and across the health system to coordinate the development
    and implementation of EHRM-related activities. OFC works closely with
    VHA to ensure that our clinical community's interests are represented
    and integrated into each facet of the program, including leadership,
    staffing, governance and deployment.
        We are pleased to have a new Functional Champion, Dr. David
    Massaro, as part of our leadership team. Dr. Massaro started on August
    1, 2022. He is board certified in family medicine and health
    informatics and will lead EHRM-IO's clinical and business functional
    efforts, including change management and training activities. Dr.
    Massaro formerly served within VHA in several executive roles,
    including Acting Chief of Clinical Informatics Operations for the
    Office of Health Informatics. He previously spent over a decade as a
    physician at VA.
        More VHA personnel are being integrated fully into OFC, including
    informaticists, solution experts and informatics patient safety
    experts.
                       budget overview and cost estimate
        In support of this effort, the President's Budget includes $1.8
    billion for fiscal Year 2023. This is in alignment with the new
    strategy, which adjusts the baseline requirements to align with VA's
    updated deployment plans. This funding is vital to support the 18
    currently proposed EHR deployments scheduled for fiscal Year 2023, as
    well as the pre- deployment activities at future sites. These pre-
    deployment activities typically begin 13-15 months in advance of go-
    live dates to ensure sites are equipped to receive the new EHR system.
        In fiscal Year 2023, VA currently plans to conduct EHR and
    infrastructure readiness activities at 68 sites across 7 VISNs. The
    funding will provide for:

      --EHR: Contracts for site assessments, site transitions, enterprise
            integration and site implementation, including activities such
            as site activation, training and workflow development.

      --Infrastructure: Information Technology (IT) and other
            infrastructure investments, such as IT upgrades, modifications
            to existing systems and interfaces.

      --Program management support: Government staff (e.g., salaries and
            benefits), Government administrative expenses and contractor
            support.

        Continuity of funding is integral to our ability to prepare sites
    for the deployment of the new EHR, and to execute VA's rollout
    schedule. By the end of fiscal Year 2022, EHRM-IO will have invested
    infrastructure readiness funding in 15 out of VHA's 18 VISNs. VA will
    also complete the vast majority of infrastructure modernization work in
    VISNs 10 and 20, and initial progress will be made in 13 additional
    VISNs. The FY 2023 budget also supports security, server stack and
    Local Area Network work at the final three VISNs, the initial set of
    infrastructure readiness items that the sites receive.
        In addition to the funding requested for the EHRM account, VHA's
    Medical Facilities request includes $505 million in Non-Recurring
    Maintenance funding for facility infrastructure projects required to
    support EHRM. Some of the projects funded by this request include: $43
    million at the Brockton VAMC, $45 million at the West Haven VAMC, and
    $45 million at the Dallas VAMC for required data cabling, electrical,
    heating/ventilation/air conditioning, and data center upgrades.
        As planned, the fiscal Year 2023 President's budget provides the
    necessary funding to prepare for and meet the deployment requirements
    at sites that will go live in fiscal Year 2024 and early fiscal
    Year2025. Thanks to the support of Congress, funding already provided
    in FYs 2021 and 2022 supports the IT physical infrastructure
    requirements essential to the new EHR. EHRM program funding continues
    to support site preparation activities, including the IT
    infrastructure, distinct from pre-deployment activities described
    above,a that must be completed 12-32 months prior to go-live and
    deployment activities to prepare sites for the new EHR system.
        In fiscal Year 2021, the VA OIG published two reports that each
    found deficiencies in the Department's Life Cycle Cost Estimate for
    EHRM and identified the need for an ICE for EHRM. In response to those
    reports, VA reviewed current and historical costs across the Department
    to ensure that, beginning in fiscal Year 2022, our quarterly financial
    reports to Congress provide a more complete picture.
        To address OIG's concern regarding the lack of an independent cost
    estimate, VA procured the services of the Institute for Defense
    Analyses (IDA) to develop an independent cost estimate that includes
    EHRM-related costs attributable to EHRM-IO, VHA and OIT among other
    costs related to the new EHR throughout the life cycle of the system.
    This estimate provides VA leadership with a neutral, independent
    assessment of potential costs to implement and operate a new EHR. VA
    facilitated briefings with key Congressional staff on the preliminary
    cost estimate in July--now that VA has received the draft final report
    from IDA, we have provided a copy to the Committee as promised.
        The four main drivers of differences between EHRM's estimates and
    IDA's are the deployment timeframe, sustainment, inclusion of
    productivity losses across the deployment and cost differences among
    existing elements of the deployment process.
        For the specific difference between VA's and IDA's cost estimates
    for EHR deployment, VA's estimate spanned 10 years whereas IDA's
    estimate covers a timeframe of 13 years. VA's estimate was based on the
    current 10-year contract. IDA's estimate of 13 years was derived from
    examining data on historical enterprise resource planning programs.
        In its estimate, IDA also includes the cost for some sustainment
    during the implementation phase plus 15 years of sustainment operations
    once the system is fully deployed. The specific sustainment cost point
    estimates in IDA's life cycle cost are $3.5 billion during the
    implementation phase and $17.1 billion during the 15-year fully
    deployed phase. VA's estimate did not include some of the costs for
    operations and support during the implementation phase nor any
    sustainment costs during the fully deployed phase.
        In total, IDA's estimate includes an estimated $25.9 billion in
    costs for elements not in scope of VA's estimate. These additional
    elements (i.e., acquisition, sustainment pre- and post-full deployment)
    account for about 75 percent of the cost difference between VA's
    estimate ($16.1 billion) and IDA's estimate ($49.8 billion).
        The remaining approximately 25 percent difference between VA and
    IDA estimates is due to IDA independently producing higher cost
    estimates for some of the elements common to both VA and IDA estimates.
    These increased costs were derived from VA actual costs and the IDA-
    estimated 13-year implementation schedule. Cost increases are common
    for programs of this complexity, and prior enterprise resource planning
    programs have had similar cost increases in acquisition.
        IDA's cost estimate excluded consideration of the effects of
    sustaining our current EHR, VistA. VistA must remain operable until all
    required functionality is replaced. The total cost to sustain VistA in
    fiscal Year 2021 was approximately $841 million. We expect this VistA
    cost to continue during the deployment of the Cerner system.
                                   conclusion
        Our focus is keeping Veterans at the center of everything we do and
    our top priority remains and continues to be advancing a culture of
    safety and high reliability, with the goal of zero incidents of patient
    harm. Veterans deserve high-quality health care--that means health care
    that is timely, safe, Veteran-centric, equitable, evidence- based and
    efficient.
        Thus, during the remainder of this year, we are working on ensuring
    the stability and resiliency of the EHR system and making improvements
    to the system, including usability improvements for our health care
    personnel. We are staying engaged with past deployment sites and
    providing support to our front-line personnel as well as fixing those
    issues they have identified. We are holding ourselves and Oracle Cerner
    accountable and continuing the work to deliver a more successful EHR,
    which will ensure delivery of world-class care to our Veterans.
        While modernizing VA's EHR is a fundamental change in how business
    and health care work processes are performed within VA, it presents us
    with opportunities to transform the way we deliver health care, and to
    standardize that delivery across the enterprise to achieve improvements
    in patient safety and efficiency in health care deliver. Because this
    initiative is so transformative in terms of how Veteran care is
    provided, the success of the project depends on how well we prepare and
    support the people who use it. Be assured that the resources you have
    invested in VA's new EHR system, when fully implemented, will support
    VA in delivering world-class health care and will improve access,
    outcomes and the experience for Veterans for decades to come.
        Finally, we want to acknowledge what may be top-of-mind for many of
    our stakeholders, including Members of the subcommittee. We understand
    the uncertainty this type of innovation can bring as meaningful,
    industry-shifting change often does. In a rollout of this scale and
    complexity, challenges are expected, they are inevitable, and we are
    prepared to address them. We learned much from our first deployment
    almost 2 years ago and have improved our deployment strategies. In
    fact, in the years ahead, a successful EHR deployment must reflect what
    we have learned, with each challenge helping to better inform and
    position the next deployment.
        I again extend my gratitude to Congress for your continued support
    and shared commitment to serving Veterans with excellence. With your
    continued support, VA will realize the full promise of a modern
    integrated health record to cultivate the health and well-being of
    Veterans. We are happy to respond to any questions that you may have.

        Senator Heinrich. Thank you, Deputy Secretary. We will
    proceed with questions using the standard five-minute rounds,
    and senators will be recognized in the order that they arrive.
    I will start by recognizing myself for five minutes.
        Deputy Secretary Remy, do you have any thoughts or
    impressions you can share with us about the life cycle cost
    analysis that was provided by IDA? Does it align with what you
    are seeing, or whether you have concerns with parts of their
    approach? Either way, just give us your honest impressions.
        Mr. Remy. Mr. Chairman, thank you for the question. Indeed,
    it is important for us to have this Independent Life Cycle Cost
    Estimate, and we thank the Office of the Inspector General for
    pointing out the need for a VA to do that. Having this
    information is a data point that will help us build toward the
    future. There are differences and distinctions between our
    program office estimates and this life cycle cost estimate.
        You mentioned them earlier that it goes for 3 years longer
    than our deployment schedule of 10 years currently does, that
    it allows for 15 years the same tail at the end, and it also
    covers other potential costs based on risk assessments that IDA
    did when they were performing the life cycle cost estimate. So
    we recognize that those are differences and distinctions, but
    we believe that the information in the estimate is valuable to
    us, as we look toward the future.
        Senator Heinrich. How much does it cost annually for the
    Department to maintain and update the VistA System? And am I
    correct in assuming that these costs will be eliminated once
    the Cerner record is fully deployed across the system?
        Mr. Remy. Mr. Chairman, we estimate that amount to be
    around $800--$900 million on an annual basis to maintain the
    VistA System. Understand, however, that VistA is integrated
    with many other components of our IT systems, our financial
    management system, and other systems that rely upon VistA as a
    component of their operation. So as we modernize, we will be
    modernizing those other systems as well, but immediately we
    would not be able to draw down the usage of VistA because it is
    integrated with other things that we are using.
        Senator Heinrich. So tell me about how those costs will be
    impacted as the Cerner record is fully deployed? Do you see a
    transition there as opposed to a cliff? Talk to us about how
    you are going to integrate that.
        Mr. Remy. Sir, I believe the way you have described it is
    accurate, that there will be a transition. I can't here say
    that we can identify, and quantify any specific cost savings
    that will result over time, because of the integration of the
    VistA System with other components of our technology.
        Senator Heinrich. This committee has requested, both in
    meetings and in our annual bills, metrics to measure whether
    this EHR implementation is good for individual veterans, or is
    a good use of taxpayer dollars. And while VA has shared really
    hundreds of technical metrics, we are still not entirely sure
    what constitutes success. So by what measures should we be able
    to tell whether the Department is succeeding?
        Mr. Remy. Starting with what you have just described, is
    our veteran experience. We have engaged in this endeavor to
    enhance outcomes for our veteran population, and we want to
    make sure that our clinicians have the tools to achieve that.
    And so we have engaged in a process to gather information from
    our clinicians and our veterans to understand what needs they
    have that aren't being met by the system, and how we can go
    about meeting those needs.
        Now, that is a general statement not a specific KPI, if you
    will, to measure success. Success at the end of the day though,
    is measured by the performance of the system to achieve that
    outcome enhancement for our veterans, and that is what we are
    looking toward. When you look at the system's operations and
    you understand one of the things that is obligated in the
    contract for our contractors to provide, is that the system be
    operable 99.9 percent of the time.
        I mentioned in my opening remarks some frustrations around
    outages and degradations; that is one of the measures that we
    look at to determine: Is the system reaching the people that it
    needs to, when it needs to?
        Senator Heinrich. VA always emphasized effective change
    management as key to really successful implementation, yet the
    provider feedback to the initial rollout has been quite
    negative. As the shift from VistA is happening, what is VA
    doing to support clinicians across that transition?
        Mr. Remy. Mr. Chairman, we are arm in arm with those
    clinicians. We recognized that the initial rollout was not a
    rollout that had all of the components that it should have at
    the time, and presented a significant number of challenges. And
    so we have learned from that, and what we are doing now is to
    make sure that we have at-the-elbow support for our clinician
    community, our provider community, as they use this system at
    each site that is deployed.
        I know we are almost out of time, but if you will, I would
    like to ask my colleagues to add to that response. Dr. Adirim.
        Senator Heinrich. Please, Dr. Adirim.
        Dr. Adirim. Yeah. Now, and I think the Deputy Secretary
    pointed out a very important thing that we did in the
    subsequent four deployments, was ensuring ample at-the-elbow
    support with an adequate amount of time for our medical
    personnel to learn the system. But we have done a number of
    other things that I learned while at DOD, DOD being very
    successful, almost done with their CONUS deployment, that
    included ensuring that we have local leadership engagement, and
    we do this in multiple ways.
        We do this several weeks before deployment engaging
    leadership to leadership, ensuring they have what they need,
    and that they are fully activated to help their people get
    through the deployments.
        We have also started, and we have already done two, and we
    have four more planned, called VISN Director Workshops well in
    advance of deployment, so that site leadership and deployment
    leadership understand what is expected of them and what they
    need to do in order to prepare their medical personnel. So we
    have learned a lot, not only from VA's first experience, but
    also from other health systems, how we support our personnel.
        And lastly, VHA did something really smart, they activated
    their VISN Clinical Resource Hubs to this effort, as well as,
    centrally, they have EHRM, they called the NESSU, which is the
    Supplemental Staffing Unit, to provide support to operations,
    whether it is virtual or onsite, for nursing, mental health,
    and so on, during this post-go-live period to ensure that
    operations can be maintained and support our veterans as we are
    going through this.
        Senator Heinrich. Great. Thank you, Doctor.
        Ranking Member Boozman has very graciously allowed Senator
    Tester to go next.
        Senator Tester. And I want to thank Senator Boozman. And I
    want to thank you, Mr. Chairman, also, for the opening
    statements. And I want to thank both of you for your leadership
    on these issues.
        As you guys were giving your opening statements it occurred
    to me that not everybody is focused on what is going on with
    electronic health records in the VA. Not everybody in the
    Senate it is a top priority, but I can tell you for the two men
    in my left it is a top priority. And I would say the same thing
    for Senator Moran and myself.
        Getting this right is really important. The contract was
    awarded in May of 2018, and we are, depending on what the
    numbers come in at the end of the fiscal year, somewhere
    between $7- and $9 billion into this outfit. And to be honest
    with you, from where I sit, I don't know that we have got a
    return on investment to speak of at all.
        And so we have got long ways to go. And as I think I have
    told you, Mr. Remy, if I had all the money that was
    appropriated for computer programs for my time in the State
    Legislature and here, we would make a serious dent in the
    national debt, truthfully. We have got more work to do. And
    even though you guys, as Senator Heinrich has pointed out,
    weren't the people here when this contract was awarded, it is
    your responsibility now to either figure it out and move
    forward, or figure out a different way to go.
        So look, the VA does not have any more installs planned for
    the rest of this year, and I think you have got a plan to
    restart in 2023, and then during that time you are working
    along with the folks from Oracle Cerner to fix any issues that
    are out there to make this thing work, so you can launch it
    somewhere else.
        Deputy Secretary Remy, what data-driven set of metrics are
    you going to use to decide whether to take this system into
    more facilities? That is a little different question than
    Senator Heinrich just asked. But what are you going to be
    looking at to say, all right, we are ready to fly, we are going
    to move out?
        Mr. Remy. Thank you, Senator. And you we have a Site
    Readiness Deployment Checklist that we have developed
    alongside, we being are VHA, an OEHR, or EHRM-IO, along with
    our contractor that spells out the types of things that will
    need to be done before we can go live.
        And I can speak to some of those things. They include
    things like training, which is critically important for those
    people that are going to be using it, infrastructure, patient
    safety protections, and those types of things that we go
    through before we determine that a site is ready to go live.
    And this is an iterative process. It is not a static process.
        As we are working with a site for potential deployment, we
    work through these issues to make sure that they have them
    covered. An example of the effectiveness of that checklist was
    Boise, recently, where we determined we wouldn't go live as we
    were going through the checklist.
        Senator Tester. Okay. So as you are looking at this
    situation right now, and as we don't plan any installations
    until 2023, so I am going to ask you an unfair question: When
    do you anticipate, in 2023, that you are going to be able to
    implement and install?
        Mr. Remy. Well, we have a schedule, but we are looking
    closely at this schedule, and we are realizing that there are
    issues that need to be resolved before we can go live. Right
    now our schedule would have us go to another installation in
    early-2023, January/February. If we have to push that back, we
    will push that back.
        Senator Tester. So the question is, and I get it, and I
    don't want anything implemented before it is ready for prime
    time, on the same token we have got an investment in a program
    that needs to start delivering at sometime. And so are the
    problems with the program so intense that it is going to take
    months, and months, and months to solve them?
        Mr. Remy. Some of the problems are challenging, and I mean
    we talk about people processes and technology, and making sure
    that we have each of those categories right as we move forward.
    We have learned things from prior deployments that will help us
    to enhance the system, so when we roll it out the next time we
    can roll it out in a way that can be more successful. We are
    learning things from our recent deployments, even, that that
    can help us move forward.
        So I don't know that I would necessarily, sir, say that the
    problems are that intense, but the problems are real enough
    that we are not moving forward until we resolve some of them.
        Senator Tester. I got you. I just, I am concerned. We are
    into this damn near 5 years, it will be 5 years in May, and we
    are still, truthfully, I mean we are still, and we haven't done
    a damn thing. I mean we have implemented, and they have been a
    train wreck, in my opinion. And so the question really is here,
    and by the way, when Cerner comes up on the next panel it is
    the same thing, you guys have to be working really, really
    close with these folks, they have to be held accountable, they
    have to hold themselves accountable, you have to hold
    yourselves accountable. And we need to get this damn thing
    done. Because it is really hard for me to go back to the people
    anywhere and say, you are spending my money really well back
    there. Okay? You got the drift?
        Mr. Remy. Yes.
        Senator Tester. All right. Thank you.
        Senator Heinrich. Thank you, Senator.
        Ranking Member Boozman.
        Senator Boozman. So Senator Tester, I think the Secretary
    is playing in the football game tonight.
        Senator Tester. He is?
        Senator Boozman. I think that is tonight, isn't it?
        Senator Tester. It is
        Senator Boozman. Well tell him, from Senator Tester and I,
    that if he gets injured he still has to come and testify.
        Mr. Remy. Well, his testimony is before the football game,
    sir.
        Senator Boozman. Oh, that is right. Very good; well, we
    will talk to him and wish him good luck. I don't know who
    talked him into that, but that is tough deal.
        Secretary Remy, we all agree that the rollout as a rollout
    has fallen short, far short of expectations. Congress has
    appropriated $8.5 billion over the last 5 years. This year's
    request another $1.75 billion. We have talked about that at
    length.
        I guess my question is, you know, we are in a situation
    now, as both Senators Heinrich and Tester alluded to, we have
    only deployed in five locations that still are struggling, and
    I guess the question is: How can we work together to get back
    on track? You know, what do we need to do to help you? And then
    again, you know, what do you all need to do, and the
    contractors, to get us in the situation that we need to be?
        So I guess my question would be a couple things right away.
    Do you have the right balance of support staff necessary for
    the project to be successful? And what efforts is VA
    undertaking to identify lessons learned and ways to improve the
    deployment to other sites? How much of the change management
    work is done by VA employees? How much by contractors? What
    specific efforts are underway to enhance your change management
    initiatives, and to ensure your personnel embrace the new
    system? And what is VHA doing to underscore with clinicians the
    Department's full commitment to EHRM? In turn, how are VAMC
    directors and facility personnel held accountable for
    implementing Cerner?
        That is a lot. But can you kind of chip away at that a
    little bit?
        Mr. Remy. I can, Mr. Boozman. And let me start with the
    last point first. And if you will, I would like to ask Dr.
    Elnahal to speak to VHA's commitment. One of the important
    things about having a confirmed Under Secretary is to have the
    messaging delivered to his community, if it is acceptable.
        Dr. Elnahal. Thank you, sir. And thank you, Senator, for
    the question.
        So as the Deputy Secretary mentioned in his opening
    remarks, and as the Secretary has mentioned recently, I do have
    deep concerns about the system as it is functioning for
    frontline employees and service to veterans. I had a chance to
    see that, myself, in a visit to Columbus, Ohio, recently, a
    really great leadership team there who wants to get this right.
    And even more importantly, a great set of frontline folks who
    were in front of veterans trying to use the system, and I
    actually saw them in action using the system.
        And what I will say is, my commitment, as again, the Deputy
    mentioned, is to actually get to a modern electronic health
    record that meets the clinical needs of our veterans, and right
    now I saw folks struggling with the system deeply.
        Among the most concerning things that I saw was a
    phenomenon whereby our frontline clinicians when they put it in
    an order, or trying to interface with the system, they were not
    confident in many cases, and in many clinical settings, that
    those orders were actually getting where they needed to go on
    behalf of their veterans. And there was a lot of manual rework,
    recheck that had to be done to meet those veterans' needs.
        What I took from that, was a need to come back, speak to my
    colleague, Dr. Adirim, Deputy Secretary, about ensuring that
    that is the case going forward by doing a deep investigation
    into that problem, but also to do a broader look at how these
    workflows can improve the configuration of the system against
    those clinical processes that we need to do for vets that also
    needs to improve.
        I will say that there was a nucleus of folks at that
    facility as well, in some areas, who did find promise in the
    system. In the urgent care setting, some of the surgical,
    medical staff said that they appreciated it and found promise
    in it. So I think we can get there. But right now I did not see
    a system that was meeting most of the frontline clinician's
    needs. And that concerns me. And I know it also concerns the
    Deputy. And we are doing what we can to try and address those
    issues.
        Senator Boozman. So as a provider, you know, looking at the
    system you said that there was concern about the orders
    actually getting where they needed to go. What effect would
    that have on a patient, on patient care? Can you give an
    example?
        Dr. Elnahal. Well, one thing I will say, Senator, is that
    this is not a new issue, per se, this is something that was
    discovered in the circumstances around the unknown queue, and
    some other issues. What I was surprised by was the frequency by
    which I was seeing that, and the number of staff who mentioned
    that to me.
        And so what that meant was, that is a call to action for us
    to configure this system in a way where the workflows are
    intuitive, where clinicians, for example, don't necessarily
    have 10 to 20 choices for any given order or action in front of
    them. Looking at our training and ensuring that where the
    system can be configured better we can do that, but also that
    we have a confirmation that the broad swath of employees being
    trained know what to do in front of a veteran.
        And so those issues we have to focus on, Senator. In full
    transparency, those things absolutely we need to improve.
        Senator Boozman. Okay.
        Mr. Remy. Senator, I know you had a list of questions, and
    we are over time, I am happy to touch on the staffing and
    change management, if you would like.
        Senator Boozman. Yes sir, if you don't mind.
        Mr. Remy. Absolutely. I mean, one of the things that flows
    from what Dr. Elnahal described is: How do you get a workforce
    to embrace change management in an environment where they see
    challenges? And the answer to that question is to provide them
    with the tools to overcome those challenges, to recognize that
    the outcome of the hard work that they put in to making sure
    that this system works is going to be improved outcomes for the
    veterans that they service.
        And so what we have attempted to do is to provide them with
    those tools, to understand the challenges when they present,
    and to take swift affirmative action to resolve and remediate
    any problems that might arise. And we are doing that through
    our staff, through our governance process for the EHRM program,
    and through our work with the contractors, with Oracle and
    Cerner, to make sure that they have those tools to get past
    those challenges.
        In terms of staffing, we are in the process of hiring; Dr.
    Adirim can speak to some of the staffing activities that are
    going on in the EHRM Integration Office. Similarly, I mentioned
    earlier that we have just brought on a new functional champion
    who is our linchpin with the VHA, and there is some hiring
    going on in VHA as well, to make sure that we have people that
    are available to do the work.
        And then lastly, you ask about the role of contractors,
    whether that be the contract that we have with Cerner, or other
    contractors, and we utilize those contractors to assist us in
    delivering the system through training, and other activities
    that can help bolster our workforce. But the core of the work
    that is being done is being done by our team at VA.
        Senator Boozman. Thank you. So initially, we predicted 10
    years. I don't think we can get that done. Do you have a
    somewhat concrete figure as to how long it is going to take us
    to actually get this thing?
        Mr. Remy. Well, sir, the original plan was 10 years, as you
    mentioned, and we have been working feverishly to try to make
    that target. We have made some adjustments in the out years to
    the time for pre-deployment, deployment activities that might
    shorten that based on lessons learned to be able to meet the
    10-year mark.
        IDA, in their cost estimates, says they believe that
    initial deployment time period is 13 years. And we have been
    looking at, if we have to go beyond the 10 years, what does
    that look like? We don't have here today a specific time period
    beyond the 10 years because we are still looking at all the
    factors that might present, so that we have to go beyond the 10
    years, if we do, nor do we have a dollar amount attributable to
    that, but I can assure you we are looking at what it would take
    to make sure that we deploy the system in a safe, effective
    manner for our clinicians, and our veterans, and if that needs
    to go beyond 10 years we are working through the process of
    determining what that time period might be.
        Senator Boozman. So let me ask you this. I will note that
    the VA has not yet allowed IDA to publicly release their
    independent cost estimate, we were told we can expect it to be
    released in mid-August, and now it is nearly October. It is
    challenging to hold an open hearing where a significant part of
    the discussion is about a document that you won't release. When
    can we expect this estimate to be released to the public?
        Mr. Remy. Yeah, I understand your question. And we have
    provided the full document to the committee. It is a
    competition-sensitive document that we would have to pay
    attention to if we were to release it more broadly. But I can
    ask Mr. Rychalski to respond to that because he has had
    experience with this type of thing before. Jon?
        Mr. Rychalski. Okay. So it is. I guess it is, right now it
    is a legal procurement matter. One thing I will say is coming
    out of DOD, they would not release the cost estimates publicly,
    I think through FOIA requests, they would allow people to come
    in and look at pieces of it for that reason. So what I would
    say is, it is the legal and procurement communities are looking
    at it now to decide if it can be publicly released, or released
    kind of how DOD does it, which is a little bit more limited.
        Senator Boozman. Thank you, Mr. Chairman.
        Senator Heinrich. Senator Tester.
        Senator Tester. I have got a real quick question for you,
    Dr. Elnahal. And thanks for being here. It is good to have you
    on board. You talked about the Columbus visit. Health care
    people are under a lot of pressure just doing their job,
    period. This adds another level of stress. Are you concerned
    about VA burnout, employee burnout?
        Dr. Elnahal. What I will say, Senator, is that is
    definitely in play, in the medical centers who have done this.
    And it was something I personally observed when I went to
    Columbus.
        Senator Tester. You personally observed burnout?
        Dr. Elnahal. I personally folks telling me that the system
    was stressful to use, and leadership was telling me that folks
    were leaving, in part because of the difficulty of the
    workflows. Now, that said I will just put this into context.
    You know, EHR deployments, more generally, make things
    difficult in their initial phases. And we are in the IOC phase.
    But nonetheless that is a phenomenon that was communicated
    clearly to me when I was at Columbus.
        Senator Tester. Do you, okay so change is hard, and they
    are difficult to implement; is there any way that you can, or
    do you have plans for trying to deal with this upfront with the
    next rollout?
        Dr. Elnahal. Well, I will mention that what Dr. Adirim
    mentioned around the clinical support teams, from our clinical
    resource hubs, the supplemental staffing units that we send are
    helping. What we have to do, ultimately, that will not only
    impact burnout, but more importantly what the veteran
    experience is, is configure the system in such a way that is
    intuitive to our frontline clinicians, and allows the system to
    get the job done for the clinical needs of veterans.
        Senator Tester. Do you think that is possible?
        Dr. Elnahal. I do, Senator. I do think it is possible. I
    think it will require a significant amount of work, but we can
    get there.
        Senator Tester. All right. Thank you. Thank you, Mr.
    Chairman.
        Senator Heinrich. Thank you for your question, Senator. I
    think that will wrap up our first panel for now, and we will
    allow our second panel to get situated.
        Thank you all very much, for testifying today.
        Mr. Remy. Thank you, Mr. Chairman. Thank you, Senators.

                                    PANEL II

        Senator Heinrich. Our second panel today is intended to
    provide the non-VA perspective.
        And for that discussion I would welcome, Mike Sicilia,
    Executive Vice President of Industries for Oracle, who is
    overseeing Oracle Cerner's efforts; Brian Rieksts, of the
    Institute for Defense Analyses, or IDA, who oversaw the
    development of the Independent Life Cycle Cost Estimate; and
    David Case, VA's Deputy Inspector General who oversees work
    related to VA's EHRM effort, including inadequacies of VA's
    initial life cycle cost estimate and patient safety concerns.
        We appreciate all of you being here today to discuss your
    roles in reviewing this initiative. I am going to recognize
    each witness for five minutes for your opening statements, and
    your full, written testimony will all be included in the record
    today.
        We will begin with Mike Sicilia of Oracle. Mr. Sicilia.
    STATEMENT OF MR. MIKE SICILIA, EXECUTIVE VICE
                PRESIDENT, INDUSTRIES, ORACLE
        Mr. Sicilia. Chairman Heinrich, Ranking Member Boozman, and
    Members of the Committee; thank you for the opportunity to
    speak with you today.
        As you probably know, approximately 4 months ago Oracle
    acquired Cerner, along with its VA EHRM program. As I stated in
    my written testimony, I hope you will agree, we believe our
    acquisition will reinvigorate this program and deliver on the
    promise of a single, longitudinal health record, from
    enlistment, and active-duty service at DOD, through retirement
    and elder care at VA.
        We believe we can deliver a system that will leapfrog
    existing commercial EHRs and deliver for our service members
    and veterans the gold standard for modern health care
    technology. In the process, we are confident that delivering a
    modern EHR system will improve patient outcomes through
    analytics and machine learning, all while enabling advanced
    care delivery channels like telemedicine and mobility.
        Importantly, for this committee, we believe we can deliver
    this system within the existing budget envelope envisioned for
    the current program scope without the need for any additional
    funds. To date Oracle Cerner, Millennium EHR is fully deployed
    for the Coast Guard, and is deployed at more than half of DOD
    medical facilities serving over 200,000 end users. I think a
    fair assessment is that the Coast Guard and DOD deployments are
    on track and proceeding successfully.
        At the VA the story is a little different. Millennium is
    deployed at five medical centers and their associated
    facilities, and work is underway at over 40 medical centers for
    their upcoming deployments. Some of the delays are pandemic
    related, but it is also true that there have been technical,
    functional and training challenges at VA facilities. As we have
    examined the underlying causes for these delays and challenges,
    our conclusion is that we have found nothing that can't be
    addressed in reasonably short order to get us back on workable
    schedule, and within budget.
        We know we have a lot to prove with deployments next year
    at larger more complex sites. We view the next year as a key
    window for building momentum and turning the corner just as we
    have done at DOD where initial adoption was also challenging
    and is now proceeding apace and with strong results.
        We recognize that training must be improved, and have
    recently engaged a third-party firm to evaluate the current
    training program. We are announcing today that we will be
    engaging Accenture to work with us to make the training much
    more efficient, applicable, and useful for caregivers.
        We are also committed to communicating timely and accurate
    information about our progress. This week we launched a
    dashboard that catalogs our to-do list and progress being made.
    We encourage you to view the dashboard frequently, and hope it
    will keep everyone, including us, focused on deliverables and
    dates.
        I have already alluded to cost, but let me be clear about
    value. We intend to rewrite the Millennium EHR as a stateless
    cloud application which will deliver a modern user interface,
    ease of use, mobility, voice recognition, and self-service. It
    will have machine learning based clinical decision support, and
    analytics that are built in from the ground up. We intend to
    deliver a beta of this new system in 2023, and we commit to
    deliver it across VA as a cost-free upgrade under the current
    contract.
        To be clear, this is a 10-year, $10 billion contract that
    already has had 700 million, or so, in additional funding due
    to additional scope. That is $10.7 billion. Even with delays,
    and perhaps an extended timeline for deployment. That is the
    number we intend to deliver the existing system for including
    rewriting the system for the cloud.

        [Clerk's Note: The following Addendum was received for Mr.
    Mike Sicilia]
         Addendum to Testimony of Mike Sicilia, EVP-Industries, Oracle
        Thank you for the opportunity to participate in the Senate
    Appropriations Committee, subcommittee on Military Construction,
    Veterans Affairs and Related Agencies hearing on ``VA's Electronic
    Health Record Modernization: An Update on Rollout, Cost and Schedule''
    held on September 21, 2022. As VA's new EHRM partner, I appreciated the
    opportunity to update the subcommittee on Oracle's commitment to the
    program and plans to get it back on track and keep our costs in line.
        In my testimony I committed that Oracle intends to deliver the EHR
    system across the entire VA for the amounts contemplated in the current
    contract under the current scope, barring big new requirements being
    added by VA. I reiterate that commitment today, but also wish to
    clarify that in my comments I did not mean to imply the contract
    ceiling had been raised or is being contemplated to be raised from $10
    billion to $10.7 billion. It of course has not and remains unchanged
    from the original IDIQ contract award in 2018. We take seriously our
    responsibility to be good stewards of taxpayer dollars and will
    continue to look for opportunities to bring efficiencies and cost
    savings to this program.
        Thank you.

        The most important changes Oracle can bring to this
    program, in the short term, relate to improving the EHR
    system's performance and modernizing its technology.
        On August 4th in Kansas City we hosted a Federal Leadership
    Summit to discuss performance and stability issues. Coming out
    of that meeting we have a two-fold plan. First, we are working
    on more than 40 different technical operational improvements
    that we expect to lead to improve performance and greater
    stability for the Federal enclave.
        Second, at the end of July, I announced our intention to
    move the Cerner application at our own expense, and with of
    course Federal approval, to a modern hyperscale cloud data
    center, which will deliver a foundation for better performance
    and stability for end users.
        And last, we are intently focused on improving
    functionality, fixing problems like the unknown queue, or other
    items listed on the dashboard, and working with VA National
    Councils where we believe changes can be made to improve
    clinical efficiency and productivity in the short term.
        The changes Oracle is bringing to the EHRM program are
    significant, and 4 months into owning Cerner, we are confident
    that the problems are being, and will continue to be fixed,
    while at the same time we focus on delivering a far superior
    and modern Federal EHR on an aggressive time schedule.
        We look forward to delivering for our Nation's veterans.
    Thank you.
        [The statement follows:]
                     Prepared Statement of Mr. Mike Sicilia
                                  introduction
        Chairman Heinrich, Ranking Member Boozman and members of the
    Committee, thank you for the opportunity to speak with you today about
    the Dept. of Veterans Affairs' (VA) Electronic Health Record
    Modernization (EHRM) program and Oracle Cerner's Electronic Health
    Records (EHR) system (Millennium).
        I am Mike Sicilia, Executive Vice President for Industries at
    Oracle. I am responsible for Oracle's Global Health Business Unit,
    including Oracle Cerner.
        Today is my first appearance before your Committee, so I want to
    begin by committing to you to be completely candid and transparent
    about where we are, where we are going, and what changes Oracle can
    bring to the Federal EHR.
        I also want to thank you for your past and current support of the
    EHRM program. We expect any program of this magnitude to receive
    intense scrutiny and oversight, and we understand concerns about delays
    and costs, which I will discuss further in my testimony. As VA's new
    partner in this effort, we have committed to getting the program on
    track and keeping our costs in line. In the coming months we hope to
    demonstrate to the Committee that we can deliver for the VA and our
    Nation's veterans so that we will continue to earn your support for the
    program.
        Oracle is a leading enterprise software vendor with more than forty
    years of experience building and developing some of the most advanced,
    mission-critical, secure and performant technology around the world for
    governments, critical infrastructure, and commercial enterprises.
        Oracle employs over 160,000 employees with more than 50,000
    developers and engineers, and in the last 10 years we have spent more
    than $56 billion on research and development. Oracle holds more than
    18,500 patents worldwide. Oracle is in both the infrastructure business
    with the world's leading autonomous database as well as the
    applications business with a full suite of high-performance enterprise
    applications across all industries.
        Oracle is also a leading hyperscale cloud service provider with
    global reach across industries and governments. Of added importance
    here, Oracle operates fully certified government cloud regions under
    the Intelligence Community's Commercial Cloud Enterprise (``C2E'')
    program and is fully qualified under the DoD's upcoming Joint
    Warfighter Cloud Capability Program (``JWCC'').
        Oracle is also a leading cloud applications company with Software
    as a Service (``SaaS'') products across Enterprise Resource Planning,
    Human Capital Management, Supply Chain, and Customer Experience as well
    as industry specific cloud applications ranging from pharmaceuticals to
    banking and retail to utilities. Our systems are performant, scalable
    and secure, and there is nothing in the Federal EHR scope outside of
    our core capabilities.
        Importantly, we have a demonstrated track record rewriting
    extremely complex applications from client-server technology to new,
    modern, stateless web applications which we plan to do here as well.
        As you know, this past June Oracle completed its acquisition of
    Cerner. With this acquisition we brought together one of the world's
    most formidable and capable infrastructure and applications companies
    with one of the leading healthcare applications companies. Oracle's
    engineering expertise together with Cerner's clinical expertise is a
    very powerful combination that we believe will serve our Nation's
    veterans and their caregivers extremely well.
        Our rationale for acquiring Cerner was straightforward. Healthcare
    IT in this country and around the world is significantly out-of-date
    and the associated health data is stove piped. Systems are aging
    without any hope of bringing modern technology innovations like
    analytics and machine learning to improve patient outcomes. Compared to
    banking, telecommunications, transportation, utilities, or any other
    mission critical sector, healthcare IT is furthest behind the
    modernization curve. Across the healthcare industry, EHR systems are
    often bespoke and running on-premises. The VA's current system--VistA--
    is certainly one example of extremely old technology incapable of
    bringing state of the art capabilities to our Nation's veterans.
        Our intention is to lead the way with a new generation of modern,
    cloud-native, highly performant and secure EHR applications embracing
    mobility, self-service, analytics and ease of use, including virtual
    care, such as telemedicine, leading to better patient outcomes. Better
    patient outcomes inevitably lead to reduced health care costs. And
    modernization provides a rare chance to ensure caregivers are enabled
    by the tools they use, decreasing the administrative burden on
    caregivers. Unlike Cerner alone, Oracle brings an order of magnitude
    more engineering resources and scale to this formidable challenge and
    opportunity.
        While Oracle is new to the EHR business, Oracle does have years of
    experience advancing medical research, powering clinical trials,
    reducing healthcare costs and providing public health authorities and
    policymakers with essential data to improve public health. During the
    COVID-19 pandemic, Oracle was honored to collaborate with the Centers
    for Disease Control (CDC) and the National Institutes of Health (NIH)
    to support COVID-19 related systems of record. We assisted in
    electronically pre-screening over six hundred thousand individuals
    willing to participate in COVID-19 vaccine clinical trials and then
    supported the CDC with the creation of the v-Safe After Vaccination
    Health Checker and the Health Partner Ordering Portal (HPOP) to support
    the distribution of vaccine, diagnostic and therapeutic supplies.
        Finally, we worked with the CDC to build a national data repository
    for COVID-19 vaccination data in the U.S. We are accustomed to rising
    to the moment to handle large, complex tasks when our Nation needs it
    most.
        I give you all this introduction so that you understand our
    acquisition of Cerner and assumption of its EHRM contract with the VA
    is well within our capabilities, given our size, expertise, and
    resources.
        You should consider that in effect VA, the Department of Defense
    (DoD) and the Coast Guard obtained a new, vastly more resourced
    technology partner overnight to augment Cerner. We also strongly
    believe in this mission and consider it not only a contractual
    obligation but a moral one to improve healthcare for our Nation's
    veterans and their caregivers. We intend to exceed expectations.
        The EHRM program is the largest health IT modernization project in
    history. The point of the program is three-fold: First, to offer a
    single interoperable longitudinal health record from a person's
    enlistment with DoD through lifelong care at VA. Second, to replace the
    current system conceived in the 1970s and deployed in the 1980s that is
    plainly and objectively past its prime. And third, to improve patient
    outcomes by deploying modern technologies across the entire data set of
    active duty and retired service members to ensure the highest quality
    of care. Doing so would literally leap-frog the commercial, private
    care systems and put DoD and VA at the leading edge of health care
    technical innovation.
        When fully deployed across the VA healthcare system, 171 medical
    centers will go from using 130 different instances of the current VistA
    EHR to using a single enterprise-wide EHR that is shared between VA,
    DoD and Coast Guard. Problems currently experienced with VistA go away,
    such as: data silos creating gaps in patient data and care, veterans
    being tethered to specific medical centers, non-standardized workflows
    across the VA healthcare system, and a patchwork of VistA instances
    that inhibits the full potential of data analytics. These benefits of a
    modern EHR remain an extremely important goal worth protecting as they
    deliver significant improvements in healthcare services and outcomes
    for our Nation's service members and veterans.
        To date, the Millennium EHR has been fully deployed for the Coast
    Guard and is deployed at more than half of DoD medical facilities
    serving over 200,000 end users. I think a fair assessment is that the
    Coast Guard and DoD deployments are on track and proceeding
    successfully.
        At VA the story is a little different. Millennium is deployed at
    five medical centers and their associated facilities, which is far
    slower than anticipated, and work is underway at over 40 medical
    centers for their upcoming deployments. Some of the delays were due to
    the COVID pandemic, but it is also true that there have been
    substantial technical and training challenges at VA facilities. As we
    have examined the underlying causes for these delays and technical and
    training challenges, our conclusion is that we have found nothing that
    can't be addressed in reasonably short order, in part because we have
    shifted Oracle's top talent to work on the DoD/VA EHR system as the
    company's number one priority.
        We believe this moment is a unique opportunity to leapfrog VA into
    the future and make VA's EHR the gold standard for EHR modernization
    globally. But I also need to be clear that modernization requires
    change. There is no amount of money and no amount of work that can
    transform VistA into a modern system capable of delivering for our
    Nation's veterans.
        We can and will do our part to deliver the most performant,
    feature-rich technology within the existing budget envelope for the
    current scope of work, but caregivers must also invest in training and
    learning a new system. Because a system is different does not make it
    unworkable. I am highly attuned to over-worked caregivers who are being
    asked to invest in learning a new technology without an immediate
    improvement in their day-to-day work. But I am also highly attuned to
    what a modern longitudinal health record with all health data in the
    same system can do to improve the care for our Nation's veterans.
        Here is how we plan to address the current problems and get this
    system back on track:

        Performance: On August 4, 2022, in Kansas City, we hosted a Federal
    Leadership Summit with VA, DoD, the Federal Electronic Health Record
    Modernization office (FEHRM) and Leidos to discuss the Federal EHR
    system's performance and recent stability issues. Coming out of that
    meeting, we have a two-fold plan.

        First, we are working on more than 40 different technical
    operational improvement projects for the Federal Enclave that we expect
    to lead to improved performance and greater stability. We have made
    progress already, completing four projects and expecting seven,
    potentially eight, more to be complete by the end of the year. As we
    work on these projects, we will continue to analyze the system and make
    other fixes as needed. These plans are detailed in a letter we sent to
    VA on September 2, 2022, and which is attached for your reference.
        Second, at the end of July I announced our intention to move the
    Cerner application--with, of course, the approval of VA, DoD and Coast
    Guard--to a modern, hyperscale cloud data center, which will deliver a
    foundation for better performance and stability for end-users.
        Once achieved, this move will provide a scalable, modern platform
    for us to deliver the kind of modern technologies users have come to
    expect like mobility and predictive analytics. This is the same
    Generation 2 Cloud infrastructure that underpins Oracle's customers'
    most critical workloads in sectors like Financial Services,
    Telecommunications and Utilities.
        Another advantage of moving the EHR system is that our cloud is a
    second-generation cloud with security built-in from the start.
    Infrastructure security patches are applied automatically with no
    downtime, removing the possibility of human error which is a major
    cause of security vulnerabilities. Oracle maintains all the highest
    government security classifications.
        Moving to our cloud data center will be provided at no extra cost
    to the Coast Guard, DoD or VA, and as I will explain later we believe
    it will help substantially compress costs in the long- run.
        We are cognizant of our Federal partners' concern raised at the
    August 4th summit that the entire Federal Enclave first be stabilized
    and are prioritizing that work in conjunction with the move to an
    Oracle cloud data center. We commit to maintaining the right balance of
    Oracle expertise, Oracle technology, and Oracle infrastructure to meet
    both goals.
        Finally, we are working with VA to approve expanding our
    relationship with our current partners to assist with the work of
    making the Federal Enclave more stable and performant. We hope to
    announce this expansion soon, and believe that this expanded
    partnership will bring much-needed expertise and capabilities for a
    system as complex as this one.
        Design: The second category of issues relates to system design. We
    have heard from providers about challenges with workflows and design
    that is not as easy or intuitive as it should be. While workflows are
    decided by the VA's National Councils and in coordination with DoD we
    will be engaging with the councils where we believe design changes can
    be made to improve clinical efficiency and productivity.
        And of course, one widely known design issue, the so-called
    ``unknown queue,'' was recently reported on by the VA Inspector
    General. Despite its name, the unknown queue was not a bug, it was a
    backstop to account for patient scheduling tasks to facilities or
    providers that were not recognized by the system. These scheduling
    tasks were not lost, rather they were routed for manual review and
    processing, but the fact is the process initially resulted in far too
    many actions being routed to this queue and the manual review was not
    being completed in a timely manner.
        While that has largely been corrected and current instances of
    orders entering the queue are minimal, we still committed to providing
    updates that would further reduce the chance of an order being entered
    incorrectly. On August 1, 2022, we delivered updates to VA to alert
    providers in their message center when an order is unable to be
    scheduled. A second update alerts providers when they go to close a
    patient's chart. VA has these updates for testing, and we will
    implement them when VA is ready. These updates were provided by us at
    no cost to VA. Screenshots of the updates are attached for your
    reference.
        We also are working with VA related to the rescheduling of canceled
    appointments and the potential that some appointments were not properly
    rerouted. We will keep the Committee updated as we learn more.
        Functionality and Millennium's Future: When it comes to
    functionality of the EHR, there are small things that become big things
    in practice and deserve rapid improvement. For example, we learned that
    in Columbus lab employees have not been printing in batches, which
    delays printing labels for vials of blood when multiple bloodwork tests
    are ordered. We have learned the check-in process for appointments has
    been slowed. These are two seemingly small examples in an overall EHR
    modernization, but they have a big impact on the amount of time a user
    spends in the EHR instead of with the veteran, and how much time the
    veteran spends in the facility. We are working to fix these and other
    similar issues that we are aware of to make the system more functional
    and efficient.
        In previous testimony in July, I spoke to functionality concerns
    with the pharmacy module. Following VA's updated pharmacy requirements,
    we are currently working on seven important updates to pharmacy that
    will be delivered over the course of the next year and will make the
    pharmacy module function much better for users and veterans. The first
    three updates will be provided to VA by early next year.
        At the same time, as I promised in July in testimony before the
    authorizing committees, we are developing new pharmacy functionalities
    from the ground-up. After starting this work, it quickly became
    apparent that we need to develop these pharmacy functionalities not as
    a separate module but as an integrated part of the new, modern EHR.
        We are currently investing substantial resources to progressively
    rewrite the Oracle Cerner Millennium EHR as a modern, stateless web
    application, which will include pharmacy functionality. The system will
    have a modern web-based user interface. It will be mobile friendly,
    meaning users can bring their own device. It will include voice
    recognition, and ML- based clinical decision support and analytics that
    are built-in from the ground up. In short, it will be a fully modern
    cloud-based EHR system. Not only will this deliver the longitudinal
    record from enlistment through retirement and lifelong care at VA, but
    also includes the hierarchical view of the entire DoD/VA population
    against which analytics, AI, and machine learning can be deployed. We
    will keep VA, DoD and Coast Guard updated and engaged as we work on
    this modern EHR system, and of course will seek appropriate approvals
    for deployment as necessary. And while I don't want to over-promise
    here, our intent is to deliver a beta version of the new EHR, with
    pharmacy functionality included, in 2023.
        Our plan--and our commitment to you--is to deliver all this
    functionality as an upgrade to the current system as part of our
    existing obligations under the current contract, at no extra cost to
    the government. Let me say that again--we plan to deliver a fully
    modern cloud-based EHR for the DoD, Coast Guard, and VA as part of our
    existing contract with the government.
        Additionally, in the short-term we plan to show VA users and
    veterans a glimpse of what the future system will look like. As I
    mentioned before, during the pandemic Oracle created direct patient
    facing applications like v-Safe. We are expanding these patient facing
    applications so that patients can easily keep their providers
    informed--each day if they like--about their health status. It is a
    simple, intuitive application that patients will be able to use to
    connect with their providers, view appointments and keep up to date on
    new benefit announcements. These new features will be added to Oracle
    Cerner EHR commercial products and therefore will be available to VA
    and DoD at no additional cost. We plan to meet with VA, DoD and
    Veterans Service Organizations to explain these features and assess
    applicability for focused areas like mental health or burn pit exposure
    awareness as two examples. It is the kind of addition to the system
    that will benefit veterans nationwide, all at once, while the longer
    hospital by hospital implementations progress. I have attached to this
    testimony sample screenshots of the new application. We look forward to
    working with VA and DoD to obtain their thoughts around rolling out
    some or all of these features in the coming months.
        Nonetheless we will continue to invest in new technologies that we
    believe can assist our nation's veterans in short order, regardless of
    where their local VA center is on the system rollout schedule.
        Training: Modern applications should require little to no training.
    Certainly, an EHR system has a level of complexity and medical
    specificity that will require some training, but our goal is to make
    this system as easy to use as anything else you do online. The best way
    to succeed is to win over users with user interfaces that are intuitive
    and functionality that exceeds practitioner's needs and expectations.
    When we do that, we believe we will create greater user satisfaction
    and combat inertia for acceptance of the new system.
        Over time we will achieve that goal, especially when we eventually
    move Millennium to a modern stateless web application. But in the
    short-term, we recognize that training must be improved for users new
    to the Millennium system. Unless a VA provider recently joined VA from
    another healthcare system that used Millennium or another commercial
    product, VA providers are used to VistA and unfamiliar with Millennium.
    This is of course natural--VistA has been in use for several decades,
    so VA users know its tricks, shortcuts and workflows.
        Similarly, it is important to understand that Oracle Cerner EHR's
    are deployed in tens of thousands of healthcare facilities across the
    country without incident.
        We understand the challenge that change presents and the answer is
    two-fold. First, as I mentioned making the system more intuitive will
    help. Second, we are working with VA to revamp training. We recently
    engaged a third-party firm to evaluate the current training program. We
    have preliminary results and expect final recommendations in the coming
    two to 3 months. We will be engaging Accenture to implement the needed
    changes and work with us to make the training much more efficient,
    applicable and useful.
        We know that nobody wants to sit through hours of training to learn
    a new system, so we will make it more targeted and impactful--and in so
    doing hopefully create better momentum and inertia for user adoption at
    rollout sites in 2023 than we have seen in the past.
        Transparency: Another issue that has clearly been a problem is the
    dissemination of timely and accurate information, whether positive or
    negative. That has led to increased oversight by Congress and by the VA
    OIG, both of which we welcome. I of course commit to continue to come
    before this Committee and to work with you as often as is necessary,
    but to begin this work, we recently launched an electronic dashboard
    that we will make available to all of you which catalogues our ``to
    do'' list and progress being made. We hope this dashboard will
    supplement VA's monthly reporting, assist in the Committee's tracking
    and oversight, and keep everyone focused on deliverables and dates.
        Timing: With new site deployments set for next year, we have the
    time to make fixes and updates to the system related to stability and
    performance as I discussed earlier. We expect to be ready to pick up
    with a full deployment schedule next year as set forth by VA and DoD,
    including at major facilities in Seattle, Portland, and other cities.
    By the end of next year DoD will be fully deployed, and we anticipate
    VA will have nearly 30 additional facilities live on the new EHR.
        Under the original deployment schedule, more facilities would be
    using Millennium by the end of 2023, and we acknowledge that the
    rollout is behind. The pandemic obviously caused some of this delay and
    presented challenges with the initial rollout in Spokane that might
    otherwise have been avoided. But, as is often said, we are where we
    are--so from our perspective we look at how we can gain efficiencies in
    our deployment methodology to get back on a course to deploy more
    efficiently, and we look forward to working with VA on a full master
    schedule.
        We believe that the combination of steps I have described--improved
    system performance and stability, design and workflow fixes, enhanced
    functionality, revamped training and more--will put us in a position to
    accelerate deployments in 2024 and beyond. The number one thing we can
    do is make a system that VA providers want--and are willing to learn
    and adopt. That happens with a modern system that makes work easier and
    where tangible benefits are seen for patients.
        We know we have a lot to prove with deployments next year at larger
    more complex sites. We view the next year as a key window for building
    momentum and turning the corner, just as we have done at DoD where
    initial adoption was similarly challenging and is now proceeding apace
    and with strong results.
        Cost: The Inspector General has written that it believes there will
    be cost overruns, and we have seen the Institute for Defense Analyses
    (IDA) preliminary cost estimates for the next 28 years of lifecycle of
    the EHRM system. While I am not able to critique the IDA report one way
    or another, I will make four points.
        First, as for Oracle's part, we intend to deliver the EHR system
    across the entire VA for the amounts contemplated in the current
    contract under the current scope. This Committee rightfully focuses on
    the cost of the EHRM program. We recognize that our portion of the
    program is the lion's share of the budget, and we take seriously our
    responsibility to be good stewards of these taxpayer dollars.
        As such, we intend to deliver even more than was ever originally
    contemplated as we upgrade Millennium to our data center and modernize
    it to a Stateless web application. Those upgrades will be done at our
    cost, not the government's.
        Of course, if there are big new pieces of functionality not
    included in the current scope of the contract, that's a different
    discussion. However, if there are significant cost overruns related to
    the current contract, we are prepared to bear those costs and remain
    within the existing budget envelope. So let's talk dollars and cents,
    this is a 10-year, $10 billion contract that already has had $700
    million or so in additional funding. That's $10.7 billion. Even with
    delays and perhaps an extended timeline for deployment, that's the
    number we intend to deliver for--again, barring some big new piece of
    functionality being added that is not currently contemplated in the
    contract.
        Second, having been in this industry for almost 30 years, I am
    unaware of any point in history where the cost of technology has gone
    up, not down, nor am I sure I can predict the State or the cost of
    technology 28 years from now. I would anecdotally point out that 15
    years ago mobile devices and cloud computing didn't even exist. Mobile
    computing and the cloud have turned the economics of technology upside
    down.
        Third, and again we have not seen the full final IDA report, but
    from the preliminary slides it does not appear that a cost comparison
    was done against the cost of maintaining or upgrading VistA for 28
    years. VistA has 130 different instances running using various VA-
    owned and maintained data centers. VistA is programmed in a language
    with a limited programmer-base where finding programmers in the coming
    decades will only become more difficult and expensive. As MITRE
    reported in 2015, VistA's ability to deliver new capabilities is
    stalled and is in danger of becoming obsolete. At the end of IDA's
    budget window, VistA would be 68 years old, would continue to fragment,
    and would have no viable path for modernization.
        It seems intuitive that a system conceived and developed in the
    1970s and 80s, by definition, would have a far greater lifecycle cost
    than a modern cloud EHR at scale developed in the 2020s. And once a
    modern cloud enterprise EHR is fully deployed, cost benefits from
    improved healthcare delivery will exist.
        Fourth, with Millennium, VA is moving to an enterprise approach
    with one system instead of operating 130 instances, and eventually this
    system will move to the cloud. IDA of course couldn't have known our
    plans here as it conducted its analysis, but we believe the appropriate
    baseline for this program is not the current Millennium system but the
    next generation cloud product we are developing and that will be the
    system running for the lifecycle.
        I can't contemplate a scenario where operating Millennium could
    cost more than VistA. In our experience, as technology improves, costs
    go down, and we believe we can compress costs in the coming years.
    Moving to a cloud-based system by definition will reduce the cost of
    maintenance dedicated to physical infrastructure for the Federal EHR.
    It also puts the onus--and staffing requirements--on Oracle as we will
    be responsible for running the data centers, providing updates and
    security patches and making upgrades for capacity needs freeing VA
    employees from many of those tasks in their own data centers.
                                  conclusions
        In conclusion, 4 months into owning Cerner, we are confident that
    the problems with the VA rollout can be fixed in a relatively short
    amount of time, and additionally that we can deliver a far superior and
    modern Federal EHR on an aggressive time schedule as part of our
    existing contract with the government. We have committed to providing
    the Committee with full transparency as we move forward, including with
    the recent launch of our dashboard. And we are dedicated to providing
    whatever resources are necessary to deliver to both DoD and VA a system
    that exceeds expectations without exceeding the contracted cost.
        Oracle is excited to be VA and DoD's new partner in this one-of-
    kind, transformational EHR modernization effort. We are confident that
    our energy, commitment and resources will benefit this program greatly.
    With a little time, we can deliver for all the veterans who served our
    Nation and deserve nothing but the best, as well as for our current
    service members who will one day be a part of our veteran community.
        We hope you will support us in this endeavor and look forward to
    working with the Committee as we move forward. I look forward to your
    questions. Thank you.

        Attachments:

      --September 2, 2022 letter from Oracle Cerner to VA

      --Screenshots of Unknown Queue updates

      --Screenshots of new patient facing application

    [GRAPHICS NOT AVAILABLE IN TIFF FORMAT]

        Senator Heinrich. Dr. Rieksts.
    STATEMENT OF DR. BRIAN RIEKSTS, PH.D., RESEARCH STAFF
                MEMBER, COST ANALYSIS AND RESEARCH
                DIVISION, INSTITUTE FOR DEFENSE ANALYSES
        Dr. Rieksts. Chairman Heinrich, Ranking Member Boozman, and
    Senator Tester, thank you for the opportunity to testify today.
        The Department of Veterans Affairs asked the Institute for
    Defense Analyses to provide an independent, life cycle cost
    estimate for the Electronic Health Record Modernization
    program. Following GAO and OIG guidance we included all program
    costs regardless of funding source. IDA estimates the life-
    cycle cost of the EHRM program to be about $50 billion in
    constant fiscal year 2022 dollars.
        These costs would span about 28 years, which includes a 13-
    year implementation period, and a 15-year period after the
    system is fully deployed. IDA also conducted a risk analysis to
    quantify the uncertainty of the cost estimate. We estimated a
    likely range of $46- to $54 billion.
        There are notable differences between the IDA cost estimate
    and the 2019 VA estimate of $16 billion. IDA estimates cost for
    a 28-year life cycle, whereas the VA estimate covers only the
    10-year time period of the current contract. The VA estimate
    does not include sustainment costs or the cost of productivity
    loss due to deploying the system, both of which are included in
    the IDA estimate.
        Our tasking was to produce this independent estimate of VA
    EHRM life-cycle costs. Estimating potential benefits such as
    improved health care delivery and possible legacy system cost
    reductions was beyond the scope of our study. Our analysis
    estimates cost ranges for EHRM cost drivers, VA has an
    opportunity to manage cost drivers such as productivity loss
    associated with deployments.
        As the system is rolled out to more facilities, emerging
    information will provide additional insights regarding risk and
    uncertainty in the cost estimate.
        Thank you for the opportunity to discuss your cost
    estimate. And I look forward to your questions.
        [The statement follows:]
                    Prepared Statement of Dr. Brian Rieksts
        Chairman Heinrich, Ranking Member Boozman, and distinguished
    Members of the Committee: Thank you for the opportunity to testify
    today.
        The Department of Veterans Affairs (VA) announced in 2017 that it
    would pursue an Electronic Health Record Modernization (EHRM) program
    to replace elements of its current aging system, the Veterans Health
    Information System and Technology Architecture (VistA). In 2022, VA
    asked the Institute for Defense Analyses (IDA) to develop an
    independent life-cycle cost estimate (LCCE) for the EHRM program.
        IDA produced an independent life-cycle estimate for all of the
    costs attributable to the EHRM program-that is, costs that are incurred
    only due to the existence of the EHRM program. IDA's cost estimate
    includes all costs of the program over its full life cycle, from
    program inception through design, development, deployment, operations
    and maintenance, and disposal. Thus, all relevant acquisition and
    sustainment costs are included in the LCCE. The IDA LCCE covers the
    implementation phase and 15 years of operations after the system is
    fully deployed to all sites.
        It is worth noting some items not in scope for the IDA independent
    cost estimate. One main item is benefits. The EHRM program has many
    potential benefits (e.g., improved healthcare delivery and reduced
    costs for legacy systems). The estimated benefits of the program,
    however, are out of scope for this study. Moreover, IDA's cost estimate
    is solely for the EHRM program. It does not include the cost of (or
    savings from) legacy electronic health record (EHR) systems.
                                    results
        IDA estimates the EHRM LCCE to be $49.8 billion (in constant Fiscal
    Year 2022 dollars). The conversion to constant dollars normalizes
    inflation to 2022 levels; actual future expenditures will be higher.
    Overall, this estimate consists of $32.7 billion during the
    implementation phase over 13 years and an additional $17.1 billion in
    sustainment costs over the following 15 years.
        The following figure shows acquisition and sustainment costs for
    the LCCE during the implementation and fully deployed phases. We note
    that some sustainment costs are required during the implementation
    phase as sites start to use the new system.
        We estimate an additional $5.2 billion of common infrastructure
    costs in addition to the LCCE. These costs would be needed without the
    EHRM program, but EHRM also requires these investments.
                             cost estimate by phase
        IDA provided a point estimate representing the 50 percent risk-
    adjusted cost. That is, the cost will be less than the point estimate
    with a likelihood of 50 percent. IDA also conducted a risk analysis to
    quantify the risk and uncertainty, estimating a range of $46-$54
    billion for the 20-80 percent cost range.
                                   comparison
        We compared the IDA estimate to the VA 2019 program office estimate
    of $16.1 billion (represented in constant Fiscal Year 2022 dollars). We
    note several differences in scope and comparable elements between the
    VA and IDA estimates. First, the VA estimate is for 10 years of
    implementation, whereas IDA's life-cycle estimate spans 28 years.
    Second, the IDA team includes sustainment and productivity loss during
    rollout as additional cost elements in its LCCE, in accordance with
    Government Accountability Office (GAO) and VA Office of Inspector
    General (OIG) guidance to include all costs, but these elements are not
    included in the VA estimate. The productivity loss cost corresponds to
    the supplemental staffing and additional community care for Veterans
    needed to mitigate the disruption in care when rolling out the system
    to facilities. The revenue loss from the disruption is also included.
    The following figure shows the costs for the VA estimate, the
    comparable costs for the IDA estimate, and the costs of additional
    elements in the IDA estimate. IDA's LCCE includes an additional $25.9
    billion beyond the scope of the VA estimate. These additional elements
    account for about 75 percent of the cost difference from the VA's
    estimate ($16.1 billion) to IDA's estimate ($49.8 billion).
                       comparison of va and ida estimates
        The costs for comparable elements differ by $7.8 billion. About
    $5.1 billion of this difference is implied by actual costs from the
    first 4 years of the program. The factors contributing to the remaining
    difference are an IDA-estimated 3-year schedule adjustment based on
    experience in other programs and additional cost risks (e.g.,
    additional development). We also note that the IDA LCCE includes $1.2
    billion less in infrastructure than the VA's estimate because IDA
    defines some costs as common infrastructure.
        Cost increases are common in programs of this complexity.
    Enterprise resource planning programs have typically had similar cost
    increases in acquisition.
                                  path forward
        Our analysis estimates the cost ranges for risks associated with
    EHRM. VA has an opportunity to mitigate some of the risks associated
    with the cost drivers. For example, productivity loss associated with
    deployments is a cost element with substantial risk. VA can manage this
    risk and seek ways to improve the efficiency of deployments.
        Going forward, information emerging from upcoming deployments will
    provide additional insights regarding risk and uncertainty in the cost
    estimate. Developing measures to assess progress in key risk areas will
    allow VA leadership to monitor uncertainty and understand the
    implication for the total life-cycle cost of this program.
        Thank you for the opportunity to discuss our cost estimate, and I
    look forward to your questions.

        Senator Heinrich. Thank You. Mr. Case.
    STATEMENT OF MR. DAVID CASE, DEPUTY INSPECTOR GENERAL,
                OFFICE OF INSPECTOR GENERAL, DEPARTMENT OF
                VETERANS AFFAIRS
        Mr. Case. Chairman Heinrich, Ranking Member Boozman, and
    Subcommittee Members, thank you for the opportunity to discuss
    the Office of Inspector General's Oversight of the VA's
    Electronic Health Record Modernization program.
        First, we want to recognize the VA employees working so
    hard in Washington, Ohio, Oregon, and across the Nation to
    ensure veterans receive timely, high-quality health care during
    the EHR transition, especially in a pandemic.
        Since April 2020, we have issued 14 EHRM reports primarily
    focused on planning, user training, and deployment activities
    at Mann-Grandstaff. They are meant to help VA leaders redress
    and avoid identified system failings in future rollouts. While
    VA has implemented 26 of our 68 total recommendations as of
    this week, considerable work remains.
        This year, we published reports related to significant
    patient safety risks at Mann-Grandstaff, including issues with
    the unknown queue, medication management, patient care
    coordination, the troubled ticket process, and gaps in
    actionable quality of care metrics. We are also aware of
    continued issues with behavioral health services and the
    pharmacy management software. We remain concerned about the
    many workarounds and mitigations that VHA employees must use to
    address these issues. They can lead to delays, increased
    errors, and affect the quality of patient care.
        Accordingly, our subject matter experts are continuously
    monitoring VA progress on OIG-recommended corrective actions
    meant to address many of these concerns.
        I want to highlight two audits we published last year.
    Because VA needs to spend billions of dollars on fiscal and IT-
    related infrastructure upgrades to support the EHR, we audited
    VA's related cost estimates following our determination that VA
    did not meet its own deadline for infrastructure upgrades at
    the pilot site.
        The cost estimates were unreliable. In general, they were
    not comprehensive, well documented, accurate, or credible. Our
    audit teams also determined VA had not reported accurate and
    complete information to Congress in the Eight Congressional
    Reports submitted through January 2021. OEHRM personnel did not
    report the estimated $2.7 billion for physical infrastructure
    upgrades, and the estimated $2.5 billion for IT infrastructure
    upgrades.
        They explained that because VHA and OIT were funding these
    upgrades, OEHRM did not need to report them. Despite VA and GAO
    guidance requiring life-cycle cost estimates to include all
    costs, regardless of funding source, these omissions
    significantly understate the program's true costs.
    Recommendations from the two audits include that VA obtain an
    independent cost estimate of the program's life cycle, and
    ensure transparency in reporting costs to Congress. These
    recommendations remain open.
        We have been briefed by the Institute for Defense Analyses
    on their draft independent cost estimate and look forward to
    reviewing the final report's methodology, findings, and
    estimates before determining how they relate to our outstanding
    recommendations.
        In April 2022, we also found VA had not executed a reliable
    and comprehensive integrated master schedule. They failed to
    meet the standards they adopted that called for a schedule that
    is comprehensive, credible, well-constructed, and controlled.
    Among the actions needed to meet the standards for an
    integrated master schedule, VA must complete a schedule risk
    analysis and develop a critical path for completion of the
    work.
        Although not every task for a 10-year project can be
    accounted for early on, VA did not use accepted strategies to
    create a schedule that can be tailored over time. VA's failure
    to meet these standards increases the risk of further delays,
    dropped activities, and budget overruns. Without a reliable
    master schedule, the accuracy of any cost estimate is at risk
    because costs are inextricably linked to the schedule of
    activities VA needs to complete.
        In closing, we remain concerned with the 15 recommendations
    open for longer than 1 year. The success of the EHR
    implementation is dependent on VA's transparency, careful
    planning, and the recognition and remediation of patient care
    and safety concerns. Not only those risks identified by our
    oversight work, but by VHA's own experts and end users who rely
    on the EHR for everyday clinical decisionmaking.
        Chairman Heinrich, this concludes my statement. I would be
    happy to answer any questions you or other committee members
    may have.
        [The statement follows:]
                      Prepared Statement of Mr. David Case
        Chairman Heinrich, Ranking Member Boozman, and subcommittee
    members, thank you for the opportunity to discuss the Office of
    Inspector General's (OIG) oversight of the Department of Veterans
    Affairs' electronic health of record modernization (EHRM) program. The
    OIG recognizes the enormity and complexity of converting VA's
    electronic health record (EHR) system for millions of veterans
    receiving VA care and acknowledges the significant work and commitment
    of VA staff to accomplish this task. Over the more than 2 years that
    OIG staff have been engaging with employees at the first deployment
    site-the Mann-Grandstaff VA Medical Center (VAMC) in Spokane,
    Washington-and other VA locations using the new EHR, oversight teams
    have observed VA employees' unwavering commitment to this transition
    while prioritizing the care of patients during the COVID-19 pandemic.
    Facility staff challenges have been exacerbated, however, by the lack
    of prompt remediation of problems that the OIG and others have
    identified in numerous oversight reports published since April 2020.
        The OIG has published 14 reports addressing the EHRM program and
    system implementation between April 2020 and this hearing with a total
    of 68 recommendations. Though this statement does not detail all of
    these reports and their findings, a comprehensive list of
    recommendations has been included in the appendixes. Each oversight
    report is meant to help VA improve the new system's implementation and
    support the provision of prompt, quality health care for veterans.
    Failure to satisfactorily complete the corrective actions associated
    with these recommendations can increase risks to patient safety and the
    ability to provide high-caliber care as the new EHR system rolls out
    nationwide. Fully addressing oversight recommendations can help
    minimize considerable cost escalations and delays in future site
    deployments as well. The OIG is therefore concerned about the five
    recommendations that have been open (not implemented or fully
    addressed) for longer than 2 years-with 21 total recommendations open
    for more than 1 year. While the OIG follows up with VA on open
    recommendations every 90 days, VA program officials can submit evidence
    of sustained progress or satisfaction of corrective actions at any time
    to facilitate closing recommendations.
        As detailed in this statement, OIG staff have found VA did not
    complete timely critical infrastructure upgrades for the initial
    rollout and provided unreliable and incomplete estimates on
    infrastructure upgrade costs, has not adequately prepared for the
    rollouts (including realistic scheduling and effective user training),
    failed to be fully transparent, and stove-piped governance with
    decision-making that has not appropriately engaged Veterans Health
    Administration (VHA) end users of the new EHR system. Many of these
    issues are still of concern to the OIG, as evidenced by the number of
    open recommendations.
        This testimony highlights those OIG reports with findings that
    illustrate three broad categories of concern: (1) IT and physical
    infrastructure deficiencies and unreliable cost estimates for
    addressing them, (2) readiness concerns that include the lack of a
    comprehensive master schedule and ineffective training that was not
    transparently reported, and (3) implementation issues that affect
    patient care and safety as well as concerns with remediation and
    mitigation strategies.
           infrastructure deficiencies and unreliable cost estimates
        The OIG's oversight in April 2020 focused on VA's preparation for
    the system's initial deployment at the Mann-Grandstaff VAMC and the
    condition of VA's physical and information technology (IT)
    infrastructure prior to system deployment. Two 2021 reports (published
    in May and July) resulted from audits that examined cost estimates for
    needed physical and IT-related infrastructure upgrades nationwide. For
    the new EHR system to operate as intended, VHA facilities need these
    infrastructure upgrades, but they are generally funded from different
    sources. Because the life-cycle cost estimates for infrastructure
    upgrades did not account for costs from all VA components' budgets,
    some estimated costs were not included in mandated reports to Congress.
    Transparent and reliable cost estimates are critical for Congress to
    make informed budgeting decisions. VA senior leaders also depend on
    these cost estimates to plan program budgets, approve acquisitions, and
    monitor program execution. The OIG determined the existing physical and
    IT infrastructure was inadequate for the new system at initial
    deployment sites, and pertinent life-cycle cost estimates for
    infrastructure upgrades were unreliable and likely underreported by
    approximately $5 billion. These two reports recommended that VA obtain
    an independent cost estimate for the EHR program's life-cycle costs,
    which the VA is obtaining from the Institute for Defense Analyses. The
    OIG has been briefed on the draft and will review the final report's
    methodology, findings, and estimates before determining how they relate
    to outstanding recommendations.
        The 2020 OIG report focused on the gaps in VA's efforts to update
    the Mann-Grandstaff VAMC's physical and information technology (IT)
    infrastructure to support the new system.\1\ The OIG found that VA did
    not meet its own timelines to complete critical physical and IT
    infrastructure upgrades at the facility. The problems with planning
    identified in this report were shown in greater detail in the 2021 OIG
    reports that found deficient and unreliable physical and IT
    infrastructure cost estimates. Many of the recommendations to resolve
    these issues remain open.
    ---------------------------------------------------------------------------
        \1\ ``Physical infrastructure'' refers to the underlying foundation
    that supports the system, such as electrical; cabling; and heating,
    ventilation, and air-conditioning. ``IT infrastructure'' includes
    network components such as wide and local area networks, end-user
    devices (e.g., desktop and laptop computers, and monitors), and medical
    devices.
    ---------------------------------------------------------------------------
        deficiencies in infrastructure readiness for deploying va's new
                  electronic health record system (april 2020)
        To deliver patient care using the new EHR system, significant
    upgrades are needed to VA's physical and IT infrastructure.\2\ The OIG
    audited VA's infrastructure readiness activities at the Mann-Grandstaff
    VAMC in anticipation of the initial March 2020 go-live date. In 2019,
    then Office of Electronic Health Record Modernization (OEHRM) leaders
    testified before the House of Representatives that having
    infrastructure in place 6 months before deploying the Cerner system was
    a program goal to help ensure smooth deployment, but the OIG found they
    had not been completed at the facility even 5 months prior to the March
    2020 go-live.\3\ In fact, the OIG found some infrastructure upgrades
    intended to mitigate diminished system performance were not projected
    to be completed until months after going live. In sum, VA committed to
    an aggressive, but apparently unrealistic, deployment date of March
    2020 without having the necessary information about the facility's
    infrastructure.
    ---------------------------------------------------------------------------
        \2\ VA OIG, Deficiencies in Infrastructure Readiness for Deploying
    VA's New Electronic Health Record System, April 27, 2020.
        \3\ In 2021, VA transitioned EHRM program management from the
    Office of Electronic Health Record Modernization (OEHRM) to the EHRM
    Integration Office (EHRM IO). EHRM IO now has responsibility for all
    recommendations assigned to OEHRM. Cerner Corporation was acquired by
    Oracle Corporation on June 7, 2022; this statement will refer to the
    entity as ``Cerner,'' as it was referred to at the time of the reviews
    discussed in this statement.
    ---------------------------------------------------------------------------
        The OIG made seven recommendations for corrective action to the
    then executive director of OEHRM, and an eighth recommendation to the
    Mann-Grandstaff VAMC director. These recommendations, of which two
    remain open as not implemented, can be found in appendix A of this
    statement. Given the time elapsed since this report's publication, it
    is concerning that one of the open recommendations calls on OEHRM to
    evaluate physical infrastructure for consistency with its program's
    requirements and monitor those evaluations.
        deficiencies in reporting reliable physical infrastructure cost
                   estimates for the ehrm program (may 2021)
        This audit was conducted to determine if VA developed and reported
    reliable physical infrastructure upgrade cost estimates for the new EHR
    system.\4\ As discussed previously, VHA medical facilities need
    significant physical infrastructure upgrades, such as electrical work,
    cabling, heating, ventilation, and cooling to successfully deploy the
    new EHR system. The audit examined whether VHA's cost estimates met VA
    standards and were comprehensive, well documented, accurate, and
    credible. It also reviewed whether OEHRM reported these cost estimates
    to Congress in accordance with statutory mandates.
    ---------------------------------------------------------------------------
        \4\ VA OIG, Deficiencies in Reporting Reliable Physical
    Infrastructure Cost Estimates for the Electronic Health Record
    Modernization Program, May 25, 2021.
    ---------------------------------------------------------------------------
        VHA and OEHRM shared responsibilities for estimating and reporting
    physical infrastructure upgrade costs. VHA developed the physical
    infrastructure upgrade cost estimates, while OEHRM was responsible for
    reporting all program life-cycle cost estimates to Congress in
    accordance with the Veterans Benefits and Transition Act of 2018.\5\ In
    May 2021, the act required quarterly reporting on the EHRM program's
    status, including annual and life-cycle cost estimates and defined the
    program as any activities to procure or implement the new EHR system.
    In early 2019, VA's Office of General Counsel determined that physical
    infrastructure upgrades must be funded from accounts specifically
    available for construction-type purposes, such as VHA's nonrecurring
    maintenance and minor construction funds.
    ---------------------------------------------------------------------------
        \5\ The law was signed on December 31, 2018, and it became Public
    Law 115-407.
    ---------------------------------------------------------------------------
       vha cost estimates for physical infrastructure upgrades needed in
                 support of the ehrm program were not reliable
        The OIG found VHA's cost estimates were not reliable under VA
    standards and Government Accountability Office (GAO) guidance.\6\ These
    standards and guidance state that cost estimates should be
    comprehensive, well documented, accurate, and credible. However,
    neither of VHA's formal cost estimates for physical infrastructure,
    dated June 2019 ($2.7 billion) and November 2019 ($1.1 billion), fully
    met these criteria, and thus could be significantly understated. In
    addition, VA lacked effective quality controls and procedures to
    evaluate the estimates and had conducted insufficient planning from the
    start.
    ---------------------------------------------------------------------------
        \6\ VA Cost Estimating Guide, ver. 2.2, August 17, 2016; GAO, Cost
    Estimating and Assessment Guide, GAO-20-195G, March 2020.
    ---------------------------------------------------------------------------
    1. Cost Estimates Were Not Comprehensive
        Comprehensive cost estimates provide officials with reasonable
    assurance that all costs are included so they can make well-informed
    decisions. VHA's November 2019 estimate, totaling about $1.1 billion
    for physical infrastructure upgrades nationally, only reflected about
    25 percent of nationwide cabling costs, understating the costs by at
    least $481 million. Also, the June and November 2019 estimates omitted
    estimated costs of upgrades paid with minor construction funds.
    2. Cost Estimates Were Not Well Documented
        Sufficient documentation supports an estimate's validity and
    provides an audit trail allowing the estimate to be easily recreated
    and updated. Both June and November estimates lacked evidence they were
    approved by senior leaders, and they did not have enough detail to
    allow an independent party to trace the costs or determine if costs
    were double-counted.
    3. Cost Estimates Were Not Accurate
        Neither cost estimate met the standard for accuracy-that is, free
    of mathematical errors and not overly conservative or optimistic. The
    June 2019 estimate had errors omitting about $90 million of fiscal year
    2021 construction design costs. The November 2019 estimate omitted
    escalation costs for upgrades expected to take place in future years
    and did not include the cost of completely upgrading the cabling
    required at VHA facilities nationwide.
    4. Cost Estimates Were Not Credible
        Credible cost estimates identify limitations of the data and
    assumptions and are to be measured against independent or third-party
    cost estimates. Both estimates lacked a risk and uncertainty analysis,
    which is used to disclose the likelihood actual costs may differ from
    estimated costs. VHA did not conduct this type of analysis because VA
    did not have accurate assessments of what infrastructure upgrades were
    needed at its facilities. Both estimates also lacked a sensitivity
    analysis, which is used to explain how much impact each cost factor has
    on the overall estimate. Both cost estimates were also not compared to
    a third-party cost estimate, a best practice in validating the
    reliability and reasonableness of cost estimates. Using the planned and
    obligated costs at VA's three planned initial operating capability
    sites, the OIG team statistically projected program-wide physical
    infrastructure costs to be between approximately $3.1 and $3.7
    billion.\7\ Notably, VHA's June 2020 estimate projects physical
    infrastructure upgrade costs to be about $3.1 billion, consistent with
    the OIG team's low-end projection.
    ---------------------------------------------------------------------------
        \7\ The three facilities were the Seattle, American Lake, and Mann-
    Grandstaff VAMCs, all located in Washington State.
    ---------------------------------------------------------------------------
    5. Lack of Effective Quality Controls and Procedures to Evaluate
            Estimates
        Deficient quality controls contributed to the unreliability of both
    cost estimates. Independent cost estimates-a control used to validate
    the data and determine the reasonableness of a VA estimate-are required
    by VA policy to be performed on all major IT programs, but an
    independent cost estimate was not performed on either estimate.
    6. Insufficient Planning at the Program's Start
        Consistent with findings from the April 2020 OIG report, the audit
    team found neither OEHRM nor VHA knew the true state of infrastructure
    at facilities at the time the Cerner contract was signed, and, when
    this audit was completed in March 2021, VHA was still identifying
    necessary infrastructure upgrades. As of January 2021, infrastructure
    requirements continue to be defined, making it difficult for VHA to
    identify gaps in infrastructure and estimate related costs.
          oehrm did not include cost estimates for upgrading physical
                     infrastructure in reports to congress
        The OIG found that OEHRM did not include the cost of physical
    infrastructure upgrades in quarterly reports to Congress, which are
    intended to meet the program's requirements under the Veterans Benefits
    and Transition Act.\8\ This is significant, as it understated the
    program's cost in reports submitted to Congress. The reports gave the
    impression that these costs were included because seven of the eight
    reports said that infrastructure costs include ``physical
    infrastructure at VA medical centers and other sites.'' To the
    contrary, these reports did not include the $2.7 billion for physical
    infrastructure upgrades as identified in the June 2019 estimate OEHRM
    received from VHA. OEHRM said it did not disclose these estimates
    because the upgrades were outside its funding responsibility, but this
    is contrary to the explicit requirements of statute and to VA and GAO
    guidance that a life-cycle cost estimate include all costs, regardless
    of source.\9\ The VA Electronic Health Record Transparency Act of 2021
    modified VA's reporting requirement to mandate the inclusion of costs
    expended by any VA element.\10\
    ---------------------------------------------------------------------------
        \8\ OEHRM produced its ninth report after the OIG report was
    drafted and did not include physical infrastructure upgrade costs in
    that document.
        \9\ The Veterans Benefits and Transition Act of 2018 defines the
    EHRM program as ``any activities ... to procure or implement an
    electronic health or medical record system to replace'' the existing
    electronic health record system and ``any contracts or agreements
    entered into by [VA] to carry out, support, or analyze'' these
    activities. Because physical infrastructure upgrades are necessary for
    system implementation, those costs should be included in life-cycle
    cost estimates under the statute's plain language.
        \10\ The law was signed on June 23, 2022 and became Public Law 117-
    154.
    ---------------------------------------------------------------------------
        The OIG made five recommendations to VA, which can be found in
    appendix B. Three of the recommendations, which pertain to the need for
    an independent cost estimate of the program's life cycle and ensuring
    transparency in reporting costs to Congress, remain open. As previously
    mentioned, the OIG has been briefed by the Institute for Defense
    Analyses on their draft independent cost estimate and looks forward to
    receiving the final report for review.
    unreliable it infrastructure cost estimates for the ehrm program (july
                                     2021)
        Of EHRM's estimated $16.1 billion total program cost from 2021, VA
    estimated about $4.3 billion would be directed for IT infrastructure
    upgrades.\11\ This audit examined whether OEHRM-developed cost
    estimates were well-documented, comprehensive, credible, and accurate,
    and whether OEHRM reported to Congress all IT infrastructure upgrade
    costs, including future technology updates.\12\
    ---------------------------------------------------------------------------
        \11\ VA OIG, Unreliable Information Technology Infrastructure Cost
    Estimates for the Electronic Health Record Modernization Program, July
    7, 2021.
        \12\ Technology refreshment is the process of replacing certain
    infrastructure on a regular schedule, instead of using the systems or
    devices until they can no longer function. For example, devices like
    laptops are replaced every 4 years.
    ---------------------------------------------------------------------------
    IT Infrastructure Upgrade Cost Estimates Were Not Reliable but
            Improvements Have Been Made
        As discussed previously, reliable estimates should be well-
    documented, comprehensive, credible, and accurate. The audit team
    evaluated two estimates OEHRM provided to Congress dated December 2018
    and August 2020-each estimating about $4.3 billion for the IT
    infrastructure upgrades. Neither met the reliability criteria, and the
    OIG could not evaluate their accuracy because they lacked documentation
    to support many of the calculations. Like the physical infrastructure
    cost audit, VA did not complete an independent cost estimate, which
    could have revealed the OIG-identified issues sooner.
        In January 2021, in part due to discussions with the audit team,
    OEHRM began developing procedures that align with cost-estimating
    guidance and include controls to help address the issues identified in
    the OIG report. During the audit, the team noted that VA also began
    making improvements to the cost model used to develop the estimate,
    facilitating more detailed support.
    IT Infrastructure Costs Were Omitted and Not Updated for Accuracy
        The OIG found OEHRM did not include costs for critical program-
    related IT infrastructure upgrades in the estimates reported to
    Congress, effectively underreporting program cost estimates by nearly
    $2.5 billion. The $2.5 billion is for IT infrastructure upgrades that
    VA's Office of Information and Technology (OIT) and VHA are expected to
    fund.\13\ Like the physical infrastructure costs, OEHRM officials
    stated they felt the omitted costs were outside their scope of
    responsibility, but neither OIT nor VHA reported these costs to
    Congress, despite VA and GAO guidance requiring life-cycle cost
    estimates to include all costs, regardless of source. The costs should
    have been disclosed by OEHRM. VA did make changes to projected costs
    starting in the November 2021 report to Congress, but because VA was
    still developing the independent cost estimate, there was no certainty
    the updates were reliable.
    ---------------------------------------------------------------------------
        \13\ OIT is expected to fund some upgrades for the local area
    network, end-user devices, phones, and Wi-Fi, while VHA is expected to
    fund upgrades mostly for medical devices.
    ---------------------------------------------------------------------------
        Without all critical IT infrastructure upgrade costs accurately
    presented, Congress lacks the comprehensive picture of total program
    costs needed to make informed oversight and investment decisions. As
    mentioned previously, VA's reporting requirements have been updated by
    the VA Electronic Health Record Transparency Act of 2021.
        All six recommendations to the executive director of OEHRM are
    listed in appendix C and remain open. The recommendations relate to
    obtaining independent cost estimates for IT infrastructure, ensuring
    the costs are estimated in line with VA policy, maintaining full and
    complete accounting for the costs, and ensuring complete and updated
    transparency of the costs with Congress.
       lack of readiness exhibited by no integrated master schedule and
                              ineffective training
        Exploring program costs and projections further, the OIG reported
    in April 2022 that VA had not executed a reliable, comprehensive
    schedule for system implementation. This could result in schedule
    delays and leave VA vulnerable to billions of dollars in cost overruns.
    Without that schedule, Congress and the public cannot rely on VA
    timeline projections for completing the work or be assured that the
    program will be completed within budget.
        The OIG also examined the flawed implementation at Mann-Grandstaff
    VAMC that was brought on by inadequate planning. Deficiencies the OIG
    detected at Mann-Grandstaff VAMC in April 2020 revealed the need for
    prompt corrective measures as additional facilities were switching to
    the new EHR system. Yet many issues remained unresolved prior to
    deployment, particularly problems identified in the OIG's July 2021
    report on the development, delivery, and assessment of staff training
    and proficiency.
     the ehrm program did not fully meet the standards for a high-quality,
                      reliableschedule (april 2022 report)
        To implement the program successfully and within budget, it is
    imperative that VA develop a reliable integrated master schedule
    (IMS).\14\ GAO guidance, which OEHRM adopted, States that a high-
    quality, reliable schedule should be comprehensive, credible, well-
    constructed, and controlled. The IMS is designed to cover the entire
    required scope of work-of both government staff and contractors-needed
    to complete the program. VA should use it as a road map to monitor
    progress, complete the work, identify potential problems and track
    their resolution, and promote accountability. While not every task for
    a 10-year project can be accounted for early on, strategies exist to
    create a tailorable, comprehensive schedule to minimize the risk of
    delays, dropped activities (some of which are prerequisites for
    others), and budget overruns. While VA may have received a draft
    independent cost estimate for the program since the OIG's two audits,
    without a reliable IMS, the developed cost estimates' accuracy are at
    risk because they are inextricably linked to the schedule of activities
    VA that needs to complete.
    ---------------------------------------------------------------------------
        \14\ VA OIG, The Electronic Health Record Modernization Program Did
    Not Fully Meet the Standards for a High-Quality, Reliable Schedule,
    April 25, 2022.
    ---------------------------------------------------------------------------
    VA Did Not Have a High-Quality, Reliable IMS
        The OIG found that neither the overall IMS nor five of its
    underlying individual project schedules fully met GAO standards adopted
    by OEHRM for a high-quality, reliable schedule. VA failed to meet all
    aspects of the following scheduling standards:

      --Comprehensive. The IMS should reflect the entire scope of program
            work in some level of detail. However, the OIG determined that
            the IMS did not capture all work for the program's duration and
            was missing VHA and OIT activities.

      --Credible. A credible IMS should include a complete schedule risk
            analysis, which can give a level of confidence in meeting a
            program's completion date. However, OEHRM did not do this.

      --Well-constructed. A ``critical path'' determines the earliest date
            a program can be completed to help managers examine the effects
            of activity slippages, but no overall IMS critical path was
            created.

      --Controlled. A controlled IMS should include a baseline schedule,
            used for managing the program and conducting trend analyses
            over time to assess program performance. However, OEHRM's
            program baseline only covered events through April 2020.

        The OIG identified several root causes for OEHRM's failures:

      --Did not adequately coordinate with various offices. VHA and OIT
            leaders said OEHRM did not collaborate with them, so the
            schedules did not include all work to be performed by these
            entities.

      --Did not conduct a schedule risk analysis because it lacked
            procedures. Despite the importance of completing this analysis,
            OEHRM did not have procedures in place on when and how to
            conduct it.

      --Focused on near-term deployment of the system at the initial
            operating sites. OEHRM only required development of site-
            specific schedules after task orders for those sites were
            awarded. Applying that strategy, VA would not have a high-
            quality, reliable IMS until it starts deploying the system at
            the last sites, which are planned to go live in fiscal Year
            2028.

      --Did not enforce its own scheduling standards or have tools in place
            to assess compliance. While OEHRM's schedule management plan
            stresses compliance with GAO guidance, task orders to Cerner do
            not require the IMS to align with them. Additionally, OEHRM's
            schedule management plan requires staff to use specific
            software to assess whether EHRM project schedules comply with
            GAO standards. However, a tool was not available from March
            2020 to June 2021.

      --Lacked consistent guidance on roles, resulting in confusion over
            the assignment of IMS development and documenting how work was
            broken down. Internal planning and contract documents
            inconsistently assigned responsibilities for developing and
            maintaining the program's work breakdown structure (WBS) and
            the IMS. The WBS defines all work needed to complete the
            program. Guidance inconsistently assigned these
            responsibilities to VA or one of its contractors-

      --Booz Allen Hamilton, Inc., or Cerner, leading to confusion.\15\
            Cerner accepted responsibility for the WBS and, in July 2020,
            worked with VA to create it. While Cerner is responsible for
            developing the IMS, VA should ensure contract requirements are
            consistent with internal guidance.
    ---------------------------------------------------------------------------
        \15\ Booz Allen Hamilton, Inc. staff support EHRM activities. Their
    work included gathering input from VA administrations or offices to
    develop schedules for VA activities.

      --Did not clearly define IMS contract requirements. Cerner was
            contractually required to develop and maintain an IMS for the
            program under VA's task orders; however, the task orders did
            not clearly establish a timeline for when a complete IMS would
            be developed. Without a clear timeline, OEHRM required Cerner
            to develop site-specific project schedules as task orders were
            awarded. Following this process, future work not yet on task
    ---------------------------------------------------------------------------
            order would be unaccounted for in the IMS.

        VA has a responsibility to ensure there is a complete IMS that
    meets scheduling standards. VA needs a high-quality, reliable IMS to
    strengthen the credibility of the program's timeline. Without one, VA
    can neither demonstrate how slippages will affect the overall timeline
    nor assure stakeholders that the reported timeline is realistic and
    achievable. Any schedule delays that extend the program beyond 10 years
    are also likely to result in billions of dollars in cost overruns. The
    OIG estimated the average cost per year of a schedule delay is
    potentially about $1.95 billion.
        For this report, the OIG made six recommendations, found in
    appendix D, and all are open.
     training deficiencies for va's new ehr system at the mann-grandstaff
                            vamc (july 2021 report)
        The OIG reviewed the training given to Mann-Grandstaff VAMC
    staff.\16\ Problems were identified similar to those found by
    Department of Defense (DoD) for training on the new EHR system. Even
    before deployment, the healthcare inspection team identified governance
    challenges as VHA did not have a defined role in decision-making or
    oversight related to training activities. In reviewing the training,
    the OIG found training content, delivery, and assessment failures.
    ---------------------------------------------------------------------------
        \16\ VA OIG, Training Deficiencies with VA's New Electronic Health
    Record System at the Mann-Grandstaff VA Medical Center in Spokane,
    Washington, July 8, 2021.
    ---------------------------------------------------------------------------
        The inspection team reviewed the training content on the software
    and the more than 900 new workflows. New workflows result in changes to
    how end users perform their jobs, such as scheduling consults
    (referrals) or how a provider performs an exam. The OIG found the
    classroom training and supplemental material were insufficient.
    Facility leaders and staff told the OIG that training did not prepare
    them for going live with the new system, teach them how to apply what
    they learned to their work, or explain the meaning behind the process
    of which buttons to push (``buttonology'').
        The OIG identified four aspects of training delivery that may have
    negatively affected the new EHR system's use: (1) insufficient time for
    training, (2) limitations with the training domain (a close facsimile
    for users' practice), (3) challenges with user role assignments (these
    dictate the capabilities on which an employee is trained), and (4) gaps
    in training support. Facility leaders and staff raised concerns with
    Cerner classroom trainers, including their lack of clinical knowledge,
    EHR expertise, and an inability to address questions.
        Finally, the OIG found OEHRM failed to effectively evaluate the
    training. The OIG conducted a follow- up administrative investigation
    into the inaccurate and incomplete data OEHRM provided about trainees'
    post-training tests after OIG staff requested ``any and all data'' from
    the training evaluation plan that OEHRM's Change Management leaders
    submitted.\17\ While the investigation did not find that the two Change
    Management leaders intentionally sought to mislead OIG healthcare
    inspectors, their lack of due care and diligence resulted in inaccurate
    information being submitted to OIG staff. Most concerning, the Change
    Management's then executive director and the director for training
    strategy did not disclose they removed some data from consideration or
    that they questioned data reliability. They delayed production of
    underlying proficiency check data and instead provided one slide with
    three summary statistics containing significant errors that resulted in
    doubling the reported trainee proficiency check pass rate from 44 to 89
    percent. In addition, officials admitted the evaluation plan was
    actually ``immature'' and ``in its infancy'' and was not implemented,
    contrary to the evaluation plan submitted to the OIG that showed
    training was being assessed immediately after it was completed by
    employees.
    ---------------------------------------------------------------------------
        \17\ VA OIG, Senior Staff Gave Inaccurate Information to OIG
    Reviewers of Electronic Health Record Training, July 14, 2022.
    ---------------------------------------------------------------------------
        Had the OIG relied on the information provided, Congress and the
    public would have been misled as to how trainees had performed in the
    tests. The culture of accountability the Secretary and Deputy Secretary
    are promoting by mandating training on engaging with the OIG and other
    measures is critical; however, this investigation underscores the need
    for leaders overseeing the EHRM program to reinforce those values and
    the requirement for timeliness, completeness, and accuracy in all
    responses to OIG requests for information. The OIG made four
    recommendations, found in appendix E, and one remains open.\18\
    ---------------------------------------------------------------------------
        \18\ Two of the recommendations ask VA to examine if administrative
    action should be taken concerning the conduct or performance of the
    senior leaders. As an independent oversight authority, the OIG cannot
    mandate administrative action or dictate a specific outcome.
    ---------------------------------------------------------------------------
        The OIG made 11 recommendations in the July 2021 report to improve
    the training program, which can be found in appendix F, and seven are
    still open.
       new patient scheduling system needs improvement as va expands its
                     implementation (november 2021 report)
        This report assessed the implementation of the EHR system's patient
    scheduling component at the Columbus clinic and Mann-Grandstaff
    VAMC.\19\ The OIG found VHA and OEHRM did not fully resolve known
    significant limitations in the scheduling system, leading to reduced
    effectiveness and increased risk of patient care delays. The problems
    identified in this report have persisted through the OIG's 2022
    reports, such as schedulers developing work-arounds for unresolved
    issues and problematic data migrated from legacy systems. OEHRM leaders
    did not provide scheduling staff with adequate chances to identify
    limitations in the new scheduling system before implementation, nor did
    leaders assess Cerner's compliance with contract terms for handling
    trouble tickets submitted by users. OEHRM leaders were aware of the
    system's issues before and after Columbus's implementation, but the
    issues were not resolved even in late 2021. That said, VHA staff told
    the OIG that the new system should help greatly, and schedulers
    reported positive experiences. For example, schedulers said the new
    system was more user-friendly than the legacy system, making video
    visits easier to schedule, among other upgrades. The OIG made eight
    recommendations, found in appendix G, and all remain open.
    ---------------------------------------------------------------------------
        \19\ VA OIG, New Patient Scheduling System Needs Improvement as VA
    Expands Its Implementation, November 10, 2021.
    ---------------------------------------------------------------------------
            implementation deficiencies and the lack of remediation
        The OIG has sustained a strong focus on the patient safety aspects
    of the EHRM program, starting with its April 2020 report that reviewed
    VA's readiness to ``go live'' at the initial site and the potential
    impact of the transition on patients' access to high-quality care. The
    findings include that the Mann-Grandstaff VAMC lacked adequate staffing
    and formal, written guidance to navigate the transition's strains. The
    OIG also found that the risk mitigations facility leaders would employ
    during the planned go-live period were inadequate to address the gaps
    in the new EHR system capabilities and presented a potential yet
    significant risk to patient safety.
        In 2022, the OIG published a series of reports that examined a
    range of user and veteran concerns with inadequate planning and
    implementation, which if left unremedied could pose patient safety
    risks and additional instances of harm in future rollouts. Three OIG
    reports released in March 2022 identified EHRM issues connected to
    medication management, care coordination, and the ticketing process
    used by Mann-Grandstaff VAMC providers to request help and resolve
    problems.
        Finally, in July 2022, the OIG determined that the new EHR system
    directed thousands of medical orders to an ``unknown queue'' that were
    not evident to the clinical and administrative staff required to
    address them. The OIG also found that VHA determined the unknown queue
    created significant risk and caused harm to multiple patients. As
    recently as June 2022, hundreds of orders remained in the unknown queue
    across VA sites implementing the new system.
     review of access to care and capabilities during va's transition to a
     new electronic health record at the mann-grandstaff va medical center
                                  (april 2020)
        VA expected a productivity drop associated with the facility's
    preparations for going live with the new EHR system.\20\ Mann-
    Grandstaff VAMC leaders consulted with DoD staff, who transitioned to
    the Cerner system in 2017 and experienced a 30-percent decrease in
    productivity for the subsequent months. VA had plans to mitigate the
    impact on facility personnel for the March 2020 go-live event,
    including adding facility staff, enhancing clinical space, changing
    clinic processes, and a greater use of community care. At publication,
    however, the OIG did not find evidence of VA providing final guidance
    to Mann-Grandstaff VAMC leaders on carrying out these plans.
    ---------------------------------------------------------------------------
        \20\ VA OIG, Review of Access to Care and Capabilities during VA's
    Transition to a New Electronic Health Record System at the Mann-
    Grandstaff VA Medical Center Spokane Washington, April 27, 2020.
    ---------------------------------------------------------------------------
        Some of the problems that emerged were foreseeable. OEHRM and
    Cerner determined in July 2019 that not all anticipated capabilities of
    the new EHR would be available for the March 2020 go-live date. Mann-
    Grandstaff VAMC leaders and staff told the OIG of concerns related to
    the deployment of limited capability sets that led to significant gaps
    in functionality. For example, the MyHealtheVet portal was the most
    frequently used method for patients to request prescription refills,
    but it would not be connected to the new EHR.\21\ Facility leaders and
    staff told the OIG of safety concerns related to losing access to the
    MyHealtheVet electronic refill portal. The OIG was unable to determine
    all potential patient safety risks associated with the new EHR, but the
    work-around for the electronic prescription refill process alone
    presented significant concerns as it could have impacted a patient's
    ability to fill a life-sustaining medication after go-live. Follow-on
    work, discussed later in this statement, conducted by the OIG after
    Mann-Grandstaff VAMC began using the new EHR system, validated numerous
    of these medication management and prescription delivery services.
    ---------------------------------------------------------------------------
        \21\ My HealtheVet, Get to Know Rx Refill Options, https://
    www.myhealth.va.gov/mhv-portal-web/ss20180423-prescription- refill-
    options-for-veterans. (The website was accessed on July 6, 2021.)
    MyHealtheVet is an online personal health portal patients can access to
    schedule appointments, view medical records, refill prescriptions, and
    send secure messages to their care providers.
    ---------------------------------------------------------------------------
        The OIG made eight recommendations, of which three remain open. The
    three that remain open call for VA to evaluate the impact of the new
    EHR implementation on productivity and provide operational guidance to
    facilities on mitigating the impact of the transition and any
    undeveloped aspects of the software on users and patients. The
    recommendations' text and status can be found in appendix H.
        A trilogy of reports released in March 2022 responded to many
    complaints submitted to the OIG hotline and requests from congressional
    offices following the new EHR's deployment at the Mann-Grandstaff VAMC.
    OIG healthcare inspections staff began work on two efforts to address
    several priority concerns-medication management and patient care
    coordination. During this work, the OIG team identified further
    challenges with the trouble ticketing process for system users to
    submit concerns or requests for help, and the OIG team determined that
    some previously identified deficiencies were still unresolved.
    Consequently, the healthcare oversight team started a third effort to
    examine why problems were not addressed and to highlight the underlying
    causal factors. When VA responded to the three reports in early March
    2022-nearly 18 months after going live in October 2020-VA actions to
    resolve issues were limited. The OIG identified 46 issues that were
    unresolved after the OIG completed its inspection in June 2021, but
    only seven were resolved as of March 2022. \22\
    ---------------------------------------------------------------------------
        \22\ The allegations substantiated but unresolved in the trilogy of
    reports date from March 2022. VA requested an extension until September
    16, 2022 on providing its first update as to the status of its work to
    resolve these issues, so at this time, the OIG does not have an update
    on VA's progress.
    ---------------------------------------------------------------------------
      medication management deficiencies after the new ehr go-live at the
                   mann- grandstaff vamc (march 2022 report)
        EHRs can improve clinical decision-making and minimize human error,
    but the risk of patient harm increases when systems have poor
    usability, workflows, or data inputs. The first in the trilogy of
    healthcare inspections focused on medication management for patients
    subject to the new EHR at Mann- Grandstaff VAMC.\23\ This included
    tracking and managing lists of medication, ordering, and promptly
    getting them to patients. Ensuring patients receive the correct
    medication in a timely manner is critical, given many patients are
    older with numerous medical conditions treated with multiple
    medications.
    ---------------------------------------------------------------------------
        \23\ VA OIG, Medication Management Deficiencies after the New
    Electronic Health Record Go-Live at the Mann-Grandstaff VA Medical
    Center in Spokane, Washington, March 17, 2022.
    ---------------------------------------------------------------------------
        The OIG grouped the various complaints regarding medication
    management into three categories: data migration, medication orders,
    and medication reconciliation.
    Data Migration
        For this report, data migration focused on the transfer of patient
    information from VA's legacy EHR to the new system. Deficiencies were
    found with patient contact information, patient medication lists, and
    formulary lists that included medications and supplies unavailable at
    the facility.

      --Patient Contact Information: Prior to going live, VA migrated
            contact information and clinical data for approximately 88,000
            veterans to the new EHR. The OIG found that outdated DoD data
            overwrote VHA's patient contact information, such as name,
            address, telephone number, and email address when data were
            migrated to the new EHR. Consequently, VA patients were delayed
            in receiving medications through the mail order pharmacy
            system.

      --Medication Lists: The OIG substantiated that medication lists,
            migrated as ``free text'' per VHA's request, contained
            inaccuracies. Because medication lists did not import properly,
            care providers used work-arounds, including manual reentry to
            generate accurate medication lists. Staff described this
            process as ``overwhelming'' and time-consuming.

      --Medication Formulary: The new EHR's formulary included many
            medications not available at Mann-Grandstaff or on VA's
            national formulary. Consequently, care providers unknowingly
            selected nonformulary or unavailable supplies. These selections
            increased risks for errors, potentially raised costs for VA,
            and added work for care providers and pharmacy staff.
    Medication Orders
        The OIG substantiated 10 of 12 allegations related to the
    mismanagement of medication orders. The identified problems affect
    every aspect of the process from orders failing to process to patients'
    recurring future medication orders being automatically discontinued
    without notice to providers.

                                  Summary of Medication Order Allegations and Findings

    ----------------------------------------------------------------------------------------------------------------
              Medication Orders                       Allegations               OIG Determination         Status
    ----------------------------------------------------------------------------------------------------------------
         Future Order Discontinuance       The new EHR discontinued future   Substantiated           Unresolved
                                            medication orders written by
                                            providers.

                                           Discontinued future medication    Substantiated           Unresolved
                                            orders required providers to
                                            write ``stat'' or place
                                            immediate orders, causing
                                            medication delays for patients.

                                           Discontinued future medication    Substantiated           Unresolved
                                            orders led absent providers to
                                            arrange for colleagues to write
                                            orders for recurring
                                            medications, creating
                                            inefficiencies and increasing
                                            risks for orders being missed
                                            and possible patient safety
                                            issues.
                                          --------------------------------------------------------------------------
          Unauthorized Orders Placed       Registered nurses could order     Substantiated           Unresolved
                                            medications without provider
                                            approval.
                                          --------------------------------------------------------------------------
       Outpatient Orders Not Processed     Pharmacy staff did not process    Not Substantiated       Not Applicable
                                            outpatient orders.
                                           Some outpatient orders failed to  Substantiated           Unresolved
                                            process and appeared missing to
                                            nonpharmacy staff.
                                          --------------------------------------------------------------------------
             Lack of Notification          Notifications were not sent to    Substantiated           Unresolved
                                            prescribing providers and
                                            pharmacists about future
                                            recurring injectable medication
                                            orders that were discontinued
                                            or outpatient medication orders
                                            that did not process.
                                          --------------------------------------------------------------------------
               Confusing Alerts            Medication alerts were            Substantiated           Unresolved
                                            confusing, and providers did
                                            not receive training on
                                            interpreting them.
                                          --------------------------------------------------------------------------
         Prescription Status Unclear       Providers were unable to assess   Substantiated           Unresolved
                                            the status of a filled
                                            prescription order.
                                          --------------------------------------------------------------------------
         Lack of Tracking for Mailed       Pharmacy staff were unable to     Not Substantiated       Not Applicable
            Controlled Substances           consistently track mailed
                                            controlled substance
                                            prescriptions.

                                           Nonpharmacy staff could not       Substantiated           Unresolved
                                            consistently track mailed
                                            controlled substance
                                            prescriptions.
                                          -----------------------------------
     Prescription Drug Monitoring Program  After completing a PDMP query,    Substantiated           Unresolved
                                            providers' notes were not
                                            automatically populated in
                                            alignment with VHA policy,
                                            requiring additional work for
                                            providers.
    ----------------------------------------------------------------------------------------------------------------

    Medication Reconciliation
        The OIG substantiated that inaccurate medication lists in the new
    EHR challenged staff conducting reconciliations. This critical process
    identifies and resolves any medication discrepancies found in an EHR
    with the information supplied by the patient or caregiver. Accurate
    medication lists guide providers' treatment decisions, and inaccuracies
    could have significant health consequences for a patient. The OIG
    observed that poor training led to a knowledge gap that contributed to
    errors and helped explain varying user experiences.

                              Summary of Medication Reconciliation Allegations and Findings

    ----------------------------------------------------------------------------------------------------------------
          Medication Reconciliation                   Allegations               OIG Determination         Status
    ----------------------------------------------------------------------------------------------------------------
        Medication List Discontinuity      Staff had to update medication    Substantiated           Unresolved
                                            lists at every visit because
                                            prior medication information
                                            revisions did not carry over.

                                           Medications disappeared from      Substantiated           Unresolved
                                            reconciled medication lists,
                                            and lists were inaccurate after
                                            reconciliation.

                                           Staff manually entered            Substantiated           Unresolved
                                            medication lists post-
                                            reconciliation, which increased
                                            risk for error and safety
                                            concerns.

                                           Medication reconciliation         Substantiated           Unresolved
                                            required a significant amount
                                            of time to complete per
                                            patient.
                                          --------------------------------------------------------------------------
         Medication List Inaccuracies      Discontinued and expired          Substantiated           Unresolved
                                            medications were not viewable
                                            during reconciliation, creating
                                            a patient safety issue.

                                           Medications administered in a     Substantiated           Unresolved
                                            clinic did not appear on
                                            medication lists, creating a
                                            patient safety issue.
                                          --------------------------------------------------------------------------
    Medication Lists Unsuited for Patient  Medication lists were not         Substantiated           Unresolved
                     Use                    patient-friendly.
    ----------------------------------------------------------------------------------------------------------------

        The two report recommendations can be found in appendix I. VA
    concurred with the first recommendation, which requires extensive
    software updates that VA indicated may take over a year from
    publication to implement. The second recommendation called for VA to
    ensure medication management issues related to the new EHR identified
    after the inspection be reported to the OIG. VA did not concur with
    this recommendation, citing the difficulty of a continuous, open
    reporting requirement to the OIG. This is not an open-ended
    recommendation, however, and could be closed after VA demonstrates an
    effective and sustainable process to identify and address patient
    safety issues. VA already must provide this information to the OIG
    regardless of whether VA concurs with the recommendation, and the OIG
    will continue this oversight work.
     care coordination deficiencies after the new ehr go-live at the mann-
                      grandstaff vamc (march 2022 report)
        The second report in the trilogy addressed an expansive list of
    allegations categorized as care coordination concerns.\24\ Care
    coordination involves numerous EHR functions that facilitate how care
    is synchronized both among healthcare providers and directly with the
    patient. As an example of these challenges, the VAMC's coordinator for
    the new EHR's patient portal reported a backlog after the go- live of
    over 300 voicemail messages from patients unable to access the portal.
    During the pandemic, the portal was a central means for patients to
    communicate with providers.
    ---------------------------------------------------------------------------
        \24\ VA OIG, Care Coordination Deficiencies after the New
    Electronic Health Record Go-Live at the Mann-Grandstaff VA Medical
    Center in Spokane, Washington, March 17, 2022.
    ---------------------------------------------------------------------------
        The OIG further sorted the allegations into eight categories. Each
    had multiple deficiencies:

        1. Patient Record Flags: Patient record flags denoting patients at
    high risk for suicide and disruptive behavior in the legacy EHR failed
    to activate for some Mann-Grandstaff VAMC patients. Some identified
    concerns about patient record flag functionality in the new EHR stemmed
    from system design, while others related to deficits in training on the
    new EHR's workflow. The flags are not as obvious in the new system as
    they were in the legacy EHR. In some new EHR views, staff had to
    navigate multiple steps to find information about the flag and relevant
    precautions. Of the six substantiated allegations, only two remained
    unresolved: the visibility of the flag and national-level data sharing
    of active record flags for patients at high risk for suicide.

        2. Data Migration: As previously discussed, deficiencies were found
    in the migration of patient information, such as incorrect patient
    names, genders, and contact information. Discussions continued between
    VA and DoD regarding business rule updates needed to improve
    interoperability and ensure accurate data migration in the face of
    policy differences between VA and DoD.

        3. Scheduling Process: Initial allegations received by the OIG
    cited delays in scheduling and inadequate appointment information and
    reminders in the new EHR. Reminders to veterans and caregivers did not
    always specify if appointments were by telephone rather than in-person,
    resulting in some patients traveling to the facility for telephone
    appointments. The OIG was also alerted to problems with the new self-
    scheduling tool that resulted in Washington State patients
    inadvertently self-scheduling appointments at the Columbus clinic. Of
    the five related substantiated allegations, four remained unresolved,
    particularly related to delays in scheduling primary care appointments,
    the type of appointment, and the information contained on appointment
    reminders.

        4. VA Video Connect: This VHA telehealth service technology enables
    veterans to meet virtually with VA healthcare providers from anywhere,
    using encrypted video. The OIG substantiated some allegations that
    appointments failed due to broken links, incorrect time zones, and
    links being sent to outdated email addresses. VA needed to completely
    resolve only the last allegation, as some veterans were still having to
    contact DoD to have their contact information updated.

        5. Referral Management: Deficiencies in implementing the Ambulatory
    Referral Management function decreased care providers' ability to
    manage patients' referrals in the provider's own clinical service,
    particularly in the behavioral health department, and with other
    outpatient services in VHA. These breakdowns could lead to delays and
    affect patient experiences at VHA more generally. For example,
    providers had no easy way to determine if a referral had been acted on.
    Certain aspects of system configuration, workflow errors,
    interoperability deficits, and insufficient training contributed to
    staffs' difficulties with handling referrals. The three substantiated
    issues remained unresolved.

        6. Laboratory Orders: The OIG was alerted to ``disappearing''
    laboratory orders that never reached lab personnel. The system
    configurations and training deficits were factors in these failures.
    Ordering providers were shown a confusing array of options.
    Additionally, staff were challenged in tracking the orders, and many
    results were delayed in being returned. These issues created more
    opportunities for human error as staff used work-arounds to get results
    that informed care delivery. These three substantiated issues were
    unresolved.

        7. Patient Portal and Secure Messaging: As mentioned above, when
    the new EHR went live, many patients could not access the portal,
    affecting access to tools that supported coordination of care, such as
    secure messaging and online prescription refills. VA staff reported
    that system changes completed by OIT resolved some causes of this
    disruption, while other resolutions were in progress.

        8. Documentation Processes: While the OIG did not substantiate all
    allegations received related to documentation process problems,
    facility staff reported experiencing challenges in effectively
    navigating and using some of the new EHR capabilities. Insufficient
    end-user training and misperceptions about certain new EHR
    functionalities appeared to be the sources of the difficulties. VA
    started using a new method, the financial identification number (FIN),
    to document workload associated with between-visit care, which VHA did
    not historically record. This required numerous steps for providers,
    creating additional work and confusion. Another example is a
    configuration issue in which not all International Classification of
    Disease 10 diagnostic codes were available in the new EHR, affecting
    providers' ability to correctly code patient diagnoses. Of the three
    substantiated allegations, the FIN and diagnostic codes were unresolved
    at the time of publication.
        For this report, the OIG made one recommendation, located in
    appendix J, and it remains open.
        ticket process concerns and underlying factors contributing to
     medication management and care coordination deficiencies (march 2022
                                    report)
        The OIG issued this third report to provide an analysis of the
    persistent issues with the ticket process used for reporting problems
    and requesting assistance at Mann-Grandstaff VAMC, including
    identifying the underlying causal factors.\25\ From the October 2020
    go-live date through March 31, 2021, new EHR end users placed over
    38,700 tickets. OIG staff analyzed the help ticket system for key terms
    for each allegation and checked 4,094 tickets related to the issues
    discussed in the two prior reports.
    ---------------------------------------------------------------------------
        \25\ VA OIG, Ticket Process Concerns and Underlying Factors
    Contributing to Deficiencies after the New Electronic Health Record Go-
    Live at the Mann-Grandstaff VA Medical Center in Spokane, Washington,
    March 17, 2022.
    ---------------------------------------------------------------------------
    Ticket Process Challenges
        The OIG team reviewed ticket comments to understand facility
    staffs' frustration with getting fixes and changes. Although VA
    initiated a strategic review to address these concerns, there were
    limited process changes. The ticket process challenges the OIG found
    include the following:

      --Cerner's service desk support staff were not able to view and
            replicate reported issues. While Cerner had a mirror version of
            the DoD EHR, a mirror version of the Mann-Grandstaff VAMC's EHR
            was not built.\26\ OEHRM staff were frustrated that when Cerner
            support staff could not reproduce a reported issue they closed
            the ticket, potentially delaying the problem's resolution.
    ---------------------------------------------------------------------------
        \26\ In the response VA gave to the OIG before publication, VA said
    Cerner service desk support staff had given access to the EHR's
    production version. The OIG will review VA's evidence during the
    follow-up process to determine if that is the case.

      --The same Cerner staff closed tickets before resolving the issues.
            Closing tickets without resolving the concerns could result in
            patient safety issues as well as the propagation of similar
            issues at future implementation sites. Facility staff also
    ---------------------------------------------------------------------------
            reported feeling a lack of support.

      --Ticket status was not communicated to end users. As part of VA's
            agreement with Cerner, end users were to be notified and given
            the opportunity to review whether the proposed or implemented
            resolution addressed the reported issue before Cerner closed
            the ticket. Mann-Grandstaff VAMC staff reported during 2021
            that Cerner's service desk staff were unhelpful or rude.

      --Mann-Grandstaff VAMC staff sometimes created work-arounds instead
            of placing tickets. Due to the challenges, Mann-Grandstaff VAMC
            users began creating work-arounds to accomplish tasks, which
            can increase patient safety risks, create inefficiencies, and
            bypass safeguards.

        This report validated deficient ticket processes identified earlier
    in VA's ``Electronic Health Record Comprehensive Lessons Learned''
    report released in July 2021.\27\ While VA had identified proposed
    measures to monitor these process changes, their July 2021 report said
    the measures had not been finalized and were under review.
    ---------------------------------------------------------------------------
        \27\ VA, Electronic Health Record Comprehensive Lessons Learned
    Report, November 2021. The report was initially released in July 2021
    and updated in November 2021.
    ---------------------------------------------------------------------------
    Underlying Factors of Substantiated Allegations in Companion
            Inspections
        To probe into the causes of the allegations in the two companion
    inspections regarding medication management and care coordination
    issues, the inspection team identified five underlying factors:

        1. EHR Usability Problems. Poor usability has been linked to
    increased patient safety risks, inefficiencies, and care provider
    frustration and stress. Among other issues, the OIG found that the user
    interface was not optimized for workflows; inefficient navigation
    hampered staff; patient datawere in different sections of the EHR; and
    restrictive definitions of user roles assignments, which defined
    employees' capabilities in the system, limited the information staff
    could see.

        2. Training Deficits. The OIG found insufficient training content,
    support, and an approach to training that did not provide staff with
    the underlying reasons for the actions they should take.

        3. Interoperability Challenges. Staff must have access to
    information needed to perform their work from within and across VHA.
    This was hampered by the data migration issues previously discussed,
    the failure of information to transfer to the Consolidated Mail
    Outpatient Pharmacy, and information not properly transferring to
    national-level VHA databases.

        4. Fixes and Refinement Needs. The OIG identified that some
    substantiated allegations were unresolved and required fixes after
    going live, as well as refinements to address errors in system
    workflows and changes to components of the new EHR. For example, staff
    were initially unable to view patients' service-connected conditions
    noted by the Veterans Benefits Administration from the new EHR, which
    led to an inability to document these conditions for healthcare
    delivery purposes.

        5. Problem Resolution Process Challenges. Successful EHR
    implementation requires effective pathways for resolving identified
    problems, and as discussed in this trilogy of reports, the ticket
    process for resolving questions and concerns had several deficiencies.

        For this report, the OIG made three recommendations, found in
    appendix K, and all are open.
     the new ehr's unknown queue caused multiple patient harm events (july
                                  2022 report)
        This review looked at one aspect of the question of whether the new
    EHR resulted in any patient harm.\28\ In May 2021, after VHA identified
    several patient safety concerns, a VHA National Center for Patient
    Safety team went to Mann-Grandstaff VAMC with their work continuing
    through the year. In late 2021, the team drafted a report and held a
    Safety Summit where they ranked dozens of safety concerns based on
    severity, identifying the ``unknown queue'' as one of the most severe.
    ---------------------------------------------------------------------------
        \28\ VA OIG, The New Electronic Health Record's Unknown Queue
    Caused Multiple Events of Patient Harm, July 14, 2022.
    ---------------------------------------------------------------------------
        Information about patient harm due to the new EHR was presented to
    the VA Deputy Secretary in November 2021. In December 2021, the Deputy
    Secretary forwarded information about harms due to the unknown queue to
    the executive director of EHRM IO. From October 24, 2020, through May
    8, 2022, VHA identified 1,134 patient safety events related to the new
    EHR. VHA's analysis identified one catastrophic patient harm (death or
    major permanent loss of function) and two major patient harm cases
    (permanent lessening of bodily functioning), one of which was related
    to the unknown queue.\29\
    ---------------------------------------------------------------------------
        \29\ VA defines ``catastrophic harm'' as ``death or major permanent
    loss of function (sensory, motor, physiologic, or intellectual) not
    related to the natural course of the patient's illness or underlying
    condition (i.e., acts of commission or omission).'' VA defines ``major
    harm as ``permanent lessening of bodily functioning (sensory, motor,
    physiologic, or intellectual) not related to the natural course of the
    patient's illness or underlying condition (i.e., acts of commission or
    omission).'' [bolding not added by the OIG]
    ---------------------------------------------------------------------------
        The intent of the unknown queue was to capture orders entered by
    providers that the new EHR cannot deliver to the intended location. The
    new EHR's design allowed providers to select locations from a drop-down
    menu that, depending on the specific order, the system would not be
    recognize as a ``match.'' This ``mismatch'' would send orders to the
    unknown queue and not to the requested service location to initiate the
    ordered care. Notably, the new EHR did not alert the healthcare
    providers that the order was not delivered to the intended location.
        Orders from care providers began populating the unknown queue
    immediately after the facility went live. Staff had to re-input the
    orders after discovering the issue, expending many hours of labor then
    and during the clinical reviews that assessed the harm patients may
    have suffered. Cerner did take steps with VA to mitigate the problem at
    Mann-Grandstaff VAMC by removing unmapped locations in September 2021.
    As of February 2022, an alert was being sent if a provider created an
    order with an unmapped location. However, prior to March 2022, VHA
    could not generate a report of unknown queue orders itself. Cerner
    acknowledged that the unknown queue's ongoing risk would require
    mitigation at future go-live sites, noting the need to continuously
    reinforce the guidance on managing the queue.
        The OIG found that Cerner did not inform VA end users of the
    unknown queue or provide guidance to address the unknown queue in
    advance of going live with the new EHR. A Cerner vice president,
    identified by the company's general counsel as an unknown queue subject
    matter expert, also reported having no knowledge that VA was told about
    it before going live. Following the OIG's transmittal of the draft
    report to VA in June 2022, Cerner provided EHRM IO with documentation
    that asserted a VA leader approved the use of the unknown queue in
    January 2020. However, that VA leader and their supervisor told OIG
    staff they had no awareness of the unknown queue prior to going live.
        VHA itself assessed the risk as major severity, frequently
    occurring, and very difficult to detect and initiated a clinical review
    in June 2021 to ensure orders were acted on and to assess patients for
    harm. VHA's clinical reviewers conducted 1,286 assessments and
    identified 148 adverse events (with an additional one later found by
    VHA to be a major harm, bringing the total to 149) for patients:

      --Major harm: 2

      --Moderate harm: 52

      --Minor harm: 95

        As an example of major harm, a provider entered a psychiatric care
    order for a patient experiencing homelessness and identified as at-risk
    for suicide. The new EHR sent the order to the unknown queue. The
    patient was not scheduled for follow-up care and later contacted the
    Veterans Crisis Line reporting a razor in hand and a plan to take their
    own life. The patient was hospitalized for psychiatric care.
        The OIG has concerns with the effectiveness of the plan to mitigate
    the unknown queue's safety risk. Facility leaders reported using the
    mitigation process to monitor and manage the queue but shared that
    steps in the process could still lead to orders remaining in the queue.
    In June 2022, when the OIG met with VA leaders to discuss this report,
    VA said that work to address the unknown queue was considered complete
    and that, on average, there were 28 orders in the unknown queue report.
    However, on that day, the OIG generated a report showing 522 total
    orders across the six VA facilities using the new EHR. The OIG made two
    recommendations, found in appendix L, and both are open.\30\
    ---------------------------------------------------------------------------
        \30\ Appendices M and N are the recommendations flowing from two
    additional OIG reports on EHR implementation. The first relates to the
    availability and use of data in the new EHR, and the second is a joint
    report with the DoD Office of Inspector General on the progress of VA
    and DoD in their interoperability efforts.
    ---------------------------------------------------------------------------
                                   conclusion
        This subcommittee and VA have focused tremendous resources on the
    successful transition to the new EHR system. The OIG's work on the
    topic reveals there are still considerable challenges, particularly
    regarding the true costs and scope-especially given the lack of a
    reliable and comprehensive integrated master schedule. Additionally,
    physical and IT infrastructure upgrades at all VHA facilities remain,
    as does the need for effective training and practice before VA staff
    can properly use the new EHR.
        The OIG is committed to providing impactful and practical
    recommendations that flow from its oversight work to help VA deploy the
    new EHR efficiently and in a manner that improves veterans'
    experiences. While each report has specific recommendations intended to
    improve the EHRM program, there are broader concerns that many of the
    recommendations reflect. A primary concern is governance: Are the right
    structures in place to identify potential issues to prevent their
    occurrence, to prioritize those issues that may affect prompt quality
    care to patients, and to resolve those issues before additional
    deployments? Another key concern is transparency: Is there transparency
    among EHRM IO, the facilities, VHA, OIT, and Oracle Cerner? Full and
    candid information sharing will help build confidence that issues are
    being identified, prioritized, and adequately addressed. As VA moves
    toward deployment in more complex facilities, proper governance and
    transparency will be necessary to get it right. Failures in these areas
    risk cascading problems that put the entire program in jeopardy. The
    OIG will continue to monitor EHRM efforts to help recommend
    improvements needed to fulfill its promise to the veteran community and
    make the most effective use of taxpayer dollars. Chairman Heinrich,
    this concludes my statement. I would be happy to answer any questions
    you or other members may have.
      appendix a. va responses to recommendations: deficiencies in infra-
     structure readiness for deploying va's new ehr system, april 27, 2020
        1. The executive director of OEHRM should establish an
    infrastructure-readiness schedule for future deployment sites that
    incorporates lessons learned from the DoD. Status: Closed October 1,
    2020.

        2. The executive director of OEHRM should reassess the enterprise-
    wide deployment schedule to ensure projected milestones are realistic
    and achievable, considering the time needed for facilities to complete
    infrastructure upgrades. Status: Closed October 1, 2020.

        3. The executive director of OEHRM should implement tools to
    comprehensively monitor the status and progress of medical devices at
    the enterprise level. Status: Closed September 21, 2021.

        4. The executive director of OEHRM should standardize
    infrastructure requirements in conjunction with the VHA and the OIT and
    ensure those requirements are disseminated to all necessary staff.
    Status: Closed July 16, 2021.

        5. The executive director of OEHRM should evaluate physical
    infrastructure for consistency with OEHRM requirements and monitor
    completion of those evaluations. Status: Open. VA's targeted completion
    date: March 2021.

        6. The executive director of OEHRM should fill infrastructure-
    readiness team vacancies until optimal staffing levels are attained.
    Status: Closed September 12, 2022.

        7. The executive director of OEHRM should ensure physical security
    assessments are completed and addressed at future EHR deployment sites.
    Status: Open. VA's targeted completion date: None initially provided.

        8. The Mann-Grandstaff VAMC director should ensure all access
    points to physical infrastructure are secured and inaccessible to
    unauthorized individuals. Status: Closed October 1, 2020.
    appendix b. va responses to recommendations: deficiencies in reporting
     reliable physical infrastructure cost estimates for the ehrm program,
                                  may 25, 2021
        1. The executive director for OEHRM should ensure an independent
    cost estimate is performed for program life cycle cost estimates
    including related physical infrastructure costs funded by VHA. Status:
    Open. VA's targeted completion date: 9--12 months from contract start.

        2. The VA assistant secretary for management and chief financial
    officer should ensure the Office of Programming, Analysis and
    Evaluation, or another office performing its duties, conducts
    independent cost estimates as required by VA financial policy, and
    performs an independent estimate of EHRM program life cycle cost
    estimates including physical infrastructure. Status: Open. VA's
    targeted completion date: 9--12 months from contract start.

        3. The director of special engineering projects for VHA's Office of
    Healthcare Environment and Facilities Programs should develop a
    reliable cost estimate for EHRM program-related physical infrastructure
    in accordance with VA cost-estimating standards and incorporate costs
    for upgrade needs identified in facility self-assessments and scoping
    sessions. Status: Closed July 26, 2022.

        4. The director of special engineering projects should also
    continuously update physical infrastructure cost estimates based on
    emerging requirements and identified project needs. Status: Closed
    January 20, 2022.

        5. The executive director for OEHRM should ensure costs for
    physical infrastructure upgrades funded by VHA or other sources needed
    to support the EHRM program are disclosed in program life cycle cost
    estimates presented to Congress. Status: Open. VA's targeted completion
    date: July 31, 2021.
      appendix c. va responses to recommendations: unreliable information
    technology infrastructure cost estimates for the ehrm program, july 7,
                                      2021
        1. The executive director of OEHRM should ensure an independent
    cost estimate is performed for program life-cycle cost estimates
    related to IT infrastructure costs. Status: Open. VA's targeted
    completion date: This is part of the strategic review and will be
    provided as soon as information is available.

        2. The executive director of OEHRM should reassess the cost
    estimate for EHRM program-related IT infrastructure and refine as
    needed to comply with VA's cost-estimating standards. Status: Open.
    VA's targeted completion date: Under active revision as part of the
    strategic review and will be provided as soon as information is
    available.

        3. The executive director of OEHRM should develop procedures for
    cost-estimating staff that align with VA cost-estimating guidance.
    Status: Open.VA's targeted completion date: Under active revision as
    part of the strategic review and will be provided as soon as
    information is available.

        4. The executive director of OEHRM should ensure costs for all IT
    infrastructure upgrades funded by OIT and VHA or other sources needed
    to support the EHRM program are disclosed in program life- cycle cost
    estimates presented to Congress. Status: Open. VA's targeted completion
    date: This is part of the strategic review and will be provided as soon
    as information is available.

        5. The executive director of OEHRM should formalize agreements with
    OIT and VHA identifying the expected contributions from each entity
    toward IT infrastructure upgrades in support of the EHRM program.
    Status: Open. VA's targeted completion date: This is part of the
    strategic review and will be provided as soon as information is
    available.

        6. The executive director of OEHRM should establish procedures that
    identify when life-cycle cost estimates should be updated and ensure
    those updated estimates are disclosed in the program's congressionally
    mandated reports. Status: Open. VA's targeted completion date: This is
    part of the strategic review and will be provided as soon as
    information is available.
     appendix d. va responses to recommendations: the ehrm program did not
     fully meet the standards for a high quality, reliable schedule, april
                                    25, 2022
        1. The EHRM program management office executive director should
    comply with internal guidance and ensure the development of an IMS that
    complies with standards adopted from GAO for scheduling, Status: Open.
    VA's targeted completion date: December 2022.

        2. The EHRM program management office executive director should
    take action to improve stakeholder coordination in the development of
    the program schedules to ensure activities from all relevant VA
    entities are included. Status: Open. VA's targeted completion date:
    August 2022.

        3. The EHRM program management office executive director should
    develop procedures for when and how staff should perform an initial
    schedule risk analysis and conduct periodic updates as needed. Status:
    Open. VA's targeted completion date: December 2022.

        4. The EHRM program management office executive director should
    ensure consistency between contract language and program office plans
    or other guidance identifying the entity or individuals responsible for
    developing and maintaining the program's WBS and IMS. Status: Open.
    VA's targeted completion date: November 2022.

        5. The EHRM program management office executive director should
    evaluate the contract requirements for schedule management and modify
    as needed to ensure clear roles and expectations for further
    development and maintenance of the IMS. Status: Open. VA's targeted
    completion date: December 2022.

        6. The EHRM program management office executive director should
    comply with the Federal Acquisition Regulation and issue guidance to
    accept deliverables not separately priced before invoice payment.
    Status: Open. VA's targeted completion date: May 2022.
        appendix e. va responses to recommendations: senior staff gave
     inaccurate information to oig reviewers of ehr training, july 14, 2022
        1. Issue a clarifying communication to the office's personnel that
    all staff have a right to speak directly and openly with OIG staff
    without fear of retaliation, and that, irrespective of any processes
    established to facilitate the flow of information, EHRM IO personnel
    are encouraged to communicate directly with OIG staff when needed to
    proactively clarify requests and avoid confusion. Status: Closed
    September 7, 2022.

        2. Provide clear guidance that the office's personnel must provide
    timely, complete, and accurate responses to requests for all data or
    information without alteration, unless other formats are requested,
    with full disclosure of the methodology, any data limitations, or other
    relevant context. This includes prompt OIG access to entire datasets
    consistent with the Inspector General Act of 1978, as amended. Status:
    Closed September 7, 2022.

        3. Determine whether any administrative action should be taken with
    respect to the conduct or performance of the executive director of
    Change Management. Status: Open. VA's targeted completion date: July
    2022.

        4. Determine whether any administrative action should be taken with
    respect to the conduct or performance of Change Management's director
    for training strategy. Status: Closed August 15, 2022.
    appendix f. va responses to recommendations: training deficiencies with
    va's new ehr system at the mann-grandstaff vamc in spokane, washington,
                                  july 8, 2021
        1. The USH explores the establishment of a group of VHA staff
    composed of core user roles with expertise in VHA operations and Cerner
    EHR use with data architect level knowledge to lead the effort of
    generating optimized VHA clinical and administrative workflows. Status:
    Open. VA's targeted completion date: September 2021.

        2. The deputy secretary establishes an EHR training domain that
    ensures close proximation to the production environment and is readily
    available to all end users during and following training. Status: Open.
    VA's targeted completion date: January 2022.

        3. The deputy secretary ensures end users receive training time
    sufficient to impart the skills necessary to use the new EHR prior to
    implementation. Status: Open. VA's targeted completion date: January
    2022.

        4. The deputy secretary ensures the user role assignment process
    addresses identified facility leaders and staff concerns. Status: Open.
    VA's targeted completion date: January 2022.
        5. The deputy secretary ensures Cerner trainers and adoption
    coaches have the capability to deliver end user training on Cerner and
    VHA EHR software workflows. Status: Open. VA's targeted completion
    date: January 2022.

        6. The deputy secretary evaluates the process of super user
    selection and takes action as indicated. Status: Closed February 1,
    2022.

        7. The deputy secretary reviews OEHRM's performance-based service
    assessments for Cerner's execution of training to determine whether
    multiple, recurrent concerns are being accurately captured and
    addressed. Status: Open. VA's targeted completion date: January 2022.

        8. The deputy secretary oversees the revision of an OEHRM training
    evaluation plan and ensures implementation of stated objectives.
    Status: Open. VA's targeted completion date: January 2022.

        9. The deputy secretary reviews the EHRM governance structure and
    takes action as indicated to ensure the under secretary for health
    (USH) role in directing and prioritizing EHRM efforts is commensurate
    with VHA's role in providing safe patient care. Status: Closed February
    1, 2022.

        10. The USH establishes guidelines and training to capture new EHR-
    related patient complaints, including patient advocacy. Status: Closed
    August 5, 2022.

        11. The USH ensures an assessment of employee morale following
    implementation of a new EHR and takes action as indicated. Status:
    Closed February 1, 2022.
      appendix g. va responses to recommendations: new patient scheduling
    system needs improvement as va expands its implementation, november 10,
                                      2021
        1. The USH coordinates with the OEHRM executive director to
    continue to make improvements to the scheduling training as needed to
    address feedback from schedulers. Status: Open. VA's targeted
    completion date: January 2022.

        2. The USH coordinates with the OEHRM executive director to require
    that some schedulers from each clinic fully test the scheduling
    capabilities of their clinics, solicit feedback from the schedulers to
    identify system or process issues, and make improvements as needed.
    Status: Open. VA's targeted completion date: November 2021.

        3. The USH coordinates with the OEHRM executive director to issue
    guidance to facility staff on which date fields in the new system
    schedulers should use to measure patient wait times. Status: Open. VA's
    targeted completion date: February 2022.

        4. The USH coordinates with the OEHRM executive director to develop
    a mechanism to track and then monitor all tickets related to the new
    scheduling system, and then ensure OEHRM evaluates whether Cerner
    effectively resolved the tickets within the timeliness metrics
    established in the contract. Status: Open. VA's targeted completion
    date: December 2021.

        5. The USH coordinates with the OEHRM executive director to develop
    a strategy to identify and resolve additional scheduling issues in a
    timely manner as OEHRM deploys the new EHR at future facilities.
    Status: Open. VA's targeted completion date: December 2021.

        6. The USH coordinates with the OEHRM executive director to develop
    a mechanism to assess whether facility employees accurately scheduled
    patient appointments in the new scheduling system, and then ensure
    facility leaders conduct routine scheduling audits. Status: Open. VA's
    targeted completion date: July 2022.

        7. The USH coordinates with the OEHRM executive director to
    evaluate whether patients received care within the time frames directed
    by VHA policy when scheduled through the new system. Status: Open. VA's
    targeted completion date: July 2022.

        8. The OIG recommends that the VA OEHRM executive director provide
    guidance to schedulers to consistently address system limitations until
    problems are resolved. Status: Open. VA's targeted completion date:
    December 2021.
     appendix h. va responses to recommendations: review of access to care
      and capabilities during va's transition to a new ehr system at the
                      mann-grandstaff vamc, april 27, 2020
        1. The USH, in conjunction with OEHRM evaluates the impact of the
    new EHR implementation on productivity and provides operational
    guidance and required resources to facilities prior to go-live. Status:
    Open. VA's targeted completion date: Initial response at Initial
    Operating Capability go-live; revised versions at subsequent go-live
    dates.

        2. The USH, in conjunction with OEHRM, identifies the impact of the
    mitigation strategies on user and patient experience at go-live and
    takes action, as needed. Status: Open. VA's targeted completion date:
    Initial response at IOC go-live; revised versions at subsequent go-live
    dates.

        3. The executive director, OEHRM, in conjunction with the USH,
    ensures that clear guidance is given to facility staff on what EHR
    capabilities will be available at go-live. Status: Closed January 13,
    2021.

        4. The USH, in conjunction with OEHRM, reevaluates the EHRM
    deployment timeline to minimize the number of required mitigation
    strategies at go-live. Status: Open. VA's targeted completion date: May
    2020.

        5. The veterans integrated service network (VISN) director
    collaborates with facility leaders to implement VA-provided operational
    guidance and supports required resources needed throughout the
    transition to the new EHR system. Status: Closed July 31, 2021.

        6. The VISN director ensures that positions required for the
    transition to the new EHR system are staffed and trained prior to go-
    live. Status: Closed October 16, 2020.

        7. The Mann-Grandstaff VAMC director ensures that community care
    consults are managed through go- live to ensure accuracy and
    completeness, and to avoid the need for manual reentry after go-live.
    Status: Closed September 22, 2021.

        8. The Mann-Grandstaff VAMC director ensures that patients receive
    medication refills in a timely manner throughout the transition to the
    new EHR system. Status: Closed September 22, 2021.
      appendix i. va responses to recommendations: medication management
      deficiencies after the new ehr go-live at the mann-grandstaff vamc,
                                 march 17, 2022
        1. The deputy secretary ensures that substantiated and unresolved
    allegations discussed in this report are reviewed and addressed.
    Status: Open. VA's targeted completion date: May 2022.

        2. The deputy secretary ensures medication management issues
    related to the new EHR that are identified subsequent to this
    inspection be reported to the OIG for further analysis. Status: Open.
    VA's targeted completion date: None as VA did not concur with the
    recommendation.
        appendix j. va responses to recommendations: care coordination
      deficiencies after the new ehr go-live at the mann-grandstaff vamc,
                                 march 17, 2022
        1. The deputy secretary ensures that substantiated and unresolved
    allegations noted in this report are reviewed and addressed. Status:
    Open. VA's targeted completion date: May 2022.
     appendix k. va responses to recommendations: ticket process concerns
     and underlying factors contributing to deficiencies after the new ehr
              go-live at the mann-grandstaff vamc, march 17, 2022
        1. The deputy secretary completes an evaluation of the new EHR
    problem resolution processes and takes action as warranted. Status:
    Open. VA's targeted completion date: March 2022.

        2. The deputy secretary completes an evaluation of the underlying
    factors of substantiated allegations identified in this report and
    takes action as warranted. Status: Open. VA's targeted completion date:
    May 2022.

        3. The deputy secretary ensures the EHRM deployment schedule
    reflects resolution of the allegations and concerns discussed in this
    report. Status: Open. VA's targeted completion date: March 2022.
      appendix l. va responses to recommendations: the new ehr's unknown
          queue caused multiple events of patient harm, july 14, 2022
        1. The deputy secretary reviews the process that led to Cerner's
    failure to provide VA substantive information of the unknown queue and
    takes action as indicated. Status: Open. VA's targeted completion date:
    October 2022.

        2. The deputy secretary evaluates the unknown queue technology and
    mitigation process and takes action as indicated. Status: Open. VA's
    targeted completion date: October 2022.
      appendix m. va responses to recommendations: deficits with metrics
     following implementation of the new ehr at the mann-grandstaff vamc,
                                  june 1, 2022
        1. The deputy secretary completes an evaluation of gaps in new EHR
    metrics and takes action as warranted. Status: Open. VA's targeted
    completion date: October 2022.

        2. The deputy secretary completes an evaluation of factors
    affecting the availability of metrics and takes action as warranted.
    Status: Open. VA's targeted completion date: October 2022.
      appendix n. va, dod, and fehrm responses to recommendations: joint
           audit of the dod and the va efforts to achieve ehr system
                         interoperability, may 5, 2022
        1. We recommend that the deputy secretary of defense and deputy
    secretary of veterans affairs review the actions of the Federal
    Electronic Health Record Modernization Program Office (FEHRM) and
    direct the FEHRM to develop processes and procedures in accordance with
    the FEHRM charter and the National Defense Authorization Acts. Status:
    Open.

        VA's targeted completion date: September 30, 2022. DoD's targeted
    completion date: None specified.

        2. We recommend that the director of the FEHRM, in coordination
    with the director of the Defense Health Agency; program executive
    director for EHRMI; and program manager for DoD Healthcare Management
    System Modernization:

          a. Determine the type of patient health care information that
            constitutes a complete patient EHR. Status: Open. FEHRM's
            targeted completion date: August 31, 2022.

          b. Develop and implement a plan for migrating legacy patient
            health care information needed for a patient's complete EHR
            once the FEHRM determines the health care data domains of
            patient health care information that constitutes a complete
            patient EHR. Status: Open. FEHRM's targeted completion date:
            August 31, 2022.

          c. Develop and implement a plan for creating interfaces that
            would allow medical devices to connect and transfer patient
            health care information to Cerner Millennium. Status: Open.
            FEHRM's targeted completion date: One year after resources have
            been approved and allocated, the FEHRM will develop a plan to
            create interfaces between medical devices and the Federal EHR.

        Senator Heinrich. Thank you. Senator Tester has another
    commitment, so I am going to let him go first.
        Senator Tester. Thank you very much, Mr. Chairman. I
    appreciate this.
        Mr. Sicilia, are you the point person for Oracle's efforts
    here with EHR?
        Mr. Sicilia. Yes, I am. I am ultimately responsible for the
    business.
        Senator Tester. Perfect. You are you the right person then
    to have in front of us. How is the communication, how would you
    rate the communication between yourself and VA leadership?
    Would you say it is good, poor, superior?
        Mr. Sicilia. I would say it is very good. I have regular
    meetings with Deputy Secretary Remy, with Dr. Adirim, and I
    certainly look forward to working with Dr. Elnahal.
        Senator Tester. And how often do you meet?
        Mr. Sicilia. We meet twice a month with Deputy Secretary
    Remy, and Dr. Adirim as part of our regularly scheduled
    leadership calls; and sometimes I have one-on-one calls with
    her as needed as well. I would say there are no barriers to
    communication between Oracle and the VA.
        Senator Tester. And as and it is made clear during these
    meetings, what you guys need and what their expectations are?
        Mr. Sicilia. Yes. I think we are in pretty good hands, and
    as I mentioned in my testimony, I think having a dashboard that
    all of us can see.
        Senator Tester. Yes.
        Mr. Sicilia. You can see where the ball and court is for
    every effort is very valuable.
        Senator Tester. So one of the advantages that happened when
    Oracle acquired this, as you are a much bigger company than
    Cerner was, you have much more capacity, you have much more
    ability to solve problems, big problems. But Oracle is a big
    company. Where does this EHR land on the list of what needs to
    be done in priorities for Oracle?
        Mr. Sicilia. This is the most important effort we have
    going on at the company. We have recast over 2,000 people,
    existing Oracle employees, to now work specifically on the VA-
    EHRM program, in addition to the existing Cerner team.
        Senator Tester. So your opening statement was--I mean who
    could argue with what you said--I mean, it was pretty good
    stuff. The only thing you didn't talk about is timeline. What
    is Oracle's timeline for getting this thing going?
        Mr. Sicilia. We will be ready, from a technical
    infrastructure perspective, whenever the VA wishes to resume go
    lives, and if they wish to add more parallelism down the road
    to compress, what now looks like it could be 13, backed into 10
    years. We are ready----
        Senator Tester. Okay. So I am holding VA accountable, but
    what I am not holding VA accountable for is the computer
    program that works, that make sure we don't burn out employees,
    and make sure veterans get the health care they need. They
    can't go live until it meets those measures that we have talked
    about before, and we can talk about more now. What is the
    timeline to making sure that program is going to work so they
    feel comfortable?
        Mr. Sicilia. I think we have a pretty good visibility into
    changes that are needed. For example, the unknown queues,
    talked about many times.
        Senator Tester. I got you.
        Mr. Sicilia. We delivered that fix on August 1st, as
    promised, to the VA. As Dr. Elnahal reported, there are
    provider frustrations, caregiver frustrations with the system.
    What we need is the VHA Councils to provide the content for how
    they want the system reconfigured, if so, because that is not
    necessarily a programming change, we don't have to change the
    code of the system, these are switches that need to be flipped.
    And frankly that is collaboration between Oracle, and the VHA
    Councils, and the VA.
        Senator Tester. And so that takes me back to my first
    question. This thing doesn't get done without good
    communication.
        Mr. Sicilia. That is correct.
        Senator Tester. And so very good, but yet I heard you just
    say that you need more input from VHA.
        Mr. Sicilia. I think with Dr. Elnahal's leadership, I think
    that will increase rapidly. And again, I believe the best way
    to hold us all accountable is to put everything that we need on
    the dashboard so we can all see at the same time, and we don't
    have to wait for the next hearing to have an update. We should
    be able to have very good telemetry into everything that is
    going on with a public-facing dashboard, and as we intend to
    do.
        Senator Tester. Thank you. And thank you for your
    statement, and thank you for the commitment to deliver it with
    the agreed amount. I think that timeliness is something that is
    important here, as on one hand I say don't roll this thing out
    until you are ready for prime time, and on the other hand, we
    can't wait forever either. Okay?
        Mr. Sicilia. Agreed.
        Senator Tester. All right. Thank you.
        This question is for Mc. Case. And it is my last question.
    The work that you do I very much appreciate, as Inspector
    General. I sent a letter to the VA after our last hearing,
    asking them to speed up the closing on their EHR
    recommendations as Chairman of the Senate Veterans Affairs'
    Committee. Mr. Case, can you give us an update on whether you
    have seen a speeding up of the VA side, giving you the
    information that you need to be able to close out these
    recommendations?
        Mr. Case. Yes, we have. We have seen some reasonable
    progress over the last couple of weeks, especially, and we have
    been able to close out six recommendations over the last few
    days, which include some of the recommendations addressing
    training. There is another piece of information we got that may
    allow us to close out another recommendation.
        So, we have seen some good effort, but there is much more
    to do, particularly with recommendations dealing with the
    medication management, care coordination, and even some early
    recommendations, and particularly those pertinent to budget and
    our recommendations concerning having an integrated master
    schedule in place. We are told that will be coming this year.
    But, we will see.
        Senator Tester. Okay. Thank you. And I want to thank
    everybody who testified. I would get to you too, but we are
    having too much fun.
        Thank you, thank you to both of you for having this
    hearing.
        Senator Heinrich. No. Thanks for coming.
        Ranking Member Boozman.
        Senator Boozman. Thank you, Mr. Chairman.
        Dr. Rieksts, the IDA cost estimate looks out over 28 years.
    I would like to focus more on the immediate future; your
    estimate is the VA will take 13, not 10 years to complete. Can
    you explain the discrepancy, why the extra 3 years? And then
    also, you estimate the implementation phase will cost us
    roughly $33 billion, primarily from the acquisition bucket.
    Will you expand on that number and explain how your estimate
    differs from the VA's?
        Dr. Rieksts. Sure. Thank you for the question. First, on
    the 13-year implementation period, we based an extra 3 years in
    our estimate based on a risk analysis. So we estimated a range
    of one to five additional years over the 10-year period that
    will be required. And that is based on both looking at
    historical programs and the challenges that they have had, and
    then events that have happened with the current program that
    have led to the delays that don't indicate that this program
    would behave differently than historical programs.
        And the second point in your question is the implementation
    period and what that includes that is not in the VA cost
    estimate. So one of the big elements is the productivity loss
    that we estimate, that is not in scope and included in the VA
    estimate, and that is the cost of sending veterans out to the
    community, to private caregivers to get care while there is a
    disruption at the facility, and clinicians are busy in
    training, and learning where to click on the new system. And
    that also would include additional staffing that is needed at a
    facility at the go-lives to help those physicians and
    clinicians at a facility. That is one element.
        Another element is, there are some sustainment costs after
    you turn on the initial sites. You need to host and sustain the
    system, and that is also included in our cost estimate. And
    then there are some differences between the IDA estimate and
    the VA estimate of about $8 billion in comparable elements.
        And that is due to both learning from the actual costs that
    have been incurred today. Our estimate has come after their
    estimate, so we leverage at the costs that have been incurred
    to date, and also doing risk analysis of, for example, some of
    the development activities that are needed to make sure the
    system is suitable for veterans, and performs the way that it
    is intended to meet the functions that are required.
        Senator Boozman. Very good. Thank you. The VA OIG has
    issued numerous reports critical of EHRM programs execution and
    transparency. Among these criticisms VA has moved forward with
    deployments without the necessary IT infrastructure in place,
    VA's EHRM and infrastructure cost estimates were not
    comprehensive, well-documented, accurate, or credible. VA's
    reports to Congress did not include cost estimates for physical
    infrastructure, leaving out $2.7 billion in cost, and the
    program is marred by ineffective training, and no integrated
    master schedule.
        Mr. Case, given those criticisms and what you know of the
    IDA independent cost estimate, what should this subcommittee
    make of the testimony today? What will be the true cost of the
    program? And how long do you think it is going to take? Who is
    more correct, VA or IDA? Is the answer somewhere in the middle?
    I know that is difficult, but tell us what you are thinking.
        Mr. Case. Looking at the cost and what IDA has produced, I
    think it is a serious document, is how I would characterize it.
    And as noted, even from the $16 billion that was estimated by
    VA, there is an additional $8 billion that IDA sees there,
    comparing apples to apples.
        One critical aspect of IDA's approach is they include a
    risk analysis. It is good to be optimistic, but we have to
    understand that there is risk to all this, particularly in a
    complicated system like this.
        How long it will take? I think we can only reach an
    informed answer on that once we see an integrated master
    schedule. They have nothing at VA that tells us how they are
    going to get from start to finish, site by site, and what is
    included in that effort. They have promised one this year, and
    we hope to see it, and it is particularly important at this
    juncture, given that they are moving into complex facilities
    next year, and it is a very short time fuse between complex
    facilities.
        As I recall, the current schedule has Ann Arbor going live,
    which is a complex facility, on January 28th. And then the next
    complex facility, which is Seattle, just 1 month later on March
    5th, and the next set of complex facilities even shorter, at
    the end of March, which is both Cincinnati and Dayton.
        Can they get lessons learned implemented in time to go
    complex facility by facility in that period of time? And, will
    issues appear during that short window that they can correct
    between facilities? It took a long time for the unknown queue
    to appear, for remediations to be put in place, and to be
    addressed to where it is now.
        All that is so uncertain that we really can't say how long
    it is going to take. Or, at least, the IG has no confidence to
    be able to say that.
        Senator Boozman. Good. Thank you. Can I ask one more? Thank
    you, Mr. Chairman.
        Mr. Sicilia, we have heard reports of inadequate IT and
    physical infrastructure at VA facilities. What is your
    assessment of VA's IT infrastructure and some of the challenges
    you anticipate that will need to be resolved to successfully
    deploy the system across the Nation? What changes need to be
    made to the program to expedite employment, and ensure the VA
    clinicians accept, and embrace the new system? And what will
    require additional effort? And what will require additional
    investment?
        Mr. Sicilia. Well, hopefully over time, Senator, there will
    be less of that infrastructure needed. As we move to a modern,
    stateless web application you need less infrastructure and not
    more infrastructure. And that is, I think, one of the issues
    that is a challenge with stability and configurations today, is
    also with the cost estimates a long time, because there is an
    assumption that the technology stands still. And obviously as
    it changes you need less of it because the new computers are,
    frankly, far more powerful.
        As far as the view into internal IT needs at the VA,
    sometimes it is just things like printers, and routers, and
    things like that. I think that will continue as a matter of
    course, and it is not uncommon. In any technology, in any
    technology rollout at the local IT landscape sometimes needs to
    change, because it is quite old, and quite attuned to the new
    systems.
        But my hope is that as we get more systems live, that
    becomes a far more repeatable process, and a repeatable cost
    because we have plenty of lessons learned. And if I can share
    with you why I am optimistic of the changes, we are pretty well
    through here, the DOD implementations. In fact, we have two
    more go lives this week, the same system, same computers, same
    code, and we are on budget for the entire program at the DOD
    and the Coast Guard from the original contract scope.
        So that is what gives me hope that all the costs that are
    under our control are certainly achievable with the current
    scope.
        Senator Boozman. Good. Thank you, Mr. Chair.
        Senator Heinrich. Thank you. Mr. Sicilia, drill down a
    little bit more on that last point. What do you see as the
    nature of the differences in the rollout between, for example,
    the Coast Guard and the VA? Is it the complexity of the
    population within the VA? What are the other contributing
    factors to why those rollouts have gone differently?
        Mr. Sicilia. You know, certainly, I think there are several
    factors. The first is, as I mentioned in my testimony, I think,
    frankly, the training and that is on us, the training at the VA
    sites was not as strong as it was at some of the DOD sites. It
    is also not lost on me that the complexity of the population in
    the VA is different than the complexity of the population among
    active service personnel. And therefore, more intense
    workflows, more configurations to the system may be needed.
        I think one of the things that has worked well at DOD is,
    if you will, snapping a chalk line with some of the
    configurations, and it is, which I think is complicating
    situation at the VA, is that you have 171 facilities with 133
    unique versions of a system. And sometimes it is difficult,
    from a change management perspective, to snap that chalk line
    because everybody wants everything they have today to work
    exactly the way it does today in the brand new system. And that
    is difficult when you have so many different custom versions,
    and so many different custom systems.
        And I think where we have seen success, and where we have
    been able to increase our velocity at the DOD, is to snap that,
    you know, that line and have that configuration fixed. And I
    was very encouraged to hear Dr. Elnahal, who I think I agreed
    with everything that he said, about understanding these
    configurations of workflows and making them simpler to use for
    physicians.
        And that is something that we can do today. This doesn't
    require any big investment, or any big change out of
    infrastructure, technical infrastructure, all that will happen
    over time too. But we can certainly reconfigure these workflows
    today. There is no barrier to doing that.
        Senator Heinrich. Talk about that, you know, much of what
    has been done early on with the system and the fixes were
    specific to individual locations. And now we are we are talking
    about moving to the cloud, and having a more ubiquitous system.
    What are the challenges in just making sure that that is a
    seamless transition as well?
        Mr. Sicilia. I would say that some of the fixes were
    specific to individual locations. But things like the unknown
    queue were, you know, a design feature or flaw depending on how
    you want to look at it.
        Senator Heinrich. Right.
        Mr. Sicilia. Of the system from day one. And certainly,
    everybody benefits from that fix, everybody who is live today,
    and who will go live next year. So that is, I think that those,
    both of those things will happen.
        As far as moving to a modern, stateless web application,
    you know, eliminating some of the technology stack as a result,
    our plan is to do that incrementally. I think it would be a
    mistake to just replace everything all at once, and say here is
    the brand new system flip the switch and here you go.
        Now, the benefit that we have is that the Cerner Millennium
    system is written on top of the Oracle database, so there is no
    data migration that has to happen, and that is one of the
    things that is a huge problem that is what is happening today,
    it has got to migrate all the data from the old system to the
    new system. We don't want to repeat that process. That is not a
    good idea. So what we will do is deliver new modern modules on
    top of the existing system that will run in parallel to the
    Cerner Millennium system, and over time councils, and certainly
    practitioners, can decide when they wish to update those
    things. I think that is the most risk-free way to move forward.
        Senator Heinrich. Given your last 4 months of experience,
    as well as the experience that DOD and Coast Guard, knowing
    full well what you are generally in for at this point, what do
    you think is a realistic timeline for safe deployment at the
    entirety of the Department's facilities?
        Mr. Sicilia. I still think we can do it 10 years that was
    originally contemplated. I don't really--comparing this to
    other large commercial programs that we roll out, like
    electricity grid management and, you know, things like this
    which are at the national infrastructure level, it is not
    uncommon that in the beginning, is where you have the most
    bumps in the road, if you will, and you pick up, you pick up
    over time.
        Senator Heinrich. Sure.
        Mr. Sicilia. Yeah, I think agreement on those workflows,
    and making sure they are as simple as we possibly can, eases
    impediment. I am very confident in our new training program,
    and bringing Accenture in to take that. So I don't see a reason
    why I could tell you today that it is not possible to deliver
    on time.
        Senator Heinrich. And you have mentioned several times the
    necessity of getting a dashboard that everybody agrees has the
    metrics that are necessary to get this where it needs to go. Do
    you feel like that that is something everyone agrees on now,
    and is communicating adequately around?
        Mr. Sicilia. I do. And the first version of the dashboard
    is now live, and accessible for all the members of the
    committee. And we will certainly work with VA to add content as
    they say fit to that as well. But everything that we are
    responsible before is now listed on that dashboard.
        Senator Heinrich. I want to get to Senator Hagerty. I
    appreciate him being here for this hearing. But I have one more
    question before we transition to him.
        And that is simply the issue, Mr. Sicilia, around outages
    and stability at the system. Give us some transparency into
    that, and how we get to what is a very high standard, and not
    unlike the standard you would apply to a power utility, 99.9
    system uptime. Talk to us about that and what your plans are.
        Mr. Sicilia. Yeah. Our early view of this, 4 months into
    the system, was a bit under-resourced technically, so we have
    added computer capacity to the system at our expense, and we
    will continue to do that over time, to make sure that all
    future go-lives are adequately staffed.
        We also have over 40 different programs going on, seven of
    which we completed already for system, shall we say, internal
    audits, where we are auditing the system and looking at all of
    the technical functionality of the system, I am confident that
    we can address the stability, and we have made great progress
    towards getting to the 99.9 over the last few months.
        We are actually in the process, where we plan to do to put
    those numbers out publicly, as we do for our business in
    general at Oracle. We are just in the process of working with
    VA right now on the exact--to make sure that we agree on the
    exact calculation of uptime, because we don't want to put a
    number out there that they may disagree with.
        Senator Heinrich. Great.
        Mr. Sicilia. So that will also be a public metric as well.
        Senator Heinrich. And we look forward to that.
        Senator Hagerty.
        Senator Hagerty. Thank you, Chairman Heinrich, and Ranking
    Member Boozman. I appreciate your chairing this. And to our
    witnesses, thank you for being here.
        You know, one of the most important missions of the Federal
    government is fulfilling the promise that we have made to our
    bravest men and women and their families for the sacrifices
    that they have made, sacrifices that we will never be able to
    fully repay. And I know that this subcommittee is committed to
    ensuring that our 17 million veterans, 430,000 of them living
    in my Home State of Tennessee, get the health care that they
    deserve.
        The Electronic Health Record Modernization program is one
    of the key programs in the VA Department that has undertaken to
    provide the best possible care to our veterans. This
    subcommittee has provided nearly $8.5 billion to this program
    over the past 5 years. As my colleagues have covered,
    unfortunately this program has been plagued by delays, by
    problems, and by safety concerns.
        So I would like to turn to you, Mr. Sicilia. You mentioned
    in your opening remarks that Oracle has recently acquired
    Cerner, and that Oracle brings world-leading expertise in
    software, and cloud infrastructure, and health IT. I am certain
    that there is significant value added that was contemplated
    when the acquisition was undertaken.
        So Mr. Sicilia, can you go through the process of
    describing the resources, and the capabilities and the
    expertise that Oracle is going to bring to bear above and
    beyond what Cerner was able to provide on its own?
        Mr. Sicilia. Yeah. As I stated, we have already repurposed
    thousands of engineers, software engineers to move into a
    central organization, wherein their sole focus is the health
    mission and the mission of this program, and to provide
    additional engineering capacity to the program.
        In my view, and owning this now for 4 months, that is where
    I think the single biggest lack was on the Cerner side. I think
    engineering horsepower in terms of people who have built very
    large scale systems for a living, both at the infrastructure
    level, and the application level, for very large mission-
    critical systems, like the world's clinical trials networks,
    the world's power grids and utilities, that same caliber of
    folks who are used to building very high, of highly available
    systems, are now working on the Cerner system. So I am
    confident.
        I do believe that the Cerner team brought us tremendous
    clinical expertise. I mean obviously there are over 1,500
    people who work at Cerner, have some medical degree, are either
    a doctor, or a nurse, or a radiologist, or something. So I
    think putting those two things together is a marriage well
    made. And I am confident that we at Oracle have the engineering
    capacity that is needed to add significance, and make
    significant velocity changes to the program.
        Senator Hagerty. Beyond the thousands of engineers that you
    have added, are there other areas of actions or specific
    improvements that you have undertaken since June to make this
    more effective?
        Mr. Sicilia. We have sponsored--well as you know, this
    implementation is the VA, plus the DOD, plus the Coast Guard,
    so it is all three, so the Federal enclave. You know, I think
    bringing together all of our Federal partners in this has been
    very valuable. So we have sponsored the meetings at the Cerner
    Headquarters in Kansas City.
        We continue to meet, so on a very regular basis, and now
    have a joint leadership meeting among the Oracle team, the
    Oracle Cerner team which is a holding a subsidiary at this
    point; the VA, the DOD, Leidos, et cetera, and everybody who is
    involved in this. And I think that that leadership meeting has
    led to, or the regular leadership meeting has led to great
    improvements, and actually the sharing of successes.
        Senator Hagerty. I am glad to hear that. I am sure you are
    looking at best observed practices across this and trying to
    make certain that would benefit from the Federal government's
    broader experience here, particularly as you try to bring the
    VA up to par.
        Would you mind to discuss, Mr. Sicilia, the impact, or what
    you anticipate the impact to be by having a centralized
    interoperable modern health record system, and how will that
    impact the quality of care for our veterans?
        Mr. Sicilia. Well, the premise is the longitudinal health
    record, which means that, you know, as either an active-duty
    service member or a veteran you don't have to go fish around to
    a bunch of different websites, or fax machines, or all the
    things that plague people from getting their health information
    today, into an aggregate stack, an electronic stack.
        And you can imagine that from a quality care perspective,
    every time you see a new provider not having to start over, and
    saying, you know, here is all the things that I have been
    treated for in the past, the medications, all of that is
    readily available, there is a tremendous time saving, and a
    tremendous, you know, lack of fatigue factor.
        I think sometimes people just get so fatigued of going
    through the process, of trying to get all their records
    together, to go to a new appointment that they give up.
        Senator Hagerty. Yeah. And the frustration too, I mean, and
    people can go to the VA--go to the VA in Memphis, and then go
    to the one in Nashville, and they can't get their records
    transferred.
        Mr. Sicilia. Right, and so if you are the VA, and you are
    traveling seeing family out of state, you ought to be able to
    have an application that tells you exactly where you can go and
    make an appointment right away, and have your records follow
    you, so you don't have to go, you don't have to go start over.
        And then, obviously, from the provider standpoint,
    providers suffer too from not having longitudinal records. I
    mean, they don't know enough about the patient that they are
    treating, and they would love to be able to know a lot of
    things ahead of time, before they walk into the room and have
    to spend the first 15 minutes of an appointment starting over.
    So I think that those benefits will be tremendous for both
    patients and providers, and will be.
        Senator Hagerty. Well, given the magnitude of investment
    that this committee has put in place, I am glad to hear you are
    kicking it into overdrive with the acquisition, putting new
    resources in place. We are looking forward to seeing this
    getting done.
        Mr. Sicilia. Thank you.
        Senator Hagerty. Thank you. Thank you, Mr. Chairman.
        Senator Heinrich. Ranking Member Boozman, do you have any
    follow-up questions?
        Senator Boozman. No. Thank you.
        Senator Heinrich. I want to thank all of our panelists and
    the senators who participated in today's hearing. This is an
    incredibly important endeavor. We certainly appreciate
    everyone's efforts to make sure it is done correctly for the
    benefit of all of our veterans.

                         ADDITIONAL COMMITTEE QUESTIONS

        And finally, I will keep the hearing record open for one
    week. Committee members who would like to submit written
    questions for the record should try to do so by 5:00. Should
    actually do so by 5:00 p.m. Wednesday, September the 28th.
                    Questions Submitted to Mr. Mike Sicilia
                 Questions Submitted by Senator Martin Heinrich
        Question. Since the contract was signed, issues have arisen that
    require shifts in what was originally planned. Oracle's commitment to
    doing much of this new work within the scope of the initial contract is
    appreciated. Your testimony indicated that on top of the $10.0 billion
    contract there already has been an additional $700 million in
    additional requirements identified, for a total of $10.7 billion. The
    clarification provided subsequent to the hearing that the contract
    ceiling is not contemplated to be raised is appreciated. What specific
    changes, by issue and amount, comprise the additional $700 million
    Oracle referenced?
        Answer. With any contract of this size and duration, there will
    always be a need for prudent, new programmatic requirements to account
    for emerging realities to be able to adjust course and implement
    lessons learned. As such, additional requirements have been identified
    that were not previously contemplated in the original Indefinite
    Delivery Indefinite Quantity (IDIQ) ceiling price. These additional
    requirements include but are not limited to the work related the VA
    side of interfaces, new requirements including Pharmacy enhancements,
    the standalone scheduling deployment in Columbus and additional
    solutions such as ones for clinicians to chart vitals data directly
    into the EHR.
        Oracle Cerner expects to be able to identify and leverage cost
    efficiencies to achieve taxpayer's savings over the ten-year term.
    Accordingly, Oracle continues to be committed to continuously
    evaluating opportunities to bring efficiencies and will continue to
    work with VA now, and over the life of the contract to deliver on those
    efficiencies through modern, enterprise-based technologies. Our
    commitment to efficiencies gained through the evolution of technology
    over time is expected to balance and offset many of the new
    requirements.
        Question. How close are Oracle Cerner and VA from having a rollout
    that is ``replicable'' and a better sense of what it will cost, on
    average, to deploy at a facility?
        Answer. As the number of facilities with different types of care
    and clinical complexity go-live, we will be able to better standardize
    deployment processes and solutions. Specifically, facilities with
    varying types of care and complexity may require new workflows, order
    sets, and capabilities that may not already be live and thus would be
    standardized over time as they are introduced to sites and accessed by
    end users. The new deployment schedule should help accelerate that
    process by pulling in capabilities earlier, such as the pharmacy
    enhancements. While difficult to eliminate all the variance between
    sites, having a holistic, replicable process will be key to better
    predict costs and scope of the work.
        This is intrinsic to our deployment methodology; however, it is
    dependent on a framework of a refinement of the system with a steady
    and clear baseline of scope and requirements. For this reason, as
    deployments resume, it will be critical to have a clear, detailed
    checklist of both requirements to resume go-lives uninterrupted and an
    ongoing set of standardized readiness criteria for site deployments
    going forward.
        Question. According to Oracle Cerner and VA, one of the significant
    differences between the first deployment of the Cerner EHR at Mann-
    Grandstaff and the subsequent deployments was the commitment to
    training. What are Oracle's plans to make the training more useful,
    effective and efficient?
        Answer. VA and Oracle Cerner remain committed to providing training
    that prepares end users to use their new EHR in the delivery of health
    care to Veterans through ongoing improvements reflective of experience
    gained and lessons learned.
        Actions taken following deployment at Mann-Grandstaff include
    4,000+ content improvements, enhancement of all courses for virtual
    delivery, conversion of some foundational content to self-paced
    computer-based courses to provide flexibility, creation of training to
    support hands-on practice, and development of more targeted training in
    specialty areas. The initial impact of these efforts is reflected in
    improved training satisfaction survey results, a return to productivity
    baseline in many areas, and productivity levels as illustrated by key
    performance indicators (e.g., decreased time in the EHR, specifically
    at urgent care centers and in labs).
        While progress has been made, there remains work to be done. The
    EHRM training program, one of significant size and complexity, is
    executed in accordance with government-defined requirements and
    government-determined priorities. While VA has contracted with Oracle
    Cerner to provide technical training on the new EHR, end user feedback
    reflects more expansive expectations. To better understand these needs
    and associated opportunities, Oracle Cerner engaged a third-party to
    conduct an independent assessment of the EHRM training program and
    recommendations offered to VA by Oracle Cerner thus far at our own
    expense. The result of this effort was the identification of 10 (10)
    root cause issues, six (6) high-level recommendations and more than 25
    initiatives.
        As I mentioned in my testimony, we will be engaging Accenture to
    assist in implementing the changes needed to make the training much
    more efficient, applicable, and useful. The scope of Accenture's
    support of this aspect of the EHRM program is currently under
    consideration and we will keep the committee apprised of progress as
    that process moves forward.
        Question. Oracle announced plans to implement a cloud-based
    solution to improve stability and enhance the EHR system. Given this
    shift, what adjustments to current site assessment findings will need
    to be factored in to adjust the estimates for this shift from data
    centers?
        Answer. The current state reviews (CSR) incorporate both technology
    and infrastructure site reviews and are capturing the site-specific
    infrastructure and workflows needed to understand the scope and
    requirements for deployments. I don't anticipate any changes needed to
    the CSR requirements to account for the future cloud-based solution.
        Question. In recent months there have been many instances of system
    instability--either degradations or outages. The contract includes a
    99.9 percent standard for system uptime, with a required ``credit'' to
    be provided by Oracle Cerner when the standard is not met. Is the 99.9
    percent standard an appropriate expectation to meet?
        Answer. Yes, the service-level commitment (SLC) is industry
    standard and an appropriate expectation. Delivering a single common
    health record between VA, DoD and the U.S. Coast Guard is the first of
    its kind and is technically challenging, but the benefits are enormous.
    Veterans and Service members will have greater flexibility in where and
    how they seek care by no longer having to carry their own paper records
    or being tethered to their data siloed in specific medical centers.
        However, the benefits of this system can only be realized if it is
    reliable and completely trusted by the people using it. We are working
    on more than 40 different technical operational improvement projects
    for the Federal Enclave that we continue to lead to improved
    performance and greater stability. We have made progress already,
    completing 12 projects and expecting 4 more to be completed by the end
    of the year.
        We are seeing benefits from our architecture and operation reviews,
    including July and September without a major incident and decreased
    frequency of incidents related to change. Specifically, we had a 60
    percent month over month reduction in incidents related to change after
    Oracle introduced more rigorous methodology and oversight.
        Question. At one point in March 2022, the system was down for over
    20 hours, which resulted in a credit of a little over $100,000. Given
    the scope and scale of the overall contract, do you see the credits
    required when Oracle does not meet this target as appropriately
    incentivizing?
        Answer. We will leave it to our client to determine the appropriate
    levels consistent with the contractual requirements. With that said,
    our incentive goes well beyond this single component. With our
    acquisition of Cerner, we see our work with the VA and DoD as the
    opportunity to be a show piece for what Oracle will bring broadly to
    the entire health IT industry. We are driven by delivering what has
    been promised and exceeding expectations with the VA EHR setting the
    gold standard with a new generation of modern, cloud-native, highly
    performant and secure EHR applications embracing mobility, self-
    service, analytics and ease of use to solve some of the biggest
    challenges facing our veterans and VA clinicians.

                                     ______

                   Questions Submitted by Senator John Hoeven
        Question. The safekeeping of our Nation's servicemembers and
    veterans' medical records is essential not only for an individual's
    privacy, but for our National security. What systems does Oracle Cerner
    have in place so that medical records are secure?
        Answer. Oracle is a leading enterprise software vendor with more
    than forty years of experience building and developing some of the most
    advanced, mission-critical, secure and performan technology around the
    world for governments, critical infrastructure, and commercial
    enterprises. As an example, Oracle operates fully certified government
    cloud regions under the Intelligence Community's Commercial Cloud
    Enterprise (``C2E'') program and is fully qualified under the DoD's
    upcoming Joint Warfighter Cloud Capability Program (``JWCC'').
        Oracle's cyber security expertise together with Cerner's expertise
    in securing sensitive patient data is a very powerful combination that
    we believe will enhance security and data protection for our Nation's
    servicemember and veterans.
        The on-going modernization of the Federal electronic health record
    system across VA and DoD enhances health record protections in three
    main areas. First, incorporating regulatory best practices in support
    of HIPAA and NIST requirements. Second, the transformation from
    disparate systems running across multiple data centers to an enterprise
    system decreases system dependencies and surface area susceptible to
    security issues while enhancing system agility to be able keep pace
    with the ever-evolving cyber security threats. Third, the additional
    protections in moving the Cerner application, with the approval of VA,
    DoD and Coast Guard, to a modern, hyperscale cloud data center. This is
    the same Generation 2 Cloud Infrastructure that underpins Oracle's
    customers' most critical workloads.

                                     ______

                    Questions Submitted to Dr. Brian Rieksts
                 Questions Submitted by Senator Martin Heinrich
        Question. VA has not provided a revised schedule for the remainder
    of deployments, but the IDA analysis suggests it could take 3 years
    longer than originally planned. Understanding there are a lot of
    unknowns with future deployments, what is the basis for that
    assessment?
        Answer. The current program has already experienced schedule slips.
    For example, Seattle Veterans Affairs Medical Center (VAMC) was
    originally projected to go live in March 2020. The current projection
    is that this site is not likely to deploy until at least June 2023.\1\
    ---------------------------------------------------------------------------
        \1\ Veterans Affairs Office of Public and Intergovernmental
    Affairs, ``VA extends delay of upcoming electronic health record
    deployments to June 2023 to address technical and other system
    performance issues,'' October 13, 2022. https://www.va.gov/opa/
    pressrel/pressrelease.cfm?id= 5833, accessed October 18, 2022.
    ---------------------------------------------------------------------------
        The Institute for Defense Analyses (IDA) estimated the overall
    deployment schedule using historical data on enterprise resource
    planning (ERP) systems.\2\ Those data show that the average schedule
    slip was 31 percent, which translates into a mean expected schedule
    adjustment of 3 years when applied to the Electronic Health Record
    Modernization (EHRM) initial 10-year deployment schedule. The
    historical data also have a standard deviation of 32 percent. Using
    this historical schedule variation, IDA estimates a 20 percent chance
    the deployments will be completed within 11 years and an 80 percent
    chance the deployments will be completed within 15 years.
    ---------------------------------------------------------------------------
        \2\ H. Gebre-Mariam et al., ``Assessing Enterprise Resource
    Planning (ERP) Cost, Schedule and Size Growth,'' June 2017, 2017 ICEAA
    Professional Development & Training Workshop.
    ---------------------------------------------------------------------------
        Question. Oracle Cerner indicated plans to shift to a cloud model,
    rather than data center requirements at each facility. Would this
    result in savings compared to the IDA estimate? If so, what would the
    potential magnitude of the savings be and where would they be realized?
        Answer. The planned requirements to shift to the Oracle cloud were
    not available for our cost estimate, so we do not have an estimate of
    this change. Costs for hosting and infrastructure could decrease;
    however, additional cybersecurity costs may be required to shift from
    current operations to the Oracle cloud. The net effect of these changes
    depends, in part, on the specific changes in requirements.
        A shift to the Oracle cloud would have an effect on some of the
    EHRM cost elements. We identified hosting ($5B) as the largest cost
    element in the IDA estimate affected by the shift. Oracle Cerner cloud
    services would still require some hosting costs; however, savings may
    be achieved if hosting costs decrease.
        Question. The IDA estimate treats the costs of ``common
    infrastructure'' as being on top of the EHR estimate. What criteria did
    IDA use in identifying and estimating this category?
        Answer. IDA defines common infrastructure as costs that would still
    be incurred without the EHRM program. We engaged with VA staff and
    reviewed documentation to identify common physical or information
    technology (IT) infrastructure costs that would be incurred regardless
    of EHRM. For example, common infrastructure includes shared services
    that are required by other programs, such as the Financial Management
    Business Transformation (FMBT) or Veterans Affairs Logistics Redesign
    (VALOR). Another example of common infrastructure is maintenance (e.g.,
    cabling, climate control) that is planned regardless of EHRM. These
    costs are reported separately from the life-cycle cost estimate (LCCE)
    because they are not required exclusively by EHRM.
        Some cost elements are not exclusively common or EHRM-specific
    infrastructure. For example, WiFi is common infrastructure, but the
    increase in WiFi density required by EHRM is EHRM-specific. In these
    cases, we estimated the share of the overall costs that are EHRM-
    specific infrastructure. To address the uncertainty in the
    categorization of EHRM-specific and common infrastructure, we estimated
    high and low values for the percent of common infrastructure and
    included this estimate in our risk analysis.

                                     ______

                     Questions Submitted to Mr. David Case
                 Questions Submitted by Senator Martin Heinrich
        Question. VA Office of Inspector General reports related to EHRM
    implementation often flag areas where VA may not have followed
    standard, acceptable procedures in moving forward, such as with the
    previous cost estimate report to Congress and the development of a
    master schedule. What does OIG perceive as the greatest risk to the
    initiative going forward?
        Answer. The OIG is concerned about any risk that could impact
    patient safety. In our three March 2022 reports about the
    implementation at Mann-Grandstaff VA Medical Center, Medication
    Management Deficiencies after the New Electronic Health Record Go-Live
    at the Mann-Grandstaff VA Medical Center in Spokane, Washington, Care
    Coordination Deficiencies after the New Electronic Health Record Go-
    Live at the Mann-Grandstaff VA Medical Center in Spokane, Washington,
    and Ticket Process Concerns and Underlying Factors Contributing to
    Deficiencies after the New Electronic Health Record Go-Live at the
    Mann-Grandstaff VA Medical Center in Spokane, Washington, we discussed
    issues related to patient safety, medication management, and care
    coordination. In our July 2022 report on the unknown queue, The New
    Electronic Health Record's Unknown Queue Caused Multiple Events of
    Patient Harm, we focused on the importance of resolving the queue
    because it requires human activity to identify what is in the queue,
    remove the order, and reroute the order. Those mitigations can lead to
    delays in care, which could impact patient safety. We have also
    recommended that VA ensure that the employee training program is
    conducted in an efficient and effective manner and improvements to the
    new EHR system are implemented speedily so that VA providers can return
    to predeployment productivity levels as quickly and safely as possible.
        We identified issues with VA's costs estimates and master schedule.
    Our 2021 reports Unreliable Information Technology Infrastructure Cost
    Estimates for the Electronic Health Record Modernization Program and
    Deficiencies in Reporting Reliable Physical Infrastructure Cost
    Estimates for the EHRM Program indicated that existing physical and IT
    infrastructure at VA medical facilities was inadequate for the new
    system and pertinent life cycle cost estimates were unreliable and
    underestimated possibly by about $5 billion. We note that the
    Department is finalizing the life cycle cost estimate prepared by the
    Institute for Defense Analyses. Once that is done, we will review the
    report and supporting materials to determine which, if any, of our
    recommendations from our two reviews of VA's physical and information
    technology cost estimates could be closed.
        However, as we noted in our report, The Electronic Health Record
    Modernization Program Did Not Fully Meet the Standards for a High-
    Quality, Reliable Schedule, VA must also develop a reliable,
    comprehensive schedule for full system implementation. Identified
    deficiencies could result in schedule delays and leave VA vulnerable to
    billions of dollars in cost overruns. Without that schedule, Congress
    cannot rely on VA's timelines for completing the work or be assured
    that the program will be completed within budget or in line with the
    independent cost estimate.
        Question. Given the number of recommendations that remain open
    after more than a year, has VA been collaborative in working on the
    issues that have been raised?
        Answer. We saw some reasonable progress recently in September 2022,
    and we were able to close out six recommendations shortly before the
    subcommittee's hearing. This included some of the recommendations
    addressing training deficiencies from our July 2021 report, Training
    Deficiencies with VA's New Electronic Health Record System at the Mann-
    Grandstaff VA Medical Center in Spokane, Washington. While we have seen
    some improved efforts, there is much more to do with recommendations
    addressing medication management and care coordination, and
    recommendations pertinent to costs and having an integrated master
    schedule in place.

                                     ______

                   Questions Submitted by Senator John Hoeven
        Question. The VA Office of Inspector General has a number of
    recommendations that remain open with regard to the VA's electronic
    health record (EHR) modernization.
        Of the open recommendations, which are of the greatest concern in
    terms of patient safety, privacy, and cost?
        Answer. The OIG is concerned about any risk that could impact
    patient safety. In our three March 2022 reports about the
    implementation at Mann-Grandstaff VA Medical Center, Medication
    Management Deficiencies after the New Electronic Health Record Go-Live
    at the Mann-Grandstaff VA Medical Center in Spokane, Washington, Care
    Coordination Deficiencies after the New Electronic Health Record Go-
    Live at the Mann-Grandstaff VA Medical Center in Spokane, Washington,
    and Ticket Process Concerns and Underlying Factors Contributing to
    Deficiencies after the New Electronic Health Record Go-Live at the
    Mann-Grandstaff VA Medical Center in Spokane, Washington, we made
    recommendations related to patient safety, medication management, and
    care coordination. From our July 2022 report on the unknown queue, The
    New Electronic Health Record's Unknown Queue Caused Multiple Events of
    Patient Harm, there is an open recommendation related to evaluating the
    mitigation process because it requires human activity to identify what
    is in the queue, remove the order, and reroute the order. Those
    mitigations can lead to delays in care, which could impact patient
    safety. We have also recommended that VA ensure that the employee
    training program is conducted in an efficient and effective manner and
    improvements to the new EHR system are implemented speedily so that VA
    providers can return to predeployment productivity levels as quickly
    and safely as possible.
        Our 2021 reports Unreliable Information Technology Infrastructure
    Cost Estimates for the Electronic Health Record Modernization Program
    and Deficiencies in Reporting Reliable Physical Infrastructure Cost
    Estimates for the EHRM Program indicated that existing physical and IT
    infrastructure at VA medical facilities was inadequate for the new
    system and pertinent life cycle cost estimates were unreliable and
    underestimated possibly by about $5 billion. We note that the
    Department is finalizing the life cycle cost estimate prepared by the
    Institute for Defense Analyses. This estimate was done by VA in
    response to our recommendations. Once that is done, we will review the
    report and supporting materials to determine which, if any, of our
    recommendations from our two reviews of VA's physical and information
    technology cost estimates could be closed.
        However, as we noted in our report, The Electronic Health Record
    Modernization Program Did Not Fully Meet the Standards for a High-
    Quality, Reliable Schedule, VA must also develop a reliable,
    comprehensive schedule for full system implementation. Identified
    deficiencies could result in schedule delays and leave VA vulnerable to
    billions of dollars in cost overruns. Without that schedule, Congress
    cannot rely on VA's timelines for completing the work or be assured
    that the program will be completed within budget or in line with the
    independent cost estimate.

                              SUBCOMMITTEE RECESS

        Senator Heinrich. And with that we stand adjourned. Thank
    you all.
        [Whereupon, at 11:22 a.m., Wednesday, Septemeber 21, the
    hearing was adjourned, and the subcommittee was recessed, to
    reconvene at a time subject to the call of the Chair.]

                                  [all]