[Senate Hearing 117-]
[From the U.S. Government Publishing Office]




 
  VA'S ELECTRONIC HEALTH RECORD MODERNIZATION: AN UPDATE ON ROLLOUT, 
                           COST, AND SCHEDULE

                              ----------                              


                     WEDNESDAY, SEPTEMBER 21, 2022

                                       U.S. Senate,
           Subcommittee of the Committee on Appropriations,
                                                    Washington, DC.
    The subcommittee met at 10:01 a.m. in room SD-124, Dirksen 
    
Senate Office Building, Hon. Martin Heinrich (chairmain) 
presiding.
    Present: Senators Heinrich, Tester, Boozman, and Hagerty.

                     DEPARTMENT OF VETERANS AFFAIRS

STATEMENT OF HON. DONALD M. REMY, DEPUTY SECRETARY, 
            DEPARTMENT OF VETERANS AFFAIRS

              OPENING STATEMENT OF SENATOR MARTIN HEINRICH

    Senator Heinrich. Good morning. This hearing of the 
Military Construction, Veterans Affairs, and Related Agencies 
Appropriations Subcommittee is now called to order.
    Today, we will discuss VA's efforts around Electronic 
Health Record Modernization, or EHRM. This is a large but 
important initiative that was promised to result in a single 
health record for an individual, from service entry through a 
veteran's life, with seamless health data sharing.
    Unfortunately, implementation efforts to date have been 
plagued by delays, provider complaints, patient safety 
concerns, and questions about the accuracy of reporting to 
Congress, particularly related to cost.
    Our goal today is to address these issues over two panels. 
With that, I would like to welcome Donald Remy, Deputy 
Secretary of Veterans Affairs, and the accountable official for 
VA's Electronic Health Record Modernization Initiative. He is 
accompanied by Dr. Terry Adirim, the Program Executive Director 
of the EHRM Integration Office; Dr. Elnahal, VA's Under 
Secretary for Health; and Mr. Jon Rychalski--all right, three 
for four is not too bad--Assistant Secretary for Management and 
Chief Financial Officer.
    I want to start just by reiterating the committee's support 
for VA's EHRM objective. This is an extremely important effort 
to solve a decade-long problem, decades-long problem, and is 
essential that VA get it right for the health and safety of 
veterans. In addition, this is a major investment, and VA has a 
responsibility to taxpayers to ensure the system works, and its 
success can be measured.
    In 2018 when VA announced it was signing a sole-source 
contract with Cerner Corporation, now known as Oracle Cerner, 
VA estimated the EHRM Initiative would cost $16 billion over 10 
years. Congress was told that this amount incorporated the full 
cost of deploying and operating the new health record across 
VA's enterprise, including $10 billion for Cerner Corporation 
to provide the record itself.
    Since that initial estimate we learned that VA did not 
include all of the costs required for a successful 
implementation, and did not report those omissions to Congress 
in a timely way. This lack of transparency was disappointing, 
and the committee is glad that VA has made the effort to be 
more forthright under new leadership.
    Based on a recent independent cost estimate completed by 
the Institute for Defense Analyses, at VA's request, the cost 
to the Department could be significantly more than the initial 
estimate. The nearly $50 billion estimate assumes a longer 
deployment timeline, and 15 years of sustainment costs, as well 
as additional related and necessary costs that VA did not 
initially contemplate.
    The components and elements raised by IDA are reasonable 
and appropriate to consider, which is why it is standard 
practice to have a third party complete a life-cycle cost 
estimate before contracts are awarded.
    It is irresponsible that this step was skipped by VA when 
rushing through a sole-source contract in 2018, though I will 
acknowledge that none of today's witnesses, who are responsible 
for EHRM today, were part of that decision. We will discuss 
that estimate more in the second panel, and would appreciate 
VA's view of it as well.
    To date Congress has appropriated $8.2 billion since fiscal 
year 2018 to VA's dedicated account for this effort, though 
more has been spent. Of this money VA has allotted about $4.1 
billion, 40 percent of the contract ceiling, toward the Oracle 
Cerner contract, and the system has been deployed to five sites 
with 166 to go. I understand that there was a lot of upfront 
costs to the system, but would like a clear understanding of 
what that funding has bought us.
    Closures related to COVID had a huge impact on rollout, but 
that is not the only challenge this effort is facing. Of 
significant concern is how the new system has been received by 
providers. This initiative will not succeed without provider 
buy-in. It is not surprising that productivity decreased 
following rollout, but it is not clear whether it has improved 
over time.
    I understand that providers from the first rollout site, 
which deployed nearly 2 years ago, are still raising concerns. 
More alarming, are concerns that the new system is a risk to 
patient safety, and reports of repeated system degradations and 
outages. We know VA and Oracle Cerner have taken steps to 
address these issues, including training processes. And I look 
forward to hearing about how that collaboration is moving 
forward.
    Finally, I would like to discuss VA's deployment schedule. 
Last year VA took a strategic pause to assess the program and 
lessons learned. The last deployments were in June, and VA has 
postponed all planned rollouts until 2023 while still intending 
to meet the goal to be fully deployed by 2028. I am glad VA is 
not rushing deployments until there is more confidence in the 
likelihood of success, but the Department needs to be 
straightforward with Congress about what is reasonable and 
achievable. This effort is too important, and needs to succeed.
    And with that, I would recognize Ranking Member Boozman for 
his opening comments as well.

                   STATEMENT OF SENATOR JOHN BOOZMAN

    Senator Boozman. Thank you Mr. Chairman. And Deputy 
Secretary Remy, thank you for you and your team being here 
today to discuss the VA's ongoing efforts to develop and deploy 
its new electronic medical record.
    Over the last 5 years this subcommittee has appropriated 
nearly $8.5 billion towards this endeavor, and we are 
deliberating on the fiscal year 2023 request of another $1.75 
billion. At the outset we were told that this program would 
cost no more than $16 billion, and would be complete in 10 
years. In the years since VA has deployed the new system at 
only a small handful of sites; and those rollouts have been 
challenging, to say the least.
    This is precisely why we are holding this hearing today. We 
want to get a sense of where the EHRM program is today, what 
the path forward is, and what the true cost is.
    And thank you, Mr. Chairman, very much for facilitating 
this. And again, this is just a very, as so many things are 
that have to do with veterans, this is certainly a very 
bipartisan effort, just to see if we can get things moving in 
the right direction.
    Certainly, the pandemic slowed the deployment and training, 
and we must not forget that, but the repeated system outages, 
persistent patient safety concerns, and lack of productivity at 
the deployment sites indicate problems much deeper than the 
challenge COVID posed. VA took an intentional pause to 
reevaluate the program, and has brought in new leadership, and 
now Cerner has been acquired by Oracle, meaning there is a 
completely new leadership team in place.
    I look forward to hearing how the new team, on all sides, 
will work together to get this right, and get deployments back 
on track. Our second panel will have representatives from 
Oracle, the IG, and from the Institute for Defense Analyses who 
recently finished work on the first Independent Life Cycle Cost 
Estimate for the EHRM program. This estimate is very different 
than the numbers VA has promised, namely, that is based on the 
assumption that VA cannot deploy the system in the 10-year 
window it has promised. Beyond that, the estimate also shows 
the potential for costs significantly higher than the $16 
billion VA number. I look forward to hearing this new 
perspective on the program, its timeline, and the associated 
costs.
    It is always appreciated hearing from the Office of the 
Inspector General, and we value the work that you do; so Mr. 
Case, thank you for being here. Mr. Rieksts, welcome, and we 
look forward to learning more about your analysis. I also want 
to welcome Mr. Sicilia from Oracle. Oracle recently completed 
its acquisition of Cerner, meaning you are now on the hook to 
see the rest of this deployment through successfully.
    I look forward to hearing your plan to do so, and how you 
can leverage your work with other government entities to help 
accelerate the VA deployment.
    And again, thank you very much, Mr. Chairman. And I yield 
back.
    Senator Heinrich. Thank you Senator.
    Deputy Secretary Remy, you are recognized for your opening 
statement. And your full written testimony will be included in 
the record as well.

                SUMMARY STATEMENT OF HON. DONALD M. REMY

    Mr. Remy. Good morning, Senator. Good morning, Chairman 
Heinrich, Ranking Member Boozman, Senator Tester.
    Chairman Heinrich, you have already introduced the expert 
team I have here with me today, so I won't go through those 
introductions again.
    I simply want to say thank you for your consideration of 
the President's fiscal year 2023 Budget Request, and this 
opportunity to update you on VA's initiative to modernize our 
electronic health record, both so critical to veterans, their 
families, caregivers, and survivors.
    The resources this committee has repeatedly secured for VA 
and the EHRM effort reflect our close partnership, and your 
continued focus on our shared commitment to improving Veteran's 
access, outcomes, and experiences.
    In delivering world-class health care, VA adheres to the 
principles of high reliability organizations. Our fundamental 
goal, as an HRO, is achieving zero harm. Just one vet harmed is 
one vet too many.
    And right now the bottom line is that the Cerner system is 
not delivering for veterans in the ways that it should, not 
even close. It needs major improvements, whether a system 
outage lasts for one minute, one hour, or one day, that outage 
is unacceptable. So on behalf of veterans we serve, and 
providers serving them, we couldn't be more frustrated, that is 
why we are holding Cerner and ourselves accountable. That is 
why we are applying lessons learned from every deployment to 
continue to improve. That is why we have paused on all future 
go-lives until 2023 to get this right. And make no mistake we 
will get this right. We have to, for veterans, for providers.
    But these challenges cannot and will not stop us from 
modernizing the record system, because it is what veterans 
need. Our nearly 40-year-old Legacy system has served us well, 
but it has reached the end of its life cycle. Now now we need 
an EHR system where vets can access their health care records 
in one place, from the first day they put on their uniforms to 
the last day of their lives.
    A system that empowers vets to receive care anywhere, 
whether it is from DOD, VA, or Community providers, without 
having to worry about cumbersome paperwork, or potentially 
harmful gaps in their records. A system that helps providers 
understand injuries that veterans suffered 50 years ago, so 
that they can provide those vets with the best possible care, 
today.
    In other words, getting this new system right is essential 
to delivering a lifetime of world-class health care and 
benefits to our veteran population. So we are continuing to 
drive toward these goals by making sure that future EHR 
deployments reflect what we learn, with each challenge 
informing and better positioning us for the next deployment.
    During the remainder of this calendar year, and beyond, we 
are working with the contractor to ensure stability and 
resiliency of the system. We are using this interim period to 
make improvements like testing at our most complex sites, and 
adding capability enhancements to improve usability for our 
health care personnel. We are also fully engaged with past 
deployment sites, closely assessing their experiences, 
providing support, and applying lessons learned for future 
deployments, while also helping our hard-working, frontline 
users fix issues as they are identified.
    We have added a new functional champion, Dr. David Massaro 
to help lead the EHRM effort, along with Dr. Adirim; and if any 
future sites fail to meet our integrated readiness criteria for 
any reason we will not go live, we simply will not subject our 
veterans to avoidable risk.
    In short, we are continuously improving, continuously 
reviewing past deployments, and continuously holding ourselves, 
and Oracle, accountable to get this right before future 
deployments, not after. Whenever we make a decision at VA we 
ask ourselves one simple question: What does it mean for the 
veterans we serve, for their families, their caregivers, and 
survivors?
    In the case of EHRM the answer is clear, veterans need a 
system supporting our high reliability organization, delivering 
them world-class health care for decades to come, the new 
system hasn't yet achieved those goals, and we won't rest until 
it does. Until it serves veterans as well as they have served 
all of us.
    We look forward to your questions.
    [The statement follows:]
                  Prepared Statement of Donald M. Remy
    Good morning, Chairman Heinrich, Ranking Member Boozman and 
distinguished Members of the subcommittee. Thank you for the 
opportunity to testify today in support of VA's initiative to modernize 
its electronic health record (EHR) system. I am accompanied today by 
VA's experts on this initiative, Dr. Shereef Elnahal, Under Secretary 
for Health; Mr. Jon Rychalski, Assistant Secretary for Management and 
Chief Financial Officer; and Dr. Terry Adirim, Program Executive 
Director, Electronic Health Record Modernization Integration Office 
(EHRM-IO).
    I want to begin by thanking Congress and this subcommittee for your 
continued support and shared commitment to Veterans. The resources you 
have invested in VA's EHRM effort will improve access, outcomes and 
experiences for Veterans. Successful deployment of a modern EHR is 
essential in the delivery of lifetime, world-class health care and 
benefits for Veterans, as well as to set the standard for U.S. health 
care writ large. We will get this right. With a unified, seamless, 
trusted information flow between VA and the Department of Defense 
(DoD), we can further empower Veterans and their families, caregivers 
and survivors to achieve and sustain health and wellness. In addition, 
we can enable care teams to deliver best-in-class access and outcomes 
while enhancing VA's ability to innovate and advance Veteran care and 
services.
    I look forward to further engagement with you and your staffs to 
ensure that we are successful-and I assure you that we remain committed 
to full transparency regarding our deployment efforts. Veterans and 
patient safety are at the center of everything we do. In delivering 
world-class health care to Veterans, VA adheres to the principles of a 
High Reliability Organization and our fundamental goal is to achieve 
zero patient harm. To those important ends, I wanted to provide a 
program update, including what continues to be a sometimes challenging, 
but much more informed deployment and operational plan moving forward.
    Our charge has been clear: create a single, seamless, integrated 
health record for Veterans, starting with their military service days. 
This complete record within a single system allows those who care for 
the Nation's Veterans to proactively prepare for the future and deliver 
the benefits, care and services those Veterans have earned.
    This is one of the most complex clinical and business 
transformation endeavors in the Department's history. But the 
complexity and challenges associated with this effort should not deter 
us from modernizing our technology and processes. This is an 
opportunity for VA to fundamentally transform health care for Veterans 
through standardization of its operations to deliver consistent, high-
quality care whenever, wherever Veterans seek it.
    Our nearly 40-year-old legacy system has served us well, but it has 
reached the end of its life-cycle-and given its limitations, it needs 
replacing. As Secretary McDonough said, this is a leap forward we can 
and must get right.
    We acknowledge that the first deployment at Mann-Grandstaff VA 
Medical Center (VAMC) in Spokane, Washington was problematic. The 
mistakes identified in the months following the Spokane deployment are 
unacceptable. We are holding ourselves and our vendor accountable to 
get these issues resolved at Mann-Grandstaff and our other deployment 
sites.
    Since the deployment in Spokane nearly 2 years ago, VA has applied 
the lessons learned from that experience to improve future deployments. 
We conducted a Department-wide strategic review that identified patient 
safety and other areas for improvement and used these lessons to change 
our deployment strategies with a focus on reducing risk and improving 
adoption. VA is unequivocally committed to providing safe, effective 
care to Veterans.
    This EHR modernization effort is led by the EHRM Integration Office 
with Dr. Terry Adirim, as the Program Executive Director, responsible 
for integrating efforts across the enterprise-wide, to include the 
Veterans Health Administration (VHA), the Office of Management, and 
other offices. We are excited to have on board as the first Senate-
confirmed Under Secretary for Health since 2017--Dr. Shereef Elnahal, 
whose leadership of VHA will be critical to the success of this effort. 
In addition, Mr. Jon Rychalski has led VA's Office of Management for 
over 4 years and can address the updated independent cost estimate 
(ICE).
                      ehrm: the plan going forward
    Any implementation of this scale and complexity comes with inherent 
challenges. While we are working diligently to address them, we also 
know change like this can be challenging and, as such, have always 
viewed this process iteratively. We are currently in the initial 
operating capability (IOC) phase. In this phase, we are learning what 
is working, what is not working-and applying the lessons learned moving 
forward.
                          deployment schedule
    Following the 2020 Mann-Grandstaff VAMC deployment and strategic 
review in 2021, VA revised its EHR deployment schedule through the 
first quarter (Q) of fiscal year 2024. We understand, and VA has always 
made clear, that the deployment schedule is subject to change based on 
unforeseen events that may prevent a safe and successful deployment. 
This may include a determination that a site may not be ready for 
deployment due to implementation tasks not being completed on time or 
an assessment by EHRM-IO and Veterans Health Administration (VHA) 
leaders that a timeline adjustment is needed for a specific clinical 
site.
    In preparation for deployments, EHRM-IO employs a detailed 
integrated readiness criteria checklist to assess risk at future sites. 
Additionally, we now use a continuous feedback loop with deployed sites 
to capture improvement opportunities and to drive future changes at 
sites not yet deployed. Pre-deployment activities are underway in 
Veterans Integrated Service Networks (VISN) 10 and 20, as well as 
preparation activities for site deployments in VISNs 12 and 23 
scheduled later in fiscal Year 2023 and in early fiscal Year 2024.
    Demonstrating the value of the new readiness process put in place, 
VA decided to postpone its planned go-live at Boise VAMC, originally 
scheduled for July 23, 2022. This decision was based on concerns that 
the site and Cerner had not completed all the tasks on the site 
deployment readiness checklist. A new launch date for Boise VAMC has 
not been determined. We also shifted Puget Sound VA Health Care System 
(HCS), which includes the American Lake and Seattle VAMCs, from the 
original date of August 2022 to March 2023; and the VA Portland HCS, 
which includes the Portland and Portland-Vancouver VAMCs, from November 
2022 to April 2023. These decisions were based on system stability 
concerns. Moving the deployment of these larger, more complex sites 
allows Oracle Cerner more time to deliver on its commitment to 
stabilize the system and implement our top priority capability 
enhancements.
    We have paused going live at sites until 2023 to get this right. We 
are using our readiness checklist to determine their viability, and as 
always, we will adjust to ensure we are deploying a safe and effective 
EHR system. During the remainder of the calendar year, VA will be 
actively working on updates to the system, which includes testing at 
the Department's most complex facilities, as well as adding new 
capability enhancements. We are also still very much engaged with our 
past deployment sites, closely monitoring and assessing user 
experience, adoption of the new system and lessons learned.
    The full EHR deployment schedule through 2028 is currently under 
development with VHA and VA's Office of Information and Technology 
(OIT) and will be ready in fall 2022.
                        capability enhancements
    VA is committed and working diligently to resolve the challenges 
and issues identified in the strategic review, and by the Office of 
Inspector General (OIG) and the Government Accountability Office (GAO). 
We already have made progress on many of the issues identified. As of 
September 2022, we have closed 20 of the 68 OIG recommendations and are 
working with OIG to close an additional 12. However, some of the 
remaining recommendations are complex that cannot be closed out until 
the IOC phase is complete. Additionally, we are focused on ensuring 
technology stability and system enhancements, as well as on rigorous 
processes to manage budget and expenditures, aligning them with 
schedule, requirements and performance, among many other program 
improvements. Given the lessons learned from recent deployments, we 
also anticipate improving metrics, system stability, user adoption and 
training.
    In terms of capability enhancements, VA currently is focused on 
four priority areas: pharmacy, suicide prevention, research and revenue 
cycle. Some of these enhancements are above the baseline requirements 
in the original contract but are necessary to ensure that our medical 
providers can deliver care safely according to VA policy and to meet 
the unique needs of Veterans. A notable example of progress is the task 
order modification for seven pharmacy capability enhancements, which 
was awarded to Cerner Government Services on July 6, 2022. The 
preliminary timeline for development of all 7 enhancements was 13-36 
months. Oracle Cerner recently indicated that it would deliver the top 
three capability enhancements prioritized by the pharmacy community in 
6-9 months. In the interim, VA has engaged MITRE experts to evaluate 
and provide recommendations to optimize the current pharmacy process to 
reduce burden on our medical personnel.
    Another key concern among clinicians has been the visibility and 
prominence of patient behavioral health record flags. Flags are 
currently configured and available in PowerChart and FirstNet as part 
of core commercially available capabilities. However, these flags can 
be bypassed by clinicians, so we are working to enhance them--in all 
Cerner applications--to prompt clinicians to address them without the 
ability to move forward until appropriate action has been taken. In the 
meantime, staff have been trained on the workflow of accessing the 
alerts via an additional click from within those applications. We are 
in the process of adding additional mental health and patient record 
flags with task order modification award anticipated in the next 1-3 
months. Once awarded, we anticipate having the ability to add three 
capabilities in 2-4 months, and an additional 2 capabilities in 18-24 
months.
    Like other EHR systems, the Cerner EHR system includes a queue to 
capture erroneous orders. The ``unknown queue,'' is not a defect of the 
EHR, but rather how the system is designed. It functions to catch 
orders that cannot be delivered and completed so that they can be 
reviewed by staff for correction.
    The problem with the unknown queue at Mann-Grandstaff VAMC was 
related to a failure of communication, training and processes. 
Unfortunately, responsible Mann- Grandstaff VAMC personnel initially 
were not aware of the unknown queue and how to work with this feature 
when the new Cerner EHR system was deployed. Subsequent actions have 
been taken to ensure that the queue is working optimally, including 
ensuring order locations are configured properly, adjusting workflows, 
identifying staff to monitor the queue who are trained in its use, 
developing tip sheets and additional resources, among others. Almost 
all this work was completed prior to the 2022 deployments.
    VA now has a process in place for facilities to track orders in the 
unknown queue daily and to assign facility staff to correct and 
resubmit the orders in a timely manner.
    The issues discussed in the recent OIG report regarding the unknown 
queue were useful to further enhance its operation. VA has implemented 
corrective actions and reported them to OIG via a memorandum, dated 
July 6, 2022, requesting closure of the recommendations, and is 
diligently working to ensure that all facilities that have already 
deployed or are deploying in the future are adhering to the appropriate 
processes.
    Proper training is an important element of a successful deployment. 
This means providing timely tailored, well-constructed coursework that 
requires active participation. VA has taken several actions to address 
identified training concerns, including:

  --Engaging with independent consultants (McKinsey & Co) to review the 
        contents and delivery of the training program, collecting end-
        user feedback and other related data and providing 
        recommendations for improvements in the training program based 
        on industry best practices;

  --Conducting interviews on content areas of concern with super users 
        and Clinical Councils;

  --Working with EHRM-IO to incorporate feedback from listening 
        sessions with super users at Puget Sound HCS regarding virtual 
        super-user training;

  --Piloting transition of 400-level courses including Sign-On Fair/
        Favorites Fair courses to local sites' super users and/or local 
        sites' informatics teams (this is part of the strategy to 
        transition ownership of appropriate activities from Cerner to 
        VHA for long-term sustainment); and

  --Implementing ongoing training content updates based on lessons 
        learned, system changes and feedback from active EHR users.
                           system reliability
    VA continues to actively address concerns regarding system outages 
and degradations and is holding Cerner accountable. Not only are these 
episodes frustrating and disruptive to our medical personnel, but they 
potentially could put Veterans' safety at risk. We are also working 
collaboratively with DoD, the Federal Electronic Health Record 
Modernization Program Office, Cerner and Leidos to ensure stability of 
the Federal network. We are instituting prevention strategies and 
working to recognize problems earlier and improve notification 
procedures. Further, Cerner has committed to upgrading the current 
system and to the introduction of procedures for responding more 
quickly to service disruptions to ensure a better, more reliable user 
experience.
    Cerner has failed to meet the 99.9 percent service uptime Service 
Level Agreement for 7 out of the last 13 months (June 2021 through July 
2022) and the Department has received financial credits for Cerner's 
failure in meeting the contractual level of performance. To further 
hold them accountable, VA sent a second Letter of Concern to Cerner on 
August 5, 2022, reiterating our concerns and directing Cerner to 
provide their technical and operational roadmap to remedy the ongoing 
system instability issues within 30 calendar days.
    These problems put our medical professionals' ability to deliver 
safe and effective care to Veterans at risk. Cerner's failure to 
resolve the system instability issues may result in the use of other 
contractual remedies within the Government's authority.
                        new functional champion
    VHA's involvement with the EHRM program is critical to the success 
of the EHR modernization initiative. The Office of the Functional 
Champion (OFC) is VHA's representative embedded within EHRM-IO and 
engaged across the Department. The OFC will lead functional initiatives 
to support VA's medical personnel, including collaborating daily across 
VA offices and across the health system to coordinate the development 
and implementation of EHRM-related activities. OFC works closely with 
VHA to ensure that our clinical community's interests are represented 
and integrated into each facet of the program, including leadership, 
staffing, governance and deployment.
    We are pleased to have a new Functional Champion, Dr. David 
Massaro, as part of our leadership team. Dr. Massaro started on August 
1, 2022. He is board certified in family medicine and health 
informatics and will lead EHRM-IO's clinical and business functional 
efforts, including change management and training activities. Dr. 
Massaro formerly served within VHA in several executive roles, 
including Acting Chief of Clinical Informatics Operations for the 
Office of Health Informatics. He previously spent over a decade as a 
physician at VA.
    More VHA personnel are being integrated fully into OFC, including 
informaticists, solution experts and informatics patient safety 
experts.
                   budget overview and cost estimate
    In support of this effort, the President's Budget includes $1.8 
billion for fiscal Year 2023. This is in alignment with the new 
strategy, which adjusts the baseline requirements to align with VA's 
updated deployment plans. This funding is vital to support the 18 
currently proposed EHR deployments scheduled for fiscal Year 2023, as 
well as the pre- deployment activities at future sites. These pre-
deployment activities typically begin 13-15 months in advance of go-
live dates to ensure sites are equipped to receive the new EHR system.
    In fiscal Year 2023, VA currently plans to conduct EHR and 
infrastructure readiness activities at 68 sites across 7 VISNs. The 
funding will provide for:

  --EHR: Contracts for site assessments, site transitions, enterprise 
        integration and site implementation, including activities such 
        as site activation, training and workflow development.

  --Infrastructure: Information Technology (IT) and other 
        infrastructure investments, such as IT upgrades, modifications 
        to existing systems and interfaces.

  --Program management support: Government staff (e.g., salaries and 
        benefits), Government administrative expenses and contractor 
        support.

    Continuity of funding is integral to our ability to prepare sites 
for the deployment of the new EHR, and to execute VA's rollout 
schedule. By the end of fiscal Year 2022, EHRM-IO will have invested 
infrastructure readiness funding in 15 out of VHA's 18 VISNs. VA will 
also complete the vast majority of infrastructure modernization work in 
VISNs 10 and 20, and initial progress will be made in 13 additional 
VISNs. The FY 2023 budget also supports security, server stack and 
Local Area Network work at the final three VISNs, the initial set of 
infrastructure readiness items that the sites receive.
    In addition to the funding requested for the EHRM account, VHA's 
Medical Facilities request includes $505 million in Non-Recurring 
Maintenance funding for facility infrastructure projects required to 
support EHRM. Some of the projects funded by this request include: $43 
million at the Brockton VAMC, $45 million at the West Haven VAMC, and 
$45 million at the Dallas VAMC for required data cabling, electrical, 
heating/ventilation/air conditioning, and data center upgrades.
    As planned, the fiscal Year 2023 President's budget provides the 
necessary funding to prepare for and meet the deployment requirements 
at sites that will go live in fiscal Year 2024 and early fiscal 
Year2025. Thanks to the support of Congress, funding already provided 
in FYs 2021 and 2022 supports the IT physical infrastructure 
requirements essential to the new EHR. EHRM program funding continues 
to support site preparation activities, including the IT 
infrastructure, distinct from pre-deployment activities described 
above,a that must be completed 12-32 months prior to go-live and 
deployment activities to prepare sites for the new EHR system.
    In fiscal Year 2021, the VA OIG published two reports that each 
found deficiencies in the Department's Life Cycle Cost Estimate for 
EHRM and identified the need for an ICE for EHRM. In response to those 
reports, VA reviewed current and historical costs across the Department 
to ensure that, beginning in fiscal Year 2022, our quarterly financial 
reports to Congress provide a more complete picture.
    To address OIG's concern regarding the lack of an independent cost 
estimate, VA procured the services of the Institute for Defense 
Analyses (IDA) to develop an independent cost estimate that includes 
EHRM-related costs attributable to EHRM-IO, VHA and OIT among other 
costs related to the new EHR throughout the life cycle of the system. 
This estimate provides VA leadership with a neutral, independent 
assessment of potential costs to implement and operate a new EHR. VA 
facilitated briefings with key Congressional staff on the preliminary 
cost estimate in July--now that VA has received the draft final report 
from IDA, we have provided a copy to the Committee as promised.
    The four main drivers of differences between EHRM's estimates and 
IDA's are the deployment timeframe, sustainment, inclusion of 
productivity losses across the deployment and cost differences among 
existing elements of the deployment process.
    For the specific difference between VA's and IDA's cost estimates 
for EHR deployment, VA's estimate spanned 10 years whereas IDA's 
estimate covers a timeframe of 13 years. VA's estimate was based on the 
current 10-year contract. IDA's estimate of 13 years was derived from 
examining data on historical enterprise resource planning programs.
    In its estimate, IDA also includes the cost for some sustainment 
during the implementation phase plus 15 years of sustainment operations 
once the system is fully deployed. The specific sustainment cost point 
estimates in IDA's life cycle cost are $3.5 billion during the 
implementation phase and $17.1 billion during the 15-year fully 
deployed phase. VA's estimate did not include some of the costs for 
operations and support during the implementation phase nor any 
sustainment costs during the fully deployed phase.
    In total, IDA's estimate includes an estimated $25.9 billion in 
costs for elements not in scope of VA's estimate. These additional 
elements (i.e., acquisition, sustainment pre- and post-full deployment) 
account for about 75 percent of the cost difference between VA's 
estimate ($16.1 billion) and IDA's estimate ($49.8 billion).
    The remaining approximately 25 percent difference between VA and 
IDA estimates is due to IDA independently producing higher cost 
estimates for some of the elements common to both VA and IDA estimates. 
These increased costs were derived from VA actual costs and the IDA-
estimated 13-year implementation schedule. Cost increases are common 
for programs of this complexity, and prior enterprise resource planning 
programs have had similar cost increases in acquisition.
    IDA's cost estimate excluded consideration of the effects of 
sustaining our current EHR, VistA. VistA must remain operable until all 
required functionality is replaced. The total cost to sustain VistA in 
fiscal Year 2021 was approximately $841 million. We expect this VistA 
cost to continue during the deployment of the Cerner system.
                               conclusion
    Our focus is keeping Veterans at the center of everything we do and 
our top priority remains and continues to be advancing a culture of 
safety and high reliability, with the goal of zero incidents of patient 
harm. Veterans deserve high-quality health care--that means health care 
that is timely, safe, Veteran-centric, equitable, evidence- based and 
efficient.
    Thus, during the remainder of this year, we are working on ensuring 
the stability and resiliency of the EHR system and making improvements 
to the system, including usability improvements for our health care 
personnel. We are staying engaged with past deployment sites and 
providing support to our front-line personnel as well as fixing those 
issues they have identified. We are holding ourselves and Oracle Cerner 
accountable and continuing the work to deliver a more successful EHR, 
which will ensure delivery of world-class care to our Veterans.
    While modernizing VA's EHR is a fundamental change in how business 
and health care work processes are performed within VA, it presents us 
with opportunities to transform the way we deliver health care, and to 
standardize that delivery across the enterprise to achieve improvements 
in patient safety and efficiency in health care deliver. Because this 
initiative is so transformative in terms of how Veteran care is 
provided, the success of the project depends on how well we prepare and 
support the people who use it. Be assured that the resources you have 
invested in VA's new EHR system, when fully implemented, will support 
VA in delivering world-class health care and will improve access, 
outcomes and the experience for Veterans for decades to come.
    Finally, we want to acknowledge what may be top-of-mind for many of 
our stakeholders, including Members of the subcommittee. We understand 
the uncertainty this type of innovation can bring as meaningful, 
industry-shifting change often does. In a rollout of this scale and 
complexity, challenges are expected, they are inevitable, and we are 
prepared to address them. We learned much from our first deployment 
almost 2 years ago and have improved our deployment strategies. In 
fact, in the years ahead, a successful EHR deployment must reflect what 
we have learned, with each challenge helping to better inform and 
position the next deployment.
    I again extend my gratitude to Congress for your continued support 
and shared commitment to serving Veterans with excellence. With your 
continued support, VA will realize the full promise of a modern 
integrated health record to cultivate the health and well-being of 
Veterans. We are happy to respond to any questions that you may have.

    Senator Heinrich. Thank you, Deputy Secretary. We will 
proceed with questions using the standard five-minute rounds, 
and senators will be recognized in the order that they arrive. 
I will start by recognizing myself for five minutes.
    Deputy Secretary Remy, do you have any thoughts or 
impressions you can share with us about the life cycle cost 
analysis that was provided by IDA? Does it align with what you 
are seeing, or whether you have concerns with parts of their 
approach? Either way, just give us your honest impressions.
    Mr. Remy. Mr. Chairman, thank you for the question. Indeed, 
it is important for us to have this Independent Life Cycle Cost 
Estimate, and we thank the Office of the Inspector General for 
pointing out the need for a VA to do that. Having this 
information is a data point that will help us build toward the 
future. There are differences and distinctions between our 
program office estimates and this life cycle cost estimate.
    You mentioned them earlier that it goes for 3 years longer 
than our deployment schedule of 10 years currently does, that 
it allows for 15 years the same tail at the end, and it also 
covers other potential costs based on risk assessments that IDA 
did when they were performing the life cycle cost estimate. So 
we recognize that those are differences and distinctions, but 
we believe that the information in the estimate is valuable to 
us, as we look toward the future.
    Senator Heinrich. How much does it cost annually for the 
Department to maintain and update the VistA System? And am I 
correct in assuming that these costs will be eliminated once 
the Cerner record is fully deployed across the system?
    Mr. Remy. Mr. Chairman, we estimate that amount to be 
around $800--$900 million on an annual basis to maintain the 
VistA System. Understand, however, that VistA is integrated 
with many other components of our IT systems, our financial 
management system, and other systems that rely upon VistA as a 
component of their operation. So as we modernize, we will be 
modernizing those other systems as well, but immediately we 
would not be able to draw down the usage of VistA because it is 
integrated with other things that we are using.
    Senator Heinrich. So tell me about how those costs will be 
impacted as the Cerner record is fully deployed? Do you see a 
transition there as opposed to a cliff? Talk to us about how 
you are going to integrate that.
    Mr. Remy. Sir, I believe the way you have described it is 
accurate, that there will be a transition. I can't here say 
that we can identify, and quantify any specific cost savings 
that will result over time, because of the integration of the 
VistA System with other components of our technology.
    Senator Heinrich. This committee has requested, both in 
meetings and in our annual bills, metrics to measure whether 
this EHR implementation is good for individual veterans, or is 
a good use of taxpayer dollars. And while VA has shared really 
hundreds of technical metrics, we are still not entirely sure 
what constitutes success. So by what measures should we be able 
to tell whether the Department is succeeding?
    Mr. Remy. Starting with what you have just described, is 
our veteran experience. We have engaged in this endeavor to 
enhance outcomes for our veteran population, and we want to 
make sure that our clinicians have the tools to achieve that. 
And so we have engaged in a process to gather information from 
our clinicians and our veterans to understand what needs they 
have that aren't being met by the system, and how we can go 
about meeting those needs.
    Now, that is a general statement not a specific KPI, if you 
will, to measure success. Success at the end of the day though, 
is measured by the performance of the system to achieve that 
outcome enhancement for our veterans, and that is what we are 
looking toward. When you look at the system's operations and 
you understand one of the things that is obligated in the 
contract for our contractors to provide, is that the system be 
operable 99.9 percent of the time.
    I mentioned in my opening remarks some frustrations around 
outages and degradations; that is one of the measures that we 
look at to determine: Is the system reaching the people that it 
needs to, when it needs to?
    Senator Heinrich. VA always emphasized effective change 
management as key to really successful implementation, yet the 
provider feedback to the initial rollout has been quite 
negative. As the shift from VistA is happening, what is VA 
doing to support clinicians across that transition?
    Mr. Remy. Mr. Chairman, we are arm in arm with those 
clinicians. We recognized that the initial rollout was not a 
rollout that had all of the components that it should have at 
the time, and presented a significant number of challenges. And 
so we have learned from that, and what we are doing now is to 
make sure that we have at-the-elbow support for our clinician 
community, our provider community, as they use this system at 
each site that is deployed.
    I know we are almost out of time, but if you will, I would 
like to ask my colleagues to add to that response. Dr. Adirim.
    Senator Heinrich. Please, Dr. Adirim.
    Dr. Adirim. Yeah. Now, and I think the Deputy Secretary 
pointed out a very important thing that we did in the 
subsequent four deployments, was ensuring ample at-the-elbow 
support with an adequate amount of time for our medical 
personnel to learn the system. But we have done a number of 
other things that I learned while at DOD, DOD being very 
successful, almost done with their CONUS deployment, that 
included ensuring that we have local leadership engagement, and 
we do this in multiple ways.
    We do this several weeks before deployment engaging 
leadership to leadership, ensuring they have what they need, 
and that they are fully activated to help their people get 
through the deployments.
    We have also started, and we have already done two, and we 
have four more planned, called VISN Director Workshops well in 
advance of deployment, so that site leadership and deployment 
leadership understand what is expected of them and what they 
need to do in order to prepare their medical personnel. So we 
have learned a lot, not only from VA's first experience, but 
also from other health systems, how we support our personnel.
    And lastly, VHA did something really smart, they activated 
their VISN Clinical Resource Hubs to this effort, as well as, 
centrally, they have EHRM, they called the NESSU, which is the 
Supplemental Staffing Unit, to provide support to operations, 
whether it is virtual or onsite, for nursing, mental health, 
and so on, during this post-go-live period to ensure that 
operations can be maintained and support our veterans as we are 
going through this.
    Senator Heinrich. Great. Thank you, Doctor.
    Ranking Member Boozman has very graciously allowed Senator 
Tester to go next.
    Senator Tester. And I want to thank Senator Boozman. And I 
want to thank you, Mr. Chairman, also, for the opening 
statements. And I want to thank both of you for your leadership 
on these issues.
    As you guys were giving your opening statements it occurred 
to me that not everybody is focused on what is going on with 
electronic health records in the VA. Not everybody in the 
Senate it is a top priority, but I can tell you for the two men 
in my left it is a top priority. And I would say the same thing 
for Senator Moran and myself.
    Getting this right is really important. The contract was 
awarded in May of 2018, and we are, depending on what the 
numbers come in at the end of the fiscal year, somewhere 
between $7- and $9 billion into this outfit. And to be honest 
with you, from where I sit, I don't know that we have got a 
return on investment to speak of at all.
    And so we have got long ways to go. And as I think I have 
told you, Mr. Remy, if I had all the money that was 
appropriated for computer programs for my time in the State 
Legislature and here, we would make a serious dent in the 
national debt, truthfully. We have got more work to do. And 
even though you guys, as Senator Heinrich has pointed out, 
weren't the people here when this contract was awarded, it is 
your responsibility now to either figure it out and move 
forward, or figure out a different way to go.
    So look, the VA does not have any more installs planned for 
the rest of this year, and I think you have got a plan to 
restart in 2023, and then during that time you are working 
along with the folks from Oracle Cerner to fix any issues that 
are out there to make this thing work, so you can launch it 
somewhere else.
    Deputy Secretary Remy, what data-driven set of metrics are 
you going to use to decide whether to take this system into 
more facilities? That is a little different question than 
Senator Heinrich just asked. But what are you going to be 
looking at to say, all right, we are ready to fly, we are going 
to move out?
    Mr. Remy. Thank you, Senator. And you we have a Site 
Readiness Deployment Checklist that we have developed 
alongside, we being are VHA, an OEHR, or EHRM-IO, along with 
our contractor that spells out the types of things that will 
need to be done before we can go live.
    And I can speak to some of those things. They include 
things like training, which is critically important for those 
people that are going to be using it, infrastructure, patient 
safety protections, and those types of things that we go 
through before we determine that a site is ready to go live. 
And this is an iterative process. It is not a static process.
    As we are working with a site for potential deployment, we 
work through these issues to make sure that they have them 
covered. An example of the effectiveness of that checklist was 
Boise, recently, where we determined we wouldn't go live as we 
were going through the checklist.
    Senator Tester. Okay. So as you are looking at this 
situation right now, and as we don't plan any installations 
until 2023, so I am going to ask you an unfair question: When 
do you anticipate, in 2023, that you are going to be able to 
implement and install?
    Mr. Remy. Well, we have a schedule, but we are looking 
closely at this schedule, and we are realizing that there are 
issues that need to be resolved before we can go live. Right 
now our schedule would have us go to another installation in 
early-2023, January/February. If we have to push that back, we 
will push that back.
    Senator Tester. So the question is, and I get it, and I 
don't want anything implemented before it is ready for prime 
time, on the same token we have got an investment in a program 
that needs to start delivering at sometime. And so are the 
problems with the program so intense that it is going to take 
months, and months, and months to solve them?
    Mr. Remy. Some of the problems are challenging, and I mean 
we talk about people processes and technology, and making sure 
that we have each of those categories right as we move forward. 
We have learned things from prior deployments that will help us 
to enhance the system, so when we roll it out the next time we 
can roll it out in a way that can be more successful. We are 
learning things from our recent deployments, even, that that 
can help us move forward.
    So I don't know that I would necessarily, sir, say that the 
problems are that intense, but the problems are real enough 
that we are not moving forward until we resolve some of them.
    Senator Tester. I got you. I just, I am concerned. We are 
into this damn near 5 years, it will be 5 years in May, and we 
are still, truthfully, I mean we are still, and we haven't done 
a damn thing. I mean we have implemented, and they have been a 
train wreck, in my opinion. And so the question really is here, 
and by the way, when Cerner comes up on the next panel it is 
the same thing, you guys have to be working really, really 
close with these folks, they have to be held accountable, they 
have to hold themselves accountable, you have to hold 
yourselves accountable. And we need to get this damn thing 
done. Because it is really hard for me to go back to the people 
anywhere and say, you are spending my money really well back 
there. Okay? You got the drift?
    Mr. Remy. Yes.
    Senator Tester. All right. Thank you.
    Senator Heinrich. Thank you, Senator.
    Ranking Member Boozman.
    Senator Boozman. So Senator Tester, I think the Secretary 
is playing in the football game tonight.
    Senator Tester. He is?
    Senator Boozman. I think that is tonight, isn't it?
    Senator Tester. It is
    Senator Boozman. Well tell him, from Senator Tester and I, 
that if he gets injured he still has to come and testify.
    Mr. Remy. Well, his testimony is before the football game, 
sir.
    Senator Boozman. Oh, that is right. Very good; well, we 
will talk to him and wish him good luck. I don't know who 
talked him into that, but that is tough deal.
    Secretary Remy, we all agree that the rollout as a rollout 
has fallen short, far short of expectations. Congress has 
appropriated $8.5 billion over the last 5 years. This year's 
request another $1.75 billion. We have talked about that at 
length.
    I guess my question is, you know, we are in a situation 
now, as both Senators Heinrich and Tester alluded to, we have 
only deployed in five locations that still are struggling, and 
I guess the question is: How can we work together to get back 
on track? You know, what do we need to do to help you? And then 
again, you know, what do you all need to do, and the 
contractors, to get us in the situation that we need to be?
    So I guess my question would be a couple things right away. 
Do you have the right balance of support staff necessary for 
the project to be successful? And what efforts is VA 
undertaking to identify lessons learned and ways to improve the 
deployment to other sites? How much of the change management 
work is done by VA employees? How much by contractors? What 
specific efforts are underway to enhance your change management 
initiatives, and to ensure your personnel embrace the new 
system? And what is VHA doing to underscore with clinicians the 
Department's full commitment to EHRM? In turn, how are VAMC 
directors and facility personnel held accountable for 
implementing Cerner?
    That is a lot. But can you kind of chip away at that a 
little bit?
    Mr. Remy. I can, Mr. Boozman. And let me start with the 
last point first. And if you will, I would like to ask Dr. 
Elnahal to speak to VHA's commitment. One of the important 
things about having a confirmed Under Secretary is to have the 
messaging delivered to his community, if it is acceptable.
    Dr. Elnahal. Thank you, sir. And thank you, Senator, for 
the question.
    So as the Deputy Secretary mentioned in his opening 
remarks, and as the Secretary has mentioned recently, I do have 
deep concerns about the system as it is functioning for 
frontline employees and service to veterans. I had a chance to 
see that, myself, in a visit to Columbus, Ohio, recently, a 
really great leadership team there who wants to get this right. 
And even more importantly, a great set of frontline folks who 
were in front of veterans trying to use the system, and I 
actually saw them in action using the system.
    And what I will say is, my commitment, as again, the Deputy 
mentioned, is to actually get to a modern electronic health 
record that meets the clinical needs of our veterans, and right 
now I saw folks struggling with the system deeply.
    Among the most concerning things that I saw was a 
phenomenon whereby our frontline clinicians when they put it in 
an order, or trying to interface with the system, they were not 
confident in many cases, and in many clinical settings, that 
those orders were actually getting where they needed to go on 
behalf of their veterans. And there was a lot of manual rework, 
recheck that had to be done to meet those veterans' needs.
    What I took from that, was a need to come back, speak to my 
colleague, Dr. Adirim, Deputy Secretary, about ensuring that 
that is the case going forward by doing a deep investigation 
into that problem, but also to do a broader look at how these 
workflows can improve the configuration of the system against 
those clinical processes that we need to do for vets that also 
needs to improve.
    I will say that there was a nucleus of folks at that 
facility as well, in some areas, who did find promise in the 
system. In the urgent care setting, some of the surgical, 
medical staff said that they appreciated it and found promise 
in it. So I think we can get there. But right now I did not see 
a system that was meeting most of the frontline clinician's 
needs. And that concerns me. And I know it also concerns the 
Deputy. And we are doing what we can to try and address those 
issues.
    Senator Boozman. So as a provider, you know, looking at the 
system you said that there was concern about the orders 
actually getting where they needed to go. What effect would 
that have on a patient, on patient care? Can you give an 
example?
    Dr. Elnahal. Well, one thing I will say, Senator, is that 
this is not a new issue, per se, this is something that was 
discovered in the circumstances around the unknown queue, and 
some other issues. What I was surprised by was the frequency by 
which I was seeing that, and the number of staff who mentioned 
that to me.
    And so what that meant was, that is a call to action for us 
to configure this system in a way where the workflows are 
intuitive, where clinicians, for example, don't necessarily 
have 10 to 20 choices for any given order or action in front of 
them. Looking at our training and ensuring that where the 
system can be configured better we can do that, but also that 
we have a confirmation that the broad swath of employees being 
trained know what to do in front of a veteran.
    And so those issues we have to focus on, Senator. In full 
transparency, those things absolutely we need to improve.
    Senator Boozman. Okay.
    Mr. Remy. Senator, I know you had a list of questions, and 
we are over time, I am happy to touch on the staffing and 
change management, if you would like.
    Senator Boozman. Yes sir, if you don't mind.
    Mr. Remy. Absolutely. I mean, one of the things that flows 
from what Dr. Elnahal described is: How do you get a workforce 
to embrace change management in an environment where they see 
challenges? And the answer to that question is to provide them 
with the tools to overcome those challenges, to recognize that 
the outcome of the hard work that they put in to making sure 
that this system works is going to be improved outcomes for the 
veterans that they service.
    And so what we have attempted to do is to provide them with 
those tools, to understand the challenges when they present, 
and to take swift affirmative action to resolve and remediate 
any problems that might arise. And we are doing that through 
our staff, through our governance process for the EHRM program, 
and through our work with the contractors, with Oracle and 
Cerner, to make sure that they have those tools to get past 
those challenges.
    In terms of staffing, we are in the process of hiring; Dr. 
Adirim can speak to some of the staffing activities that are 
going on in the EHRM Integration Office. Similarly, I mentioned 
earlier that we have just brought on a new functional champion 
who is our linchpin with the VHA, and there is some hiring 
going on in VHA as well, to make sure that we have people that 
are available to do the work.
    And then lastly, you ask about the role of contractors, 
whether that be the contract that we have with Cerner, or other 
contractors, and we utilize those contractors to assist us in 
delivering the system through training, and other activities 
that can help bolster our workforce. But the core of the work 
that is being done is being done by our team at VA.
    Senator Boozman. Thank you. So initially, we predicted 10 
years. I don't think we can get that done. Do you have a 
somewhat concrete figure as to how long it is going to take us 
to actually get this thing?
    Mr. Remy. Well, sir, the original plan was 10 years, as you 
mentioned, and we have been working feverishly to try to make 
that target. We have made some adjustments in the out years to 
the time for pre-deployment, deployment activities that might 
shorten that based on lessons learned to be able to meet the 
10-year mark.
    IDA, in their cost estimates, says they believe that 
initial deployment time period is 13 years. And we have been 
looking at, if we have to go beyond the 10 years, what does 
that look like? We don't have here today a specific time period 
beyond the 10 years because we are still looking at all the 
factors that might present, so that we have to go beyond the 10 
years, if we do, nor do we have a dollar amount attributable to 
that, but I can assure you we are looking at what it would take 
to make sure that we deploy the system in a safe, effective 
manner for our clinicians, and our veterans, and if that needs 
to go beyond 10 years we are working through the process of 
determining what that time period might be.
    Senator Boozman. So let me ask you this. I will note that 
the VA has not yet allowed IDA to publicly release their 
independent cost estimate, we were told we can expect it to be 
released in mid-August, and now it is nearly October. It is 
challenging to hold an open hearing where a significant part of 
the discussion is about a document that you won't release. When 
can we expect this estimate to be released to the public?
    Mr. Remy. Yeah, I understand your question. And we have 
provided the full document to the committee. It is a 
competition-sensitive document that we would have to pay 
attention to if we were to release it more broadly. But I can 
ask Mr. Rychalski to respond to that because he has had 
experience with this type of thing before. Jon?
    Mr. Rychalski. Okay. So it is. I guess it is, right now it 
is a legal procurement matter. One thing I will say is coming 
out of DOD, they would not release the cost estimates publicly, 
I think through FOIA requests, they would allow people to come 
in and look at pieces of it for that reason. So what I would 
say is, it is the legal and procurement communities are looking 
at it now to decide if it can be publicly released, or released 
kind of how DOD does it, which is a little bit more limited.
    Senator Boozman. Thank you, Mr. Chairman.
    Senator Heinrich. Senator Tester.
    Senator Tester. I have got a real quick question for you, 
Dr. Elnahal. And thanks for being here. It is good to have you 
on board. You talked about the Columbus visit. Health care 
people are under a lot of pressure just doing their job, 
period. This adds another level of stress. Are you concerned 
about VA burnout, employee burnout?
    Dr. Elnahal. What I will say, Senator, is that is 
definitely in play, in the medical centers who have done this. 
And it was something I personally observed when I went to 
Columbus.
    Senator Tester. You personally observed burnout?
    Dr. Elnahal. I personally folks telling me that the system 
was stressful to use, and leadership was telling me that folks 
were leaving, in part because of the difficulty of the 
workflows. Now, that said I will just put this into context. 
You know, EHR deployments, more generally, make things 
difficult in their initial phases. And we are in the IOC phase. 
But nonetheless that is a phenomenon that was communicated 
clearly to me when I was at Columbus.
    Senator Tester. Do you, okay so change is hard, and they 
are difficult to implement; is there any way that you can, or 
do you have plans for trying to deal with this upfront with the 
next rollout?
    Dr. Elnahal. Well, I will mention that what Dr. Adirim 
mentioned around the clinical support teams, from our clinical 
resource hubs, the supplemental staffing units that we send are 
helping. What we have to do, ultimately, that will not only 
impact burnout, but more importantly what the veteran 
experience is, is configure the system in such a way that is 
intuitive to our frontline clinicians, and allows the system to 
get the job done for the clinical needs of veterans.
    Senator Tester. Do you think that is possible?
    Dr. Elnahal. I do, Senator. I do think it is possible. I 
think it will require a significant amount of work, but we can 
get there.
    Senator Tester. All right. Thank you. Thank you, Mr. 
Chairman.
    Senator Heinrich. Thank you for your question, Senator. I 
think that will wrap up our first panel for now, and we will 
allow our second panel to get situated.
    Thank you all very much, for testifying today.
    Mr. Remy. Thank you, Mr. Chairman. Thank you, Senators.


 
                                PANEL II

    Senator Heinrich. Our second panel today is intended to 
provide the non-VA perspective.
    And for that discussion I would welcome, Mike Sicilia, 
Executive Vice President of Industries for Oracle, who is 
overseeing Oracle Cerner's efforts; Brian Rieksts, of the 
Institute for Defense Analyses, or IDA, who oversaw the 
development of the Independent Life Cycle Cost Estimate; and 
David Case, VA's Deputy Inspector General who oversees work 
related to VA's EHRM effort, including inadequacies of VA's 
initial life cycle cost estimate and patient safety concerns.
    We appreciate all of you being here today to discuss your 
roles in reviewing this initiative. I am going to recognize 
each witness for five minutes for your opening statements, and 
your full, written testimony will all be included in the record 
today.
    We will begin with Mike Sicilia of Oracle. Mr. Sicilia.
STATEMENT OF MR. MIKE SICILIA, EXECUTIVE VICE 
            PRESIDENT, INDUSTRIES, ORACLE
    Mr. Sicilia. Chairman Heinrich, Ranking Member Boozman, and 
Members of the Committee; thank you for the opportunity to 
speak with you today.
    As you probably know, approximately 4 months ago Oracle 
acquired Cerner, along with its VA EHRM program. As I stated in 
my written testimony, I hope you will agree, we believe our 
acquisition will reinvigorate this program and deliver on the 
promise of a single, longitudinal health record, from 
enlistment, and active-duty service at DOD, through retirement 
and elder care at VA.
    We believe we can deliver a system that will leapfrog 
existing commercial EHRs and deliver for our service members 
and veterans the gold standard for modern health care 
technology. In the process, we are confident that delivering a 
modern EHR system will improve patient outcomes through 
analytics and machine learning, all while enabling advanced 
care delivery channels like telemedicine and mobility.
    Importantly, for this committee, we believe we can deliver 
this system within the existing budget envelope envisioned for 
the current program scope without the need for any additional 
funds. To date Oracle Cerner, Millennium EHR is fully deployed 
for the Coast Guard, and is deployed at more than half of DOD 
medical facilities serving over 200,000 end users. I think a 
fair assessment is that the Coast Guard and DOD deployments are 
on track and proceeding successfully.
    At the VA the story is a little different. Millennium is 
deployed at five medical centers and their associated 
facilities, and work is underway at over 40 medical centers for 
their upcoming deployments. Some of the delays are pandemic 
related, but it is also true that there have been technical, 
functional and training challenges at VA facilities. As we have 
examined the underlying causes for these delays and challenges, 
our conclusion is that we have found nothing that can't be 
addressed in reasonably short order to get us back on workable 
schedule, and within budget.
    We know we have a lot to prove with deployments next year 
at larger more complex sites. We view the next year as a key 
window for building momentum and turning the corner just as we 
have done at DOD where initial adoption was also challenging 
and is now proceeding apace and with strong results.
    We recognize that training must be improved, and have 
recently engaged a third-party firm to evaluate the current 
training program. We are announcing today that we will be 
engaging Accenture to work with us to make the training much 
more efficient, applicable, and useful for caregivers.
    We are also committed to communicating timely and accurate 
information about our progress. This week we launched a 
dashboard that catalogs our to-do list and progress being made. 
We encourage you to view the dashboard frequently, and hope it 
will keep everyone, including us, focused on deliverables and 
dates.
    I have already alluded to cost, but let me be clear about 
value. We intend to rewrite the Millennium EHR as a stateless 
cloud application which will deliver a modern user interface, 
ease of use, mobility, voice recognition, and self-service. It 
will have machine learning based clinical decision support, and 
analytics that are built in from the ground up. We intend to 
deliver a beta of this new system in 2023, and we commit to 
deliver it across VA as a cost-free upgrade under the current 
contract.
    To be clear, this is a 10-year, $10 billion contract that 
already has had 700 million, or so, in additional funding due 
to additional scope. That is $10.7 billion. Even with delays, 
and perhaps an extended timeline for deployment. That is the 
number we intend to deliver the existing system for including 
rewriting the system for the cloud.

    [Clerk's Note: The following Addendum was received for Mr. 
Mike Sicilia]
     Addendum to Testimony of Mike Sicilia, EVP-Industries, Oracle
    Thank you for the opportunity to participate in the Senate 
Appropriations Committee, subcommittee on Military Construction, 
Veterans Affairs and Related Agencies hearing on ``VA's Electronic 
Health Record Modernization: An Update on Rollout, Cost and Schedule'' 
held on September 21, 2022. As VA's new EHRM partner, I appreciated the 
opportunity to update the subcommittee on Oracle's commitment to the 
program and plans to get it back on track and keep our costs in line.
    In my testimony I committed that Oracle intends to deliver the EHR 
system across the entire VA for the amounts contemplated in the current 
contract under the current scope, barring big new requirements being 
added by VA. I reiterate that commitment today, but also wish to 
clarify that in my comments I did not mean to imply the contract 
ceiling had been raised or is being contemplated to be raised from $10 
billion to $10.7 billion. It of course has not and remains unchanged 
from the original IDIQ contract award in 2018. We take seriously our 
responsibility to be good stewards of taxpayer dollars and will 
continue to look for opportunities to bring efficiencies and cost 
savings to this program.
    Thank you.

    The most important changes Oracle can bring to this 
program, in the short term, relate to improving the EHR 
system's performance and modernizing its technology.
    On August 4th in Kansas City we hosted a Federal Leadership 
Summit to discuss performance and stability issues. Coming out 
of that meeting we have a two-fold plan. First, we are working 
on more than 40 different technical operational improvements 
that we expect to lead to improve performance and greater 
stability for the Federal enclave.
    Second, at the end of July, I announced our intention to 
move the Cerner application at our own expense, and with of 
course Federal approval, to a modern hyperscale cloud data 
center, which will deliver a foundation for better performance 
and stability for end users.
    And last, we are intently focused on improving 
functionality, fixing problems like the unknown queue, or other 
items listed on the dashboard, and working with VA National 
Councils where we believe changes can be made to improve 
clinical efficiency and productivity in the short term.
    The changes Oracle is bringing to the EHRM program are 
significant, and 4 months into owning Cerner, we are confident 
that the problems are being, and will continue to be fixed, 
while at the same time we focus on delivering a far superior 
and modern Federal EHR on an aggressive time schedule.
    We look forward to delivering for our Nation's veterans. 
Thank you.
    [The statement follows:]
                 Prepared Statement of Mr. Mike Sicilia
                              introduction
    Chairman Heinrich, Ranking Member Boozman and members of the 
Committee, thank you for the opportunity to speak with you today about 
the Dept. of Veterans Affairs' (VA) Electronic Health Record 
Modernization (EHRM) program and Oracle Cerner's Electronic Health 
Records (EHR) system (Millennium).
    I am Mike Sicilia, Executive Vice President for Industries at 
Oracle. I am responsible for Oracle's Global Health Business Unit, 
including Oracle Cerner.
    Today is my first appearance before your Committee, so I want to 
begin by committing to you to be completely candid and transparent 
about where we are, where we are going, and what changes Oracle can 
bring to the Federal EHR.
    I also want to thank you for your past and current support of the 
EHRM program. We expect any program of this magnitude to receive 
intense scrutiny and oversight, and we understand concerns about delays 
and costs, which I will discuss further in my testimony. As VA's new 
partner in this effort, we have committed to getting the program on 
track and keeping our costs in line. In the coming months we hope to 
demonstrate to the Committee that we can deliver for the VA and our 
Nation's veterans so that we will continue to earn your support for the 
program.
    Oracle is a leading enterprise software vendor with more than forty 
years of experience building and developing some of the most advanced, 
mission-critical, secure and performant technology around the world for 
governments, critical infrastructure, and commercial enterprises.
    Oracle employs over 160,000 employees with more than 50,000 
developers and engineers, and in the last 10 years we have spent more 
than $56 billion on research and development. Oracle holds more than 
18,500 patents worldwide. Oracle is in both the infrastructure business 
with the world's leading autonomous database as well as the 
applications business with a full suite of high-performance enterprise 
applications across all industries.
    Oracle is also a leading hyperscale cloud service provider with 
global reach across industries and governments. Of added importance 
here, Oracle operates fully certified government cloud regions under 
the Intelligence Community's Commercial Cloud Enterprise (``C2E'') 
program and is fully qualified under the DoD's upcoming Joint 
Warfighter Cloud Capability Program (``JWCC'').
    Oracle is also a leading cloud applications company with Software 
as a Service (``SaaS'') products across Enterprise Resource Planning, 
Human Capital Management, Supply Chain, and Customer Experience as well 
as industry specific cloud applications ranging from pharmaceuticals to 
banking and retail to utilities. Our systems are performant, scalable 
and secure, and there is nothing in the Federal EHR scope outside of 
our core capabilities.
    Importantly, we have a demonstrated track record rewriting 
extremely complex applications from client-server technology to new, 
modern, stateless web applications which we plan to do here as well.
    As you know, this past June Oracle completed its acquisition of 
Cerner. With this acquisition we brought together one of the world's 
most formidable and capable infrastructure and applications companies 
with one of the leading healthcare applications companies. Oracle's 
engineering expertise together with Cerner's clinical expertise is a 
very powerful combination that we believe will serve our Nation's 
veterans and their caregivers extremely well.
    Our rationale for acquiring Cerner was straightforward. Healthcare 
IT in this country and around the world is significantly out-of-date 
and the associated health data is stove piped. Systems are aging 
without any hope of bringing modern technology innovations like 
analytics and machine learning to improve patient outcomes. Compared to 
banking, telecommunications, transportation, utilities, or any other 
mission critical sector, healthcare IT is furthest behind the 
modernization curve. Across the healthcare industry, EHR systems are 
often bespoke and running on-premises. The VA's current system--VistA--
is certainly one example of extremely old technology incapable of 
bringing state of the art capabilities to our Nation's veterans.
    Our intention is to lead the way with a new generation of modern, 
cloud-native, highly performant and secure EHR applications embracing 
mobility, self-service, analytics and ease of use, including virtual 
care, such as telemedicine, leading to better patient outcomes. Better 
patient outcomes inevitably lead to reduced health care costs. And 
modernization provides a rare chance to ensure caregivers are enabled 
by the tools they use, decreasing the administrative burden on 
caregivers. Unlike Cerner alone, Oracle brings an order of magnitude 
more engineering resources and scale to this formidable challenge and 
opportunity.
    While Oracle is new to the EHR business, Oracle does have years of 
experience advancing medical research, powering clinical trials, 
reducing healthcare costs and providing public health authorities and 
policymakers with essential data to improve public health. During the 
COVID-19 pandemic, Oracle was honored to collaborate with the Centers 
for Disease Control (CDC) and the National Institutes of Health (NIH) 
to support COVID-19 related systems of record. We assisted in 
electronically pre-screening over six hundred thousand individuals 
willing to participate in COVID-19 vaccine clinical trials and then 
supported the CDC with the creation of the v-Safe After Vaccination 
Health Checker and the Health Partner Ordering Portal (HPOP) to support 
the distribution of vaccine, diagnostic and therapeutic supplies.
    Finally, we worked with the CDC to build a national data repository 
for COVID-19 vaccination data in the U.S. We are accustomed to rising 
to the moment to handle large, complex tasks when our Nation needs it 
most.
    I give you all this introduction so that you understand our 
acquisition of Cerner and assumption of its EHRM contract with the VA 
is well within our capabilities, given our size, expertise, and 
resources.
    You should consider that in effect VA, the Department of Defense 
(DoD) and the Coast Guard obtained a new, vastly more resourced 
technology partner overnight to augment Cerner. We also strongly 
believe in this mission and consider it not only a contractual 
obligation but a moral one to improve healthcare for our Nation's 
veterans and their caregivers. We intend to exceed expectations.
    The EHRM program is the largest health IT modernization project in 
history. The point of the program is three-fold: First, to offer a 
single interoperable longitudinal health record from a person's 
enlistment with DoD through lifelong care at VA. Second, to replace the 
current system conceived in the 1970s and deployed in the 1980s that is 
plainly and objectively past its prime. And third, to improve patient 
outcomes by deploying modern technologies across the entire data set of 
active duty and retired service members to ensure the highest quality 
of care. Doing so would literally leap-frog the commercial, private 
care systems and put DoD and VA at the leading edge of health care 
technical innovation.
    When fully deployed across the VA healthcare system, 171 medical 
centers will go from using 130 different instances of the current VistA 
EHR to using a single enterprise-wide EHR that is shared between VA, 
DoD and Coast Guard. Problems currently experienced with VistA go away, 
such as: data silos creating gaps in patient data and care, veterans 
being tethered to specific medical centers, non-standardized workflows 
across the VA healthcare system, and a patchwork of VistA instances 
that inhibits the full potential of data analytics. These benefits of a 
modern EHR remain an extremely important goal worth protecting as they 
deliver significant improvements in healthcare services and outcomes 
for our Nation's service members and veterans.
    To date, the Millennium EHR has been fully deployed for the Coast 
Guard and is deployed at more than half of DoD medical facilities 
serving over 200,000 end users. I think a fair assessment is that the 
Coast Guard and DoD deployments are on track and proceeding 
successfully.
    At VA the story is a little different. Millennium is deployed at 
five medical centers and their associated facilities, which is far 
slower than anticipated, and work is underway at over 40 medical 
centers for their upcoming deployments. Some of the delays were due to 
the COVID pandemic, but it is also true that there have been 
substantial technical and training challenges at VA facilities. As we 
have examined the underlying causes for these delays and technical and 
training challenges, our conclusion is that we have found nothing that 
can't be addressed in reasonably short order, in part because we have 
shifted Oracle's top talent to work on the DoD/VA EHR system as the 
company's number one priority.
    We believe this moment is a unique opportunity to leapfrog VA into 
the future and make VA's EHR the gold standard for EHR modernization 
globally. But I also need to be clear that modernization requires 
change. There is no amount of money and no amount of work that can 
transform VistA into a modern system capable of delivering for our 
Nation's veterans.
    We can and will do our part to deliver the most performant, 
feature-rich technology within the existing budget envelope for the 
current scope of work, but caregivers must also invest in training and 
learning a new system. Because a system is different does not make it 
unworkable. I am highly attuned to over-worked caregivers who are being 
asked to invest in learning a new technology without an immediate 
improvement in their day-to-day work. But I am also highly attuned to 
what a modern longitudinal health record with all health data in the 
same system can do to improve the care for our Nation's veterans.
    Here is how we plan to address the current problems and get this 
system back on track:

    Performance: On August 4, 2022, in Kansas City, we hosted a Federal 
Leadership Summit with VA, DoD, the Federal Electronic Health Record 
Modernization office (FEHRM) and Leidos to discuss the Federal EHR 
system's performance and recent stability issues. Coming out of that 
meeting, we have a two-fold plan.

    First, we are working on more than 40 different technical 
operational improvement projects for the Federal Enclave that we expect 
to lead to improved performance and greater stability. We have made 
progress already, completing four projects and expecting seven, 
potentially eight, more to be complete by the end of the year. As we 
work on these projects, we will continue to analyze the system and make 
other fixes as needed. These plans are detailed in a letter we sent to 
VA on September 2, 2022, and which is attached for your reference.
    Second, at the end of July I announced our intention to move the 
Cerner application--with, of course, the approval of VA, DoD and Coast 
Guard--to a modern, hyperscale cloud data center, which will deliver a 
foundation for better performance and stability for end-users.
    Once achieved, this move will provide a scalable, modern platform 
for us to deliver the kind of modern technologies users have come to 
expect like mobility and predictive analytics. This is the same 
Generation 2 Cloud infrastructure that underpins Oracle's customers' 
most critical workloads in sectors like Financial Services, 
Telecommunications and Utilities.
    Another advantage of moving the EHR system is that our cloud is a 
second-generation cloud with security built-in from the start. 
Infrastructure security patches are applied automatically with no 
downtime, removing the possibility of human error which is a major 
cause of security vulnerabilities. Oracle maintains all the highest 
government security classifications.
    Moving to our cloud data center will be provided at no extra cost 
to the Coast Guard, DoD or VA, and as I will explain later we believe 
it will help substantially compress costs in the long- run.
    We are cognizant of our Federal partners' concern raised at the 
August 4th summit that the entire Federal Enclave first be stabilized 
and are prioritizing that work in conjunction with the move to an 
Oracle cloud data center. We commit to maintaining the right balance of 
Oracle expertise, Oracle technology, and Oracle infrastructure to meet 
both goals.
    Finally, we are working with VA to approve expanding our 
relationship with our current partners to assist with the work of 
making the Federal Enclave more stable and performant. We hope to 
announce this expansion soon, and believe that this expanded 
partnership will bring much-needed expertise and capabilities for a 
system as complex as this one.
    Design: The second category of issues relates to system design. We 
have heard from providers about challenges with workflows and design 
that is not as easy or intuitive as it should be. While workflows are 
decided by the VA's National Councils and in coordination with DoD we 
will be engaging with the councils where we believe design changes can 
be made to improve clinical efficiency and productivity.
    And of course, one widely known design issue, the so-called 
``unknown queue,'' was recently reported on by the VA Inspector 
General. Despite its name, the unknown queue was not a bug, it was a 
backstop to account for patient scheduling tasks to facilities or 
providers that were not recognized by the system. These scheduling 
tasks were not lost, rather they were routed for manual review and 
processing, but the fact is the process initially resulted in far too 
many actions being routed to this queue and the manual review was not 
being completed in a timely manner.
    While that has largely been corrected and current instances of 
orders entering the queue are minimal, we still committed to providing 
updates that would further reduce the chance of an order being entered 
incorrectly. On August 1, 2022, we delivered updates to VA to alert 
providers in their message center when an order is unable to be 
scheduled. A second update alerts providers when they go to close a 
patient's chart. VA has these updates for testing, and we will 
implement them when VA is ready. These updates were provided by us at 
no cost to VA. Screenshots of the updates are attached for your 
reference.
    We also are working with VA related to the rescheduling of canceled 
appointments and the potential that some appointments were not properly 
rerouted. We will keep the Committee updated as we learn more.
    Functionality and Millennium's Future: When it comes to 
functionality of the EHR, there are small things that become big things 
in practice and deserve rapid improvement. For example, we learned that 
in Columbus lab employees have not been printing in batches, which 
delays printing labels for vials of blood when multiple bloodwork tests 
are ordered. We have learned the check-in process for appointments has 
been slowed. These are two seemingly small examples in an overall EHR 
modernization, but they have a big impact on the amount of time a user 
spends in the EHR instead of with the veteran, and how much time the 
veteran spends in the facility. We are working to fix these and other 
similar issues that we are aware of to make the system more functional 
and efficient.
    In previous testimony in July, I spoke to functionality concerns 
with the pharmacy module. Following VA's updated pharmacy requirements, 
we are currently working on seven important updates to pharmacy that 
will be delivered over the course of the next year and will make the 
pharmacy module function much better for users and veterans. The first 
three updates will be provided to VA by early next year.
    At the same time, as I promised in July in testimony before the 
authorizing committees, we are developing new pharmacy functionalities 
from the ground-up. After starting this work, it quickly became 
apparent that we need to develop these pharmacy functionalities not as 
a separate module but as an integrated part of the new, modern EHR.
    We are currently investing substantial resources to progressively 
rewrite the Oracle Cerner Millennium EHR as a modern, stateless web 
application, which will include pharmacy functionality. The system will 
have a modern web-based user interface. It will be mobile friendly, 
meaning users can bring their own device. It will include voice 
recognition, and ML- based clinical decision support and analytics that 
are built-in from the ground up. In short, it will be a fully modern 
cloud-based EHR system. Not only will this deliver the longitudinal 
record from enlistment through retirement and lifelong care at VA, but 
also includes the hierarchical view of the entire DoD/VA population 
against which analytics, AI, and machine learning can be deployed. We 
will keep VA, DoD and Coast Guard updated and engaged as we work on 
this modern EHR system, and of course will seek appropriate approvals 
for deployment as necessary. And while I don't want to over-promise 
here, our intent is to deliver a beta version of the new EHR, with 
pharmacy functionality included, in 2023.
    Our plan--and our commitment to you--is to deliver all this 
functionality as an upgrade to the current system as part of our 
existing obligations under the current contract, at no extra cost to 
the government. Let me say that again--we plan to deliver a fully 
modern cloud-based EHR for the DoD, Coast Guard, and VA as part of our 
existing contract with the government.
    Additionally, in the short-term we plan to show VA users and 
veterans a glimpse of what the future system will look like. As I 
mentioned before, during the pandemic Oracle created direct patient 
facing applications like v-Safe. We are expanding these patient facing 
applications so that patients can easily keep their providers 
informed--each day if they like--about their health status. It is a 
simple, intuitive application that patients will be able to use to 
connect with their providers, view appointments and keep up to date on 
new benefit announcements. These new features will be added to Oracle 
Cerner EHR commercial products and therefore will be available to VA 
and DoD at no additional cost. We plan to meet with VA, DoD and 
Veterans Service Organizations to explain these features and assess 
applicability for focused areas like mental health or burn pit exposure 
awareness as two examples. It is the kind of addition to the system 
that will benefit veterans nationwide, all at once, while the longer 
hospital by hospital implementations progress. I have attached to this 
testimony sample screenshots of the new application. We look forward to 
working with VA and DoD to obtain their thoughts around rolling out 
some or all of these features in the coming months.
    Nonetheless we will continue to invest in new technologies that we 
believe can assist our nation's veterans in short order, regardless of 
where their local VA center is on the system rollout schedule.
    Training: Modern applications should require little to no training. 
Certainly, an EHR system has a level of complexity and medical 
specificity that will require some training, but our goal is to make 
this system as easy to use as anything else you do online. The best way 
to succeed is to win over users with user interfaces that are intuitive 
and functionality that exceeds practitioner's needs and expectations. 
When we do that, we believe we will create greater user satisfaction 
and combat inertia for acceptance of the new system.
    Over time we will achieve that goal, especially when we eventually 
move Millennium to a modern stateless web application. But in the 
short-term, we recognize that training must be improved for users new 
to the Millennium system. Unless a VA provider recently joined VA from 
another healthcare system that used Millennium or another commercial 
product, VA providers are used to VistA and unfamiliar with Millennium. 
This is of course natural--VistA has been in use for several decades, 
so VA users know its tricks, shortcuts and workflows.
    Similarly, it is important to understand that Oracle Cerner EHR's 
are deployed in tens of thousands of healthcare facilities across the 
country without incident.
    We understand the challenge that change presents and the answer is 
two-fold. First, as I mentioned making the system more intuitive will 
help. Second, we are working with VA to revamp training. We recently 
engaged a third-party firm to evaluate the current training program. We 
have preliminary results and expect final recommendations in the coming 
two to 3 months. We will be engaging Accenture to implement the needed 
changes and work with us to make the training much more efficient, 
applicable and useful.
    We know that nobody wants to sit through hours of training to learn 
a new system, so we will make it more targeted and impactful--and in so 
doing hopefully create better momentum and inertia for user adoption at 
rollout sites in 2023 than we have seen in the past.
    Transparency: Another issue that has clearly been a problem is the 
dissemination of timely and accurate information, whether positive or 
negative. That has led to increased oversight by Congress and by the VA 
OIG, both of which we welcome. I of course commit to continue to come 
before this Committee and to work with you as often as is necessary, 
but to begin this work, we recently launched an electronic dashboard 
that we will make available to all of you which catalogues our ``to 
do'' list and progress being made. We hope this dashboard will 
supplement VA's monthly reporting, assist in the Committee's tracking 
and oversight, and keep everyone focused on deliverables and dates.
    Timing: With new site deployments set for next year, we have the 
time to make fixes and updates to the system related to stability and 
performance as I discussed earlier. We expect to be ready to pick up 
with a full deployment schedule next year as set forth by VA and DoD, 
including at major facilities in Seattle, Portland, and other cities. 
By the end of next year DoD will be fully deployed, and we anticipate 
VA will have nearly 30 additional facilities live on the new EHR.
    Under the original deployment schedule, more facilities would be 
using Millennium by the end of 2023, and we acknowledge that the 
rollout is behind. The pandemic obviously caused some of this delay and 
presented challenges with the initial rollout in Spokane that might 
otherwise have been avoided. But, as is often said, we are where we 
are--so from our perspective we look at how we can gain efficiencies in 
our deployment methodology to get back on a course to deploy more 
efficiently, and we look forward to working with VA on a full master 
schedule.
    We believe that the combination of steps I have described--improved 
system performance and stability, design and workflow fixes, enhanced 
functionality, revamped training and more--will put us in a position to 
accelerate deployments in 2024 and beyond. The number one thing we can 
do is make a system that VA providers want--and are willing to learn 
and adopt. That happens with a modern system that makes work easier and 
where tangible benefits are seen for patients.
    We know we have a lot to prove with deployments next year at larger 
more complex sites. We view the next year as a key window for building 
momentum and turning the corner, just as we have done at DoD where 
initial adoption was similarly challenging and is now proceeding apace 
and with strong results.
    Cost: The Inspector General has written that it believes there will 
be cost overruns, and we have seen the Institute for Defense Analyses 
(IDA) preliminary cost estimates for the next 28 years of lifecycle of 
the EHRM system. While I am not able to critique the IDA report one way 
or another, I will make four points.
    First, as for Oracle's part, we intend to deliver the EHR system 
across the entire VA for the amounts contemplated in the current 
contract under the current scope. This Committee rightfully focuses on 
the cost of the EHRM program. We recognize that our portion of the 
program is the lion's share of the budget, and we take seriously our 
responsibility to be good stewards of these taxpayer dollars.
    As such, we intend to deliver even more than was ever originally 
contemplated as we upgrade Millennium to our data center and modernize 
it to a Stateless web application. Those upgrades will be done at our 
cost, not the government's.
    Of course, if there are big new pieces of functionality not 
included in the current scope of the contract, that's a different 
discussion. However, if there are significant cost overruns related to 
the current contract, we are prepared to bear those costs and remain 
within the existing budget envelope. So let's talk dollars and cents, 
this is a 10-year, $10 billion contract that already has had $700 
million or so in additional funding. That's $10.7 billion. Even with 
delays and perhaps an extended timeline for deployment, that's the 
number we intend to deliver for--again, barring some big new piece of 
functionality being added that is not currently contemplated in the 
contract.
    Second, having been in this industry for almost 30 years, I am 
unaware of any point in history where the cost of technology has gone 
up, not down, nor am I sure I can predict the State or the cost of 
technology 28 years from now. I would anecdotally point out that 15 
years ago mobile devices and cloud computing didn't even exist. Mobile 
computing and the cloud have turned the economics of technology upside 
down.
    Third, and again we have not seen the full final IDA report, but 
from the preliminary slides it does not appear that a cost comparison 
was done against the cost of maintaining or upgrading VistA for 28 
years. VistA has 130 different instances running using various VA- 
owned and maintained data centers. VistA is programmed in a language 
with a limited programmer-base where finding programmers in the coming 
decades will only become more difficult and expensive. As MITRE 
reported in 2015, VistA's ability to deliver new capabilities is 
stalled and is in danger of becoming obsolete. At the end of IDA's 
budget window, VistA would be 68 years old, would continue to fragment, 
and would have no viable path for modernization.
    It seems intuitive that a system conceived and developed in the 
1970s and 80s, by definition, would have a far greater lifecycle cost 
than a modern cloud EHR at scale developed in the 2020s. And once a 
modern cloud enterprise EHR is fully deployed, cost benefits from 
improved healthcare delivery will exist.
    Fourth, with Millennium, VA is moving to an enterprise approach 
with one system instead of operating 130 instances, and eventually this 
system will move to the cloud. IDA of course couldn't have known our 
plans here as it conducted its analysis, but we believe the appropriate 
baseline for this program is not the current Millennium system but the 
next generation cloud product we are developing and that will be the 
system running for the lifecycle.
    I can't contemplate a scenario where operating Millennium could 
cost more than VistA. In our experience, as technology improves, costs 
go down, and we believe we can compress costs in the coming years. 
Moving to a cloud-based system by definition will reduce the cost of 
maintenance dedicated to physical infrastructure for the Federal EHR. 
It also puts the onus--and staffing requirements--on Oracle as we will 
be responsible for running the data centers, providing updates and 
security patches and making upgrades for capacity needs freeing VA 
employees from many of those tasks in their own data centers.
                              conclusions
    In conclusion, 4 months into owning Cerner, we are confident that 
the problems with the VA rollout can be fixed in a relatively short 
amount of time, and additionally that we can deliver a far superior and 
modern Federal EHR on an aggressive time schedule as part of our 
existing contract with the government. We have committed to providing 
the Committee with full transparency as we move forward, including with 
the recent launch of our dashboard. And we are dedicated to providing 
whatever resources are necessary to deliver to both DoD and VA a system 
that exceeds expectations without exceeding the contracted cost.
    Oracle is excited to be VA and DoD's new partner in this one-of-
kind, transformational EHR modernization effort. We are confident that 
our energy, commitment and resources will benefit this program greatly. 
With a little time, we can deliver for all the veterans who served our 
Nation and deserve nothing but the best, as well as for our current 
service members who will one day be a part of our veteran community.
    We hope you will support us in this endeavor and look forward to 
working with the Committee as we move forward. I look forward to your 
questions. Thank you.

    Attachments:

  --September 2, 2022 letter from Oracle Cerner to VA

  --Screenshots of Unknown Queue updates

  --Screenshots of new patient facing application
































    Senator Heinrich. Dr. Rieksts.
STATEMENT OF DR. BRIAN RIEKSTS, PH.D., RESEARCH STAFF 
            MEMBER, COST ANALYSIS AND RESEARCH 
            DIVISION, INSTITUTE FOR DEFENSE ANALYSES
    Dr. Rieksts. Chairman Heinrich, Ranking Member Boozman, and 
Senator Tester, thank you for the opportunity to testify today.
    The Department of Veterans Affairs asked the Institute for 
Defense Analyses to provide an independent, life cycle cost 
estimate for the Electronic Health Record Modernization 
program. Following GAO and OIG guidance we included all program 
costs regardless of funding source. IDA estimates the life-
cycle cost of the EHRM program to be about $50 billion in 
constant fiscal year 2022 dollars.
    These costs would span about 28 years, which includes a 13-
year implementation period, and a 15-year period after the 
system is fully deployed. IDA also conducted a risk analysis to 
quantify the uncertainty of the cost estimate. We estimated a 
likely range of $46- to $54 billion.
    There are notable differences between the IDA cost estimate 
and the 2019 VA estimate of $16 billion. IDA estimates cost for 
a 28-year life cycle, whereas the VA estimate covers only the 
10-year time period of the current contract. The VA estimate 
does not include sustainment costs or the cost of productivity 
loss due to deploying the system, both of which are included in 
the IDA estimate.
    Our tasking was to produce this independent estimate of VA 
EHRM life-cycle costs. Estimating potential benefits such as 
improved health care delivery and possible legacy system cost 
reductions was beyond the scope of our study. Our analysis 
estimates cost ranges for EHRM cost drivers, VA has an 
opportunity to manage cost drivers such as productivity loss 
associated with deployments.
    As the system is rolled out to more facilities, emerging 
information will provide additional insights regarding risk and 
uncertainty in the cost estimate.
    Thank you for the opportunity to discuss your cost 
estimate. And I look forward to your questions.
    [The statement follows:]
                Prepared Statement of Dr. Brian Rieksts
    Chairman Heinrich, Ranking Member Boozman, and distinguished 
Members of the Committee: Thank you for the opportunity to testify 
today.
    The Department of Veterans Affairs (VA) announced in 2017 that it 
would pursue an Electronic Health Record Modernization (EHRM) program 
to replace elements of its current aging system, the Veterans Health 
Information System and Technology Architecture (VistA). In 2022, VA 
asked the Institute for Defense Analyses (IDA) to develop an 
independent life-cycle cost estimate (LCCE) for the EHRM program.
    IDA produced an independent life-cycle estimate for all of the 
costs attributable to the EHRM program-that is, costs that are incurred 
only due to the existence of the EHRM program. IDA's cost estimate 
includes all costs of the program over its full life cycle, from 
program inception through design, development, deployment, operations 
and maintenance, and disposal. Thus, all relevant acquisition and 
sustainment costs are included in the LCCE. The IDA LCCE covers the 
implementation phase and 15 years of operations after the system is 
fully deployed to all sites.
    It is worth noting some items not in scope for the IDA independent 
cost estimate. One main item is benefits. The EHRM program has many 
potential benefits (e.g., improved healthcare delivery and reduced 
costs for legacy systems). The estimated benefits of the program, 
however, are out of scope for this study. Moreover, IDA's cost estimate 
is solely for the EHRM program. It does not include the cost of (or 
savings from) legacy electronic health record (EHR) systems.
                                results
    IDA estimates the EHRM LCCE to be $49.8 billion (in constant Fiscal 
Year 2022 dollars). The conversion to constant dollars normalizes 
inflation to 2022 levels; actual future expenditures will be higher. 
Overall, this estimate consists of $32.7 billion during the 
implementation phase over 13 years and an additional $17.1 billion in 
sustainment costs over the following 15 years.
    The following figure shows acquisition and sustainment costs for 
the LCCE during the implementation and fully deployed phases. We note 
that some sustainment costs are required during the implementation 
phase as sites start to use the new system.
    We estimate an additional $5.2 billion of common infrastructure 
costs in addition to the LCCE. These costs would be needed without the 
EHRM program, but EHRM also requires these investments.
                         cost estimate by phase
    IDA provided a point estimate representing the 50 percent risk-
adjusted cost. That is, the cost will be less than the point estimate 
with a likelihood of 50 percent. IDA also conducted a risk analysis to 
quantify the risk and uncertainty, estimating a range of $46-$54 
billion for the 20-80 percent cost range.
                               comparison
    We compared the IDA estimate to the VA 2019 program office estimate 
of $16.1 billion (represented in constant Fiscal Year 2022 dollars). We 
note several differences in scope and comparable elements between the 
VA and IDA estimates. First, the VA estimate is for 10 years of 
implementation, whereas IDA's life-cycle estimate spans 28 years. 
Second, the IDA team includes sustainment and productivity loss during 
rollout as additional cost elements in its LCCE, in accordance with 
Government Accountability Office (GAO) and VA Office of Inspector 
General (OIG) guidance to include all costs, but these elements are not 
included in the VA estimate. The productivity loss cost corresponds to 
the supplemental staffing and additional community care for Veterans 
needed to mitigate the disruption in care when rolling out the system 
to facilities. The revenue loss from the disruption is also included. 
The following figure shows the costs for the VA estimate, the 
comparable costs for the IDA estimate, and the costs of additional 
elements in the IDA estimate. IDA's LCCE includes an additional $25.9 
billion beyond the scope of the VA estimate. These additional elements 
account for about 75 percent of the cost difference from the VA's 
estimate ($16.1 billion) to IDA's estimate ($49.8 billion).
                   comparison of va and ida estimates
    The costs for comparable elements differ by $7.8 billion. About 
$5.1 billion of this difference is implied by actual costs from the 
first 4 years of the program. The factors contributing to the remaining 
difference are an IDA-estimated 3-year schedule adjustment based on 
experience in other programs and additional cost risks (e.g., 
additional development). We also note that the IDA LCCE includes $1.2 
billion less in infrastructure than the VA's estimate because IDA 
defines some costs as common infrastructure.
    Cost increases are common in programs of this complexity. 
Enterprise resource planning programs have typically had similar cost 
increases in acquisition.
                              path forward
    Our analysis estimates the cost ranges for risks associated with 
EHRM. VA has an opportunity to mitigate some of the risks associated 
with the cost drivers. For example, productivity loss associated with 
deployments is a cost element with substantial risk. VA can manage this 
risk and seek ways to improve the efficiency of deployments.
    Going forward, information emerging from upcoming deployments will 
provide additional insights regarding risk and uncertainty in the cost 
estimate. Developing measures to assess progress in key risk areas will 
allow VA leadership to monitor uncertainty and understand the 
implication for the total life-cycle cost of this program.
    Thank you for the opportunity to discuss our cost estimate, and I 
look forward to your questions.

    Senator Heinrich. Thank You. Mr. Case.
STATEMENT OF MR. DAVID CASE, DEPUTY INSPECTOR GENERAL, 
            OFFICE OF INSPECTOR GENERAL, DEPARTMENT OF 
            VETERANS AFFAIRS
    Mr. Case. Chairman Heinrich, Ranking Member Boozman, and 
Subcommittee Members, thank you for the opportunity to discuss 
the Office of Inspector General's Oversight of the VA's 
Electronic Health Record Modernization program.
    First, we want to recognize the VA employees working so 
hard in Washington, Ohio, Oregon, and across the Nation to 
ensure veterans receive timely, high-quality health care during 
the EHR transition, especially in a pandemic.
    Since April 2020, we have issued 14 EHRM reports primarily 
focused on planning, user training, and deployment activities 
at Mann-Grandstaff. They are meant to help VA leaders redress 
and avoid identified system failings in future rollouts. While 
VA has implemented 26 of our 68 total recommendations as of 
this week, considerable work remains.
    This year, we published reports related to significant 
patient safety risks at Mann-Grandstaff, including issues with 
the unknown queue, medication management, patient care 
coordination, the troubled ticket process, and gaps in 
actionable quality of care metrics. We are also aware of 
continued issues with behavioral health services and the 
pharmacy management software. We remain concerned about the 
many workarounds and mitigations that VHA employees must use to 
address these issues. They can lead to delays, increased 
errors, and affect the quality of patient care.
    Accordingly, our subject matter experts are continuously 
monitoring VA progress on OIG-recommended corrective actions 
meant to address many of these concerns.
    I want to highlight two audits we published last year. 
Because VA needs to spend billions of dollars on fiscal and IT-
related infrastructure upgrades to support the EHR, we audited 
VA's related cost estimates following our determination that VA 
did not meet its own deadline for infrastructure upgrades at 
the pilot site.
    The cost estimates were unreliable. In general, they were 
not comprehensive, well documented, accurate, or credible. Our 
audit teams also determined VA had not reported accurate and 
complete information to Congress in the Eight Congressional 
Reports submitted through January 2021. OEHRM personnel did not 
report the estimated $2.7 billion for physical infrastructure 
upgrades, and the estimated $2.5 billion for IT infrastructure 
upgrades.
    They explained that because VHA and OIT were funding these 
upgrades, OEHRM did not need to report them. Despite VA and GAO 
guidance requiring life-cycle cost estimates to include all 
costs, regardless of funding source, these omissions 
significantly understate the program's true costs. 
Recommendations from the two audits include that VA obtain an 
independent cost estimate of the program's life cycle, and 
ensure transparency in reporting costs to Congress. These 
recommendations remain open.
    We have been briefed by the Institute for Defense Analyses 
on their draft independent cost estimate and look forward to 
reviewing the final report's methodology, findings, and 
estimates before determining how they relate to our outstanding 
recommendations.
    In April 2022, we also found VA had not executed a reliable 
and comprehensive integrated master schedule. They failed to 
meet the standards they adopted that called for a schedule that 
is comprehensive, credible, well-constructed, and controlled. 
Among the actions needed to meet the standards for an 
integrated master schedule, VA must complete a schedule risk 
analysis and develop a critical path for completion of the 
work.
    Although not every task for a 10-year project can be 
accounted for early on, VA did not use accepted strategies to 
create a schedule that can be tailored over time. VA's failure 
to meet these standards increases the risk of further delays, 
dropped activities, and budget overruns. Without a reliable 
master schedule, the accuracy of any cost estimate is at risk 
because costs are inextricably linked to the schedule of 
activities VA needs to complete.
    In closing, we remain concerned with the 15 recommendations 
open for longer than 1 year. The success of the EHR 
implementation is dependent on VA's transparency, careful 
planning, and the recognition and remediation of patient care 
and safety concerns. Not only those risks identified by our 
oversight work, but by VHA's own experts and end users who rely 
on the EHR for everyday clinical decisionmaking.
    Chairman Heinrich, this concludes my statement. I would be 
happy to answer any questions you or other committee members 
may have.
    [The statement follows:]
                  Prepared Statement of Mr. David Case
    Chairman Heinrich, Ranking Member Boozman, and subcommittee 
members, thank you for the opportunity to discuss the Office of 
Inspector General's (OIG) oversight of the Department of Veterans 
Affairs' electronic health of record modernization (EHRM) program. The 
OIG recognizes the enormity and complexity of converting VA's 
electronic health record (EHR) system for millions of veterans 
receiving VA care and acknowledges the significant work and commitment 
of VA staff to accomplish this task. Over the more than 2 years that 
OIG staff have been engaging with employees at the first deployment 
site-the Mann-Grandstaff VA Medical Center (VAMC) in Spokane, 
Washington-and other VA locations using the new EHR, oversight teams 
have observed VA employees' unwavering commitment to this transition 
while prioritizing the care of patients during the COVID-19 pandemic. 
Facility staff challenges have been exacerbated, however, by the lack 
of prompt remediation of problems that the OIG and others have 
identified in numerous oversight reports published since April 2020.
    The OIG has published 14 reports addressing the EHRM program and 
system implementation between April 2020 and this hearing with a total 
of 68 recommendations. Though this statement does not detail all of 
these reports and their findings, a comprehensive list of 
recommendations has been included in the appendixes. Each oversight 
report is meant to help VA improve the new system's implementation and 
support the provision of prompt, quality health care for veterans. 
Failure to satisfactorily complete the corrective actions associated 
with these recommendations can increase risks to patient safety and the 
ability to provide high-caliber care as the new EHR system rolls out 
nationwide. Fully addressing oversight recommendations can help 
minimize considerable cost escalations and delays in future site 
deployments as well. The OIG is therefore concerned about the five 
recommendations that have been open (not implemented or fully 
addressed) for longer than 2 years-with 21 total recommendations open 
for more than 1 year. While the OIG follows up with VA on open 
recommendations every 90 days, VA program officials can submit evidence 
of sustained progress or satisfaction of corrective actions at any time 
to facilitate closing recommendations.
    As detailed in this statement, OIG staff have found VA did not 
complete timely critical infrastructure upgrades for the initial 
rollout and provided unreliable and incomplete estimates on 
infrastructure upgrade costs, has not adequately prepared for the 
rollouts (including realistic scheduling and effective user training), 
failed to be fully transparent, and stove-piped governance with 
decision-making that has not appropriately engaged Veterans Health 
Administration (VHA) end users of the new EHR system. Many of these 
issues are still of concern to the OIG, as evidenced by the number of 
open recommendations.
    This testimony highlights those OIG reports with findings that 
illustrate three broad categories of concern: (1) IT and physical 
infrastructure deficiencies and unreliable cost estimates for 
addressing them, (2) readiness concerns that include the lack of a 
comprehensive master schedule and ineffective training that was not 
transparently reported, and (3) implementation issues that affect 
patient care and safety as well as concerns with remediation and 
mitigation strategies.
       infrastructure deficiencies and unreliable cost estimates
    The OIG's oversight in April 2020 focused on VA's preparation for 
the system's initial deployment at the Mann-Grandstaff VAMC and the 
condition of VA's physical and information technology (IT) 
infrastructure prior to system deployment. Two 2021 reports (published 
in May and July) resulted from audits that examined cost estimates for 
needed physical and IT-related infrastructure upgrades nationwide. For 
the new EHR system to operate as intended, VHA facilities need these 
infrastructure upgrades, but they are generally funded from different 
sources. Because the life-cycle cost estimates for infrastructure 
upgrades did not account for costs from all VA components' budgets, 
some estimated costs were not included in mandated reports to Congress. 
Transparent and reliable cost estimates are critical for Congress to 
make informed budgeting decisions. VA senior leaders also depend on 
these cost estimates to plan program budgets, approve acquisitions, and 
monitor program execution. The OIG determined the existing physical and 
IT infrastructure was inadequate for the new system at initial 
deployment sites, and pertinent life-cycle cost estimates for 
infrastructure upgrades were unreliable and likely underreported by 
approximately $5 billion. These two reports recommended that VA obtain 
an independent cost estimate for the EHR program's life-cycle costs, 
which the VA is obtaining from the Institute for Defense Analyses. The 
OIG has been briefed on the draft and will review the final report's 
methodology, findings, and estimates before determining how they relate 
to outstanding recommendations.
    The 2020 OIG report focused on the gaps in VA's efforts to update 
the Mann-Grandstaff VAMC's physical and information technology (IT) 
infrastructure to support the new system.\1\ The OIG found that VA did 
not meet its own timelines to complete critical physical and IT 
infrastructure upgrades at the facility. The problems with planning 
identified in this report were shown in greater detail in the 2021 OIG 
reports that found deficient and unreliable physical and IT 
infrastructure cost estimates. Many of the recommendations to resolve 
these issues remain open.
---------------------------------------------------------------------------
    \1\ ``Physical infrastructure'' refers to the underlying foundation 
that supports the system, such as electrical; cabling; and heating, 
ventilation, and air-conditioning. ``IT infrastructure'' includes 
network components such as wide and local area networks, end-user 
devices (e.g., desktop and laptop computers, and monitors), and medical 
devices.
---------------------------------------------------------------------------
    deficiencies in infrastructure readiness for deploying va's new 
              electronic health record system (april 2020)
    To deliver patient care using the new EHR system, significant 
upgrades are needed to VA's physical and IT infrastructure.\2\ The OIG 
audited VA's infrastructure readiness activities at the Mann-Grandstaff 
VAMC in anticipation of the initial March 2020 go-live date. In 2019, 
then Office of Electronic Health Record Modernization (OEHRM) leaders 
testified before the House of Representatives that having 
infrastructure in place 6 months before deploying the Cerner system was 
a program goal to help ensure smooth deployment, but the OIG found they 
had not been completed at the facility even 5 months prior to the March 
2020 go-live.\3\ In fact, the OIG found some infrastructure upgrades 
intended to mitigate diminished system performance were not projected 
to be completed until months after going live. In sum, VA committed to 
an aggressive, but apparently unrealistic, deployment date of March 
2020 without having the necessary information about the facility's 
infrastructure.
---------------------------------------------------------------------------
    \2\ VA OIG, Deficiencies in Infrastructure Readiness for Deploying 
VA's New Electronic Health Record System, April 27, 2020.
    \3\ In 2021, VA transitioned EHRM program management from the 
Office of Electronic Health Record Modernization (OEHRM) to the EHRM 
Integration Office (EHRM IO). EHRM IO now has responsibility for all 
recommendations assigned to OEHRM. Cerner Corporation was acquired by 
Oracle Corporation on June 7, 2022; this statement will refer to the 
entity as ``Cerner,'' as it was referred to at the time of the reviews 
discussed in this statement.
---------------------------------------------------------------------------
    The OIG made seven recommendations for corrective action to the 
then executive director of OEHRM, and an eighth recommendation to the 
Mann-Grandstaff VAMC director. These recommendations, of which two 
remain open as not implemented, can be found in appendix A of this 
statement. Given the time elapsed since this report's publication, it 
is concerning that one of the open recommendations calls on OEHRM to 
evaluate physical infrastructure for consistency with its program's 
requirements and monitor those evaluations.
    deficiencies in reporting reliable physical infrastructure cost 
               estimates for the ehrm program (may 2021)
    This audit was conducted to determine if VA developed and reported 
reliable physical infrastructure upgrade cost estimates for the new EHR 
system.\4\ As discussed previously, VHA medical facilities need 
significant physical infrastructure upgrades, such as electrical work, 
cabling, heating, ventilation, and cooling to successfully deploy the 
new EHR system. The audit examined whether VHA's cost estimates met VA 
standards and were comprehensive, well documented, accurate, and 
credible. It also reviewed whether OEHRM reported these cost estimates 
to Congress in accordance with statutory mandates.
---------------------------------------------------------------------------
    \4\ VA OIG, Deficiencies in Reporting Reliable Physical 
Infrastructure Cost Estimates for the Electronic Health Record 
Modernization Program, May 25, 2021.
---------------------------------------------------------------------------
    VHA and OEHRM shared responsibilities for estimating and reporting 
physical infrastructure upgrade costs. VHA developed the physical 
infrastructure upgrade cost estimates, while OEHRM was responsible for 
reporting all program life-cycle cost estimates to Congress in 
accordance with the Veterans Benefits and Transition Act of 2018.\5\ In 
May 2021, the act required quarterly reporting on the EHRM program's 
status, including annual and life-cycle cost estimates and defined the 
program as any activities to procure or implement the new EHR system. 
In early 2019, VA's Office of General Counsel determined that physical 
infrastructure upgrades must be funded from accounts specifically 
available for construction-type purposes, such as VHA's nonrecurring 
maintenance and minor construction funds.
---------------------------------------------------------------------------
    \5\ The law was signed on December 31, 2018, and it became Public 
Law 115-407.
---------------------------------------------------------------------------
   vha cost estimates for physical infrastructure upgrades needed in 
             support of the ehrm program were not reliable
    The OIG found VHA's cost estimates were not reliable under VA 
standards and Government Accountability Office (GAO) guidance.\6\ These 
standards and guidance state that cost estimates should be 
comprehensive, well documented, accurate, and credible. However, 
neither of VHA's formal cost estimates for physical infrastructure, 
dated June 2019 ($2.7 billion) and November 2019 ($1.1 billion), fully 
met these criteria, and thus could be significantly understated. In 
addition, VA lacked effective quality controls and procedures to 
evaluate the estimates and had conducted insufficient planning from the 
start.
---------------------------------------------------------------------------
    \6\ VA Cost Estimating Guide, ver. 2.2, August 17, 2016; GAO, Cost 
Estimating and Assessment Guide, GAO-20-195G, March 2020.
---------------------------------------------------------------------------
1. Cost Estimates Were Not Comprehensive
    Comprehensive cost estimates provide officials with reasonable 
assurance that all costs are included so they can make well-informed 
decisions. VHA's November 2019 estimate, totaling about $1.1 billion 
for physical infrastructure upgrades nationally, only reflected about 
25 percent of nationwide cabling costs, understating the costs by at 
least $481 million. Also, the June and November 2019 estimates omitted 
estimated costs of upgrades paid with minor construction funds.
2. Cost Estimates Were Not Well Documented
    Sufficient documentation supports an estimate's validity and 
provides an audit trail allowing the estimate to be easily recreated 
and updated. Both June and November estimates lacked evidence they were 
approved by senior leaders, and they did not have enough detail to 
allow an independent party to trace the costs or determine if costs 
were double-counted.
3. Cost Estimates Were Not Accurate
    Neither cost estimate met the standard for accuracy-that is, free 
of mathematical errors and not overly conservative or optimistic. The 
June 2019 estimate had errors omitting about $90 million of fiscal year 
2021 construction design costs. The November 2019 estimate omitted 
escalation costs for upgrades expected to take place in future years 
and did not include the cost of completely upgrading the cabling 
required at VHA facilities nationwide.
4. Cost Estimates Were Not Credible
    Credible cost estimates identify limitations of the data and 
assumptions and are to be measured against independent or third-party 
cost estimates. Both estimates lacked a risk and uncertainty analysis, 
which is used to disclose the likelihood actual costs may differ from 
estimated costs. VHA did not conduct this type of analysis because VA 
did not have accurate assessments of what infrastructure upgrades were 
needed at its facilities. Both estimates also lacked a sensitivity 
analysis, which is used to explain how much impact each cost factor has 
on the overall estimate. Both cost estimates were also not compared to 
a third-party cost estimate, a best practice in validating the 
reliability and reasonableness of cost estimates. Using the planned and 
obligated costs at VA's three planned initial operating capability 
sites, the OIG team statistically projected program-wide physical 
infrastructure costs to be between approximately $3.1 and $3.7 
billion.\7\ Notably, VHA's June 2020 estimate projects physical 
infrastructure upgrade costs to be about $3.1 billion, consistent with 
the OIG team's low-end projection.
---------------------------------------------------------------------------
    \7\ The three facilities were the Seattle, American Lake, and Mann-
Grandstaff VAMCs, all located in Washington State.
---------------------------------------------------------------------------
5. Lack of Effective Quality Controls and Procedures to Evaluate 
        Estimates
    Deficient quality controls contributed to the unreliability of both 
cost estimates. Independent cost estimates-a control used to validate 
the data and determine the reasonableness of a VA estimate-are required 
by VA policy to be performed on all major IT programs, but an 
independent cost estimate was not performed on either estimate.
6. Insufficient Planning at the Program's Start
    Consistent with findings from the April 2020 OIG report, the audit 
team found neither OEHRM nor VHA knew the true state of infrastructure 
at facilities at the time the Cerner contract was signed, and, when 
this audit was completed in March 2021, VHA was still identifying 
necessary infrastructure upgrades. As of January 2021, infrastructure 
requirements continue to be defined, making it difficult for VHA to 
identify gaps in infrastructure and estimate related costs.
      oehrm did not include cost estimates for upgrading physical 
                 infrastructure in reports to congress
    The OIG found that OEHRM did not include the cost of physical 
infrastructure upgrades in quarterly reports to Congress, which are 
intended to meet the program's requirements under the Veterans Benefits 
and Transition Act.\8\ This is significant, as it understated the 
program's cost in reports submitted to Congress. The reports gave the 
impression that these costs were included because seven of the eight 
reports said that infrastructure costs include ``physical 
infrastructure at VA medical centers and other sites.'' To the 
contrary, these reports did not include the $2.7 billion for physical 
infrastructure upgrades as identified in the June 2019 estimate OEHRM 
received from VHA. OEHRM said it did not disclose these estimates 
because the upgrades were outside its funding responsibility, but this 
is contrary to the explicit requirements of statute and to VA and GAO 
guidance that a life-cycle cost estimate include all costs, regardless 
of source.\9\ The VA Electronic Health Record Transparency Act of 2021 
modified VA's reporting requirement to mandate the inclusion of costs 
expended by any VA element.\10\
---------------------------------------------------------------------------
    \8\ OEHRM produced its ninth report after the OIG report was 
drafted and did not include physical infrastructure upgrade costs in 
that document.
    \9\ The Veterans Benefits and Transition Act of 2018 defines the 
EHRM program as ``any activities ... to procure or implement an 
electronic health or medical record system to replace'' the existing 
electronic health record system and ``any contracts or agreements 
entered into by [VA] to carry out, support, or analyze'' these 
activities. Because physical infrastructure upgrades are necessary for 
system implementation, those costs should be included in life-cycle 
cost estimates under the statute's plain language.
    \10\ The law was signed on June 23, 2022 and became Public Law 117-
154.
---------------------------------------------------------------------------
    The OIG made five recommendations to VA, which can be found in 
appendix B. Three of the recommendations, which pertain to the need for 
an independent cost estimate of the program's life cycle and ensuring 
transparency in reporting costs to Congress, remain open. As previously 
mentioned, the OIG has been briefed by the Institute for Defense 
Analyses on their draft independent cost estimate and looks forward to 
receiving the final report for review.
unreliable it infrastructure cost estimates for the ehrm program (july 
                                 2021)
    Of EHRM's estimated $16.1 billion total program cost from 2021, VA 
estimated about $4.3 billion would be directed for IT infrastructure 
upgrades.\11\ This audit examined whether OEHRM-developed cost 
estimates were well-documented, comprehensive, credible, and accurate, 
and whether OEHRM reported to Congress all IT infrastructure upgrade 
costs, including future technology updates.\12\
---------------------------------------------------------------------------
    \11\ VA OIG, Unreliable Information Technology Infrastructure Cost 
Estimates for the Electronic Health Record Modernization Program, July 
7, 2021.
    \12\ Technology refreshment is the process of replacing certain 
infrastructure on a regular schedule, instead of using the systems or 
devices until they can no longer function. For example, devices like 
laptops are replaced every 4 years.
---------------------------------------------------------------------------
IT Infrastructure Upgrade Cost Estimates Were Not Reliable but 
        Improvements Have Been Made
    As discussed previously, reliable estimates should be well-
documented, comprehensive, credible, and accurate. The audit team 
evaluated two estimates OEHRM provided to Congress dated December 2018 
and August 2020-each estimating about $4.3 billion for the IT 
infrastructure upgrades. Neither met the reliability criteria, and the 
OIG could not evaluate their accuracy because they lacked documentation 
to support many of the calculations. Like the physical infrastructure 
cost audit, VA did not complete an independent cost estimate, which 
could have revealed the OIG-identified issues sooner.
    In January 2021, in part due to discussions with the audit team, 
OEHRM began developing procedures that align with cost-estimating 
guidance and include controls to help address the issues identified in 
the OIG report. During the audit, the team noted that VA also began 
making improvements to the cost model used to develop the estimate, 
facilitating more detailed support.
IT Infrastructure Costs Were Omitted and Not Updated for Accuracy
    The OIG found OEHRM did not include costs for critical program-
related IT infrastructure upgrades in the estimates reported to 
Congress, effectively underreporting program cost estimates by nearly 
$2.5 billion. The $2.5 billion is for IT infrastructure upgrades that 
VA's Office of Information and Technology (OIT) and VHA are expected to 
fund.\13\ Like the physical infrastructure costs, OEHRM officials 
stated they felt the omitted costs were outside their scope of 
responsibility, but neither OIT nor VHA reported these costs to 
Congress, despite VA and GAO guidance requiring life-cycle cost 
estimates to include all costs, regardless of source. The costs should 
have been disclosed by OEHRM. VA did make changes to projected costs 
starting in the November 2021 report to Congress, but because VA was 
still developing the independent cost estimate, there was no certainty 
the updates were reliable.
---------------------------------------------------------------------------
    \13\ OIT is expected to fund some upgrades for the local area 
network, end-user devices, phones, and Wi-Fi, while VHA is expected to 
fund upgrades mostly for medical devices.
---------------------------------------------------------------------------
    Without all critical IT infrastructure upgrade costs accurately 
presented, Congress lacks the comprehensive picture of total program 
costs needed to make informed oversight and investment decisions. As 
mentioned previously, VA's reporting requirements have been updated by 
the VA Electronic Health Record Transparency Act of 2021.
    All six recommendations to the executive director of OEHRM are 
listed in appendix C and remain open. The recommendations relate to 
obtaining independent cost estimates for IT infrastructure, ensuring 
the costs are estimated in line with VA policy, maintaining full and 
complete accounting for the costs, and ensuring complete and updated 
transparency of the costs with Congress.
   lack of readiness exhibited by no integrated master schedule and 
                          ineffective training
    Exploring program costs and projections further, the OIG reported 
in April 2022 that VA had not executed a reliable, comprehensive 
schedule for system implementation. This could result in schedule 
delays and leave VA vulnerable to billions of dollars in cost overruns. 
Without that schedule, Congress and the public cannot rely on VA 
timeline projections for completing the work or be assured that the 
program will be completed within budget.
    The OIG also examined the flawed implementation at Mann-Grandstaff 
VAMC that was brought on by inadequate planning. Deficiencies the OIG 
detected at Mann-Grandstaff VAMC in April 2020 revealed the need for 
prompt corrective measures as additional facilities were switching to 
the new EHR system. Yet many issues remained unresolved prior to 
deployment, particularly problems identified in the OIG's July 2021 
report on the development, delivery, and assessment of staff training 
and proficiency.
 the ehrm program did not fully meet the standards for a high-quality, 
                  reliableschedule (april 2022 report)
    To implement the program successfully and within budget, it is 
imperative that VA develop a reliable integrated master schedule 
(IMS).\14\ GAO guidance, which OEHRM adopted, States that a high-
quality, reliable schedule should be comprehensive, credible, well-
constructed, and controlled. The IMS is designed to cover the entire 
required scope of work-of both government staff and contractors-needed 
to complete the program. VA should use it as a road map to monitor 
progress, complete the work, identify potential problems and track 
their resolution, and promote accountability. While not every task for 
a 10-year project can be accounted for early on, strategies exist to 
create a tailorable, comprehensive schedule to minimize the risk of 
delays, dropped activities (some of which are prerequisites for 
others), and budget overruns. While VA may have received a draft 
independent cost estimate for the program since the OIG's two audits, 
without a reliable IMS, the developed cost estimates' accuracy are at 
risk because they are inextricably linked to the schedule of activities 
VA that needs to complete.
---------------------------------------------------------------------------
    \14\ VA OIG, The Electronic Health Record Modernization Program Did 
Not Fully Meet the Standards for a High-Quality, Reliable Schedule, 
April 25, 2022.
---------------------------------------------------------------------------
VA Did Not Have a High-Quality, Reliable IMS
    The OIG found that neither the overall IMS nor five of its 
underlying individual project schedules fully met GAO standards adopted 
by OEHRM for a high-quality, reliable schedule. VA failed to meet all 
aspects of the following scheduling standards:

  --Comprehensive. The IMS should reflect the entire scope of program 
        work in some level of detail. However, the OIG determined that 
        the IMS did not capture all work for the program's duration and 
        was missing VHA and OIT activities.

  --Credible. A credible IMS should include a complete schedule risk 
        analysis, which can give a level of confidence in meeting a 
        program's completion date. However, OEHRM did not do this.

  --Well-constructed. A ``critical path'' determines the earliest date 
        a program can be completed to help managers examine the effects 
        of activity slippages, but no overall IMS critical path was 
        created.

  --Controlled. A controlled IMS should include a baseline schedule, 
        used for managing the program and conducting trend analyses 
        over time to assess program performance. However, OEHRM's 
        program baseline only covered events through April 2020.

    The OIG identified several root causes for OEHRM's failures:

  --Did not adequately coordinate with various offices. VHA and OIT 
        leaders said OEHRM did not collaborate with them, so the 
        schedules did not include all work to be performed by these 
        entities.

  --Did not conduct a schedule risk analysis because it lacked 
        procedures. Despite the importance of completing this analysis, 
        OEHRM did not have procedures in place on when and how to 
        conduct it.

  --Focused on near-term deployment of the system at the initial 
        operating sites. OEHRM only required development of site-
        specific schedules after task orders for those sites were 
        awarded. Applying that strategy, VA would not have a high-
        quality, reliable IMS until it starts deploying the system at 
        the last sites, which are planned to go live in fiscal Year 
        2028.

  --Did not enforce its own scheduling standards or have tools in place 
        to assess compliance. While OEHRM's schedule management plan 
        stresses compliance with GAO guidance, task orders to Cerner do 
        not require the IMS to align with them. Additionally, OEHRM's 
        schedule management plan requires staff to use specific 
        software to assess whether EHRM project schedules comply with 
        GAO standards. However, a tool was not available from March 
        2020 to June 2021.

  --Lacked consistent guidance on roles, resulting in confusion over 
        the assignment of IMS development and documenting how work was 
        broken down. Internal planning and contract documents 
        inconsistently assigned responsibilities for developing and 
        maintaining the program's work breakdown structure (WBS) and 
        the IMS. The WBS defines all work needed to complete the 
        program. Guidance inconsistently assigned these 
        responsibilities to VA or one of its contractors-

  --Booz Allen Hamilton, Inc., or Cerner, leading to confusion.\15\ 
        Cerner accepted responsibility for the WBS and, in July 2020, 
        worked with VA to create it. While Cerner is responsible for 
        developing the IMS, VA should ensure contract requirements are 
        consistent with internal guidance.
---------------------------------------------------------------------------
    \15\ Booz Allen Hamilton, Inc. staff support EHRM activities. Their 
work included gathering input from VA administrations or offices to 
develop schedules for VA activities.

  --Did not clearly define IMS contract requirements. Cerner was 
        contractually required to develop and maintain an IMS for the 
        program under VA's task orders; however, the task orders did 
        not clearly establish a timeline for when a complete IMS would 
        be developed. Without a clear timeline, OEHRM required Cerner 
        to develop site-specific project schedules as task orders were 
        awarded. Following this process, future work not yet on task 
---------------------------------------------------------------------------
        order would be unaccounted for in the IMS.

    VA has a responsibility to ensure there is a complete IMS that 
meets scheduling standards. VA needs a high-quality, reliable IMS to 
strengthen the credibility of the program's timeline. Without one, VA 
can neither demonstrate how slippages will affect the overall timeline 
nor assure stakeholders that the reported timeline is realistic and 
achievable. Any schedule delays that extend the program beyond 10 years 
are also likely to result in billions of dollars in cost overruns. The 
OIG estimated the average cost per year of a schedule delay is 
potentially about $1.95 billion.
    For this report, the OIG made six recommendations, found in 
appendix D, and all are open.
 training deficiencies for va's new ehr system at the mann-grandstaff 
                        vamc (july 2021 report)
    The OIG reviewed the training given to Mann-Grandstaff VAMC 
staff.\16\ Problems were identified similar to those found by 
Department of Defense (DoD) for training on the new EHR system. Even 
before deployment, the healthcare inspection team identified governance 
challenges as VHA did not have a defined role in decision-making or 
oversight related to training activities. In reviewing the training, 
the OIG found training content, delivery, and assessment failures.
---------------------------------------------------------------------------
    \16\ VA OIG, Training Deficiencies with VA's New Electronic Health 
Record System at the Mann-Grandstaff VA Medical Center in Spokane, 
Washington, July 8, 2021.
---------------------------------------------------------------------------
    The inspection team reviewed the training content on the software 
and the more than 900 new workflows. New workflows result in changes to 
how end users perform their jobs, such as scheduling consults 
(referrals) or how a provider performs an exam. The OIG found the 
classroom training and supplemental material were insufficient. 
Facility leaders and staff told the OIG that training did not prepare 
them for going live with the new system, teach them how to apply what 
they learned to their work, or explain the meaning behind the process 
of which buttons to push (``buttonology'').
    The OIG identified four aspects of training delivery that may have 
negatively affected the new EHR system's use: (1) insufficient time for 
training, (2) limitations with the training domain (a close facsimile 
for users' practice), (3) challenges with user role assignments (these 
dictate the capabilities on which an employee is trained), and (4) gaps 
in training support. Facility leaders and staff raised concerns with 
Cerner classroom trainers, including their lack of clinical knowledge, 
EHR expertise, and an inability to address questions.
    Finally, the OIG found OEHRM failed to effectively evaluate the 
training. The OIG conducted a follow- up administrative investigation 
into the inaccurate and incomplete data OEHRM provided about trainees' 
post-training tests after OIG staff requested ``any and all data'' from 
the training evaluation plan that OEHRM's Change Management leaders 
submitted.\17\ While the investigation did not find that the two Change 
Management leaders intentionally sought to mislead OIG healthcare 
inspectors, their lack of due care and diligence resulted in inaccurate 
information being submitted to OIG staff. Most concerning, the Change 
Management's then executive director and the director for training 
strategy did not disclose they removed some data from consideration or 
that they questioned data reliability. They delayed production of 
underlying proficiency check data and instead provided one slide with 
three summary statistics containing significant errors that resulted in 
doubling the reported trainee proficiency check pass rate from 44 to 89 
percent. In addition, officials admitted the evaluation plan was 
actually ``immature'' and ``in its infancy'' and was not implemented, 
contrary to the evaluation plan submitted to the OIG that showed 
training was being assessed immediately after it was completed by 
employees.
---------------------------------------------------------------------------
    \17\ VA OIG, Senior Staff Gave Inaccurate Information to OIG 
Reviewers of Electronic Health Record Training, July 14, 2022.
---------------------------------------------------------------------------
    Had the OIG relied on the information provided, Congress and the 
public would have been misled as to how trainees had performed in the 
tests. The culture of accountability the Secretary and Deputy Secretary 
are promoting by mandating training on engaging with the OIG and other 
measures is critical; however, this investigation underscores the need 
for leaders overseeing the EHRM program to reinforce those values and 
the requirement for timeliness, completeness, and accuracy in all 
responses to OIG requests for information. The OIG made four 
recommendations, found in appendix E, and one remains open.\18\
---------------------------------------------------------------------------
    \18\ Two of the recommendations ask VA to examine if administrative 
action should be taken concerning the conduct or performance of the 
senior leaders. As an independent oversight authority, the OIG cannot 
mandate administrative action or dictate a specific outcome.
---------------------------------------------------------------------------
    The OIG made 11 recommendations in the July 2021 report to improve 
the training program, which can be found in appendix F, and seven are 
still open.
   new patient scheduling system needs improvement as va expands its 
                 implementation (november 2021 report)
    This report assessed the implementation of the EHR system's patient 
scheduling component at the Columbus clinic and Mann-Grandstaff 
VAMC.\19\ The OIG found VHA and OEHRM did not fully resolve known 
significant limitations in the scheduling system, leading to reduced 
effectiveness and increased risk of patient care delays. The problems 
identified in this report have persisted through the OIG's 2022 
reports, such as schedulers developing work-arounds for unresolved 
issues and problematic data migrated from legacy systems. OEHRM leaders 
did not provide scheduling staff with adequate chances to identify 
limitations in the new scheduling system before implementation, nor did 
leaders assess Cerner's compliance with contract terms for handling 
trouble tickets submitted by users. OEHRM leaders were aware of the 
system's issues before and after Columbus's implementation, but the 
issues were not resolved even in late 2021. That said, VHA staff told 
the OIG that the new system should help greatly, and schedulers 
reported positive experiences. For example, schedulers said the new 
system was more user-friendly than the legacy system, making video 
visits easier to schedule, among other upgrades. The OIG made eight 
recommendations, found in appendix G, and all remain open.
---------------------------------------------------------------------------
    \19\ VA OIG, New Patient Scheduling System Needs Improvement as VA 
Expands Its Implementation, November 10, 2021.
---------------------------------------------------------------------------
        implementation deficiencies and the lack of remediation
    The OIG has sustained a strong focus on the patient safety aspects 
of the EHRM program, starting with its April 2020 report that reviewed 
VA's readiness to ``go live'' at the initial site and the potential 
impact of the transition on patients' access to high-quality care. The 
findings include that the Mann-Grandstaff VAMC lacked adequate staffing 
and formal, written guidance to navigate the transition's strains. The 
OIG also found that the risk mitigations facility leaders would employ 
during the planned go-live period were inadequate to address the gaps 
in the new EHR system capabilities and presented a potential yet 
significant risk to patient safety.
    In 2022, the OIG published a series of reports that examined a 
range of user and veteran concerns with inadequate planning and 
implementation, which if left unremedied could pose patient safety 
risks and additional instances of harm in future rollouts. Three OIG 
reports released in March 2022 identified EHRM issues connected to 
medication management, care coordination, and the ticketing process 
used by Mann-Grandstaff VAMC providers to request help and resolve 
problems.
    Finally, in July 2022, the OIG determined that the new EHR system 
directed thousands of medical orders to an ``unknown queue'' that were 
not evident to the clinical and administrative staff required to 
address them. The OIG also found that VHA determined the unknown queue 
created significant risk and caused harm to multiple patients. As 
recently as June 2022, hundreds of orders remained in the unknown queue 
across VA sites implementing the new system.
 review of access to care and capabilities during va's transition to a 
 new electronic health record at the mann-grandstaff va medical center 
                              (april 2020)
    VA expected a productivity drop associated with the facility's 
preparations for going live with the new EHR system.\20\ Mann-
Grandstaff VAMC leaders consulted with DoD staff, who transitioned to 
the Cerner system in 2017 and experienced a 30-percent decrease in 
productivity for the subsequent months. VA had plans to mitigate the 
impact on facility personnel for the March 2020 go-live event, 
including adding facility staff, enhancing clinical space, changing 
clinic processes, and a greater use of community care. At publication, 
however, the OIG did not find evidence of VA providing final guidance 
to Mann-Grandstaff VAMC leaders on carrying out these plans.
---------------------------------------------------------------------------
    \20\ VA OIG, Review of Access to Care and Capabilities during VA's 
Transition to a New Electronic Health Record System at the Mann-
Grandstaff VA Medical Center Spokane Washington, April 27, 2020.
---------------------------------------------------------------------------
    Some of the problems that emerged were foreseeable. OEHRM and 
Cerner determined in July 2019 that not all anticipated capabilities of 
the new EHR would be available for the March 2020 go-live date. Mann-
Grandstaff VAMC leaders and staff told the OIG of concerns related to 
the deployment of limited capability sets that led to significant gaps 
in functionality. For example, the MyHealtheVet portal was the most 
frequently used method for patients to request prescription refills, 
but it would not be connected to the new EHR.\21\ Facility leaders and 
staff told the OIG of safety concerns related to losing access to the 
MyHealtheVet electronic refill portal. The OIG was unable to determine 
all potential patient safety risks associated with the new EHR, but the 
work-around for the electronic prescription refill process alone 
presented significant concerns as it could have impacted a patient's 
ability to fill a life-sustaining medication after go-live. Follow-on 
work, discussed later in this statement, conducted by the OIG after 
Mann-Grandstaff VAMC began using the new EHR system, validated numerous 
of these medication management and prescription delivery services.
---------------------------------------------------------------------------
    \21\ My HealtheVet, Get to Know Rx Refill Options, https://
www.myhealth.va.gov/mhv-portal-web/ss20180423-prescription- refill-
options-for-veterans. (The website was accessed on July 6, 2021.) 
MyHealtheVet is an online personal health portal patients can access to 
schedule appointments, view medical records, refill prescriptions, and 
send secure messages to their care providers.
---------------------------------------------------------------------------
    The OIG made eight recommendations, of which three remain open. The 
three that remain open call for VA to evaluate the impact of the new 
EHR implementation on productivity and provide operational guidance to 
facilities on mitigating the impact of the transition and any 
undeveloped aspects of the software on users and patients. The 
recommendations' text and status can be found in appendix H.
    A trilogy of reports released in March 2022 responded to many 
complaints submitted to the OIG hotline and requests from congressional 
offices following the new EHR's deployment at the Mann-Grandstaff VAMC. 
OIG healthcare inspections staff began work on two efforts to address 
several priority concerns-medication management and patient care 
coordination. During this work, the OIG team identified further 
challenges with the trouble ticketing process for system users to 
submit concerns or requests for help, and the OIG team determined that 
some previously identified deficiencies were still unresolved. 
Consequently, the healthcare oversight team started a third effort to 
examine why problems were not addressed and to highlight the underlying 
causal factors. When VA responded to the three reports in early March 
2022-nearly 18 months after going live in October 2020-VA actions to 
resolve issues were limited. The OIG identified 46 issues that were 
unresolved after the OIG completed its inspection in June 2021, but 
only seven were resolved as of March 2022. \22\
---------------------------------------------------------------------------
    \22\ The allegations substantiated but unresolved in the trilogy of 
reports date from March 2022. VA requested an extension until September 
16, 2022 on providing its first update as to the status of its work to 
resolve these issues, so at this time, the OIG does not have an update 
on VA's progress.
---------------------------------------------------------------------------
  medication management deficiencies after the new ehr go-live at the 
               mann- grandstaff vamc (march 2022 report)
    EHRs can improve clinical decision-making and minimize human error, 
but the risk of patient harm increases when systems have poor 
usability, workflows, or data inputs. The first in the trilogy of 
healthcare inspections focused on medication management for patients 
subject to the new EHR at Mann- Grandstaff VAMC.\23\ This included 
tracking and managing lists of medication, ordering, and promptly 
getting them to patients. Ensuring patients receive the correct 
medication in a timely manner is critical, given many patients are 
older with numerous medical conditions treated with multiple 
medications.
---------------------------------------------------------------------------
    \23\ VA OIG, Medication Management Deficiencies after the New 
Electronic Health Record Go-Live at the Mann-Grandstaff VA Medical 
Center in Spokane, Washington, March 17, 2022.
---------------------------------------------------------------------------
    The OIG grouped the various complaints regarding medication 
management into three categories: data migration, medication orders, 
and medication reconciliation.
Data Migration
    For this report, data migration focused on the transfer of patient 
information from VA's legacy EHR to the new system. Deficiencies were 
found with patient contact information, patient medication lists, and 
formulary lists that included medications and supplies unavailable at 
the facility.

  --Patient Contact Information: Prior to going live, VA migrated 
        contact information and clinical data for approximately 88,000 
        veterans to the new EHR. The OIG found that outdated DoD data 
        overwrote VHA's patient contact information, such as name, 
        address, telephone number, and email address when data were 
        migrated to the new EHR. Consequently, VA patients were delayed 
        in receiving medications through the mail order pharmacy 
        system.

  --Medication Lists: The OIG substantiated that medication lists, 
        migrated as ``free text'' per VHA's request, contained 
        inaccuracies. Because medication lists did not import properly, 
        care providers used work-arounds, including manual reentry to 
        generate accurate medication lists. Staff described this 
        process as ``overwhelming'' and time-consuming.

  --Medication Formulary: The new EHR's formulary included many 
        medications not available at Mann-Grandstaff or on VA's 
        national formulary. Consequently, care providers unknowingly 
        selected nonformulary or unavailable supplies. These selections 
        increased risks for errors, potentially raised costs for VA, 
        and added work for care providers and pharmacy staff.
Medication Orders
    The OIG substantiated 10 of 12 allegations related to the 
mismanagement of medication orders. The identified problems affect 
every aspect of the process from orders failing to process to patients' 
recurring future medication orders being automatically discontinued 
without notice to providers.

                              Summary of Medication Order Allegations and Findings
 
----------------------------------------------------------------------------------------------------------------
          Medication Orders                       Allegations               OIG Determination         Status
----------------------------------------------------------------------------------------------------------------
     Future Order Discontinuance       The new EHR discontinued future   Substantiated           Unresolved
                                        medication orders written by
                                        providers.
 
                                       Discontinued future medication    Substantiated           Unresolved
                                        orders required providers to
                                        write ``stat'' or place
                                        immediate orders, causing
                                        medication delays for patients.
 
                                       Discontinued future medication    Substantiated           Unresolved
                                        orders led absent providers to
                                        arrange for colleagues to write
                                        orders for recurring
                                        medications, creating
                                        inefficiencies and increasing
                                        risks for orders being missed
                                        and possible patient safety
                                        issues.
                                      --------------------------------------------------------------------------
      Unauthorized Orders Placed       Registered nurses could order     Substantiated           Unresolved
                                        medications without provider
                                        approval.
                                      --------------------------------------------------------------------------
   Outpatient Orders Not Processed     Pharmacy staff did not process    Not Substantiated       Not Applicable
                                        outpatient orders.
                                       Some outpatient orders failed to  Substantiated           Unresolved
                                        process and appeared missing to
                                        nonpharmacy staff.
                                      --------------------------------------------------------------------------
         Lack of Notification          Notifications were not sent to    Substantiated           Unresolved
                                        prescribing providers and
                                        pharmacists about future
                                        recurring injectable medication
                                        orders that were discontinued
                                        or outpatient medication orders
                                        that did not process.
                                      --------------------------------------------------------------------------
           Confusing Alerts            Medication alerts were            Substantiated           Unresolved
                                        confusing, and providers did
                                        not receive training on
                                        interpreting them.
                                      --------------------------------------------------------------------------
     Prescription Status Unclear       Providers were unable to assess   Substantiated           Unresolved
                                        the status of a filled
                                        prescription order.
                                      --------------------------------------------------------------------------
     Lack of Tracking for Mailed       Pharmacy staff were unable to     Not Substantiated       Not Applicable
        Controlled Substances           consistently track mailed
                                        controlled substance
                                        prescriptions.
 
                                       Nonpharmacy staff could not       Substantiated           Unresolved
                                        consistently track mailed
                                        controlled substance
                                        prescriptions.
                                      -----------------------------------
 Prescription Drug Monitoring Program  After completing a PDMP query,    Substantiated           Unresolved
                                        providers' notes were not
                                        automatically populated in
                                        alignment with VHA policy,
                                        requiring additional work for
                                        providers.
----------------------------------------------------------------------------------------------------------------

Medication Reconciliation
    The OIG substantiated that inaccurate medication lists in the new 
EHR challenged staff conducting reconciliations. This critical process 
identifies and resolves any medication discrepancies found in an EHR 
with the information supplied by the patient or caregiver. Accurate 
medication lists guide providers' treatment decisions, and inaccuracies 
could have significant health consequences for a patient. The OIG 
observed that poor training led to a knowledge gap that contributed to 
errors and helped explain varying user experiences.

                          Summary of Medication Reconciliation Allegations and Findings
 
----------------------------------------------------------------------------------------------------------------
      Medication Reconciliation                   Allegations               OIG Determination         Status
----------------------------------------------------------------------------------------------------------------
    Medication List Discontinuity      Staff had to update medication    Substantiated           Unresolved
                                        lists at every visit because
                                        prior medication information
                                        revisions did not carry over.
 
                                       Medications disappeared from      Substantiated           Unresolved
                                        reconciled medication lists,
                                        and lists were inaccurate after
                                        reconciliation.
 
                                       Staff manually entered            Substantiated           Unresolved
                                        medication lists post-
                                        reconciliation, which increased
                                        risk for error and safety
                                        concerns.
 
                                       Medication reconciliation         Substantiated           Unresolved
                                        required a significant amount
                                        of time to complete per
                                        patient.
                                      --------------------------------------------------------------------------
     Medication List Inaccuracies      Discontinued and expired          Substantiated           Unresolved
                                        medications were not viewable
                                        during reconciliation, creating
                                        a patient safety issue.
 
                                       Medications administered in a     Substantiated           Unresolved
                                        clinic did not appear on
                                        medication lists, creating a
                                        patient safety issue.
                                      --------------------------------------------------------------------------
Medication Lists Unsuited for Patient  Medication lists were not         Substantiated           Unresolved
                 Use                    patient-friendly.
----------------------------------------------------------------------------------------------------------------

    The two report recommendations can be found in appendix I. VA 
concurred with the first recommendation, which requires extensive 
software updates that VA indicated may take over a year from 
publication to implement. The second recommendation called for VA to 
ensure medication management issues related to the new EHR identified 
after the inspection be reported to the OIG. VA did not concur with 
this recommendation, citing the difficulty of a continuous, open 
reporting requirement to the OIG. This is not an open-ended 
recommendation, however, and could be closed after VA demonstrates an 
effective and sustainable process to identify and address patient 
safety issues. VA already must provide this information to the OIG 
regardless of whether VA concurs with the recommendation, and the OIG 
will continue this oversight work.
 care coordination deficiencies after the new ehr go-live at the mann-
                  grandstaff vamc (march 2022 report)
    The second report in the trilogy addressed an expansive list of 
allegations categorized as care coordination concerns.\24\ Care 
coordination involves numerous EHR functions that facilitate how care 
is synchronized both among healthcare providers and directly with the 
patient. As an example of these challenges, the VAMC's coordinator for 
the new EHR's patient portal reported a backlog after the go- live of 
over 300 voicemail messages from patients unable to access the portal. 
During the pandemic, the portal was a central means for patients to 
communicate with providers.
---------------------------------------------------------------------------
    \24\ VA OIG, Care Coordination Deficiencies after the New 
Electronic Health Record Go-Live at the Mann-Grandstaff VA Medical 
Center in Spokane, Washington, March 17, 2022.
---------------------------------------------------------------------------
    The OIG further sorted the allegations into eight categories. Each 
had multiple deficiencies:

    1. Patient Record Flags: Patient record flags denoting patients at 
high risk for suicide and disruptive behavior in the legacy EHR failed 
to activate for some Mann-Grandstaff VAMC patients. Some identified 
concerns about patient record flag functionality in the new EHR stemmed 
from system design, while others related to deficits in training on the 
new EHR's workflow. The flags are not as obvious in the new system as 
they were in the legacy EHR. In some new EHR views, staff had to 
navigate multiple steps to find information about the flag and relevant 
precautions. Of the six substantiated allegations, only two remained 
unresolved: the visibility of the flag and national-level data sharing 
of active record flags for patients at high risk for suicide.

    2. Data Migration: As previously discussed, deficiencies were found 
in the migration of patient information, such as incorrect patient 
names, genders, and contact information. Discussions continued between 
VA and DoD regarding business rule updates needed to improve 
interoperability and ensure accurate data migration in the face of 
policy differences between VA and DoD.

    3. Scheduling Process: Initial allegations received by the OIG 
cited delays in scheduling and inadequate appointment information and 
reminders in the new EHR. Reminders to veterans and caregivers did not 
always specify if appointments were by telephone rather than in-person, 
resulting in some patients traveling to the facility for telephone 
appointments. The OIG was also alerted to problems with the new self-
scheduling tool that resulted in Washington State patients 
inadvertently self-scheduling appointments at the Columbus clinic. Of 
the five related substantiated allegations, four remained unresolved, 
particularly related to delays in scheduling primary care appointments, 
the type of appointment, and the information contained on appointment 
reminders.

    4. VA Video Connect: This VHA telehealth service technology enables 
veterans to meet virtually with VA healthcare providers from anywhere, 
using encrypted video. The OIG substantiated some allegations that 
appointments failed due to broken links, incorrect time zones, and 
links being sent to outdated email addresses. VA needed to completely 
resolve only the last allegation, as some veterans were still having to 
contact DoD to have their contact information updated.

    5. Referral Management: Deficiencies in implementing the Ambulatory 
Referral Management function decreased care providers' ability to 
manage patients' referrals in the provider's own clinical service, 
particularly in the behavioral health department, and with other 
outpatient services in VHA. These breakdowns could lead to delays and 
affect patient experiences at VHA more generally. For example, 
providers had no easy way to determine if a referral had been acted on. 
Certain aspects of system configuration, workflow errors, 
interoperability deficits, and insufficient training contributed to 
staffs' difficulties with handling referrals. The three substantiated 
issues remained unresolved.

    6. Laboratory Orders: The OIG was alerted to ``disappearing'' 
laboratory orders that never reached lab personnel. The system 
configurations and training deficits were factors in these failures. 
Ordering providers were shown a confusing array of options. 
Additionally, staff were challenged in tracking the orders, and many 
results were delayed in being returned. These issues created more 
opportunities for human error as staff used work-arounds to get results 
that informed care delivery. These three substantiated issues were 
unresolved.

    7. Patient Portal and Secure Messaging: As mentioned above, when 
the new EHR went live, many patients could not access the portal, 
affecting access to tools that supported coordination of care, such as 
secure messaging and online prescription refills. VA staff reported 
that system changes completed by OIT resolved some causes of this 
disruption, while other resolutions were in progress.

    8. Documentation Processes: While the OIG did not substantiate all 
allegations received related to documentation process problems, 
facility staff reported experiencing challenges in effectively 
navigating and using some of the new EHR capabilities. Insufficient 
end-user training and misperceptions about certain new EHR 
functionalities appeared to be the sources of the difficulties. VA 
started using a new method, the financial identification number (FIN), 
to document workload associated with between-visit care, which VHA did 
not historically record. This required numerous steps for providers, 
creating additional work and confusion. Another example is a 
configuration issue in which not all International Classification of 
Disease 10 diagnostic codes were available in the new EHR, affecting 
providers' ability to correctly code patient diagnoses. Of the three 
substantiated allegations, the FIN and diagnostic codes were unresolved 
at the time of publication.
    For this report, the OIG made one recommendation, located in 
appendix J, and it remains open.
    ticket process concerns and underlying factors contributing to 
 medication management and care coordination deficiencies (march 2022 
                                report)
    The OIG issued this third report to provide an analysis of the 
persistent issues with the ticket process used for reporting problems 
and requesting assistance at Mann-Grandstaff VAMC, including 
identifying the underlying causal factors.\25\ From the October 2020 
go-live date through March 31, 2021, new EHR end users placed over 
38,700 tickets. OIG staff analyzed the help ticket system for key terms 
for each allegation and checked 4,094 tickets related to the issues 
discussed in the two prior reports.
---------------------------------------------------------------------------
    \25\ VA OIG, Ticket Process Concerns and Underlying Factors 
Contributing to Deficiencies after the New Electronic Health Record Go-
Live at the Mann-Grandstaff VA Medical Center in Spokane, Washington, 
March 17, 2022.
---------------------------------------------------------------------------
Ticket Process Challenges
    The OIG team reviewed ticket comments to understand facility 
staffs' frustration with getting fixes and changes. Although VA 
initiated a strategic review to address these concerns, there were 
limited process changes. The ticket process challenges the OIG found 
include the following:

  --Cerner's service desk support staff were not able to view and 
        replicate reported issues. While Cerner had a mirror version of 
        the DoD EHR, a mirror version of the Mann-Grandstaff VAMC's EHR 
        was not built.\26\ OEHRM staff were frustrated that when Cerner 
        support staff could not reproduce a reported issue they closed 
        the ticket, potentially delaying the problem's resolution.
---------------------------------------------------------------------------
    \26\ In the response VA gave to the OIG before publication, VA said 
Cerner service desk support staff had given access to the EHR's 
production version. The OIG will review VA's evidence during the 
follow-up process to determine if that is the case.

  --The same Cerner staff closed tickets before resolving the issues. 
        Closing tickets without resolving the concerns could result in 
        patient safety issues as well as the propagation of similar 
        issues at future implementation sites. Facility staff also 
---------------------------------------------------------------------------
        reported feeling a lack of support.

  --Ticket status was not communicated to end users. As part of VA's 
        agreement with Cerner, end users were to be notified and given 
        the opportunity to review whether the proposed or implemented 
        resolution addressed the reported issue before Cerner closed 
        the ticket. Mann-Grandstaff VAMC staff reported during 2021 
        that Cerner's service desk staff were unhelpful or rude.

  --Mann-Grandstaff VAMC staff sometimes created work-arounds instead 
        of placing tickets. Due to the challenges, Mann-Grandstaff VAMC 
        users began creating work-arounds to accomplish tasks, which 
        can increase patient safety risks, create inefficiencies, and 
        bypass safeguards.

    This report validated deficient ticket processes identified earlier 
in VA's ``Electronic Health Record Comprehensive Lessons Learned'' 
report released in July 2021.\27\ While VA had identified proposed 
measures to monitor these process changes, their July 2021 report said 
the measures had not been finalized and were under review.
---------------------------------------------------------------------------
    \27\ VA, Electronic Health Record Comprehensive Lessons Learned 
Report, November 2021. The report was initially released in July 2021 
and updated in November 2021.
---------------------------------------------------------------------------
Underlying Factors of Substantiated Allegations in Companion 
        Inspections
    To probe into the causes of the allegations in the two companion 
inspections regarding medication management and care coordination 
issues, the inspection team identified five underlying factors:

    1. EHR Usability Problems. Poor usability has been linked to 
increased patient safety risks, inefficiencies, and care provider 
frustration and stress. Among other issues, the OIG found that the user 
interface was not optimized for workflows; inefficient navigation 
hampered staff; patient datawere in different sections of the EHR; and 
restrictive definitions of user roles assignments, which defined 
employees' capabilities in the system, limited the information staff 
could see.

    2. Training Deficits. The OIG found insufficient training content, 
support, and an approach to training that did not provide staff with 
the underlying reasons for the actions they should take.

    3. Interoperability Challenges. Staff must have access to 
information needed to perform their work from within and across VHA. 
This was hampered by the data migration issues previously discussed, 
the failure of information to transfer to the Consolidated Mail 
Outpatient Pharmacy, and information not properly transferring to 
national-level VHA databases.

    4. Fixes and Refinement Needs. The OIG identified that some 
substantiated allegations were unresolved and required fixes after 
going live, as well as refinements to address errors in system 
workflows and changes to components of the new EHR. For example, staff 
were initially unable to view patients' service-connected conditions 
noted by the Veterans Benefits Administration from the new EHR, which 
led to an inability to document these conditions for healthcare 
delivery purposes.

    5. Problem Resolution Process Challenges. Successful EHR 
implementation requires effective pathways for resolving identified 
problems, and as discussed in this trilogy of reports, the ticket 
process for resolving questions and concerns had several deficiencies.

    For this report, the OIG made three recommendations, found in 
appendix K, and all are open.
 the new ehr's unknown queue caused multiple patient harm events (july 
                              2022 report)
    This review looked at one aspect of the question of whether the new 
EHR resulted in any patient harm.\28\ In May 2021, after VHA identified 
several patient safety concerns, a VHA National Center for Patient 
Safety team went to Mann-Grandstaff VAMC with their work continuing 
through the year. In late 2021, the team drafted a report and held a 
Safety Summit where they ranked dozens of safety concerns based on 
severity, identifying the ``unknown queue'' as one of the most severe.
---------------------------------------------------------------------------
    \28\ VA OIG, The New Electronic Health Record's Unknown Queue 
Caused Multiple Events of Patient Harm, July 14, 2022.
---------------------------------------------------------------------------
    Information about patient harm due to the new EHR was presented to 
the VA Deputy Secretary in November 2021. In December 2021, the Deputy 
Secretary forwarded information about harms due to the unknown queue to 
the executive director of EHRM IO. From October 24, 2020, through May 
8, 2022, VHA identified 1,134 patient safety events related to the new 
EHR. VHA's analysis identified one catastrophic patient harm (death or 
major permanent loss of function) and two major patient harm cases 
(permanent lessening of bodily functioning), one of which was related 
to the unknown queue.\29\
---------------------------------------------------------------------------
    \29\ VA defines ``catastrophic harm'' as ``death or major permanent 
loss of function (sensory, motor, physiologic, or intellectual) not 
related to the natural course of the patient's illness or underlying 
condition (i.e., acts of commission or omission).'' VA defines ``major 
harm as ``permanent lessening of bodily functioning (sensory, motor, 
physiologic, or intellectual) not related to the natural course of the 
patient's illness or underlying condition (i.e., acts of commission or 
omission).'' [bolding not added by the OIG]
---------------------------------------------------------------------------
    The intent of the unknown queue was to capture orders entered by 
providers that the new EHR cannot deliver to the intended location. The 
new EHR's design allowed providers to select locations from a drop-down 
menu that, depending on the specific order, the system would not be 
recognize as a ``match.'' This ``mismatch'' would send orders to the 
unknown queue and not to the requested service location to initiate the 
ordered care. Notably, the new EHR did not alert the healthcare 
providers that the order was not delivered to the intended location.
    Orders from care providers began populating the unknown queue 
immediately after the facility went live. Staff had to re-input the 
orders after discovering the issue, expending many hours of labor then 
and during the clinical reviews that assessed the harm patients may 
have suffered. Cerner did take steps with VA to mitigate the problem at 
Mann-Grandstaff VAMC by removing unmapped locations in September 2021. 
As of February 2022, an alert was being sent if a provider created an 
order with an unmapped location. However, prior to March 2022, VHA 
could not generate a report of unknown queue orders itself. Cerner 
acknowledged that the unknown queue's ongoing risk would require 
mitigation at future go-live sites, noting the need to continuously 
reinforce the guidance on managing the queue.
    The OIG found that Cerner did not inform VA end users of the 
unknown queue or provide guidance to address the unknown queue in 
advance of going live with the new EHR. A Cerner vice president, 
identified by the company's general counsel as an unknown queue subject 
matter expert, also reported having no knowledge that VA was told about 
it before going live. Following the OIG's transmittal of the draft 
report to VA in June 2022, Cerner provided EHRM IO with documentation 
that asserted a VA leader approved the use of the unknown queue in 
January 2020. However, that VA leader and their supervisor told OIG 
staff they had no awareness of the unknown queue prior to going live.
    VHA itself assessed the risk as major severity, frequently 
occurring, and very difficult to detect and initiated a clinical review 
in June 2021 to ensure orders were acted on and to assess patients for 
harm. VHA's clinical reviewers conducted 1,286 assessments and 
identified 148 adverse events (with an additional one later found by 
VHA to be a major harm, bringing the total to 149) for patients:

  --Major harm: 2

  --Moderate harm: 52

  --Minor harm: 95

    As an example of major harm, a provider entered a psychiatric care 
order for a patient experiencing homelessness and identified as at-risk 
for suicide. The new EHR sent the order to the unknown queue. The 
patient was not scheduled for follow-up care and later contacted the 
Veterans Crisis Line reporting a razor in hand and a plan to take their 
own life. The patient was hospitalized for psychiatric care.
    The OIG has concerns with the effectiveness of the plan to mitigate 
the unknown queue's safety risk. Facility leaders reported using the 
mitigation process to monitor and manage the queue but shared that 
steps in the process could still lead to orders remaining in the queue. 
In June 2022, when the OIG met with VA leaders to discuss this report, 
VA said that work to address the unknown queue was considered complete 
and that, on average, there were 28 orders in the unknown queue report. 
However, on that day, the OIG generated a report showing 522 total 
orders across the six VA facilities using the new EHR. The OIG made two 
recommendations, found in appendix L, and both are open.\30\
---------------------------------------------------------------------------
    \30\ Appendices M and N are the recommendations flowing from two 
additional OIG reports on EHR implementation. The first relates to the 
availability and use of data in the new EHR, and the second is a joint 
report with the DoD Office of Inspector General on the progress of VA 
and DoD in their interoperability efforts.
---------------------------------------------------------------------------
                               conclusion
    This subcommittee and VA have focused tremendous resources on the 
successful transition to the new EHR system. The OIG's work on the 
topic reveals there are still considerable challenges, particularly 
regarding the true costs and scope-especially given the lack of a 
reliable and comprehensive integrated master schedule. Additionally, 
physical and IT infrastructure upgrades at all VHA facilities remain, 
as does the need for effective training and practice before VA staff 
can properly use the new EHR.
    The OIG is committed to providing impactful and practical 
recommendations that flow from its oversight work to help VA deploy the 
new EHR efficiently and in a manner that improves veterans' 
experiences. While each report has specific recommendations intended to 
improve the EHRM program, there are broader concerns that many of the 
recommendations reflect. A primary concern is governance: Are the right 
structures in place to identify potential issues to prevent their 
occurrence, to prioritize those issues that may affect prompt quality 
care to patients, and to resolve those issues before additional 
deployments? Another key concern is transparency: Is there transparency 
among EHRM IO, the facilities, VHA, OIT, and Oracle Cerner? Full and 
candid information sharing will help build confidence that issues are 
being identified, prioritized, and adequately addressed. As VA moves 
toward deployment in more complex facilities, proper governance and 
transparency will be necessary to get it right. Failures in these areas 
risk cascading problems that put the entire program in jeopardy. The 
OIG will continue to monitor EHRM efforts to help recommend 
improvements needed to fulfill its promise to the veteran community and 
make the most effective use of taxpayer dollars. Chairman Heinrich, 
this concludes my statement. I would be happy to answer any questions 
you or other members may have.
  appendix a. va responses to recommendations: deficiencies in infra- 
 structure readiness for deploying va's new ehr system, april 27, 2020
    1. The executive director of OEHRM should establish an 
infrastructure-readiness schedule for future deployment sites that 
incorporates lessons learned from the DoD. Status: Closed October 1, 
2020.

    2. The executive director of OEHRM should reassess the enterprise-
wide deployment schedule to ensure projected milestones are realistic 
and achievable, considering the time needed for facilities to complete 
infrastructure upgrades. Status: Closed October 1, 2020.

    3. The executive director of OEHRM should implement tools to 
comprehensively monitor the status and progress of medical devices at 
the enterprise level. Status: Closed September 21, 2021.

    4. The executive director of OEHRM should standardize 
infrastructure requirements in conjunction with the VHA and the OIT and 
ensure those requirements are disseminated to all necessary staff. 
Status: Closed July 16, 2021.

    5. The executive director of OEHRM should evaluate physical 
infrastructure for consistency with OEHRM requirements and monitor 
completion of those evaluations. Status: Open. VA's targeted completion 
date: March 2021.

    6. The executive director of OEHRM should fill infrastructure-
readiness team vacancies until optimal staffing levels are attained. 
Status: Closed September 12, 2022.

    7. The executive director of OEHRM should ensure physical security 
assessments are completed and addressed at future EHR deployment sites. 
Status: Open. VA's targeted completion date: None initially provided.

    8. The Mann-Grandstaff VAMC director should ensure all access 
points to physical infrastructure are secured and inaccessible to 
unauthorized individuals. Status: Closed October 1, 2020.
appendix b. va responses to recommendations: deficiencies in reporting 
 reliable physical infrastructure cost estimates for the ehrm program, 
                              may 25, 2021
    1. The executive director for OEHRM should ensure an independent 
cost estimate is performed for program life cycle cost estimates 
including related physical infrastructure costs funded by VHA. Status: 
Open. VA's targeted completion date: 9--12 months from contract start.

    2. The VA assistant secretary for management and chief financial 
officer should ensure the Office of Programming, Analysis and 
Evaluation, or another office performing its duties, conducts 
independent cost estimates as required by VA financial policy, and 
performs an independent estimate of EHRM program life cycle cost 
estimates including physical infrastructure. Status: Open. VA's 
targeted completion date: 9--12 months from contract start.

    3. The director of special engineering projects for VHA's Office of 
Healthcare Environment and Facilities Programs should develop a 
reliable cost estimate for EHRM program-related physical infrastructure 
in accordance with VA cost-estimating standards and incorporate costs 
for upgrade needs identified in facility self-assessments and scoping 
sessions. Status: Closed July 26, 2022.

    4. The director of special engineering projects should also 
continuously update physical infrastructure cost estimates based on 
emerging requirements and identified project needs. Status: Closed 
January 20, 2022.

    5. The executive director for OEHRM should ensure costs for 
physical infrastructure upgrades funded by VHA or other sources needed 
to support the EHRM program are disclosed in program life cycle cost 
estimates presented to Congress. Status: Open. VA's targeted completion 
date: July 31, 2021.
  appendix c. va responses to recommendations: unreliable information 
technology infrastructure cost estimates for the ehrm program, july 7, 
                                  2021
    1. The executive director of OEHRM should ensure an independent 
cost estimate is performed for program life-cycle cost estimates 
related to IT infrastructure costs. Status: Open. VA's targeted 
completion date: This is part of the strategic review and will be 
provided as soon as information is available.

    2. The executive director of OEHRM should reassess the cost 
estimate for EHRM program-related IT infrastructure and refine as 
needed to comply with VA's cost-estimating standards. Status: Open. 
VA's targeted completion date: Under active revision as part of the 
strategic review and will be provided as soon as information is 
available.

    3. The executive director of OEHRM should develop procedures for 
cost-estimating staff that align with VA cost-estimating guidance. 
Status: Open.VA's targeted completion date: Under active revision as 
part of the strategic review and will be provided as soon as 
information is available.

    4. The executive director of OEHRM should ensure costs for all IT 
infrastructure upgrades funded by OIT and VHA or other sources needed 
to support the EHRM program are disclosed in program life- cycle cost 
estimates presented to Congress. Status: Open. VA's targeted completion 
date: This is part of the strategic review and will be provided as soon 
as information is available.

    5. The executive director of OEHRM should formalize agreements with 
OIT and VHA identifying the expected contributions from each entity 
toward IT infrastructure upgrades in support of the EHRM program. 
Status: Open. VA's targeted completion date: This is part of the 
strategic review and will be provided as soon as information is 
available.

    6. The executive director of OEHRM should establish procedures that 
identify when life-cycle cost estimates should be updated and ensure 
those updated estimates are disclosed in the program's congressionally 
mandated reports. Status: Open. VA's targeted completion date: This is 
part of the strategic review and will be provided as soon as 
information is available.
 appendix d. va responses to recommendations: the ehrm program did not 
 fully meet the standards for a high quality, reliable schedule, april 
                                25, 2022
    1. The EHRM program management office executive director should 
comply with internal guidance and ensure the development of an IMS that 
complies with standards adopted from GAO for scheduling, Status: Open. 
VA's targeted completion date: December 2022.

    2. The EHRM program management office executive director should 
take action to improve stakeholder coordination in the development of 
the program schedules to ensure activities from all relevant VA 
entities are included. Status: Open. VA's targeted completion date: 
August 2022.

    3. The EHRM program management office executive director should 
develop procedures for when and how staff should perform an initial 
schedule risk analysis and conduct periodic updates as needed. Status: 
Open. VA's targeted completion date: December 2022.

    4. The EHRM program management office executive director should 
ensure consistency between contract language and program office plans 
or other guidance identifying the entity or individuals responsible for 
developing and maintaining the program's WBS and IMS. Status: Open. 
VA's targeted completion date: November 2022.

    5. The EHRM program management office executive director should 
evaluate the contract requirements for schedule management and modify 
as needed to ensure clear roles and expectations for further 
development and maintenance of the IMS. Status: Open. VA's targeted 
completion date: December 2022.

    6. The EHRM program management office executive director should 
comply with the Federal Acquisition Regulation and issue guidance to 
accept deliverables not separately priced before invoice payment. 
Status: Open. VA's targeted completion date: May 2022.
    appendix e. va responses to recommendations: senior staff gave 
 inaccurate information to oig reviewers of ehr training, july 14, 2022
    1. Issue a clarifying communication to the office's personnel that 
all staff have a right to speak directly and openly with OIG staff 
without fear of retaliation, and that, irrespective of any processes 
established to facilitate the flow of information, EHRM IO personnel 
are encouraged to communicate directly with OIG staff when needed to 
proactively clarify requests and avoid confusion. Status: Closed 
September 7, 2022.

    2. Provide clear guidance that the office's personnel must provide 
timely, complete, and accurate responses to requests for all data or 
information without alteration, unless other formats are requested, 
with full disclosure of the methodology, any data limitations, or other 
relevant context. This includes prompt OIG access to entire datasets 
consistent with the Inspector General Act of 1978, as amended. Status: 
Closed September 7, 2022.

    3. Determine whether any administrative action should be taken with 
respect to the conduct or performance of the executive director of 
Change Management. Status: Open. VA's targeted completion date: July 
2022.

    4. Determine whether any administrative action should be taken with 
respect to the conduct or performance of Change Management's director 
for training strategy. Status: Closed August 15, 2022.
appendix f. va responses to recommendations: training deficiencies with 
va's new ehr system at the mann-grandstaff vamc in spokane, washington, 
                              july 8, 2021
    1. The USH explores the establishment of a group of VHA staff 
composed of core user roles with expertise in VHA operations and Cerner 
EHR use with data architect level knowledge to lead the effort of 
generating optimized VHA clinical and administrative workflows. Status: 
Open. VA's targeted completion date: September 2021.

    2. The deputy secretary establishes an EHR training domain that 
ensures close proximation to the production environment and is readily 
available to all end users during and following training. Status: Open. 
VA's targeted completion date: January 2022.

    3. The deputy secretary ensures end users receive training time 
sufficient to impart the skills necessary to use the new EHR prior to 
implementation. Status: Open. VA's targeted completion date: January 
2022.

    4. The deputy secretary ensures the user role assignment process 
addresses identified facility leaders and staff concerns. Status: Open. 
VA's targeted completion date: January 2022.
    5. The deputy secretary ensures Cerner trainers and adoption 
coaches have the capability to deliver end user training on Cerner and 
VHA EHR software workflows. Status: Open. VA's targeted completion 
date: January 2022.

    6. The deputy secretary evaluates the process of super user 
selection and takes action as indicated. Status: Closed February 1, 
2022.

    7. The deputy secretary reviews OEHRM's performance-based service 
assessments for Cerner's execution of training to determine whether 
multiple, recurrent concerns are being accurately captured and 
addressed. Status: Open. VA's targeted completion date: January 2022.

    8. The deputy secretary oversees the revision of an OEHRM training 
evaluation plan and ensures implementation of stated objectives. 
Status: Open. VA's targeted completion date: January 2022.

    9. The deputy secretary reviews the EHRM governance structure and 
takes action as indicated to ensure the under secretary for health 
(USH) role in directing and prioritizing EHRM efforts is commensurate 
with VHA's role in providing safe patient care. Status: Closed February 
1, 2022.

    10. The USH establishes guidelines and training to capture new EHR-
related patient complaints, including patient advocacy. Status: Closed 
August 5, 2022.

    11. The USH ensures an assessment of employee morale following 
implementation of a new EHR and takes action as indicated. Status: 
Closed February 1, 2022.
  appendix g. va responses to recommendations: new patient scheduling 
system needs improvement as va expands its implementation, november 10, 
                                  2021
    1. The USH coordinates with the OEHRM executive director to 
continue to make improvements to the scheduling training as needed to 
address feedback from schedulers. Status: Open. VA's targeted 
completion date: January 2022.

    2. The USH coordinates with the OEHRM executive director to require 
that some schedulers from each clinic fully test the scheduling 
capabilities of their clinics, solicit feedback from the schedulers to 
identify system or process issues, and make improvements as needed. 
Status: Open. VA's targeted completion date: November 2021.

    3. The USH coordinates with the OEHRM executive director to issue 
guidance to facility staff on which date fields in the new system 
schedulers should use to measure patient wait times. Status: Open. VA's 
targeted completion date: February 2022.

    4. The USH coordinates with the OEHRM executive director to develop 
a mechanism to track and then monitor all tickets related to the new 
scheduling system, and then ensure OEHRM evaluates whether Cerner 
effectively resolved the tickets within the timeliness metrics 
established in the contract. Status: Open. VA's targeted completion 
date: December 2021.

    5. The USH coordinates with the OEHRM executive director to develop 
a strategy to identify and resolve additional scheduling issues in a 
timely manner as OEHRM deploys the new EHR at future facilities. 
Status: Open. VA's targeted completion date: December 2021.

    6. The USH coordinates with the OEHRM executive director to develop 
a mechanism to assess whether facility employees accurately scheduled 
patient appointments in the new scheduling system, and then ensure 
facility leaders conduct routine scheduling audits. Status: Open. VA's 
targeted completion date: July 2022.

    7. The USH coordinates with the OEHRM executive director to 
evaluate whether patients received care within the time frames directed 
by VHA policy when scheduled through the new system. Status: Open. VA's 
targeted completion date: July 2022.

    8. The OIG recommends that the VA OEHRM executive director provide 
guidance to schedulers to consistently address system limitations until 
problems are resolved. Status: Open. VA's targeted completion date: 
December 2021.
 appendix h. va responses to recommendations: review of access to care 
  and capabilities during va's transition to a new ehr system at the 
                  mann-grandstaff vamc, april 27, 2020
    1. The USH, in conjunction with OEHRM evaluates the impact of the 
new EHR implementation on productivity and provides operational 
guidance and required resources to facilities prior to go-live. Status: 
Open. VA's targeted completion date: Initial response at Initial 
Operating Capability go-live; revised versions at subsequent go-live 
dates.

    2. The USH, in conjunction with OEHRM, identifies the impact of the 
mitigation strategies on user and patient experience at go-live and 
takes action, as needed. Status: Open. VA's targeted completion date: 
Initial response at IOC go-live; revised versions at subsequent go-live 
dates.

    3. The executive director, OEHRM, in conjunction with the USH, 
ensures that clear guidance is given to facility staff on what EHR 
capabilities will be available at go-live. Status: Closed January 13, 
2021.

    4. The USH, in conjunction with OEHRM, reevaluates the EHRM 
deployment timeline to minimize the number of required mitigation 
strategies at go-live. Status: Open. VA's targeted completion date: May 
2020.

    5. The veterans integrated service network (VISN) director 
collaborates with facility leaders to implement VA-provided operational 
guidance and supports required resources needed throughout the 
transition to the new EHR system. Status: Closed July 31, 2021.

    6. The VISN director ensures that positions required for the 
transition to the new EHR system are staffed and trained prior to go-
live. Status: Closed October 16, 2020.

    7. The Mann-Grandstaff VAMC director ensures that community care 
consults are managed through go- live to ensure accuracy and 
completeness, and to avoid the need for manual reentry after go-live. 
Status: Closed September 22, 2021.

    8. The Mann-Grandstaff VAMC director ensures that patients receive 
medication refills in a timely manner throughout the transition to the 
new EHR system. Status: Closed September 22, 2021.
  appendix i. va responses to recommendations: medication management 
  deficiencies after the new ehr go-live at the mann-grandstaff vamc, 
                             march 17, 2022
    1. The deputy secretary ensures that substantiated and unresolved 
allegations discussed in this report are reviewed and addressed. 
Status: Open. VA's targeted completion date: May 2022.

    2. The deputy secretary ensures medication management issues 
related to the new EHR that are identified subsequent to this 
inspection be reported to the OIG for further analysis. Status: Open. 
VA's targeted completion date: None as VA did not concur with the 
recommendation.
    appendix j. va responses to recommendations: care coordination 
  deficiencies after the new ehr go-live at the mann-grandstaff vamc, 
                             march 17, 2022
    1. The deputy secretary ensures that substantiated and unresolved 
allegations noted in this report are reviewed and addressed. Status: 
Open. VA's targeted completion date: May 2022.
 appendix k. va responses to recommendations: ticket process concerns 
 and underlying factors contributing to deficiencies after the new ehr 
          go-live at the mann-grandstaff vamc, march 17, 2022
    1. The deputy secretary completes an evaluation of the new EHR 
problem resolution processes and takes action as warranted. Status: 
Open. VA's targeted completion date: March 2022.

    2. The deputy secretary completes an evaluation of the underlying 
factors of substantiated allegations identified in this report and 
takes action as warranted. Status: Open. VA's targeted completion date: 
May 2022.

    3. The deputy secretary ensures the EHRM deployment schedule 
reflects resolution of the allegations and concerns discussed in this 
report. Status: Open. VA's targeted completion date: March 2022.
  appendix l. va responses to recommendations: the new ehr's unknown 
      queue caused multiple events of patient harm, july 14, 2022
    1. The deputy secretary reviews the process that led to Cerner's 
failure to provide VA substantive information of the unknown queue and 
takes action as indicated. Status: Open. VA's targeted completion date: 
October 2022.

    2. The deputy secretary evaluates the unknown queue technology and 
mitigation process and takes action as indicated. Status: Open. VA's 
targeted completion date: October 2022.
  appendix m. va responses to recommendations: deficits with metrics 
 following implementation of the new ehr at the mann-grandstaff vamc, 
                              june 1, 2022
    1. The deputy secretary completes an evaluation of gaps in new EHR 
metrics and takes action as warranted. Status: Open. VA's targeted 
completion date: October 2022.

    2. The deputy secretary completes an evaluation of factors 
affecting the availability of metrics and takes action as warranted. 
Status: Open. VA's targeted completion date: October 2022.
  appendix n. va, dod, and fehrm responses to recommendations: joint 
       audit of the dod and the va efforts to achieve ehr system 
                     interoperability, may 5, 2022
    1. We recommend that the deputy secretary of defense and deputy 
secretary of veterans affairs review the actions of the Federal 
Electronic Health Record Modernization Program Office (FEHRM) and 
direct the FEHRM to develop processes and procedures in accordance with 
the FEHRM charter and the National Defense Authorization Acts. Status: 
Open.

    VA's targeted completion date: September 30, 2022. DoD's targeted 
completion date: None specified.

    2. We recommend that the director of the FEHRM, in coordination 
with the director of the Defense Health Agency; program executive 
director for EHRMI; and program manager for DoD Healthcare Management 
System Modernization:

      a. Determine the type of patient health care information that 
        constitutes a complete patient EHR. Status: Open. FEHRM's 
        targeted completion date: August 31, 2022.

      b. Develop and implement a plan for migrating legacy patient 
        health care information needed for a patient's complete EHR 
        once the FEHRM determines the health care data domains of 
        patient health care information that constitutes a complete 
        patient EHR. Status: Open. FEHRM's targeted completion date: 
        August 31, 2022.

      c. Develop and implement a plan for creating interfaces that 
        would allow medical devices to connect and transfer patient 
        health care information to Cerner Millennium. Status: Open. 
        FEHRM's targeted completion date: One year after resources have 
        been approved and allocated, the FEHRM will develop a plan to 
        create interfaces between medical devices and the Federal EHR.

    Senator Heinrich. Thank you. Senator Tester has another 
commitment, so I am going to let him go first.
    Senator Tester. Thank you very much, Mr. Chairman. I 
appreciate this.
    Mr. Sicilia, are you the point person for Oracle's efforts 
here with EHR?
    Mr. Sicilia. Yes, I am. I am ultimately responsible for the 
business.
    Senator Tester. Perfect. You are you the right person then 
to have in front of us. How is the communication, how would you 
rate the communication between yourself and VA leadership? 
Would you say it is good, poor, superior?
    Mr. Sicilia. I would say it is very good. I have regular 
meetings with Deputy Secretary Remy, with Dr. Adirim, and I 
certainly look forward to working with Dr. Elnahal.
    Senator Tester. And how often do you meet?
    Mr. Sicilia. We meet twice a month with Deputy Secretary 
Remy, and Dr. Adirim as part of our regularly scheduled 
leadership calls; and sometimes I have one-on-one calls with 
her as needed as well. I would say there are no barriers to 
communication between Oracle and the VA.
    Senator Tester. And as and it is made clear during these 
meetings, what you guys need and what their expectations are?
    Mr. Sicilia. Yes. I think we are in pretty good hands, and 
as I mentioned in my testimony, I think having a dashboard that 
all of us can see.
    Senator Tester. Yes.
    Mr. Sicilia. You can see where the ball and court is for 
every effort is very valuable.
    Senator Tester. So one of the advantages that happened when 
Oracle acquired this, as you are a much bigger company than 
Cerner was, you have much more capacity, you have much more 
ability to solve problems, big problems. But Oracle is a big 
company. Where does this EHR land on the list of what needs to 
be done in priorities for Oracle?
    Mr. Sicilia. This is the most important effort we have 
going on at the company. We have recast over 2,000 people, 
existing Oracle employees, to now work specifically on the VA-
EHRM program, in addition to the existing Cerner team.
    Senator Tester. So your opening statement was--I mean who 
could argue with what you said--I mean, it was pretty good 
stuff. The only thing you didn't talk about is timeline. What 
is Oracle's timeline for getting this thing going?
    Mr. Sicilia. We will be ready, from a technical 
infrastructure perspective, whenever the VA wishes to resume go 
lives, and if they wish to add more parallelism down the road 
to compress, what now looks like it could be 13, backed into 10 
years. We are ready----
    Senator Tester. Okay. So I am holding VA accountable, but 
what I am not holding VA accountable for is the computer 
program that works, that make sure we don't burn out employees, 
and make sure veterans get the health care they need. They 
can't go live until it meets those measures that we have talked 
about before, and we can talk about more now. What is the 
timeline to making sure that program is going to work so they 
feel comfortable?
    Mr. Sicilia. I think we have a pretty good visibility into 
changes that are needed. For example, the unknown queues, 
talked about many times.
    Senator Tester. I got you.
    Mr. Sicilia. We delivered that fix on August 1st, as 
promised, to the VA. As Dr. Elnahal reported, there are 
provider frustrations, caregiver frustrations with the system. 
What we need is the VHA Councils to provide the content for how 
they want the system reconfigured, if so, because that is not 
necessarily a programming change, we don't have to change the 
code of the system, these are switches that need to be flipped. 
And frankly that is collaboration between Oracle, and the VHA 
Councils, and the VA.
    Senator Tester. And so that takes me back to my first 
question. This thing doesn't get done without good 
communication.
    Mr. Sicilia. That is correct.
    Senator Tester. And so very good, but yet I heard you just 
say that you need more input from VHA.
    Mr. Sicilia. I think with Dr. Elnahal's leadership, I think 
that will increase rapidly. And again, I believe the best way 
to hold us all accountable is to put everything that we need on 
the dashboard so we can all see at the same time, and we don't 
have to wait for the next hearing to have an update. We should 
be able to have very good telemetry into everything that is 
going on with a public-facing dashboard, and as we intend to 
do.
    Senator Tester. Thank you. And thank you for your 
statement, and thank you for the commitment to deliver it with 
the agreed amount. I think that timeliness is something that is 
important here, as on one hand I say don't roll this thing out 
until you are ready for prime time, and on the other hand, we 
can't wait forever either. Okay?
    Mr. Sicilia. Agreed.
    Senator Tester. All right. Thank you.
    This question is for Mc. Case. And it is my last question. 
The work that you do I very much appreciate, as Inspector 
General. I sent a letter to the VA after our last hearing, 
asking them to speed up the closing on their EHR 
recommendations as Chairman of the Senate Veterans Affairs' 
Committee. Mr. Case, can you give us an update on whether you 
have seen a speeding up of the VA side, giving you the 
information that you need to be able to close out these 
recommendations?
    Mr. Case. Yes, we have. We have seen some reasonable 
progress over the last couple of weeks, especially, and we have 
been able to close out six recommendations over the last few 
days, which include some of the recommendations addressing 
training. There is another piece of information we got that may 
allow us to close out another recommendation.
    So, we have seen some good effort, but there is much more 
to do, particularly with recommendations dealing with the 
medication management, care coordination, and even some early 
recommendations, and particularly those pertinent to budget and 
our recommendations concerning having an integrated master 
schedule in place. We are told that will be coming this year. 
But, we will see.
    Senator Tester. Okay. Thank you. And I want to thank 
everybody who testified. I would get to you too, but we are 
having too much fun.
    Thank you, thank you to both of you for having this 
hearing.
    Senator Heinrich. No. Thanks for coming.
    Ranking Member Boozman.
    Senator Boozman. Thank you, Mr. Chairman.
    Dr. Rieksts, the IDA cost estimate looks out over 28 years. 
I would like to focus more on the immediate future; your 
estimate is the VA will take 13, not 10 years to complete. Can 
you explain the discrepancy, why the extra 3 years? And then 
also, you estimate the implementation phase will cost us 
roughly $33 billion, primarily from the acquisition bucket. 
Will you expand on that number and explain how your estimate 
differs from the VA's?
    Dr. Rieksts. Sure. Thank you for the question. First, on 
the 13-year implementation period, we based an extra 3 years in 
our estimate based on a risk analysis. So we estimated a range 
of one to five additional years over the 10-year period that 
will be required. And that is based on both looking at 
historical programs and the challenges that they have had, and 
then events that have happened with the current program that 
have led to the delays that don't indicate that this program 
would behave differently than historical programs.
    And the second point in your question is the implementation 
period and what that includes that is not in the VA cost 
estimate. So one of the big elements is the productivity loss 
that we estimate, that is not in scope and included in the VA 
estimate, and that is the cost of sending veterans out to the 
community, to private caregivers to get care while there is a 
disruption at the facility, and clinicians are busy in 
training, and learning where to click on the new system. And 
that also would include additional staffing that is needed at a 
facility at the go-lives to help those physicians and 
clinicians at a facility. That is one element.
    Another element is, there are some sustainment costs after 
you turn on the initial sites. You need to host and sustain the 
system, and that is also included in our cost estimate. And 
then there are some differences between the IDA estimate and 
the VA estimate of about $8 billion in comparable elements.
    And that is due to both learning from the actual costs that 
have been incurred today. Our estimate has come after their 
estimate, so we leverage at the costs that have been incurred 
to date, and also doing risk analysis of, for example, some of 
the development activities that are needed to make sure the 
system is suitable for veterans, and performs the way that it 
is intended to meet the functions that are required.
    Senator Boozman. Very good. Thank you. The VA OIG has 
issued numerous reports critical of EHRM programs execution and 
transparency. Among these criticisms VA has moved forward with 
deployments without the necessary IT infrastructure in place, 
VA's EHRM and infrastructure cost estimates were not 
comprehensive, well-documented, accurate, or credible. VA's 
reports to Congress did not include cost estimates for physical 
infrastructure, leaving out $2.7 billion in cost, and the 
program is marred by ineffective training, and no integrated 
master schedule.
    Mr. Case, given those criticisms and what you know of the 
IDA independent cost estimate, what should this subcommittee 
make of the testimony today? What will be the true cost of the 
program? And how long do you think it is going to take? Who is 
more correct, VA or IDA? Is the answer somewhere in the middle? 
I know that is difficult, but tell us what you are thinking.
    Mr. Case. Looking at the cost and what IDA has produced, I 
think it is a serious document, is how I would characterize it. 
And as noted, even from the $16 billion that was estimated by 
VA, there is an additional $8 billion that IDA sees there, 
comparing apples to apples.
    One critical aspect of IDA's approach is they include a 
risk analysis. It is good to be optimistic, but we have to 
understand that there is risk to all this, particularly in a 
complicated system like this.
    How long it will take? I think we can only reach an 
informed answer on that once we see an integrated master 
schedule. They have nothing at VA that tells us how they are 
going to get from start to finish, site by site, and what is 
included in that effort. They have promised one this year, and 
we hope to see it, and it is particularly important at this 
juncture, given that they are moving into complex facilities 
next year, and it is a very short time fuse between complex 
facilities.
    As I recall, the current schedule has Ann Arbor going live, 
which is a complex facility, on January 28th. And then the next 
complex facility, which is Seattle, just 1 month later on March 
5th, and the next set of complex facilities even shorter, at 
the end of March, which is both Cincinnati and Dayton.
    Can they get lessons learned implemented in time to go 
complex facility by facility in that period of time? And, will 
issues appear during that short window that they can correct 
between facilities? It took a long time for the unknown queue 
to appear, for remediations to be put in place, and to be 
addressed to where it is now.
    All that is so uncertain that we really can't say how long 
it is going to take. Or, at least, the IG has no confidence to 
be able to say that.
    Senator Boozman. Good. Thank you. Can I ask one more? Thank 
you, Mr. Chairman.
    Mr. Sicilia, we have heard reports of inadequate IT and 
physical infrastructure at VA facilities. What is your 
assessment of VA's IT infrastructure and some of the challenges 
you anticipate that will need to be resolved to successfully 
deploy the system across the Nation? What changes need to be 
made to the program to expedite employment, and ensure the VA 
clinicians accept, and embrace the new system? And what will 
require additional effort? And what will require additional 
investment?
    Mr. Sicilia. Well, hopefully over time, Senator, there will 
be less of that infrastructure needed. As we move to a modern, 
stateless web application you need less infrastructure and not 
more infrastructure. And that is, I think, one of the issues 
that is a challenge with stability and configurations today, is 
also with the cost estimates a long time, because there is an 
assumption that the technology stands still. And obviously as 
it changes you need less of it because the new computers are, 
frankly, far more powerful.
    As far as the view into internal IT needs at the VA, 
sometimes it is just things like printers, and routers, and 
things like that. I think that will continue as a matter of 
course, and it is not uncommon. In any technology, in any 
technology rollout at the local IT landscape sometimes needs to 
change, because it is quite old, and quite attuned to the new 
systems.
    But my hope is that as we get more systems live, that 
becomes a far more repeatable process, and a repeatable cost 
because we have plenty of lessons learned. And if I can share 
with you why I am optimistic of the changes, we are pretty well 
through here, the DOD implementations. In fact, we have two 
more go lives this week, the same system, same computers, same 
code, and we are on budget for the entire program at the DOD 
and the Coast Guard from the original contract scope.
    So that is what gives me hope that all the costs that are 
under our control are certainly achievable with the current 
scope.
    Senator Boozman. Good. Thank you, Mr. Chair.
    Senator Heinrich. Thank you. Mr. Sicilia, drill down a 
little bit more on that last point. What do you see as the 
nature of the differences in the rollout between, for example, 
the Coast Guard and the VA? Is it the complexity of the 
population within the VA? What are the other contributing 
factors to why those rollouts have gone differently?
    Mr. Sicilia. You know, certainly, I think there are several 
factors. The first is, as I mentioned in my testimony, I think, 
frankly, the training and that is on us, the training at the VA 
sites was not as strong as it was at some of the DOD sites. It 
is also not lost on me that the complexity of the population in 
the VA is different than the complexity of the population among 
active service personnel. And therefore, more intense 
workflows, more configurations to the system may be needed.
    I think one of the things that has worked well at DOD is, 
if you will, snapping a chalk line with some of the 
configurations, and it is, which I think is complicating 
situation at the VA, is that you have 171 facilities with 133 
unique versions of a system. And sometimes it is difficult, 
from a change management perspective, to snap that chalk line 
because everybody wants everything they have today to work 
exactly the way it does today in the brand new system. And that 
is difficult when you have so many different custom versions, 
and so many different custom systems.
    And I think where we have seen success, and where we have 
been able to increase our velocity at the DOD, is to snap that, 
you know, that line and have that configuration fixed. And I 
was very encouraged to hear Dr. Elnahal, who I think I agreed 
with everything that he said, about understanding these 
configurations of workflows and making them simpler to use for 
physicians.
    And that is something that we can do today. This doesn't 
require any big investment, or any big change out of 
infrastructure, technical infrastructure, all that will happen 
over time too. But we can certainly reconfigure these workflows 
today. There is no barrier to doing that.
    Senator Heinrich. Talk about that, you know, much of what 
has been done early on with the system and the fixes were 
specific to individual locations. And now we are we are talking 
about moving to the cloud, and having a more ubiquitous system. 
What are the challenges in just making sure that that is a 
seamless transition as well?
    Mr. Sicilia. I would say that some of the fixes were 
specific to individual locations. But things like the unknown 
queue were, you know, a design feature or flaw depending on how 
you want to look at it.
    Senator Heinrich. Right.
    Mr. Sicilia. Of the system from day one. And certainly, 
everybody benefits from that fix, everybody who is live today, 
and who will go live next year. So that is, I think that those, 
both of those things will happen.
    As far as moving to a modern, stateless web application, 
you know, eliminating some of the technology stack as a result, 
our plan is to do that incrementally. I think it would be a 
mistake to just replace everything all at once, and say here is 
the brand new system flip the switch and here you go.
    Now, the benefit that we have is that the Cerner Millennium 
system is written on top of the Oracle database, so there is no 
data migration that has to happen, and that is one of the 
things that is a huge problem that is what is happening today, 
it has got to migrate all the data from the old system to the 
new system. We don't want to repeat that process. That is not a 
good idea. So what we will do is deliver new modern modules on 
top of the existing system that will run in parallel to the 
Cerner Millennium system, and over time councils, and certainly 
practitioners, can decide when they wish to update those 
things. I think that is the most risk-free way to move forward.
    Senator Heinrich. Given your last 4 months of experience, 
as well as the experience that DOD and Coast Guard, knowing 
full well what you are generally in for at this point, what do 
you think is a realistic timeline for safe deployment at the 
entirety of the Department's facilities?
    Mr. Sicilia. I still think we can do it 10 years that was 
originally contemplated. I don't really--comparing this to 
other large commercial programs that we roll out, like 
electricity grid management and, you know, things like this 
which are at the national infrastructure level, it is not 
uncommon that in the beginning, is where you have the most 
bumps in the road, if you will, and you pick up, you pick up 
over time.
    Senator Heinrich. Sure.
    Mr. Sicilia. Yeah, I think agreement on those workflows, 
and making sure they are as simple as we possibly can, eases 
impediment. I am very confident in our new training program, 
and bringing Accenture in to take that. So I don't see a reason 
why I could tell you today that it is not possible to deliver 
on time.
    Senator Heinrich. And you have mentioned several times the 
necessity of getting a dashboard that everybody agrees has the 
metrics that are necessary to get this where it needs to go. Do 
you feel like that that is something everyone agrees on now, 
and is communicating adequately around?
    Mr. Sicilia. I do. And the first version of the dashboard 
is now live, and accessible for all the members of the 
committee. And we will certainly work with VA to add content as 
they say fit to that as well. But everything that we are 
responsible before is now listed on that dashboard.
    Senator Heinrich. I want to get to Senator Hagerty. I 
appreciate him being here for this hearing. But I have one more 
question before we transition to him.
    And that is simply the issue, Mr. Sicilia, around outages 
and stability at the system. Give us some transparency into 
that, and how we get to what is a very high standard, and not 
unlike the standard you would apply to a power utility, 99.9 
system uptime. Talk to us about that and what your plans are.
    Mr. Sicilia. Yeah. Our early view of this, 4 months into 
the system, was a bit under-resourced technically, so we have 
added computer capacity to the system at our expense, and we 
will continue to do that over time, to make sure that all 
future go-lives are adequately staffed.
    We also have over 40 different programs going on, seven of 
which we completed already for system, shall we say, internal 
audits, where we are auditing the system and looking at all of 
the technical functionality of the system, I am confident that 
we can address the stability, and we have made great progress 
towards getting to the 99.9 over the last few months.
    We are actually in the process, where we plan to do to put 
those numbers out publicly, as we do for our business in 
general at Oracle. We are just in the process of working with 
VA right now on the exact--to make sure that we agree on the 
exact calculation of uptime, because we don't want to put a 
number out there that they may disagree with.
    Senator Heinrich. Great.
    Mr. Sicilia. So that will also be a public metric as well.
    Senator Heinrich. And we look forward to that.
    Senator Hagerty.
    Senator Hagerty. Thank you, Chairman Heinrich, and Ranking 
Member Boozman. I appreciate your chairing this. And to our 
witnesses, thank you for being here.
    You know, one of the most important missions of the Federal 
government is fulfilling the promise that we have made to our 
bravest men and women and their families for the sacrifices 
that they have made, sacrifices that we will never be able to 
fully repay. And I know that this subcommittee is committed to 
ensuring that our 17 million veterans, 430,000 of them living 
in my Home State of Tennessee, get the health care that they 
deserve.
    The Electronic Health Record Modernization program is one 
of the key programs in the VA Department that has undertaken to 
provide the best possible care to our veterans. This 
subcommittee has provided nearly $8.5 billion to this program 
over the past 5 years. As my colleagues have covered, 
unfortunately this program has been plagued by delays, by 
problems, and by safety concerns.
    So I would like to turn to you, Mr. Sicilia. You mentioned 
in your opening remarks that Oracle has recently acquired 
Cerner, and that Oracle brings world-leading expertise in 
software, and cloud infrastructure, and health IT. I am certain 
that there is significant value added that was contemplated 
when the acquisition was undertaken.
    So Mr. Sicilia, can you go through the process of 
describing the resources, and the capabilities and the 
expertise that Oracle is going to bring to bear above and 
beyond what Cerner was able to provide on its own?
    Mr. Sicilia. Yeah. As I stated, we have already repurposed 
thousands of engineers, software engineers to move into a 
central organization, wherein their sole focus is the health 
mission and the mission of this program, and to provide 
additional engineering capacity to the program.
    In my view, and owning this now for 4 months, that is where 
I think the single biggest lack was on the Cerner side. I think 
engineering horsepower in terms of people who have built very 
large scale systems for a living, both at the infrastructure 
level, and the application level, for very large mission-
critical systems, like the world's clinical trials networks, 
the world's power grids and utilities, that same caliber of 
folks who are used to building very high, of highly available 
systems, are now working on the Cerner system. So I am 
confident.
    I do believe that the Cerner team brought us tremendous 
clinical expertise. I mean obviously there are over 1,500 
people who work at Cerner, have some medical degree, are either 
a doctor, or a nurse, or a radiologist, or something. So I 
think putting those two things together is a marriage well 
made. And I am confident that we at Oracle have the engineering 
capacity that is needed to add significance, and make 
significant velocity changes to the program.
    Senator Hagerty. Beyond the thousands of engineers that you 
have added, are there other areas of actions or specific 
improvements that you have undertaken since June to make this 
more effective?
    Mr. Sicilia. We have sponsored--well as you know, this 
implementation is the VA, plus the DOD, plus the Coast Guard, 
so it is all three, so the Federal enclave. You know, I think 
bringing together all of our Federal partners in this has been 
very valuable. So we have sponsored the meetings at the Cerner 
Headquarters in Kansas City.
    We continue to meet, so on a very regular basis, and now 
have a joint leadership meeting among the Oracle team, the 
Oracle Cerner team which is a holding a subsidiary at this 
point; the VA, the DOD, Leidos, et cetera, and everybody who is 
involved in this. And I think that that leadership meeting has 
led to, or the regular leadership meeting has led to great 
improvements, and actually the sharing of successes.
    Senator Hagerty. I am glad to hear that. I am sure you are 
looking at best observed practices across this and trying to 
make certain that would benefit from the Federal government's 
broader experience here, particularly as you try to bring the 
VA up to par.
    Would you mind to discuss, Mr. Sicilia, the impact, or what 
you anticipate the impact to be by having a centralized 
interoperable modern health record system, and how will that 
impact the quality of care for our veterans?
    Mr. Sicilia. Well, the premise is the longitudinal health 
record, which means that, you know, as either an active-duty 
service member or a veteran you don't have to go fish around to 
a bunch of different websites, or fax machines, or all the 
things that plague people from getting their health information 
today, into an aggregate stack, an electronic stack.
    And you can imagine that from a quality care perspective, 
every time you see a new provider not having to start over, and 
saying, you know, here is all the things that I have been 
treated for in the past, the medications, all of that is 
readily available, there is a tremendous time saving, and a 
tremendous, you know, lack of fatigue factor.
    I think sometimes people just get so fatigued of going 
through the process, of trying to get all their records 
together, to go to a new appointment that they give up.
    Senator Hagerty. Yeah. And the frustration too, I mean, and 
people can go to the VA--go to the VA in Memphis, and then go 
to the one in Nashville, and they can't get their records 
transferred.
    Mr. Sicilia. Right, and so if you are the VA, and you are 
traveling seeing family out of state, you ought to be able to 
have an application that tells you exactly where you can go and 
make an appointment right away, and have your records follow 
you, so you don't have to go, you don't have to go start over.
    And then, obviously, from the provider standpoint, 
providers suffer too from not having longitudinal records. I 
mean, they don't know enough about the patient that they are 
treating, and they would love to be able to know a lot of 
things ahead of time, before they walk into the room and have 
to spend the first 15 minutes of an appointment starting over. 
So I think that those benefits will be tremendous for both 
patients and providers, and will be.
    Senator Hagerty. Well, given the magnitude of investment 
that this committee has put in place, I am glad to hear you are 
kicking it into overdrive with the acquisition, putting new 
resources in place. We are looking forward to seeing this 
getting done.
    Mr. Sicilia. Thank you.
    Senator Hagerty. Thank you. Thank you, Mr. Chairman.
    Senator Heinrich. Ranking Member Boozman, do you have any 
follow-up questions?
    Senator Boozman. No. Thank you.
    Senator Heinrich. I want to thank all of our panelists and 
the senators who participated in today's hearing. This is an 
incredibly important endeavor. We certainly appreciate 
everyone's efforts to make sure it is done correctly for the 
benefit of all of our veterans.

                     ADDITIONAL COMMITTEE QUESTIONS

    And finally, I will keep the hearing record open for one 
week. Committee members who would like to submit written 
questions for the record should try to do so by 5:00. Should 
actually do so by 5:00 p.m. Wednesday, September the 28th.
                Questions Submitted to Mr. Mike Sicilia
                Questions Submitted by Senator Heinrich
    Question. Since the contract was signed, issues have arisen that 
require shifts in what was originally planned. Oracle's commitment to 
doing much of this new work within the scope of the initial contract is 
appreciated. Your testimony indicated that on top of the $10.0 billion 
contract there already has been an additional $700 million in 
additional requirements identified, for a total of $10.7 billion. The 
clarification provided subsequent to the hearing that the contract 
ceiling is not contemplated to be raised is appreciated. What specific 
changes, by issue and amount, comprise the additional $700 million 
Oracle referenced?
    Answer. With any contract of this size and duration, there will 
always be a need for prudent, new programmatic requirements to account 
for emerging realities to be able to adjust course and implement 
lessons learned. As such, additional requirements have been identified 
that were not previously contemplated in the original Indefinite 
Delivery Indefinite Quantity (IDIQ) ceiling price. These additional 
requirements include but are not limited to the work related the VA 
side of interfaces, new requirements including Pharmacy enhancements, 
the standalone scheduling deployment in Columbus and additional 
solutions such as ones for clinicians to chart vitals data directly 
into the EHR.
    Oracle Cerner expects to be able to identify and leverage cost 
efficiencies to achieve taxpayer's savings over the ten-year term. 
Accordingly, Oracle continues to be committed to continuously 
evaluating opportunities to bring efficiencies and will continue to 
work with VA now, and over the life of the contract to deliver on those 
efficiencies through modern, enterprise-based technologies. Our 
commitment to efficiencies gained through the evolution of technology 
over time is expected to balance and offset many of the new 
requirements.
    Question. How close are Oracle Cerner and VA from having a rollout 
that is ``replicable'' and a better sense of what it will cost, on 
average, to deploy at a facility?
    Answer. As the number of facilities with different types of care 
and clinical complexity go-live, we will be able to better standardize 
deployment processes and solutions. Specifically, facilities with 
varying types of care and complexity may require new workflows, order 
sets, and capabilities that may not already be live and thus would be 
standardized over time as they are introduced to sites and accessed by 
end users. The new deployment schedule should help accelerate that 
process by pulling in capabilities earlier, such as the pharmacy 
enhancements. While difficult to eliminate all the variance between 
sites, having a holistic, replicable process will be key to better 
predict costs and scope of the work.
    This is intrinsic to our deployment methodology; however, it is 
dependent on a framework of a refinement of the system with a steady 
and clear baseline of scope and requirements. For this reason, as 
deployments resume, it will be critical to have a clear, detailed 
checklist of both requirements to resume go-lives uninterrupted and an 
ongoing set of standardized readiness criteria for site deployments 
going forward.
    Question. According to Oracle Cerner and VA, one of the significant 
differences between the first deployment of the Cerner EHR at Mann-
Grandstaff and the subsequent deployments was the commitment to 
training. What are Oracle's plans to make the training more useful, 
effective and efficient?
    Answer. VA and Oracle Cerner remain committed to providing training 
that prepares end users to use their new EHR in the delivery of health 
care to Veterans through ongoing improvements reflective of experience 
gained and lessons learned.
    Actions taken following deployment at Mann-Grandstaff include 
4,000+ content improvements, enhancement of all courses for virtual 
delivery, conversion of some foundational content to self-paced 
computer-based courses to provide flexibility, creation of training to 
support hands-on practice, and development of more targeted training in 
specialty areas. The initial impact of these efforts is reflected in 
improved training satisfaction survey results, a return to productivity 
baseline in many areas, and productivity levels as illustrated by key 
performance indicators (e.g., decreased time in the EHR, specifically 
at urgent care centers and in labs).
    While progress has been made, there remains work to be done. The 
EHRM training program, one of significant size and complexity, is 
executed in accordance with government-defined requirements and 
government-determined priorities. While VA has contracted with Oracle 
Cerner to provide technical training on the new EHR, end user feedback 
reflects more expansive expectations. To better understand these needs 
and associated opportunities, Oracle Cerner engaged a third-party to 
conduct an independent assessment of the EHRM training program and 
recommendations offered to VA by Oracle Cerner thus far at our own 
expense. The result of this effort was the identification of 10 (10) 
root cause issues, six (6) high-level recommendations and more than 25 
initiatives.
    As I mentioned in my testimony, we will be engaging Accenture to 
assist in implementing the changes needed to make the training much 
more efficient, applicable, and useful. The scope of Accenture's 
support of this aspect of the EHRM program is currently under 
consideration and we will keep the committee apprised of progress as 
that process moves forward.
    Question. Oracle announced plans to implement a cloud-based 
solution to improve stability and enhance the EHR system. Given this 
shift, what adjustments to current site assessment findings will need 
to be factored in to adjust the estimates for this shift from data 
centers?
    Answer. The current state reviews (CSR) incorporate both technology 
and infrastructure site reviews and are capturing the site-specific 
infrastructure and workflows needed to understand the scope and 
requirements for deployments. I don't anticipate any changes needed to 
the CSR requirements to account for the future cloud-based solution.
    Question. In recent months there have been many instances of system 
instability--either degradations or outages. The contract includes a 
99.9 percent standard for system uptime, with a required ``credit'' to 
be provided by Oracle Cerner when the standard is not met. Is the 99.9 
percent standard an appropriate expectation to meet?
    Answer. Yes, the service-level commitment (SLC) is industry 
standard and an appropriate expectation. Delivering a single common 
health record between VA, DoD and the U.S. Coast Guard is the first of 
its kind and is technically challenging, but the benefits are enormous. 
Veterans and Service members will have greater flexibility in where and 
how they seek care by no longer having to carry their own paper records 
or being tethered to their data siloed in specific medical centers.
    However, the benefits of this system can only be realized if it is 
reliable and completely trusted by the people using it. We are working 
on more than 40 different technical operational improvement projects 
for the Federal Enclave that we continue to lead to improved 
performance and greater stability. We have made progress already, 
completing 12 projects and expecting 4 more to be completed by the end 
of the year.
    We are seeing benefits from our architecture and operation reviews, 
including July and September without a major incident and decreased 
frequency of incidents related to change. Specifically, we had a 60 
percent month over month reduction in incidents related to change after 
Oracle introduced more rigorous methodology and oversight.
    Question. At one point in March 2022, the system was down for over 
20 hours, which resulted in a credit of a little over $100,000. Given 
the scope and scale of the overall contract, do you see the credits 
required when Oracle does not meet this target as appropriately 
incentivizing?
    Answer. We will leave it to our client to determine the appropriate 
levels consistent with the contractual requirements. With that said, 
our incentive goes well beyond this single component. With our 
acquisition of Cerner, we see our work with the VA and DoD as the 
opportunity to be a show piece for what Oracle will bring broadly to 
the entire health IT industry. We are driven by delivering what has 
been promised and exceeding expectations with the VA EHR setting the 
gold standard with a new generation of modern, cloud-native, highly 
performant and secure EHR applications embracing mobility, self-
service, analytics and ease of use to solve some of the biggest 
challenges facing our veterans and VA clinicians.

                                 ______
                                 

               Questions Submitted by Senator John Hoeven
    Question. The safekeeping of our Nation's servicemembers and 
veterans' medical records is essential not only for an individual's 
privacy, but for our National security. What systems does Oracle Cerner 
have in place so that medical records are secure?
    Answer. Oracle is a leading enterprise software vendor with more 
than forty years of experience building and developing some of the most 
advanced, mission-critical, secure and performan technology around the 
world for governments, critical infrastructure, and commercial 
enterprises. As an example, Oracle operates fully certified government 
cloud regions under the Intelligence Community's Commercial Cloud 
Enterprise (``C2E'') program and is fully qualified under the DoD's 
upcoming Joint Warfighter Cloud Capability Program (``JWCC'').
    Oracle's cyber security expertise together with Cerner's expertise 
in securing sensitive patient data is a very powerful combination that 
we believe will enhance security and data protection for our Nation's 
servicemember and veterans.
    The on-going modernization of the Federal electronic health record 
system across VA and DoD enhances health record protections in three 
main areas. First, incorporating regulatory best practices in support 
of HIPAA and NIST requirements. Second, the transformation from 
disparate systems running across multiple data centers to an enterprise 
system decreases system dependencies and surface area susceptible to 
security issues while enhancing system agility to be able keep pace 
with the ever-evolving cyber security threats. Third, the additional 
protections in moving the Cerner application, with the approval of VA, 
DoD and Coast Guard, to a modern, hyperscale cloud data center. This is 
the same Generation 2 Cloud Infrastructure that underpins Oracle's 
customers' most critical workloads.

                                 ______
                                 

                Questions Submitted to Dr. Brian Rieksts
             Questions Submitted by Senator Martin Heinrich
    Question. VA has not provided a revised schedule for the remainder 
of deployments, but the IDA analysis suggests it could take 3 years 
longer than originally planned. Understanding there are a lot of 
unknowns with future deployments, what is the basis for that 
assessment?
    Answer. The current program has already experienced schedule slips. 
For example, Seattle Veterans Affairs Medical Center (VAMC) was 
originally projected to go live in March 2020. The current projection 
is that this site is not likely to deploy until at least June 2023.\1\
---------------------------------------------------------------------------
    \1\ Veterans Affairs Office of Public and Intergovernmental 
Affairs, ``VA extends delay of upcoming electronic health record 
deployments to June 2023 to address technical and other system 
performance issues,'' October 13, 2022. https://www.va.gov/opa/
pressrel/pressrelease.cfm?id= 5833, accessed October 18, 2022.
---------------------------------------------------------------------------
    The Institute for Defense Analyses (IDA) estimated the overall 
deployment schedule using historical data on enterprise resource 
planning (ERP) systems.\2\ Those data show that the average schedule 
slip was 31 percent, which translates into a mean expected schedule 
adjustment of 3 years when applied to the Electronic Health Record 
Modernization (EHRM) initial 10-year deployment schedule. The 
historical data also have a standard deviation of 32 percent. Using 
this historical schedule variation, IDA estimates a 20 percent chance 
the deployments will be completed within 11 years and an 80 percent 
chance the deployments will be completed within 15 years.
---------------------------------------------------------------------------
    \2\ H. Gebre-Mariam et al., ``Assessing Enterprise Resource 
Planning (ERP) Cost, Schedule and Size Growth,'' June 2017, 2017 ICEAA 
Professional Development & Training Workshop.
---------------------------------------------------------------------------
    Question. Oracle Cerner indicated plans to shift to a cloud model, 
rather than data center requirements at each facility. Would this 
result in savings compared to the IDA estimate? If so, what would the 
potential magnitude of the savings be and where would they be realized?
    Answer. The planned requirements to shift to the Oracle cloud were 
not available for our cost estimate, so we do not have an estimate of 
this change. Costs for hosting and infrastructure could decrease; 
however, additional cybersecurity costs may be required to shift from 
current operations to the Oracle cloud. The net effect of these changes 
depends, in part, on the specific changes in requirements.
    A shift to the Oracle cloud would have an effect on some of the 
EHRM cost elements. We identified hosting ($5B) as the largest cost 
element in the IDA estimate affected by the shift. Oracle Cerner cloud 
services would still require some hosting costs; however, savings may 
be achieved if hosting costs decrease.
    Question. The IDA estimate treats the costs of ``common 
infrastructure'' as being on top of the EHR estimate. What criteria did 
IDA use in identifying and estimating this category?
    Answer. IDA defines common infrastructure as costs that would still 
be incurred without the EHRM program. We engaged with VA staff and 
reviewed documentation to identify common physical or information 
technology (IT) infrastructure costs that would be incurred regardless 
of EHRM. For example, common infrastructure includes shared services 
that are required by other programs, such as the Financial Management 
Business Transformation (FMBT) or Veterans Affairs Logistics Redesign 
(VALOR). Another example of common infrastructure is maintenance (e.g., 
cabling, climate control) that is planned regardless of EHRM. These 
costs are reported separately from the life-cycle cost estimate (LCCE) 
because they are not required exclusively by EHRM.
    Some cost elements are not exclusively common or EHRM-specific 
infrastructure. For example, WiFi is common infrastructure, but the 
increase in WiFi density required by EHRM is EHRM-specific. In these 
cases, we estimated the share of the overall costs that are EHRM-
specific infrastructure. To address the uncertainty in the 
categorization of EHRM-specific and common infrastructure, we estimated 
high and low values for the percent of common infrastructure and 
included this estimate in our risk analysis.

                                 ______
                                 

                 Questions Submitted to Mr. David Case
             Questions Submitted by Senator Martin Heinrich
    Question. VA Office of Inspector General reports related to EHRM 
implementation often flag areas where VA may not have followed 
standard, acceptable procedures in moving forward, such as with the 
previous cost estimate report to Congress and the development of a 
master schedule. What does OIG perceive as the greatest risk to the 
initiative going forward?
    Answer. The OIG is concerned about any risk that could impact 
patient safety. In our three March 2022 reports about the 
implementation at Mann-Grandstaff VA Medical Center, Medication 
Management Deficiencies after the New Electronic Health Record Go-Live 
at the Mann-Grandstaff VA Medical Center in Spokane, Washington, Care 
Coordination Deficiencies after the New Electronic Health Record Go-
Live at the Mann-Grandstaff VA Medical Center in Spokane, Washington, 
and Ticket Process Concerns and Underlying Factors Contributing to 
Deficiencies after the New Electronic Health Record Go-Live at the 
Mann-Grandstaff VA Medical Center in Spokane, Washington, we discussed 
issues related to patient safety, medication management, and care 
coordination. In our July 2022 report on the unknown queue, The New 
Electronic Health Record's Unknown Queue Caused Multiple Events of 
Patient Harm, we focused on the importance of resolving the queue 
because it requires human activity to identify what is in the queue, 
remove the order, and reroute the order. Those mitigations can lead to 
delays in care, which could impact patient safety. We have also 
recommended that VA ensure that the employee training program is 
conducted in an efficient and effective manner and improvements to the 
new EHR system are implemented speedily so that VA providers can return 
to predeployment productivity levels as quickly and safely as possible.
    We identified issues with VA's costs estimates and master schedule. 
Our 2021 reports Unreliable Information Technology Infrastructure Cost 
Estimates for the Electronic Health Record Modernization Program and 
Deficiencies in Reporting Reliable Physical Infrastructure Cost 
Estimates for the EHRM Program indicated that existing physical and IT 
infrastructure at VA medical facilities was inadequate for the new 
system and pertinent life cycle cost estimates were unreliable and 
underestimated possibly by about $5 billion. We note that the 
Department is finalizing the life cycle cost estimate prepared by the 
Institute for Defense Analyses. Once that is done, we will review the 
report and supporting materials to determine which, if any, of our 
recommendations from our two reviews of VA's physical and information 
technology cost estimates could be closed.
    However, as we noted in our report, The Electronic Health Record 
Modernization Program Did Not Fully Meet the Standards for a High-
Quality, Reliable Schedule, VA must also develop a reliable, 
comprehensive schedule for full system implementation. Identified 
deficiencies could result in schedule delays and leave VA vulnerable to 
billions of dollars in cost overruns. Without that schedule, Congress 
cannot rely on VA's timelines for completing the work or be assured 
that the program will be completed within budget or in line with the 
independent cost estimate.
    Question. Given the number of recommendations that remain open 
after more than a year, has VA been collaborative in working on the 
issues that have been raised?
    Answer. We saw some reasonable progress recently in September 2022, 
and we were able to close out six recommendations shortly before the 
subcommittee's hearing. This included some of the recommendations 
addressing training deficiencies from our July 2021 report, Training 
Deficiencies with VA's New Electronic Health Record System at the Mann-
Grandstaff VA Medical Center in Spokane, Washington. While we have seen 
some improved efforts, there is much more to do with recommendations 
addressing medication management and care coordination, and 
recommendations pertinent to costs and having an integrated master 
schedule in place.

                                 ______
                                 

               Questions Submitted by Senator John Hoeven
    Question. The VA Office of Inspector General has a number of 
recommendations that remain open with regard to the VA's electronic 
health record (EHR) modernization.
    Of the open recommendations, which are of the greatest concern in 
terms of patient safety, privacy, and cost?
    Answer. The OIG is concerned about any risk that could impact 
patient safety. In our three March 2022 reports about the 
implementation at Mann-Grandstaff VA Medical Center, Medication 
Management Deficiencies after the New Electronic Health Record Go-Live 
at the Mann-Grandstaff VA Medical Center in Spokane, Washington, Care 
Coordination Deficiencies after the New Electronic Health Record Go-
Live at the Mann-Grandstaff VA Medical Center in Spokane, Washington, 
and Ticket Process Concerns and Underlying Factors Contributing to 
Deficiencies after the New Electronic Health Record Go-Live at the 
Mann-Grandstaff VA Medical Center in Spokane, Washington, we made 
recommendations related to patient safety, medication management, and 
care coordination. From our July 2022 report on the unknown queue, The 
New Electronic Health Record's Unknown Queue Caused Multiple Events of 
Patient Harm, there is an open recommendation related to evaluating the 
mitigation process because it requires human activity to identify what 
is in the queue, remove the order, and reroute the order. Those 
mitigations can lead to delays in care, which could impact patient 
safety. We have also recommended that VA ensure that the employee 
training program is conducted in an efficient and effective manner and 
improvements to the new EHR system are implemented speedily so that VA 
providers can return to predeployment productivity levels as quickly 
and safely as possible.
    Our 2021 reports Unreliable Information Technology Infrastructure 
Cost Estimates for the Electronic Health Record Modernization Program 
and Deficiencies in Reporting Reliable Physical Infrastructure Cost 
Estimates for the EHRM Program indicated that existing physical and IT 
infrastructure at VA medical facilities was inadequate for the new 
system and pertinent life cycle cost estimates were unreliable and 
underestimated possibly by about $5 billion. We note that the 
Department is finalizing the life cycle cost estimate prepared by the 
Institute for Defense Analyses. This estimate was done by VA in 
response to our recommendations. Once that is done, we will review the 
report and supporting materials to determine which, if any, of our 
recommendations from our two reviews of VA's physical and information 
technology cost estimates could be closed.
    However, as we noted in our report, The Electronic Health Record 
Modernization Program Did Not Fully Meet the Standards for a High-
Quality, Reliable Schedule, VA must also develop a reliable, 
comprehensive schedule for full system implementation. Identified 
deficiencies could result in schedule delays and leave VA vulnerable to 
billions of dollars in cost overruns. Without that schedule, Congress 
cannot rely on VA's timelines for completing the work or be assured 
that the program will be completed within budget or in line with the 
independent cost estimate.

                          SUBCOMMITTEE RECESS

    Senator Heinrich. And with that we stand adjourned. Thank 
you all.
    [Whereupon, at 11:22 a.m., Wednesday, Septemeber 21, the 
hearing was adjourned, and the subcommittee was recessed, to 
reconvene at a time subject to the call of the Chair.]