[House Hearing, 117 Congress]
[From the U.S. Government Publishing Office]


                   TRUSTWORTHY AI: MANAGING THE RISKS
                       OF ARTIFICIAL INTELLIGENCE

=======================================================================
 
                                HEARING

                               BEFORE THE

                SUBCOMMITTEE ON RESEARCH AND TECHNOLOGY

                                 OF THE

                      COMMITTEE ON SCIENCE, SPACE,
                             AND TECHNOLOGY

                                 OF THE

                        HOUSE OF REPRESENTATIVES

                    ONE HUNDRED SEVENTEENTH CONGRESS

                             SECOND SESSION

                               __________

                           SEPTEMBER 29, 2022

                               __________

                           Serial No. 117-70

                               __________

 Printed for the use of the Committee on Science, Space, and Technology

[GRAPHIC NOT AVAILABLE IN TIFF FORMAT]                                     

       Available via the World Wide Web: http://science.house.gov
       
                              __________

                                
                    U.S. GOVERNMENT PUBLISHING OFFICE                    
48-617PDF                  WASHINGTON : 2023                    
          
-----------------------------------------------------------------------------------       

              COMMITTEE ON SCIENCE, SPACE, AND TECHNOLOGY

             HON. EDDIE BERNICE JOHNSON, Texas, Chairwoman
ZOE LOFGREN, California              FRANK LUCAS, Oklahoma, 
SUZANNE BONAMICI, Oregon                 Ranking Member
AMI BERA, California                 MO BROOKS, Alabama
HALEY STEVENS, Michigan,             BILL POSEY, Florida
    Vice Chair                       RANDY WEBER, Texas
MIKIE SHERRILL, New Jersey           BRIAN BABIN, Texas
JAMAAL BOWMAN, New York              ANTHONY GONZALEZ, Ohio
MELANIE A. STANSBURY, New Mexico     MICHAEL WALTZ, Florida
BRAD SHERMAN, California             JAMES R. BAIRD, Indiana
ED PERLMUTTER, Colorado              DANIEL WEBSTER, Florida
JERRY McNERNEY, California           MIKE GARCIA, California
PAUL TONKO, New York                 STEPHANIE I. BICE, Oklahoma
BILL FOSTER, Illinois                YOUNG KIM, California
DONALD NORCROSS, New Jersey          RANDY FEENSTRA, Iowa
DON BEYER, Virginia                  JAKE LaTURNER, Kansas
SEAN CASTEN, Illinois                CARLOS A. GIMENEZ, Florida
CONOR LAMB, Pennsylvania             JAY OBERNOLTE, California
DEBORAH ROSS, North Carolina         PETER MEIJER, Michigan
GWEN MOORE, Wisconsin                JAKE ELLZEY, TEXAS
DAN KILDEE, Michigan                 MIKE CAREY, OHIO
SUSAN WILD, Pennsylvania
LIZZIE FLETCHER, Texas
VACANCY
                                 ------                                

                Subcommittee on Research and Technology

                HON. HALEY STEVENS, Michigan, Chairwoman
MELANIE A. STANSBURY, New Mexico     RANDY FEENSTRA, Iowa, 
PAUL TONKO, New York                     Ranking Member
GWEN MOORE, Wisconsin                ANTHONY GONZALEZ, Ohio
SUSAN WILD, Pennsylvania             JAMES R. BAIRD, Indiana
BILL FOSTER, Illinois                JAKE LaTURNER, Kansas
CONOR LAMB, Pennsylvania             PETER MEIJER, Michigan
DEBORAH ROSS, North Carolina         JAKE ELLZEY, TEXAS
                        
                        
                        C  O  N  T  E  N  T  S

                           September 29, 2022

                                                                   Page

Hearing Charter..................................................     2

                           Opening Statements

Statement by Representative Haley Stevens, Chairwoman, 
  Subcommittee on Research and Technology, Committee on Science, 
  Space, and Technology, U.S. House of Representatives...........     9
    Written Statement............................................    10

Statement by Representative Randy Feenstra, Ranking Member, 
  Subcommittee on Research and Technology, Committee on Science, 
  Space, and Technology, U.S. House of Representatives...........    10
    Written Statement............................................    12

Written statement by Representative Eddie Bernice Johnson, 
  Chairwoman, Committee on Science, Space, and Technology, U.S. 
  House of Representatives.......................................    13

                               Witnesses:

Ms. Elham Tabassi, Chief of Staff, Information Technology 
  Laboratory, National Institute of Standards and Technology
    Oral Statement...............................................    14
    Written Statement............................................    17

Dr. Charles Isbell, Dean and John P. Imlay, Jr. Chair of the 
  College of Computing, Georgia Institute of Technology
    Oral Statement...............................................    28
    Written Statement............................................    30

Mr. Jordan Crenshaw, Vice President of the Chamber Technology 
  Engagement Center, U.S. Chamber of Commerce
    Oral Statement...............................................    36
    Written Statement............................................    38

Ms. Navrina Singh, Founder and Chief Executive Officer, Credo AI
    Oral Statement...............................................    49
    Written Statement............................................    51

Discussion.......................................................    61

             Appendix I: Answers to Post-Hearing Questions

Ms. Elham Tabassi, Chief of Staff, Information Technology 
  Laboratory, National Institute of Standards and Technology.....    86

Mr. Jordan Crenshaw, Vice President of the Chamber Technology 
  Engagement Center, U.S. Chamber of Commerce....................    87

            Appendix II: Additional Material for the Record

Document submitted by Representative Brad Sherman, Committee on 
  Science, Space, and Technology, U.S. House of Representatives
    ``Engineered Intelligence: Creating a Successor Species,'' 
      Representative Brad Sherman................................    92

 
                   TRUSTWORTHY AI: MANAGING THE RISKS
                       OF ARTIFICIAL INTELLIGENCE

                              ----------                              


                      THURSDAY, SEPTEMBER 29, 2022

                  House of Representatives,
           Subcommittee on Research and Technology,
               Committee on Science, Space, and Technology,
                                                   Washington, D.C.

    The Subcommittee met, pursuant to notice, at 10:42 a.m., in 
room 2318, Rayburn House Office Building, Hon. Haley Stevens 
[Chairwoman of the Subcommittee] presiding.

[GRAPHICS NOT AVAILABLE IN TIFF FORMAT]

    Chairwoman Stevens. Welcome to the Research and Technology 
hearing to examine the harmful impacts associated with 
artificial intelligence (AI) systems, as well as the 
opportunities with our artificial intelligence systems, the 
activities that academia, government, and industry are 
conducting to prevent, mitigate, and manage AI risks as these 
new technologies proliferate.
    I'm thrilled to be joined by this distinguished panel of 
witnesses, all of whom are in the room with us today. It is 
great to see your faces and to be together the first time since 
a March 2020 hearing, I believe.
    It is also of deep importance to be discussing the benefits 
and the challenges of artificial intelligence, the potential to 
influence many aspects of our lives and support our economic 
and national security. The applications in our everyday lives 
span from merely convenient like recommending your next movie, 
to transformational, like aiding doctors in earlier detection 
of disease. In my home State of Michigan, advances in 
artificial intelligence by automakers are accelerating the 
development of autonomous vehicles that will lead to reduced 
traffic and increased road safety. Artificial intelligence 
systems are also increasingly used to analyze massive amounts 
of data to propel research in fields to enhance our 
understanding of the universe and cosmology, to synthetic 
biology, to weather prediction. Call our ancestors.
    But ill-conceived or untested applications of artificial 
intelligence have also on occasion caused damage. We have 
already seen ways AI systems can amplify, perpetuate, or 
exacerbate inequitable outcomes. Researchers have shown that AI 
systems making decisions in high-risk situations, such as 
credit or housing, can be biased against already disadvantaged 
communities, causing harm. This is why we need to encourage 
people developing or deploying AI systems to be thoughtful 
about what they're putting out into the world. We must develop 
the tools, methodologies, and standards to ensure that AI 
products and services are safe and secure, accurate, free of 
harmful bias, and otherwise trustworthy. We are in a moment of 
trust.
    Since taking over this gavel of the Research and Technology 
Subcommittee a few years ago, I have worked with my colleagues 
on both sides of the aisle to promote trustworthy AI. We're 
working together. I was proud to secure trustworthy AI 
provisions in the CHIPS and Science Act that was passed and 
signed into law just last month, which also promotes the--or 
includes the Promoting Digital Privacy Technologies Act, which 
passed the House and awaits a vote in the Senate, supports 
privacy-enhanced data sets and tools for training AI systems.
    Additionally, this Committee led the development of the 
2020 National AI Initiative Act to accelerate and coordinate 
Federal investments in research standards and education of 
trustworthy AI. In that act we also directed NIST (National 
Institute of Standards and Technology) to develop an AI Risk 
Management Framework (AI RMF) to help organizations understand 
and mitigate the risks associated with these technologies.
    We're all excited to be having today's hearing and to 
discuss the progress of this work and the many other things 
that NIST is doing to promote trustworthy AI. Academia and 
industry are supporting ethical approaches to artificial 
intelligence. Universities across the country are adopting 
principles for responsible use of AI and incorporating ethics 
into their computer science (CS) curricula. Industry is moving 
past theoretical principles into practical approaches to 
mitigating AI risks. There's more to do, there's jobs to be 
had, and people's lives are being impacted.
    With that, we're here in Congress to ensure that the United 
States continues to lead the world in artificial intelligence 
and trustworthy artificial intelligence. And we thank our 
witnesses for their time.
    [The prepared statement of Chairwoman Stevens follows:]

    Good morning and welcome to today's Research and Technology 
hearing to examine the harmful impacts associated with 
artificial intelligence systems, and the activities that 
academia, government, and industry are conducting to prevent, 
mitigate, and manage AI risks. I am thrilled to be joined by 
our distinguished panel of witnesses. It is great to be with 
you all in person today, and I look forward to hearing your 
testimony.
    Artificial intelligence has the potential to benefit many 
aspects of our lives and support our economic and national 
security. The applications in our everyday lives span from 
merely convenient, like recommending your next movie, to 
transformational, like aiding doctors in earlier detection of 
disease. In my home state of Michigan, advances in AI by 
automakers are accelerating the development of autonomous 
vehicles that will lead to reduced traffic and increased road 
safety. AI systems are also increasingly used to analyze 
massive amounts of data to propel research in fields to enhance 
our understanding of the universe in cosmology to synthetic 
biology to weather prediction.
    But ill-conceived or untested applications of AI have also 
caused great harm. We have already seen ways AI systems can 
amplify, perpetuate, or exacerbate inequitable outcomes. 
Researchers have shown that AI systems making decisions in 
high-risk situations, such as credit or housing, can be biased 
against already disadvantaged communities.
    This is why we need to encourage people developing or 
deploying AI systems to be thoughtful about what they are 
putting out into the world. We must develop the tools, 
methodologies, and standards to ensure that AI products and 
services are safe and secure, accurate, free of harmful bias, 
and otherwise trustworthy.
    Since taking over the gavel of the Research and Technology 
Subcommittee, I have worked with my colleagues on both sides of 
the aisle to promote trustworthy AI. I was proud to secure 
trustworthy AI provisions in the CHIPS and Science Act--which 
the President signed into law last month. My Promoting Digital 
Privacy Technologies Act, which passed the House and awaits a 
vote in the Senate, supports privacy-enhanced datasets and 
tools for training AI systems. Additionally, this Committee led 
the development of the 2020 National AI Initiative Act to 
accelerate and coordinate Federal investments in research, 
standards, and education of trustworthy AI. In that Act, we 
also directed NIST to develop an AI risk management framework 
to help organizations understand and mitigate the risks 
associated with these technologies. I look forward to hearing 
about the progress of this work and the many other things NIST 
is doing to promote trustworthy AI in today's discussion.
    Academia and industry are also supporting ethical 
approaches to AI. Universities across the country are adopting 
principles for responsible use of AI and incorporating ethics 
into their computer science curricula. Industry is moving past 
theoretical principles into practical approaches to mitigating 
AI risks. But there is still much more to do.
    I'm looking forward to hearing more about this work from 
our witnesses today and to discussing what we here in Congress 
can do to ensure the United States leads the world in 
trustworthy artificial intelligence. I'd like to again thank 
our witnesses for joining us today.

    Chairwoman Stevens. With that, the Chair is going to 
recognize Ranking Member Mr. Feenstra for an opening statement.
    Mr. Feenstra. Thank you, Chairwoman Stevens, for holding 
this important hearing today. I very much value of this 
hearing. And I also want to thank Ranking Member Lucas for 
attending today. I'm very grateful for that also. And also to 
the distinguished panel that we have before us, it's--I 
appreciate the time and effort that you have taken to come here 
and to give testimony on this important topic.
    Artificial intelligence is fundamentally changing the way 
we solve some of our society's biggest challenges. From 
healthcare to transportation, commerce to cybersecurity, AI 
technologies are revolutionizing almost every aspect of our 
daily life. But with every new and emerging technology comes 
new and evolving challenges and risks. Over the years, the 
Science Committee has held several hearings on AI, discussing 
challenges rang ranging from ethics to the work force needs. I 
hope we can use today's hearing as an opportunity to further 
these important discussions and shed light on the importance of 
enabling safe and trustworthy AI.
    To do that, we have to first define what makes AI safe and 
trustworthy, and I believe our witnesses can help us shed light 
on that today. But in general, I think we can agree that safe 
and trustworthy AI will meet certain criteria, like including 
accuracy, privacy, and reliability. Additionally, it is 
important that trustworthy AI systems utilize robust data, 
while also protecting the safety and security of the user data.
    Some other important factors of trustworthy AI includes 
transparency, fairness, accountability, and the mitigation of 
harmful biases. These factors are particularly important to 
keep in mind as these technologies are being deployed for the 
use in our daily lives. It is also critical that the data used 
in AI technologies is accurate because the input data is the 
foundation, the literal foundation of AI. So that must be our 
general goal, transparent and fair AI with accurate data and 
strong privacy protections. We can ensure that by having the 
standards and evaluation methods in place for these 
technologies.
    The integration of trustworthy AI in key industries has the 
most potential use and significant competition to advance U.S. 
industry. AI and other industries of the future like quantum 
science can revolutionize how business and economics operate, 
improving efficiency, expanding services, and integrating 
operations. The key to these benefits, of course, is the 
trustworthy of AI.
    Here in Congress, Members of the Science Committee 
introduced the bipartisan National Artificial Intelligence 
Initiative Act in 2020, which was made into law through the 
Fiscal Year 2021 NDAA. The legislation created a broad national 
security to accelerate investments of responsible AI research, 
development, and standards, as well as education for AI work 
force. It facilitated a new public-private partnership to 
ensure that the United States leads the world in the 
development and the use of AI systems.
    Related to today's hearing, the initiatives require the 
National Institute of Standards and Technology, NIST, to create 
the framework for managing risk associated with AI systems and 
best practices sharing to advance trustworthy AI systems.
    As a leader in AI research, measurement, evaluation and 
standards, NIST has been developing their voluntary AI Risk 
Management Framework since this last July. The framework has 
been developed through a consensus-driven, open, transparent, 
and collaborative process with multiple workshops for industry 
to provide input. I look forward to hearing more about the 
progress NIST is making in implementing this directive and 
finalizing this important guidance from Ms. Tabassi. I believe 
that AI risk management from this framework will be critical 
for our industry to better mitigate risk associated with AI 
technologies, as well as promote the incorporation of 
trustworthiness in every stage from design to evaluation of AI 
technologies.
    I'm also looking forward to hearing from the U.S. Chamber 
of Commerce to learn more about the work through the Commission 
on the Artificial Intelligence Competitiveness, Inclusion, and 
Innovation and how they are working to help build customer 
confidence in AI technologies.
    I want to thank our witnesses again for their 
participation. I thank Madam Chair for putting this hearing on. 
And with that, I yield back.
    [The prepared statement of Mr. Feenstra follows:]

    Thank you, Chairwoman Stevens, for holding today's hearing 
on this important issue.
    And thank you, to our distinguished panel of witnesses for 
joining us heretoday. Artificial intelligence is fundamentally 
changing the way we solve some of our society's biggest 
challenges.
    From healthcare to transportation; commerce to 
cybersecurity; A.I. technologies are revolutionizing almost 
every aspect of daily life. But with every new and emerging 
technology comes new and evolving challenges and risks. Over 
the years, the Science Committee has held several hearings on 
A.I., discussing challenges ranging from ethics to workforce 
needs.
    I hope we can use today's hearing as an opportunity to 
further these important discussions, and to shed light on the 
importance of enabling safe and trustworthy A.I. To do that, we 
have to first define what makes A.I. safe and trustworthy. I 
believe our witnesses can help shed light on this today.
    But in general, I think we can agree that safe and 
trustworthy A.I. will meet certain criteria like including 
accuracy, privacy, and reliability. Additionally, it is 
important that trustworthy A.I. systems utilize robust data 
while also protecting the safety and security of user data.
    Some other important factors of trustworthy A.I. include 
transparency, fairness, accountability, and mitigation of 
harmful biases. These factors are particularly important to 
keep in mind, as these technologies are being deployed for use 
in our daily lives.
    It is also critical that data used by A.I. technologies is 
accurate because the input data is the foundation of A.I. So 
that must be our general goal: transparent and fair A.I. with 
accurate data and strong privacy protections.
    We can ensure that by having standards and evaluation 
methods in place for these technologies. The integration of 
trustworthy A.I. in key industries has the potential to be a 
significant competitive advantage for U.S. industry. A.I. and 
other industries of the future like quantum sciences can 
revolutionize how businesses and economies operate, improving 
efficiency, expanding services, and integrating operations. The 
key to these benefits, of course, is the trustworthiness of 
A.I.
    Here in Congress, Members of the Science Committee 
introduced the bipartisan National Artificial Intelligence 
Initiative Act of 2020, which was made law through the FY21 
NDAA. This legislation created a broad national strategy to 
accelerate investments in responsible A.I. research, 
development, and standards, as well as education for the A.I. 
workforce. It facilitated new public-private partnerships to 
ensure the U.S. leads the world in the development and use of 
responsible A.I. systems.
    Related to today's hearing, this initiative required the 
National Institute of Standards and Technology (NIST) to create 
a framework for managing risks associated with A.I. systems and 
best practices for sharing data to advance trustworthy A.I. 
systems. As a leader in A.I. research, measurement, evaluation, 
and standards, NIST has been developing its voluntary A.I. Risk 
Management Framework since last July. The framework has been 
developed through a consensus-driven, open, transparent, and 
collaborative process with multiple workshops for industry to 
provide input.
    I look forward to hearing more about the progress NIST is 
making in implementing this directive and finalizing this 
important guidance from Ms. Tabassi. I believe the A.I. Risk 
Management Framework will be a critical tool for industry to 
better mitigate risks associated with A.I. technologies as well 
as promote the incorporation of trustworthiness into every 
stage from design to evaluation of A.I. technologies.
    I am also looking forward to hearing from the U.S. Chamber 
of Commerce to learn more about their work through the 
Commission on Artificial Intelligence Competitiveness, 
Inclusion, and Innovation, and how they are working to help 
build consumer confidence in A.I. technologies.
    I want to thank our witnesses again for their 
participation. Madam Chair, I yield back.

    Chairwoman Stevens. At some point in time, they will recall 
and remember that we had today's hearing that is now actually 
both meeting in person and virtually, so a couple of reminders 
to Members. First, Members and staff who are attending in 
person may choose to be masked. It's not a requirement. Any 
individuals with symptoms, a positive test, or exposure to 
someone with COVID-19 should wear a mask while present.
    Members who are attending virtually should keep their video 
feed on as long as they're present in the hearing. Members are 
responsible for their own microphones. Please keep your 
microphones muted or off unless you are speaking.
    Additionally, if Members have documents they wish to submit 
for the record, please keep them--or please email them to the 
Committee Clerk, whose email address was circulated prior to 
the hearing.
    If there are Members who wish to submit additional opening 
statements, your statements will be added to the record at this 
point.
    [The prepared statement of Chairwoman Johnson follows:]

    Thank you, Chairwoman Stevens and Ranking Member Feenstra, 
for holding today's hearing. And welcome to our esteemed panel 
of witnesses.
    We are here today to learn more about the development of 
trustworthy artificial intelligence and the work being done to 
reduce the risks posed by AI systems.
    Recent advances in computing and software engineering, 
combined with an increase in the availability of data, have 
enabled rapid developments in the capabilities of AI systems. 
These systems are now deployed across every sector of our 
society and economy, including education, law enforcement, 
medicine, and transportation. These are sectors for which AI 
carries the potential for both great benefit, and great harm.
    One significant risk across sectors is harmful bias, which 
can occur when an AI system produces results that are 
systemically prejudiced. Bias in AI can amplify, perpetuate, 
and exacerbate existing structural inequalities in our society, 
or create new ones. The bias may arise from non-representative 
training data, implicit biases in the humans who design the 
system, and many other factors. It is often the result of the 
complex interactions among the human, organizational, and 
technical factors involved in the development of AI systems. 
Consequently, the solution to these problems is not a purely 
technical one. We must ensure that the writing, testing, and 
deployment of AI systems is an inclusive, thoughtful and 
accountable process that results in AI that is safe, 
trustworthy, and free of harmful bias.
    That goal remained central in our development of the 
National Artificial Intelligence Initiative Act, which I led 
alongside Ranking Member Lucas and which we enacted last 
Congress. In the National AI Initiative Act, we directed the 
National Science Foundation (NSF) to support research and 
education in trustworthy AI. As we train the next generation of 
AI researchers, we must not treat ethics as something separate 
from technology development. The law specifically directs NSF 
to integrate ethics research and technology education from the 
earliest stages and establishes faculty fellowships in 
technology ethics. The recently enacted CHIPS and Science Act 
further directs NSF to require ethics statements in its award 
proposals to ensure researchers consider the potential societal 
implications of their work.
    As we will learn more about today, the National AI 
Initiative Act also directed the National Institute of 
Standards and Technology to develop a framework for trustworthy 
AI, in addition to carrying out measurement research and 
standards development to enable the implementation of such a 
framework.
    While AI systems continue to make rapid progress, the 
activities carried out under the National AI Initiative Act 
will be key to grappling with the sociotechnical questions 
posed by rapidly advancing AI systems.
    I look forward to hearing more from our witnesses today and 
to discussing what more the United States can do to ensure we 
are the world leader in the development of trustworthy AI. 
Thank you, and I yield back my time.

    Chairwoman Stevens. And at this time, I'd like to introduce 
our witnesses. Our first witness is Elham Tabassi. Ms. Tabassi 
is the Chief of Staff for the Information Technology Laboratory 
at the National Institute of Standards and Technology. She 
leads NIST's trustworthy and responsible AI program that aims 
to cultivate trust in the design, development, and use of AI 
technologies by improving measurement science, standards, and 
related tools. Ms. Tabassi is a member of the National AI 
Research Task Force and has been at NIST since 1999.
    Our next witness is Dr. Charles Isbell. Dr. Isbell is the 
Dean and John P. Imlay, Jr. Chair of the College of Computing 
at Georgia Tech. His recent work focuses on building autonomous 
systems that can interact with large numbers of other 
intelligence agents, including humans and AI systems. Dr. 
Isbell also studies the effects of AI bias and pursues reform 
in computing education, focusing on broadening participation 
and access. He is an elected fellow of AAAI (Association for 
the Advancement of Artificial Intelligence), ACM (Association 
for Computing Machinery), and the American Academy of Arts and 
Sciences.
    Our third witness is Mr. Jordan Crenshaw. Mr. Crenshaw 
serves as the Vice President of the U.S. Chamber of Commerce's 
Technology Engagement Center. He also manages the Chamber's 
Privacy Working Group and which is comprised of nearly 300 
companies and trade associations in which developed model 
privacy legislation and principles. Prior to his current 
position, Mr. Crenshaw led the Chamber's Telecommunication and 
E-Commerce Policy Committee, which analyzes Federal privacy, 
cloud computing, broadband internet, e-commerce and broadcast 
policies.
    Our final witness is Ms. Navrina Singh. Ms. Singh is the 
Founder and Chief Executive Officer (CEO) of Credo. Credo AI 
helps organizations to monitor, measure, and manage AI 
introduce risk. Prior to co-founding Credo AI, Ms. Singh was 
the Director and Principal of Product in Microsoft Cloud and 
AI, where she built natural language-based conversational AI 
products. Currently, Ms. Singh serves as a member of the 
National AI Advisory Committee, which is tasked with advising 
the President and the National AI Initiative Office on topics 
related to the National AI Initiative.
    As our witnesses know--should know, you will each have 5 
minutes for your spoken testimony. Your written testimony will 
be included in the record for the hearing. They're great 
testimonies. When you have completed your spoken testimony, 
we'll begin with questions. Each Member will have 5 minutes to 
question the panel.
    We will start with Ms. Tabassi.

        TESTIMONY OF MS. ELHAM TABASSI, CHIEF OF STAFF,

               INFORMATION TECHNOLOGY LABORATORY,

         NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY

    Ms. Tabassi. Good morning, Chairwoman Stevens, Ranking 
Member Feenstra, and distinguished Members of the Subcommittee. 
I am Elham Tabassi, and I serve as the lead for the Trustworthy 
and Responsible AI program at the Department of Commerce's 
National Institute of Standards and Technology known as NIST. 
Thank you for the opportunity to testify today on NIST's effort 
to advance the trustworthy and responsible development and use 
of artificial intelligence. This Committee is well aware of the 
importance of advancing research and standards to cultivate 
trust in AI. Thank you for your dedication to this important 
issue and for your support of NIST's role.
    Artificial Intelligence holds the promise to revolutionize 
and enhance our society and economy, but the development and 
use of these systems are not without challenges or risks. 
Through robust collaboration with stakeholders across 
government, industry, civil groups, and academia, NIST works to 
advance research, standards, measurements, and tools to manage 
these risks and realize the full promise of this technology for 
all Americans.
    Among its work, NIST is developing the AI Risk Management 
Framework, or AI RMF, to provide guidance on mapping, 
measuring, and managing risks associated with AI. Like the 
well-known cybersecurity and privacy frameworks, the AI RMF 
will provide a set of outcomes that enable dialog, 
understanding, and actions to manage AI risks. Critically, the 
framework will focus on managing risks not just to 
organizations, but also to individuals and society. This 
approach is reflective of the sociotechnical nature of AI 
systems as a product of the complex human, organizational, and 
technical factors involved in their design and development.
    As is the case with all our publications, NIST is taking a 
stakeholder-driven and open process to coordinate the 
development of the framework. From the start of this initiative 
last year, NIST has engaged a broad range of stakeholders, 
including through several workshops and public comment 
opportunities. Based on stakeholder feedback, and consistent 
with congressional direction, NIST is on track to publish the 
final AI RMF 1.0 in January 2023. The technology and standards 
landscape for AI will continue to evolve. Therefore, NIST 
intends for the framework and related guidance to be updated 
over time to reflect new knowledge, awareness, and practices.
    Building off the RMF there is much more work to do to 
develop additional guidance, standards, measures, and tools to 
evaluate and measure AI trustworthiness, especially for 
specific characteristics and use cases. For example, NIST has 
significantly expanded its research efforts to mitigate harmful 
bias with a focus on sociotechnical approach.
    To support the advancement of AI standards, NIST seeks to 
bolster knowledge, leadership, and coordination on AI, 
including by engaging with other government agencies within 
United States and internationally. NIST engages with partners 
around the world, including through the Organization for 
Economic Cooperation and Development, OECD, and the U.S.-EU 
Trade and Technology Council (TTC) to advance shared goals in 
trustworthy and responsible AI.
    NIST also coordinates with other Federal agencies and leads 
several policymaking and interagency efforts. This includes 
administering the National Artificial Intelligence Advisory 
Committee or NAIAC, which advises the President and the 
National AI Initiative Office.
    Advancing research and standards that contribute to more 
secure, private, fair, rights-affirming, and world-leading 
digital economy is a top priority for NIST. Thank you for the 
opportunity to present on NIST's activities to improve 
trustworthy and responsible AI. I look forward to your 
questions.
    [The prepared statement of Ms. Tabassi follows:]
    
[GRAPHICS NOT AVAILABLE IN TIFF FORMAT]
    
    Chairwoman Stevens. Dr. Isbell.

                TESTIMONY OF DR. CHARLES ISBELL,

               DEAN AND JOHN P. IMLAY, JR. CHAIR

                  OF THE COLLEGE OF COMPUTING,

                GEORGIA INSTITUTE OF TECHNOLOGY

    Dr. Isbell. Thank you, Subcommittee Chair Stevens, Ranking 
Members Feenstra and Lucas, and distinguished Members of the 
Subcommittee. I'm Charles Isbell. I'm a Professor in and Dean 
for the College of Computing at Georgia Tech. Thank you for the 
opportunity to be here today.
    So by way of explaining my background, let me note that 
while I tend to focus on statistical machine learning, my 
research passion is actually interactive artificial 
intelligence. As noted at the top of the hearing, there, the 
fundamental research goal is to understand how to build 
autonomous agents who must live and interact with large numbers 
of other intelligent agents, some of whom may be human. But I'm 
also an educator. As such, I spend much of my energy focusing 
on providing access to all those who wish to be a part of this 
ongoing conversation around the role of AI and computing in our 
lives. My discussion today and answers to your questions you 
ask will be informed by both my research and educator selves.
    So let us begin this discussion by defining our terms. 
There are many potential definitions of AI. My favorite one is 
that it is the art and science of making computers act the way 
they do in the movies. In the movies, computers are often semi-
magical and anthropomorphic. They do things that if humans did 
them, we would say they required intelligence.
    This definition is borne out in our use of AI in the 
everyday world. We use the infrastructure of AI to search 
billions upon billions of documents to find the answers to a 
staggering variety of questions, often expressed literally as 
questions. We use automatically tagged images to organize our 
photos. And we use that same infrastructure to plan optimal 
routes for trips, even altering our routes on the fly in the 
face of changes in traffic. In fact, we let our cars mostly 
drive themselves in that very same traffic playing the role of 
a tireless chauffeur.
    As noted by the Chair, we're able to automatically detect 
tumors from X-rays, even those that are trained--that trained 
doctors find difficult to see. We let computers finish our 
sentences as we type text and use search engines, sometimes 
facilitating a subtle shift from prediction of our behavior to 
influence over our behavior. Often, we take advantage of these 
services by using our phones to interpret a wide variety of 
spoken commands.
    So in some very important sense, AI already exists. It is 
not the AI of fanciful science fiction, neither benevolent 
intelligence working with humans as we traverse the galaxy, nor 
malevolent AI that seeks humanity's destruction. Nonetheless, 
we are living every day with machines who make decisions that 
if humans made them, we would attribute to intelligence. And 
the machines often make those decisions faster, and some might 
argue better, than humans would.
    Yet like all computing systems, at bottom, AI simply makes 
us more efficient. It amplifies our ability to make decisions, 
including bad ones, all too often automating the biases baked 
into our data and that of its developers. By way of example, 
according to the Marshall Project, most States use some form of 
automated risk assessment at some stage in the criminal justice 
system. We set out to predict recidivism as if that means the 
chance of committing a crime again, when in fact, what we're 
actually predicting is the chance of being arrested and 
convicted again. As with the shift from predicting behavior to 
influencing it, this distinction is subtle, but important. 
Without recognition of the difference, one can create a 
feedback loop and make things worse, without even noticing it.
    Although we sometimes act as if the machine is doing the 
work, it is worth noting that these machines are making 
decisions with us, with humans. They are partners, and as with 
any partner, it is important that we understand what our 
partner is doing and why. To make AI trustworthy, we need a 
more informed citizenry, something we can accomplish by 
requiring that our AI partners are more transparent on the one 
hand, but that we are more savvy on the other.
    So speaking of definitions, by transparency, I mean that an 
AI algorithm should be inspectable, that the kind of data the 
algorithm uses to build its model should be available, and the 
decisions that such algorithms make should be understandable. 
In other words, as we deploy these algorithms, each algorithm 
should be able to explain its output. ``This applicant was 
assigned this score because'' is more useful and less prone to 
misuse than just ``This applicant was assigned this score.''
    But to really understand such machines, much less to create 
them, we should strive for all of our citizens to not only be 
literate, but to be competent. That is, they must understand 
computing and computational thinking and how it fits into 
problem solving in their everyday lives. In the long term, one 
of the key solutions to AI bias will be bringing a wider group 
of people into computing education and into machine learning 
more specifically. We have to improve the number and the 
diversity of those entering the field and participating in and 
influencing the conversation because it is the right thing to 
do, but also because it is the only way for us to compete.
    It should not be lost that putting these two thoughts 
together suggests that the process by which we build AI 
algorithms is a shared effort that requires a wide swath of 
citizens to be informed and engaged and for developers to 
accept the responsibility for including the users of and 
sometimes targets of those systems in the development process 
itself. As a field, we have not caught up to the reality of the 
responsibility that we hold, and it is something that we simply 
must do. We must move from tool sets and skill sets to 
mindsets, incorporating responsibility in all that we do from 
the ground up.
    I'm very excited for this hearing. I think advances in AI 
are essential to our economic and social future. These are all 
areas in which funding--the funding power of the National 
Science Foundation and NIST as well can make a huge difference. 
So thank you very much, and I look forward to your questions.
    [The prepared statement of Dr. Isbell follows:]
    
[GRAPHICS NOT AVAILABLE IN TIFF FORMAT]
    
    Chairwoman Stevens. OK, Georgia Tech, you convinced me. I'm 
signing up for his class.
    Dr. Isbell. Done.
    Chairwoman Stevens. All right. With that, we're going to 
hear from Mr. Crenshaw for 5 minutes. Thanks.

        TESTIMONY OF MR. JORDAN CRENSHAW, VICE PRESIDENT

          OF THE CHAMBER TECHNOLOGY ENGAGEMENT CENTER,

                    U.S. CHAMBER OF COMMERCE

    Mr. Crenshaw. Thank you, Chair Stevens, Ranking Members 
Feenstra and Lucas, and Members of the Research and Technology 
Subcommittee. Good morning, and thank you. My name is Jordan 
Crenshaw, and I'm the vice president of the U.S. Chamber of 
Commerce's Technology Engagement Center. It's my pleasure to 
talk to you today about how we--business, government, and 
citizens--can work together to build trustworthy artificial 
intelligence.
    AI is changing the world as we know it. By 2030, AI will 
have a $16 trillion impact on the global economy. But from a 
practical level, what does that mean? AI is helping forecasters 
and emergency management better track the intensification of 
hurricanes and chart out evacuation and emergency preparedness. 
It's allowing researchers to more easily pinpoint virus 
mutations and tailor vaccines for new variants. It's also 
bolstering our cyber defenses against an evolving digital 
threat landscape. And finally, AI has the potential to fill the 
gaps where we have worker shortages, like patient monitoring 
where we have nursing shortages, and help tackle supply chain 
issues where we have a lack of available truckers.
    The United States is not operating in a vacuum. Its 
strategic competitors also realize the benefits of this crucial 
technology. For example, prior to the invasion of Ukraine, 
China and Russia agreed to cooperate on developing emerging 
technologies, specifically noting artificial intelligence. When 
it comes to AI, we are in a race we must win. AI is here now, 
and it's not going away. We cannot ignore it, and we cannot 
afford to sit on the sidelines and allow those who do not share 
our democratic values to set the standard for the world.
    For the research and deployment of AI to be successful, 
Americans must have trust in the technology. And while AI has 
many benefits, as I previously mentioned, in the wrong hands 
like those of our adversaries, there could be harms. Americans 
are united in the belief that we must beat our competitors as 
well. In fact, according to polling by the U.S. Chamber of 
Commerce, 85 percent of Americans believe the United States 
should lead in AI, and nearly that same number believes that we 
are best positioned as a nation to develop those ethical 
standards for its use.
    We agree. It's why the Chamber earlier this year 
established its Commission on AI Competitiveness, Inclusion, 
and Innovation, led by your former congressional colleagues, 
Representatives John Delaney and Mike Ferguson, and it's 
comprised of experts in business, academia, and civil society. 
The Commission has been tasked with developing policy 
recommendations in three core areas: trustworthiness, work 
force preparation, and international competitiveness. Our 
Commission held field hearings in Austin, Silicon Valley, 
Cleveland, London, and here in D.C.. And we've heard from a 
variety of stakeholders and look forward to presenting you with 
our recommendations early next year.
    In the meantime, while we wait for the Commission to 
finalize its report, we offer the following observations about 
what it will take to maintain trustworthy AI leadership. The 
Federal Government has a significant role to play in conducting 
fundamental research in trustworthy AI. The Chamber was pleased 
to see passage of the CHIPS and Science Act and hopes to see 
the necessary appropriations to carry out the science 
provisions. We encourage continued investment in STEM (science, 
technology, engineering, and mathematics) education. We need a 
trained, skilled, and diverse work force that can bring 
together multiple voices for coding and developing systems.
    AI is only as good, though, as the data it uses. That is 
why it is key that both government and the private sector team 
up to ensure there is quality data for more accurate and 
trustworthy AI. Governments should prioritize improving access 
to its own data and models and ways that respect individual 
privacy. At the same time, while we talk about privacy, as 
Congress looks to address these types of issues, it's important 
that we look at issues to determine whether or not we inhibit 
the collection of sensitive data and other types of data that 
could inhibit deploying trustworthy AI systems.
    Fourth, we need to increase widespread access to shared 
computing resources. However, many small startups and academic 
institutions lack sufficient computing resources to help 
develop solutions to artificial intelligence. That's why 
Congress took the critical step of establishing the Research--
passing the Resource Task Force Act of 2020. Now the National 
Science Foundation and the White House's Office of Science and 
Technology Policy should fully implement the law and 
expeditiously develop a roadmap to unlock AI innovation across 
multiple stakeholders.
    Finally, we also are encouraged and are thankful for the 
work by NIST in its development of the AI Risk Management 
Framework, which is a consensus-driven, cross-sector, and 
voluntary framework to leverage best practices.
    These recommendations are only the beginning. And I thank 
you for your time to address how the business community can 
partner with you to maintain trustworthy AI leadership. We 
thank you for your leadership, and I look forward to your 
questions.
    [The prepared statement of Mr. Crenshaw follows:]
    
[GRAPHICS NOT AVAILABLE IN TIFF FORMAT]
    
    Chairwoman Stevens. Thank you.
    With that, Ms. Singh, yes.

                TESTIMONY OF MS. NAVRINA SINGH,

         FOUNDER AND CHIEF EXECUTIVE OFFICER, CREDO AI

    Ms. Singh. Madam Chair, Ranking Member Feenstra and Lucas, 
and Members of the Subcommittee, thank you for the opportunity 
to testify today and to be part of this distinguished panel of 
witnesses. My name is Navrina Singh. I'm the Founder and CEO of 
Credo AI, a venture-backed startup. In addition, I'm a member 
of the National AI Advisory Committee that is advising 
President Biden as part of the National AI Initiative.
    Trustworthy artificial intelligence is a topic that is 
deeply personal to me. Growing up in India as a girl who 
aspired to be an engineer, I learned early on that I faced an 
uphill battle for no reason other than my gender. Part of my 
passion for the subject and the main reason I founded Credo AI 
in March 2020 is because I experienced firsthand what is at 
stake. While AI is an exciting and ultimately very useful 
technology, unless we create a culture of accountability, 
transparency, and governance around it, we risk unchecked 
growth and algorithms that may unintentionally encode the same 
types of societal ceilings and perceptions that I experienced 
as a girl in India and that many others still experience today.
    Members of the Subcommittee know very well the power and 
potential of AI when used responsibly. While it is a 
transformational technology that is evolving rapidly, I realize 
that there are different points of view on its perceived 
advantages. But one thing we can all agree on is AI is not 
going away, which is why we owe it to ourselves and to the 
world that our children will inherit to ensure robust 
compliance and governance structures to keep pace with the AI 
development.
    As the Subcommittee studies the question of how to manage 
AI risk and build trustworthy AI, we think three key 
considerations merit special attention. First, I want to focus 
on full AI lifecycle, from design to development, to testing 
and validation, to production and use. That means building AI 
systems responsibly continuously. It is fit for purpose, fair, 
transparent, safe and secure, privacy-preserving, and 
auditable.
    Second, context is paramount. We believe that achieving 
trustworthy AI depends on shared understanding, that governance 
and oversight of AI is industry-specific, application-specific, 
model-specific, and data-specific to ensure that it is fit for 
purpose. This necessitates a collaborative approach to metric 
alignment, and associated assessments.
    Third, transparency reporting and system assessments are 
critical for responsible AI governance. Reporting requirements 
that promote and incentivize public disclosure of AI system 
behaviors act as a key driver for establishment of standards 
and benchmark. And fundamental to this is access to compliant 
and comprehensive data for assessments. For these reasons, we 
at Credo AI advocate for context base, full AI lifecycle 
governance of AI systems with reporting requirements that are 
specific, regular, and transparent.
    If you truly want to be a global leader in AI, then our 
focus should be on building responsible technology aligned with 
our societal values. Responsible AI is also a competitive 
advantage. It allows companies to deploy AI at scale with 
confidence, and this transparency promotes trust with consumers 
in this technology. Government has a critical role to play 
here, working together through public-private partnerships to 
ensure the right set of standards exist to further innovation 
in the space. And we urge the policymakers and standard-setting 
bodies to prioritize establishing context-focused standards and 
benchmarks that are globally interoperable and can help 
eliminate some of the guesswork.
    My 8-year-old daughter told me recently that she wants to 
be an inventor and a social media influencer when she grows up. 
While I'm grateful that in this country my daughter will have 
the opportunity to follow her dreams, we owe it to her and the 
generations that will follow to ensure that we build AI which 
is developed responsibly and ethically.
    Thank you for the opportunity to appear before you, and I 
look forward to your questions.
    [The prepared statement of Ms. Singh follows:]
    
[GRAPHICS NOT AVAILABLE IN TIFF FORMAT]
    
    Chairwoman Stevens. Well, thank you.
    And at this point, we're going to turn to our first round 
of questions, and the Chair is going to recognize herself for 5 
minutes.
    In hearing your testimony, as I reflect on my time pursuing 
a master's in philosophy of which my parents never understood 
why I got, but we were asking the ethical question about 
artificial intelligence that some ask in the theoretical space 
that can a AI replace human behavior? Can--does AI threaten 
what we do as people seeking to overtake, you know, the 
decisions that we make as people?
    Today's hearing is a little bit more instructive to the 
theoretical question. Today's hearing is saying, hey, we have 
artificial intelligence, and it is being utilized, but how is 
it being utilized? How is it being implemented? And is it 
implementing fairly and accurately for the best outcomes for 
society and for humanity?
    So in 2019, NIST developed the strategy for Federal 
engagement in developing technical standards and tools for 
artificial intelligence. And, Ms. Tabassi, I'm just wondering 
if you could touch briefly because your testimony got me 
thinking on this, what was included in this strategy and why it 
is important that we have strategies for engaging in the 
development of technical standards for artificial intelligence. 
Has NIST's work on AI management framework revealed new or 
underdeveloped areas for standardization with regard to 
trustworthy AI systems? And then, because we want to hear from 
you on that, but then I want to hear from, I guess, Crenshaw, 
Mr. Crenshaw, about the--you know, how beneficial it is to 
industry actors for the Federal Government to lay out 
priorities and standards for critical technologies and 
artificial intelligence. Are you using these?
    But let's start with you, Ms. Tabassi.
    Ms. Tabassi. Thank you very much for the question, 
Chairwoman. Yes, in 2019, we developed a plan for Federal 
Government engagement in development of technical standards, 
and it has several recommendations on bolstering research 
that's really important for development of good, technically 
solid, scientifically valid standards, but also importance of 
public-private partnership and coordination across the 
government on bolstering our engagements in the standard 
development and importance of international cooperations on 
development of standards that are technically sound and correct 
but also reflect our shared democratic values.
    Let me also say that it also lists standards that are 
related and needed for a trustworthy, responsible AI and of 
course, many of the standards that's happening for information 
technology and software systems can be related to artificial 
intelligence and can be used there but also need for other 
standards for addressing issues such as bias and explainability 
and trustworthy.
    Chairwoman Stevens. Great. And, Mr. Crenshaw, I mean, are 
you using these or, I mean, is this helpful to what you were 
talking about?
    Mr. Crenshaw. The NIST process is incredibly helpful. It is 
getting the conversation started and providing the guidance 
that's necessary for industry to look to. It's incredibly 
important, too, to have buy-in from the affected stakeholder 
community. And I have to applaud NIST for the work that they 
have done through their multiple rounds of comment, their 
multiple rounds of public engagement and public meetings to 
really get this right. And I think it's incredibly important, 
the work they are doing, that there is a set of guidelines for 
industry to look to. I think, you know, on the domestic level, 
that that is a guiding light for industry.
    I would note, it's also important to remember standards 
bodies internationally as well. In order for us to maintain our 
leadership in this front, we need to make sure that we have 
American interests represented with American businesses and 
American policymakers being aware of that. We do know that our 
competitors are trying to pack those bodies, and we want to 
make sure that we are represented as well. I think yesterday--
--
    Chairwoman Stevens. So are you suggesting more investment?
    Mr. Crenshaw. I'm suggesting more participation, so----
    Chairwoman Stevens. Well, we did just reauthorize NIST, 
but, you know, Dr. Isbell, what I was kind of getting at was 
the Turing test, which I know you're familiar with. But I don't 
know if that's really the question now, is it, you know, in 
terms of improving these outcomes with AI? And maybe this is 
too philosophical of a question, but is it the Turing test that 
that we should be focused on or what is the question that we 
should be focused on with the fair implementation of AI across 
a multitude of sectors that are determining our economy at 
grand scale with 5 seconds left?
    Dr. Isbell. There is no question too philosophical. The 
short answer is, it's not the Turing test. It's about the 
actual impact and outcomes on real people. And you have to 
bring those real people in to understand those outcomes.
    Chairwoman Stevens. And with that, I'm going to now 
recognize Mr. Feenstra, our Ranking Member, for 5 minutes.
    Mr. Feenstra. Thank you, Chairman Stevens--Chairwoman 
Stevens, and thank you for those questions. Thank you again for 
all witnesses. I really enjoyed your testimonies.
    You know, there's extensive research going on in my home 
State and my universities and--concerning AI, how it's being 
applied now and into the future. Iowa State's AI Institute for 
Resilient Agriculture is bringing together experts to lay the 
groundwork for developing AI-driven predictive plant models to 
increase the resiliency of agriculture. Researchers at the 
University of Northern Iowa are aiming to use AI to improve 
healthcare outcomes, increase privacy, online security, and 
create predictive maintenance systems for our products. And 
then in the University of Iowa, they're utilizing AI to improve 
the effectiveness of cancer screenings, as well as the work to 
identify and address biases in AI and healthcare models. You 
know, these are just a few examples that are out there, and 
they're limitless.
    And I would just like to say, Dr. Isbell, I'm an academic 
also, and I teach--or did teach consumer behavior. And when you 
start looking at consumer behavior, there's a tremendous amount 
of AI being used, good and bad.
    Ms. Tabassi, I understand that AI won't be replacing 
doctors, all right? I understand that, won't be replacing 
nurses. But we also have the opportunity to learn about 
healthcare-related AI and research, as I just mentioned. 
Fostering trust in AI will be critical to utilizing 
applications such as these in the healthcare sector. And this 
is just one example.
    My question to you, if I can flip my page, can you explain 
how an AI Risk Management Framework will--broadly applied 
across the different sectors and industries to minimize the 
negative impacts of AI systems and maximize positive outcomes? 
You can use any specific sector examples in healthcare if you 
wish, but I'd like to know more about that.
    Ms. Tabassi. Thank you so very much for the question, 
Ranking Member Feenstra. And all of the examples that you said 
just show the potential of AI to really change our lives for 
better. I'm going to use the last example that you brought up, 
the cancer screening. So if you have a cancer screening tool, 
first, as mentioned several times, we wanted to make sure that 
it's accurate, it's working well, but beyond that the accuracy 
should also be balanced with associated risks and impact that 
it can have. So the question comes up about the bias or 
fairness. Does it advantage or disadvantage certain 
demographics? Beyond that there's questions about the 
vulnerability and security and resilience of the AI model, we 
all hear that AI systems are brittle. Can that cause negative 
consequences? The issue of the privacy, the data that's used to 
train the models, can we make sure that the privacy is 
preserved and the training data are not inferred from the 
models?
    And then on top of that is we heard about the 
explainability also. If the tool comes out and gives, for 
example, an outcome or prediction that there is a cancer there, 
that's a very serious message to be carried to the doctor to 
the patient. So explainability on how the model decides that 
there's a cancer there, and another level of complexity, the 
explanation needed for physician versus technician versus 
patient is different. AI RMF is trying to provide a shared 
lexicon, interoperable way to address all of these questions, 
but also provide a measurable process, metrics and methodology 
to measure them and manage these risks.
    Mr. Feenstra. Thank you so much for that. That's great 
information.
    Mr. Crenshaw, in your testimony you say that trust is a 
partnership? I 100 percent agree. And only when government and 
industry work side by side can trust be built. How did NIST 
work with industry in developing the AI Risk Management 
Framework? And how is having a tool like the framework going to 
strengthen consumer confidence when it comes to building trust 
in the AI systems?
    Mr. Crenshaw. Well, I think as I said, Congressman, trust 
is essential. And I think NIST has done a great job of really 
instilling trust in their work with the business community by 
being open and transparent. If you look at the the comment 
record, it's comments from across the board, everyone from 
civil society all the way to industry and developers. And 
they're really looking to develop a robust record. That I 
believe is a really great example for other agencies as they're 
looking at tackling this issue to look at. So they've had 
multiple stakeholder sessions. They've come in and actually 
spoken with our members and tried to get a good feel for where 
they're at. And it really--the partnership has been excellent, 
and I think it's a great example for other agencies moving 
forward in this space.
    Mr. Feenstra. Thank you, Mr. Crenshaw, I have questions for 
Dr. Isbell and Ms. Singh, but I ran out of time. So with that, 
thank you for your testimony. I yield back.
    Chairwoman Stevens. Great. And with that, we're going to 
hear from Dr. Foster for 5 minutes of questioning.
    Mr. Foster. Thank you, Madam Chair.
    So my first general question is this discussion converging? 
You know, I've been chairing the Task Force on AI and Financial 
Services for the last several years, and it strikes me that the 
complexity of AI behavior is increasing much more rapidly than 
our ability to categorize and regulate it. You know, an example 
of that is a simple neural net classifier that's operating on a 
static data set to calculate credit scores or something like 
that has a relatively--it's an enormous, but it's a relatively 
finite range of behaviors to categorize, OK?
    On the other hand, interactive AI, which is an agent which 
is learning from other intelligent agents and guiding its 
behavior, has an enormously larger space of behaviors to 
characterize. And I just don't even see how you can possibly 
explain how an intelligent agent might react in any given 
circumstances. Like you can say general things like, you know, 
this child is a fast learner but makes a lot of mistakes, but 
that doesn't give you the granularity of detail you need.
    And so I'm just wondering, since you've been all thinking 
about this, do you get the feeling that it is converging or 
not? No? Dr. Isbell?
    Dr. Isbell. The short answer is no. The problems that we're 
talking about are exponential. All of our solutions are linear. 
You might as well ask the question whether human behavior is 
converging and we know how to understand or regulate that. And 
of course, the answer is no, but that does not mean that there 
are not things that we can do to make progress. And I do think 
a lot of the discussions that we've had just in the last couple 
of years around fairness, accountability, thinking about how to 
educate people to be in the--to be a part of these discussions 
do make real progress, and that progress doesn't--is very 
sudden, and makes very sudden changes, so it's a good thing.
    Mr. Foster. Any other thoughts on this? Yes, Dr. Singh?
    Ms. Singh. Congressman, I think that's a great question. I 
believe we are making progress toward convergence. But one of 
the key areas that I spoke about earlier is how important 
context is to this work. So one of the core acts that we have 
as standards emerge in this space is really thinking about 
context, the applications, and how we can make progress toward 
the right metrics and assessments, along with the specific 
reporting requirements. And we are seeing globally as well as 
the great work that NIST is doing that there is a convergence 
that has started to happen in terms of having those contextual 
conversations.
    Mr. Foster. Any other thoughts? It's a huge question. Let's 
see--many of you have emphasized education and the need for an 
educated public. So if you had to choose between a public that 
knew statistics or knew calculus, which would you take? I'm a 
physicist, so I naturally lean toward calculus, but it seems 
like what I use every day as a politician, statistics are 
relevant. And probably for AI, I think you're in the same bin. 
And do you have any--well, all right, Dr. Isbell--but you have 
to deal with curricula, so you're on the seat again.
    Dr. Isbell. I'm not speaking for all of my colleagues. I 
think the answer is, if I had to choose for most people, it 
would be statistics, but I'd also like them to know information 
theory and linear algebra. But fundamentally, it's about 
problem solving around data mattering as opposed to just the 
algorithms and the processes that you go through. And with that 
you can solve a lot of the problems or at least address and 
think about the problems that are coming down the pike.
    Mr. Foster. Any other thoughts from any of you? What do you 
use every day, statistics or calculus? I think--yes, machine 
learning. It's--backpropagation is the chain rule, and I don't 
think there's much other calculus anywhere in it. But anyway, 
the--now, actually, this was for Mr. Crenshaw. You've 
emphasized international competition, and it strikes me that a 
lot of the countries that are clobbering us, you can't get out 
of high school without knowing calculus and probably 
statistics. There's all sorts of people showing up at school 
boards, you know, unhappy that we're not supporting their 
preferred theology or mythology. But very few school boards are 
being inundated by people, you know, demanding that our kids 
know statistics and calculus. What--is there some--is there 
work to be done there?
    Mr. Crenshaw. There's definitely work to be done on the 
education front. We need to prioritize STEM education to ensure 
that we have the fundamental knowledge base for students across 
the country to get into this field because we are going to need 
more coders and ethicists in this field who actually can assist 
with our leadership.
    The other thing I think would be important to note, too, is 
that we also need to make sure that we have talent in this 
country and retain talent and still attract talent. And one of 
the things that we found out through our AI Commission is that 
we, you know are going to lose the talent race if we don't deal 
with our immigration issues in this country as well and make 
sure that we can retain talent after we've educated them here 
in the United States, make sure that we can keep our talent to 
ensure that we have people who know how to make ethical AI 
work.
    Mr. Foster. Thank you. And we in a bipartisan way on this 
Committee have been doing everything we can to try to drag that 
across the finish line. I think we came within one Senator of 
doing something significant in the CHIPS and Science.
    Anyway, my time's up and will yield back.
    Chairwoman Stevens. And with that, we will hear from the 
Ranking Member of the Full Committee who we're so grateful is 
here, Mr. Lucas for 5 minutes of questioning.
    Mr. Lucas. Thank you, Madam Chairman. Ms. Tabassi, in the 
AI Initiative that we passed in Congress last year, we gave 
NIST the difficult task of defining what makes AI safe and 
trustworthy. Can you walk us through the process of how NIST 
determined that definition of trustworthiness? And while you're 
thinking about that, do you think this measure of 
trustworthiness also helps with the measuring of fairness in AI 
systems, please?
    Ms. Tabassi. Thank you so very much, Ranking Member Lucas, 
for the question. In terms of the process of developing a 
definition of the trustworthiness, I want to thank the kind of 
work that has been mentioned about the NIST process. But the 
process has been an open, transparent, collaborative process. 
There has been many definitions and proposals for definition 
for trustworthiness, so we ran a stakeholder-driven effort to 
converge to the extent possible on the definition of the 
trustworthiness. And that, as was mentioned, include rounds of 
workshops and public comment and a listening session. So that 
was the process.
    Your second part of the question is about the fairness. So 
fairness is one of the aspects of the trustworthiness as it's 
mentioned in the AI RMF. And fairness, as it was mentioned, is 
a complicated concept because it can depend on societal values 
and can change from context to context. But that's also part of 
one of the aspects of the trustworthiness mentioned in the AI 
RMF.
    Mr. Lucas. Ms. Singh, in your testimony, you illustrate why 
you cannot have a one-size-fits-all definition of an 
algorithmic fairness. How does the AI Risk Management Framework 
exemplify this?
    Ms. Singh. As I previously stated, I really commend NIST 
for the Risk Management Framework and how they're thinking 
through not only mapping different applications, but measuring 
and then overall management of those. At Credo AI, we are 
really focused on operationalizing responsible AI tenets and 
ensuring that continuous oversight and governance is provided 
of these systems. And I think for us it is really critical that 
there are governance assets based on the context of AI 
application that gets generated that inspires that trust that 
Ms. Tabassi was just talking about.
    Mr. Lucas. Mr. Crenshaw, do you foresee U.S. industry 
widely adopting and utilizing the Risk Management Framework 
since it's a voluntary tool, or will it need to be 
incentivized? While you're thinking about that, do you 
anticipate U.S. standard bodies will play a role in encouraging 
the utilization of the framework?
    Mr. Crenshaw. I think there's definitely a role there. I 
think they also have really gotten the conversation out about 
the need to develop standards. When it comes to the NIST Risk 
Management Framework, I think what we've seen of it is 
promising. Obviously, we'll have to comment on the final 
product when it comes out. But I think it is a promising 
product. And, you know, I think, given the fact that we've had 
such robust stakeholder input, I do anticipate that, you know, 
given the direction things are going, we definitely could see 
stakeholder engagement to support the framework. And I think 
that's a good thing because we need guidelines and standards to 
get behind so we can develop trust.
    Mr. Lucas. Ms. Singh, do you have any thoughts on this 
point?
    Ms. Singh. I think multistakeholder engagement is going to 
be critical in the process. And as--you know, we've been 
invited to give feedback on the NIST RMF, and we've done that 
actively over the past couple of months. As mentioned, I think 
there's a little bit more work to be done in terms of ensuring 
that we are looking at different applications and context.
    Mr. Lucas. Ms. Tabassi, any thoughts?
    Ms. Tabassi. In terms of the adoption, I think that the 
adoption and use of the AI RMF would be based on the value that 
it provides and also giving awareness that these things exist 
is also very important. I thank again the Committee and all of 
my panelists for the kind words about the process. And in terms 
of the context and specific use, agreed that a lot more work 
needs to be done. And we have a call for contribution 
particularly for that.
    Mr. Lucas. One last question, and I come back to you, Ms. 
Tabassi. Why is it important for democratic nations to lead the 
development of international standards for trustworthy AI 
systems?
    Ms. Tabassi. I believe it's important to affirm our shared 
democratic values of openness, protection of democracy and 
human rights, and design and develop technologies that 
operationalizes those values. And we need standards for 
technologies that are rights-affirming and show those values.
    Mr. Lucas. Just the way I intend to answer questions about 
that in my town meeting someday. Thank you. Yield back, Madam 
Chair.
    Chairwoman Stevens. With that, we are going to hear from 
the Congresswoman from North Carolina, Ms. Ross, for 5 minutes 
of questioning.
    Ms. Ross. Thank you very much, Chairwoman Stevens and 
Ranking Member Feenstra. And thank you to the panelists for 
joining us today. On April 29th of last year [inaudible] 
represents a larger problem of cybersecurity and privacy issues 
in this country. AI innovation happens fast, and we need 
legislation that's equipped to grow into this quickly expanding 
sector. For my constituents in the Research Triangle and for 
national security more broadly, we need to invest in long-term 
structural infrastructure that ensures better cybersecurity and 
privacy in our tech sector. We also need to look at how AI 
affects the arts and our creators, and we all have many of them 
in our district. So I look forward to hearing from our 
witnesses on how we can ensure that systems of machine learning 
can be created with consideration for individual privacy, 
corporate privacy, intellectual property, and national 
security. But since none of the folks who have asked questions 
yet have talked about intellectual property, and I serve on the 
Judiciary Subcommittee on that, I'm going to ask Ms. Tabassi--
I'm sorry if I mispronounced your name--to say I want to thank 
you for your important work on the draft of the Artificial 
Intelligence Risk Management Framework.
    But I also want to talk a little bit about intellectual 
property because the United States takes our intellectual 
property protections very seriously. And without those 
protections, there's a significant threat to American 
creativity, ingenuity, jobs, and our economy. And AI offers 
opportunities to artists and creators to enhance the creation 
process in many ways, but that also presents risks. And there 
are services and sites available today that use art, books, 
music, and other American-made works as inputs to train AI.
    Based on what is happening with image-generating AI 
currently on the web, we can already see that artists will have 
to compete with AI creations in their own style and trained on 
their own content when they were either--neither consulted nor 
compensated for this. And as a matter of fact, there was a 
recent article that I just read about that. Is this issue on 
NIST's radar screen, and what can we do about it?
    Ms. Tabassi. Thank you so very much for the question, 
Congresswoman. And we have actually received comments to that 
effect to AI RMF. And that's a serious problem, certainly 
something that would be part of the discussions in the future 
drafts of the RMF. A lot of work needs to be done, and that 
would definitely be part of the discussion. Thank you.
    Ms. Ross. OK. I do have a couple of other questions. Dr. 
Isbell, your written testimony talks about the Marshall Project 
and the use of risk assessment in the criminal justice system. 
How can transparency increase the ability of individuals to 
protect their information and avoid undue scrutiny? And to whom 
should individuals direct their concerns if they believe that 
their data has been misused?
    Dr. Isbell. So it's a very--it's actually quite a difficult 
problem because the data that we have is out there everywhere, 
and we leave a trail everywhere that we go. Fundamentally, 
there has to be policy and there has to be infrastructure. This 
is a role that government has to provide a mechanism by which 
people can can deal with issues where their data had been 
misused. It is not a thing that will naturally come from 
industry. It is not a thing that naturally comes from the 
educational sector. It is something that has to be dealt with 
by the legal system.
    Ms. Ross. And can you tell us about any law enforcement 
practices that we should be aware of as we're considering 
changes to the legal system?
    Dr. Isbell. Well, I think the short answer is you have to 
think very carefully about and look at the way that the systems 
that are out there are currently being used and how they're 
currently being misused. And having done that, it takes you 
down a path toward understanding how you have to try to address 
those one at a time. It's a pervasive thing that touches 
everything. I--we don't have time to talk about this now, but 
you--earlier, someone made a comment that doctors will not be 
replaced by AI. Well, they're already being replaced by AI, and 
they're being done in an unregulated way that's having an 
impact on people. And you have to be--you have to recognize 
that and you have to address it context by context and one case 
at a time.
    Ms. Ross. Thank you, Madam Chairman, and I yield back.
    Chairwoman Stevens. Great. And with that, we're going to 
hear from Dr. Baird of Indiana for 5 minutes of questioning.
    Mr. Baird. Thank you, Madam Chair. And I appreciate you and 
Ranking Member Feenstra for holding this important hearing. And 
I really appreciate, I always do, the expertise of the 
witnesses and their ability to answer our questions and it's 
very important and very specific.
    My first question goes to Dr. Isbell. And I want to know 
what role have universities played in the development of the AI 
Risk Management Framework? And more broadly, how are 
universities helping to shape the future of AI by engaging in 
public-private partnerships, Dr. Isbell?
    Dr. Isbell. So the--higher education in general is--
universities have participated by being invited in and being a 
part of the conversations. Individuals and organizations have 
continued to participate in all of these discussions around 
standards, including things that NIST has done, but also 
through operations of institutes that have been created, for 
example, by NSF. What the universities do, what our role is, is 
to do the basic research that exists to create the basic 
research, ask the basic questions, and then educate the 
students who are going to go forward and to do that work. A lot 
of the work that we do, a lot of where we play that role isn't 
actually identifying the fundamental problems. That is sort of 
what academic freedom allows you to do, and that's what we 
continue to do. The environment that we create is one that is--
that allows us to ask these questions and to make them 
available for industry, to make them available for government 
to take the next step. That's what we do.
    Mr. Baird. Well, thank you very much. Ms. Tabassi, to your 
knowledge, has the People's Republic of China developed a 
similar tool to the AI Risk Management Framework? And what 
about any of our allies? And so what role if any has NIST 
played in sharing findings and the best practices with the 
international community, particularly our allies? So if you 
have any thoughts in that area, I would appreciate it.
    Ms. Tabassi. Thank you so very much for the question, 
Congressman. In terms of cooperation and collaboration with our 
allies, the stakeholder engagement effort that we run includes 
our international partners, so they have been involved in terms 
of providing input to the AI RMF, coming to our workshops and 
participating in those events, but we also interact with them 
and talk with them in forums such as Trade and Technology 
Council, QUAD, or OECD. So there is a good, strong, robust 
engagement going on that way.
    Mr. Baird. Thank you. Then my last question goes to Ms. 
Singh. So in creating the tools to help companies develop 
responsible AI, what are some of the most common concerns with 
AI systems that your company has seen?
    Ms. Singh. Thank you so much for that question. You know, 
if responsibly and not built artificial intelligence is going 
to have very varying impacts on different use cases. So across 
the companies that we work with, one of the things that is 
critical is, again, really having a holistic view of from the 
time you're designing the AI system to the actual use, making 
sure that you're interrogating the technical systems, you're 
interrogating the processes, as well as you're interrogating 
the outputs. So this goes back to really identifying any 
unintended consequences that could appear in the entire AI 
lifecycle.
    Mr. Baird. Thank you very much. And I appreciate the 
witnesses' responses. And with that, Madam Chair, I yield back.
    Mr. McNerney [presiding]. Well, I was going to--I think I'm 
the next questioner, and I was going to thank the Chairwoman 
for this great hearing, but I certainly want to thank the 
panelists. Your testimony is great. What a great, incredible 
subject. I want to get right to questions though.
    Ms. Tabassi, how might standards and assessments be 
developed and--for explainability and interoperability
    Ms. Tabassi. We do that the same way that we do for any 
type of other standards. With true stakeholder engagements and 
working with a whole community. Broad stakeholder engagement 
underlines everything we do at NIST and explainability, 
interoperability are difficult, complex topics. We do have some 
foundational research going on. Our researchers are working on 
this, but we also augment it with the work of the whole 
community.
    Mr. McNerney. OK. Well, I've been on standards committees, 
and I know what kind of work goes on. So you're saying it's a 
similar process or would be a similar process?
    Ms. Tabassi. Correct. Part of it, doing the internal 
research, providing technical contributions, working with the 
whole community on strengthening the research and taking the 
contributions to the standard development organizations and 
hopefully see them through become international standards.
    Mr. McNerney. Thank you.
    Dr. Isbell, in math and physics, systems and solutions are 
considered unstable if small changes in the initial conditions 
result in large changes in the solutions and outputs. Are AI 
systems unstable in terms of the data input? And, if so, how 
can that be mitigated?
    Dr. Isbell. Some of them are. There's a wide range of ways 
of doing AI and machine learning. Some of them are quite 
stable, and some of them are less stable. There's a lot of 
theory behind this and a lot of work that's been done over 
decades to get there.
    I think the most important thing actually is not the sort 
of instability that you're talking about with small changes but 
that we don't actually understand how the set of parameters 
that go into the way that we build these systems have that 
impact. It's actually less about the data in that sense and 
more about the way that we build the systems in the first 
place. And that has remained largely unexplored.
    Mr. McNerney. Well, thank you. That'd be a great area for 
research. Thank you.
    Ms. Tabassi, can you touch briefly on what's included in 
the strategy of engaging technical standards for tools for 
artificial intelligence?
    Ms. Tabassi. Thank you for that question, Congressman. And, 
yes, happy to. So that strategy for working toward the standard 
was developed in 2019. And we are basically implementing the 
recommendations of that plan since it has been developed in 
2019. What's in the plan? Basically talks about standards, 
standard development processes, talks about AI standards, 
what's needed, and concludes with recommendations on what's 
needed to maintain U.S. leadership in development of the 
technical standards and recommendations very broadly is about 
strengthening research for development of scientifically valid 
standards, public-private partnership, to be able to do that 
research and build those foundations, and international 
cooperations for development of standards.
    I just also want to note, that plan was also developed in a 
stakeholder-driven effort with a lot of input from the 
community.
    Mr. McNerney. Thank you. So what what extent is the United 
States already collaborating with the EU and other likeminded 
nations on developing standards for trustworthy AI?
    Ms. Tabassi. Multiple ways. One of them is by expert-to-
expert scientists working on what we call pre-standardization 
research to actually provide the scientific foundations for the 
standards and then cooperation by to the standard meeting and 
seeing them through to become international standard, but also 
at the forum such as TTC and QUAD.
    Mr. McNerney. Well, thank you.
    Mr. Crenshaw, I didn't want to leave you out. Would the 
Chamber and presumably many U.S. businesses support the 
development of a United States AI regulatory law?
    Mr. Crenshaw. I think, given the state of the technology, 
we believe it's premature to get into prescriptive regulation. 
We support voluntary frameworks like we see at NIST. A few 
areas, though, I think, you know, we would like to see 
regulation is for things like consumer privacy. We'd like to 
see a national standard put in place. But at the same time, we 
want to make sure that the process at NIST can work itself out 
first before we start making any kind of determinations on 
regulation. And it's also an issue, though, our own AI 
Commission is working through as well to make recommendations 
for.
    Mr. McNerney. Thank you. My time has expired, and I'm going 
to call on Mr. LaTurner. You're up for 5 minutes.
    Mr. LaTurner. Thank you, Mr. Chairman. I appreciate it. Ms. 
Singh, in your testimony, you talk about the need for 
policymakers to establish benchmarks for fairness when it comes 
to responsible AI, yet you also talked about how industry-
specific and context-driven artificial intelligence factors 
preclude standard-setting bodies from creating a one-size-fits-
all metrics. In a context-specific field, how can Congress 
create meaningful regulation that ensures AI systems retain 
algorithmic fairness?
    Ms. Singh. Thank you so much for that question. I think the 
work that NIST is doing is a good example of the public-private 
partnership that is needed to ensure that we are doing 
thoughtful policymaking and standards that are very context-
specific. As I've stated previously, you know, in artificial 
intelligence, the question that we should be asking ourselves 
right now is how can governance and oversight keep up with the 
development of artificial intelligence? And so we believe that 
standards are going to be critical, especially as we think 
about transparency reporting. And transparency reporting, is 
going to be a complete view into the AI lifecycle that can help 
with benchmarking.
    Mr. LaTurner. What could we be doing differently with our--
with Congress and the public-private partnerships? Do you have 
any recommendations on how we could be doing it better?
    Ms. Singh. Yes, thank you so much for that question. You 
know, we've given some feedback to NIST on that. I think we 
have to really step back and think about the AI application, as 
well as what the impact to the stakeholders within that AI 
application is. And I think going back to context-centric 
metrics, as well as context-centric reporting requirements is 
one of the first steps we believe is going to help move this 
industry forward.
    Mr. LaTurner. How can developing responsible AI give the 
United States an economic and societal competitive advantage 
over other countries
    Ms. Singh. Thank you. I think that is a fantastic question. 
We at Credo AI believe that responsible AI is a competitive 
advantage because it is not only going to help United States 
and the companies here deploy AI with confidence, but as we 
make sure that the standards that emerge which are aligned with 
our societal values, that is going to promote more consumer 
trust, which, as you can imagine, is going to further bolster 
our leadership in artificial intelligence.
    Mr. LaTurner. Thank you, Ms. Singh.
    Dr. Isbell, you state in your testimony that there are many 
occasions where tech workers cannot be certain how AI 
algorithms reach the correct answer, and these algorithms are 
known as, quote, black-box models. If for any reason these 
types of algorithms reach an incorrect or biased outcome like 
the ones you describe in your testimony, it can be nearly 
impossible to diagnose. If we want to solve the problem of 
black-box models by making an algorithm's data set more 
transparent, then what countermeasures can we take to bolster 
AI security from hackers? To your knowledge, are there any 
examples of AI developers that have already--that are already 
addressing this issue?
    Dr. Isbell. So there's a great amount--there's a large 
amount of work that's being done in academia at the level of 
basic research to understand differential privacy, to 
understand how it is that people can interfere and break into 
the way that machine learning algorithms actually work. So 
there's a lot of work. It's in early stages, but a lot of great 
stuff is being done. How much of the--not a lot of that has 
necessarily been deployed in the systems that are out there now 
I think in large part because the incentives haven't 
necessarily been there.
    What drives industry and drives the people who build these 
systems and deploy them to do--to touch on this is requirements 
that either through the market or through policy, that if they 
don't do this, they're simply not going to be able to deploy 
their systems and to have them used and adopted by large groups 
of people.
    So there's a lot of work that's been done out there, a lot 
of specific things. I would start with differential privacy, 
and there's lots of researchers that have done great work on 
this. But at the end of the day, it's really going to be about 
creating the incentives for people to want to take advantage of 
what we know in order to keep things secure.
    Mr. LaTurner. Thank you. Mr. Chairman, I yield back.
    Chairwoman Stevens. Great. And with that, we're going to 
hear from Mr. Beyer of the Commonwealth of Virginia for 5 
minutes of questioning.
    Mr. Beyer. Thank you, Madam Chair, very much. And thank the 
witnesses for really interesting feedback. But also thank my 
colleagues, Democrats and Republicans, for some very good 
questions.
    Ms. Tabassi, I know you take on this tremendous task of 
managing, developing the AI Risk Management Framework. You 
heard from Mr. Crenshaw what the Chamber is doing with its 
commission. And I think you've heard pushback about how we're 
not ready to have mandatory standards, that we're still so 
early that we're--we don't want to overreact. We don't want to 
overregulate. But at the same time is it not naive to think 
that we can make this voluntary indefinitely, that at some 
point there won't be a need for clarity in terms of what is 
demanded and expected from businesses in AI?
    Ms. Tabassi. Thank you very much for that very thoughtful 
question, Congressman. So NIST AI RMF is a voluntary framework 
just like any other frameworks that NIST has developed. And the 
use and adoption of that, at least, I believe, would be based 
on the value that it provides. And another strength of the 
voluntary process that we are doing is based on the stakeholder 
engagement and stakeholder-driven process that we are following 
in development of this voluntary tool. It gives the opportunity 
to the whole community to provide their input, their comments. 
So by the end, the final tool would be a more effective 
resource that everybody that participate in development of that 
would have a buy-in in that.
    So by that, I think, having the value on using this and 
having buy-in because of participation in the process of 
developing it, would help with its adoption. NIST is a 
nonregulatory agency, and the things we put out are voluntary.
    Mr. Beyer. We know that, so thank you. I understand you're 
nonregulatory and ultimately it will come back to us and then 
come back to us just based on dangers.
    Dr. Isbell, I was fascinated by your testimony. Because so 
much of what we talked about today is concern about biases, but 
you also had a wonderful paragraph about the upside of machine 
learning and artificial intelligence. Can you expand on that a 
little bit? It seems to me that we as human beings dramatically 
underestimate the potential for what artificial intelligence 
can bring humanity.
    Dr. Isbell. So there's a particular law, and I forget 
what--escapes me right now. But what the law says is that we 
overestimate the short term and we underestimate the long term. 
And I think that's exactly what's been happening with AI. There 
was a lot of hype back in the 1970's and 1980's before the AI 
winter with all the great changes that AI was going to bring to 
the world. They were wrong. They were overhyped.
    But it's turned out that the impact that AI has had has 
been profound and far deeper than anything anyone even imagined 
back then. It has infiltrated every part of our life, and I use 
infiltrate in a positive way. We will be doing a better job of 
detecting when people are sick in ways that we were never able 
to do. We will be able to help people to make decisions they 
otherwise would not have ever been able to make. We will be 
able to connect with one another in ways that we have not been 
able to connect with one another before. And a large part of it 
will be because of computing, and it'll be because of AI. It's 
all very positive. The opportunities in front of us are huge, 
and it will take us--it will help us to solve big problems that 
we currently have a hard time thinking through and those 
problems over decades and even over centuries.
    The problem that we have, of course, is that we have to set 
up the incentives to allow people to do that, and we have to 
make certain that everyday people understand enough of what's 
actually going on so that they can make rational decisions 
about how to use that technology in their own lives.
    Mr. Beyer. Dr. Isbell, I'd love to have a question for the 
record if you could find one of your research assistants to 
find out the name of that law.
    Dr. Isbell. I will.
    Mr. Beyer. Dr. Vint Cerf told it to me 30 years ago, and 
I've always attributed it to him, but it probably has a deeper 
root.
    Dr. Isbell. Absolutely.
    Mr. Beyer. Very powerful.
    Dr. Singh, one quick question. You know, we've been 
struggling with facial recognition technology on police 
bodycams. Now, is this something that you're working on, too, 
that the notion that people of color, especially women of 
color, are picked up inaccurately much more frequently than 
others?
    Ms. Singh. Thank you so much for that question. We at Credo 
AI work across a diverse range of applications, including 
facial recognition. And as I stated previously, I think any 
artificial intelligence that is not developed responsibly is 
going to impact all of us, and especially the marginalized 
communities, which in the past have been excluded because of 
gender, ethnicity, color, are at a higher disadvantage here. So 
building responsible AI is not just competitive advantage, but 
it is going to serve humanity really well.
    Mr. Beyer. Madam Chair, I yield back.
    Chairwoman Stevens. Thank you. And with that, we're going 
to hear from Mr. Gonzalez of Ohio for 5 minutes of questioning.
    Mr. Gonzalez. Thank you, Chairwoman Stevens, Ranking Member 
Feenstra, for holding this hearing. Thanks to all the witnesses 
for your testimonies.
    Ms. Tabassi, we talked a little bit about the AI Risk 
Management Framework, and that was helpful. I'm curious, has 
China developed a similar tool? What is China doing 
specifically around this?
    Ms. Tabassi. Right. So I believe it was in 2017 that China 
put a very ambitious domestic AI plan out. To the best of my 
knowledge, there isn't anything that they're doing similar to 
the AI RMF. If they're doing it domestically, I don't know. 
But--yes.
    Mr. Gonzalez. OK. Thank you.
    Mr. Crenshaw, I'm going to switch to you for a second. 
Unlike most countries that have a top-down, government-led 
approach, the United States has a bottoms-up, industry-led 
approach to standards setting, which I think is appropriate. We 
employ a voluntary system which relies on industry 
participation and leadership. This market-driven approach 
enables competition, ensures transparency, and takes advantage 
of consensus-building to drive us to the best possible 
outcomes. Can you explain how the U.S. approach to AI through 
the AI Risk Management Framework drives innovation?
    Mr. Crenshaw. Well, I think it's interesting to know, 
during one of our hearings, we actually had one of the cochairs 
of the National AI Advisory Committee come testify, Miriam 
Vogel. And she said the reason we needed to maintain leadership 
in this country is because we have a brand of trust compared to 
other countries. And it's important that we have standards in 
place that are voluntary, that will be adaptable to this new 
and developing technology but at the same time will look at 
things like risk. And it's important that we have real firm 
guidance in place.
    And another--I think, as I said before as well, when it 
comes to international standards bodies, we need to make sure 
that the United States is well-represented. The CHIPS and 
Science Act actually helped provide funding to ensure we can 
participate in that space. But, you know, at the same time, 
too, as companies look at things like developing implementation 
for compliance or following guidelines, if they go out there 
and say we're following this guideline and then they're found 
not to be, there is some teeth there.
    Mr. Gonzalez. Yes.
    Mr. Crenshaw. So there are agencies that can enforce there 
as well.
    Mr. Gonzalez. Great.
    Mr. Crenshaw. So there is great trust to be had by 
establishing leadership and trust against other countries.
    Mr. Gonzalez. Dr. Isbell, with your role on campus as a 
Professor and Dean, what do you believe the appropriate role of 
the university is--are in shaping the future of AI?
    Dr. Isbell. Twofold. One is to do research. We have one of 
the best systems in the world around basic research. Our 
research ones are amazing. And all the way down to our research 
twos and even our community colleges are able to bring people 
in and to think about and engage in the conversation around AI 
or any other large, important issue. So the research is 
important, and maintaining and supporting that is important.
    But the second and perhaps the most obvious is the 
fundamental mission, which is educating people, not just 
educating the people who are going to do the research, but I 
think importantly, and especially when it comes to AI and 
machine learning, is educating everyone else who is not going 
to do AI and machine learning research but will be affected by 
it, who will be adjacent to it, and will be far away. As I told 
my son who's deeply into history, you will not be able to get a 
degree in history in 5 years without knowing machine learning 
and AI because it's still going to be data-driven. And so our 
responsibility is to make certain that everyone is a part of 
that conversation.
    Mr. Gonzalez. Great. And then I agree 100 percent on the 
research point, actually, on both points. But, you know, one 
thing we talk about a lot on this Committee is how do we get 
the research--the incredible research that's happening on our 
university campuses out into the public space and then driving 
innovation in the private sector? So what do you think we need 
to be doing to have a--I'll just call it a more robust sort of 
flywheel of research taking place on college campuses, leads to 
innovation, leads to private companies, et cetera, et cetera?
    Dr. Isbell. So we actually do pretty well with that, I 
think, but I think the biggest problem right now is that 
there's a mismatch between what the company--pick whatever your 
favorite company is--wants to do in the next 6 months to a year 
versus what the basic research that's looking out 5 or 10 years 
actually is. Support through organizations like NSF, for 
example, to help partner with those companies, to partner with 
industry to help do the basic research, universities, I think, 
is the best way to get that translational work done from the 
lab out into the world. And when it works, it works very well.
    Mr. Gonzalez. Thank you. I yield back.
    Chairwoman Stevens. Thank you.
    With that, we'll hear from Congressman Sherman of 
California for 5 minutes of questioning.
    Mr. Sherman. Thank you, and thank you for allowing me to 
participate in this Subcommittee's hearing. Without objection, 
I'd like to enter into the record an article I wrote 22 years 
ago, ``Engineered Intelligence: Creating Our Successors' 
Species.''
    My line of questioning is going to be about things that 
won't affect us until the second half of this century. But 
since they relate to whether humankind will continue to be in 
domination of the planet Earth, they're important. We're--right 
now, the computer engineers and the bioengineers are racing to 
create a new level of intelligence. And the last time there was 
a higher level, a new level of intelligence appeared on the 
planet is when our ancestors said hello to Neanderthal. It did 
not work out well for Neanderthal.
    So my focus is on whether we're going to see artificial 
intelligence that has general intelligence, self-awareness, and 
what I call the ambition, or survival instinct, or care. And 
that third thing I should go into more, I tend to think that 
our successor species would be biological because even the 
dumbest worm seems to care if you try to turn it off or kill 
it, whereas the smartest computers we have so far don't care if 
you unplug them.
    So my concern is what are we doing to prevent or monitor 
for general intelligence, self-awareness, and ambition or 
survival instinct? Or are we just going to ignore those issues 
and focus on things that affect us in the next decade? Ms. 
Tabassi?
    Ms. Tabassi. Thank you very much, Congressman, for the 
question. It's hard to determine when or if we can reach or the 
community can reach to an artificial general intelligence. I 
will say that that's----
    Mr. Sherman. Well, I think we're going to get there 
someday.
    Ms. Tabassi. Right.
    Mr. Sherman. We just don't know----
    Ms. Tabassi. Very good, very good. So we don't know when 
we're going to get there. So from the NIST point of view, we 
think that that's one reason to work on foundational 
principles. That's why it's now timely----
    Mr. Sherman. Is anybody doing any technical research about 
how we can get very useful computers, that we somehow put 
something in there, a governor if you will, that prevents 
general intelligence or prevents self-awareness, or prevents 
ambition and caring? Is anybody doing the research as to how we 
can get what we want without getting what we don't want?
    Ms. Tabassi. I'm not aware of that research being done at 
our laboratory at NIST, across the academia, and the community. 
I don't know. Thank you for the question.
    Mr. Sherman. I'll ask the other witnesses. Is anybody aware 
of us trying to prevent, as we try to harvest the benefits of 
artificial intelligence, the creation of an ambitious, self-
aware computer that may very well decide that we're irrelevant 
to this planet? Is anybody figuring out how to do that, or is 
it just an issue we're all aware of but aren't really trying to 
confront? Does anyone just--yes, Mr.--yes, Doctor?
    Dr. Isbell. So I guess the--yes, and thank you for the 
question. Actually, you know, one of the reasons I got into AI 
in the first place were these what I'd consider pretty 
existential and philosophical questions around what does it 
mean to build intelligence? I think the answer is that people 
discuss these issues all the time. They try to figure it out, 
they try to work it through. We don't have any large research, 
at least that I'm aware of, any large research agendas around 
preventing the issue--preventing general intelligence in part 
because we have no idea how to get there from here. And I think 
one of the things that I would leave----
    Mr. Sherman. What about those two other issues, how to 
prevent self-awareness, how to monitor for self-awareness, how 
to prevent ambition or survival instinct, how to monitor for 
survival instinct?
    Dr. Isbell. I don't think it's done in those terms. I don't 
think it's done in those terms. It's done in simpler terms 
around preventing harm.
    Mr. Sherman. Well, we're going to concentrate on the harm 
that could occur in the next decade----
    Dr. Isbell. That's right.
    Mr. Sherman [continuing]. The Nation or artists that lose 
their creativity and the benefits of their creativity, and it 
doesn't seem like anybody's worried about the problems we'll 
confront in the second half of this century. And with that, I 
yield back.
    Chairwoman Stevens. Great. And with that, we're going to go 
to another round of questions because we're just having so much 
fun here. And the Chair is going to recognize herself for 5 
minutes. I think this question about where and how we're 
determining the ethics is very important. Obviously, we have so 
much respect for NIST and an understanding of the role that 
standards play. We could go philosophical again and ask our 
standards, ethics, and how the ethics arrive out of standards 
that come from rigorous processes that are inputted by--you 
know, we talked about the companies, we've heard from Dr. 
Isbell about the people, the people element that needs to get 
involved with the standards.
    But, Dr. Isbell, some universities are already including 
ethics as a curriculum and long have. You go into a philosophy 
department, you're going to get an ethics course. Hopefully, 
people take it. But ethics as a curriculum requirement for 
computer science degrees in particular, a great start, but it's 
often obviously sometimes a separate course and may not be 
directly connected to what students are learning in other 
courses.
    You've changed your approach at Georgia Tech, and so I just 
wondering if you could elaborate on what you're doing to 
integrate ethics education and how you're assessing its 
effectiveness. And I also just--because that's a question I 
know you can answer it, but I just really want to applaud you 
for a segment in your testimony that I encourage everyone to 
look at where you said computing has long been an intellectual 
wild west where things change so fast that the priority was 
always to fix--to find what's next, to find the better 
solution. Now, we've succeeded in finding solutions so good 
that they are intertwined in nearly every area of our personal 
lives and communities. So can our laws move fast enough? Can 
our ethics move fast enough? And where and how do we find this 
arising? Thank you.
    Dr. Isbell. Sure. Thank you for the question. I really 
appreciate it. I will say that, you know, people in my field 
have spent 40, 50 years trying to convince everyone that what 
we did was really important, and it turns out, we were right. 
And then what we're living with now are the consequences of 
having been right.
    So when it comes to ethics and responsibility, I think 
the--you know, Georgia Tech, we've had that as a requirement 
for CS going back at least about 30 years. But what we had done 
wrong--and not just us, but I think the way that we approached 
this--is that we treat it, as you say, a separate class, 
something that gets stapled on at the end. It's a requirement. 
Nobody takes it till their last semester. It doesn't get 
integrated into the rest of the curriculum and it can't.
    So one of the things that we did recently is we kept it as 
a requirement, and we made it a prerequisite for our junior 
yearlong design classes. So by the time you're a sophomore, you 
know just enough to be dangerous. You're at a place where 
you're being forced to think carefully about the consequences 
of the systems that you build, and then you're asked to build 
such a big system. This is before you take Intro to AI. This 
before you take Intro to Machine Learning. This is before you 
take Introduction to Cybersecurity and Privacy. So it puts you 
in a place where the people further down the chain can actually 
now ask you the direct questions that they couldn't do before 
because you wouldn't have the language or the experience to be 
able to do that.
    That is what's important. When we claim that something is 
important, we have to operationalize it in our curriculum in 
the way that we teach people from the very beginning and not 
toward the end, which is the natural thing to do if you aren't 
very careful about how important you think that it is.
    Chairwoman Stevens. And certainly to Mr. Crenshaw, I'm sure 
you have some some thoughts about this as well. And, you know, 
we applaud the the point about, hey, we want to drive a--you 
know, American leadership of what we're doing with artificial 
intelligence.
    And thank you, Ms. Singh, by the way. I've just so 
thoroughly enjoyed your--not only your testimony, but the 
answers to your questions. But how do we balance these things 
out, right? You know, we sometimes see, you know, too much of a 
good thing, per se. And we don't--you know, we like standards. 
We're doing standards. You've said you like the risk 
management. But, you know, in some ways, right, we see 
companies getting pushback because they haven't self-regulated 
and the ethics component isn't there. And so, you know, where 
and how do we find that balance? And maybe that's articulated 
through boards. Which--how does that populate? And maybe Ms. 
Singh can chime in, too.
    Mr. Crenshaw. I think it's critically important, one note 
to make, that we have the critical decisionmakers in companies 
involved in this process as well. Not only do technologists 
have a role, but C-suite does as well. And also, you know, we 
need more education out there about the need to build in 
ethical AI into standards for companies and how they operate. 
I've talked to some companies that are actually developing 
their own ethical frameworks and have full-time ethicists who 
are being brought on. We had a hearing actually at the 
Cleveland Clinic about 4 months ago in which they've now 
brought on an ethicist as well, as they're using AI to treat 
their patients. So it's important, and I think companies are 
beginning to see this.
    Chairwoman Stevens. Yes.
    Ms. Singh. Thank you, Chairwoman. I think, today, we've 
established that AI is not a technical problem. It's a 
sociotechnical problem that really needs multistakeholder 
perspective and viewpoints. So I totally agree that there is a 
need for education. There's a need for involvement from 
multiple stakeholders. But if I may, I think the companies we 
work with, they're still struggling with what does good look 
like. And this is where we believe that government has a 
critical role to play in thoughtful policymaking and in these 
standards to at least give that context to these companies 
because everyone right now, even if they're trying to self-
regulate, do not know what does good look like. So our ask 
right now is really making sure that there is more transparency 
around how these systems are built and deployed.
    Chairwoman Stevens. Yes, right. And there's also certainly 
examples from throughout history where the notion of good has 
gotten it wrong.
    But with that, why don't I turn it over to Mr. Feenstra, 
for 5 minutes of questioning. Thank you.
    Mr. Feenstra. Thank you, Madam Chair. I'm so glad that we 
could have an extra round of questions.
    And Dr. Isbell, thank you again for all your comments. I've 
been enjoying listening to you. And, as academics, to me, the 
challenge is--I finished my dissertation on maternity 
healthcare in rural America. And the challenge is, you know, we 
talk about ethics, but there's this fine line of how we access 
data and the barriers that are put on to try to get the data. 
And so how do we thread that needle of, you know, there's a 
need to have the data and to create trustworthy AI systems, and 
yet there's that balancing act of ethics. Can you dive into 
that a little bit?
    Dr. Isbell. I mean, I do have my opinions about how to 
solve all problems around ethics, which is a very deeply 
difficult question. I think the best way of thinking about it 
is to help people to articulate explicitly what it is that--
what the tradeoffs are and where they want to live in that 
space of tradeoffs. If people can understand the tradeoffs, 
they can make informed decisions. I guarantee you that, first 
off, there's more data out about you out there in the world 
than you have ever imagined and that people know more about you 
than you wish that they did, and that could be a good thing 
because one day, it may save your life. On the other hand, it's 
a lot--it's your privacy, and it's who you are, and people 
shouldn't just be able to get access to that data just because 
they can.
    Mr. Feenstra. Is there any data, though, that you'd say 
that would be beneficial that, you know, you look at and say, 
OK, this is captive that we can't get at that might be helpful 
as we move into trustworthiness and AI?
    Dr. Isbell. I think that that's a conversation that 
involves, as we've been saying all along, all the stakeholders 
who are involved.
    I will add one thing, though, which is, although I think 
that bottom-up thinking is good and it's something that's 
driven us to innovation, it says right there in this chamber 
that, ``Where there is no vision, the people perish.''
    Mr. Feenstra. That's right.
    Dr. Isbell. And the vision has to come from elected 
officials, it has to come from government, and it has to be a 
conversation about where it is we agree we want to go.
    Mr. Feenstra. Yes, I agree. Thank you, very, very good and 
thoughtful words.
    Ms. Singh, very intrigued by what your organization does. 
So if you look at how we build the appropriate safety and 
security into products, do you do you see a role in government? 
Or how do we incentivize going down this path, especially in 
the private sector? I mean, I think the private sector has some 
accountability in going down this path. But do you see anything 
that we can do? You know, we can put parameters, I get that. 
But we also, to me, have to do something to allow people to say 
I want to. Do you have any thoughts on that?
    Ms. Singh. Thank you so much for that question because I 
certainly do have many thoughts on it. But one that I would 
love to reemphasize here is the companies we work with right 
now, they are recognizing the importance of transparency 
reporting and disclosures because that transparency is helping 
them build trust with the consumers and truly get that 
competitive advantage. While one of the reasons that these 
companies are not sharing these transparency reports broadly is 
because they don't know how their competitors or others in the 
market stack up to it.
    Mr. Feenstra. Yes.
    Ms. Singh. So at Credo AI, we are big proponents of you 
know, the government coming up with standards that cannot only 
mandate disclosures, but I think we will--it will propel a 
thoughtful benchmarking across these AI applications.
    Mr. Feenstra. Yes, I mean, that's a great thought, that you 
can be protective in your data, but if we say--if the 
government says, wait a minute, this is universal data that 
everybody could use, that can be a gamechanger a little bit. 
Again, ethics plays a vital role in that. Thank you.
    With that, I am out of time. Thank you.
    Chairwoman Stevens. Yes. And we'll hear from Dr. McNerney 
for 5 minutes of additional questioning.
    Mr. McNerney. Well, good. Now that you're back, I can thank 
you for having this hearing. It's great. And again, I want to 
thank the witnesses.
    Ms. Singh, I feel bad about leaving you out first round, 
but I have two big concerns about AI, and I'll throw the first 
one to you. The first one is--and machine learning, which has 
really overtaken AI--that AI will overtake an increasing number 
of decisionmaking from humans, pushing us more and more into 
irrelevance and sort of dehumanizing us. What can we do to 
prevent that, you know, pushing us aside with the 
decisionmaking capability of AI?
    Ms. Singh. Thank you so much for that question. You know, 
with any disruptive technology, be it AI, we see there are huge 
economic impacts. And we see that in, you know, changes in work 
force, the role that humans will play in the future of work. 
But as we step back and think about it, I think we have a great 
opportunity right now to invest more in education. As Dr. 
Isbell mentioned, I'm excited his son is going to be getting 
educated on AI because I think that's going to be critical. But 
thinking about reskilling and upskilling in this age of AI is 
going to give us a competitive edge.
    Mr. McNerney. So that's a great answer, educate more people 
so that we can utilize the AI in a more productive way than 
letting it make decisions for us. That's basically what you're 
saying, right?
    Ms. Singh. Yes, absolutely.
    Mr. McNerney. Very good. OK. Thank you.
    The next one, I guess I'll go to Dr. Isbell again. AI--one 
of my other concerns about AI is that it's being used to 
monitor humans and our behaviors, our habits, especially either 
in autocratic nations or by businesses that would like to be 
able to influence our decisionmaking in terms of the way we 
spend our money. What do you think is a way to mitigate that 
issue?
    Dr. Isbell. So first off, you're right, that's exactly what 
happened, and it's been happening for a long time. Black Friday 
is a thing that happens because it gets people to buy things, 
right, so this is hardly new. What has happened is computing 
and AI has made it much more efficient and easier to deploy.
    My answer to that--I have two. One is that it's education. 
It's making people aware of what's happening and allowing them 
to make reasonable decisions. The other is that there are 
policies and there are technical mechanisms that we can employ. 
We can encourage people to develop and to deploy that will 
allow them--that will allow people to understand what is being 
happened--what is happening to them. You are in fact being 
studied. You--your data is in fact predicting this behavior, 
and you're doing this. And giving people the tools, not just 
the stuff that they know--the education they learn on their own 
but the technical tools that allow others to monitor the 
monitors, that is a place that has a lot of potential and not 
one that we've invested a great deal into.
    Mr. McNerney. Well, the French postmodernists in the 
1930's--1930's and 1940's were sort of warning us that the 
government would be getting more and more information about us 
and being able to use that information to control our political 
decisionmaking as individuals, and that's sort of what I was 
worried about. And now what we're seeing with social media is 
that these--some of these companies are using information to 
direct people into political bubbles that may advocate violence 
or other sorts of extreme behavior. And I think that's one of 
the issues we have--that I'm having with how do we tamp that 
down? Do you have any recommendations, Mr. Crenshaw, on how we 
could go about doing that?
    Mr. Crenshaw. Well, I think when it comes to anytime we're 
looking at the use of algorithms, we have to look at it from a 
risk-based approach. And I think we also need to realize that 
there are some benefits also to artificial intelligence that 
we've seen. And, you know, one of the things I wanted to note 
is that what we've learned is that the more people know about 
AI, the less scared or concerned they are about it. And I think 
that's why education about artificial intelligence is so 
important. But companies also need to build in ethics and 
ethical decisionmaking into their AI as well, too. And we see 
companies that are leading in this space.
    Mr. McNerney. But it's hard to regulate that. And I'm 
thrilled that we're hearing about companies hiring ethicists, 
but how do we get that as a part of the corporate mindset that, 
you know, we need to do this in the future? So--it's not 
something we can regulate I don't think.
    Mr. Crenshaw. I agree that C-suite needs to be involved. It 
needs to be part of corporate culture is building in ethics 
into artificial intelligence. But at the same time, I think 
with the work we're seeing at agencies like NIST are getting us 
in the right direction toward where we want to be.
    Mr. McNerney. Thank you. I yield back.
    Chairwoman Stevens. Thank you. And with that, I don't 
believe we have any other questions. So we're going to bring 
the hearing to a close. Do we have one more? Oh, did Baird come 
back? OK, hold on. I'm not closing. Where is he? Dr. Baird? 
He's not coming? Well, we got questions for the record, too. 
OK. We're prepared to close. All right. Well, we're prepared to 
close. But honestly, we're not going to close the door on the 
conversation because this has only brought up more questions. 
And in fact, we could probably have a hearing on a couple of 
different subsets that we discussed today. I believe with this 
Committee, and as Mr. Gonzalez who, you know, we have been so 
privileged to work with during his couple of terms here in the 
Congress, mentioned, you know, taking research applications, 
commercializing them, recognizing where our economy filters in.
    We also recognize that we're in a leadership moment, and 
this is--you know, we have been deeply privileged to have Dr. 
McNerney through his tenure, his mighty tenure in the Congress 
on this Committee, and he's so, so dedicated to this Committee, 
but this is a leadership moment for the United States of 
America. And we are going to shape how the world's going to go 
on this. We want to be able to shape how the world's going to 
go, and we've got to be prepared to do some of the deeper work. 
It's not just the question of harm, but it's also the questions 
of, you know, the meta challenges that come before us that are 
somewhat brought on by AI. It's forcing us to be more 
collaborative. It is forcing us to come together in ways that 
we didn't last century.
    I left out that I was working at a digital research lab 
before coming to this body and we did the taxonomy, Mr. 
Crenshaw, on the IoT (Internet of things) jobs, you know, how 
companies are going to have to hire. We did this in partnership 
with Manpower Group and a host of other industry and academic 
partners. Digital ethicists came up. That was one of the job 
profiles we came up with. That was just 5, 6 years ago. And I 
mentioned Turing test, and we were so possessed when I was in 
school by the Turing test, like we thought that was going to be 
the question. And Mr. Sherman sort of got to that in his 
questions, you know, are we worried about replacing humanity? 
No, we are talking about what Mr.--or Dr. Isbell said in his 
testimony, culture, changing culture and how we influence 
culture through the laws we pass in this body.
    And we have been addressing some meta challenges. I didn't 
have the privilege of having Mr. Feenstra here last term, but I 
know we would have been working together on the trade deal, the 
USMCA (United States-Mexico-Canada Agreement). You have unions 
and the Chamber came together to pass USMCA. This time around, 
we passed Inflation Reduction Act. For the first time ever, you 
know, we're dealing with climate. You've got the environmental 
groups and the industry partners, my automakers saying they 
want the same thing.
    So these digital applications, these complex artificial 
intelligence systems that we're putting into place, they're 
asking us to come together. So, Ms. Tabassi, I--you know, we're 
going to come back to you because we just--we think NIST solves 
all of our problems, the mighty agency that can with a little. 
And we're excited about that, and we--and we're going to come 
visit you and we're going to talk about how you're stitching 
together with your risk management what Dr. Isbell said and 
what Ms. Singh is saying. Who's at the table? Who's at the 
table? You know, we solve some problems in ones and twos, and 
then we look at some of the broader challenges. But overall, 
we're wildly optimistic. We're working on the vision, and we're 
excited that we had this time together today. Hopefully, the 
rest of the Congress tunes in on C-SPAN later.
    But with that, we're going to close it. We're going to 
leave the record open for a couple of weeks for additional 
questions for the record, and our witnesses are excused. Thank 
you.
    [Whereupon, at 12:29 p.m., the Subcommittee was adjourned.]

                               Appendix I

                              ----------                              

                   Answers to Post-Hearing Questions

[GRAPHICS NOT AVAILABLE IN TIFF FORMAT]

                              Appendix II

                              ----------                              

                   Additional Material for the Record


           Document submitted by Representative Brad Sherman

[GRAPHICS NOT AVAILABLE IN TIFF FORMAT]

                                 [all]