    [House Hearing, 117 Congress]
    [From the U.S. Government Publishing Office]

                            BEYOND I, ROBOT: ETHICS,
                            ARTIFICIAL INTELLIGENCE,
                              AND THE DIGITAL AGE

    =======================================================================

                                 VIRTUAL HEARING

                                   BEFORE THE

                     TASK FORCE ON ARTIFICIAL INTELLIGENCE

                                     OF THE

                        COMMITTEE ON FINANCIAL SERVICES

                         U.S. HOUSE OF REPRESENTATIVES

                        ONE HUNDRED SEVENTEENTH CONGRESS

                                 FIRST SESSION

                                   __________

                                OCTOBER 13, 2021

                                   __________

           Printed for the use of the Committee on Financial Services

                               Serial No. 117-52

    [GRAPHIC NOT AVAILABLE IN TIFF FORMAT]

                                   __________

                        U.S. GOVERNMENT PUBLISHING OFFICE
    46-195 PDF                 WASHINGTON : 2022

    -----------------------------------------------------------------------------------

                     HOUSE COMMITTEE ON FINANCIAL SERVICES

                     MAXINE WATERS, California, Chairwoman

    CAROLYN B. MALONEY, New York         PATRICK McHENRY, North Carolina,
    NYDIA M. VELAZQUEZ, New York             Ranking Member
    BRAD SHERMAN, California             FRANK D. LUCAS, Oklahoma
    GREGORY W. MEEKS, New York           BILL POSEY, Florida
    DAVID SCOTT, Georgia                 BLAINE LUETKEMEYER, Missouri
    AL GREEN, Texas                      BILL HUIZENGA, Michigan
    EMANUEL CLEAVER, Missouri            ANN WAGNER, Missouri
    ED PERLMUTTER, Colorado              ANDY BARR, Kentucky
    JIM A. HIMES, Connecticut            ROGER WILLIAMS, Texas
    BILL FOSTER, Illinois                FRENCH HILL, Arkansas
    JOYCE BEATTY, Ohio                   TOM EMMER, Minnesota
    JUAN VARGAS, California              LEE M. ZELDIN, New York
    JOSH GOTTHEIMER, New Jersey          BARRY LOUDERMILK, Georgia
    VICENTE GONZALEZ, Texas              ALEXANDER X. MOONEY, West Virginia
    AL LAWSON, Florida                   WARREN DAVIDSON, Ohio
    MICHAEL SAN NICOLAS, Guam            TED BUDD, North Carolina
    CINDY AXNE, Iowa                     DAVID KUSTOFF, Tennessee
    SEAN CASTEN, Illinois                TREY HOLLINGSWORTH, Indiana
    AYANNA PRESSLEY, Massachusetts       ANTHONY GONZALEZ, Ohio
    RITCHIE TORRES, New York             JOHN ROSE, Tennessee
    STEPHEN F. LYNCH, Massachusetts      BRYAN STEIL, Wisconsin
    ALMA ADAMS, North Carolina           LANCE GOODEN, Texas
    RASHIDA TLAIB, Michigan              WILLIAM TIMMONS, South Carolina
    MADELEINE DEAN, Pennsylvania         VAN TAYLOR, Texas
    ALEXANDRIA OCASIO-CORTEZ, New York   PETE SESSIONS, Texas
    JESUS ``CHUY'' GARCIA, Illinois
    SYLVIA GARCIA, Texas
    NIKEMA WILLIAMS, Georgia
    JAKE AUCHINCLOSS, Massachusetts

                       Charla Ouertatani, Staff Director
                     TASK FORCE ON ARTIFICIAL INTELLIGENCE

                        BILL FOSTER, Illinois, Chairman

    BRAD SHERMAN, California             ANTHONY GONZALEZ, Ohio, Ranking
    SEAN CASTEN, Illinois                    Member
    AYANNA PRESSLEY, Massachusetts       BARRY LOUDERMILK, Georgia
    ALMA ADAMS, North Carolina           TED BUDD, North Carolina
    SYLVIA GARCIA, Texas                 TREY HOLLINGSWORTH, Indiana
    JAKE AUCHINCLOSS, Massachusetts      VAN TAYLOR, Texas

                               C O N T E N T S

                                  ----------
                                                                       Page
    Hearing held on:
        October 13, 2021.............................................     1
    Appendix:
        October 13, 2021.............................................    25

                                   WITNESSES
                          Wednesday, October 13, 2021

    Broussard, Meredith, Associate Professor, Arthur L. Carter
      Journalism Institute of New York University....................     4
    Cooper, Aaron, Vice President, Global Policy, BSA--The Software
      Alliance.......................................................    12
    King, Meg, Director, Science and Technology Innovation Program,
      The Wilson Center..............................................     8
    Vogel, Miriam, President and CEO, EqualAI........................     6
    Yong, Jeffery, Principal Advisor, Financial Stability Institute,
      Bank for International Settlements.............................    10

                                    APPENDIX

    Prepared statements:
        Broussard, Meredith..........................................    26
        Cooper, Aaron................................................    33
        King, Meg....................................................    74
        Vogel, Miriam................................................    79
        Yong, Jeffery................................................    88

                  Additional Material Submitted for the Record

    Garcia, Hon. Sylvia:
        Written responses to questions for the record submitted to
          Jeffery Yong...............................................    90

                            BEYOND I, ROBOT: ETHICS,
                            ARTIFICIAL INTELLIGENCE,
                              AND THE DIGITAL AGE

                                  ----------

                          Wednesday, October 13, 2021

                 U.S. House of Representatives,
                 Task Force on Artificial Intelligence,
                               Committee on Financial Services,
                                                       Washington, D.C.
        The task force met, pursuant to notice, at 12 p.m., via
    Webex, Hon. Bill Foster [chairman of the task force] presiding.
        Members present: Representatives Foster, Casten, Pressley,
    Adams, Garcia of Texas, Auchincloss; Gonzalez of Ohio,
    Loudermilk, Budd, and Taylor.
        Chairman Foster. The Task Force on Artificial Intelligence
    will come to order.
        Without objection, the Chair is authorized to declare a
    recess of the task force at any time. Also, without objection,
    members of the full Financial Services Committee who are not
    members of this task force are authorized to participate in
    today's hearing.
        As a reminder, I ask all Members to keep themselves muted
    when they are not being recognized by the Chair. The staff has
    been instructed not to mute Members, except when a Member is
    not being recognized by the Chair and there is inadvertent
    background noise.
        Members are also reminded that they may only participate in
    one remote proceeding at a time. If you are participating
    today, please keep your camera on, and if you choose to attend
    a different remote proceeding, please turn your camera off.
        Today's hearing is entitled, ``Beyond I, Robot: Ethics,
    Artificial Intelligence, and the Digital Age.''
        I now recognize myself for 4 minutes to give an opening
    statement.
        Thank you, everyone, for joining us today at a time when
    the power and perils of artificial intelligence (AI) are very
    much on people's minds. Each generation has its own cautionary
    tales about AI. Recent big-screen adaptations--The Matrix,
    Terminator, and Tron--echo the episodes of the old 1960s Star
    Trek starring William Shatner as Captain James D. Kirk of the
    Starship Enterprise, and those episodes themselves were taken
    from the short stories of Isaac Asimov, Arthur Clarke, and all
    of the old masters of 1950s sci-fi pulp magazines. And
    parenthetically, I should offer our congratulations that today,
    William Shatner was able to boldly go into suborbital near
    space where X-15 pilots have been boldly going since the late
    1950s. But I digress. Asimov's classic, I, Robot, showed us
    what can happen when we deploy technology or AI without fully
    comprehending its consequences.
        There is an ancient joke in AI that I first heard as an
    undergraduate back in the 1970s about an all-powerful AI that
    was given a simple command: Maximize paperclip production. It
    thought about it for a moment and then began killing off all
    humans on Earth because humans interfere with paperclip
    production.
        Now, it may have taken us 50 years, but we are kind of
    there. Facebook's AI was given the simple command, ``maximize
    Facebook's profits,'' whereupon they thought for a moment and
    then began killing off all rational political debate in our
    country because that interferes with Facebook's profits. And
    the situations with social media in Myanmar and around the
    world are even uglier and more deadly.
        In previous hearings of this task force, we have looked at
    the biases and unexpected side effects of using AI in financial
    services and housing. We have also looked at the implications
    of artificial intelligence's voracious appetite for personal
    data and the implications for privacy, at technological
    approaches to maximally preserve privacy while retaining AI's
    effectiveness, and the importance of secure digital identity.
        In this hearing, we are going to take a closer look at the
    frameworks for developing, monitoring, and recognizing AI to
    ensure that the technology we develop and deploy will be of
    overall benefit to society.
        In past hearings, we have examined instances of algorithmic
    bias that have produced discriminatory effects in the lending
    space. We have seen facial recognition technology that is far
    less effective at identifying minorities correctly, despite the
    fact that the developers of these tools did not include a
    discriminatory line of code in their products. So, we clearly
    cannot allow technology to treat humans differently based on
    race and appearance, unless, of course, perhaps we are
    explicitly correcting for past unjust biases, which brings up a
    set of issues that my father struggled with as a civil rights
    lawyer back in the 1950s, and continues with us today. We have
    to understand whether we should hold AI to standards that are
    higher than we would expect of an ordinary human-based
    decision-making process.
        As we start defining frameworks for developing and
    performance-testing AI, it seems possible that we are starting
    to place requirements on AI that are more strict than we would
    ever place on human decision-makers. For example, most of our
    witnesses today have advocated for defining minimum diversity
    standards for the training datasets for AI, but we have never
    considered requiring that a human bank officer would have a
    minimum number of friends of different races or protected
    classes, even though it might arguably result in more fair
    decision-making. And we may already be seeing the positive
    results of holding AI to higher standards than humans with the
    recent reports that fintech apps were apparently more effective
    than human-based banks in issuing Paycheck Protection Program
    (PPP) loans to minority customers.
        As policymakers, we also have to understand to what extent
    we should concentrate on so-called black-box testing that only
    focuses on the inputs and outputs from opaque neural networks
    and other decision-making algorithms, or whether we should
    expect ourselves and the public to receive and to understand a
    detailed explanation of what goes on under the hood. So, there
    is a lot to be examined here. It is my hope that in this
    dialogue, we will discover which frameworks exist and which
    should be created or fleshed out to ensure that AI is working
    effectively and safely for everyone.
        And the Chair will now recognize the ranking member of the
    task force, Mr. Gonzalez of Ohio, for 5 minutes for an opening
    statement.
        Mr. Gonzalez of Ohio. Thank you, Chairman Foster, for your
    leadership on ethics in AI and for convening today's hearing,
    and I also thank our witnesses for being here. It is vital that
    Congress continues to consider how we can best promote
    innovative advancement in the private sector while also
    ensuring that AI is both transparent and ethical. Today's
    hearing provides an opportunity to hear directly from industry
    experts and stakeholders on the importance of this topic.
        A few months ago, the task force held a similar hearing
    examining how human-centered AI can address systemic racism.
    One of our witnesses at that hearing, Professor Rayid Ghani of
    Carnegie Mellon University, testified that algorithms
    themselves are neither inherently biased or unbiased, but work
    by analyzing past data and making generalizations about future
    outcomes. I believe that these discussions on bias and
    algorithms are important to have. We must acknowledge and
    recognize these technologies at times are not perfect due to
    the inherent nature of a technology created by humans. It is
    vital, though, that we do not take steps backwards by
    overregulating this industry, which may have a chilling effect
    on the deployment of these technologies.
        If there are problems with AI and algorithms, we should not
    abandon our push to innovate and move forward. It is through
    further innovation that we are likely going to be able to fix
    these issues and to improve the technology. As Chairman Foster
    recognized, we have seen the benefits in the disbursement of
    PPP loans. I think that is an important thing for us to keep in
    mind as we continue forward.
        We should also continue to work with the experts in
    industry in order to move forward in a bipartisan way that both
    celebrates technical advancements and ensures that there is
    transparency and fairness through the use of artificial
    intelligence. There have been multiple efforts in the
    government and the private sector to address this issue, and we
    have seen tremendous advances not only in AI technology, but in
    efforts to address bias in algorithms internally. There is
    recognition of a business incentive to have transparent
    algorithms that are fair and ethical.
        Beyond the obvious concerns of ethics and transparency, I
    am also looking forward to learning more today from our
    witnesses about ways that we can strengthen data transparency
    for families, and consider reforms that would protect our
    children from being targeted by harmful algorithms. As the
    financial internet and the traditional internet merge--and we
    have seen recently-reported social media companies, like
    TikTok, employing algorithms that promote inappropriate content
    to young users--I think it is extremely troubling and extremely
    timely that we start to discuss these things. An AI-powered
    world where parents have no control over what content or
    products are being fed to their kids, no transparency around
    the algorithms that are funneling the content, and no control
    over the underlying data itself is not an ideal outcome.
        In summary, AI has great promise to innovate industries
    like the financial services sector, but there are still
    opportunities to improve. I look forward to hearing from our
    witnesses today how Congress should be thinking about this
    balance, and I yield back.
        Chairman Foster. Thank you.
        The Chair will now recognize the Chair of the full
    Financial Services Committee, the gentlewoman from California,
    Chairwoman Waters, for 1 minute.
        [No response.]
        Chairman Foster. It is my understanding that she is not
    able to make it right now, so we will move on.
        Today, we welcome the testimony of our distinguished
    witnesses: Ms. Meredith Broussard, an associate professor at
    the Arthur L. Carter Journalism Institute of New York
    University; Ms. Miriam Vogel, the president and CEO of EqualAI;
    Ms. Meg King, the director of the Science and Technology
    Innovation Program at the Wilson Center; Mr. Jeffery Yong,
    principal advisor at the Financial Stability Institute of the
    Bank for International Settlements; and Mr. Aaron Cooper, the
    vice president for global policy at BSA--The Software Alliance.
        Witnesses are reminded that their oral testimony will be
    limited to 5 minutes. You should be able to see a timer on your
    screen which indicates how much time you have left. I would ask
    you to be mindful of the timer, and quickly wrap up your
    testimony once the time has expired, so that we can be
    respectful of both the witnesses' and the members' time.
        And without objection, your written statements will be made
    a part of the record.
        Ms. Broussard, you are now recognized for 5 minutes to give
    an oral presentation of your testimony.

    STATEMENT OF MEREDITH BROUSSARD, ASSOCIATE PROFESSOR, ARTHUR L.
           CARTER JOURNALISM INSTITUTE OF NEW YORK UNIVERSITY

        Ms. Broussard. Thank you. Chairman Foster, members of the
    task force, thank you for hosting this important hearing and
    for giving me the opportunity to testify. My name is Meredith
    Broussard. I am a professor at NYU, the research director at
    the NYU Alliance for Public Interest Technology, and author of
    the book, ``Artificial Unintelligence: How Computers
    Misunderstand the World.'' In my written testimony, I explore a
    practical vision for recognizing AI, and in my short time, I'll
    talk about AI generally as well as discrimination algorithmic
    auditing and regulatory sandboxes.
        The first thing I want to say is that AI is not what we see
    in Hollywood. There is no robot apocalypse coming. There is no
    singularity. We do not need to prepare for artificial general
    intelligence because these things are imaginary. What is real
    is that AI is math, very complicated and beautiful math.
    Machine learning, the most popular kind of AI, is a poorly-
    chosen term because it suggests that there is a brain or
    sentience inside the computer. There is not. When we do machine
    learning, we take a large set of historical data and instruct
    the computer to create a model based on patterns and values in
    that dataset. The model can then be used to predict or make
    decisions based on past data. The more data you put in, the
    more precise your predictions will become. However, all
    historical datasets have bias. For example, if you feed in data
    on who has gotten a mortgage in the past in the United States
    and ask the computer to make similar decisions in the future,
    you will get an AI that offers mortgages to more White people
    than people of color.
        AI needs to be regulated because it has all of the flaws of
    any human process, plus some. My own regulatory vision begins
    with frameworks, high-level governance models that guide a
    company's use of AI and data. A company can make sure its
    frameworks are implemented by performing regular algorithmic
    audits, ideally using a regulatory sandbox. The process could
    be monitored by regulators using tools we already have, namely
    compliance processes inside existing regulatory agencies.
    Agencies and companies might decide which AIs need to be
    regulated and monitored by looking at the user and the context.
    Automated license plate readers used at toll booths might be a
    reasonable use of AI. Automated license plate readers used by
    police as dragnet surveillance might be an unreasonable use of
    AI.
        An open secret in the AI world is everyone knows that these
    systems discriminate. Any conversation about a robot apocalypse
    is a deliberate distraction from the harms that AI systems are
    causing today. Right now, AI is preventing people from getting
    mortgages. A recent investigation by The Markup found that
    nationally, loan applicants of color were 40 to 80 percent more
    likely to be turned down by mortgage approval algorithms as
    compared to their White counterparts.
        When the International Baccalaureate used AI to assign
    student grades during the pandemic, high-achieving, low-income
    students received terrible grades, which prevented them from
    getting college credits that would allow them to graduate early
    and incur less student loan debt.
        AI is used to generate secret predictive consumer scores,
    like health risk scores or identity and fraud scores. It is
    likely that Black, Indigenous, and People of Color (BIPOC)
    people are systematically disadvantaged by most of these
    scoring systems. The EU's proposed AI regulation calls for
    categorizing AI into high and low risk, which I think is a good
    strategy. A low-risk use might be using facial recognition to
    unlock your phone. A high-risk use might be the police using
    facial recognition on real-time surveillance video feeds.
    Facial recognition has been shown to consistently misidentify
    people with darker skin; people of color are at a high risk of
    being harmed by facial recognition when it is used in policing.
    In the U.S., we can register and audit high-risk AI to ensure
    that AI is not harming citizens.
        The process for uncovering algorithmic bias is called
    algorithmic auditing. ORCAA, a company I consult with, performs
    bespoke algorithmic audits in context, asking how an algorithm
    might fail and for whom. Audits can show how an algorithm might
    be racist, or sexist, or ableist, or might discriminate
    illegally. Once we identify a problem, it can be addressed, or
    the algorithm can be discarded. There is also software like
    Parity, or Aequitas, or AI Fairness 360, that can evaluate
    algorithms for 1 of 21 known kinds of mathematical fairness.
        I'm enthusiastic about the potential of a regulatory
    sandbox, a protected environment where companies can test their
    algorithms for bias. If and when the bias is discovered, they
    can then address the issue in their code and rerun the test
    until they're in compliance with acceptable thresholds. I'm
    currently working with ORCAA to develop a regulatory sandbox
    prototype. In our version, regulators would also have a limited
    view inside the sandbox to see if the company is auditing their
    algorithms for bias and fixing the problems that they find
    without the companies revealing any trade secrets.
        Thank you for the opportunity to testify today on this
    important topic, and I welcome your questions.
        [The prepared statement of Ms. Broussard can be found on
    page 26 of the appendix.]
        Chairman Foster. Thank you, Ms. Broussard, and I have to
    say I am fascinated with the thought of figuring out for which
    of the 21 definitions of fairness you will be advocating.
        Ms. Vogel, you are now recognized for 5 minutes to give an
    oral presentation of your testimony.

         STATEMENT OF MIRIAM VOGEL, PRESIDENT AND CEO, EQUALAI

        Ms. Vogel. Chairman Foster, Ranking Member Gonzalez, and
    distinguished members of the task force, thank you for
    conducting this important hearing and for the opportunity to
    provide this testimony. My name is Miriam Vogel. I'm president
    and CEO of EqualAI, a nonprofit founded to reduce unconscious
    bias in AI systems. At EqualAI, we are AI net positive. We
    believe AI is and will be a powerful tool to advance our lives,
    economy, and opportunities to thrive, but only if we're
    vigilant to ensure that the AI we use does not perpetuate and
    mass produce historical and new forms of bias and
    discrimination.
        We're at a critical juncture. AI is increasingly becoming
    an important part of our daily lives, but decades of progress
    made and lives lost to promote equal opportunity can be
    unwritten in a few lines of code. And the perpetrators of this
    disparity may not even realize the harm they're causing. For
    instance, we can see our country's long history of housing
    discrimination now replicated at scale in mortgage approval
    algorithms that determine creditworthiness using proxies for
    race and class.
        At EqualAI, we try to help avoid such harms by supporting
    three main stakeholders: companies; policymakers; and lawyers.
    Often, our work involves helping organizations understand they
    are effectively AI companies because they are now using AI in
    pivotal functions. As such, they need an AI governance plan,
    particularly given that with AI, as you know, key assessments
    occur behind the proverbial black box where inputs and
    operations are generally unknown to the end user.
        As discussed in your past hearings, implicit bias
    infiltrates AI in a variety of ways. Our operating thesis is
    that bias can embed in each of the human touch points
    throughout the AI lifecycle, from the ideation phase deciding
    what the problem is you even want to solve with AI, to the
    design, data collection, development, testing, and monitoring
    phases. But we are optimistic and we think each touch point is
    also an opportunity to identify and eliminate harmful biases.
    As such, risk management should occur at each stage of the AI
    lifecycle.
        There are several helpful frameworks to identify and reduce
    harms in the AI systems, including GAO's, GSA's, and the
    important efforts under way at NIST. The EqualAI framework
    offers five pillars to consider when establishing responsible
    governance, including, first, invest in the pipeline. Our basis
    tenet is that AI needs to be created by and for a broader
    cross-section of our population. There are several
    organizations promoting diversity in tech effectively right
    now--AINU, AI4All, and several others--and we need to support
    these efforts.
        Second, hire and promote people with your values. To create
    and sustain a diverse workplace and produce better AI, AI
    programs used in H.R. functions should be checked routinely to
    ensure they're in sync with the values of your organization and
    our country.
        Third, evaluate your data. The more we know about datasets,
    the safer we are as a society. We encourage identifying gaps in
    data so that they can be rectified and, at a minimum, clarified
    for end users.
        Fourth, test your AI. AI should be checked for bias on a
    routine basis. As you know, AI constantly iterates and learns
    new patterns as it is fed new data. On our website,
    EqualAI.org, we offer a checklist to help get you started, and
    we offer additional steps to take in our written testimony. We
    highly recommend as well the use of routine audits.
        And fifth, redefine the team. An often-overlooked
    opportunity to reduce bias in AI is by creating testing teams
    that include those underrepresented in the AI creation and the
    underlying datasets.
        There are numerous ways that Congress can play a key role
    in ensuring more effective, inclusive AI. Several are listed in
    our testimony. A few include, one, Congress can reinforce the
    applicability of laws prohibiting discrimination to AI-
    supported determinations. Two, Congress can lead by example,
    create a framework for AI procurement, acquisition, and
    development and ask vendors if they do the same.
        Three, incentivize investment in the future of work. Like
    all transformative technologies, AI will eliminate jobs, but it
    will also open up opportunities. To lead in the AI revolution,
    safeguard our economy, and support greater prosperity among
    more communities, we should re-skill our workforce by
    understanding what jobs are likely to emerge, and offering
    incentives for upscaling and loan forgiveness for those
    committing to a term in public service.
        Finally, we enthusiastically support the bill of rights put
    forward by the White House Office of Science and Technology
    Policy last week to level-set expectations and inform the
    public about their rights.
        In conclusion, we believe we're at a critical juncture to
    ensure that AI is built by and for a broader cross-section of
    our population. It's not only the right thing to do; a strong
    U.S. economy and our leadership depend on it. Thank you for the
    opportunity to testify, and I look forward to your questions.
        [The prepared statement of Ms. Vogel can be found on page
    79 of the appendix.]
        Chairman Foster. Thank you, Ms. Vogel, and I echo your
    enthusiasm for the White House's effort to come up with an AI
    bill of rights, though I don't believe I have seen even a draft
    of it at this point.
        Ms. King, you are now recognized for 5 minutes to give an
    oral presentation of your testimony.

        STATEMENT OF MEG KING, DIRECTOR, SCIENCE AND TECHNOLOGY
                 INNOVATION PROGRAM, THE WILSON CENTER

        Ms. King. Thank you, Chairman Foster, Ranking Member
    Gonzalez, and members of the AI Task Force for inviting me to
    testify today. My name is Meg King. I'm the director of the
    Science and Technology Innovation Program at the Wilson Center,
    a nonpartisan think tank created by Congress nearly 60 years
    ago. My program studies the policy opportunities and challenges
    of emerging technologies, and investigates methods to foster
    more open science and to build serious games. We also offer
    hands-on training programs, called the Technology Labs, to
    Legislative and Executive Branch staff on a variety of issues,
    including artificial intelligence. Next month, we will offer a
    series of individual trainings on AI for Members as well.
        As with any technological evolution, the benefits of AI
    come with associated costs and risks. Focusing only on the
    benefits misses the nuances of the potentials and pitfalls of
    this advance. To help the task force understand the risks to
    any industry and, in particular, the financial services
    industry, I will focus my remarks on the nature of AI
    generally, to understand the environment in which creation is
    occurring.
        Today, there aren't significant incentives for the private
    sector to include ethics directly in the development process.
    At the current pace of advancement, companies cannot afford to
    develop slowly, or a competitor might be able to bring a
    similar product to market faster. Largely due to consumer trust
    concerns, international organizations, regions, and private
    companies have all begun to issue ethical frameworks for AI.
    Most are very vague principles, as you mentioned, Chairman
    Foster, with little guidance as to application.
        Two that this committee should pay close attention to are
    the Organisation for Economic Co-operation and Development
    (OECD), and the European Commission (EC). In addition to their
    principles on AI, the OECD is developing process and technical
    guidelines ranging from pinpointing new research to making
    available software advances which will become part of a
    publicly-available interactive tool for developers and
    policymakers alike. As Ms. Broussard noted, European regulators
    announced a risk-based plan this year to establish transparency
    requirements, including biometric identification and chatbots.
    Chatbots, in particular, are expected to have a significant
    impact on the financial services industry as many companies see
    value in customer service process improvement and the prospect
    of gaining more insight into customer needs in order to sell
    more financial products.
        As regulators ask developers more questions about the
    ethics of their AI systems, they have the potential to slow the
    process, which could cost businesses money. However, if ethical
    concerns are identified too late in the development process,
    companies could face considerable financial loss if not
    addressed properly. No ethical AI framework should be static,
    as AI systems will continue to evolve, as will our interaction
    with them. Key components, however, should be consistent, and
    that, specifically for the financial sector, should include
    explainability, data inputs, testing, and system life cycle.
    Explainable Artificial Intelligence (XAI) is the method to ask
    questions about the outcomes of AI systems and how they achieve
    them. It helps developers and policymakers identify problems
    and failures, possible sources of bias, and helps users access
    explanations. There are a number of techniques available to
    carry out XAI, as well as open source tools, which make these
    techniques more accessible.
        In the financial sector, XAI will become critical as
    predictive models increasingly perform calculations during live
    transactions, for example, to evaluate risk or the opportunity
    of offering a financial product or specific transaction to a
    customer. Establishing a clear process for XAI will be critical
    to address flaws identified in these real-time systems and
    should be an area of focus for the committee.
        Additionally, producing policies on how these systems will
    be used and in what context will be helpful. Without context,
    data pulled from a mix of public/private records can produce
    inaccurate results and discriminate in access to financial
    products. One of the near-term questions this committee should
    ask about systems you will encounter in your oversight is how
    the COVID-19 pandemic experience is factored into these
    systems. One promising possibility to address the data input
    problem might be to synthesize artificial financial data to
    correct for inaccurate or biased historical data. Just today, a
    major tech company announced acquisition of a synthetic data
    startup. Watch this space.
        While quality assurance is part of most development
    processes, there are currently no enforceable standards for
    testing AI systems, and, therefore, testing is uneven at best.
    Additionally, users are far removed from AI system developers.
    Carefully assessing the growing field of Machine Learning
    Operations Tools (MLOps) and machine learning operations and
    identifying ways the committee can participate in that process
    will be useful.
        AI breaks, often in unpredictable ways, at unpredictable
    times. Participants in the Wilson Center's AI Lab have seen AI
    function spectacularly using a deep learning language model to
    produce the first-ever AI-drafted legislation, as well as fail
    when a particular image loaded into a publicly-available
    generative adversarial network produced a distorted picture of
    a monster rather than a human. Lab learners also study why
    accuracy levels matter, as they use a toy supply chain
    optimization model to predict whether and why a package will
    arrive on time and how to improve the prediction by changing
    the variables used, such as product weight and length of
    purchase.
        Beyond mistakes, some AI systems carry out tasks in a way
    humans never would. Many examples exist of scenarios producing
    results developers didn't intend, like a vacuum cleaner
    injecting collected dust so it can collect even more, and a
    racing boat in a digital game looping in place to collect
    points instead of winning the race. Anyone who has played the
    game, ``20 Questions'' understands this problem. Unless you ask
    exactly the right question, you won't get the right answer.
        As more and more AI systems are built and distributed
    widely with varying levels of user expertise, this problem will
    continue. Establishing a framework of ethics for the
    development, distribution, and deployment of AI systems will
    help spot potential problems and provide more trust in them.
    Thank you.
        [The prepared statement of Ms. King can be found on page 74
    of the appendix.]
        Chairman Foster. Thank you, Ms. King.
        Mr. Yong, you are now recognized for 5 minutes to give an
    oral presentation of your testimony.

        STATEMENT OF JEFFERY YONG, PRINCIPAL ADVISOR, FINANCIAL
        STABILITY INSTITUTE, BANK FOR INTERNATIONAL SETTLEMENTS

        Mr. Yong. Thank you. Good afternoon, Chairman Foster,
    Ranking Member Gonzalez, and distinguished members of the task
    force. My name is Jeffery Yong, and I'm the principal advisor
    at the Financial Stability Institute of the Bank for
    International Settlements, or the BIS. I offer my remarks today
    entirely in my personal capacity based on a publication that I
    co-authored with my colleague, Jermy Prenio, entitled, ``FSI
    Insights No. 35: Humans keeping AI in check--emerging
    regulatory expectations in the financial sector.'' And the
    views expressed in that paper are our own and do not
    necessarily represent those of the BIS, its members, or the
    Basel ommittees. I'm appearing before the task force
    voluntarily. I would like to note that my statements here today
    are similarly my personal views, and they do not represent the
    official views of the BIS, its members, or the Basel
    Committees.
        By way of background, the Financial Stability Institute
    (FSI) is a unit within the BIS with a mandate to support
    implementation of global regulatory standards and sound
    supervisory practices by central banks and financial sectors,
    supervisory and regulatory authorities worldwide. One of the
    ways the FSI carries out this mandate is through its policy
    implementation work which involves publishing FSI Insights
    papers. The papers aim to contribute to international
    discussions on a range of contemporary, regulatory, and
    supervisory policy issues and implementation challenges faced
    by financial sector authorities.
        In preparing FSI Insight No. 35, my co-author and I found
    that regulatory expectations on the use of AI in financial
    services were at a nascent stage. Accordingly, we drafted a
    paper with four key objectives: to identify emerging common
    financial regulatory themes around AI governance; to assess how
    similar or different these common regulatory themes are viewed
    in the context of AI vis-a-vis that of traditional financial
    models; to explore how existing international financial
    regulatory standards may be applied in the context of AI
    governance; and to examine challenges in implementing the
    common regulatory themes.
        To this end, we can select a section of policy documents on
    AI governance issued by financial authorities or groups formed
    by them as well as other cross-industry AI governance guidance
    that applies to the financial sector. In total, we examined 19
    policy documents issued by 16 regional and national authorities
    and 2 international organizations. Most of these documents are
    either discussion papers or high-level principles, which
    underscores the fact that financial regulatory thinking in this
    area is at a very early stage.
        We identified five common themes that recur in policy
    documents that we examined: reliability; accountability;
    transparency; fairness; and ethics.
        On the theme of reliability, emerging supervisory
    expectations for AI and traditional models appear to be
    similar. What seems to be different is that the reliability of
    AI models is viewed from the perspective of avoiding harm to
    data subjects or consumers, for example, through
    discrimination.
        On the theme of accountability, it is acknowledged that
    both traditional and AI models require human intervention. In
    the case of AI, however, this requirement is motivated by the
    need to make sure that decisions based on AI models do not
    result in unfair or unethical outcomes. Moreover, external
    accountability is emphasized in the case of an AI model, so
    that data subjects are aware of AI-driven decisions and have
    channels for recourse and moving on transparency.
        Supervisory expectations related to explainability and
    auditability are similar for AI and traditional models.
    However, expectations or external disclosure are unique to AI
    models. This refers to expectations that firms using AI models
    should make data subjects aware of AI-driven decisions that
    impact them, including how their data is being used.
        On the theme of fairness, there's a distinct and strong
    emphasis in emerging supervisory expectations on this aspect in
    the case of AI models. Fairness is commonly described in the
    documents as avoiding discriminatory outcomes.
        Similarly, on ethics, as a distinct and strong emphasis on
    this aspect of AI models, ethics expectations are broader than
    fairness, and relate to ascertaining that consumers will not be
    exploited or harmed.
        Now, given the similarities of the themes between AI and
    traditional models, existing financial literacy standards that
    govern the use of traditional models may be applied in the
    context of AI. However, there may be scope to do more in
    defining financial regulatory expectations related to fairness
    and ethics. The use of AI in the financial sector presents
    certain challenges, and the key challenge relates to the level
    of complexity and lack of explainability. Given these
    challenges, one way to approach this is to consider a tailored
    and coordinated regulatory policy approach, meaning
    differentiating potential and conduct treatment depending on
    the risk that the AI models pose.
        With that, I conclude. Thank you.
        [The prepared statement of Mr. Yong can be found on page 88
    of the appendix.]
        Chairman Foster. Thank you, Mr. Yong.
        Mr. Cooper, you are now recognized for 5 minutes to give an
    oral presentation of your testimony.

    STATEMENT OF AARON COOPER, VICE PRESIDENT, GLOBAL POLICY, BSA--
                         THE SOFTWARE ALLIANCE

        Mr. Cooper. Thank you very much. Good afternoon, Chairman
    Foster, Ranking Member Gonzalez, and members of the AI Task
    Force. My name is Aaron Cooper. I'm vice president of global
    policy for BSA-The Software Alliance. BSA is the leading
    advocate for the global enterprise software industry. Our
    members are at the forefront of developing cutting-edge, data-
    driven services that have a significant impact on U.S. job
    creation. I commend the task force for convening today's
    important hearing, and I thank you for the opportunity to
    testify.
        Enterprise software services, including AI, are
    accelerating digital transformation in every sector of the
    economy, and BSA members are on the leading edge, providing
    businesses with the trusted tools they need to leverage the
    benefits of AI. In fact, last year, software supported more
    than 12.5 million jobs outside the tech sector. AI is not just
    about robots, self-driving vehicles, or social media. It's used
    by businesses of all sizes to improve their competitiveness.
    It's the power and industrial design that improves
    manufacturing performance and reduces environmental impact.
    It's the tool that streamlines transportation and logistics
    operations, and that detects cyberattacks and improves H.R.
    operations. In the financial services industry, AI is being
    used to reduce the risk of fraudulent transactions and deliver
    a better customer relations experience.
        While the adoption of AI can unquestionably be a force for
    good, it can also create real risks if not developed and
    deployed responsibly. We commend the task force for its work to
    explore domestic and international AI frameworks because they
    play a critical role in ensuring the responsible use of AI.
        As you explore these issues, we offer our perspective on a
    risk management approach to bias which has been a particular
    focus for BSA, and that we hope will also inform the broader
    conversation. For BSA members, earning trust and confidence in
    AI and other software services they develop is crucial, so
    confronting the risk of bias is a priority. We, therefore, set
    out to develop concrete steps companies can take to guard
    against this. The resulting framework is included in full in my
    written testimony. It is built on three key elements: impact
    assessments; risk mitigation practices; and organizational
    accountability modeled on NIST frameworks, which includes more
    than 50 actionable diagnostic statements for performing impact
    assessments that identify risks of bias and corresponding best
    practices for mitigating those risks.
        Among the unique features of the BSA framework is that it
    recognizes that these steps need to be followed at all stages
    of the AI life cycle: design; development; and deploymentt.
    Also, different businesses will have different roles throughout
    the life cycle, so risk management responsibilities will need
    to be tailored to a company's role. Who's developing the
    algorithm? Who's collecting the data, training the model, and
    ultimately deploying the system? What does that all mean in
    practice?
        A few examples. First, when designing an AI system,
    companies should clearly define the intended use and what the
    system is optimized to predict, identify who may be impacted,
    and, if the risk of bias is present, document efforts to
    mitigate that risk. They should examine data that will be used
    to train the model to ensure that it's representative and not
    tainted by historical biases.
        Second, at the development stage, they should document
    choices made in selecting features for the model and document
    how the model was tested.
        Third, at the deployment phase, they should document the
    process for monitoring the data and model and maintain a
    feedback mechanism to enable consumers to report concerns.
        And to be clear, at every phase, it is important for
    companies to have a team that brings diverse perspectives and
    background, which can help anticipate the needs and concerns of
    people who may be affected by AI in order to identify potential
    sources of bias. Bias is only one of the important ethical
    considerations for responsible AI, but addressing it is
    critical. And the risk management approach we recommend in this
    context can be tailored to address other ethical
    considerations.
        In conclusion, digital transformation across industry
    sectors is creating jobs and improving our lives, but industry,
    civil society, and academia must work together with Congress
    and other policymakers on guidelines and laws which will ensure
    that companies act responsibly in how they develop and deploy
    AI. We appreciate the task force's strong focus on these issues
    and hope that our framework on confronting bias will contribute
    meaningfully to this discussion. Thank you for the opportunity
    to testify, and I look forward to your questions.
        [The prepared statement of Mr. Cooper can be found on page
    33 of the appendix.]
        Chairman Foster. Thank you, Mr. Cooper.
        I will now recognize myself for 5 minutes for questions.
        My first general question is to Ms. King or Mr. Cooper,
    whomever wants to field it. How much should we expect of AI,
    and, in particular, should we be asking more of AI than we do
    of humans? For AI-driven cars, should the standard be that you
    should outperform humans on average or in all circumstances?
    With similar things regarding fairness as well, in general, is
    it reasonable? Are there real dangers in using human-based
    decision-making as the standard of fairness and safety for what
    is acceptable in AI?
        Mr. Cooper. I am happy to jump in. I will give one example
    and a way of thinking about this, that things which are illegal
    in the physical world should be illegal in the digital world
    when we use AI or any other system. In the realm of
    discrimination, for instance, a practice that would be
    discriminatory if a person did it, should still be
    discriminatory and illegal if an AI system does it.
        And I think what we are finding in other areas is that AI
    is increasingly being used, both in everyday features of what
    companies are doing as they go through a digital transformation
    but will also increasingly be used in more high-risk areas. And
    in those situations, we need to make sure that there is a
    proper impact assessment so we know, whether it is related to
    bias or safety or another issue, that companies are thinking
    through what those implications are going to be and taking
    steps to mitigate the risks.
        Ms. Vogel. I am happy to answer if you would like, as well.
    I think the answer is, honestly, we don't know. Certain systems
    are designed very narrowly right now, and that is because AI
    outperforms humans in those systems. But in others, with
    context, with heuristics, the shortcuts that we use as humans
    don't perform well. And as one of the Wilson Center machine-
    learning researchers who has written a paper about it reminds
    us regularly, autonomous agents optimize the room, floor, and
    function that we give them. So, until we can improve AI to a
    level where we feel comfortable that moves beyond that narrow
    capability, I don't think we have an answer yet about how to
    think about the consequences, but also the opportunities. They
    are just so varied across so many sectors at this point.
        Chairman Foster. Are there any other comments on the
    deployment decision that has to be made here? You need some
    sort of absolute standard that this is good enough for this
    application, and it is something we are going to have to pace
    because that is probably, at best, the level at which Congress
    will be specific about how these decisions should be set up.
        Another thing that I know we all struggle with is this
    question of black-box testing versus expecting that the public
    should have a detailed understanding of what goes on inside. If
    you look at the trouble that we have had trying to convince
    people to get vaccinated, it is not clear that it helps to tell
    them the details of how the immune system in the human works.
    And, that may make it better. It may make it worse. We had this
    situation very recently where we apparently fired a football
    coach, not, to my knowledge, for mistreating athletes, but for
    what went on in his private decision-making.
        Should we accept or reject algorithms based only on their
    inputs and outputs, or should we actually demand to look inside
    at all of the intermediate levels of the neural network and see
    if there are objectionable racist nodes in them? What is your
    thinking on that, the black box versus detail, and also how to
    convey that to the public? Anyone? Should I just pick someone
    at random?
        Ms. King. I am happy to jump in.
        Chairman Foster. Okay.
        Ms. King. I think it is a great question, and I think that
    the challenge also includes that even if you show the general
    public all of the nodes, it wouldn't necessarily make sense. In
    this case, you wouldn't know which are prioritized, so there is
    a balance to strike. There are intellectual property issues,
    privacy issues, and so forth. So, just opening the box, first
    of all, would be somewhat technically challenging as well as
    legally. Compliance testing, as you say, can be a helpful way
    to demonstrate compliance, safety, and legality. And to the
    extent that more data becomes available, we don't need to
    expect everyone in the general public to understand it. We have
    seen so many cases already where the limited publicly-available
    data has been used for important findings, like with the
    UnitedHealth Care Optum case, where scientists, researchers
    were able to go backwards, look at the algorithms, and identify
    biases in the algorithms.
        Chairman Foster. Thank you. And when you figure all this
    stuff out, let us know.
        The Chair will now recognize the ranking member of the task
    force, Representative Gonzalez of Ohio, for 5 minutes.
        Mr. Gonzalez of Ohio. Thank you, Chairman Foster, and thank
    you to our witnesses. Ms. Vogel, I am going to try to pick up
    where Chairman Foster just left off on compliance testing. Is
    it fair to say, based on the response you gave, that the right
    way to think about this is more to look at the outputs of AI as
    opposed to opening up the hood and trying to understand each
    individual node and network? Is that the right way to think
    about it?
        Ms. Vogel. My view is that it should be a balance. I think
    that, absolutely, the outputs are indicative. They are helpful
    to look at now because so much of the AI is already deployed,
    and so we are not at the design stages for so much of the AI in
    common use, and for that understanding, what the outputs are is
    important and helpful. I think there are elements of what is
    under the hood that would be helpful and important to
    understand, particularly when you are talking about AI used in
    a pivotal sensitive function. So, I don't think it is one or
    the other. I do think it is a balance, but no matter what I
    think, the outputs are very important to be testing and
    watching.
        Mr. Gonzalez of Ohio. No, I appreciate that. I think, as I
    mentioned in my opening statement, it is encouraging that we
    are seeing some AI algorithms produce better results,
    significantly better results in some instances, from a bias
    standpoint. And obviously, the hope is to understand what it is
    that they are doing right and doing more of that or making that
    more transparent, and then helping foster a more collaborative
    innovation environment.
        Ms. King, I want to shift to you, and I want to ask about
    transparency in AI algorithms. Also, as I mentioned in my
    opening statement, the use of algorithms in social media has
    had a detrimental effect on young users, which, as a parent, I
    find extremely problematic. Do you think that more can be done
    to ensure parents have additional transparency about their
    child's data being collected by these apps or their own, and
    how can we strike the right balance and the right line between
    encouraging innovation, managing problematic algorithms, and
    providing data sovereignty to users?
        Ms. King. Thank you, sir. As a parent as well, that is the
    one thing that terrifies me, is my children getting access to
    these capabilities. And unfortunately, I wish there was one
    significant answer that could fix it, but it is going to be a
    constant ever-moving group of things that we have to do. And
    explainability is significant in that problem because, as
    Miriam just said, we have to understand what the outputs are,
    but we have to understand enough about how we are getting there
    to be able to make informed decisions about whether there is
    too much data that is being collected or whether there isn't,
    and there are many ways to do this. One of the most popular is
    this local and interpretable model agnostic explanation. This
    was created by the University of Washington to try to see what
    happens inside, so model agnostic. It should be across models.
    That is one of many ways to do that.
        Another piece to this is, as you just mentioned, that AI
    can be positive and there are some impressive advances
    happening right now in synthetic data that can both hopefully
    correct for some of those historical data biases, but also give
    just a better picture of the people who are going to be
    impacted by the system being created. Now, of course, you have
    to understand what that synthetic data looks like, so you
    probably should have a wide group of interdisciplinary experts
    assessing that to make sure you are not missing something. But
    I think it is a combination of constantly reviewing the
    outcomes, constantly trying to take at least a sample of
    explainability across some of the most important, as Europeans
    are suggesting, high-risk models, and then also assessing kind
    of what are the new technical capabilities we are developing
    now that can help address this problem.
        Mr. Gonzalez of Ohio. Thank you. Shifting to Mr. Cooper for
    a second with my final minute, I want to ask you about BSA's AI
    risk management framework included in your testimony. One
    aspect that seems to be of importance is that a one-size-fits-
    all framework will not work for small companies and startups. I
    completely agree. Could you elaborate on why flexibility in any
    framework is important for fostering innovation?
        Mr. Cooper. Sure. Thank you very much. I think it is
    important to have flexibility in a variety of ways of achieving
    a desired outcome for a number of reasons, including that not
    all systems are going to be used for the same purposes. The
    algorithm and the data that is used to determine what shows our
    kids watch or what videos our kids watch online is one form of
    algorithm and one use of AI. But there is also database
    management, and customer relations management tools, and
    farmers who use AI in order to improve crop yield, and one set
    of regulations across-the-board isn't going to be able to be
    flexible enough to address the range of different use cases for
    AI.
        Mr. Gonzalez of Ohio. Yes, thank you. And I also think it
    is almost always the case that the higher the regulatory
    burden, the more you entrench incumbents, and the less
    innovation you have at the startup level as the regulatory
    burden is just too high to even contemplate a startup. So with
    that, I thank the witnesses, and I yield back.
        Chairman Foster. Thank you. The Chair now recognizes Ms.
    Pressley of Massachusetts for 5 minutes.
        Ms. Pressley. Thank you, Chairman Foster, for convening
    this important hearing, and to our witnesses for joining us
    here today. Certainly, systemic racial discrimination is
    widespread in the financial services industry. The damage of
    redlining, banking deserts, and employment discrimination has
    never been fully redressed or repaired in America, and all of
    the data supports those facts. Today, mortgage lenders deny
    Black applicants at a rate 80 percent higher than White
    applicants, and payday lenders continue to target low-income
    people of color, charging 500-percent interest even in the
    midst of a pandemic. Many believe artificial intelligence
    presents an opportunity to make the allocation of credit and
    risk fairer and more inclusive. However, AI technology and
    machine learning can easily go in the other direction,
    exacerbating existing bias, and reinforcing bias credit
    allocation, and making discrimination in lending even harder to
    prove.
        Cases of racial bias in AI are well-documented and have
    impacted everything from mortgage loans and tenant screening to
    student loans. The deciding factor between whether the
    technology has a positive or damaging impact could be its
    developers.
        Ms. Broussard, who is writing the algorithms that are being
    used to make important financial decisions, like
    creditworthiness? Do the teams writing these algorithms
    generally reflect the diversity of people in America?
        Ms. Broussard. Generally, these teams do not represent the
    diversity of people in America. Silicon Valley and its
    developers tend to be very pale, male, and Yale. Compared to
    overall private industry, the EEOC found that the high-tech
    sector employed a larger share of Whites, Asian Americans, and
    men, and a smaller share of African Americans, Hispanics, and
    women.
        Ms. Pressley. Thank you. In fact, in February 2020, the
    Financial Services Committee released a report on the diversity
    of America's largest banks, which found that banks were largely
    undiversified at all levels and departments. Those data points
    you offered there support that.
        Ms. Broussard, one more question, will this lack of
    diversity affect AI used by financial institutions? What is the
    impact?
        Ms. Broussard. Absolutely, yes, there is an impact. The
    problem is that people tend to embed their unconscious biases
    in the technology that they create. When we have a small and
    homogeneous group of people creating AI, that AI then gets the
    collective blind spots of the community of people who are
    creating the algorithms. So, the more diversity you have in the
    room when you are creating algorithms, the better the algorithm
    is going to be for the wide variety of people who live in
    America.
        Ms. Pressley. Thank you, Ms. Broussard. And just to further
    unpack the impact of that on people's lives, there are many
    different facets that AI companies developing these
    technologies really need to consider, from, as we are speaking
    to here, who is developing the algorithms to the AI's impact on
    job loss. A recent report from the World Economic Forum
    predicted that by 2025, the next wave of automation amplified
    by the pandemic will disrupt 85 million jobs globally.
        Ms. Vogel, what role should independent auditors play in
    helping to assess the human cost and the ethical implications
    of AI technologies so that both developers and the public can
    fully understand the ethical impacts these technologies have
    for actual consumers?
        Ms. Vogel. Thank you for that question. It is a really
    important point. We do have this growing body of experts--in
    fact, we have one on this very panel--who do this important
    work of checking in, of taking the temperature and
    understanding where these gaps are. I think it is really
    important that we build our reliance and our infrastructure to
    support more algorithmic auditing because these are the people
    who will tell us if the AI doing what we expect it to. Are we
    discriminating? Are we creating opportunity? For whom will this
    fail, and how do we create more opportunity through our
    algorithms?
        Ms. Pressley. Thank you. I agree. Frequent and independent
    audits are critical. AI-supported recommendations in the
    financial services industry directly impact people's lives and
    economic opportunities, and yet the algorithms used are trained
    on data that is rife with imbalance and discrimination. So as
    we do the work deliberately to enact long-overdue economic
    justice, we can't allow the AI industry to create new problems
    and to compound these already persistent and deeply-embedded
    inequities. Thank you, and I yield back.
        Chairman Foster. Thank you.
        The Chair now recognizes Mr. Loudermilk from Georgia for 5
    minutes.
        Mr. Loudermilk. Thank you, Mr. Chairman. One important
    thing to keep in mind as we discuss AI is the types of bias we
    need to eliminate and the types of bias that we actually want
    to keep. Sometimes, the main purpose of an algorithm is to be
    biased. For example, in loan underwriting, algorithms are
    generally used to distinguish between who can pay back a loan
    and who is not able to pay back a loan. With that in mind, we
    must work toward eliminating the types of bias that have no
    place in our financial system, such as the bias based on race
    or gender or any other factor like that.
        One important way of doing that is when an algorithm is
    being built, there should be a thorough record-keeping of
    everything that is added to the algorithm. That way, if bias is
    suspected, companies and regulators can see everything that
    went into the algorithm and see where the bias may be coming
    from. I think this would help make it where algorithms are not
    a black box and where the outputs cannot be explained, but you
    would have a record where you could see where the problems may
    be.
        Mr. Cooper, your organization's framework for AI best
    practices recommends maintaining records of the data that is
    used to train AI models. I agree with that. Expanding on that,
    do you believe that maintaining thorough records of all of the
    inputs used to build an algorithm can be useful for identifying
    the source of any unwanted bias?
        Mr. Cooper. Thank you very much, Congressman. Yes, I think
    it goes even beyond what the data is. I think that there is a
    whole set of considerations that companies need to go through
    to figure out whether there is a high risk that the AI system,
    as it is intended to be deployed or as it is being deployed,
    may have consequential impacts on people. And the decision-
    making about what those risks are and what the right mitigation
    practices are, how the data was tested, what historical biases
    may or may not be present in them, keeping a record of that as
    part of a risk management framework, can be both useful in
    order to make sure that companies are not putting systems out
    into the world or using systems that are going to lead to
    discriminatory results. But it could also be useful, as you
    say, after the fact, if there is a problem, to go back and
    audit and find out why it happened and make sure it doesn't
    happen again.
        Chairman Foster. Representative Loudermilk, I believe you
    are muted.
        Mr. Loudermilk. I don't know why it is cutting off like
    that. Can you hear me now?
        Chairman Foster. Yes, we can.
        Mr. Loudermilk. Okay.
        Chairman Foster. And feel free to exceed your time by 40
    seconds.
        Mr. Loudermilk. Thank you. Mr. Yong, in your testimony, you
    discuss the importance of accountability and transparency in
    AI. Can maintaining records when algorithms are being built
    help achieve those goals?
        Mr. Yong. Yes. Accountability is very important, especially
    when it comes to AI, and without record-keeping, there is no
    transparency. In our testimony, we mentioned that transparency
    is a prerequisite to enabling financial institutions to meet
    the other general AI governance principles. And if the AI model
    is not transparent, then it is very difficult to assess whether
    it is reliable, whether it is sound, and whether there is bias
    involved. So definitely, record-keeping is a prerequisite to
    meeting this accountability and general principles.
        Mr. Loudermilk. Thank you for that. Ms. King, you have
    written that policymakers must govern AI in a way that is
    flexible enough to adapt when technology inevitably changes,
    wand we know that it continually changes in today's
    environment. I agree with that, and I believe that is needed to
    have an environment that fosters innovation. With that in mind,
    how can policymakers ensure that AI governance remains
    flexible, but robust, at the same time?
        Ms. King. Thank you, sir, for that question, and I think it
    is all about having a set of goals that are measurable and
    achievable. One of them is, how can you explain these systems,
    and, again, the complexity here is really because these systems
    cross so many sectors. Yours obviously is financial services,
    so you have some very specific use cases to identify, which is
    helpful, but you need to be able to explain those specific use
    cases. You need to ask a lot of questions, and those questions
    will change, too, but the big ones are why was it developed.
    What are the [inaudible]? How does it possibly fail because it
    is the unexpected failure that is really a lot of the problem
    here.
        And then again, how can we correct those errors and report
    them? So if you can kind of have those four ways of addressing
    this challenge and work with companies, and you work with both
    governments who are buying this and the companies who are
    producing this to have sort of the four methods of regularly
    checking that you are getting, you are producing what you want,
    you are getting what you want out of it, and that it is not
    discriminatory, then I think that is a flexible way to move
    forward.
        And I think the sandbox concept that Ms. Broussard has
    suggested is also very helpful, because while records are great
    and it is easy for us to say, let's keep records here, if you
    have ever taken a look at the code behind some of these systems
    and how often it changes as you shift weights, it gets pretty
    complicated pretty quickly. So, the more you can have these
    kinds of places where companies and organizations can feel safe
    testing is going to be critical going forward well.
        Mr. Loudermilk. Thank you. And, Mr. Chairman, I yield back
    the balance of my time.
        Chairman Foster. Thank you,.
        The Chair now recognizes Representative Casten for 5
    minutes.
        Mr. Casten. Thank you so much. I think Mr. Loudermilk
    really hit it on the head with the transparency question, and I
    want to follow on that, but I want to specifically get to the
    auditability issue, and I think you alluded to this, Ms. King.
    It is one thing to be able to see the code. It is something
    else completely to be able to understand the code. And I say
    this as someone who, before I came to Congress, ran a utility.
    And my biggest risk was predicting revenue--did it vary with
    the weather, did it vary with economic conditions--and I built
    a genetic algorithm that figured all that stuff out. I have no
    idea how it worked, but it was amazingly effective and it made
    our investors much happier because we could predict our
    revenue.
        That is trivial. It is not at all implausible for me to
    imagine that we get to a world where an investment fund has
    figured out from looking at global data that there is about to
    be a massive human rights abuse committed, and it is shorting
    the affected businesses and properties, right? That would be
    deeply unethical, and if we understood it, it would be a
    problem, but it is totally possible that we could never
    actually understand that and saying that is what it is doing.
        So my question is, and I think all of you can answer this,
    but I am going to start with Ms. King, just because I see you
    nodding so vociferously, what is the best regulatory practice
    for ensuring that these algorithms remain auditable, and
    ensuring that they apply to everyone in the system, because
    presumably, as soon as some subset of people agree to have
    auditable algorithms, people who violate that might have an
    investing edge, whether that is a bad actor in our country or a
    foreign actor who wishes us harm. What is that standard, both
    domestically and internationally? How would you recommend we
    think about that?
        Ms. King. Thank you, sir. I will take a quick stab at it,
    and I am going to use a hypothetical because I think it is
    always helpful to have it. Yours is very complicated, and I am
    not going to try and explain your very impressive example. At
    the Wilson Center, in one of our trainings, we use a supply
    chain prediction model. Will it or won't it arrive? Will the
    USPS deliver a package on time? And you have a series of
    variables. You have product wait. You have the month that it
    was ordered. You have things that you probably, as a consumer,
    wouldn't think matter, but about 10 different variables. And as
    you play with the model and you change the variables, you
    change the weights--how much weight do we give to a particular
    variable or not--you understand more why the prediction you get
    comes out, and then you can kind of take that and you can go
    back through the system and check it.
        I would say you need a couple of standards. One is not
    going to work, unfortunately, but a couple of standards that
    have that sort of ability to use a couple of methods, probably
    a model agnostic method, if it is possible, to go back and just
    understand, at least at a strategic level. You may not be able,
    as you know very well, to go and explain the whole thing, but a
    confidence level and then explainability that you could
    achieve. So, you are looking for some sort of confidence trust
    level and some sort of agnostic model verification, and you are
    also looking to make sure as you are going through that
    process, that if you are going to have regulators as part of
    this conversation, you have a number of regulators across
    sectors. As your example points out, you can't just have
    financial services regulators. You are going to have to have
    others from other parts of the government at the table because
    of these unexpected outcomes.
        Mr. Casten. If I could, though, that approach you described
    works where there is a finite number of known inputs. If you
    are using sort of a neural network model that, for all
    practical purposes, has an infinite number of inputs, I don't
    know how you audit that at some level of complexity. To follow
    on from that, and I am sure this varies market to market, is
    there any good analysis? Is there a percentage of algorithmic
    trading or algorithmic investing, wherever it sits, that we
    really don't want to have more than X percent because now the
    algorithms are responding to algorithms? Is there a robust
    mathematical way to think about that? And maybe it is different
    for housing credit decisions than it is for equities
    investments or something else. But is there some robust way to
    think about that so that we don't sort of unwittingly introduce
    too much volatility into the system? And if any of the other
    witnesses want to chime in on this, you would be welcomed for
    your thoughts as well.
        Ms. Vogel. I can speak to auditing algorithms. What we want
    to do is, we don't want to think about auditing all algorithms
    to the same standard. We want to think about auditing
    algorithms in context because the context matters a lot. So, we
    do need to keep track of inputs. We do need explainability. We
    do need to enforce real-world laws inside algorithms. We do you
    need to be aware of bias in, bias out. And so to your point
    about thresholds, the acceptable thresholds would be determined
    based on the context.
        Mr. Casten. I see I am out of time, but I would welcome
    further thoughts offline from any of the witnesses, and I yield
    back.
        Chairman Foster. Thank you.
        The Chair now recognizes Ms. Adams from North Carolina for
    5 minutes.
        Ms. Adams. Thank you, Chairman Foster. And thank you,
    Ranking Member Gonzalez and Chairwoman Waters, for this hearing
    today. And to our witnesses, thank you for your testimony.
        Professor Broussard, in your testimony, you noted that all
    historical data sets have bias, and that AI needs to be
    regulated as soon as possible because it has all of the flaws
    of any human process plus more. You also cite in your testimony
    the potential impact of bias in AI to students and consumers of
    lower socioeconomic status, such as when the International
    Baccalaureate used AI to assign grades to students, to their
    detriment. So, building off of what my colleague, Ms. Pressley,
    discussed, would you tell us more about what happened in these
    scenarios?
        Ms. Broussard. Sure. Thank you for that question. The
    International Baccalaureate (IB) example is a situation where,
    because of the pandemic, the International Baccalaureate exams
    were canceled, and the IB decided to use an algorithm to assign
    imaginary grades to real students, which had disastrous
    consequences, because the inputs to the algorithm were things
    like a school's performance in the past. We know that the
    economic divide is particularly profound when it comes to
    America's schools, and so the students at the poor schools were
    predicted to do poorly, and the students at the rich schools
    were predicted to do well. We have a racial divide there. Who
    are the students at poor schools? They are mostly Black and
    Brown students. Who are the students at rich schools? Well,
    they are mostly White students. So, the algorithm made very
    predictable decisions that disadvantaged Black and Brown and
    poor students. This is what happens most of the time with
    algorithmic decisions.
        Ms. Adams. Would you explain what algorithmic auditing is,
    and how we can encourage public and private entities to adapt
    it as a best practice?
        Ms. Broussard. Thank you. Yes. Algorithmic auditing, as I
    mentioned in my testimony, is something that I do with a
    company called O'Neil Risk Consulting and Algorithmic Auditing,
    Inc. (ORCAA). What we do is we look at an algorithm and we ask,
    who could this algorithm negatively affect, and we look at the
    inputs to the algorithm. We do look at the code. We act as an
    information fiduciary, so we keep everything extremely private.
    We look at the outputs and we do mathematical and statistical
    analysis as necessary in order to figure out what is going on
    in the algorithm. Once you actually figure out where the
    algorithm is going wrong, you can fix it, but in a lot of
    industries now, people are pretending that there is nothing
    wrong. For example, Ms. Vogel mentioned before the Optum case.
    There is also the case of the Apple card, where a man was
    offered a credit limit that was about 10 times higher than his
    wife, even though they shared all of their finances.
        Companies are pretending that they don't collect
    information like race in order to make decisions. But, on the
    other hand, if you are using a factor, like a ZIP Code, that is
    an input to your algorithm, then, actually we have enough
    residential segregation in the United States that if you are
    using a ZIP Code, you are actually using race as a proxy. So,
    there are--
        Ms. Adams. Thank you.
        Ms. Broussard. Thank you.
        Ms. Adams. I want to move on, if I can, quickly.
        Ms. Broussard. Sure.
        Ms. Adams. Ms. Vogel, I was happy to see that part of your
    recommendations related to diversifying the AI field, including
    supporting Historically Black Colleges and Universities
    (HBCUs). Specifically, what shoud Congress be doing to ensure
    that HBCU and Minority Serving Institution (MSI) students are
    able to participate in the AI revolution that is currently
    underway?
        Ms. Vogel. Thank you for that question. We strongly believe
    that we need AI to be built by and for a broader cross-section
    of the population, both so that more can benefit from the AI,
    so that more can benefit from the economic support that comes
    from it, but also so that our AI is better. So, we need to make
    sure that we support HBCUs and MSIs to ensure that their
    students are part of this current AI revolution that is
    undeway. We know that HBCUs produce nearly 20 percent of all
    Black graduates, 25 percent of Black graduates who earned
    degrees in the disciplines of STEM technology, science,
    engineering, and math, and we need to make sure that we have
    all hands on deck. We can't afford to not bring all of these
    students into the AI revolution. Industry is depending on their
    participation.
        Ms. Adams. Thank you, ma'am. I think I am out of time. Mr.
    Chairman, I am going to yield back, but thank you very much for
    your response.
        Chairman Foster. Thank you.
        The Chair now recognizes Mr. Auchincloss of Massachusetts
    for 5 minutes.
        Mr. Auchincloss. Thank you, Mr. Chairman. I would like to
    talk about two specific applications of algorithms that have
    been and are front and center these days, and really invite the
    panel to weigh in on one or both of them. The first is the use
    of algorithms in hiring. A number of organizations, some from
    the center-left, some from the left, and some from the center-
    right, have all converged that there are somewhere between 25
    to 30 million ``hidden workers'' in the United States, people
    who could be employed, who, under the right conditions, want to
    be employed. And yet, we are not tapping into their
    productivity, and they are not getting to realize their fullest
    aspirations.
        That is obviously a multifaceted problem, but one element
    of it is algorithms that some of the biggest companies are
    using, something like 75 to 80 percent of Fortune 100s, for
    example, in how they sort resumes that get put forward. They
    are screening out resumes that have discontinuity in
    employment. They are screening out resumes of formerly-
    incarcerated individuals. They are screening out resumes that
    don't have a college degree, even for jobs that don't require a
    college degree. I welcome input from the panel on this first
    application, kind of the state of play right now in these
    resume-screening algorithms, and what can be done to improve
    them, and whether they have any role at all going forward?
        Ms. Broussard. I can offer that my colleague, Hilke
    Schellmann, has been writing about these topics, and has done
    some really excellent work in the MIT Technology Review, that
    is an in-depth review on what is going on with hiring
    algorithms.
        Mr. Cooper. Yes. This is one of the reasons why we need a
    risk management framework for when there is going to be an AI
    system that has a highly-consequential impact on somebody's
    lifec, so making sure that there is a thought process that is
    auditable about what the factors are that are considered in
    determining what resume is going to go where is a good example
    of something where we need to make sure that there is a
    thought-through and documented impact assessment, and then
    steps taken to mitigate risk.
        Mr. Auchincloss. And I would just add also that this should
    be a triple-line win for everybody. Companies don't want to be
    screening out high-quality workers for esoteric reasons. They
    are struggling for employees, as we speak. And as a society, we
    want people to be working and contributing, and people
    themselves want meaningful work. So, I would hope that this can
    be an area of actual bipartisan work going forward on how we
    encourage the private sector to be more thoughtful about their
    use of these hiring algorithms and the other elements of this
    challenge of hidden workers.
        The second area that I want to dig into and invite the
    panel to speak on is about what many of us have been reading
    about these last 2 weeks, which is Facebook's algorithm. The
    whistleblower addressing the Senate exposited that while she
    did not think Section 230 should be revised for user-generated
    content, she did think that Facebook's algorithm itself should
    be subject to liability laws. And I would welcome input from
    any of the panelists here about how that might be applicable in
    terms of Facebook, in particular, but really any social media's
    algorithm, whether that should be subject to regulation itself?
        Mr. Cooper. I am happy to jump in again. We don't represent
    Facebook, but I would say that I think it is important to make
    sure that where you have particular high risk in the way an
    algorithm is working--in this case, feeding certain videos or
    certain social media feeds to certain people, particularly
    where it has to do with children--is a high risk, and we need
    to make sure that the decision-making process is appropriate.
    And there is a combination of a regulatory aspect of that and
    also just good practices internally to make sure that there is
    organizational accountability so that when decisions are made,
    that there is somebody at a senior level who signs off on those
    decisions, and that there is documentation of why certain
    choices were made.
        Mr. Auchincloss. Yes. I can see why it would be challenging
    to try to unpick liability for an algorithm that was put into
    place, how you can draw causality directly, and yet part of me
    thinks that we have to answer that question. We have to wrestle
    with that problem because, otherwise, we are going to be in a
    place where I think organizations will be distancing themselves
    from accountability instead of embracing it by being able to
    point towards these black box algorithms and say that they are
    just part of part of their toolkit, and you can never pay cause
    and effect. I think we need to reject that explanation and hold
    companies liable for the algorithms that they choose to use.
        I yield back my time, Mr. Chairman.
        Chairman Foster. Thank you, and I would like to thank our
    witnesses for their testimony today.
        The Chair notes that some Members may have additional
    questions for these witnesses, which they may wish to submit in
    writing. Without objection, the hearing record will remain open
    for 5 legislative days for Members to submit written questions
    to these witnesses and to place their responses in the record.
    Also, without objection, Members will have 5 legislative days
    to submit extraneous materials to the Chair for inclusion in
    the record.
        This hearing is now adjourned.
        [Whereupon, at 1:16 p.m., the hearing was adjourned.]

                                A P P E N D I X

                                October 13, 2021

    [GRAPHIC NOT AVAILABLE IN TIFF FORMAT]

                                   [all]