    [Senate Hearing 118-643]
    [From the U.S. Government Publishing Office]

                                                            S. Hrg. 118-643

                        ARTIFICIAL INTELLIGENCE AND HOUSING:
                            EXPLORING PROMISE AND PERIL

    =======================================================================

                                    HEARING

                                   before the

                                SUBCOMMITTEE ON
               HOUSING, TRANSPORTATION, AND COMMUNITY DEVELOPMENT

                                     of the

                                  COMMITTEE ON
                       BANKING,HOUSING,AND URBAN AFFAIRS
                              UNITED STATES SENATE

                        ONE HUNDRED EIGHTEENTH CONGRESS

                                 SECOND SESSION

                                       ON

                 EXAMINING ARTIFICIAL INTELLIGENCE AND HOUSING

                                   __________

                                JANUARY 31, 2024

                                   __________

      Printed for the use of the Committee on Banking, Housing, and Urban
                                    Affairs

                      [GRAPHIC NOT AVAILABLE IN TIFF FORMAT]

                    Available at: https: //www.govinfo.gov /

                                   ______

                     U.S. GOVERNMENT PUBLISHING OFFICE

    60-366 PDF                WASHINGTON : 2026

                COMMITTEE ON BANKING, HOUSING, AND URBAN AFFAIRS

                           SHERROD BROWN, Ohio, Chair

    JACK REED, Rhode Island              TIM SCOTT, South Carolina
    ROBERT MENENDEZ, New Jersey          MIKE CRAPO, Idaho
    JON TESTER, Montana                  MIKE ROUNDS, South Dakota
    MARK R. WARNER, Virginia             THOM TILLIS, North Carolina
    ELIZABETH WARREN, Massachusetts      JOHN KENNEDY, Louisiana
    CHRIS VAN HOLLEN, Maryland           BILL HAGERTY, Tennessee
    CATHERINE CORTEZ MASTO, Nevada       CYNTHIA M. LUMMIS, Wyoming
    TINA SMITH, Minnesota                J.D. VANCE, Ohio
    RAPHAEL G. WARNOCK, Georgia          KATIE BOYD BRITT, Alabama
    JOHN FETTERMAN, Pennsylvania         KEVIN CRAMER, North Dakota
    LAPHONZA R. BUTLER, California       STEVE DAINES, Montana

                         Laura Swanson, Staff Director

                   Lila Nieves-Lee, Republican Staff Director

                           Elisha Tuku, Chief Counsel

                      Amber Beck, Republican Chief Counsel

                          Cameron Ricker, Chief Clerk

                          Shelvin Simmons, IT Director

                           Pat Lally, Assistant Clerk

                                     ______

       Subcommittee on Housing, Transportation, and Community Development

                          TINA SMITH, Minnesota, Chair

                   CYNTHIA M. LUMMIS, Wyoming, Ranking Member

    JACK REED, Rhode Island              MIKE CRAPO, Idaho
    ROBERT MENENDEZ, New Jersey          MIKE ROUNDS, South Dakota
    JON TESTER, Montana                  JOHN KENNEDY, Louisiana
    CATHERINE CORTEZ MASTO, Nevada       BILL HAGERTY, Tennessee
    KYRSTEN SINEMA, Arizona              J.D. VANCE, Ohio
    RAPHAEL G. WARNOCK, Georgia          KATIE BOYD BRITT, Alabama
    JOHN FETTERMAN, Pennsylvania

                    Tim Everett, Subcommittee Staff Director

            Kelsey Pristach, Republican Subcommittee Staff Director

                                      (ii)

                                C O N T E N T S

                                  ----------

                          WEDNESDAY, JANUARY 31, 2024

                                                                       Page

    Opening statement of Chair Smith.................................     1
            Prepared statement.......................................    23

    Opening statements, comments, or prepared statements of:

    Senator Lummis...................................................     3

                                   WITNESSES

    Lisa Rice, President and Chief Executive Officer, National Fair
      Housing Alliance...............................................     5
        Prepared statement...........................................    25
        Responses to written questions of:
            Chair Smith..............................................    55
            Senator Warnock..........................................    55
    Vanessa Perry, Interim Dean and Professor, The George Washington
      University School of Business; Nonresident Fellow, Housing
      Finance Policy Center, Urban Institute.........................     6
        Prepared statement...........................................    46
        Responses to written questions of:
            Chair Smith..............................................    55
            Senator Warnock..........................................    60
    Nicholas Schmidt, Partner and Artificial Intelligence Practice
      Leader, BLDS; Founder and CTO, SolasAI.........................     7
        Prepared statement...........................................    47
        Responses to written questions of:
            Chair Smith..............................................    61
            Senator Warnock..........................................    63

                  Additional Material Supplied for the Record

    Letter submitted by ACU..........................................    68
    Concept Paper submitted by AI....................................    71
    Letter submitted by the Center for AI and Digital Policy.........    88
    Letter submitted by Family Housing Fund..........................    94
    Letter submitted by Urban Institute..............................    97
    Letter submitted by NMHC/NAA.....................................   160
    ``To Err Is Automated: Have Technological Advances in the
      Mortgage Market Increased Opportunities for Black
      Homeownership?'' Working Paper.................................   163
    Letter submitted by Zillow.......................................   185

                                     (iii)

                        ARTIFICIAL INTELLIGENCE AND HOUSING:
                            EXPLORING PROMISE AND PERIL

                                  ----------

                          WEDNESDAY, JANUARY 31, 2024

                                   U.S. Senate,
      Committee on Banking, Housing, and Urban Affairs,
        Subcommittee on Housing, Transportation, and Community
                                                   Development,
                                                        Washington, DC.
        The Subcommittee met at 10:03 a.m., in room SD-538, Dirksen
    Senate Office Building, Hon. Tina Smith, Chair of the
    Subcommittee, presiding.

                 OPENING STATEMENT OF CHAIR TINA SMITH

        Chair Smith. Good morning. The Subcommittee on Housing,
    Transportation, and Community Development will come to order.
    Today's hearing will focus on both the promise and the threats
    that artificial intelligence poses in the housing sector, and
    I'm very much looking forward to our witness' testimony and to
    this conversation.
        I want to thank Ranking Member Lummis and her staff for
    your ongoing bipartisan work as we put together this hearing.
    We both share, I believe, a deep interest in how we can develop
    Federal policy that supports innovation, and expands
    opportunity for everyone to have a safe, decent, affordable
    place to live. And one of the most consequential innovations in
    recent years is artificial intelligence.
        Leader Schumer, Senator Rounds, Senator Young, and Senator
    Heinrich are leading a bipartisan effort to explore the
    impacts, opportunities, and threats that AI poses. And they
    have asked Senate committees to engage in our areas of
    expertise, which leads us to this Committee hearing today,
    examining what AI means for housing.
        So without a safe, decent, affordable place to live,
    nothing in your life works. Not your job, your family, your
    education, your health. So, a foundation question is how AI can
    help or hinder this goal. We know that some aspects of
    artificial intelligence have been around for a long time, and
    we also know that major advances are fueling the use of AI in
    finance and housing in ways that we need to understand.
        Consumers find AI when they encounter chatbots, when they
    shop online, or digital helpers that seem to be ubiquitous. And
    AI plays a role when a prospective tenant is looking to rent an
    apartment, or a renter submits a maintenance request to a
    management company, or a family tries to qualify for a home
    loan, or when a person experiencing homelessness is connected
    to services. These are powerful tools that hold great potential
    to cut costs and to target services, to reduce wait times, and
    to even reduce bias.
        But they also have the potential to bake in existing
    inequities and to reduce accountability and even limit
    opportunity. Today, AI is being used actively in every part of
    the housing continuum, from emergency homelessness services to
    mortgage financing. And as I was preparing for this hearing, I
    found endless applications.
        AI is being deployed, for example, to help connect people
    experiencing homelessness with health and housing resources. AI
    is helping to forecast more precisely and accurately where
    families are at risk of eviction so that we can better target
    assistance. Academics and advocates are using AI and machine
    learning to help understand, and map the country's zoning laws
    and codes, spanning 30,000 different localities.
        And these insights could help us to understand the dense
    and complicated rules that govern where, and how, and what
    types of housing are being built so that we can make better
    decisions about how to boost housing supply and lower costs. So
    there are many opportunities, and there are also some very real
    concerns about the threats that AI poses.
        In Minnesota, some landlords are reportedly using AI-
    generated tenant-screening reports that include incorrect and
    sometimes illegal or off-limits information. The result, it's
    even harder for people to find a place to rent in some
    circumstances, and they may never know why they were declined
    or be able to correct the record. For landlords, maybe it's
    just easier to move on to the next applicant rather than
    consider additional information, another example of how AI used
    in a bad way can be quite harmful.
        There's a lawsuit in Minnesota right now against a law firm
    that has allegedly automated the process of filing evictions
    for landlords. In 1 month, the firm filed 400 eviction
    complaints. These eviction filings lacked much detail about why
    the eviction was happening, and seemed to routinely lack basic
    information about lease terms or to include significant errors
    around lease states, and rental amounts, and payment
    information.
        So the fact that a firm allegedly leaned on AI to generate
    a large number of eviction filings with false information,
    apparently without any meaningful review by an attorney, that's
    a big problem. Not only is the eviction illegal, but that
    eviction will live on in public records and hurt the tenant
    into the future.
        AI is also increasingly part of how people buy homes. It is
    used in credit scoring models and automated valuation models
    which determine the value of a home. How AI is deployed has
    major implications for a person's credit scores, their mortgage
    rates, and whether home ownership and wealth building is even
    within reach.
        We know that we have historic, systemic challenges with
    fairness and equity in this country. My own hometown of
    Minneapolis, has some of the greatest disparities in home
    ownership between Black and White families of anywhere in the
    country. So we need to carefully explore whether AI is
    extending and reinforcing these biases, and how it has the
    potential to correct them.
        Our excellent witnesses here today have an unenviable task
    in your opening statements to ground us in both these
    opportunities and threats in AI and housing in 5 minutes. I
    look forward to hearing from you, and I also look very much
    forward to the questions from my colleagues as we follow-up
    after your testimony.
        With any innovation there are both opportunities and
    challenges that we need to balance, and our job is to think
    about these complex issues so that we can develop the best
    public policy. So I very much look forward to this
    conversation, and I now turn to Senator Lummis for her opening
    statement.

             OPENING STATEMENT OF SENATOR CYNTHIA M. LUMMIS

        Senator Lummis. Well, thank you Madam Chair, and thanks to
    our witnesses for being here today.
        Yesterday, I was on an airplane sitting next to a woman who
    was flying into D.C. so the health care business could have
    their first workshop on AI and health care in some public
    health fields, and how to define AI and use guidelines in a way
    that will make the sideboards between existing technology and
    AI more clear.
        You are way ahead of the game that health care was. And Ms.
    Rice, it's so nice to see you again because you are the one who
    came to my office and explained to me how AI can be used in a
    nefarious way. And before that, I never even would've
    considered that AI could be used in a way that reinforces bad
    behavior, and so I'm so pleased to see you here today. And all
    of you, thank you for coming. So now we're going to explore the
    range of promises and perils that artificial intelligence can
    offer to housing in the United States.
        So there are many potential benefits from the use of AI in
    housing. Incorporating more information into housing decisions
    might expand access to credit. Automated tools may expedite the
    process of approving new housing developments. In rural areas,
    AI can ease the appraisals process and accelerate supply
    chains. Now, I'm the broken record on the Banking Committee for
    my support establishing a regulatory framework for digital
    assets. But I see some parallels where we have a new technology
    rapidly changing in a space, and a need for thoughtful
    consideration of how existing regulations may apply.
        So from today's hearing, I'd like to better understand what
    parts of AI fit in today's regulatory framework and what needs
    to evolve. We've already seen straying into some areas that
    create perils because they're poorly thought out from the
    regulatory aspect.
        This summer, in response to the potential use of AI when
    providing financial services to customers, the Securities and
    Exchange Commission proposed a rule called ``conflicts of
    interest'' in predictive data analysis. It was an overreaching
    rule, and the scope got into even using basic spreadsheets as
    covered technology. So what that did was put unneeded burdens
    on a new use of an old technology.
        So we need rules and regulations governing AI to be scoped
    appropriately, not using it as an excuse to force decades-old
    technology to comply with a new and unnecessary set of
    impractical regulations. So I'm delighted to see you here to
    help us parse the difference. And this is a really qualified
    panel, so this should be a really informative morning.
        Concerns about AI merit discussion. We need a financial
    system that gives families access to building wealth and create
    strong communities through home ownership. Credit decisions
    should be based on who can repay not on the color of their skin
    or the community they call home. We should recognize the risk
    that AI can, if not trained on complete data and within safe
    guards, reinforce bias and discrimination. But aggressive
    action prohibiting innovative AI use is perilous, and we need
    to find that balance.
        Today's hearing presents an opportunity to explore the
    application of AI within the housing space, and I look forward
    to hearing the testimonies of our witnesses. I'm optimistic
    about the prospect of harnessing AI to uplift and expand the
    American housing experience. So let's leverage innovation to
    improve access to housing in Wyoming and all over the United
    States. Thank you for being here. Thank you, Madam Chair.
        Chair Smith. Thank you so much, Senator Lummis. And I also
    am just really grateful that Senator Rounds is here. I know
    that you have done a lot of work in this space across the board
    and your leadership of that bipartisan task force that you and
    Senator Schumer are leading. So thank you very much. I'm glad
    you're here. Thank you.
        So we have three witnesses with us today. I'll introduce
    each of them. Dr. Vanessa Perry is interim dean and professor
    at the George Washington University School of Business and
    nonresident fellow at the Urban Institute's Housing Finance
    Policy Center. Dr. Perry has extensive experience in the use of
    AI in mortgage finance and home ownership.
        She previously held positions at the Department of Housing
    and Urban Development, the Consumer Finance Protection Bureau,
    and Freddie Mac. Dr. Perry earned her Ph.D. from the University
    of North Carolina at Chapel Hill and an MBA from Washington
    University in St. Louis.
        Ms. Lisa Rice is the president and CEO of the National Fair
    Housing Alliance. Ms. Rice leads the Alliance's work on
    eliminating housing discrimination, and addressing the evolving
    role of artificial intelligence in housing. She has provided
    expert testimony to the Banking and Housing Committee, the
    bipartisan Senate AI Forum, and the House Financial Services
    Committee. Ms. Rice previously served as a CEO of the Toledo
    Fair Housing Center and the Northwest Ohio Development Agency.
        And we have Nick Schmidt with us today, who is a partner
    and artificial intelligence practice leader at BLDS, and the
    founder and CTO of SolasAI--did I say that correctly? Close
    enough, he says. BLDS is a consulting firm that specializes in
    statistics and economics.
        Mr. Schmidt's work focuses on algorithmic fairness,
    explainable AI, and model governance practices. He founded
    SolasAI, a compliance-focused AI platform that works to
    identify and mitigate bias in algorithmic decision making. He
    earned his MBA from the University of Chicago's Booth School of
    Business.
        We'll begin with Ms. Rice, and just go down the line. You
    each have 5 minutes for your opening statements, and each of
    you have a clock in front of you which will count down the
    time. And your full written statement will be made a part of
    the record. Ms. Rice, you can begin.

    STATEMENT OF LISA RICE, PRESIDENT AND CHIEF EXECUTIVE OFFICER,
                     NATIONAL FAIR HOUSING ALLIANCE

        Ms. Rice. Chairwoman Smith, Ranking Member Lummis, and
    other distinguished Members of the Senate Subcommittee on
    Housing, Transportation, and Community Development, thank you
    for the opportunity to testify during this hearing on
    artificial intelligence.
        The National Fair Housing Alliance works to eliminate all
    forms of housing and lending discrimination, and ensure equal
    opportunities for all people. We are the trade association for
    over 170 Fair housing organizations throughout the U.S. and its
    territories. We know that technology is the new civil and human
    rights frontier. That is why we are committed to creating
    automated systems that are fair, explainable, and trustworthy.
        It is why we also create and promote policies, frameworks,
    and other tools to advance responsible AI principles.
    Artificial intelligence holds great promise for improving
    systems, democratizing opportunities, lowering costs, and
    increasing productivity. Yet it also holds great dangers for
    perpetuating bias, spreading misinformation, excluding people
    from necessary services, and generating other harms.
        AI is used extensively in the housing and financial
    sectors, including in credit scoring, tenant screening,
    automated underwriting risk-based pricing, dynamic rental,
    pricing marketing, and many other areas. All these processes
    can manifest bias and extensive harm to consumers, communities,
    and our economy. Large language models like ChatGPT can also
    present challenges like data privacy, security bias, and
    accuracy concerns.
        Through our work, we have uncovered discrimination in all
    these systems. For example, our investigation against a major
    insurance company in which we examined a third-party scoring
    system used by the insurer, showed the system was overcharging
    Black consumers at rates that exceeded their commensurate level
    of risk. Our findings in that case are similar to findings by
    researchers at Berkeley that found algorithmic pricing models
    discriminate against Black and Latino consumers by overcharging
    them to the tune of $765 million per year.
        Consumers who live in communities with few mainstream
    banks, like rural areas and communities of color, can often be
    negatively impacted by AI systems that rely heavily on data
    contained in the credit reporting agencies. For example, the
    CFPB reports that consumers in southern rural areas of the U.S.
    disproportionately have lower credit scores because, in part,
    they have lower access to physical and online mainstream
    financial services and products.
        While AI can present serious threats, it can also be used
    for good. For example, NFHA uses AI tools to conduct research
    that is helping expand credit and housing opportunities for
    underserved groups and communities. Just this month, we
    released important research completed in conjunction with
    FairPlay AI, documenting a novel approach for de-biasing
    algorithmic models that simultaneously optimizes for high
    quality model performance.
        Most previous research on fairness techniques have shown a
    tradeoff lower moderate, a model accuracy for increased
    fairness. The novel approach at the center of our research,
    which is based on distribution matching, presents the ability
    to optimize models for both fairness and accuracy.
        In addition to de-biasing systems, AI can be used to detect
    discrimination risks, build systems that can expand access to
    credit, provide financial services to consumers more
    efficiently, optimize privacy, identify barriers to fair and
    affordable housing and zoning ordinances, and other positive
    activities.
        To ensure AI can be optimized for good, Congress must first
    do everything possible to ensure Federal agencies, researchers,
    and NGO's can apply existing laws and standards to the use of
    automated systems. Congress must ensure agencies have
    sufficient resources to tackle the issues, and Congress must
    pass legislation that both supports innovation and protects our
    society to ensure the U.S. can continue to lead the world in
    the development of the responsible AI. Thank you.
        Chair Smith. Thank you very much. Dr. Perry.

      STATEMENT OF VANESSA PERRY, INTERIM DEAN AND PROFESSOR, THE
     GEORGE WASHINGTON UNIVERSITY SCHOOL OF BUSINESS; NONRESIDENT
         FELLOW, HOUSING FINANCE POLICY CENTER, URBAN INSTITUTE

        Ms. Perry. Thank you. Good morning. My name is Vanessa
    Perry. I'm a professor and interim dean at the George
    Washington University School of Business, and a nonresident
    fellow at the Housing Finance Policy Center at the Urban
    Institute focusing on consumers housing and financial markets.
    I want to thank Chairwoman Smith and Ranking Member Lummis for
    inviting me to testify on the impact of artificial
    intelligence, which is being employed increasingly throughout
    the housing and mortgage industry.
        Based on my three decades of research in this field,
    regulation of AI warrants urgent attention. While these models
    can enhance efficiency, they can have unintended impacts on
    fairness and equity. Compared to traditional models, AI relies
    on a wider range of data inputs and more complex combinations
    thereof.
        Although intricate multivariate algorithms have been used
    in the mortgage industry for years, AI models have the
    potential to incorporate new data sources due to their
    complexity. It is difficult, but not impossible for anyone
    other than AI developers to scrutinize and monitor their
    inputs.
        AI is already widespread in the mortgage market. AI digital
    marketing models target prospective home buyers, and
    communications with customers are intermediated by AI chatbots.
    Credit scoring companies and mortgage underwriting systems use
    AI to evaluate credit risk. AI models are used for property
    valuation, loan servicing, and loss mitigation. Because these
    models rely on historical data, there is the potential for them
    to systematize and amplify discrimination and inequality. For
    example, due to the legacy of redlining and segregation, and
    their effects on present day neighborhood conditions and home
    values, why should we expect AI to produce estimates that are
    both accurate and fair? And absent guardrails, how would we
    know if AI models were to incorporate data elements such as GPS
    location that serve as a proxy for race, gender, or ability?
        AI models are not subject to human error, and they enable
    more accurate and consistent decisions. Depending on how they
    are developed, their enhanced capabilities could expand access
    to home ownership for households currently underrepresented in
    the mortgage market. For example, AI can produce faster and
    less subjective estimates than human property appraisals, and
    can devise credit scores for those who lack a traditional
    credit history.
        To address concerns about AI's impact on access to the
    housing finance system for underrepresented communities, my
    colleagues and I have proposed five factors summarizing the
    societal, ethical, legal, and practical issues that should be
    considered in the development and implementation of AI. They
    form a memorable acronym, SCALE.
        First, societal values. Algorithms tell us what factors the
    developers think are important in what order, and to what
    degree. AI models should consider the socioeconomic and
    historical context such as past discrimination, and should
    align with prevailing legal and ethical paradigms, such as
    disparate impact law, individual freedom, privacy, and racial
    equity.
        Next, is contextual integrity. Model inputs should be
    relevant to the mortgage and housing domain, and may differ
    substantively from those used for other or less consequential
    context.
        Then, accuracy. Models should be reliable, error-free,
    unbiased, and representative of all demographic and income
    groups across varying macroeconomic conditions.
        Next, legality. The model and its inputs should not
    incorporate characteristics protected by fair lending laws, or
    generate unjustified disparate impacts based on these
    characteristics.
        Finally, expanded opportunity AI models should
    significantly increase access to credit in addition to offering
    greater cost efficiency or risk assessment benefits. This
    criterion represents a higher bar than existing regulatory
    frameworks, and has perhaps the most promising impact on the
    economy and communities.
        This framework could inform new or expanded regulations
    such as guidance for the use of certain types of data, for
    example, an individual's social media profile for certain
    purposes, such as mortgage lending decisions. If designed to do
    so, AI models can increase access to home ownership and
    eradicate the effects of systemic discrimination while
    increasing accuracy and efficiency in the mortgage value chain.
        We need new laws on the Federal level that utilize the
    scale factors previously enumerated at every stage of the AI
    life cycle. Thank you, again, for the opportunity to testify
    here today, and I look forward to answering your questions.
        Chair Smith. Thank you very much. Now we'll hear from Mr.
    Schmidt.

         STATEMENT OF NICHOLAS SCHMIDT, PARTNER AND ARTIFICIAL
      INTELLIGENCE PRACTICE LEADER, BLDS; FOUNDER AND CTO, SOLASAI

        Mr. Schmidt. Thank you, Chair Smith, Ranking Member Lummis,
    Senator Rounds, and the esteemed Members of the Subcommittee.
    Thank you for this opportunity to share my experience and
    insights into how AI is revolutionizing the housing sector, and
    how we can make it safer, fairer, and more accountable.
        I'm Nicholas Schmidt, the founder and CTO of SolasAI. I'm
    also the practice leader at BLDS. I've spent nearly 25 years
    working to ensure that data, algorithms, and statistics are
    applied fairly. And I've spent the last decade devoted to the
    responsible and ethical application of AI and machine learning,
    and with a significant portion of my work focused on credit and
    housing.
        Examples of AI, including the amazing ability of generative
    AI, and the horror of deep fakes and what they might do to our
    democracy, drive our fear, our fascination speculation. But
    really, as amazing, and as scary as those truly are, the more
    mundane examples of AI in machine learning are just as
    important, especially in housing.
        We must recognize that these technologies are already
    shaping many aspects of the industry and getting distracted by
    those sort of shinier aspects of AI, may take our eyes off the
    ball at a time that is really critical.
        So what we should consider is that while an AI algorithm
    could help enable people to get a home loan for the first time,
    it could also drive a decision to evict someone from their home
    or apartment without sign sufficient justification. Because of
    the former, we want to encourage the adoption of the
    technology, because of the latter, we want to make sure that
    it's only used responsibly. This is only possible through
    effective regulation.
        When considering this, what is really important to remember
    is that these systems are driven by human decisions.
    Unfortunately, this human-centric nature of AI development is
    overlooked, but AI is not self-operating, and claiming so is a
    dangerous notion. It absolves us of the responsibility, and
    also ignores the numerous opportunities we have to shape AI for
    the better.
        Just to illustrate this, let's consider a mortgage
    underwriting algorithm. I'm going to go through some of the
    ways in which a human impacts how that algorithm ultimately
    operates determining what data to use. Should you use something
    like education or purchase history, how do you define
    delinquency? That's going to make a very large impact on people
    who have income insecurity, but are ultimately able to pay
    their mortgages over time. That'll make a big difference over
    whether or not they'll actually get a loan.
        What algorithm should the lender use? That affects not only
    discrimination, fairness, accountability, but also potentially
    the safety and soundness of the lender? No algorithm is
    perfect. We have to make a decision. Is the algorithm good
    enough to put into production? And finally, someone has to
    decide how to use the algorithm. All of these decision points
    offer an opportunity for smart intervention.
        And when considering such regulation that comes out of this
    intervention, we should remember that these technologies are
    not new to housing. We do not need to reinvent the wheel.
    Instead, we should learn from the practices and frameworks to
    create a strong and reasonable regulatory environment.
        In particular, there are three things I'll, I'll mention.
    The Federal Reserve's SR11-7 Policy, the NIST AI Risk
    Management Framework, and SP 1270 guidance and the disparate
    impact and disparate treatment framework from the Fair Housing
    Act, and potentially E-COA--we shall see what the courts say
    about that. But out of these frameworks and practices, I want
    to distill this down to four principles. Fairness,
    transparency, accountability, and materiality.
        Fairness, disparate impact and disparate treatment. And
    regulators should set expectations that bias and discrimination
    should be measured and mitigated in AI systems. But really,
    importantly, fairness is not just about antidiscrimination, and
    it is an issue that everyone here has a self-interest in.
        What is being put into some, not all, but some AI systems
    is not fair to anyone regardless of their race, ethnicity, sex,
    or age. We need to empower regulators to set and enforce
    principles-based standards for dating inclusion.
        Transparency. If a person receives a negative outcome from
    an AI-based decision, they should have the right to understand
    why.
        Accountability. If you are negatively impacted, you should
    have a right to appeal if it's a sufficiently high-stakes
    decision. Additionally, people or entities that deploy AI
    systems irresponsibly should face the appropriate consequences.
        And materiality. Businesses need to understand and make
    risk-based decisions. You can't have, for example, an Excel
    spreadsheet subject to the same regulations and, you know,
    requirements and oversight as a credit underwriting algorithm
    that is scoring millions of people. Doing so, ultimately, is
    ineffective for all of our goals.
        So by considering these four sets of principles, we really
    can develop a balanced regulatory approach. And by fostering
    collaboration among all of the stakeholders, we really do have
    a potential to harness AI's value in housing. With this. I
    really do believe that AI can serve as a force for good. Thank
    you very much for the opportunity to share my insight.
        Chair Smith. Great. Thank you so much to all of our
    panelists. And we'll now begin a round of questioning from
    colleagues. And I will start.
        So each of you, it seems to me, are highlighting both the
    opportunities and also the risks of AI. But I'm struck by how--
    I guess I want to dive in a little bit more into how you--we
    know that AI used in housing could pose significant risks. But
    I'm wondering if you could talk a bit more about what the
    beneficial applications for AI might be.
        Let's say just take for example a goal of figuring out how
    to increase home ownership amongst, traditionally, you know,
    people who've been traditionally unable to buy their own home.
    AI is a tool. How could AI, just as an example, be used as a
    tool to advance that goal? And anybody, I'd love to hear what
    you have to say about that, if it's implemented correctly is
    right.
        Ms. Rice. I can start. As I shared, we have just released
    research this month using AI to help expand access to credit
    for people who are creditworthy and have been kept out of the
    financial mainstream. There are many organizations, and our
    research shows that we can increase fairness and access by
    about 13 percent for underserved groups.
        There are many organizations--well, I take that back, not
    many, but there are some organizations that are working on
    using AI and other forms of technologies to incorporate factors
    that are currently not included in the underwriting systems and
    the credit scoring systems that we utilize today in order to
    determine if people can access credit opportunity.
        So I'll give you an example. Rental housing payment
    information. This is a very viable factor. It is an important
    and highly predictive variable that is not included in the
    underscoring systems that we utilize today. That missing
    factor, that missing information, that missing data could
    really open up the aperture for opportunity, particularly for
    people who live in rural communities.
        As I stated before, the CFPB's research and other research
    shows that people in rural areas have disproportionately lower
    credit scores, not because they're less creditworthy, but
    because they don't have access to the mainstream financial
    services that regularly report positive data to the credit
    repositories. So if you access credit outside the financial
    mainstream, using rental housing payment information can really
    increase access to affordable credit opportunities.
        Chair Smith. So if AI is including sort of better
    information, less biased information, potentially information
    that isn't included in regular decision making, you could
    actually have a net positive impact on the fairness of the
    system.
        Ms. Rice. That's right.
        Chair Smith. Right. Thank you. Let me ask kind of a
    different question. Like, how do we address this so-called
    black box phenomenon in AI where, you know, you have an AI tool
    that includes machine learning, so it is constantly learning
    and changing the way it's, you know, improving supposedly the
    way that it's making decision-making, but it seems to sort of
    spit out an answer at the end.
        And the question is, like, where did that answer come from?
    Could maybe Mr. Schmidt or Dr. Perry, could you just address
    that? How do we actually get accountability when you have that
    kind of a technological tool?
        Mr. Schmidt. So there are two issues here. One is directly
    your question and how do we deal with these incredibly opaque
    algorithms? But then there's another important piece, which is,
    should we even be worrying about this in high-risk situations,
    or should we be using more interpretable--requiring more
    interpretable, more understandable algorithms?
        And the way that this is first sort of surfaced over the
    last 5 or 10 years is been this idea that in order to have
    accuracy, you had to have an opaque black box. By being able to
    incorporate lots of interactions and nonlinear relationships
    and all sorts of difficult math, you would get a more accurate
    measure of, say, credit outcomes.
        What has happened over the recent years is that people have
    realized that you don't need to have these incredibly opaque
    algorithms. You can actually develop ones that are pretty
    understandable. And so I believe that those are going to become
    much more commonly used in these high-rate, high-stakes
    decisions.
        Chair Smith. Interesting. Thank you. Dr. Perry, I'm out of
    time, if there's anything that you'd like to add, I'd love to
    hear from you.
        Ms. Perry. I just wanted to add that because these models
    are dynamic, they change often. The question is, which version
    of a model even would you scrutinize or monitor?
        Chair Smith. Yeah. Thank you. Senator Lummis.
        Senator Lummis. Thank you, Madam Chair. Mr. Schmidt,
    welcome. You have an interesting job and an important one in
    terms of helping us understand the four criteria you mentioned
    in your testimony. With regard to transparency, what level of
    explanation do you think companies and organizations should
    provide to consumers?
        Mr. Schmidt. So I think that there are two principles that,
    that need to apply. One of them is transparency into the
    process that they have undergone, and that will allow people to
    understand whether or not the data that was used was fair as
    well as the process, and whether or not the data was right. So
    were they using some factor, you know, education in a situation
    where education really didn't belong, or maybe they got the
    education of the person wrong. Being able to understand and
    appeal those decisions is very important.
        The other area where this is really important is having
    some sort of understanding of how to improve one's outcome. So
    you got rejected and you got told that, you know, had, I don't
    know, number of delinquencies in the past 60 days. Maybe that
    one's easy to figure out how to rectify, but in these complex
    systems it could be much more difficult than that. And giving
    people a clear framework to move from being rejected to being
    accepted, I think could be a very helpful public policy goal.
        Senator Lummis. Another question, this one about data
    privacy. How do you think companies that use AI models for
    housing should be thinking about data privacy?
        Mr. Schmidt. I would like to preface this by saying that
    I'm not a data privacy expert, but what I have seen in my
    experience, particularly with large lenders and other
    organizations that are really leading the financial system, is
    that data privacy is very high priority, and making sure that
    there are not data leakages as people spend inordinate amounts
    of time on. So I think that that is something that is very well
    recognized and something that the industry is really striving
    toward.
        Senator Lummis. So what is the weakest link here? You know,
    when you named those four criteria, which one is crying in need
    of improvement?
        Mr. Schmidt. The one that keeps me up at night is fairness.
    And this is both from, again, antidiscrimination in the
    traditional sense of the word, as well as fairness for
    everybody in this room. The quality of models being built
    across all industries is low. That is absolutely not universal.
    There are brilliant people doing very brilliant work, but there
    is a lot of stuff going on there that is extremely low quality.
        And what we need is to have the same kind of, or similar,
    model governance standards that were put forward by the Fed and
    the OCC, put into lower tier lending institutions and places
    like the health care industry. Because you are absolutely
    right, that the health, the financial system, financial
    industry, is light years ahead of every other industry. And the
    potential for damage if strong model governance is not put into
    place, astronomical.
        Senator Lummis. Can you help with that? Have you studied
    this enough that you're beginning to see how to break through
    those areas where the data is being used in a very
    unsophisticated way?
        Mr. Schmidt. These are mostly people problems. The problem
    with data scientists and data science is that there are--
    colloquially, I'd say they're drunk on algorithms. They get
    very excited by the math, and they get very excited by the next
    new thing, and they don't take a step away and say, ``What is
    it I'm doing?''
        And so, I think that the first step is making people answer
    questions, making the model builders answer questions. What is
    it I'm putting in here? Why am I putting it in this model? What
    purpose does it serve? And the way that that can happen
    effectively is by requiring effective model governance.
        And if I may, the other thing that is really important is
    having lawyers in the middle of the room. And in the questions,
    I always tell my clients that lawyers are the best data
    scientists because they're the most skeptical and negative
    people in a room, and that's really what is needed in data
    science. So we need lawyers, and compliance, and people like
    all the ones who are laughing in this room, because you are the
    people who are going to question the data scientists and get
    them to start doing better work.
        Senator Lummis. Well, thank you. As a lawyer, I resemble
    that remark.
        [Laughter.]
        Senator Lummis. So thank you very much. I yield back. Thank
    you, Madam Chair.
        Chair Smith. Thank you, Senator Lummis. Senator Menendez.
        Senator Menendez. Thank you, Madam Chair. And as a lawyer,
    I'm not going to speak as a lawyer today, but as a legislator,
    so I can avoid that.
        Because of the way AI learns by studying large sets of
    data, if there is a bias in the underlying data, it can become
    encoded in AI's decision-making process. And when it comes to
    housing, the historical data reflects years of redlining,
    disparities in wealth, access to credit, appraisal bias, among
    others. So Ms. Rice, Dr. Perry, how can we work to ensure that
    AI doesn't stratify or even expand existing disparities in
    housing?
        Ms. Rice. Sure. I could take a first crack at that. There
    are multiple ways that we have to work to make sure that AI
    doesn't magnify existing inequities. So one way is to take
    steps to audit and monitor AI, to examine it preproduction,
    predeployment, and also post-deployment.
        We have developed, actually, what we believe is a state-of-
    the-art framework. It's called purpose process and monitoring.
    We believe that it's a state-of-the-art auditing model
    framework to help ensure that algorithms are fair, and
    equitable, and that they do not pose harm to consumers.
    Unfortunately, we do know, Senator Menendez, that many models
    that have been deployed are exacting harms on consumers. We
    have clear evidence of that. So what the purpose monitoring
    process and monitoring framework does, is it helps model
    developers answer all of the questions that Nick just talked
    about. Right? Is this a good use for whatever problem it is
    that you're trying to solve? Does it need an algorithmic or an
    AI solution? Because there are many problems that that need to
    be solved that don't necessarily need an AI solution.
        It examines the data, it examines the model development,
    and the efficacy and accuracy of that model. And then, it also
    looks at the potential and examines the potential for harms.
    And it identifies, or I should say, compels model developers to
    look at less discriminatory alternatives while they are
    designing the model.
        Models can also change post-deployment, right, their
    effectiveness. Just because a model is effective on day one
    doesn't mean it is going to be still be highly effective and
    accurate on day 200. So you have to continually be auditing and
    monitoring that model to make sure that it is still
    efficacious. Thank you.
        Senator Menendez. Dr. Perry, anything you want to add to
    that?
        Ms. Perry. Yes. I would just like to add that in order to,
    sort of, prevent these models from producing, reproducing
    existing discrimination or past discrimination, the one thing
    to do is to intentionally design models that explicitly remove
    those effects and reduce those effects. And actually, that's
    one of the great promises of these tools; is that they can be
    programmed explicitly to remove the effects of past
    discrimination because they are more capable of identifying
    them than the sort of traditional models that we've relied on
    in this industry for so long.
        Senator Menendez. Yeah. Let me follow up this line of
    questioning. That problem is further magnified by the challenge
    of explainability. That is the model--the results of AI models
    and the steps that led them to where they are can be difficult
    to parse due to the complexity of the algorithms and the
    underlying data. How should we think about issues of
    accountability and oversight when it comes to these AI models
    and the data that they use? And I welcome anybody's.
        Ms. Perry. I'll start. My opinion is to effectively
    monitor, we need to examine, one, the guiding principles that
    are used to create these models, the inputs which data could
    potentially be used and which are prohibited by the models. And
    then finally, the effects; so the entire process with the
    understanding that regulatory agencies may not necessarily have
    the resources to do the volume of monitoring and enforcement
    that we are accustomed to. So we definitely need stronger
    protections and processes in place to make this happen.
        Ms. Rice. And Senator Menendez, I'll just add that a couple
    of weeks ago, we held the first our inaugural responsible AI
    symposium. And I had the privilege of hosting a segment with
    the heads of all of the Federal regulatory agencies. And I
    posed this question to them, and they each said that we
    currently have existing laws on the books that effectively
    govern many aspects of AI development and usage. And they are
    working to make sure that their agencies are adept and can
    enforce and exercise effective oversight over the use of AI,
    particularly in financial services.
        Now, one of the challenges is getting staff that are
    educated and trained on these systems and usages. That is a
    huge lift. I will tell you, when we stood up our responsible AI
    division, we had to comb the globe to find someone to head it
    up. And we did find someone in Canada, not in the United
    States. So we have to really do a good job of making sure that
    our citizens, our students, are well educated in this area.
        The other thing that they mentioned is that they need
    resources. They need effective resources, and they need to be
    using AI tools themselves in order to effectively police this
    area.
        Chair Smith. Thank you very much. Senator Rounds.
        Senator Rounds. Thank you, Madam Chair. And let me just,
    first of all, say thank you for doing this hearing today for
    both of you.
        The AI working group, when we started the program, one of
    the things that we talked about was is we wanted every single
    committee to be able to participate because of the expertise on
    the committees and the individuals who participated in meetings
    like this to share their points of view and their
    recommendations. So I really appreciate your jumping on as
    early as you have. You're probably one of the first
    subcommittees to actually have a significant opportunity to
    talk about a specific area in which AI does make a difference.
        And I also want to say, Ms. Rice, what you've indicated in
    terms of the personnel issue is something that we have to take
    into account. And we're never going to have enough experts on
    this, and so to be able to share those experts is one of the
    reasons why NIST, I think, is going to be very important, being
    a location where we can find individuals from industry who can
    come in and participate and help to bring that expertise to the
    committees, as well as to the organizations responsible for
    overseeing AI development.
        I also wanted to--Mr. Schmidt, I think you would agree with
    me on this, but I want to get your thoughts. I think that a
    company is responsible and has to abide by the Fair Credit
    Reporting Act, the Fair Housing Act, the Equal Credit
    Opportunity Act, regardless of what technology that they're
    using, AI or otherwise. Correct?
        Mr. Schmidt. Correct.
        Senator Rounds. So really, one of the challenges we've got
    is how do we make sure that within an algorithm or within a
    decision-making process, because AI really is simply a very,
    very fast decision-making process that is now available to us,
    but one in which a computer system and a use of formulas
    actually learns from itself, whether or not it was making
    correct decisions and modifies it, as Dr. Perry indicated, can
    modify itself along the way.
        And so what I'm curious about is, I don't think we can be
    afraid of the technology, but I think we've got to be able to
    leverage the best of it while at the same time being able to
    provide the regulators the ability to ask the questions and to
    make modifications or request modifications, should there be
    biases identified. Can you think of any program that we've ever
    made that doesn't have some biases built into it?
        Mr. Schmidt. No.
        Senator Rounds. So really, the challenge for us is how do
    we identify those biases? And in fact, any adult who is
    responsible for making decisions today, is there a human that
    doesn't have biases built into their decision making?
        Mr. Schmidt. I've yet to meet one.
        Senator Rounds. Yeah. So really, the question for us is how
    do we overcome those human attributes and allow a machine to do
    a better job of making decisions that eliminates the biases
    that we're trying to get out of our decision-making processes.
    Fair enough?
        Mr. Schmidt. Yes.
        Senator Rounds. OK. I know that we're going to have a lot
    of discussions about the problems and the challenges that we're
    going to have anytime we bring in a new type of technology. But
    I wanted to close with this, and it's going to come right back
    down to what Ms. Rice had indicated. Suitable AI talent is
    going to be in high demand.
        JPMorgan said in May of 2023 that it had hired 900 data
    scientists, 600 machine learning engineers, and 200 AI
    researchers to execute its technology initiatives. Google has
    hired thousands of researchers and engineers to work on machine
    learning and AI.
        However, I believe that it's critical that AI solutions
    reach beyond our largest companies. We learned at the end of
    the last year that between 2022 and 2023, the number of people
    experiencing homelessness on a given night increased in the
    United States by 12 percent to 653,104 people. The highest
    number of people recorded since the inception of the annual
    count in 2007. Researchers have used AI to identify individuals
    most at risk for losing their housing to target with
    homelessness prevention assistance.
        Now, I think this is one of the things that we should do a
    better job on, and if we can eliminate some of the challenges
    we have, just in terms of the number of individuals that we
    have to impact, AI could actually help us identify those
    individuals before they become homeless.
        Mr. Schmidt, could you discuss what steps we need to take
    now to provide the AI workforce of tomorrow, and how we make
    certain that AI talent is reaching areas like homelessness
    prevention?
        Mr. Schmidt. To address the latter half of your question, I
    think that it is very important that we encourage organizations
    like Ms. Rice's, as well as academia, and the type of work that
    Ms. Perry is doing, because quite frankly, industry is not
    likely to jump on those sorts of things. And we have to put the
    incentives in the right places and give the right people the
    right money and ability to put forward those sorts of programs.
    So that's what I would say about that.
        And then in terms of the talent gap, what we really need to
    do is concentrate on both STEM and liberal arts. What we need
    are well-rounded people. I've worked in tech-type companies my
    entire career, and I've always liked hiring English majors and
    history majors, and teaching them how to do statistics, because
    they're the ones who are really thoughtful.
        And so what we want to do is we want to encourage that. We
    want to encourage an educational system that is focused on
    questioning things, really getting into why are we doing
    things? And then the technology is secondary. My company,
    actually, so I founded it out 4 years ago. The first Python
    program I ever wrote became my company, and so I learned that.
    You know, I guess you can teach an old dog new tricks. I
    learned it later, and I think that many people can learn those
    kinds of things if they're given the right education to start
    with.
        Thank you. And thank you for your leniency on the time
    today, Madam Chair.
        Chair Smith. Absolutely. Thank you, Senator Rounds. Senator
    Warnock.
        Senator Warnock. Thank you, Madam Chair. And amen to Mr.
    Nicholas Schmidt on hiring folks with education and the
    humanities who think critically about these issues. Georgia is
    in a housing crisis, and it is not getting better. Since 2019,
    rent in Georgia has increased 13.7 percent. This is one of the
    highest increases in the country, but Georgia's not alone.
    According to Moody's Analytics, the amount that Americans are
    spending on rent is at a record high. For the first time last
    year, the average apartment cost over 30 percent, over 30
    percent of a family's monthly income.
        One factor driving rents up may be the use of AI software
    by some of the nation's largest and most powerful rental
    property management companies. As a ProPublica report details,
    this AI software looks at data across multiple companies and
    multiple buildings in the same area, and then recommends a
    monthly rental price. I am concerned that these tools may
    enable property management to coordinate pricing, coordinate
    escalation, inflate rents, and stifle competition.
        Ms. Rice, folks across Georgia are feeling the rental
    pinch. How can we ensure that AI driven rental software is not
    being weaponized to artificially inflate the rental market?
        Ms. Rice. Senator Warnock, thank you for that question. And
    you're talking about dynamic pricing systems, and you're
    absolutely right. We are very concerned about these systems and
    believe that they are contributing to the increase in rental
    housing prices.
        And because the systems are operated by a few companies
    that amass data amongst a number of different landlords
    throughout a region, there is a great propensity for reducing
    competition because those systems are conveying to all of those
    different landlords, essentially, the same dynamic, a kind of
    dynamic pricing.
        So a land an apartment complex that may not be getting a
    lot of inquiries can still elevate their prices because their
    competitors are getting a lot of inquiries. Right? So it does
    really work sort of cross purposes when you think about fair
    and affordable housing principles.
        The other danger is that there is very little transparency
    around these systems. So if a consumer wants to know why did
    the price increase from day one to day two the only information
    the consumer gets back is, ``Well, this is what the algorithm
    is saying the price is today.'' There's no explainability or
    transparency around why that price increase happened. So
    unfortunately, this is an area where we do need increased
    regulation.
        We're trying to use current fair housing laws to tackle
    this problem. We're also trying to use current antitrust laws
    to tackle this process, and we're not getting where we really
    need to be. So we do need increased regulation here. The only
    thing that I will add is that local jurisdictions may be able
    to use tools that they have to tamp down on the utilization of
    dynamic pricing algorithms, but that remains to be seen.
        Senator Warnock. So in an ironic way, and AI is here to
    stay, which is why we're having a conversation, but in a way,
    it sort of mitigates against natural competition in the market
    that this is falsely derived through these algorithms, and
    they're opaque. It's difficult to get through to ask the kinds
    of questions that we need to ask.
        So this is a unique threat that Congress has to address.
    This is affecting renters, but home ownership and the impact of
    these algorithms are impacting communities of color. Buying a
    home access to credit has been a long significant barrier.
    Techniques like machine learning and artificial intelligence
    can improve credit access by incorporating more types of data,
    like paying utility bills and rental history to demonstrate
    that someone is creditworthy, but they can recreate in code
    historical disparities and discrimination in the housing
    market.
        So, I'd like to quickly ask Dr. Perry. Dr. Perry, as credit
    scoring models become increasingly sophisticated, how do we
    ensure that these models do not perpetuate through the
    algorithm's historical discrimination?
        Ms. Perry. Thanks for the question. In order to ensure that
    these models, many of which operate in a sort of black box
    which is one of the key issues that we are concerned with, is
    really to require that we adopt ways to monitor and scrutinize
    every aspect of the model development process.
        I can't say it enough how important it is to have a set of
    guiding principles or goals that drive the development from the
    outset. Then we need to know exactly what inputs are currently
    being used and what inputs could potentially be used. Then we
    need to look at the way those inputs are combined. And finally,
    we need to do effects testing, but we can't just look at
    effects the way that we currently look at, say, test for
    disparate impact, because this is far more complicated than
    that and requires far more information.
        And, you know, these models take, you know, a long time to
    develop, but they also are changing on a dynamic basis, and
    that's why it's really important to know what the goals of the
    model are upfront.
        That said, I think there's really great potential because
    for everything you can program the AI to optimize, you can also
    program it to optimize opportunity and fairness. You can tell
    it explicitly, do not look at factors that are correlated with
    race, ethnicity, ability, or any other kinds of protected
    factors or prohibited factors.
        Chair Smith. Thank you. Thank you very much.
        Senator Warnock. Thank you very much, Madam Chair.
        Chair Smith. Yeah. Senator Cortez Masto.
        Senator Cortez Masto. Thank you, Madam Chair and our
    ranking member for this great conversation. I'm going to
    follow-up on this, but I'm going to ask Dr. Schmidt. If you
    would, Mr. Schmidt, talk a little bit about automated valuation
    models and home appraisals. Right? There's positives because it
    should lower costs. It should hopefully decrease bias, which we
    have been talking about. But you also note that the data used
    to build an automated evaluation model will be biased, could be
    biased. Right? And you have software program to address this
    issue. So, talk to us how you're decreasing that discrimination
    as the initial data going into these models.
        Mr. Schmidt. Sure. So the problem with any kind of pricing
    model, automated valuation model is that it is necessarily
    going to be looking at location, location, location. You know,
    that's what drives price. Well, what do we know about locations
    and housing in America? It's got a history that goes up to
    today of discrimination. You know, that has not ended.
        And so if you are building a model that incorporates
    information about location, which you have to do to get an
    accurate model, you are building a discriminatory model. But
    there are two things that kind of make it better. One of them
    is relative to the alternative. If you think about the
    potential bias, not necessarily, but potential bias of a human
    appraiser, and there's been a lot of investigations and work on
    that. It's shown that there's a really high risk of human
    appraisal bias, very unlikely that that will be explicitly
    built into an automated valuation model.
        So what that means is that within a particular geography
    that you're controlling for in one of these models, you're
    likely to remove that idiosyncratic bias from the appraiser.
    But the fact that you are controlling from one geography to
    another means that you're incorporating that geographic level
    discrimination.
        So I think of AVMs as having--they're a step, maybe a small
    step in the right direction. They are also much cheaper in
    rural settings. My understanding is in rural areas, it can be
    impossible to get an appraisal, a human appraisal. If there's
    good enough data, then an AVM could provide one. So there are
    many reasons why an AVM would represent just a sort of uniform
    improvement relative to current not ideal practices.
        Senator Cortez Masto. So let me ask you this, because this
    is a concern. I see where we're going, and I think my
    colleague's absolutely right. AI is here, and people are going
    to start using it, and they'll start using an appraisal
    process.
        I know in Las Vegas, in the metropolitan area, home values
    in a majority of our Black neighborhoods were devalued by at
    least 12 percent. That's over $22,000 difference compared to
    neighborhoods with fewer than 1 percent of Black residents. How
    do we ensure that that bias doesn't get put into these models?
        Mr. Schmidt. So it will to some----
        Senator Cortez Masto. So then how do we adjust for these
    differences?
        Mr. Schmidt. Yes. But there is yet another aspect to it,
    which is--and Ms. Rice was talking about it, and it's actually
    the center of my work--called Searching for Less Discriminatory
    Alternative Models. And it actually works better in AI and
    machine learning.
        And what it turns out in machine learning models is that
    you can have thousands, really an infinite number of ways to
    specify these models, including certain information, excluding
    other information, changing what are called hyperparameters,
    which are kind of the knobs and dials that slightly change the
    algorithm. All of these can have, or changing the algorithm
    itself, all of these can have an effect on the bias as well as
    the quality of the model.
        And machine learning, it turns out, there are just way more
    opportunities. And so what I've developed, and what Ms. Rice's
    organization has developed, and other very good organizations
    are working on, are finding those fairer models, and that's
    where things can continue to improve.
        Senator Cortez Masto. OK. Any other comments, Ms. Rice?
        Ms. Rice. Thank you, Senator Cortez Masto. I will say and
    remind us that not every issue warrants a technological
    response. So there is still--let's still remember that we have
    human beings that are highly trained to perform these types of
    appraisals, and we have to use them.
        Now, that said, AVMs, while they can be a useful tool, they
    are not as accurate in every community, so they don't have the
    same level of performance. And so we do find that AVM models
    are less accurate. In fact, in some communities, they have a 15
    percent degree of accuracy.
        So I think you'd agree you would not want to use an AVM
    model in a community that has a 15 percent degree of accuracy.
    That we're finding that in rural areas, and we're finding it in
    predominantly Black and Latino areas. AVM models are much more
    predictive in areas that have a lot of new housing development,
    and that are predominantly White.
        So, we are working on tools to make AVM models fairer. What
    will make AVM models fairer is if we change the construct or
    the approach that we currently use to determine property
    values. Right now, we use a sales comparison approach, which as
    Nick has already said, requires that I look at what the house
    across the street from me sold, or the house around the block
    from me sold.
        We have worked extensively with the insurance sector to
    change their property valuation models to make them fairer by
    considering the cost of reconstruction, and making sure that we
    have accurate data that can assess what is that reconstruction
    cost, and that gets embedded or enfolded into the valuation
    process.
        Senator Cortez Masto. Thank you. Thank you very much.
        Chair Smith. Thank you so much. Senator Lummis. We're going
    to move to a second round of questions for anybody who has
    them.
        Senator Lummis. Thank you, Madam Chair. And I'd like to
    start just by saying, Mr. Schmidt, you finally gave me an
    explanation for what I can tell people. My son-in-law does--and
    I've wondered how to explain what he does for years. Now I
    know. He manages teams of people who are drunk on algorithms,
    to make a product that's useful by sober people.
        [Laughter.]
        Senator Lummis. So thank you for that. I finally know how
    to explain what he does. Panel, thank you so much. I have the
    same question for each one of you to conclude my questioning.
    And I'd like to start with you, Dr. Perry. What regulatory
    approach would you advocate for this Committee to pursue? What
    guiding principles can you point us toward? How can you help us
    be relevant in this discussion?
        Ms. Perry. Thank you, Senator Lummis, for the question. I
    think that of the most important consideration is holding AI
    accountable for expanding opportunity. More so than anything, I
    think that is of critical importance. It's also important to
    move really very quickly because the industry is already
    adopting and applying these tools in ways we are not even
    aware.
        Senator Lummis. Thank you. Ms. Rice.
        Ms. Rice. Senator Lummis, thank you so much for the
    question. First, I'd say I'd echo what I said earlier, that we
    already have a bevy of existing laws and regulations that apply
    in this sector. And so, what we can do is help you understand
    the extent of that. And one of the things that we are thinking
    about is pulling together sort of a matrix, you know, some
    agencies that explain what are the existing laws and
    regulations that are in place, and then understanding what we
    need to do to enforce those existing laws and standards.
        But the second, I will note, is that there are gaps. There
    are clearly gaps, and we're finding that--we talked about a gap
    with the dynamic rent, the rental dynamic pricing schemes. And
    so, listing where there are gaps, and then helping legislators
    understand what it would really take to fill those gaps.
        And here I do advocate for an intersectoral approach,
    because you're going to need academicians, you're going to need
    developers, you're going to need industry, and you're going to
    need civil rights and human rights organizations to understand,
    you know, how to design the legislation so that we are
    addressing those gaps that do exist.
        Senator Lummis. Thank you. Mr. Schmidt.
        Mr. Schmidt. So, I would really repeat what my copanelists
    said. I think we need to move fast. I think it's extremely
    important to recognize that existing regulations already cover
    90 percent of it. What I have seen the Consumer Financial
    Protection Bureau, CFPP, and the OCC doing, I cannot say enough
    for. I think they're really moving in the right direction.
    They're doing excellent work, and they should be empowered to
    keep going in those directions. What I see is that industry is
    responding in a way that is really right.
        The other thing I would say is just that we need to make
    sure that our regulators have the education, they have the
    experience that they need. What the bureau has done from 2016
    to today, I think they've done an amazing amount of self-
    education and bringing on really good people. And they're doing
    a great job considering the resources they have, but they could
    use more.
        Senator Lummis. Thank you. And Ms. Rice.
        Ms. Rice. Senator Lummis, can I just mention one thing?
    Section 230, the Communications Decency Act. We are going to
    have to revisit it.
        Senator Lummis. Thank you for that. This has been a great
    panel. Thanks for your work in this area. This is all new to
    me. So your guidance is deeply appreciated. Thanks, Madam
    Chair, for holding this hearing. I yield back.
        Chair Smith. Thank you so much. I just have a follow-up
    question, kind of along the lines of what Senator Lummis was
    asking about. There have been some efforts at the State level
    creating legislation and guidance around AI implementation. Mr.
    Schmidt or anybody, would you just comment, do you see anything
    out there at the State level that you think is good or
    something that we should be cautious about?
        Mr. Schmidt. The New York Department of Financial Services
    just released a circular for insurance. And of course, I could
    quibble with a few things, but really, I could have written the
    thing myself and been proud of it. I think that what they have
    done is excellent and really principle-based, and based in
    existing regulation and law, and could be very effective.
        Chair Smith. That's great. Anybody else have a comment
    about State-led efforts that you think are useful or something
    to not do?
        Ms. Perry. There a number of States that have done some
    really impressive things. And the concern that I'm have now is
    that they're going to be different from each other, and it's
    going to create havoc for companies, and in particular, it's
    going to be problematic for smaller businesses to comply when
    there are different State regulations, and something that is by
    definition, crosses State boundaries.
        Chair Smith. Yeah. Thank you. That's great. Ranking Member
    Lummis, I have a feeling that you and I could--we have easily
    another hour-and-a-half of conversation with these excellent
    panelists about this topic. And I think that there's a lot more
    work for us to do around this idea of what model governance
    standards might look like. The President's executive order on
    AI, the AI Bill of Rights, and you all have laid out some
    possibilities. There's lots of work there.
        But I'm struggling to really understand like how that
    becomes implementable, you know, in industry as you're trying
    to figure out--you know, as industry is trying to figure out
    what the rules are. And I'm also really struck by how one of
    the things that happens in AI is there's an imbalance of
    information.
        Like, information is power. And if you have more
    information as a seller than a buyer, then you're going to have
    more power. I'm seeing this right now in my own home State of
    Minnesota, where we have these big institutional investors who
    are able to within a matter of minutes, when a house comes on
    the market, decide whether it's something that they want to buy
    or not, while an individual home potential homeowner can't
    possibly compete with that. And so that shifts the balance of
    power in how real estate market has worked in the past in ways
    that I think are concerning.
        So I always think that if you want to understand the
    outcome, you need to look at what the incentives are. And to
    me, the question is how do we build incentives into this system
    so that we are maximizing fairness, and accountability, and
    transparency as we as we try to set some guidelines both to
    encourage innovation, but also to make sure that these systems
    are working for the public benefit.
        So, thank you very much to all of you for this. Senator
    Cortez Masto, do you have a follow-up question at all?
        Senator Cortez Masto. Just a comment, and I appreciate the
    conversation. Let me just say I heard the be thoughtful,
    reasonable, and rapid three things Congress is challenged at
    doing. And so we are doomed from the start.
        But it really would be helpful to get your thoughts maybe
    on the top three things that we can focus on in the housing
    space, or the best model that a State has put forward. I
    understand the patchwork concern, but if there's a good model
    that can be brought to at a Federal level, I think that's worth
    looking at as well. And then finally, thank you, Ms. Rice, for
    talking about CDA and 230. I couldn't agree more. We have to do
    something about that. So, thank you.
        Chair Smith. Great. Thank you so much. Well, thank you to
    our witnesses for being here today and for providing such
    excellent testimony. Before we adjourn, I would like to enter
    several statements into the record from the Urban Institute's
    Housing Finance Policy Center, and from Zillow, as well as
    reports and articles from Dr. Perry. Is there any objection.
    Without objection, those will be entered into the record.
        For Senators who wish to submit questions for the record,
    those questions are due 1 week from today, which is Wednesday,
    February 7th. For our witnesses, you will have 45 days to
    respond to any questions for the record, and thank you again.
    And with that, this hearing is adjourned.
        [Whereupon, at 11:21 a.m., the hearing was adjourned.]
        [Prepared statements, responses to written questions, and
    additional material supplied for the record follow:]
                     PREPARED STATEMENT OF CHAIR TINA SMITH
        Today's Hearing will focus on the promise and threats that
    Artificial Intelligence poses in the housing sector, and I am very much
    looking forward to our witnesses testimony and this conversation. I
    want to thank Ranking Member Lummis and her staff for our ongoing
    bipartisan work as we put together this hearing. We both share, I
    believe, a deep interest in how we can develop Federal policy that
    supports innovation and expands opportunity for everyone to have a
    safe, decent affordable place to live.
        And one of the most consequential innovations in recent years is
    Artificial Intelligence. Leader Schumer, Senator Rounds, Senator Young,
    and Senator Heinrich are leading a bipartisan effort to explore the
    impacts, opportunities and threats that AI poses. And they have asked
    Senate Committees to engage in our areas of expertise, which leads us
    to this Committee hearing today, examining what AI means for housing.
        Without a safe, decent, affordable place to live, nothing in your
    life works--not your job, your family, your education, or your health.
        So, a foundational question is how AI can help and hinder this
    goal. We know that some aspects of Artificial Intelligence have been
    around for a long time, and we also know that major advances are
    fueling the use of AI in finance and housing in ways that we need to
    understand.
        Consumers can find AI when they encounter chat bots when they shop
    online, or digital ``helpers'' that seem to be ubiquitous.
        AI plays a role when:

        a prospective tenant is looking to rent an apartment,

        a renter submits a maintenance request to her management
            company,

        a family tries to qualify for a home loan, or when

        a person experiencing homelessness is connected to
            services.

        These are powerful tools that hold great potential to cut costs,
    target services, reduce wait times, and even reduce bias.
        But they also have the potential to bake in existing inequities,
    reduce accountability, and limit opportunity.
        Today AI is being actively used in every part of the housing
    continuum, from emergency homelessness services to mortgage financing.
    As I was preparing for this hearing, I found endless applications.
        AI is being deployed to help connect people experiencing
    homelessness to health and housing resources.
        AI is helping to forecast more precisely and accurately where
    families are at risk of eviction to help better target assistance.
        Academics and advocates are using AI and machine learning to help
    understand and map the country's zoning laws and codes, which span
    about 30,000 localities. These insights will help understand the dense
    and complicated rules that govern where, how and what type of housing
    is allowed to be built, so we can make better decisions about boosting
    housing supply and lowering costs.
        So there are many opportunities. And, there are also very real
    concerns about the threats that artificial intelligence can pose to
    Americans. In Minnesota, some landlords are reportedly using AI-
    generated tenant screening reports that include incorrect and sometimes
    illegal and off-limits information. The result--it's even harder for
    people to find a place to rent, and they may never know why they were
    declined, or be able to correct the record. For landlords, it may be
    easier to just move onto the next applicant rather than considering
    additional information.
        Another example of how AI, used in a bad way, can be quite harmful:
    There's a current lawsuit in Minnesota against a law firm that
    allegedly has ``automated'' the process of filing evictions for
    landlords. In one month, the firm filed 400 eviction complaints. These
    eviction filings lacked much detail about why the eviction was
    happening and seem to routinely lack basic information about lease
    terms and included significant errors regarding lease dates, rental
    amounts, and payment information.
        The fact that a firm allegedly leaned on AI to generate a large
    number of eviction filings with false information, apparently without
    meaningful reviewing by an attorney is a big problem. Not only is the
    eviction illegal, but that eviction will live on in public records and
    hurt the tenant into the future.
        AI is also increasingly part of how people buy homes. It's used in
    credit scoring models and automated valuation models (or AVMs), which
    determine the value of a home. How AI is deployed has major
    implications for a person's credit scores, their mortgage rates, and
    whether home ownership and wealth building is even within reach. We
    know that we have historic, systemic challenges with fairness and
    equity in in this country--my own home town of Minneapolis has some of
    the greatest disparities in home ownership between Black and White
    families anywhere in the country. We need to carefully explore whether
    AI is extending and reinforcing these biases, and how it has the
    potential to correct them.
        Our excellent witnesses have an unenviable task in your opening
    statements--to ground us in both the opportunities and threats of AI in
    housing, in 5 minutes each. I look forward to hearing from you, and I
    look forward to hearing the questions and conversation with my
    colleagues that will follow.
        As with any innovation, there are both opportunities and challenges
    that must be balanced, and our job is to think through how these
    complex issues so we can develop the best public policy. I very much
    look forward to this conversation.

                        PREPARED STATEMENT OF LISA RICE

     President and Chief Executive Officer, National Fair Housing Alliance
                                January 31, 2024

    [GRAPHIC(S) NOT AVAILABLE IN TIFF FORMAT]

                      PREPARED STATEMENT OF VANESSA PERRY
    Interim Dean and Professor, The George Washington University School of
      Business; Nonresident Fellow, Housing Finance Policy Center, Urban
                                   Institute
                                January 31, 2024
        Good morning. Thank you for inviting me to address the impact of
    Artificial intelligence, i.e., AI, which is being employed increasingly
    throughout the housing and mortgage industry. For these purposes, AI
    refers to the use of data and algorithms in place of human decisions.
    This definition includes machine learning models, which are programmed
    to imitate the way humans learn, iteratively correcting themselves to
    improve their accuracy.
        Compared to traditional models, AI relies on a wider range of data
    inputs and more complex combinations thereof. Although complex
    multivariate algorithms have been in place in the mortgage market for
    years, these models have the potential to incorporate nontraditional
    data sources. Due to their complexity, it is difficult, but not
    impossible, for anyone other than AI developers to scrutinize and
    monitor their inputs.
        AI models are already widely applied in the mortgage market. AI
    digital marketing models target prospective homebuyers and
    communications with customers are intermediated by AI chatbots. Credit
    scoring companies and mortgage underwriting systems use AI to evaluate
    credit risk. AI models are widely used for property valuation, loan
    servicing, and loss mitigation. AI regulation warrants urgent attention
    because evidence from other domains and my research with the Housing
    Finance Policy Center at the Urban Institute suggests that while these
    models can enhance efficiency, they can have unintended impacts on
    fairness and equity. \1\
    ---------------------------------------------------------------------------
         \1\ Michael Neal, Linna Zhu, Caitlin Young, Vanessa G. Perry,
    Matthew Pruitt (2023), ``Harnessing Artificial Intelligence for Equity
    in Mortgage Finance'', Urban Institute, November 6, https://
    www.urban.org/research/publication/harnessing-artificial-intelligence-
    equitymortgage-finance.
    ---------------------------------------------------------------------------
        AI models are not subject to human errors, and they enable
    efficient, accurate, and consistent decisions. Depending on how they
    are developed, their enhanced capabilities could expand access to home
    ownership for households currently underrepresented in the mortgage
    market. For example, AI can produce faster and less subjective
    estimates than human property appraisals and can devise credit scores
    for those who lack a traditional credit history.
        However, because these models rely on historical data, there is the
    potential for these models to systematize and amplify discrimination
    and inequality. \2\ For example, due to the legacy of redlining and
    segregation and their effects on present-day neighborhood conditions
    and home values, why should we expect AI models to produce estimates
    that are both accurate and fair? And absent guardrails, how would we
    know if AI models were to incorporate data elements, such as GPS
    location, that serve as a proxy for race, gender, or ability?
    ---------------------------------------------------------------------------
         \2\ Michael Neal, Linna Zhu, Vanessa G. Perry (2024), ``To Err Is
    Automated: Have Technological Advances in the Mortgage Market Increased
    Opportunities for Black Homeownership?'' Journal of the Center for
    Policy Analysis and Research, forthcoming, https://papers.ssrn.com/
    abstract=4347212.
    ---------------------------------------------------------------------------
        To address concerns about AI's impact on access to the housing
    finance system for underrepresented or marginalized communities, my
    coauthors and I have proposed five factors summarizing the societal,
    ethical, legal, and practical issues that should be considered in the
    development and implementation of AI. \3\ They form a memorable
    acronym, S.C.A.L.E., which stands for:
    ---------------------------------------------------------------------------
         \3\ Perry, V., Kirsten Martin, and Ann Schnare (2023),
    ``Algorithms for All: Can AI in the Mortgage Market Expand Access to
    Homeownership?'', AI, 4(4), 888-903, https://doi.org/10.3390/ai4040045

        Societal values. Algorithms tell us what factors the
            developer thinks are important, in what order, and to what
            degree. AI models should consider the socio-economic and
            historical context (e.g., past discrimination) and should align
            with prevailing legal and ethical paradigms, e.g., disparate
    ---------------------------------------------------------------------------
            impact law, individual freedom, and racial equity.

        Contextual integrity. In addition to its accuracy, model
            inputs should be relevant to the mortgage or housing domain,
            and may differ substantively from those used for other or less
            consequential contexts.

        Accuracy. Models should be reliable, error-free, unbiased,
            and representative of all demographic and economic groups
            across varying macroeconomic conditions.

        Legality. The model and its inputs, if used for housing or
            housing finance decisions, should not incorporate
            characteristics protected by fair lending laws or generate
            unjustified disparate impacts based on these characteristics.

        Expanded opportunity. AI models should significantly
            increase access to credit in addition to offering greater cost
            efficiency or risk assessment benefits. This criterion has
            perhaps the most promising impact on the economy and
            communities.

        In terms of policy directions, the S.C.A.L.E. framework could
    inform new or expanded regulations, such as guidance for the use of
    certain types of data--such as an individual's social media profile--
    for certain purposes, such as mortgage lending decisions.
        While the S.C.A.L.E. criteria imply that model inputs and
    algorithms, due to the complexity of AI models, regulators cannot rely
    on traditional approaches to documentation and testing. Furthermore,
    concerns about potential harms related to AI are domain-specific,
    suggesting that regulation and enforcement efforts must be targeted
    specifically to housing and mortgage applications.
        If designed to do so, AI models can increase access to home
    ownership and eradicate the effects of systemic discrimination while
    increasing accuracy and efficiency in the mortgage value chain. We need
    laws on the Federal level that turn the ``S.C.A.L.E.'' toward imposing
    these standards at every stage of the AI life cycle.
                                     ______

                     PREPARED STATEMENT OF NICHOLAS SCHMIDT
    Partner and Artificial Intelligence Practice Leader, BLDS; Founder and
                                  CTO, SolasAI
                                January 31, 2024
        Chair Smith, Ranking Member Lummis, and Members of the
    Subcommittee, thank you for hosting this important hearing and for
    giving me the opportunity to submit this testimony.
        My name is Nicholas Schmidt, and I am the founder and Chief
    Technology Officer (CTO) of SolasAI, as well as a Partner and the AI
    Practice Leader at BLDS, LLC. My passion for the responsible and fair
    application of artificial intelligence (AI) and machine learning (ML),
    including in housing, has driven my career, leading to the development
    of SolasAI, a software platform dedicated to mitigating bias and
    discrimination in algorithmic decision making.
        At SolasAI, we create software that not only addresses regulatory,
    legal, and reputational challenges but also empowers innovation in
    model development. Our work centers on reducing disparate impact and
    eliminating disparate treatment, ensuring that AI tools used in housing
    are equitable, effective, and profitable.
        Today, I am here to share insights on how properly implemented and
    regulated machine learning and artificial intelligence can transform
    the housing sector. I aim to provide a comprehensive understanding of
    the potential for AI to make high-quality housing more affordable and
    accessible, while also addressing the critical need for fairness and
    equity in these technologies.
    Defining Artificial Intelligence
        Before one can hope to craft effective laws or regulations around
    the use of artificial intelligence, we must first define it and
    understand its scope. While the term ``AI'' often conjures a vision of
    futuristic and sentient machines, in practical terms, AI encompasses a
    wide array of far less radical technologies.
        Contrary to the popular focus on--and hype around--generative AI,
    AI's impact on society extends through various types of machine
    learning (ML) and AI applications, many of which are already
    transforming the housing industry. \1\ What can be defined as AI
    includes technologies ranging from predictive analytics to automated
    decision-making systems, all of which impact the affordability,
    accessibility, and equity of housing in the United States.
    ---------------------------------------------------------------------------
         \1\ Significant ink has been spilled over the attempt to precisely
    define the difference between ``AI'' and ``ML.'' Commonly, people refer
    to deep neural network-based techniques that are typically used for
    image, text, and language recognition or generation as ``AI,'' whereas
    other techniques that are used for things like credit underwriting or
    pricing (including, especially, tree-based ensemble models) are
    referred to as ``ML.'' A more robust and thorough discussion of the
    terms can be found here: https://mitsloan.mit.edu/ideas-made-to-matter/
    machine-learning-explained and https://aima.cs.berkeley.edu/
    newchap00.pdf.
    ---------------------------------------------------------------------------
        In practical terms, ML and AI represent a class of mathematical
    algorithms that learn patterns and rules from data. These learned rules
    may then be applied to new data to inform real-world decisions. Thus,
    it is important to remember that the rules AI develops are based on
    mathematical (i.e., not human) insight, but that those rules are
    developed on historical data that encode many types of human biases.
    However, as I discuss below, despite the computer developing the rules,
    human decisions affect how the rules are developed and systems are
    used. Understanding this is essential for writing effective
    legislation.
    Human Decisions Drive Algorithms
        Beyond the confusion of what ``AI'' is, many people are unaware of
    how much human involvement is required to build and deploy an AI (or
    ``algorithmic decisioning'') system. In fact, particularly coming from
    technology companies, there seems to be a fatalistic attitude that
    implies nothing can be done to improve them. This notion, dangerous in
    the extreme, is easily proven wrong. There are numerous places where
    humans interact with AI systems before and during the deployment of the
    algorithms that shape whether the algorithms are making reasonable,
    safe, and fair decisions. Understanding the extent to which the output
    of an algorithm is dependent on the decisions that people who build the
    models make is essential because--at each of these decision points--
    there are opportunities for humans to make better decisions that can
    make the algorithms more fair, accountable, and transparent.
        Using a mortgage delinquency algorithm as an example, the human
    steps required to make such a model include, e.g.:

      1.  Choosing what the model will define as a delinquency. A data
            scientist might define delinquency as 60 days or 90 days of
            nonpayment. For people with less income security, but who are
            likely to be able to pay their bills over a longer period,
            choosing 60 days instead of 90 days may make the difference
            between being provided a loan or being rejected. Importantly,
            this decision is an entirely human-based decision.

      2.  Choosing which data will be used to predict delinquency. The
            computer only makes its rules based on the data it is provided.
            The person building the model might only include data clearly
            related to delinquency (e.g., the existing level of debt, past
            payment history, etc.), or they may include data that is not
            clearly causally related to delinquency (e.g., education or
            purchase history). The choices that the modeler makes will
            affect the fairness of the model, its accuracy, its
            reasonableness, and its reliability. While algorithms will
            choose how to weight different data (and possibly exclude
            certain data altogether), choosing which data to provide is an
            entirely human-based decision.

      3.  Choosing the type of algorithm that will build these rules. Model
            developers have many options regarding what model architecture
            they will use to develop the model (i.e., how the model will
            learn from the data). These include architectures like deep
            neural networks, gradient-boosted trees, or traditional linear
            regression. This decision has direct implications in terms of
            the transparency of the model and whether the model's decisions
            will be explainable. Some kinds of models, such as regression
            models, are inherently explainable. This means it is possible
            to know exactly how the model arrived at a decision. \2\
            Others, such as neural networks, are not transparent. The
            decisions surrounding model architecture have practical and
            legal implications, as I explain infra.
    ---------------------------------------------------------------------------
         \2\ Providing a hypothetical example of how easy it is to
    interpret a linear regression, suppose we build an AVM using linear
    regression that uses three variables to predict home price: number of
    bedrooms, number of bathrooms, and number of square feet. With linear
    regression, we know the effect of each of these variables on a
    particular house's estimated price because linear regression is
    completely transparent. Looking at the model, we could know that, for
    example, each bedroom adds $50,000 to the value. We could also
    explicitly tell that an additional bathroom adds $25,000 to the value.
    Finally, we could see that every additional 100 square feet adds $5,000
    to the value. On the other hand, a more complicated model architecture
    could learn from the interaction of all of these variables in highly
    nonlinear ways such that increasing the square footage of the house by
    100 feet adds between $0 and $50,000, depending on the number of
    bedrooms and bathrooms. While, in this particular example, a more
    complex model would be more accurate than the regression model, it
    would also likely be very difficult to interpret and fully understand
    the model. This becomes particularly problematic when we move away from
    this simple three variable example to a more realistic ML model, which
    might include dozens, hundreds, or maybe thousands of variables.

      4.  Choosing whether the algorithm is sufficiently accurate for the
            task. No algorithm is perfect. One issue is that a model might
            be very effective for some people, but not others. For example,
            it might be highly accurate for high-credit quality individuals
            but be far less accurate for those with lower credit quality. A
            person working for the lender will ultimately make a decision
            whether that trade-off is acceptable before putting the model
            into production. More generally, whether to test for different
            forms of inaccuracy, how to balance the varying costs of
            inaccuracy, and what minimum accuracy requirements are required
    ---------------------------------------------------------------------------
            are all human decisions.

      5.  Choosing how to implement and use the algorithm. Generally, an AI
            or ML model does not make a decision on its own. Frequently,
            only a subset of applicants will be scored by the model; which
            applicants are scored is a human decision. Once applicants are
            scored, what that score means and how it is used must be
            determined. For example, will a cutoff be used, or will the
            specific model work with another model or subjective rules to
            make a decision? Will the score be considered in light of other
            variables or factors? All of these are human decisions. In
            fact, when all of these decisions are examined together, it is
            clear that the entire model system, of which the model itself
            may only play a relatively modest role, is largely made up of
            interlocking human decisions that result in an ultimate
            decision.

        I take pains to point out each of these decision points because, as
    NIST-1270 puts it, ``A fallacy of objectivity can often surround these
    processes, and may create conditions where technology's capacity and
    capabilities are oversold.'' \3\
    ---------------------------------------------------------------------------
         \3\ The sentence preceding this quote summarizes the issues I
    address: ``Adding to the challenge is the reality that these systems
    are built and placed within organizational settings along with their
    accompanying--often unstated--policies and priorities, and used by
    subject matter experts and decision makers who have their own implicit
    heuristics and biases.'' NIST SP-1270, ``Towards a Standard for
    Identifying and Managing Bias in Artificial Intelligence'', p.25.
    ---------------------------------------------------------------------------
        This fallacy of objectivity has led many to conclude that not much
    can be done to regulate or effectively manage these technologies.
    However, because so much of the use of AI is driven by choices that
    people make, regulators and the law do not need to ``surrender'' to
    these emerging technologies; the space is ripe for regulation of human
    decisions. In fact, effective regulation of these human decisions can
    create fairer, more equitable outcomes without stifling innovation in
    this space. But more than just a benefit to consumers, having defined
    and reasonable regulations would give companies more clarity on how
    they can safely use AI systems.
        I next provide context for how AI is used in housing by discussing
    common use cases and outlining the challenges and opportunities they
    offer.
    The Use of AI Systems in Housing
        AI systems are increasingly being used across the housing industry,
    as companies find that many facets of housing decisions can be made far
    faster, cheaper, and more reliably than can be performed by humans.
    While, in past years, there was significant discussion about whether
    algorithms should be making such decisions, we are now in a world where
    the use of such algorithms is commonplace; companies that do not employ
    them are at great risk of losing out to those that do. Below, I outline
    several areas where I have seen algorithms used in housing and discuss
    the opportunities and challenges associated with their use.
        Many entities have used algorithms to underwrite and price
    mortgages for decades. Fannie Mae's Desktop Underwriter (DU) and
    Freddie Mac's Loan Product Advisor (LPA) have histories dating back
    nearly 30 years, while FICO introduced its first consumer score in
    1989. More recently, human appraisals are being replaced with automated
    valuation models (AVM), allowing fast and--hopefully--accurate
    assessment of the value of a home. Other ways algorithms are used
    include the provision and pricing of insurance, providing background
    screening for rentals, automating the servicing of loans, and pricing
    rental units.
        There are two noteworthy takeaways from these applications of AI in
    the housing industry. First, there is a long history of the use of
    algorithms in housing. Correspondingly, there is a wealth of experience
    in building these algorithms fairly, reliably, and transparently. As
    such, we do not have to reinvent the wheel when it comes to effectively
    regulating the responsible use of algorithms. Second, the reach of
    algorithms in the housing industry is growing fast, which will
    profoundly affect people's housing decisions. It is imperative that we
    learn from what we know about safely implementing algorithms in housing
    and apply that knowledge to these newer applications.
        However, before discussing how the history and practices of
    regulating and monitoring the use of algorithms can be extended and
    made better in the world of AI, I will discuss a particular use case of
    algorithmic decisioning--automated valuation models--because they
    represent a concrete example of many of the benefits and challenges
    inherent in the expanded use of algorithms and AI.
    Case Study in the Use of Algorithms: Automated Valuation Models
        As anyone who has purchased a house knows, obtaining an appraisal
    can be a drawn-out, stressful, expensive, and opaque process. Further,
    there is substantial evidence that human appraisals often suffer from
    significant discrimination, with Black or African American homeowners
    and people living in predominantly Black neighborhoods having their
    homes appraised for far less than they would have been if they were
    White or lived in a predominantly White neighborhood. \4\ Additionally,
    people living in rural areas have found it difficult to get an
    appraisal of their homes, and have found that appraisals are often more
    expensive. \5\ \6\ Higher costs and delays in securing an appraisal may
    make transactions in rural areas take longer, cause sales not to occur,
    or prevent home buyers or refinancers from locking in lower rates.
    ---------------------------------------------------------------------------
         \4\ http://tinyurl.com/3w5s2dak
         \5\ https://www.knock.com/blog/how-long-does-an-appraisal-take
         \6\ https://www.homelight.com/blog/buyer-how-long-does-an-
    appraisal-take/
    ---------------------------------------------------------------------------
        A potential promise of automated valuation models (AVMs) is that
    they may solve these problems. AVMs replace the job of a human
    appraiser by predicting the value of a home using a wide range of data,
    including information such as a home's square feet, number of
    bathrooms, size of yard, tax history, location, sales prices of similar
    homes, and many other factors. While they have a long history of use by
    financial institutions in valuing portfolios of properties, they are
    now being used in a way that has a far more direct impact on consumers.
    Lenders are using these estimated home values for many financial
    decisions, including as a factor in deciding whether to offer a loan,
    provide favorable terms, or offer refinancing. As such, they may
    significantly affect a person's finances and life, influencing where
    they live and how much they pay for housing. Their use is instructive
    for understanding the benefits and harms of algorithmic decisioning.
        AVMs' obvious and clear benefits are their speed, availability, and
    price. The National Association of Realtors (NAR) reported that, in
    2022, the average cost of a home appraisal was $500, with 9 percent of
    appraisals costing more than $800. \7\ Once an AVM is built and
    running, the cost of calculating an appraisal is virtually zero for the
    model owner (though borrowers could still be charged for this service).
    NAR additionally reported that the median response from realtors
    indicated that it takes 14 days for a lender to return a completed
    appraisal, with 4 percent of realtors reporting that it typically takes
    them more than 30 days to receive a completed appraisal. In comparison,
    obtaining an appraisal from an AVM is virtually instantaneous; even if
    the lender performs subsequent reviews of the AVM-based appraisal, an
    AVM-based appraisal is likely to be available substantially faster than
    a human-based appraisal. It is likely that the use of AVMs has already
    ensured that home sales have closed faster, instead of languishing or
    being rejected.
    ---------------------------------------------------------------------------
         \7\ https://www.nar.realtor/sites/default/files/documents/2022-
    appraisal-survey-09-27-2022.pdf
    ---------------------------------------------------------------------------
        Another benefit of an appraisal from an AVM is that it may be less
    discriminatory than an appraisal made by a human. A blog post from the
    CFPB describes a lawsuit where a Black couple's home appraised for
    nearly 60 percent more after they ``whitewashed'' their home--removing
    evidence that they were Black. \8\ Given that the data used in an AVM
    would not incorporate information about the race of the homeowners,
    this type of discrimination is unlikely to manifest in a well-built
    AVM. Put another way, human appraisers are subject to cognitive biases
    they may not even be aware of; AVMs are not.
    ---------------------------------------------------------------------------
         \8\ http://tinyurl.com/2jt4vtrs
    ---------------------------------------------------------------------------
        However, this is not to say that AVMs will be free from any
    discrimination or bias. \9\ AVMs are not magical--if they create
    accurate appraisals, those appraisals will follow the real estate maxim
    of ``location, location, location.'' Of course, because of the history
    of housing discrimination in the United States, a home's location
    necessarily incorporates historical and present patterns of
    discrimination and bias. As a result, because the data used to build
    AVMs is necessarily biased, the output of the models will be as well.
    AVMs also cannot help when there may be insufficient data to draw
    conclusions.
    ---------------------------------------------------------------------------
         \9\ Throughout this report, I use the terms ``bias,''
    ``discrimination,'' and ``disparities'' more or less interchangeably,
    according to their lay definitions. However, it should be noted that
    each of these has distinct definitions and interpretations in technical
    and legal settings. Defining these terms precisely becomes particularly
    important when measuring and mitigating discrimination.
    ---------------------------------------------------------------------------
        To summarize, the discriminatory effect of AVMs is likely to be
    mixed. Because an AVM does not have access to or incorporate
    information about individual sellers or borrowers, it cannot
    discriminate based on those factors. However, it will still incorporate
    price-affecting discrimination in the data it sees. As a result, we can
    view AVMs as a tool to help fight discrimination in home pricing, but
    not one that is complete or without significant residual problems.
        My work, and the software we have developed at SolasAI, measures
    bias and discrimination in models such as AVMs. As explained above,
    data may be biased due to historical patterns of discrimination; when a
    model is trained on such data, it results in a biased model. Our work
    focuses first on measuring whether a model shows evidence of unfair
    disparities and, if it does, attempts to identify the source of those
    disparities. If such disparities are found, the software searches for
    what is known as a ``less discriminatory alternative'' (LDA) model.
    These LDA models provide predictive power similar to or equal to the
    original model but have a less discriminatory effect.
        In addition to these remaining concerns about discrimination, AVMs
    raise at least two other challenges: interpretability and
    accountability. Many (but not all) AI and ML algorithms are described
    as ``black-box'' processes, which means that, while we understand what
    data went into the algorithm, and we can see what the output of the
    algorithm is, it is very difficult to understand how or why the
    algorithm made the prediction that it made. In the case of a black-box
    AVM model, a lender might not be able to provide a borrower with a
    reliable explanation for why the home they wanted to buy received a low
    appraisal. While this lack of clarity may be acceptable in low-risk
    situations, or when the algorithm gets its prediction right, having
    opaque black-box models make high-risk decisions that cannot be
    explained may not be fair to people who receive unfavorable outcomes.
        The lack of explainability also raises concerns about the quality
    of the models. If we cannot sufficiently understand why a model gives a
    particular prediction, then we should have less confidence in that
    model. The problem is that, while it may appear to be a high-quality
    model based on the data it has seen, if we cannot understand the model,
    then we cannot be sure that its decision-making process makes sense and
    is likely to continue to operate well if conditions change. As a
    result, for business-critical or high-human-impact decisions, model
    builders need to balance a desire for accuracy with the necessity that
    the algorithm be explainable. \10\
    ---------------------------------------------------------------------------
         \10\ There is a general understanding that more opaque ``black-
    box'' algorithms are better able to capture hidden patterns in data
    than less opaque, ``white-box'' algorithms. To the extent this is true,
    a black-box algorithm will likely be more accurate. However, there has
    been significant work done to show that in many contexts, less opaque
    and highly interpretable models perform just as well, or virtually as
    well, as opaque models.
    ---------------------------------------------------------------------------
        The points outlined above about AVMs apply to virtually every
    algorithm making housing-related decisions. With regard to
    discrimination, an algorithm is not inherently discriminatory, but it
    can discriminate if it is given discriminatory data, poorly built, or
    used in the wrong context. However, as we saw in the context of AVMs,
    even if they suffer from known biases, they may still represent a
    better option than human-based decisions.
        Having used AVMs to illustrate the benefits and perils of using AI
    more generally, the next section outlines concrete steps that
    legislators and regulators can take to minimize likely harm while
    encouraging and fostering safe innovation in AI.
    The Role of Regulators and Policymakers in Ensuring Responsible
            Innovation in AI
        As we consider regulations for AI in housing, the primary goal
    should be to maximize the responsible use of this technology: given its
    potential to cause extreme harm at scale, safe and sound implementation
    of AI technologies is paramount. However, given its potential to
    represent a significant technological development that delivers real
    and meaningful societal benefits, we should also aim to minimize
    regulation's potential to be overly burdensome, possibly stifling
    innovation.
        Effectively accomplishing this goal is significantly more likely if
    we consider two key factors. First, any approach to regulation should
    not be overly prescriptive. Instead, we should focus on setting clear
    risk-based principles that encourage and enforce responsible AI
    development and use, where the most impactful systems (i.e., the
    systems with the most potential to harm or benefit people, society, or
    the environment) receive the most oversight. Second, it will be
    valuable to recognize that a significant body of existing work,
    regulation, and industry practice can be applied to AI systems to make
    them safer. Looking to these tried and validated frameworks and
    policies should guide our approach to making effective regulations for
    the use of AI.
    The Benefits of a Nonprescriptive Regulatory Environment
        A principles-based and less prescriptive approach to AI regulation
    can encourage innovation while ensuring the responsible development of
    AI. It recognizes the dynamic nature of technology and compliance and
    provides flexibility for continuous improvement and adaptation.
        A principles-based framework allows for innovation in both
    technology and compliance methods. On the compliance side, advancements
    like improved Less Discriminatory Alternatives (LDA) search and
    enhanced techniques for providing Adverse Action Notices (AANs)
    demonstrate how technology and compliance can complement each other and
    evolve together. Further, innovations in technology, such as Shapley
    values, explainable boosting machines, and Wells Fargo's Python
    Interpretable Machine Learning (PiML) package illustrate the rapid
    development of new AI tools and methods that encourage responsible
    model building. A less rigid but strong regulatory environment
    encourages such advancements.
        Overly prescriptive regulations, on the other hand, risk stifling
    innovation as they may lead to a ``design-around'' mentality, where the
    focus shifts from responsible development to merely meeting specific
    regulatory criteria. This can hinder genuine progress and the
    exploration of new AI and negate the desired helpful impact of the
    regulations. It also risks enforcing technical requirements that
    quickly drift into irrelevance as technology evolves.
    Key Principles To Consider for AI Regulation
        Four fundamental principles are foundational for adopting effective
    AI regulation: materiality, fairness, accountability, and transparency.
    Developing regulations with these as guideposts will help ensure that
    AI systems serve the public interest while advancing technological
    progress.
        Materiality:
         This principle advocates for a risk-based approach in governing AI
    systems. By focusing more stringent regulation on higher-risk AI
    applications, resources will be allocated more effectively. For
    example, a company should not spend as much time reviewing a marketing
    model as they would an underwriting model that enormously impacts both
    consumers and the business. Adopting such a risk-based approach ensures
    that systems with the most significant potential impact are carefully
    monitored and promotes innovation by not overburdening lower-risk
    initiatives with unnecessary regulatory constraints. As discussed
    below, SR 11-7 provides a solid foundation for guiding how materiality
    is assessed in AI regulation.
        Fairness:
         The principle of fairness is central to the responsible deployment
    of AI. Establishing a clear understanding and expectation of fair AI
    practices is crucial, particularly in applications that significantly
    impact individuals, such as housing. Regulators should set expectations
    that bias and discrimination should be identified and mitigated in AI
    systems. Existing frameworks relating to measuring and mitigating
    disparate impact, disparate treatment, and proxy discrimination should
    guide further regulation of AI fairness.
        Accountability:
         AI systems must have accountability mechanisms, especially those
    with high impact. This involves providing individuals affected by AI
    decisions a right to appeal, ensuring that there is recourse for those
    who may be adversely impacted. Additionally, entities that deploy AI
    systems irresponsibly should face appropriate consequences.
        Transparency:
         The principle of transparency mandates clear explanations for
    decisions made by AI systems. This is fundamental to building trust in
    AI systems. Understanding the ``why'' and ``how'' behind AI-driven
    decisions is crucial for public acceptance and confidence in these
    technologies, and is further crucial to ensure that systems are fair.
        By focusing on these critical principles--materiality, fairness,
    accountability, and transparency--we can create a regulatory
    environment that encourages the development of innovative AI
    technologies and safeguards against potential harms and biases. This
    principles-based approach to AI regulation is particularly pertinent in
    housing, where the impact of AI can have profound implications on
    people's lives and the fabric of communities. Next, I discuss a number
    of frameworks that can serve as guides for future regulation.
    Using Existing Regulations and Frameworks To Guide Further AI
            Regulation
        Congress and regulators will not need to devise laws and
    regulations from scratch to achieve the objectives that I have laid
    out. There are many regulations, standards, and frameworks with a
    proven track record of setting standards for human decisions related to
    AI, holding relevant actors accountable for those standards, and
    supporting the development and deployment of these technologies.
    Importantly, in many (but of course not all) cases, the industry has
    welcomed these for providing clear and reasonable standards. I discuss
    these at a high level below.

        SR 11-7, \11\ a supervision and regulation letter from the
            Board of Governors of the Federal Reserve System, constructs
            accountability mechanisms and organizational structures to
            ensure adequate and risk-based governance of credit modelers.
            While the document highlights the importance of technical
            processes such as testing and monitoring, its primary focus is
            on principles such as effective governance structures (e.g.,
            independent validation teams with high stature and strong
            financial incentives), risk management executives with
            independent reporting chains, and documentation requirements.
    ---------------------------------------------------------------------------
         \11\ https://www.federalreserve.gov/supervisionreg/srletters/
    sr1107.htm

        NIST SP 1270, \12\ a special publication from the National
            Institute of Standards and Technology (NIST), describes
            technical, process, and cultural problems and solutions
            relating to AI bias. It highlights that many aspects of data
            and AI systems are strongly influenced by human behavior and
            decisions, and suggests that approaches from model risk
            management (e.g., SR 11-7) coupled with more novel approaches,
            such as structured feedback activities (e.g., bug bounties or
            red teaming), human-centered design, and information sharing
            are strong mitigants for managing bias in AI systems. Another
            prominent theme of NIST SP 1270 is that basic scientific rigor
            in AI needs to be improved.
    ---------------------------------------------------------------------------
         \12\ https://nvlpubs.nist.gov/nistpubs/SpecialPublications/
    NIST.SP.1270.pdf

        The disparate impact, disparate treatment, and proxying
            framework is a legal doctrine that has been developed over the
            course of decades under the Fair Housing Act (FHA), the Equal
            Credit Opportunity Act (ECOA), and Title VII of the Civil
            Rights Act of 1964. This framework sets forth requirements for
            measuring discrimination that could be used in any decision
            tool (including AI or ML models) and provides a conceptual
            framework for mitigating any meaningful disparities found
            through conducting an LDA search. Many AI tools that affect
            consumers in the housing market are likely covered by this
            framework via the FHA. Setting the expectation that this
            framework would be extended to all high-risk AI use cases
            throughout the housing industry would ensure that companies
            move towards adopting fairer models. \13\
    ---------------------------------------------------------------------------
         \13\ Schmidt, Nicholas, and B. Stephens. ``An Introduction to
    Artificial Intelligence and Solutions to the Problem of Algorithmic
    Discrimination''. Conference on Consumer Finance Law (CCFL) Quarterly
    Report. Volume 73, Number 2 (October 2019). https://arxiv.org/abs/
    1911.05755

        The NIST AI Risk Management Framework \14\ puts forward
            four risk management functions for organizations: (1) Govern,
            (2) Map (understanding the risk of AI systems in their
            operational contexts with less emphasis on their development),
            (3) Measure, and (4) Manage; and seven trustworthy
            characteristics for AI systems: (1) Safe, (2) Valid and
            Reliable, (3) Accountable and Transparent, (4) Explainable and
            Interpretable, (5) Privacy-enhanced, (6) Fair with Harmful Bias
            Managed, and (7) Secure and Resilient. Governance guidance is
            largely aligned with the risk-based principles laid out in SR
            11-7 but introduces additional governance concepts from data
            privacy, information security, and more recent academic
            research. The AI Risk Management Framework has two distinct
            strengths: it acknowledges (1) that many AI risks arrive from
            real-world problems, not computer code bugs; and (2) that
            overlaps and connections between risks, risk controls, and
            governance must be recognized. It does all this while orienting
            governance toward traditional risk-based principles focusing on
            human accountability.
    ---------------------------------------------------------------------------
         \14\ https://www.nist.gov/itl/ai-risk-management-framework
    ---------------------------------------------------------------------------
    Conclusion
        The most significant harms associated with AI are not the
    fantastical scenarios often depicted in science fiction, but real-world
    issues such as discrimination, data privacy violations, unaccountable
    decision making, and fraudulent activities. Effectively regulating AI
    systems requires recognizing these facts. As AI evolves and impacts
    more aspects of housing, policymakers, regulators, public advocacy
    groups, and industry professionals must remain vigilant and proactive.
    We each play a key role in ensuring that AI systems are not only
    technically sound and effective, but are also fair, transparent, and
    accountable. This will require ongoing collaboration, research, and
    adaptation of regulatory approaches.
        Regulating AI so that it is used responsibly and safely and so that
    it can continue to benefit consumers and industry is a complex and
    crucial task. Materiality, fairness, accountability, and transparency
    provide a framework of principles that can serve as a touchstone for
    evaluating the effectiveness of proposed regulations. Further,
    leveraging existing regulations and frameworks such as SR 11-7, NIST SP
    1270, the disparate impact framework, and the NIST AI Risk Management
    Framework, regulators and legislators can create an environment that
    fosters innovation and protects against potential harm.
        The future of AI in housing presents both exciting possibilities
    and significant responsibilities. By embracing a principles-based
    regulatory approach, drawing on existing frameworks, and remaining open
    to continuous learning and improvement, we can build AI that reaches
    its potential to improve housing affordability, accessibility, and
    equity, while safeguarding the rights and interests of all individuals.
        Thank you again for the opportunity to share my insights and
    perspectives on this critical issue.

             RESPONSES TO WRITTEN QUESTIONS OF CHAIR SMITH
                             FROM LISA RICE

    Q.1. How do we build incentives into housing and regulatory
    systems so that we are maximizing fairness, accountability, and
    transparency as we try to set guidelines both to encourage
    innovation but also to make sure these systems are working for
    the public? Can and should this be handled legislatively? Dr.
    Perry underscored the importance of holding ``AI accountable
    for expanding opportunity,'' and I welcome legislative and
    regulatory recommendations for how to best incentivize this.

    A.1. Response not received in time for publication.

    Q.2. You underscored the bevy of existing laws that apply to
    and cover most of the AI applications in housing. What are the
    legislative gaps you see in this space, and how do you
    recommend that Congress addresses them?

    A.2. Response not received in time for publication.

                                    ------

           RESPONSES TO WRITTEN QUESTIONS OF SENATOR WARNOCK
                             FROM LISA RICE

    Q.1. The Fair Housing Act \1\ and Equal Credit Opportunity Act
    \2\ are landmark statutes to prevent discrimination in housing.
    Some Federal regulators have already used authorities under
    these statutes to engage in oversight of more advanced models
    that employ novel forms of data and more sophisticated
    statistical techniques. \3\
    ---------------------------------------------------------------------------
         \1\ See supra n. 1.
         \2\ See supra n. 2.
         \3\ See Press Release, supra n. 3.

    ---------------------------------------------------------------------------
    A.1. Response not received in time for publication.

    Q.2. In your view, are Federal regulators using all available
    authorities under FHA and ECOA to engage in oversight of novel
    software or models that employ machine learning (ML) or
    artificial intelligence (AI)? Why or why not?

    A.2. Response not received in time for publication.

    Q.3. Are there any ways in which existing statutes,
    authorities, or resources fail to enable policymakers and
    regulators to engage in oversight of novel software or models
    using ML and AI? If so, what additional authorities are most
    helpful? Please provide specific examples.

    A.3. Response not received in time for publication.

                                    ------

             RESPONSES TO WRITTEN QUESTIONS OF CHAIR SMITH
                           FROM VANESSA PERRY

    Q.1. How do we build incentives into housing and regulatory
    systems so that we are maximizing fairness, accountability, and
    transparency as we try to set guidelines both to encourage
    innovation but also to make sure these systems are working for
    the public?

    A.1. Existing regulatory frameworks in the lending industry,
    including, aspects of the Fair Housing Act, e.g., the disparate
    impact standard, \1\ and Affirmatively Furthering Fair Housing
    (AFFH), can be used to establish standards for fairness and
    accountability in Artificial Intelligence and Machine Learning
    (AI/ML) systems. Existing reporting requirements (e.g., under
    the Community Reinvestment Act, the Home Mortgage Disclosure
    Act, and the Federal Housing Finance Agency's conservatorship)
    collect data that can be used to evaluate outcomes, although
    expanded, real-time data, such as loan- and transaction-level
    detail, should be readily available to better inform regulators
    and policymakers about the impact these systems are having on
    the market.
    ---------------------------------------------------------------------------
         \1\ ``Reinstatement of HUD's Discriminatory Effects Standard'',
    Housing and Urban Development Department, 03/31/2023, https://
    www.federalregister.gov/documents/2023/03/31/2023-05836/reinstatement-
    of-huds-discriminatory-effects-standard.
    ---------------------------------------------------------------------------
        The issue of transparency is controversial among scholars
    who study ethical AI due to concerns about the complex and
    dynamic nature of AI/ML models. The Consumer Financial
    Protection Bureau (CFPB) has recently issued guidance
    elaborating on the Equal Credit Opportunity Act requirement
    that creditors explain the specific reasons for denying loans,
    even when complex AI/ML models are used. \2\ A distinguishing
    feature of AI models compared to algorithmic predecessors is
    that they often apply multiple factors simultaneously that can
    adapt to varying market conditions. Heretofore, these reasons
    have been listed in simple categories, e.g., credit history, or
    insufficient cash. In the case of AI/ML models, however, there
    could be numerous factors that may not matter by themselves but
    taken together result in loan rejection. To prevent consumer
    confusion, the CFPB should continue to provide standard
    categories of feedback, supported by empirical evidence on the
    messages that consumers are most likely to understand. At the
    same time, AI/ML models could be used to recommend actionable
    steps for denied applicants to take that would increase their
    likelihood of loan approval, e.g., how much debt reduction or
    savings would be necessary to qualify for a mortgage.
    ---------------------------------------------------------------------------
         \2\ CFPB Issues Guidance on Credit Denials by Lenders Using
    Artificial Intelligence, Consumer Financial Protection Bureau, Sept.
    19, 2023, https://www.consumerfinance.gov/about-us/newsroom/cfpb-
    issues-guidance-on-credit-denials-by-lenders-using-artificial-
    intelligence/.
    ---------------------------------------------------------------------------
        Financial regulators, e.g., the OCC, perform periodic
    reviews of underwriting models, and in the case of the
    Enterprises, model changes must be approved by FHFA in advance
    of adoption. To provide opportunities for innovation as well as
    regulatory oversight, President Biden's recent executive order
    on AI relies on voluntary commitments from AI companies and
    ``red-team-style'' \3\ audits, but regulatory requirements and/
    or tax incentives could be instituted to encourage firms to
    participate.
    ---------------------------------------------------------------------------
         \3\ Red-teaming is the process of testing model effectiveness by
    applying an adversarial lens to your organization, https://www.ibm.com/
    blog/red-teaming-101-what-is-red-teaming/.

    Q.2. Can and should this be handled legislatively? You
    underscored the importance of holding ``AI accountable for
    expanding opportunity,'' and I welcome legislative and
    ---------------------------------------------------------------------------
    regulatory recommendations for how to best incentivize this.

    A.2. To ensure that AI/ML models accomplish the goal of
    expanding opportunity, \4\ legislation could be established,
    mirroring the Affirmatively Furthering Fair Housing (AFFH)
    provision of the 1968 Fair Housing Act. AFFH requires that
    entities receiving Federal funding related to housing must take
    active steps to promote fair housing practices and combat
    housing discrimination, based on an analysis of impediments to
    housing options in their jurisdictions. A similar mandate could
    be incorporated into H.R. 6580, i.e., the proposed Algorithmic
    Accountability Act of 2022, \5\ stipulating that predeployment
    impact assessments identify how an AI system expands
    opportunity for underrepresented groups (e.g., low-income,
    rural area residents, residents in historically redlined
    neighborhoods). Organizations would subsequently be evaluated
    against these goals at the post-implementation evaluation stage
    (Algorithmic Accountability Act, Sect. 3). \6\ To strengthen
    these incentives, additional legislation could create Expanded
    Opportunity Tax Credits for organizations which can demonstrate
    that the adoption of a new system has significantly impacted
    specific goals--a model that combines approaches from Low-
    Income Housing Tax Credits and the affordable housing goals
    established for Fannie Mae and Freddie Mac. In this example, a
    mortgage lender would qualify for a tax credit if they could
    demonstrate that a new AI/ML system would result in a higher
    rate of mortgage approvals for low- and moderate-income
    homebuyers, or for residents of rural or other underserved
    areas.
    ---------------------------------------------------------------------------
         \4\ Perry, V., Kirsten Martin, and Ann Schnare (2023),
    ``Algorithms for All: Can AI in the Mortgage Market Expand Access to
    Homeownership?'', AI, 4(4), 888-903, https://doi.org/10.3390/ai4040045.
         \5\ H.R. 6580, To direct the Federal Trade Commission to require
    impact assessments of automated decision systems and augmented critical
    decision processes, and for other purpose, Feb. 3, 2022, https://
    www.congress.gov/117/bills/hr6580/BILLS-117hr6580ih.pdf.
         \6\ Mokander, J., Juneja, P., Watson, D.S., et al. (2022), ``The
    U.S. Algorithmic Accountability Act of 2022 vs. The EU Artificial
    Intelligence Act: What Can They Learn From Each Other?'' Minds &
    Machines 32, 751-758. https://doi.org/10.1007/s11023-022-09612-y
    ---------------------------------------------------------------------------
        H.R. 6580, known as the Algorithmic Accountability Act,
    directs the Federal Trade Commission (FTC) to require impact
    assessments of automated decision systems and augmented
    critical decision processes. Rather than place the entire
    burden of these assessments on the FTC, all regulators of
    entities in affected industries, e.g., housing, credit, and
    education, could implement these assessments. This would allow
    specialization and alignment with the agencies' mission and
    goals.

    Q.3. We know financial technology companies and mortgage
    lenders are exploring new ways to incorporate AI into
    underwriting and property valuation. What roles do Fannie Mae
    and Freddie Mac, as GSEs, have to play in shaping industry
    implementation and responsibility for AI technology? \7\
    ---------------------------------------------------------------------------
         \7\ Neal, Michael, Linna Zhu, Caitlin Young, Vanessa G. Perry,
    Matthew Pruitt (2023a), ``Harnessing Artificial Intelligence for Equity
    in Mortgage Finance'', Urban Institute, November 6, https://
    www.urban.org/research/publication/harnessing-artificial-intelligence-
    equity-mortgage-finance.

    A.3. Approximately 70 percent of mortgage loans in the U.S. are
    financed by Fannie Mae and Freddie Mac, and most mortgage
    lenders utilize their proprietary underwriting systems. As
    conservator of the Enterprises, the Federal Housing Finance
    Administration (FHFA) has issued guidance for the managing
    risks of AI/ML systems \8\ and has approved the use of AL and
    ML models for certain applications, e.g., fraud detection,
    property valuation, and loan servicing activities. However,
    according to a recent study on AI in the mortgage industry,
    neither the Enterprises, the CFPB, nor other regulatory
    entities have provided specific standards for downstream
    adoption and management of AI by lenders, seller/servicers of
    the Enterprises, or other intermediaries in the mortgage value
    chain. Because the underwriting standards of the Enterprises
    are the primary drivers of lending practices industrywide, and
    because of the significant legal and financial risks of
    noncompliance, lenders and other market participants have been
    reluctant to experiment with AI-based innovations. \9\ As a
    result of this uncertainty, these lenders may miss
    opportunities to leverage AI/ML to advance opportunities for
    currently underserved market segments, such as models to expand
    credit scoring or increase fairness in property valuation.
    ---------------------------------------------------------------------------
         \8\ Advisory Bulletin on Artificial Intelligence/Machine Learning
    Risk Management, Federal Housing Finance Agency, AB 2022-02, Feb. 10,
    2022, https://www.fhfa.gov/SupervisionRegulation/AdvisoryBulletins/
    Pages/Artifical-Intelligence-Machine-Learning-Risk-Management.aspx.
         \9\ Neal, et al. (2023a).
    ---------------------------------------------------------------------------
        Because of the Federal Government's close monitoring and
    oversight of the Enterprises, as well as their access to data,
    model development, and computing resources, they are in a
    better position than individual lenders to lead AI/ML
    innovation in the industry via pilot programs. For example,
    Freddie Mac has used AI/ML to incorporate rental and other
    payment data sources in their models, and partnering with
    Fintech companies has been one approach to expediting the
    development and testing of these tools. \10\
    ---------------------------------------------------------------------------
         \10\ ``Zest AI Joins Forces With Freddie Mac To Help Make
    Homeownership Possible for More Americans'', PR Newswire, Nov. 18,
    2020, https://www.prnewswire.com/news-releases/zest-ai-joins-forces-
    with-freddie-mac-to-help-make-homeownership-possible-for-more-
    americans-301176336.html.
    ---------------------------------------------------------------------------
        It is important to note that historically, changes to
    underwriting models that require FHFA approval have taken years
    to implement, failing to meet the regulatory demands posed by
    the rapid pace of AI/ML innovations. For example, it has taken
    more than 5 years for the Enterprises to update their credit
    score requirement despite compelling evidence that these
    changes would have a significant and immediate impact on
    homeownership access for marginalized communities. \11\
    ---------------------------------------------------------------------------
         \11\ ``Fact Sheet: Credit Score Models and Credit Report
    Requirements'', Federal Housing Finance Agency, Mar. 23, 2023, https://
    www.fhfa.gov/Media/PublicAffairs/Pages/Fact-Sheet-FHFA-Announcement-on-
    Credit-Score-Models-March-2023.aspx.
    ---------------------------------------------------------------------------
        It is also important to understand the influence of the
    Federal Housing Administration (FHA) on AI adoption in the
    mortgage industry. The FHA Technology Open to Approved Lenders
    (TOTAL) Scorecard has been developed using AI/ML and is used
    widely by mortgage lenders along with Freddie Mac and Fannie
    Mae's underwriting systems. Relative to the conventional market
    dominated by the Enterprises, FHA insures a disproportionate
    share of mortgages for low- and moderate-income, first-time
    homebuyers and historically underrepresented racial groups. For
    example, in 2023, FHA's percentage of mortgages to Black and
    Hispanic borrowers was 30.6 percent, \12\ compared to 15.7 and
    20.7 percent for Freddie Mac and Fannie Mae, respectively. \13\
    FHA issues guidance for lenders on how to implement the FHA's
    AI-based systems but does not address lenders' use of their own
    AI applications. \14\
    ---------------------------------------------------------------------------
         \12\ ``Fact Sheet: FHA's Impact in 2023--Making Homeownership
    Possible for Hundreds of Thousands of Families'', U.S. Department of
    HUD, Nov. 15, https://www.hud.gov/sites/dfiles/PA/documents/Fact-Sheet-
    MMI-Rollout.pdf.
         \13\ ``2023 Annual Housing Activities Report'', Freddie Mac,
    https://www.freddiemac.com/about/pdf/2023-annual-housing-activities-
    report.pdf; 2023 Annual Mortgage Report, Fannie Mae, https://
    www.fanniemae.com/media/50736/display.
         \14\ Neal, Michael, Janneke Ratcliffe, and Matthew Pruitt (2023b),
    ``AI Could Alter Mortgage Lending, but Government Leadership Is
    Needed'', Urban Wire, Urban Institute, November 6, https://
    www.urban.org/urban-wire/ai-could-alter-mortgage-lending-government-
    leadership-needed.
    ---------------------------------------------------------------------------
        The U.S. Department of Housing and Urban Development (HUD)
    also influences the adoption and oversight of AI in its role as
    regulator of the Enterprises' fair lending activities along
    with FHFA. Significant, additional investments in technology
    infrastructure and expertise will be necessary for HUD and the
    Department of Justice to investigate violations or otherwise
    enforce the Fair Housing Act when AI and ML are involved.
    Currently, the agency relies heavily on third-party vendors for
    fair lending testing of these models. \15\
    ---------------------------------------------------------------------------
         \15\ Hoffberg, Adam, and Bill Reeder (2023), ``The FHA TOTAL
    Mortgage Algorithm: Providing Machine-Learning Analysis for Two
    Decades'', PD&R Edge, U.S. Dept. of HUD, June 27, https://
    www.huduser.gov/portal/pdredge/pdr-edge-pdrat50-062723.html.

    Q.4. Additionally, smaller, mission-oriented lenders such as
    Minority Depository Institutions and Community Development
    Financial Institutions tend to serve low-to-moderate income
    communities and minorities at higher proportions than other
    institutions. But we know, from one of your reports, that they
    have been slower to adopt AI than other financial institutions.
    Can you please describe this technology gap, and if it could
    further reinforce racial disparities in the mortgage industry
    ---------------------------------------------------------------------------
    and what should be done about it?

    A.4. According to a recent study, \16\ larger lenders are more
    likely to adopt AI/ML systems due to greater capacity to make
    the necessary investments in technology and human capital, used
    towards in-house or third-party resources. Fintech companies,
    for example, can charge anywhere from several thousand to
    hundreds of thousands of dollars per year to develop models
    used for customer service, underwriting, or fraud detection,
    not including additional ongoing costs for maintenance and
    administration. Taken together, these costs present significant
    barriers to AI adoption for smaller and mission-oriented
    lenders, such as CDFIs. Anecdotal evidence also suggests that
    these smaller institutions rely more heavily on relationship-
    based approaches which are more labor-intensive, less
    quantifiable, and more difficult to replicate using algorithms
    designed for the mainstream mortgage market. However, one of
    the intended benefits of AI/ML is to incorporate nontraditional
    data sources in ways that can expand access to the mortgage
    market; another potential benefit is lower costs due to
    increased efficiency. Due to these technology gaps, smaller
    institutions could miss out on opportunities that AI presents
    to create mission-focused scale economies.
    ---------------------------------------------------------------------------
         \16\ Neal, et al. (2023a).
    ---------------------------------------------------------------------------
        To ensure that smaller lenders have access to these
    technologies, regulatory agencies should partner with nonprofit
    housing organizations to provide technical assistance grants,
    similar to the Community Development Financial Institutions
    (CDFI) Program administered by the U.S. Department of the
    Treasury. This program provides grants to support investments
    in technology, capacity building, and new product development.

                                    ------

           RESPONSES TO WRITTEN QUESTIONS OF SENATOR WARNOCK
                           FROM VANESSA PERRY

    Q.1. The Fair Housing Act \1\ and Equal Credit Opportunity Act
    \2\ are landmark statutes to prevent discrimination in housing.
    Some Federal regulators have already used authorities under
    these statutes to engage in oversight of more advanced models
    that employ novel forms of data and more sophisticated
    statistical techniques. \3\
    ---------------------------------------------------------------------------
         \1\ See supra n. 1.
         \2\ See supra n. 2.
         \3\ See Press Release, supra n. 3.
    ---------------------------------------------------------------------------
        In your view, are Federal regulators using all available
    authorities under FHA and ECOA to engage in oversight of novel
    software or models that employ machine learning (ML) or
    artificial intelligence (AI)? Why or why not?

    A.1. Federal regulators, such as the U.S. Department of Housing
    and Urban Development (HUD), the U.S. Department of Justice
    (DOJ), and the Consumer Financial Protection Bureau (CFPB),
    have leveraged their authority under the Fair Housing Act and
    the Equal Credit Opportunity Act (ECOA) to regulate the use of
    Artificial Intelligence and Machine Learning models (AI//ML) in
    the mortgage industry. For example, in 2019 Facebook settled a
    Fair Housing Act case filed by HUD. The case alleged that
    Facebook's AI-driven advertising platform enabled
    discriminatory practices, such as targeting or excluding market
    segments based on prohibited factors, such as race, gender,
    nationality, religion, familial status, and disability. This
    landmark case has led to heightened awareness, increased
    scrutiny, and regulatory oversight of advertising practices in
    the mortgage industry. In addition, social media and digital
    marketing companies have instituted stricter policies to ensure
    compliance with fair housing laws.
        Additionally, in 2023, the CFPB, along with the DOJ, the
    Federal Trade Commission (FTC), and the Equal Employment
    Opportunity Commission (EEOC), issued a joint statement
    emphasizing their intentions to apply existing
    antidiscrimination laws and regulations to AI models,
    particularly when used for credit decisions and digital
    marketing of financial services. \4\ Later in 2023, the CFPB
    issued guidance reinforcing that existing legal requirements
    under ECOA for lenders to provide specific reasons for credit
    denials apply in the case of AI models. \5\ These examples
    underscore the potential for existing statutes to apply to
    emerging technological applications, such as AI/ML.
    ---------------------------------------------------------------------------
         \4\ Khan, Linda (2023), Joint Statement on Enforcement Efforts
    Against Discrimination and Bias in Automated Systems, Federal Trade
    Commission, April 25, https://www.ftc.gov/legal-library/browse/cases-
    proceedings/public-statements/joint-statement-enforcement-efforts-
    against-discrimination-bias-automated-systems.
         \5\ CFPB Issues Guidance on Credit Denials by Lenders Using
    Artificial Intelligence, CFPB, Sept. 13, https://
    www.consumerfinance.gov/about-us/newsroom/cfpb-issues-guidance-on-
    credit-denials-by-lenders-using-artificial-intelligence/.
    ---------------------------------------------------------------------------
        While these are important signals to the industry about
    best practices and legal requirements, these agencies can only
    take action after alleged violations have occurred and
    complaints have been filed. Given resource constraints
    including limited in-house expertise in evaluating AI/ML
    models, more efforts should be placed on ensuring that AI/ML
    models align with prevailing legal requirements and established
    risk management principles before implementation, such as the
    predeployment impact assessments proposed in the Algorithmic
    Accountability Act of 2022. \6\
    ---------------------------------------------------------------------------
         \6\ H.R. 6580, https://www.congress.gov/117/bills/hr6580/BILLS-
    117hr6580ih.pdf.

    Q.2. Are there any ways in which existing statutes,
    authorities, or resources fail to enable policymakers and
    regulators to engage in oversight of novel software or models
    using ML and AI? If so, what additional authorities are most
    ---------------------------------------------------------------------------
    helpful? Please provide specific examples.

    A.2. Resource constraints, regulatory bureaucracy, and
    statutory restrictions explain why existing statutes,
    authorities, or resources limit the ability of policymakers and
    regulators to oversee novel software or models using ML and AI.
    For example, under the Fair Housing Act and ECOA, HUD, and the
    CFPB, respectively, have subpoena power and can make referrals
    to the DOJ, but the DOJ is only able to make information
    requests. States have subpoena power but must defer to Federal
    regulators for cases against national banks. Congress could
    increase the effectiveness of enforcement efforts by expanding
    the subpoena power of the DOJ and the States.
        A Federal judge recently dismissed a case filed by the CFPB
    alleging that a mortgage company's communications would deter
    prospective Black mortgage applicants, ruling that the ECOA
    only applies to actual loan applicants, in contrast to the Fair
    Housing Act, which covers potential applicants. \7\ While this
    case did not relate to the use of AI/ML, it has implications
    for the ability to enforce AI/ML marketing practices. A
    legislative solution would be to align ECOA with the Fair
    Housing Act.
    ---------------------------------------------------------------------------
         \7\ Andreano, Jr., Richard (2023), ``CFPB Suffers Significant
    Defeat in ECOA Lawsuit Against Townstone Mortgage'', Consumer Finance
    Monitor, Feb. 6, https://www.consumerfinancemonitor.com/2023/02/06/
    cfpb-suffers-significant-defeat-in-ecoa-lawsuit-against-townstone-
    mortgage/.
    ---------------------------------------------------------------------------
        The Fair Housing Act's ``disparate impact'' rule prohibits
    the institution of lending policies, including the use of AI/ML
    models, that have an unjustified discriminatory effect based on
    protected characteristics. Similarly, ECOA prohibits the use of
    protected demographic characteristics in credit decisions.
    These regulations require lenders to legitimize actions that
    deny access to credit and housing opportunities but would be
    more impactful if expanded to stipulate an affirmative
    obligation by federally regulated entities to expand access to
    credit and housing opportunities.

                                    ------

             RESPONSES TO WRITTEN QUESTIONS OF CHAIR SMITH
                         FROM NICHOLAS SCHMIDT

    Q.1. How do we build incentives into housing and regulatory
    systems so that we are maximizing fairness, accountability, and
    transparency as we try to set guidelines both to encourage
    innovation but also to make sure these systems are working for
    the public? Can and should this be handled legislatively? Dr.
    Perry underscored the importance of holding ``AI accountable
    for expanding opportunity,'' and I welcome legislative and
    regulatory recommendations for how to best incentivize this.

    A.1. It is essential to recognize that innovation and the
    principles of fairness, accountability, and transparency do not
    conflict. In fact, these principles are the catalysts that
    drive sustainable and impactful innovation. Rather than
    hindering progress, these values guide the development of
    technologies that will ultimately lead to successful
    innovation, serve the public good, and ensure equitable
    outcomes. Fairness is an integral component of successful
    innovation. Innovations that exacerbate disparities based on
    race, gender, age, or other categories cannot be deemed
    successful. Implementing standards that mandate testing and
    validation of the fairness of models mitigates risk and
    promotes innovation that benefits all segments of society. The
    importance of accountability in fostering successful innovation
    becomes evident when we reflect on the causes and responses to
    the 2008-2009 housing crisis. The lack of accountability by
    individuals making investment decisions was a significant
    factor in the crisis. However, the resilience of the lending
    industry since then can be attributed, in part, to enhanced
    regulatory standards for model risk management introduced by
    the Federal Reserve and the Office of the Comptroller of the
    Currency in 2011. These standards have significantly improved
    the vetting, testing, and monitoring of algorithmic decision-
    making systems, and our economy's success demonstrates that
    accountability is a cornerstone of robust and responsible
    innovation. Transparency, likewise, plays a pivotal role in
    promoting responsible innovation within the housing sector. AI
    is, without a doubt, overhyped. As a result, there has been a
    tendency to implement AI systems without proper oversight in
    many industries. By encouraging and requiring more
    transparency, regulators and legislators will push companies to
    consider whether an insufficiently tested system is worth the
    potential for reputational harm that might result from
    disclosure through transparency requirements. For example, a
    modeler might want to use an unnecessarily and overly complex
    algorithm to assess credit quality (I have seen unnecessarily
    complex algorithms put into production far too frequently in
    unregulated industries). However, suppose the company
    understands that it has transparency requirements and must be
    able to explain why the algorithm made a decision. If the
    algorithm is too complex, the company cannot reasonably explain
    the algorithm's decisions. This may drive business and
    compliance leaders of the company to refuse to allow such a
    complex model to be used. This is beneficial because overly
    complex algorithms are often unfair and unsafe. Being
    encouraged to push back against the desire to ``move fast and
    break things'' will lead to a more considered and mature
    adoption of AI.
        While fairness, accountability, and transparency are
    compatible with and necessary for successful innovation, it is
    crucial to acknowledge the potential for legislative efforts to
    inadvertently impede progress. Successful legislation should be
    principled rather than prescriptive, offering a framework that
    fosters innovation while ensuring it operates within ethical
    boundaries. This approach allows flexibility and adaptation,
    which are essential for nurturing innovation in a rapidly
    evolving technological landscape.

    Q.2. You underscored the bevy of existing laws that apply to
    and cover most of the AI applications in housing. What are the
    legislative gaps you see in this space, and how do you
    recommend that Congress addresses them?

    A.2. With exceptions related to the provision of transparent
    explanations, standards around proxy discrimination, and
    mandating that modelers search for less discriminatory
    alternative models (which I outline in my responses to Senator
    Warnock), large lending institutions are generally well-covered
    by existing laws. However, smaller companies, including third-
    party model providers, have generally not been subject to the
    rigorous standards against which large institutions are
    assessed in terms of model governance, model quality, fairness,
    and safety. While it is likely unreasonable to require that
    smaller companies meet the compliance standards of the top
    banks, they should be subject to more rigorous standards and
    expectations. Creating legislation that provides more funding
    for existing regulators to understand these issues and
    investigate companies that do not meet minimum standards would
    help ensure that these models, which have a significant effect
    on people's ability to rent apartments or buy homes, would be
    fairer and more effective.

                                    ------

           RESPONSES TO WRITTEN QUESTIONS OF SENATOR WARNOCK
                         FROM NICHOLAS SCHMIDT

    Q.1. As credit underwriting models increasingly employ newer
    and larger sets of data, I am concerned that historical
    discrimination in housing may lead to biased training data.
        What is your current assessment of the quality of training
    data available for those building credit models?

    A.1. The quality of training data used to construct credit
    models is often inconsistent. Data quality is generally high
    for individuals with sufficient historical credit history to be
    evaluated through traditional credit scoring models, such as
    FICO or Vantage. While this observation may diverge from
    narratives frequently encountered in the press, credit bureaus
    maintain robust standards for accuracy and offer mechanisms for
    data dispute and correction. This achievement is, in part, a
    testament to effective legislative frameworks that mandate
    transparency and consumer rights to data access.
        Conversely, the emergent forms of data increasingly
    incorporated into credit assessments--particularly for those
    lacking a traditional credit score--exhibit varying degrees of
    quality. The issues stem from several factors, including the
    nascent stage of integrating such data into credit algorithms,
    the relative absence of mandates for data disclosure and
    correction, and the usage of complex algorithms that may
    obfuscate the impact of erroneous data on consumer credit
    evaluations. The latter of these issues may require significant
    efforts to uncover and address data inaccuracies, an effort
    that is often not sufficiently addressed by some model builders
    or users.
        Further, the issue of data incompleteness is more
    pronounced in these newer data streams. The dependency on
    third-party data providers that may not consistently capture
    information across all consumer segments exacerbates this
    challenge. Notably, discrepancies in data completeness often
    manifest along racial lines, underscoring the need for vigilant
    review and monitoring to prevent systemic biases.
        Despite these challenges, it is also crucial to recognize
    the potential benefits of incorporating alternative data
    sources. For individuals with limited or no traditional credit
    history, access to such data enables financial institutions to
    extend credit and housing opportunities that would otherwise be
    inaccessible. This can significantly enhance financial equity
    and provide opportunities to underserved populations.
    Therefore, while maintaining vigilance and concern about data
    quality and its associated fairness, it is essential to
    consider that its use often promotes financial inclusion.

    Q.2. What standards are currently used by industry to assess an
    AI model's quality in terms of its data inputs as well as other
    model characteristics (e.g., accuracy, bias and discrimination,
    variance, performance on sparse data, etc.)?

    A.2. Before machine learning (ML) and AI became widespread,
    most major financial services companies had robust processes to
    assess the fairness and reasonableness of decision making
    (whether subjective or algorithmic). As algorithmic decisioning
    has become more common, most companies have successfully
    applied these processes to their machine learning and AI
    algorithms. In particular, the legal framework of disparate
    treatment, disparate impact, and proxy discrimination is the
    most commonly used framework to assess fairness. This framework
    assesses both the data used to build the algorithm and the
    fairness of the algorithm's output.
        This framework has a rich history. While originating in the
    employment context, it has been extended to housing, lending,
    and other industries, including insurance and technology. Its
    advantages include its comprehensive yet flexible nature. It
    also addresses how discrimination has manifested in our
    society, and it provides an avenue to encourage inclusion
    without being unnecessarily burdensome. Finally, it is well-
    designed to be applied to questions of fairness and inclusion
    in algorithmic decisioning by machine learning and AI.
        AI, ML, and the data used for these algorithms present some
    challenges to applying this framework, particularly in
    assessing proxy discrimination and disparate impact.
    Institutions would benefit from having more clarity around what
    types of data they can use and what they are expected to do to
    minimize any disparities in their models.
        Proxy discrimination involves making a lending decision
    based at least partially on a factor so closely related to
    membership in a protected class that it is essentially just a
    stand-in for that class. The classic example of proxy
    discrimination is redlining, which is the refusal to lend in
    certain neighborhoods that were almost always heavily minority
    neighborhoods. Lenders justified this by claiming that these
    neighborhoods were riskier to lend in than neighborhoods that
    were majority White. This was a racist policy, and lenders go
    to great lengths to avoid obvious proxies like this.
        However, in machine learning and AI, modelers use far more
    data than traditional credit bureau attributes, which are much
    less likely to be proxies for a protected class. When building
    these models, modelers may want to incorporate information
    about such varied things as marketing preferences, shopping
    preferences, and patterns of speech or writing that may cross
    the line and become proxies for a protected class. While it may
    seem clear that these variables should be avoided if they might
    proxy for a protected class, the question becomes more
    difficult for at least two reasons. First, using these
    variables may increase financial inclusion for people without
    sufficient traditional credit history--even for members of the
    protected class proxied by the variable. The question may be
    whether to deny a loan to everyone without sufficient credit
    history or to use these variables, accept some people without
    sufficient credit history, but accept relatively fewer members
    of a protected class than truly should be accepted if better
    data existed. The answer to this question is ultimately a
    public policy question that individual companies in the housing
    industry cannot answer on their own without encountering
    significant regulatory and legal risks. Thus, having clearer
    regulations around the use of potentially problematic variables
    would be beneficial.
        The second reason why this is a difficult question is that
    there is not a clear dividing line between disparate impact and
    proxy discrimination. While we can make some distinctions on
    certain variables, others are less obvious. For example, a
    variable that indicates where a person shops for clothing is
    almost certainly a proxy for sex. However, a variable that
    indicates the percent of disposable income a person spends on
    clothing may not be a proxy. Suppose the second variable is
    still correlated with sex, but it is also a robust predictor of
    whatever outcome is being modeled. In that case, it might be
    considered a source of disparate impact, not proxy
    discrimination. As such, it is not necessarily illegal. If the
    first variable does represent a proxy risk and the second does
    not, then it is obvious that there is some line between the two
    where a variable moves from being a cause of disparate impact
    to being a cause of proxy discrimination. The difficulty is
    that there is no clear guidance on where that line is. Industry
    participants would benefit from regulations that set out
    clearer guidance on what constitutes proxy discrimination.
        Machine learning and AI also present new issues related to
    disparate impact. While disparate treatment represents the
    explicit inclusion of protected class status into a decision,
    disparate impact is perhaps a subtler form of discrimination.
    In a disparate impact analysis of a model, we start with a very
    simple test. In the case of a yes/no outcome, we test whether a
    protected group receives the favorable outcome less frequently
    than a member of some reference group. For example, we test
    whether Black or African American applicants receive loan
    offers less frequently than non-Hispanic White applicants. In
    the case of a continuous outcome, such as interest rates paid
    for a loan, we assess whether the protected group has, on
    average, received worse terms than the reference group. Using
    interest rates as an example, we might test whether women pay
    higher APRs than men.
        Importantly, if we find evidence of unfavorable
    differences, we do not necessarily assume that the algorithm is
    illegally discriminatory. Instead, we move to the next step of
    assessing whether the algorithm's use has sufficient business
    justification and that none of the variables in the model
    represent disparate treatment or proxy discrimination. For
    example, a well-built credit underwriting model that has passed
    rigorous model governance standards and uses traditional credit
    bureau data would almost certainly pass this hurdle.
        If the model passes this second stage, we then test whether
    a similar process can still meet the institution's legitimate
    business needs but have a smaller negative impact on protected
    class members. Searching for such an alternative model is
    called a ``search for less discriminatory alternative models''
    or ``LDA search.''
        Machine learning and AI have significantly extended the
    ability to effectively search for LDA models. In traditional
    models, the approach was to try adding or dropping a handful of
    variables and see whether that might decrease disparities while
    maintaining model quality. While sometimes effective, this
    approach often did not result in significant changes. In the
    new age of ML and AI, we have developed new techniques that
    have greatly expanded the ability to search for LDA models.
    Adopting such techniques can significantly increase equitable
    and fair outcomes in housing.
        While we have seen significant positive results from
    implementing these techniques, there is a lack of regulatory
    clarity around whether this is required, what techniques are
    allowed, and what standards will meet regulatory expectations.
    Legislation that clearly outlines that searching for less
    discriminatory models is a requirement and allows regulators to
    set the guidelines for what is acceptable will encourage
    industry participants to adopt these approaches. This could
    drive significant and positive change by making housing
    outcomes fairer and more equitable.

    Q.3. In your written testimony, you stated, ``Many . . . AI and
    ML algorithms are described as `black-box' processes . . .
    [W]hile we understand what data went into the algorithm, and we
    can see what output of the algorithm is, it is very difficult
    to understand how or why the algorithm made the prediction it
    made.'' \1\ You also note that ``a lender might not be able to
    provide a borrower with a reliable explanation for why the home
    they wanted to buy received a low appraisal.'' \2\
    ---------------------------------------------------------------------------
         \1\ Testimony of Nicholas Schmidt, U.S. Senate Committee on
    Banking, Housing, and Urban Affairs, Subcommittee on Housing,
    Transportation, and Community Development (Jan. 31, 2024), https://
    www.banking.senate.gov/imo/media/doc/schmidt-testimony-1-31-24.pdf at
    8.
         \2\ Id.
    ---------------------------------------------------------------------------
        In your experience, how commonly are ``black-box''
    algorithms used in housing?

    A.3. These algorithms are now quite commonly used in housing.
    They are used throughout the industry, most notably for
    marketing, appraisals, credit underwriting, loan pricing,
    evaluating renters, and servicing loans.

    Q.4. How can regulators better ensure that lenders and other
    institutions use models capable of providing transparent
    explanations for their decisions?

    A.4. Regulators may be able to encourage lenders to use
    appropriately transparent algorithms by assessing and judging
    them based on the quality of their models' explanations. If a
    lender knows it must give regulators evidence that its
    algorithms provide reasonably accurate explanations, it will be
    less likely to use an unnecessarily complex algorithm.
        The difficulty is that, even with relatively simple
    algorithms, all explanation methodologies have shortcomings;
    even the most robust approach will not always provide the best
    explanation for why an algorithm made a decision. This lack of
    perfection should not be a reason to reject AI and machine
    learning because well-built machine learning or AI algorithms
    significantly benefit consumers and industry participants.
        Regulators have not been clear about what constitutes a
    sufficiently robust explanation methodology. While this lack of
    clarity has been reasonable given the pace of technological
    change, we are now at a point where better guidance would
    benefit consumers and the industry. However, rather than
    prescribing a particular methodology, effective regulation
    would put forth methodologies for testing and evaluating the
    robustness of explanations. A company could then be evaluated
    on how well it tested its methodology and whether that
    methodology passed the tests it constructed. The benefits of
    this approach include that it would allow continued innovation
    by not prescribing an approach that may quickly find itself out
    of date while also providing companies with a clearer standard
    against which they can judge themselves.
                  Additional Material Supplied for the Record

    [GRAPHIC(S) NOT AVAILABLE IN TIFF FORMAT]