[Senate Hearing 118-643]
[From the U.S. Government Publishing Office]
S. Hrg. 118-643
ARTIFICIAL INTELLIGENCE AND HOUSING:
EXPLORING PROMISE AND PERIL
=======================================================================
HEARING
before the
SUBCOMMITTEE ON
HOUSING, TRANSPORTATION, AND COMMUNITY DEVELOPMENT
of the
COMMITTEE ON
BANKING,HOUSING,AND URBAN AFFAIRS
UNITED STATES SENATE
ONE HUNDRED EIGHTEENTH CONGRESS
SECOND SESSION
ON
EXAMINING ARTIFICIAL INTELLIGENCE AND HOUSING
__________
JANUARY 31, 2024
__________
Printed for the use of the Committee on Banking, Housing, and Urban
Affairs
[GRAPHIC NOT AVAILABLE IN TIFF FORMAT]
Available at: https: //www.govinfo.gov /
______
U.S. GOVERNMENT PUBLISHING OFFICE
60-366 PDF WASHINGTON : 2026
COMMITTEE ON BANKING, HOUSING, AND URBAN AFFAIRS
SHERROD BROWN, Ohio, Chair
JACK REED, Rhode Island TIM SCOTT, South Carolina
ROBERT MENENDEZ, New Jersey MIKE CRAPO, Idaho
JON TESTER, Montana MIKE ROUNDS, South Dakota
MARK R. WARNER, Virginia THOM TILLIS, North Carolina
ELIZABETH WARREN, Massachusetts JOHN KENNEDY, Louisiana
CHRIS VAN HOLLEN, Maryland BILL HAGERTY, Tennessee
CATHERINE CORTEZ MASTO, Nevada CYNTHIA M. LUMMIS, Wyoming
TINA SMITH, Minnesota J.D. VANCE, Ohio
RAPHAEL G. WARNOCK, Georgia KATIE BOYD BRITT, Alabama
JOHN FETTERMAN, Pennsylvania KEVIN CRAMER, North Dakota
LAPHONZA R. BUTLER, California STEVE DAINES, Montana
Laura Swanson, Staff Director
Lila Nieves-Lee, Republican Staff Director
Elisha Tuku, Chief Counsel
Amber Beck, Republican Chief Counsel
Cameron Ricker, Chief Clerk
Shelvin Simmons, IT Director
Pat Lally, Assistant Clerk
______
Subcommittee on Housing, Transportation, and Community Development
TINA SMITH, Minnesota, Chair
CYNTHIA M. LUMMIS, Wyoming, Ranking Member
JACK REED, Rhode Island MIKE CRAPO, Idaho
ROBERT MENENDEZ, New Jersey MIKE ROUNDS, South Dakota
JON TESTER, Montana JOHN KENNEDY, Louisiana
CATHERINE CORTEZ MASTO, Nevada BILL HAGERTY, Tennessee
KYRSTEN SINEMA, Arizona J.D. VANCE, Ohio
RAPHAEL G. WARNOCK, Georgia KATIE BOYD BRITT, Alabama
JOHN FETTERMAN, Pennsylvania
Tim Everett, Subcommittee Staff Director
Kelsey Pristach, Republican Subcommittee Staff Director
(ii)
C O N T E N T S
----------
WEDNESDAY, JANUARY 31, 2024
Page
Opening statement of Chair Smith................................. 1
Prepared statement....................................... 23
Opening statements, comments, or prepared statements of:
Senator Lummis................................................... 3
WITNESSES
Lisa Rice, President and Chief Executive Officer, National Fair
Housing Alliance............................................... 5
Prepared statement........................................... 25
Responses to written questions of:
Chair Smith.............................................. 55
Senator Warnock.......................................... 55
Vanessa Perry, Interim Dean and Professor, The George Washington
University School of Business; Nonresident Fellow, Housing
Finance Policy Center, Urban Institute......................... 6
Prepared statement........................................... 46
Responses to written questions of:
Chair Smith.............................................. 55
Senator Warnock.......................................... 60
Nicholas Schmidt, Partner and Artificial Intelligence Practice
Leader, BLDS; Founder and CTO, SolasAI......................... 7
Prepared statement........................................... 47
Responses to written questions of:
Chair Smith.............................................. 61
Senator Warnock.......................................... 63
Additional Material Supplied for the Record
Letter submitted by ACU.......................................... 68
Concept Paper submitted by AI.................................... 71
Letter submitted by the Center for AI and Digital Policy......... 88
Letter submitted by Family Housing Fund.......................... 94
Letter submitted by Urban Institute.............................. 97
Letter submitted by NMHC/NAA..................................... 160
``To Err Is Automated: Have Technological Advances in the
Mortgage Market Increased Opportunities for Black
Homeownership?'' Working Paper................................. 163
Letter submitted by Zillow....................................... 185
(iii)
ARTIFICIAL INTELLIGENCE AND HOUSING:
EXPLORING PROMISE AND PERIL
----------
WEDNESDAY, JANUARY 31, 2024
U.S. Senate,
Committee on Banking, Housing, and Urban Affairs,
Subcommittee on Housing, Transportation, and Community
Development,
Washington, DC.
The Subcommittee met at 10:03 a.m., in room SD-538, Dirksen
Senate Office Building, Hon. Tina Smith, Chair of the
Subcommittee, presiding.
OPENING STATEMENT OF CHAIR TINA SMITH
Chair Smith. Good morning. The Subcommittee on Housing,
Transportation, and Community Development will come to order.
Today's hearing will focus on both the promise and the threats
that artificial intelligence poses in the housing sector, and
I'm very much looking forward to our witness' testimony and to
this conversation.
I want to thank Ranking Member Lummis and her staff for
your ongoing bipartisan work as we put together this hearing.
We both share, I believe, a deep interest in how we can develop
Federal policy that supports innovation, and expands
opportunity for everyone to have a safe, decent, affordable
place to live. And one of the most consequential innovations in
recent years is artificial intelligence.
Leader Schumer, Senator Rounds, Senator Young, and Senator
Heinrich are leading a bipartisan effort to explore the
impacts, opportunities, and threats that AI poses. And they
have asked Senate committees to engage in our areas of
expertise, which leads us to this Committee hearing today,
examining what AI means for housing.
So without a safe, decent, affordable place to live,
nothing in your life works. Not your job, your family, your
education, your health. So, a foundation question is how AI can
help or hinder this goal. We know that some aspects of
artificial intelligence have been around for a long time, and
we also know that major advances are fueling the use of AI in
finance and housing in ways that we need to understand.
Consumers find AI when they encounter chatbots, when they
shop online, or digital helpers that seem to be ubiquitous. And
AI plays a role when a prospective tenant is looking to rent an
apartment, or a renter submits a maintenance request to a
management company, or a family tries to qualify for a home
loan, or when a person experiencing homelessness is connected
to services. These are powerful tools that hold great potential
to cut costs and to target services, to reduce wait times, and
to even reduce bias.
But they also have the potential to bake in existing
inequities and to reduce accountability and even limit
opportunity. Today, AI is being used actively in every part of
the housing continuum, from emergency homelessness services to
mortgage financing. And as I was preparing for this hearing, I
found endless applications.
AI is being deployed, for example, to help connect people
experiencing homelessness with health and housing resources. AI
is helping to forecast more precisely and accurately where
families are at risk of eviction so that we can better target
assistance. Academics and advocates are using AI and machine
learning to help understand, and map the country's zoning laws
and codes, spanning 30,000 different localities.
And these insights could help us to understand the dense
and complicated rules that govern where, and how, and what
types of housing are being built so that we can make better
decisions about how to boost housing supply and lower costs. So
there are many opportunities, and there are also some very real
concerns about the threats that AI poses.
In Minnesota, some landlords are reportedly using AI-
generated tenant-screening reports that include incorrect and
sometimes illegal or off-limits information. The result, it's
even harder for people to find a place to rent in some
circumstances, and they may never know why they were declined
or be able to correct the record. For landlords, maybe it's
just easier to move on to the next applicant rather than
consider additional information, another example of how AI used
in a bad way can be quite harmful.
There's a lawsuit in Minnesota right now against a law firm
that has allegedly automated the process of filing evictions
for landlords. In 1 month, the firm filed 400 eviction
complaints. These eviction filings lacked much detail about why
the eviction was happening, and seemed to routinely lack basic
information about lease terms or to include significant errors
around lease states, and rental amounts, and payment
information.
So the fact that a firm allegedly leaned on AI to generate
a large number of eviction filings with false information,
apparently without any meaningful review by an attorney, that's
a big problem. Not only is the eviction illegal, but that
eviction will live on in public records and hurt the tenant
into the future.
AI is also increasingly part of how people buy homes. It is
used in credit scoring models and automated valuation models
which determine the value of a home. How AI is deployed has
major implications for a person's credit scores, their mortgage
rates, and whether home ownership and wealth building is even
within reach.
We know that we have historic, systemic challenges with
fairness and equity in this country. My own hometown of
Minneapolis, has some of the greatest disparities in home
ownership between Black and White families of anywhere in the
country. So we need to carefully explore whether AI is
extending and reinforcing these biases, and how it has the
potential to correct them.
Our excellent witnesses here today have an unenviable task
in your opening statements to ground us in both these
opportunities and threats in AI and housing in 5 minutes. I
look forward to hearing from you, and I also look very much
forward to the questions from my colleagues as we follow-up
after your testimony.
With any innovation there are both opportunities and
challenges that we need to balance, and our job is to think
about these complex issues so that we can develop the best
public policy. So I very much look forward to this
conversation, and I now turn to Senator Lummis for her opening
statement.
OPENING STATEMENT OF SENATOR CYNTHIA M. LUMMIS
Senator Lummis. Well, thank you Madam Chair, and thanks to
our witnesses for being here today.
Yesterday, I was on an airplane sitting next to a woman who
was flying into D.C. so the health care business could have
their first workshop on AI and health care in some public
health fields, and how to define AI and use guidelines in a way
that will make the sideboards between existing technology and
AI more clear.
You are way ahead of the game that health care was. And Ms.
Rice, it's so nice to see you again because you are the one who
came to my office and explained to me how AI can be used in a
nefarious way. And before that, I never even would've
considered that AI could be used in a way that reinforces bad
behavior, and so I'm so pleased to see you here today. And all
of you, thank you for coming. So now we're going to explore the
range of promises and perils that artificial intelligence can
offer to housing in the United States.
So there are many potential benefits from the use of AI in
housing. Incorporating more information into housing decisions
might expand access to credit. Automated tools may expedite the
process of approving new housing developments. In rural areas,
AI can ease the appraisals process and accelerate supply
chains. Now, I'm the broken record on the Banking Committee for
my support establishing a regulatory framework for digital
assets. But I see some parallels where we have a new technology
rapidly changing in a space, and a need for thoughtful
consideration of how existing regulations may apply.
So from today's hearing, I'd like to better understand what
parts of AI fit in today's regulatory framework and what needs
to evolve. We've already seen straying into some areas that
create perils because they're poorly thought out from the
regulatory aspect.
This summer, in response to the potential use of AI when
providing financial services to customers, the Securities and
Exchange Commission proposed a rule called ``conflicts of
interest'' in predictive data analysis. It was an overreaching
rule, and the scope got into even using basic spreadsheets as
covered technology. So what that did was put unneeded burdens
on a new use of an old technology.
So we need rules and regulations governing AI to be scoped
appropriately, not using it as an excuse to force decades-old
technology to comply with a new and unnecessary set of
impractical regulations. So I'm delighted to see you here to
help us parse the difference. And this is a really qualified
panel, so this should be a really informative morning.
Concerns about AI merit discussion. We need a financial
system that gives families access to building wealth and create
strong communities through home ownership. Credit decisions
should be based on who can repay not on the color of their skin
or the community they call home. We should recognize the risk
that AI can, if not trained on complete data and within safe
guards, reinforce bias and discrimination. But aggressive
action prohibiting innovative AI use is perilous, and we need
to find that balance.
Today's hearing presents an opportunity to explore the
application of AI within the housing space, and I look forward
to hearing the testimonies of our witnesses. I'm optimistic
about the prospect of harnessing AI to uplift and expand the
American housing experience. So let's leverage innovation to
improve access to housing in Wyoming and all over the United
States. Thank you for being here. Thank you, Madam Chair.
Chair Smith. Thank you so much, Senator Lummis. And I also
am just really grateful that Senator Rounds is here. I know
that you have done a lot of work in this space across the board
and your leadership of that bipartisan task force that you and
Senator Schumer are leading. So thank you very much. I'm glad
you're here. Thank you.
So we have three witnesses with us today. I'll introduce
each of them. Dr. Vanessa Perry is interim dean and professor
at the George Washington University School of Business and
nonresident fellow at the Urban Institute's Housing Finance
Policy Center. Dr. Perry has extensive experience in the use of
AI in mortgage finance and home ownership.
She previously held positions at the Department of Housing
and Urban Development, the Consumer Finance Protection Bureau,
and Freddie Mac. Dr. Perry earned her Ph.D. from the University
of North Carolina at Chapel Hill and an MBA from Washington
University in St. Louis.
Ms. Lisa Rice is the president and CEO of the National Fair
Housing Alliance. Ms. Rice leads the Alliance's work on
eliminating housing discrimination, and addressing the evolving
role of artificial intelligence in housing. She has provided
expert testimony to the Banking and Housing Committee, the
bipartisan Senate AI Forum, and the House Financial Services
Committee. Ms. Rice previously served as a CEO of the Toledo
Fair Housing Center and the Northwest Ohio Development Agency.
And we have Nick Schmidt with us today, who is a partner
and artificial intelligence practice leader at BLDS, and the
founder and CTO of SolasAI--did I say that correctly? Close
enough, he says. BLDS is a consulting firm that specializes in
statistics and economics.
Mr. Schmidt's work focuses on algorithmic fairness,
explainable AI, and model governance practices. He founded
SolasAI, a compliance-focused AI platform that works to
identify and mitigate bias in algorithmic decision making. He
earned his MBA from the University of Chicago's Booth School of
Business.
We'll begin with Ms. Rice, and just go down the line. You
each have 5 minutes for your opening statements, and each of
you have a clock in front of you which will count down the
time. And your full written statement will be made a part of
the record. Ms. Rice, you can begin.
STATEMENT OF LISA RICE, PRESIDENT AND CHIEF EXECUTIVE OFFICER,
NATIONAL FAIR HOUSING ALLIANCE
Ms. Rice. Chairwoman Smith, Ranking Member Lummis, and
other distinguished Members of the Senate Subcommittee on
Housing, Transportation, and Community Development, thank you
for the opportunity to testify during this hearing on
artificial intelligence.
The National Fair Housing Alliance works to eliminate all
forms of housing and lending discrimination, and ensure equal
opportunities for all people. We are the trade association for
over 170 Fair housing organizations throughout the U.S. and its
territories. We know that technology is the new civil and human
rights frontier. That is why we are committed to creating
automated systems that are fair, explainable, and trustworthy.
It is why we also create and promote policies, frameworks,
and other tools to advance responsible AI principles.
Artificial intelligence holds great promise for improving
systems, democratizing opportunities, lowering costs, and
increasing productivity. Yet it also holds great dangers for
perpetuating bias, spreading misinformation, excluding people
from necessary services, and generating other harms.
AI is used extensively in the housing and financial
sectors, including in credit scoring, tenant screening,
automated underwriting risk-based pricing, dynamic rental,
pricing marketing, and many other areas. All these processes
can manifest bias and extensive harm to consumers, communities,
and our economy. Large language models like ChatGPT can also
present challenges like data privacy, security bias, and
accuracy concerns.
Through our work, we have uncovered discrimination in all
these systems. For example, our investigation against a major
insurance company in which we examined a third-party scoring
system used by the insurer, showed the system was overcharging
Black consumers at rates that exceeded their commensurate level
of risk. Our findings in that case are similar to findings by
researchers at Berkeley that found algorithmic pricing models
discriminate against Black and Latino consumers by overcharging
them to the tune of $765 million per year.
Consumers who live in communities with few mainstream
banks, like rural areas and communities of color, can often be
negatively impacted by AI systems that rely heavily on data
contained in the credit reporting agencies. For example, the
CFPB reports that consumers in southern rural areas of the U.S.
disproportionately have lower credit scores because, in part,
they have lower access to physical and online mainstream
financial services and products.
While AI can present serious threats, it can also be used
for good. For example, NFHA uses AI tools to conduct research
that is helping expand credit and housing opportunities for
underserved groups and communities. Just this month, we
released important research completed in conjunction with
FairPlay AI, documenting a novel approach for de-biasing
algorithmic models that simultaneously optimizes for high
quality model performance.
Most previous research on fairness techniques have shown a
tradeoff lower moderate, a model accuracy for increased
fairness. The novel approach at the center of our research,
which is based on distribution matching, presents the ability
to optimize models for both fairness and accuracy.
In addition to de-biasing systems, AI can be used to detect
discrimination risks, build systems that can expand access to
credit, provide financial services to consumers more
efficiently, optimize privacy, identify barriers to fair and
affordable housing and zoning ordinances, and other positive
activities.
To ensure AI can be optimized for good, Congress must first
do everything possible to ensure Federal agencies, researchers,
and NGO's can apply existing laws and standards to the use of
automated systems. Congress must ensure agencies have
sufficient resources to tackle the issues, and Congress must
pass legislation that both supports innovation and protects our
society to ensure the U.S. can continue to lead the world in
the development of the responsible AI. Thank you.
Chair Smith. Thank you very much. Dr. Perry.
STATEMENT OF VANESSA PERRY, INTERIM DEAN AND PROFESSOR, THE
GEORGE WASHINGTON UNIVERSITY SCHOOL OF BUSINESS; NONRESIDENT
FELLOW, HOUSING FINANCE POLICY CENTER, URBAN INSTITUTE
Ms. Perry. Thank you. Good morning. My name is Vanessa
Perry. I'm a professor and interim dean at the George
Washington University School of Business, and a nonresident
fellow at the Housing Finance Policy Center at the Urban
Institute focusing on consumers housing and financial markets.
I want to thank Chairwoman Smith and Ranking Member Lummis for
inviting me to testify on the impact of artificial
intelligence, which is being employed increasingly throughout
the housing and mortgage industry.
Based on my three decades of research in this field,
regulation of AI warrants urgent attention. While these models
can enhance efficiency, they can have unintended impacts on
fairness and equity. Compared to traditional models, AI relies
on a wider range of data inputs and more complex combinations
thereof.
Although intricate multivariate algorithms have been used
in the mortgage industry for years, AI models have the
potential to incorporate new data sources due to their
complexity. It is difficult, but not impossible for anyone
other than AI developers to scrutinize and monitor their
inputs.
AI is already widespread in the mortgage market. AI digital
marketing models target prospective home buyers, and
communications with customers are intermediated by AI chatbots.
Credit scoring companies and mortgage underwriting systems use
AI to evaluate credit risk. AI models are used for property
valuation, loan servicing, and loss mitigation. Because these
models rely on historical data, there is the potential for them
to systematize and amplify discrimination and inequality. For
example, due to the legacy of redlining and segregation, and
their effects on present day neighborhood conditions and home
values, why should we expect AI to produce estimates that are
both accurate and fair? And absent guardrails, how would we
know if AI models were to incorporate data elements such as GPS
location that serve as a proxy for race, gender, or ability?
AI models are not subject to human error, and they enable
more accurate and consistent decisions. Depending on how they
are developed, their enhanced capabilities could expand access
to home ownership for households currently underrepresented in
the mortgage market. For example, AI can produce faster and
less subjective estimates than human property appraisals, and
can devise credit scores for those who lack a traditional
credit history.
To address concerns about AI's impact on access to the
housing finance system for underrepresented communities, my
colleagues and I have proposed five factors summarizing the
societal, ethical, legal, and practical issues that should be
considered in the development and implementation of AI. They
form a memorable acronym, SCALE.
First, societal values. Algorithms tell us what factors the
developers think are important in what order, and to what
degree. AI models should consider the socioeconomic and
historical context such as past discrimination, and should
align with prevailing legal and ethical paradigms, such as
disparate impact law, individual freedom, privacy, and racial
equity.
Next, is contextual integrity. Model inputs should be
relevant to the mortgage and housing domain, and may differ
substantively from those used for other or less consequential
context.
Then, accuracy. Models should be reliable, error-free,
unbiased, and representative of all demographic and income
groups across varying macroeconomic conditions.
Next, legality. The model and its inputs should not
incorporate characteristics protected by fair lending laws, or
generate unjustified disparate impacts based on these
characteristics.
Finally, expanded opportunity AI models should
significantly increase access to credit in addition to offering
greater cost efficiency or risk assessment benefits. This
criterion represents a higher bar than existing regulatory
frameworks, and has perhaps the most promising impact on the
economy and communities.
This framework could inform new or expanded regulations
such as guidance for the use of certain types of data, for
example, an individual's social media profile for certain
purposes, such as mortgage lending decisions. If designed to do
so, AI models can increase access to home ownership and
eradicate the effects of systemic discrimination while
increasing accuracy and efficiency in the mortgage value chain.
We need new laws on the Federal level that utilize the
scale factors previously enumerated at every stage of the AI
life cycle. Thank you, again, for the opportunity to testify
here today, and I look forward to answering your questions.
Chair Smith. Thank you very much. Now we'll hear from Mr.
Schmidt.
STATEMENT OF NICHOLAS SCHMIDT, PARTNER AND ARTIFICIAL
INTELLIGENCE PRACTICE LEADER, BLDS; FOUNDER AND CTO, SOLASAI
Mr. Schmidt. Thank you, Chair Smith, Ranking Member Lummis,
Senator Rounds, and the esteemed Members of the Subcommittee.
Thank you for this opportunity to share my experience and
insights into how AI is revolutionizing the housing sector, and
how we can make it safer, fairer, and more accountable.
I'm Nicholas Schmidt, the founder and CTO of SolasAI. I'm
also the practice leader at BLDS. I've spent nearly 25 years
working to ensure that data, algorithms, and statistics are
applied fairly. And I've spent the last decade devoted to the
responsible and ethical application of AI and machine learning,
and with a significant portion of my work focused on credit and
housing.
Examples of AI, including the amazing ability of generative
AI, and the horror of deep fakes and what they might do to our
democracy, drive our fear, our fascination speculation. But
really, as amazing, and as scary as those truly are, the more
mundane examples of AI in machine learning are just as
important, especially in housing.
We must recognize that these technologies are already
shaping many aspects of the industry and getting distracted by
those sort of shinier aspects of AI, may take our eyes off the
ball at a time that is really critical.
So what we should consider is that while an AI algorithm
could help enable people to get a home loan for the first time,
it could also drive a decision to evict someone from their home
or apartment without sign sufficient justification. Because of
the former, we want to encourage the adoption of the
technology, because of the latter, we want to make sure that
it's only used responsibly. This is only possible through
effective regulation.
When considering this, what is really important to remember
is that these systems are driven by human decisions.
Unfortunately, this human-centric nature of AI development is
overlooked, but AI is not self-operating, and claiming so is a
dangerous notion. It absolves us of the responsibility, and
also ignores the numerous opportunities we have to shape AI for
the better.
Just to illustrate this, let's consider a mortgage
underwriting algorithm. I'm going to go through some of the
ways in which a human impacts how that algorithm ultimately
operates determining what data to use. Should you use something
like education or purchase history, how do you define
delinquency? That's going to make a very large impact on people
who have income insecurity, but are ultimately able to pay
their mortgages over time. That'll make a big difference over
whether or not they'll actually get a loan.
What algorithm should the lender use? That affects not only
discrimination, fairness, accountability, but also potentially
the safety and soundness of the lender? No algorithm is
perfect. We have to make a decision. Is the algorithm good
enough to put into production? And finally, someone has to
decide how to use the algorithm. All of these decision points
offer an opportunity for smart intervention.
And when considering such regulation that comes out of this
intervention, we should remember that these technologies are
not new to housing. We do not need to reinvent the wheel.
Instead, we should learn from the practices and frameworks to
create a strong and reasonable regulatory environment.
In particular, there are three things I'll, I'll mention.
The Federal Reserve's SR11-7 Policy, the NIST AI Risk
Management Framework, and SP 1270 guidance and the disparate
impact and disparate treatment framework from the Fair Housing
Act, and potentially E-COA--we shall see what the courts say
about that. But out of these frameworks and practices, I want
to distill this down to four principles. Fairness,
transparency, accountability, and materiality.
Fairness, disparate impact and disparate treatment. And
regulators should set expectations that bias and discrimination
should be measured and mitigated in AI systems. But really,
importantly, fairness is not just about antidiscrimination, and
it is an issue that everyone here has a self-interest in.
What is being put into some, not all, but some AI systems
is not fair to anyone regardless of their race, ethnicity, sex,
or age. We need to empower regulators to set and enforce
principles-based standards for dating inclusion.
Transparency. If a person receives a negative outcome from
an AI-based decision, they should have the right to understand
why.
Accountability. If you are negatively impacted, you should
have a right to appeal if it's a sufficiently high-stakes
decision. Additionally, people or entities that deploy AI
systems irresponsibly should face the appropriate consequences.
And materiality. Businesses need to understand and make
risk-based decisions. You can't have, for example, an Excel
spreadsheet subject to the same regulations and, you know,
requirements and oversight as a credit underwriting algorithm
that is scoring millions of people. Doing so, ultimately, is
ineffective for all of our goals.
So by considering these four sets of principles, we really
can develop a balanced regulatory approach. And by fostering
collaboration among all of the stakeholders, we really do have
a potential to harness AI's value in housing. With this. I
really do believe that AI can serve as a force for good. Thank
you very much for the opportunity to share my insight.
Chair Smith. Great. Thank you so much to all of our
panelists. And we'll now begin a round of questioning from
colleagues. And I will start.
So each of you, it seems to me, are highlighting both the
opportunities and also the risks of AI. But I'm struck by how--
I guess I want to dive in a little bit more into how you--we
know that AI used in housing could pose significant risks. But
I'm wondering if you could talk a bit more about what the
beneficial applications for AI might be.
Let's say just take for example a goal of figuring out how
to increase home ownership amongst, traditionally, you know,
people who've been traditionally unable to buy their own home.
AI is a tool. How could AI, just as an example, be used as a
tool to advance that goal? And anybody, I'd love to hear what
you have to say about that, if it's implemented correctly is
right.
Ms. Rice. I can start. As I shared, we have just released
research this month using AI to help expand access to credit
for people who are creditworthy and have been kept out of the
financial mainstream. There are many organizations, and our
research shows that we can increase fairness and access by
about 13 percent for underserved groups.
There are many organizations--well, I take that back, not
many, but there are some organizations that are working on
using AI and other forms of technologies to incorporate factors
that are currently not included in the underwriting systems and
the credit scoring systems that we utilize today in order to
determine if people can access credit opportunity.
So I'll give you an example. Rental housing payment
information. This is a very viable factor. It is an important
and highly predictive variable that is not included in the
underscoring systems that we utilize today. That missing
factor, that missing information, that missing data could
really open up the aperture for opportunity, particularly for
people who live in rural communities.
As I stated before, the CFPB's research and other research
shows that people in rural areas have disproportionately lower
credit scores, not because they're less creditworthy, but
because they don't have access to the mainstream financial
services that regularly report positive data to the credit
repositories. So if you access credit outside the financial
mainstream, using rental housing payment information can really
increase access to affordable credit opportunities.
Chair Smith. So if AI is including sort of better
information, less biased information, potentially information
that isn't included in regular decision making, you could
actually have a net positive impact on the fairness of the
system.
Ms. Rice. That's right.
Chair Smith. Right. Thank you. Let me ask kind of a
different question. Like, how do we address this so-called
black box phenomenon in AI where, you know, you have an AI tool
that includes machine learning, so it is constantly learning
and changing the way it's, you know, improving supposedly the
way that it's making decision-making, but it seems to sort of
spit out an answer at the end.
And the question is, like, where did that answer come from?
Could maybe Mr. Schmidt or Dr. Perry, could you just address
that? How do we actually get accountability when you have that
kind of a technological tool?
Mr. Schmidt. So there are two issues here. One is directly
your question and how do we deal with these incredibly opaque
algorithms? But then there's another important piece, which is,
should we even be worrying about this in high-risk situations,
or should we be using more interpretable--requiring more
interpretable, more understandable algorithms?
And the way that this is first sort of surfaced over the
last 5 or 10 years is been this idea that in order to have
accuracy, you had to have an opaque black box. By being able to
incorporate lots of interactions and nonlinear relationships
and all sorts of difficult math, you would get a more accurate
measure of, say, credit outcomes.
What has happened over the recent years is that people have
realized that you don't need to have these incredibly opaque
algorithms. You can actually develop ones that are pretty
understandable. And so I believe that those are going to become
much more commonly used in these high-rate, high-stakes
decisions.
Chair Smith. Interesting. Thank you. Dr. Perry, I'm out of
time, if there's anything that you'd like to add, I'd love to
hear from you.
Ms. Perry. I just wanted to add that because these models
are dynamic, they change often. The question is, which version
of a model even would you scrutinize or monitor?
Chair Smith. Yeah. Thank you. Senator Lummis.
Senator Lummis. Thank you, Madam Chair. Mr. Schmidt,
welcome. You have an interesting job and an important one in
terms of helping us understand the four criteria you mentioned
in your testimony. With regard to transparency, what level of
explanation do you think companies and organizations should
provide to consumers?
Mr. Schmidt. So I think that there are two principles that,
that need to apply. One of them is transparency into the
process that they have undergone, and that will allow people to
understand whether or not the data that was used was fair as
well as the process, and whether or not the data was right. So
were they using some factor, you know, education in a situation
where education really didn't belong, or maybe they got the
education of the person wrong. Being able to understand and
appeal those decisions is very important.
The other area where this is really important is having
some sort of understanding of how to improve one's outcome. So
you got rejected and you got told that, you know, had, I don't
know, number of delinquencies in the past 60 days. Maybe that
one's easy to figure out how to rectify, but in these complex
systems it could be much more difficult than that. And giving
people a clear framework to move from being rejected to being
accepted, I think could be a very helpful public policy goal.
Senator Lummis. Another question, this one about data
privacy. How do you think companies that use AI models for
housing should be thinking about data privacy?
Mr. Schmidt. I would like to preface this by saying that
I'm not a data privacy expert, but what I have seen in my
experience, particularly with large lenders and other
organizations that are really leading the financial system, is
that data privacy is very high priority, and making sure that
there are not data leakages as people spend inordinate amounts
of time on. So I think that that is something that is very well
recognized and something that the industry is really striving
toward.
Senator Lummis. So what is the weakest link here? You know,
when you named those four criteria, which one is crying in need
of improvement?
Mr. Schmidt. The one that keeps me up at night is fairness.
And this is both from, again, antidiscrimination in the
traditional sense of the word, as well as fairness for
everybody in this room. The quality of models being built
across all industries is low. That is absolutely not universal.
There are brilliant people doing very brilliant work, but there
is a lot of stuff going on there that is extremely low quality.
And what we need is to have the same kind of, or similar,
model governance standards that were put forward by the Fed and
the OCC, put into lower tier lending institutions and places
like the health care industry. Because you are absolutely
right, that the health, the financial system, financial
industry, is light years ahead of every other industry. And the
potential for damage if strong model governance is not put into
place, astronomical.
Senator Lummis. Can you help with that? Have you studied
this enough that you're beginning to see how to break through
those areas where the data is being used in a very
unsophisticated way?
Mr. Schmidt. These are mostly people problems. The problem
with data scientists and data science is that there are--
colloquially, I'd say they're drunk on algorithms. They get
very excited by the math, and they get very excited by the next
new thing, and they don't take a step away and say, ``What is
it I'm doing?''
And so, I think that the first step is making people answer
questions, making the model builders answer questions. What is
it I'm putting in here? Why am I putting it in this model? What
purpose does it serve? And the way that that can happen
effectively is by requiring effective model governance.
And if I may, the other thing that is really important is
having lawyers in the middle of the room. And in the questions,
I always tell my clients that lawyers are the best data
scientists because they're the most skeptical and negative
people in a room, and that's really what is needed in data
science. So we need lawyers, and compliance, and people like
all the ones who are laughing in this room, because you are the
people who are going to question the data scientists and get
them to start doing better work.
Senator Lummis. Well, thank you. As a lawyer, I resemble
that remark.
[Laughter.]
Senator Lummis. So thank you very much. I yield back. Thank
you, Madam Chair.
Chair Smith. Thank you, Senator Lummis. Senator Menendez.
Senator Menendez. Thank you, Madam Chair. And as a lawyer,
I'm not going to speak as a lawyer today, but as a legislator,
so I can avoid that.
Because of the way AI learns by studying large sets of
data, if there is a bias in the underlying data, it can become
encoded in AI's decision-making process. And when it comes to
housing, the historical data reflects years of redlining,
disparities in wealth, access to credit, appraisal bias, among
others. So Ms. Rice, Dr. Perry, how can we work to ensure that
AI doesn't stratify or even expand existing disparities in
housing?
Ms. Rice. Sure. I could take a first crack at that. There
are multiple ways that we have to work to make sure that AI
doesn't magnify existing inequities. So one way is to take
steps to audit and monitor AI, to examine it preproduction,
predeployment, and also post-deployment.
We have developed, actually, what we believe is a state-of-
the-art framework. It's called purpose process and monitoring.
We believe that it's a state-of-the-art auditing model
framework to help ensure that algorithms are fair, and
equitable, and that they do not pose harm to consumers.
Unfortunately, we do know, Senator Menendez, that many models
that have been deployed are exacting harms on consumers. We
have clear evidence of that. So what the purpose monitoring
process and monitoring framework does, is it helps model
developers answer all of the questions that Nick just talked
about. Right? Is this a good use for whatever problem it is
that you're trying to solve? Does it need an algorithmic or an
AI solution? Because there are many problems that that need to
be solved that don't necessarily need an AI solution.
It examines the data, it examines the model development,
and the efficacy and accuracy of that model. And then, it also
looks at the potential and examines the potential for harms.
And it identifies, or I should say, compels model developers to
look at less discriminatory alternatives while they are
designing the model.
Models can also change post-deployment, right, their
effectiveness. Just because a model is effective on day one
doesn't mean it is going to be still be highly effective and
accurate on day 200. So you have to continually be auditing and
monitoring that model to make sure that it is still
efficacious. Thank you.
Senator Menendez. Dr. Perry, anything you want to add to
that?
Ms. Perry. Yes. I would just like to add that in order to,
sort of, prevent these models from producing, reproducing
existing discrimination or past discrimination, the one thing
to do is to intentionally design models that explicitly remove
those effects and reduce those effects. And actually, that's
one of the great promises of these tools; is that they can be
programmed explicitly to remove the effects of past
discrimination because they are more capable of identifying
them than the sort of traditional models that we've relied on
in this industry for so long.
Senator Menendez. Yeah. Let me follow up this line of
questioning. That problem is further magnified by the challenge
of explainability. That is the model--the results of AI models
and the steps that led them to where they are can be difficult
to parse due to the complexity of the algorithms and the
underlying data. How should we think about issues of
accountability and oversight when it comes to these AI models
and the data that they use? And I welcome anybody's.
Ms. Perry. I'll start. My opinion is to effectively
monitor, we need to examine, one, the guiding principles that
are used to create these models, the inputs which data could
potentially be used and which are prohibited by the models. And
then finally, the effects; so the entire process with the
understanding that regulatory agencies may not necessarily have
the resources to do the volume of monitoring and enforcement
that we are accustomed to. So we definitely need stronger
protections and processes in place to make this happen.
Ms. Rice. And Senator Menendez, I'll just add that a couple
of weeks ago, we held the first our inaugural responsible AI
symposium. And I had the privilege of hosting a segment with
the heads of all of the Federal regulatory agencies. And I
posed this question to them, and they each said that we
currently have existing laws on the books that effectively
govern many aspects of AI development and usage. And they are
working to make sure that their agencies are adept and can
enforce and exercise effective oversight over the use of AI,
particularly in financial services.
Now, one of the challenges is getting staff that are
educated and trained on these systems and usages. That is a
huge lift. I will tell you, when we stood up our responsible AI
division, we had to comb the globe to find someone to head it
up. And we did find someone in Canada, not in the United
States. So we have to really do a good job of making sure that
our citizens, our students, are well educated in this area.
The other thing that they mentioned is that they need
resources. They need effective resources, and they need to be
using AI tools themselves in order to effectively police this
area.
Chair Smith. Thank you very much. Senator Rounds.
Senator Rounds. Thank you, Madam Chair. And let me just,
first of all, say thank you for doing this hearing today for
both of you.
The AI working group, when we started the program, one of
the things that we talked about was is we wanted every single
committee to be able to participate because of the expertise on
the committees and the individuals who participated in meetings
like this to share their points of view and their
recommendations. So I really appreciate your jumping on as
early as you have. You're probably one of the first
subcommittees to actually have a significant opportunity to
talk about a specific area in which AI does make a difference.
And I also want to say, Ms. Rice, what you've indicated in
terms of the personnel issue is something that we have to take
into account. And we're never going to have enough experts on
this, and so to be able to share those experts is one of the
reasons why NIST, I think, is going to be very important, being
a location where we can find individuals from industry who can
come in and participate and help to bring that expertise to the
committees, as well as to the organizations responsible for
overseeing AI development.
I also wanted to--Mr. Schmidt, I think you would agree with
me on this, but I want to get your thoughts. I think that a
company is responsible and has to abide by the Fair Credit
Reporting Act, the Fair Housing Act, the Equal Credit
Opportunity Act, regardless of what technology that they're
using, AI or otherwise. Correct?
Mr. Schmidt. Correct.
Senator Rounds. So really, one of the challenges we've got
is how do we make sure that within an algorithm or within a
decision-making process, because AI really is simply a very,
very fast decision-making process that is now available to us,
but one in which a computer system and a use of formulas
actually learns from itself, whether or not it was making
correct decisions and modifies it, as Dr. Perry indicated, can
modify itself along the way.
And so what I'm curious about is, I don't think we can be
afraid of the technology, but I think we've got to be able to
leverage the best of it while at the same time being able to
provide the regulators the ability to ask the questions and to
make modifications or request modifications, should there be
biases identified. Can you think of any program that we've ever
made that doesn't have some biases built into it?
Mr. Schmidt. No.
Senator Rounds. So really, the challenge for us is how do
we identify those biases? And in fact, any adult who is
responsible for making decisions today, is there a human that
doesn't have biases built into their decision making?
Mr. Schmidt. I've yet to meet one.
Senator Rounds. Yeah. So really, the question for us is how
do we overcome those human attributes and allow a machine to do
a better job of making decisions that eliminates the biases
that we're trying to get out of our decision-making processes.
Fair enough?
Mr. Schmidt. Yes.
Senator Rounds. OK. I know that we're going to have a lot
of discussions about the problems and the challenges that we're
going to have anytime we bring in a new type of technology. But
I wanted to close with this, and it's going to come right back
down to what Ms. Rice had indicated. Suitable AI talent is
going to be in high demand.
JPMorgan said in May of 2023 that it had hired 900 data
scientists, 600 machine learning engineers, and 200 AI
researchers to execute its technology initiatives. Google has
hired thousands of researchers and engineers to work on machine
learning and AI.
However, I believe that it's critical that AI solutions
reach beyond our largest companies. We learned at the end of
the last year that between 2022 and 2023, the number of people
experiencing homelessness on a given night increased in the
United States by 12 percent to 653,104 people. The highest
number of people recorded since the inception of the annual
count in 2007. Researchers have used AI to identify individuals
most at risk for losing their housing to target with
homelessness prevention assistance.
Now, I think this is one of the things that we should do a
better job on, and if we can eliminate some of the challenges
we have, just in terms of the number of individuals that we
have to impact, AI could actually help us identify those
individuals before they become homeless.
Mr. Schmidt, could you discuss what steps we need to take
now to provide the AI workforce of tomorrow, and how we make
certain that AI talent is reaching areas like homelessness
prevention?
Mr. Schmidt. To address the latter half of your question, I
think that it is very important that we encourage organizations
like Ms. Rice's, as well as academia, and the type of work that
Ms. Perry is doing, because quite frankly, industry is not
likely to jump on those sorts of things. And we have to put the
incentives in the right places and give the right people the
right money and ability to put forward those sorts of programs.
So that's what I would say about that.
And then in terms of the talent gap, what we really need to
do is concentrate on both STEM and liberal arts. What we need
are well-rounded people. I've worked in tech-type companies my
entire career, and I've always liked hiring English majors and
history majors, and teaching them how to do statistics, because
they're the ones who are really thoughtful.
And so what we want to do is we want to encourage that. We
want to encourage an educational system that is focused on
questioning things, really getting into why are we doing
things? And then the technology is secondary. My company,
actually, so I founded it out 4 years ago. The first Python
program I ever wrote became my company, and so I learned that.
You know, I guess you can teach an old dog new tricks. I
learned it later, and I think that many people can learn those
kinds of things if they're given the right education to start
with.
Thank you. And thank you for your leniency on the time
today, Madam Chair.
Chair Smith. Absolutely. Thank you, Senator Rounds. Senator
Warnock.
Senator Warnock. Thank you, Madam Chair. And amen to Mr.
Nicholas Schmidt on hiring folks with education and the
humanities who think critically about these issues. Georgia is
in a housing crisis, and it is not getting better. Since 2019,
rent in Georgia has increased 13.7 percent. This is one of the
highest increases in the country, but Georgia's not alone.
According to Moody's Analytics, the amount that Americans are
spending on rent is at a record high. For the first time last
year, the average apartment cost over 30 percent, over 30
percent of a family's monthly income.
One factor driving rents up may be the use of AI software
by some of the nation's largest and most powerful rental
property management companies. As a ProPublica report details,
this AI software looks at data across multiple companies and
multiple buildings in the same area, and then recommends a
monthly rental price. I am concerned that these tools may
enable property management to coordinate pricing, coordinate
escalation, inflate rents, and stifle competition.
Ms. Rice, folks across Georgia are feeling the rental
pinch. How can we ensure that AI driven rental software is not
being weaponized to artificially inflate the rental market?
Ms. Rice. Senator Warnock, thank you for that question. And
you're talking about dynamic pricing systems, and you're
absolutely right. We are very concerned about these systems and
believe that they are contributing to the increase in rental
housing prices.
And because the systems are operated by a few companies
that amass data amongst a number of different landlords
throughout a region, there is a great propensity for reducing
competition because those systems are conveying to all of those
different landlords, essentially, the same dynamic, a kind of
dynamic pricing.
So a land an apartment complex that may not be getting a
lot of inquiries can still elevate their prices because their
competitors are getting a lot of inquiries. Right? So it does
really work sort of cross purposes when you think about fair
and affordable housing principles.
The other danger is that there is very little transparency
around these systems. So if a consumer wants to know why did
the price increase from day one to day two the only information
the consumer gets back is, ``Well, this is what the algorithm
is saying the price is today.'' There's no explainability or
transparency around why that price increase happened. So
unfortunately, this is an area where we do need increased
regulation.
We're trying to use current fair housing laws to tackle
this problem. We're also trying to use current antitrust laws
to tackle this process, and we're not getting where we really
need to be. So we do need increased regulation here. The only
thing that I will add is that local jurisdictions may be able
to use tools that they have to tamp down on the utilization of
dynamic pricing algorithms, but that remains to be seen.
Senator Warnock. So in an ironic way, and AI is here to
stay, which is why we're having a conversation, but in a way,
it sort of mitigates against natural competition in the market
that this is falsely derived through these algorithms, and
they're opaque. It's difficult to get through to ask the kinds
of questions that we need to ask.
So this is a unique threat that Congress has to address.
This is affecting renters, but home ownership and the impact of
these algorithms are impacting communities of color. Buying a
home access to credit has been a long significant barrier.
Techniques like machine learning and artificial intelligence
can improve credit access by incorporating more types of data,
like paying utility bills and rental history to demonstrate
that someone is creditworthy, but they can recreate in code
historical disparities and discrimination in the housing
market.
So, I'd like to quickly ask Dr. Perry. Dr. Perry, as credit
scoring models become increasingly sophisticated, how do we
ensure that these models do not perpetuate through the
algorithm's historical discrimination?
Ms. Perry. Thanks for the question. In order to ensure that
these models, many of which operate in a sort of black box
which is one of the key issues that we are concerned with, is
really to require that we adopt ways to monitor and scrutinize
every aspect of the model development process.
I can't say it enough how important it is to have a set of
guiding principles or goals that drive the development from the
outset. Then we need to know exactly what inputs are currently
being used and what inputs could potentially be used. Then we
need to look at the way those inputs are combined. And finally,
we need to do effects testing, but we can't just look at
effects the way that we currently look at, say, test for
disparate impact, because this is far more complicated than
that and requires far more information.
And, you know, these models take, you know, a long time to
develop, but they also are changing on a dynamic basis, and
that's why it's really important to know what the goals of the
model are upfront.
That said, I think there's really great potential because
for everything you can program the AI to optimize, you can also
program it to optimize opportunity and fairness. You can tell
it explicitly, do not look at factors that are correlated with
race, ethnicity, ability, or any other kinds of protected
factors or prohibited factors.
Chair Smith. Thank you. Thank you very much.
Senator Warnock. Thank you very much, Madam Chair.
Chair Smith. Yeah. Senator Cortez Masto.
Senator Cortez Masto. Thank you, Madam Chair and our
ranking member for this great conversation. I'm going to
follow-up on this, but I'm going to ask Dr. Schmidt. If you
would, Mr. Schmidt, talk a little bit about automated valuation
models and home appraisals. Right? There's positives because it
should lower costs. It should hopefully decrease bias, which we
have been talking about. But you also note that the data used
to build an automated evaluation model will be biased, could be
biased. Right? And you have software program to address this
issue. So, talk to us how you're decreasing that discrimination
as the initial data going into these models.
Mr. Schmidt. Sure. So the problem with any kind of pricing
model, automated valuation model is that it is necessarily
going to be looking at location, location, location. You know,
that's what drives price. Well, what do we know about locations
and housing in America? It's got a history that goes up to
today of discrimination. You know, that has not ended.
And so if you are building a model that incorporates
information about location, which you have to do to get an
accurate model, you are building a discriminatory model. But
there are two things that kind of make it better. One of them
is relative to the alternative. If you think about the
potential bias, not necessarily, but potential bias of a human
appraiser, and there's been a lot of investigations and work on
that. It's shown that there's a really high risk of human
appraisal bias, very unlikely that that will be explicitly
built into an automated valuation model.
So what that means is that within a particular geography
that you're controlling for in one of these models, you're
likely to remove that idiosyncratic bias from the appraiser.
But the fact that you are controlling from one geography to
another means that you're incorporating that geographic level
discrimination.
So I think of AVMs as having--they're a step, maybe a small
step in the right direction. They are also much cheaper in
rural settings. My understanding is in rural areas, it can be
impossible to get an appraisal, a human appraisal. If there's
good enough data, then an AVM could provide one. So there are
many reasons why an AVM would represent just a sort of uniform
improvement relative to current not ideal practices.
Senator Cortez Masto. So let me ask you this, because this
is a concern. I see where we're going, and I think my
colleague's absolutely right. AI is here, and people are going
to start using it, and they'll start using an appraisal
process.
I know in Las Vegas, in the metropolitan area, home values
in a majority of our Black neighborhoods were devalued by at
least 12 percent. That's over $22,000 difference compared to
neighborhoods with fewer than 1 percent of Black residents. How
do we ensure that that bias doesn't get put into these models?
Mr. Schmidt. So it will to some----
Senator Cortez Masto. So then how do we adjust for these
differences?
Mr. Schmidt. Yes. But there is yet another aspect to it,
which is--and Ms. Rice was talking about it, and it's actually
the center of my work--called Searching for Less Discriminatory
Alternative Models. And it actually works better in AI and
machine learning.
And what it turns out in machine learning models is that
you can have thousands, really an infinite number of ways to
specify these models, including certain information, excluding
other information, changing what are called hyperparameters,
which are kind of the knobs and dials that slightly change the
algorithm. All of these can have, or changing the algorithm
itself, all of these can have an effect on the bias as well as
the quality of the model.
And machine learning, it turns out, there are just way more
opportunities. And so what I've developed, and what Ms. Rice's
organization has developed, and other very good organizations
are working on, are finding those fairer models, and that's
where things can continue to improve.
Senator Cortez Masto. OK. Any other comments, Ms. Rice?
Ms. Rice. Thank you, Senator Cortez Masto. I will say and
remind us that not every issue warrants a technological
response. So there is still--let's still remember that we have
human beings that are highly trained to perform these types of
appraisals, and we have to use them.
Now, that said, AVMs, while they can be a useful tool, they
are not as accurate in every community, so they don't have the
same level of performance. And so we do find that AVM models
are less accurate. In fact, in some communities, they have a 15
percent degree of accuracy.
So I think you'd agree you would not want to use an AVM
model in a community that has a 15 percent degree of accuracy.
That we're finding that in rural areas, and we're finding it in
predominantly Black and Latino areas. AVM models are much more
predictive in areas that have a lot of new housing development,
and that are predominantly White.
So, we are working on tools to make AVM models fairer. What
will make AVM models fairer is if we change the construct or
the approach that we currently use to determine property
values. Right now, we use a sales comparison approach, which as
Nick has already said, requires that I look at what the house
across the street from me sold, or the house around the block
from me sold.
We have worked extensively with the insurance sector to
change their property valuation models to make them fairer by
considering the cost of reconstruction, and making sure that we
have accurate data that can assess what is that reconstruction
cost, and that gets embedded or enfolded into the valuation
process.
Senator Cortez Masto. Thank you. Thank you very much.
Chair Smith. Thank you so much. Senator Lummis. We're going
to move to a second round of questions for anybody who has
them.
Senator Lummis. Thank you, Madam Chair. And I'd like to
start just by saying, Mr. Schmidt, you finally gave me an
explanation for what I can tell people. My son-in-law does--and
I've wondered how to explain what he does for years. Now I
know. He manages teams of people who are drunk on algorithms,
to make a product that's useful by sober people.
[Laughter.]
Senator Lummis. So thank you for that. I finally know how
to explain what he does. Panel, thank you so much. I have the
same question for each one of you to conclude my questioning.
And I'd like to start with you, Dr. Perry. What regulatory
approach would you advocate for this Committee to pursue? What
guiding principles can you point us toward? How can you help us
be relevant in this discussion?
Ms. Perry. Thank you, Senator Lummis, for the question. I
think that of the most important consideration is holding AI
accountable for expanding opportunity. More so than anything, I
think that is of critical importance. It's also important to
move really very quickly because the industry is already
adopting and applying these tools in ways we are not even
aware.
Senator Lummis. Thank you. Ms. Rice.
Ms. Rice. Senator Lummis, thank you so much for the
question. First, I'd say I'd echo what I said earlier, that we
already have a bevy of existing laws and regulations that apply
in this sector. And so, what we can do is help you understand
the extent of that. And one of the things that we are thinking
about is pulling together sort of a matrix, you know, some
agencies that explain what are the existing laws and
regulations that are in place, and then understanding what we
need to do to enforce those existing laws and standards.
But the second, I will note, is that there are gaps. There
are clearly gaps, and we're finding that--we talked about a gap
with the dynamic rent, the rental dynamic pricing schemes. And
so, listing where there are gaps, and then helping legislators
understand what it would really take to fill those gaps.
And here I do advocate for an intersectoral approach,
because you're going to need academicians, you're going to need
developers, you're going to need industry, and you're going to
need civil rights and human rights organizations to understand,
you know, how to design the legislation so that we are
addressing those gaps that do exist.
Senator Lummis. Thank you. Mr. Schmidt.
Mr. Schmidt. So, I would really repeat what my copanelists
said. I think we need to move fast. I think it's extremely
important to recognize that existing regulations already cover
90 percent of it. What I have seen the Consumer Financial
Protection Bureau, CFPP, and the OCC doing, I cannot say enough
for. I think they're really moving in the right direction.
They're doing excellent work, and they should be empowered to
keep going in those directions. What I see is that industry is
responding in a way that is really right.
The other thing I would say is just that we need to make
sure that our regulators have the education, they have the
experience that they need. What the bureau has done from 2016
to today, I think they've done an amazing amount of self-
education and bringing on really good people. And they're doing
a great job considering the resources they have, but they could
use more.
Senator Lummis. Thank you. And Ms. Rice.
Ms. Rice. Senator Lummis, can I just mention one thing?
Section 230, the Communications Decency Act. We are going to
have to revisit it.
Senator Lummis. Thank you for that. This has been a great
panel. Thanks for your work in this area. This is all new to
me. So your guidance is deeply appreciated. Thanks, Madam
Chair, for holding this hearing. I yield back.
Chair Smith. Thank you so much. I just have a follow-up
question, kind of along the lines of what Senator Lummis was
asking about. There have been some efforts at the State level
creating legislation and guidance around AI implementation. Mr.
Schmidt or anybody, would you just comment, do you see anything
out there at the State level that you think is good or
something that we should be cautious about?
Mr. Schmidt. The New York Department of Financial Services
just released a circular for insurance. And of course, I could
quibble with a few things, but really, I could have written the
thing myself and been proud of it. I think that what they have
done is excellent and really principle-based, and based in
existing regulation and law, and could be very effective.
Chair Smith. That's great. Anybody else have a comment
about State-led efforts that you think are useful or something
to not do?
Ms. Perry. There a number of States that have done some
really impressive things. And the concern that I'm have now is
that they're going to be different from each other, and it's
going to create havoc for companies, and in particular, it's
going to be problematic for smaller businesses to comply when
there are different State regulations, and something that is by
definition, crosses State boundaries.
Chair Smith. Yeah. Thank you. That's great. Ranking Member
Lummis, I have a feeling that you and I could--we have easily
another hour-and-a-half of conversation with these excellent
panelists about this topic. And I think that there's a lot more
work for us to do around this idea of what model governance
standards might look like. The President's executive order on
AI, the AI Bill of Rights, and you all have laid out some
possibilities. There's lots of work there.
But I'm struggling to really understand like how that
becomes implementable, you know, in industry as you're trying
to figure out--you know, as industry is trying to figure out
what the rules are. And I'm also really struck by how one of
the things that happens in AI is there's an imbalance of
information.
Like, information is power. And if you have more
information as a seller than a buyer, then you're going to have
more power. I'm seeing this right now in my own home State of
Minnesota, where we have these big institutional investors who
are able to within a matter of minutes, when a house comes on
the market, decide whether it's something that they want to buy
or not, while an individual home potential homeowner can't
possibly compete with that. And so that shifts the balance of
power in how real estate market has worked in the past in ways
that I think are concerning.
So I always think that if you want to understand the
outcome, you need to look at what the incentives are. And to
me, the question is how do we build incentives into this system
so that we are maximizing fairness, and accountability, and
transparency as we as we try to set some guidelines both to
encourage innovation, but also to make sure that these systems
are working for the public benefit.
So, thank you very much to all of you for this. Senator
Cortez Masto, do you have a follow-up question at all?
Senator Cortez Masto. Just a comment, and I appreciate the
conversation. Let me just say I heard the be thoughtful,
reasonable, and rapid three things Congress is challenged at
doing. And so we are doomed from the start.
But it really would be helpful to get your thoughts maybe
on the top three things that we can focus on in the housing
space, or the best model that a State has put forward. I
understand the patchwork concern, but if there's a good model
that can be brought to at a Federal level, I think that's worth
looking at as well. And then finally, thank you, Ms. Rice, for
talking about CDA and 230. I couldn't agree more. We have to do
something about that. So, thank you.
Chair Smith. Great. Thank you so much. Well, thank you to
our witnesses for being here today and for providing such
excellent testimony. Before we adjourn, I would like to enter
several statements into the record from the Urban Institute's
Housing Finance Policy Center, and from Zillow, as well as
reports and articles from Dr. Perry. Is there any objection.
Without objection, those will be entered into the record.
For Senators who wish to submit questions for the record,
those questions are due 1 week from today, which is Wednesday,
February 7th. For our witnesses, you will have 45 days to
respond to any questions for the record, and thank you again.
And with that, this hearing is adjourned.
[Whereupon, at 11:21 a.m., the hearing was adjourned.]
[Prepared statements, responses to written questions, and
additional material supplied for the record follow:]
PREPARED STATEMENT OF CHAIR TINA SMITH
Today's Hearing will focus on the promise and threats that
Artificial Intelligence poses in the housing sector, and I am very much
looking forward to our witnesses testimony and this conversation. I
want to thank Ranking Member Lummis and her staff for our ongoing
bipartisan work as we put together this hearing. We both share, I
believe, a deep interest in how we can develop Federal policy that
supports innovation and expands opportunity for everyone to have a
safe, decent affordable place to live.
And one of the most consequential innovations in recent years is
Artificial Intelligence. Leader Schumer, Senator Rounds, Senator Young,
and Senator Heinrich are leading a bipartisan effort to explore the
impacts, opportunities and threats that AI poses. And they have asked
Senate Committees to engage in our areas of expertise, which leads us
to this Committee hearing today, examining what AI means for housing.
Without a safe, decent, affordable place to live, nothing in your
life works--not your job, your family, your education, or your health.
So, a foundational question is how AI can help and hinder this
goal. We know that some aspects of Artificial Intelligence have been
around for a long time, and we also know that major advances are
fueling the use of AI in finance and housing in ways that we need to
understand.
Consumers can find AI when they encounter chat bots when they shop
online, or digital ``helpers'' that seem to be ubiquitous.
AI plays a role when:
a prospective tenant is looking to rent an apartment,
a renter submits a maintenance request to her management
company,
a family tries to qualify for a home loan, or when
a person experiencing homelessness is connected to
services.
These are powerful tools that hold great potential to cut costs,
target services, reduce wait times, and even reduce bias.
But they also have the potential to bake in existing inequities,
reduce accountability, and limit opportunity.
Today AI is being actively used in every part of the housing
continuum, from emergency homelessness services to mortgage financing.
As I was preparing for this hearing, I found endless applications.
AI is being deployed to help connect people experiencing
homelessness to health and housing resources.
AI is helping to forecast more precisely and accurately where
families are at risk of eviction to help better target assistance.
Academics and advocates are using AI and machine learning to help
understand and map the country's zoning laws and codes, which span
about 30,000 localities. These insights will help understand the dense
and complicated rules that govern where, how and what type of housing
is allowed to be built, so we can make better decisions about boosting
housing supply and lowering costs.
So there are many opportunities. And, there are also very real
concerns about the threats that artificial intelligence can pose to
Americans. In Minnesota, some landlords are reportedly using AI-
generated tenant screening reports that include incorrect and sometimes
illegal and off-limits information. The result--it's even harder for
people to find a place to rent, and they may never know why they were
declined, or be able to correct the record. For landlords, it may be
easier to just move onto the next applicant rather than considering
additional information.
Another example of how AI, used in a bad way, can be quite harmful:
There's a current lawsuit in Minnesota against a law firm that
allegedly has ``automated'' the process of filing evictions for
landlords. In one month, the firm filed 400 eviction complaints. These
eviction filings lacked much detail about why the eviction was
happening and seem to routinely lack basic information about lease
terms and included significant errors regarding lease dates, rental
amounts, and payment information.
The fact that a firm allegedly leaned on AI to generate a large
number of eviction filings with false information, apparently without
meaningful reviewing by an attorney is a big problem. Not only is the
eviction illegal, but that eviction will live on in public records and
hurt the tenant into the future.
AI is also increasingly part of how people buy homes. It's used in
credit scoring models and automated valuation models (or AVMs), which
determine the value of a home. How AI is deployed has major
implications for a person's credit scores, their mortgage rates, and
whether home ownership and wealth building is even within reach. We
know that we have historic, systemic challenges with fairness and
equity in in this country--my own home town of Minneapolis has some of
the greatest disparities in home ownership between Black and White
families anywhere in the country. We need to carefully explore whether
AI is extending and reinforcing these biases, and how it has the
potential to correct them.
Our excellent witnesses have an unenviable task in your opening
statements--to ground us in both the opportunities and threats of AI in
housing, in 5 minutes each. I look forward to hearing from you, and I
look forward to hearing the questions and conversation with my
colleagues that will follow.
As with any innovation, there are both opportunities and challenges
that must be balanced, and our job is to think through how these
complex issues so we can develop the best public policy. I very much
look forward to this conversation.
PREPARED STATEMENT OF LISA RICE
President and Chief Executive Officer, National Fair Housing Alliance
January 31, 2024
[GRAPHIC(S) NOT AVAILABLE IN TIFF FORMAT]
PREPARED STATEMENT OF VANESSA PERRY
Interim Dean and Professor, The George Washington University School of
Business; Nonresident Fellow, Housing Finance Policy Center, Urban
Institute
January 31, 2024
Good morning. Thank you for inviting me to address the impact of
Artificial intelligence, i.e., AI, which is being employed increasingly
throughout the housing and mortgage industry. For these purposes, AI
refers to the use of data and algorithms in place of human decisions.
This definition includes machine learning models, which are programmed
to imitate the way humans learn, iteratively correcting themselves to
improve their accuracy.
Compared to traditional models, AI relies on a wider range of data
inputs and more complex combinations thereof. Although complex
multivariate algorithms have been in place in the mortgage market for
years, these models have the potential to incorporate nontraditional
data sources. Due to their complexity, it is difficult, but not
impossible, for anyone other than AI developers to scrutinize and
monitor their inputs.
AI models are already widely applied in the mortgage market. AI
digital marketing models target prospective homebuyers and
communications with customers are intermediated by AI chatbots. Credit
scoring companies and mortgage underwriting systems use AI to evaluate
credit risk. AI models are widely used for property valuation, loan
servicing, and loss mitigation. AI regulation warrants urgent attention
because evidence from other domains and my research with the Housing
Finance Policy Center at the Urban Institute suggests that while these
models can enhance efficiency, they can have unintended impacts on
fairness and equity. \1\
---------------------------------------------------------------------------
\1\ Michael Neal, Linna Zhu, Caitlin Young, Vanessa G. Perry,
Matthew Pruitt (2023), ``Harnessing Artificial Intelligence for Equity
in Mortgage Finance'', Urban Institute, November 6, https://
www.urban.org/research/publication/harnessing-artificial-intelligence-
equitymortgage-finance.
---------------------------------------------------------------------------
AI models are not subject to human errors, and they enable
efficient, accurate, and consistent decisions. Depending on how they
are developed, their enhanced capabilities could expand access to home
ownership for households currently underrepresented in the mortgage
market. For example, AI can produce faster and less subjective
estimates than human property appraisals and can devise credit scores
for those who lack a traditional credit history.
However, because these models rely on historical data, there is the
potential for these models to systematize and amplify discrimination
and inequality. \2\ For example, due to the legacy of redlining and
segregation and their effects on present-day neighborhood conditions
and home values, why should we expect AI models to produce estimates
that are both accurate and fair? And absent guardrails, how would we
know if AI models were to incorporate data elements, such as GPS
location, that serve as a proxy for race, gender, or ability?
---------------------------------------------------------------------------
\2\ Michael Neal, Linna Zhu, Vanessa G. Perry (2024), ``To Err Is
Automated: Have Technological Advances in the Mortgage Market Increased
Opportunities for Black Homeownership?'' Journal of the Center for
Policy Analysis and Research, forthcoming, https://papers.ssrn.com/
abstract=4347212.
---------------------------------------------------------------------------
To address concerns about AI's impact on access to the housing
finance system for underrepresented or marginalized communities, my
coauthors and I have proposed five factors summarizing the societal,
ethical, legal, and practical issues that should be considered in the
development and implementation of AI. \3\ They form a memorable
acronym, S.C.A.L.E., which stands for:
---------------------------------------------------------------------------
\3\ Perry, V., Kirsten Martin, and Ann Schnare (2023),
``Algorithms for All: Can AI in the Mortgage Market Expand Access to
Homeownership?'', AI, 4(4), 888-903, https://doi.org/10.3390/ai4040045
Societal values. Algorithms tell us what factors the
developer thinks are important, in what order, and to what
degree. AI models should consider the socio-economic and
historical context (e.g., past discrimination) and should align
with prevailing legal and ethical paradigms, e.g., disparate
---------------------------------------------------------------------------
impact law, individual freedom, and racial equity.
Contextual integrity. In addition to its accuracy, model
inputs should be relevant to the mortgage or housing domain,
and may differ substantively from those used for other or less
consequential contexts.
Accuracy. Models should be reliable, error-free, unbiased,
and representative of all demographic and economic groups
across varying macroeconomic conditions.
Legality. The model and its inputs, if used for housing or
housing finance decisions, should not incorporate
characteristics protected by fair lending laws or generate
unjustified disparate impacts based on these characteristics.
Expanded opportunity. AI models should significantly
increase access to credit in addition to offering greater cost
efficiency or risk assessment benefits. This criterion has
perhaps the most promising impact on the economy and
communities.
In terms of policy directions, the S.C.A.L.E. framework could
inform new or expanded regulations, such as guidance for the use of
certain types of data--such as an individual's social media profile--
for certain purposes, such as mortgage lending decisions.
While the S.C.A.L.E. criteria imply that model inputs and
algorithms, due to the complexity of AI models, regulators cannot rely
on traditional approaches to documentation and testing. Furthermore,
concerns about potential harms related to AI are domain-specific,
suggesting that regulation and enforcement efforts must be targeted
specifically to housing and mortgage applications.
If designed to do so, AI models can increase access to home
ownership and eradicate the effects of systemic discrimination while
increasing accuracy and efficiency in the mortgage value chain. We need
laws on the Federal level that turn the ``S.C.A.L.E.'' toward imposing
these standards at every stage of the AI life cycle.
______
PREPARED STATEMENT OF NICHOLAS SCHMIDT
Partner and Artificial Intelligence Practice Leader, BLDS; Founder and
CTO, SolasAI
January 31, 2024
Chair Smith, Ranking Member Lummis, and Members of the
Subcommittee, thank you for hosting this important hearing and for
giving me the opportunity to submit this testimony.
My name is Nicholas Schmidt, and I am the founder and Chief
Technology Officer (CTO) of SolasAI, as well as a Partner and the AI
Practice Leader at BLDS, LLC. My passion for the responsible and fair
application of artificial intelligence (AI) and machine learning (ML),
including in housing, has driven my career, leading to the development
of SolasAI, a software platform dedicated to mitigating bias and
discrimination in algorithmic decision making.
At SolasAI, we create software that not only addresses regulatory,
legal, and reputational challenges but also empowers innovation in
model development. Our work centers on reducing disparate impact and
eliminating disparate treatment, ensuring that AI tools used in housing
are equitable, effective, and profitable.
Today, I am here to share insights on how properly implemented and
regulated machine learning and artificial intelligence can transform
the housing sector. I aim to provide a comprehensive understanding of
the potential for AI to make high-quality housing more affordable and
accessible, while also addressing the critical need for fairness and
equity in these technologies.
Defining Artificial Intelligence
Before one can hope to craft effective laws or regulations around
the use of artificial intelligence, we must first define it and
understand its scope. While the term ``AI'' often conjures a vision of
futuristic and sentient machines, in practical terms, AI encompasses a
wide array of far less radical technologies.
Contrary to the popular focus on--and hype around--generative AI,
AI's impact on society extends through various types of machine
learning (ML) and AI applications, many of which are already
transforming the housing industry. \1\ What can be defined as AI
includes technologies ranging from predictive analytics to automated
decision-making systems, all of which impact the affordability,
accessibility, and equity of housing in the United States.
---------------------------------------------------------------------------
\1\ Significant ink has been spilled over the attempt to precisely
define the difference between ``AI'' and ``ML.'' Commonly, people refer
to deep neural network-based techniques that are typically used for
image, text, and language recognition or generation as ``AI,'' whereas
other techniques that are used for things like credit underwriting or
pricing (including, especially, tree-based ensemble models) are
referred to as ``ML.'' A more robust and thorough discussion of the
terms can be found here: https://mitsloan.mit.edu/ideas-made-to-matter/
machine-learning-explained and https://aima.cs.berkeley.edu/
newchap00.pdf.
---------------------------------------------------------------------------
In practical terms, ML and AI represent a class of mathematical
algorithms that learn patterns and rules from data. These learned rules
may then be applied to new data to inform real-world decisions. Thus,
it is important to remember that the rules AI develops are based on
mathematical (i.e., not human) insight, but that those rules are
developed on historical data that encode many types of human biases.
However, as I discuss below, despite the computer developing the rules,
human decisions affect how the rules are developed and systems are
used. Understanding this is essential for writing effective
legislation.
Human Decisions Drive Algorithms
Beyond the confusion of what ``AI'' is, many people are unaware of
how much human involvement is required to build and deploy an AI (or
``algorithmic decisioning'') system. In fact, particularly coming from
technology companies, there seems to be a fatalistic attitude that
implies nothing can be done to improve them. This notion, dangerous in
the extreme, is easily proven wrong. There are numerous places where
humans interact with AI systems before and during the deployment of the
algorithms that shape whether the algorithms are making reasonable,
safe, and fair decisions. Understanding the extent to which the output
of an algorithm is dependent on the decisions that people who build the
models make is essential because--at each of these decision points--
there are opportunities for humans to make better decisions that can
make the algorithms more fair, accountable, and transparent.
Using a mortgage delinquency algorithm as an example, the human
steps required to make such a model include, e.g.:
1. Choosing what the model will define as a delinquency. A data
scientist might define delinquency as 60 days or 90 days of
nonpayment. For people with less income security, but who are
likely to be able to pay their bills over a longer period,
choosing 60 days instead of 90 days may make the difference
between being provided a loan or being rejected. Importantly,
this decision is an entirely human-based decision.
2. Choosing which data will be used to predict delinquency. The
computer only makes its rules based on the data it is provided.
The person building the model might only include data clearly
related to delinquency (e.g., the existing level of debt, past
payment history, etc.), or they may include data that is not
clearly causally related to delinquency (e.g., education or
purchase history). The choices that the modeler makes will
affect the fairness of the model, its accuracy, its
reasonableness, and its reliability. While algorithms will
choose how to weight different data (and possibly exclude
certain data altogether), choosing which data to provide is an
entirely human-based decision.
3. Choosing the type of algorithm that will build these rules. Model
developers have many options regarding what model architecture
they will use to develop the model (i.e., how the model will
learn from the data). These include architectures like deep
neural networks, gradient-boosted trees, or traditional linear
regression. This decision has direct implications in terms of
the transparency of the model and whether the model's decisions
will be explainable. Some kinds of models, such as regression
models, are inherently explainable. This means it is possible
to know exactly how the model arrived at a decision. \2\
Others, such as neural networks, are not transparent. The
decisions surrounding model architecture have practical and
legal implications, as I explain infra.
---------------------------------------------------------------------------
\2\ Providing a hypothetical example of how easy it is to
interpret a linear regression, suppose we build an AVM using linear
regression that uses three variables to predict home price: number of
bedrooms, number of bathrooms, and number of square feet. With linear
regression, we know the effect of each of these variables on a
particular house's estimated price because linear regression is
completely transparent. Looking at the model, we could know that, for
example, each bedroom adds $50,000 to the value. We could also
explicitly tell that an additional bathroom adds $25,000 to the value.
Finally, we could see that every additional 100 square feet adds $5,000
to the value. On the other hand, a more complicated model architecture
could learn from the interaction of all of these variables in highly
nonlinear ways such that increasing the square footage of the house by
100 feet adds between $0 and $50,000, depending on the number of
bedrooms and bathrooms. While, in this particular example, a more
complex model would be more accurate than the regression model, it
would also likely be very difficult to interpret and fully understand
the model. This becomes particularly problematic when we move away from
this simple three variable example to a more realistic ML model, which
might include dozens, hundreds, or maybe thousands of variables.
4. Choosing whether the algorithm is sufficiently accurate for the
task. No algorithm is perfect. One issue is that a model might
be very effective for some people, but not others. For example,
it might be highly accurate for high-credit quality individuals
but be far less accurate for those with lower credit quality. A
person working for the lender will ultimately make a decision
whether that trade-off is acceptable before putting the model
into production. More generally, whether to test for different
forms of inaccuracy, how to balance the varying costs of
inaccuracy, and what minimum accuracy requirements are required
---------------------------------------------------------------------------
are all human decisions.
5. Choosing how to implement and use the algorithm. Generally, an AI
or ML model does not make a decision on its own. Frequently,
only a subset of applicants will be scored by the model; which
applicants are scored is a human decision. Once applicants are
scored, what that score means and how it is used must be
determined. For example, will a cutoff be used, or will the
specific model work with another model or subjective rules to
make a decision? Will the score be considered in light of other
variables or factors? All of these are human decisions. In
fact, when all of these decisions are examined together, it is
clear that the entire model system, of which the model itself
may only play a relatively modest role, is largely made up of
interlocking human decisions that result in an ultimate
decision.
I take pains to point out each of these decision points because, as
NIST-1270 puts it, ``A fallacy of objectivity can often surround these
processes, and may create conditions where technology's capacity and
capabilities are oversold.'' \3\
---------------------------------------------------------------------------
\3\ The sentence preceding this quote summarizes the issues I
address: ``Adding to the challenge is the reality that these systems
are built and placed within organizational settings along with their
accompanying--often unstated--policies and priorities, and used by
subject matter experts and decision makers who have their own implicit
heuristics and biases.'' NIST SP-1270, ``Towards a Standard for
Identifying and Managing Bias in Artificial Intelligence'', p.25.
---------------------------------------------------------------------------
This fallacy of objectivity has led many to conclude that not much
can be done to regulate or effectively manage these technologies.
However, because so much of the use of AI is driven by choices that
people make, regulators and the law do not need to ``surrender'' to
these emerging technologies; the space is ripe for regulation of human
decisions. In fact, effective regulation of these human decisions can
create fairer, more equitable outcomes without stifling innovation in
this space. But more than just a benefit to consumers, having defined
and reasonable regulations would give companies more clarity on how
they can safely use AI systems.
I next provide context for how AI is used in housing by discussing
common use cases and outlining the challenges and opportunities they
offer.
The Use of AI Systems in Housing
AI systems are increasingly being used across the housing industry,
as companies find that many facets of housing decisions can be made far
faster, cheaper, and more reliably than can be performed by humans.
While, in past years, there was significant discussion about whether
algorithms should be making such decisions, we are now in a world where
the use of such algorithms is commonplace; companies that do not employ
them are at great risk of losing out to those that do. Below, I outline
several areas where I have seen algorithms used in housing and discuss
the opportunities and challenges associated with their use.
Many entities have used algorithms to underwrite and price
mortgages for decades. Fannie Mae's Desktop Underwriter (DU) and
Freddie Mac's Loan Product Advisor (LPA) have histories dating back
nearly 30 years, while FICO introduced its first consumer score in
1989. More recently, human appraisals are being replaced with automated
valuation models (AVM), allowing fast and--hopefully--accurate
assessment of the value of a home. Other ways algorithms are used
include the provision and pricing of insurance, providing background
screening for rentals, automating the servicing of loans, and pricing
rental units.
There are two noteworthy takeaways from these applications of AI in
the housing industry. First, there is a long history of the use of
algorithms in housing. Correspondingly, there is a wealth of experience
in building these algorithms fairly, reliably, and transparently. As
such, we do not have to reinvent the wheel when it comes to effectively
regulating the responsible use of algorithms. Second, the reach of
algorithms in the housing industry is growing fast, which will
profoundly affect people's housing decisions. It is imperative that we
learn from what we know about safely implementing algorithms in housing
and apply that knowledge to these newer applications.
However, before discussing how the history and practices of
regulating and monitoring the use of algorithms can be extended and
made better in the world of AI, I will discuss a particular use case of
algorithmic decisioning--automated valuation models--because they
represent a concrete example of many of the benefits and challenges
inherent in the expanded use of algorithms and AI.
Case Study in the Use of Algorithms: Automated Valuation Models
As anyone who has purchased a house knows, obtaining an appraisal
can be a drawn-out, stressful, expensive, and opaque process. Further,
there is substantial evidence that human appraisals often suffer from
significant discrimination, with Black or African American homeowners
and people living in predominantly Black neighborhoods having their
homes appraised for far less than they would have been if they were
White or lived in a predominantly White neighborhood. \4\ Additionally,
people living in rural areas have found it difficult to get an
appraisal of their homes, and have found that appraisals are often more
expensive. \5\ \6\ Higher costs and delays in securing an appraisal may
make transactions in rural areas take longer, cause sales not to occur,
or prevent home buyers or refinancers from locking in lower rates.
---------------------------------------------------------------------------
\4\ http://tinyurl.com/3w5s2dak
\5\ https://www.knock.com/blog/how-long-does-an-appraisal-take
\6\ https://www.homelight.com/blog/buyer-how-long-does-an-
appraisal-take/
---------------------------------------------------------------------------
A potential promise of automated valuation models (AVMs) is that
they may solve these problems. AVMs replace the job of a human
appraiser by predicting the value of a home using a wide range of data,
including information such as a home's square feet, number of
bathrooms, size of yard, tax history, location, sales prices of similar
homes, and many other factors. While they have a long history of use by
financial institutions in valuing portfolios of properties, they are
now being used in a way that has a far more direct impact on consumers.
Lenders are using these estimated home values for many financial
decisions, including as a factor in deciding whether to offer a loan,
provide favorable terms, or offer refinancing. As such, they may
significantly affect a person's finances and life, influencing where
they live and how much they pay for housing. Their use is instructive
for understanding the benefits and harms of algorithmic decisioning.
AVMs' obvious and clear benefits are their speed, availability, and
price. The National Association of Realtors (NAR) reported that, in
2022, the average cost of a home appraisal was $500, with 9 percent of
appraisals costing more than $800. \7\ Once an AVM is built and
running, the cost of calculating an appraisal is virtually zero for the
model owner (though borrowers could still be charged for this service).
NAR additionally reported that the median response from realtors
indicated that it takes 14 days for a lender to return a completed
appraisal, with 4 percent of realtors reporting that it typically takes
them more than 30 days to receive a completed appraisal. In comparison,
obtaining an appraisal from an AVM is virtually instantaneous; even if
the lender performs subsequent reviews of the AVM-based appraisal, an
AVM-based appraisal is likely to be available substantially faster than
a human-based appraisal. It is likely that the use of AVMs has already
ensured that home sales have closed faster, instead of languishing or
being rejected.
---------------------------------------------------------------------------
\7\ https://www.nar.realtor/sites/default/files/documents/2022-
appraisal-survey-09-27-2022.pdf
---------------------------------------------------------------------------
Another benefit of an appraisal from an AVM is that it may be less
discriminatory than an appraisal made by a human. A blog post from the
CFPB describes a lawsuit where a Black couple's home appraised for
nearly 60 percent more after they ``whitewashed'' their home--removing
evidence that they were Black. \8\ Given that the data used in an AVM
would not incorporate information about the race of the homeowners,
this type of discrimination is unlikely to manifest in a well-built
AVM. Put another way, human appraisers are subject to cognitive biases
they may not even be aware of; AVMs are not.
---------------------------------------------------------------------------
\8\ http://tinyurl.com/2jt4vtrs
---------------------------------------------------------------------------
However, this is not to say that AVMs will be free from any
discrimination or bias. \9\ AVMs are not magical--if they create
accurate appraisals, those appraisals will follow the real estate maxim
of ``location, location, location.'' Of course, because of the history
of housing discrimination in the United States, a home's location
necessarily incorporates historical and present patterns of
discrimination and bias. As a result, because the data used to build
AVMs is necessarily biased, the output of the models will be as well.
AVMs also cannot help when there may be insufficient data to draw
conclusions.
---------------------------------------------------------------------------
\9\ Throughout this report, I use the terms ``bias,''
``discrimination,'' and ``disparities'' more or less interchangeably,
according to their lay definitions. However, it should be noted that
each of these has distinct definitions and interpretations in technical
and legal settings. Defining these terms precisely becomes particularly
important when measuring and mitigating discrimination.
---------------------------------------------------------------------------
To summarize, the discriminatory effect of AVMs is likely to be
mixed. Because an AVM does not have access to or incorporate
information about individual sellers or borrowers, it cannot
discriminate based on those factors. However, it will still incorporate
price-affecting discrimination in the data it sees. As a result, we can
view AVMs as a tool to help fight discrimination in home pricing, but
not one that is complete or without significant residual problems.
My work, and the software we have developed at SolasAI, measures
bias and discrimination in models such as AVMs. As explained above,
data may be biased due to historical patterns of discrimination; when a
model is trained on such data, it results in a biased model. Our work
focuses first on measuring whether a model shows evidence of unfair
disparities and, if it does, attempts to identify the source of those
disparities. If such disparities are found, the software searches for
what is known as a ``less discriminatory alternative'' (LDA) model.
These LDA models provide predictive power similar to or equal to the
original model but have a less discriminatory effect.
In addition to these remaining concerns about discrimination, AVMs
raise at least two other challenges: interpretability and
accountability. Many (but not all) AI and ML algorithms are described
as ``black-box'' processes, which means that, while we understand what
data went into the algorithm, and we can see what the output of the
algorithm is, it is very difficult to understand how or why the
algorithm made the prediction that it made. In the case of a black-box
AVM model, a lender might not be able to provide a borrower with a
reliable explanation for why the home they wanted to buy received a low
appraisal. While this lack of clarity may be acceptable in low-risk
situations, or when the algorithm gets its prediction right, having
opaque black-box models make high-risk decisions that cannot be
explained may not be fair to people who receive unfavorable outcomes.
The lack of explainability also raises concerns about the quality
of the models. If we cannot sufficiently understand why a model gives a
particular prediction, then we should have less confidence in that
model. The problem is that, while it may appear to be a high-quality
model based on the data it has seen, if we cannot understand the model,
then we cannot be sure that its decision-making process makes sense and
is likely to continue to operate well if conditions change. As a
result, for business-critical or high-human-impact decisions, model
builders need to balance a desire for accuracy with the necessity that
the algorithm be explainable. \10\
---------------------------------------------------------------------------
\10\ There is a general understanding that more opaque ``black-
box'' algorithms are better able to capture hidden patterns in data
than less opaque, ``white-box'' algorithms. To the extent this is true,
a black-box algorithm will likely be more accurate. However, there has
been significant work done to show that in many contexts, less opaque
and highly interpretable models perform just as well, or virtually as
well, as opaque models.
---------------------------------------------------------------------------
The points outlined above about AVMs apply to virtually every
algorithm making housing-related decisions. With regard to
discrimination, an algorithm is not inherently discriminatory, but it
can discriminate if it is given discriminatory data, poorly built, or
used in the wrong context. However, as we saw in the context of AVMs,
even if they suffer from known biases, they may still represent a
better option than human-based decisions.
Having used AVMs to illustrate the benefits and perils of using AI
more generally, the next section outlines concrete steps that
legislators and regulators can take to minimize likely harm while
encouraging and fostering safe innovation in AI.
The Role of Regulators and Policymakers in Ensuring Responsible
Innovation in AI
As we consider regulations for AI in housing, the primary goal
should be to maximize the responsible use of this technology: given its
potential to cause extreme harm at scale, safe and sound implementation
of AI technologies is paramount. However, given its potential to
represent a significant technological development that delivers real
and meaningful societal benefits, we should also aim to minimize
regulation's potential to be overly burdensome, possibly stifling
innovation.
Effectively accomplishing this goal is significantly more likely if
we consider two key factors. First, any approach to regulation should
not be overly prescriptive. Instead, we should focus on setting clear
risk-based principles that encourage and enforce responsible AI
development and use, where the most impactful systems (i.e., the
systems with the most potential to harm or benefit people, society, or
the environment) receive the most oversight. Second, it will be
valuable to recognize that a significant body of existing work,
regulation, and industry practice can be applied to AI systems to make
them safer. Looking to these tried and validated frameworks and
policies should guide our approach to making effective regulations for
the use of AI.
The Benefits of a Nonprescriptive Regulatory Environment
A principles-based and less prescriptive approach to AI regulation
can encourage innovation while ensuring the responsible development of
AI. It recognizes the dynamic nature of technology and compliance and
provides flexibility for continuous improvement and adaptation.
A principles-based framework allows for innovation in both
technology and compliance methods. On the compliance side, advancements
like improved Less Discriminatory Alternatives (LDA) search and
enhanced techniques for providing Adverse Action Notices (AANs)
demonstrate how technology and compliance can complement each other and
evolve together. Further, innovations in technology, such as Shapley
values, explainable boosting machines, and Wells Fargo's Python
Interpretable Machine Learning (PiML) package illustrate the rapid
development of new AI tools and methods that encourage responsible
model building. A less rigid but strong regulatory environment
encourages such advancements.
Overly prescriptive regulations, on the other hand, risk stifling
innovation as they may lead to a ``design-around'' mentality, where the
focus shifts from responsible development to merely meeting specific
regulatory criteria. This can hinder genuine progress and the
exploration of new AI and negate the desired helpful impact of the
regulations. It also risks enforcing technical requirements that
quickly drift into irrelevance as technology evolves.
Key Principles To Consider for AI Regulation
Four fundamental principles are foundational for adopting effective
AI regulation: materiality, fairness, accountability, and transparency.
Developing regulations with these as guideposts will help ensure that
AI systems serve the public interest while advancing technological
progress.
Materiality:
This principle advocates for a risk-based approach in governing AI
systems. By focusing more stringent regulation on higher-risk AI
applications, resources will be allocated more effectively. For
example, a company should not spend as much time reviewing a marketing
model as they would an underwriting model that enormously impacts both
consumers and the business. Adopting such a risk-based approach ensures
that systems with the most significant potential impact are carefully
monitored and promotes innovation by not overburdening lower-risk
initiatives with unnecessary regulatory constraints. As discussed
below, SR 11-7 provides a solid foundation for guiding how materiality
is assessed in AI regulation.
Fairness:
The principle of fairness is central to the responsible deployment
of AI. Establishing a clear understanding and expectation of fair AI
practices is crucial, particularly in applications that significantly
impact individuals, such as housing. Regulators should set expectations
that bias and discrimination should be identified and mitigated in AI
systems. Existing frameworks relating to measuring and mitigating
disparate impact, disparate treatment, and proxy discrimination should
guide further regulation of AI fairness.
Accountability:
AI systems must have accountability mechanisms, especially those
with high impact. This involves providing individuals affected by AI
decisions a right to appeal, ensuring that there is recourse for those
who may be adversely impacted. Additionally, entities that deploy AI
systems irresponsibly should face appropriate consequences.
Transparency:
The principle of transparency mandates clear explanations for
decisions made by AI systems. This is fundamental to building trust in
AI systems. Understanding the ``why'' and ``how'' behind AI-driven
decisions is crucial for public acceptance and confidence in these
technologies, and is further crucial to ensure that systems are fair.
By focusing on these critical principles--materiality, fairness,
accountability, and transparency--we can create a regulatory
environment that encourages the development of innovative AI
technologies and safeguards against potential harms and biases. This
principles-based approach to AI regulation is particularly pertinent in
housing, where the impact of AI can have profound implications on
people's lives and the fabric of communities. Next, I discuss a number
of frameworks that can serve as guides for future regulation.
Using Existing Regulations and Frameworks To Guide Further AI
Regulation
Congress and regulators will not need to devise laws and
regulations from scratch to achieve the objectives that I have laid
out. There are many regulations, standards, and frameworks with a
proven track record of setting standards for human decisions related to
AI, holding relevant actors accountable for those standards, and
supporting the development and deployment of these technologies.
Importantly, in many (but of course not all) cases, the industry has
welcomed these for providing clear and reasonable standards. I discuss
these at a high level below.
SR 11-7, \11\ a supervision and regulation letter from the
Board of Governors of the Federal Reserve System, constructs
accountability mechanisms and organizational structures to
ensure adequate and risk-based governance of credit modelers.
While the document highlights the importance of technical
processes such as testing and monitoring, its primary focus is
on principles such as effective governance structures (e.g.,
independent validation teams with high stature and strong
financial incentives), risk management executives with
independent reporting chains, and documentation requirements.
---------------------------------------------------------------------------
\11\ https://www.federalreserve.gov/supervisionreg/srletters/
sr1107.htm
NIST SP 1270, \12\ a special publication from the National
Institute of Standards and Technology (NIST), describes
technical, process, and cultural problems and solutions
relating to AI bias. It highlights that many aspects of data
and AI systems are strongly influenced by human behavior and
decisions, and suggests that approaches from model risk
management (e.g., SR 11-7) coupled with more novel approaches,
such as structured feedback activities (e.g., bug bounties or
red teaming), human-centered design, and information sharing
are strong mitigants for managing bias in AI systems. Another
prominent theme of NIST SP 1270 is that basic scientific rigor
in AI needs to be improved.
---------------------------------------------------------------------------
\12\ https://nvlpubs.nist.gov/nistpubs/SpecialPublications/
NIST.SP.1270.pdf
The disparate impact, disparate treatment, and proxying
framework is a legal doctrine that has been developed over the
course of decades under the Fair Housing Act (FHA), the Equal
Credit Opportunity Act (ECOA), and Title VII of the Civil
Rights Act of 1964. This framework sets forth requirements for
measuring discrimination that could be used in any decision
tool (including AI or ML models) and provides a conceptual
framework for mitigating any meaningful disparities found
through conducting an LDA search. Many AI tools that affect
consumers in the housing market are likely covered by this
framework via the FHA. Setting the expectation that this
framework would be extended to all high-risk AI use cases
throughout the housing industry would ensure that companies
move towards adopting fairer models. \13\
---------------------------------------------------------------------------
\13\ Schmidt, Nicholas, and B. Stephens. ``An Introduction to
Artificial Intelligence and Solutions to the Problem of Algorithmic
Discrimination''. Conference on Consumer Finance Law (CCFL) Quarterly
Report. Volume 73, Number 2 (October 2019). https://arxiv.org/abs/
1911.05755
The NIST AI Risk Management Framework \14\ puts forward
four risk management functions for organizations: (1) Govern,
(2) Map (understanding the risk of AI systems in their
operational contexts with less emphasis on their development),
(3) Measure, and (4) Manage; and seven trustworthy
characteristics for AI systems: (1) Safe, (2) Valid and
Reliable, (3) Accountable and Transparent, (4) Explainable and
Interpretable, (5) Privacy-enhanced, (6) Fair with Harmful Bias
Managed, and (7) Secure and Resilient. Governance guidance is
largely aligned with the risk-based principles laid out in SR
11-7 but introduces additional governance concepts from data
privacy, information security, and more recent academic
research. The AI Risk Management Framework has two distinct
strengths: it acknowledges (1) that many AI risks arrive from
real-world problems, not computer code bugs; and (2) that
overlaps and connections between risks, risk controls, and
governance must be recognized. It does all this while orienting
governance toward traditional risk-based principles focusing on
human accountability.
---------------------------------------------------------------------------
\14\ https://www.nist.gov/itl/ai-risk-management-framework
---------------------------------------------------------------------------
Conclusion
The most significant harms associated with AI are not the
fantastical scenarios often depicted in science fiction, but real-world
issues such as discrimination, data privacy violations, unaccountable
decision making, and fraudulent activities. Effectively regulating AI
systems requires recognizing these facts. As AI evolves and impacts
more aspects of housing, policymakers, regulators, public advocacy
groups, and industry professionals must remain vigilant and proactive.
We each play a key role in ensuring that AI systems are not only
technically sound and effective, but are also fair, transparent, and
accountable. This will require ongoing collaboration, research, and
adaptation of regulatory approaches.
Regulating AI so that it is used responsibly and safely and so that
it can continue to benefit consumers and industry is a complex and
crucial task. Materiality, fairness, accountability, and transparency
provide a framework of principles that can serve as a touchstone for
evaluating the effectiveness of proposed regulations. Further,
leveraging existing regulations and frameworks such as SR 11-7, NIST SP
1270, the disparate impact framework, and the NIST AI Risk Management
Framework, regulators and legislators can create an environment that
fosters innovation and protects against potential harm.
The future of AI in housing presents both exciting possibilities
and significant responsibilities. By embracing a principles-based
regulatory approach, drawing on existing frameworks, and remaining open
to continuous learning and improvement, we can build AI that reaches
its potential to improve housing affordability, accessibility, and
equity, while safeguarding the rights and interests of all individuals.
Thank you again for the opportunity to share my insights and
perspectives on this critical issue.
RESPONSES TO WRITTEN QUESTIONS OF CHAIR SMITH
FROM LISA RICE
Q.1. How do we build incentives into housing and regulatory
systems so that we are maximizing fairness, accountability, and
transparency as we try to set guidelines both to encourage
innovation but also to make sure these systems are working for
the public? Can and should this be handled legislatively? Dr.
Perry underscored the importance of holding ``AI accountable
for expanding opportunity,'' and I welcome legislative and
regulatory recommendations for how to best incentivize this.
A.1. Response not received in time for publication.
Q.2. You underscored the bevy of existing laws that apply to
and cover most of the AI applications in housing. What are the
legislative gaps you see in this space, and how do you
recommend that Congress addresses them?
A.2. Response not received in time for publication.
------
RESPONSES TO WRITTEN QUESTIONS OF SENATOR WARNOCK
FROM LISA RICE
Q.1. The Fair Housing Act \1\ and Equal Credit Opportunity Act
\2\ are landmark statutes to prevent discrimination in housing.
Some Federal regulators have already used authorities under
these statutes to engage in oversight of more advanced models
that employ novel forms of data and more sophisticated
statistical techniques. \3\
---------------------------------------------------------------------------
\1\ See supra n. 1.
\2\ See supra n. 2.
\3\ See Press Release, supra n. 3.
---------------------------------------------------------------------------
A.1. Response not received in time for publication.
Q.2. In your view, are Federal regulators using all available
authorities under FHA and ECOA to engage in oversight of novel
software or models that employ machine learning (ML) or
artificial intelligence (AI)? Why or why not?
A.2. Response not received in time for publication.
Q.3. Are there any ways in which existing statutes,
authorities, or resources fail to enable policymakers and
regulators to engage in oversight of novel software or models
using ML and AI? If so, what additional authorities are most
helpful? Please provide specific examples.
A.3. Response not received in time for publication.
------
RESPONSES TO WRITTEN QUESTIONS OF CHAIR SMITH
FROM VANESSA PERRY
Q.1. How do we build incentives into housing and regulatory
systems so that we are maximizing fairness, accountability, and
transparency as we try to set guidelines both to encourage
innovation but also to make sure these systems are working for
the public?
A.1. Existing regulatory frameworks in the lending industry,
including, aspects of the Fair Housing Act, e.g., the disparate
impact standard, \1\ and Affirmatively Furthering Fair Housing
(AFFH), can be used to establish standards for fairness and
accountability in Artificial Intelligence and Machine Learning
(AI/ML) systems. Existing reporting requirements (e.g., under
the Community Reinvestment Act, the Home Mortgage Disclosure
Act, and the Federal Housing Finance Agency's conservatorship)
collect data that can be used to evaluate outcomes, although
expanded, real-time data, such as loan- and transaction-level
detail, should be readily available to better inform regulators
and policymakers about the impact these systems are having on
the market.
---------------------------------------------------------------------------
\1\ ``Reinstatement of HUD's Discriminatory Effects Standard'',
Housing and Urban Development Department, 03/31/2023, https://
www.federalregister.gov/documents/2023/03/31/2023-05836/reinstatement-
of-huds-discriminatory-effects-standard.
---------------------------------------------------------------------------
The issue of transparency is controversial among scholars
who study ethical AI due to concerns about the complex and
dynamic nature of AI/ML models. The Consumer Financial
Protection Bureau (CFPB) has recently issued guidance
elaborating on the Equal Credit Opportunity Act requirement
that creditors explain the specific reasons for denying loans,
even when complex AI/ML models are used. \2\ A distinguishing
feature of AI models compared to algorithmic predecessors is
that they often apply multiple factors simultaneously that can
adapt to varying market conditions. Heretofore, these reasons
have been listed in simple categories, e.g., credit history, or
insufficient cash. In the case of AI/ML models, however, there
could be numerous factors that may not matter by themselves but
taken together result in loan rejection. To prevent consumer
confusion, the CFPB should continue to provide standard
categories of feedback, supported by empirical evidence on the
messages that consumers are most likely to understand. At the
same time, AI/ML models could be used to recommend actionable
steps for denied applicants to take that would increase their
likelihood of loan approval, e.g., how much debt reduction or
savings would be necessary to qualify for a mortgage.
---------------------------------------------------------------------------
\2\ CFPB Issues Guidance on Credit Denials by Lenders Using
Artificial Intelligence, Consumer Financial Protection Bureau, Sept.
19, 2023, https://www.consumerfinance.gov/about-us/newsroom/cfpb-
issues-guidance-on-credit-denials-by-lenders-using-artificial-
intelligence/.
---------------------------------------------------------------------------
Financial regulators, e.g., the OCC, perform periodic
reviews of underwriting models, and in the case of the
Enterprises, model changes must be approved by FHFA in advance
of adoption. To provide opportunities for innovation as well as
regulatory oversight, President Biden's recent executive order
on AI relies on voluntary commitments from AI companies and
``red-team-style'' \3\ audits, but regulatory requirements and/
or tax incentives could be instituted to encourage firms to
participate.
---------------------------------------------------------------------------
\3\ Red-teaming is the process of testing model effectiveness by
applying an adversarial lens to your organization, https://www.ibm.com/
blog/red-teaming-101-what-is-red-teaming/.
Q.2. Can and should this be handled legislatively? You
underscored the importance of holding ``AI accountable for
expanding opportunity,'' and I welcome legislative and
---------------------------------------------------------------------------
regulatory recommendations for how to best incentivize this.
A.2. To ensure that AI/ML models accomplish the goal of
expanding opportunity, \4\ legislation could be established,
mirroring the Affirmatively Furthering Fair Housing (AFFH)
provision of the 1968 Fair Housing Act. AFFH requires that
entities receiving Federal funding related to housing must take
active steps to promote fair housing practices and combat
housing discrimination, based on an analysis of impediments to
housing options in their jurisdictions. A similar mandate could
be incorporated into H.R. 6580, i.e., the proposed Algorithmic
Accountability Act of 2022, \5\ stipulating that predeployment
impact assessments identify how an AI system expands
opportunity for underrepresented groups (e.g., low-income,
rural area residents, residents in historically redlined
neighborhoods). Organizations would subsequently be evaluated
against these goals at the post-implementation evaluation stage
(Algorithmic Accountability Act, Sect. 3). \6\ To strengthen
these incentives, additional legislation could create Expanded
Opportunity Tax Credits for organizations which can demonstrate
that the adoption of a new system has significantly impacted
specific goals--a model that combines approaches from Low-
Income Housing Tax Credits and the affordable housing goals
established for Fannie Mae and Freddie Mac. In this example, a
mortgage lender would qualify for a tax credit if they could
demonstrate that a new AI/ML system would result in a higher
rate of mortgage approvals for low- and moderate-income
homebuyers, or for residents of rural or other underserved
areas.
---------------------------------------------------------------------------
\4\ Perry, V., Kirsten Martin, and Ann Schnare (2023),
``Algorithms for All: Can AI in the Mortgage Market Expand Access to
Homeownership?'', AI, 4(4), 888-903, https://doi.org/10.3390/ai4040045.
\5\ H.R. 6580, To direct the Federal Trade Commission to require
impact assessments of automated decision systems and augmented critical
decision processes, and for other purpose, Feb. 3, 2022, https://
www.congress.gov/117/bills/hr6580/BILLS-117hr6580ih.pdf.
\6\ Mokander, J., Juneja, P., Watson, D.S., et al. (2022), ``The
U.S. Algorithmic Accountability Act of 2022 vs. The EU Artificial
Intelligence Act: What Can They Learn From Each Other?'' Minds &
Machines 32, 751-758. https://doi.org/10.1007/s11023-022-09612-y
---------------------------------------------------------------------------
H.R. 6580, known as the Algorithmic Accountability Act,
directs the Federal Trade Commission (FTC) to require impact
assessments of automated decision systems and augmented
critical decision processes. Rather than place the entire
burden of these assessments on the FTC, all regulators of
entities in affected industries, e.g., housing, credit, and
education, could implement these assessments. This would allow
specialization and alignment with the agencies' mission and
goals.
Q.3. We know financial technology companies and mortgage
lenders are exploring new ways to incorporate AI into
underwriting and property valuation. What roles do Fannie Mae
and Freddie Mac, as GSEs, have to play in shaping industry
implementation and responsibility for AI technology? \7\
---------------------------------------------------------------------------
\7\ Neal, Michael, Linna Zhu, Caitlin Young, Vanessa G. Perry,
Matthew Pruitt (2023a), ``Harnessing Artificial Intelligence for Equity
in Mortgage Finance'', Urban Institute, November 6, https://
www.urban.org/research/publication/harnessing-artificial-intelligence-
equity-mortgage-finance.
A.3. Approximately 70 percent of mortgage loans in the U.S. are
financed by Fannie Mae and Freddie Mac, and most mortgage
lenders utilize their proprietary underwriting systems. As
conservator of the Enterprises, the Federal Housing Finance
Administration (FHFA) has issued guidance for the managing
risks of AI/ML systems \8\ and has approved the use of AL and
ML models for certain applications, e.g., fraud detection,
property valuation, and loan servicing activities. However,
according to a recent study on AI in the mortgage industry,
neither the Enterprises, the CFPB, nor other regulatory
entities have provided specific standards for downstream
adoption and management of AI by lenders, seller/servicers of
the Enterprises, or other intermediaries in the mortgage value
chain. Because the underwriting standards of the Enterprises
are the primary drivers of lending practices industrywide, and
because of the significant legal and financial risks of
noncompliance, lenders and other market participants have been
reluctant to experiment with AI-based innovations. \9\ As a
result of this uncertainty, these lenders may miss
opportunities to leverage AI/ML to advance opportunities for
currently underserved market segments, such as models to expand
credit scoring or increase fairness in property valuation.
---------------------------------------------------------------------------
\8\ Advisory Bulletin on Artificial Intelligence/Machine Learning
Risk Management, Federal Housing Finance Agency, AB 2022-02, Feb. 10,
2022, https://www.fhfa.gov/SupervisionRegulation/AdvisoryBulletins/
Pages/Artifical-Intelligence-Machine-Learning-Risk-Management.aspx.
\9\ Neal, et al. (2023a).
---------------------------------------------------------------------------
Because of the Federal Government's close monitoring and
oversight of the Enterprises, as well as their access to data,
model development, and computing resources, they are in a
better position than individual lenders to lead AI/ML
innovation in the industry via pilot programs. For example,
Freddie Mac has used AI/ML to incorporate rental and other
payment data sources in their models, and partnering with
Fintech companies has been one approach to expediting the
development and testing of these tools. \10\
---------------------------------------------------------------------------
\10\ ``Zest AI Joins Forces With Freddie Mac To Help Make
Homeownership Possible for More Americans'', PR Newswire, Nov. 18,
2020, https://www.prnewswire.com/news-releases/zest-ai-joins-forces-
with-freddie-mac-to-help-make-homeownership-possible-for-more-
americans-301176336.html.
---------------------------------------------------------------------------
It is important to note that historically, changes to
underwriting models that require FHFA approval have taken years
to implement, failing to meet the regulatory demands posed by
the rapid pace of AI/ML innovations. For example, it has taken
more than 5 years for the Enterprises to update their credit
score requirement despite compelling evidence that these
changes would have a significant and immediate impact on
homeownership access for marginalized communities. \11\
---------------------------------------------------------------------------
\11\ ``Fact Sheet: Credit Score Models and Credit Report
Requirements'', Federal Housing Finance Agency, Mar. 23, 2023, https://
www.fhfa.gov/Media/PublicAffairs/Pages/Fact-Sheet-FHFA-Announcement-on-
Credit-Score-Models-March-2023.aspx.
---------------------------------------------------------------------------
It is also important to understand the influence of the
Federal Housing Administration (FHA) on AI adoption in the
mortgage industry. The FHA Technology Open to Approved Lenders
(TOTAL) Scorecard has been developed using AI/ML and is used
widely by mortgage lenders along with Freddie Mac and Fannie
Mae's underwriting systems. Relative to the conventional market
dominated by the Enterprises, FHA insures a disproportionate
share of mortgages for low- and moderate-income, first-time
homebuyers and historically underrepresented racial groups. For
example, in 2023, FHA's percentage of mortgages to Black and
Hispanic borrowers was 30.6 percent, \12\ compared to 15.7 and
20.7 percent for Freddie Mac and Fannie Mae, respectively. \13\
FHA issues guidance for lenders on how to implement the FHA's
AI-based systems but does not address lenders' use of their own
AI applications. \14\
---------------------------------------------------------------------------
\12\ ``Fact Sheet: FHA's Impact in 2023--Making Homeownership
Possible for Hundreds of Thousands of Families'', U.S. Department of
HUD, Nov. 15, https://www.hud.gov/sites/dfiles/PA/documents/Fact-Sheet-
MMI-Rollout.pdf.
\13\ ``2023 Annual Housing Activities Report'', Freddie Mac,
https://www.freddiemac.com/about/pdf/2023-annual-housing-activities-
report.pdf; 2023 Annual Mortgage Report, Fannie Mae, https://
www.fanniemae.com/media/50736/display.
\14\ Neal, Michael, Janneke Ratcliffe, and Matthew Pruitt (2023b),
``AI Could Alter Mortgage Lending, but Government Leadership Is
Needed'', Urban Wire, Urban Institute, November 6, https://
www.urban.org/urban-wire/ai-could-alter-mortgage-lending-government-
leadership-needed.
---------------------------------------------------------------------------
The U.S. Department of Housing and Urban Development (HUD)
also influences the adoption and oversight of AI in its role as
regulator of the Enterprises' fair lending activities along
with FHFA. Significant, additional investments in technology
infrastructure and expertise will be necessary for HUD and the
Department of Justice to investigate violations or otherwise
enforce the Fair Housing Act when AI and ML are involved.
Currently, the agency relies heavily on third-party vendors for
fair lending testing of these models. \15\
---------------------------------------------------------------------------
\15\ Hoffberg, Adam, and Bill Reeder (2023), ``The FHA TOTAL
Mortgage Algorithm: Providing Machine-Learning Analysis for Two
Decades'', PD&R Edge, U.S. Dept. of HUD, June 27, https://
www.huduser.gov/portal/pdredge/pdr-edge-pdrat50-062723.html.
Q.4. Additionally, smaller, mission-oriented lenders such as
Minority Depository Institutions and Community Development
Financial Institutions tend to serve low-to-moderate income
communities and minorities at higher proportions than other
institutions. But we know, from one of your reports, that they
have been slower to adopt AI than other financial institutions.
Can you please describe this technology gap, and if it could
further reinforce racial disparities in the mortgage industry
---------------------------------------------------------------------------
and what should be done about it?
A.4. According to a recent study, \16\ larger lenders are more
likely to adopt AI/ML systems due to greater capacity to make
the necessary investments in technology and human capital, used
towards in-house or third-party resources. Fintech companies,
for example, can charge anywhere from several thousand to
hundreds of thousands of dollars per year to develop models
used for customer service, underwriting, or fraud detection,
not including additional ongoing costs for maintenance and
administration. Taken together, these costs present significant
barriers to AI adoption for smaller and mission-oriented
lenders, such as CDFIs. Anecdotal evidence also suggests that
these smaller institutions rely more heavily on relationship-
based approaches which are more labor-intensive, less
quantifiable, and more difficult to replicate using algorithms
designed for the mainstream mortgage market. However, one of
the intended benefits of AI/ML is to incorporate nontraditional
data sources in ways that can expand access to the mortgage
market; another potential benefit is lower costs due to
increased efficiency. Due to these technology gaps, smaller
institutions could miss out on opportunities that AI presents
to create mission-focused scale economies.
---------------------------------------------------------------------------
\16\ Neal, et al. (2023a).
---------------------------------------------------------------------------
To ensure that smaller lenders have access to these
technologies, regulatory agencies should partner with nonprofit
housing organizations to provide technical assistance grants,
similar to the Community Development Financial Institutions
(CDFI) Program administered by the U.S. Department of the
Treasury. This program provides grants to support investments
in technology, capacity building, and new product development.
------
RESPONSES TO WRITTEN QUESTIONS OF SENATOR WARNOCK
FROM VANESSA PERRY
Q.1. The Fair Housing Act \1\ and Equal Credit Opportunity Act
\2\ are landmark statutes to prevent discrimination in housing.
Some Federal regulators have already used authorities under
these statutes to engage in oversight of more advanced models
that employ novel forms of data and more sophisticated
statistical techniques. \3\
---------------------------------------------------------------------------
\1\ See supra n. 1.
\2\ See supra n. 2.
\3\ See Press Release, supra n. 3.
---------------------------------------------------------------------------
In your view, are Federal regulators using all available
authorities under FHA and ECOA to engage in oversight of novel
software or models that employ machine learning (ML) or
artificial intelligence (AI)? Why or why not?
A.1. Federal regulators, such as the U.S. Department of Housing
and Urban Development (HUD), the U.S. Department of Justice
(DOJ), and the Consumer Financial Protection Bureau (CFPB),
have leveraged their authority under the Fair Housing Act and
the Equal Credit Opportunity Act (ECOA) to regulate the use of
Artificial Intelligence and Machine Learning models (AI//ML) in
the mortgage industry. For example, in 2019 Facebook settled a
Fair Housing Act case filed by HUD. The case alleged that
Facebook's AI-driven advertising platform enabled
discriminatory practices, such as targeting or excluding market
segments based on prohibited factors, such as race, gender,
nationality, religion, familial status, and disability. This
landmark case has led to heightened awareness, increased
scrutiny, and regulatory oversight of advertising practices in
the mortgage industry. In addition, social media and digital
marketing companies have instituted stricter policies to ensure
compliance with fair housing laws.
Additionally, in 2023, the CFPB, along with the DOJ, the
Federal Trade Commission (FTC), and the Equal Employment
Opportunity Commission (EEOC), issued a joint statement
emphasizing their intentions to apply existing
antidiscrimination laws and regulations to AI models,
particularly when used for credit decisions and digital
marketing of financial services. \4\ Later in 2023, the CFPB
issued guidance reinforcing that existing legal requirements
under ECOA for lenders to provide specific reasons for credit
denials apply in the case of AI models. \5\ These examples
underscore the potential for existing statutes to apply to
emerging technological applications, such as AI/ML.
---------------------------------------------------------------------------
\4\ Khan, Linda (2023), Joint Statement on Enforcement Efforts
Against Discrimination and Bias in Automated Systems, Federal Trade
Commission, April 25, https://www.ftc.gov/legal-library/browse/cases-
proceedings/public-statements/joint-statement-enforcement-efforts-
against-discrimination-bias-automated-systems.
\5\ CFPB Issues Guidance on Credit Denials by Lenders Using
Artificial Intelligence, CFPB, Sept. 13, https://
www.consumerfinance.gov/about-us/newsroom/cfpb-issues-guidance-on-
credit-denials-by-lenders-using-artificial-intelligence/.
---------------------------------------------------------------------------
While these are important signals to the industry about
best practices and legal requirements, these agencies can only
take action after alleged violations have occurred and
complaints have been filed. Given resource constraints
including limited in-house expertise in evaluating AI/ML
models, more efforts should be placed on ensuring that AI/ML
models align with prevailing legal requirements and established
risk management principles before implementation, such as the
predeployment impact assessments proposed in the Algorithmic
Accountability Act of 2022. \6\
---------------------------------------------------------------------------
\6\ H.R. 6580, https://www.congress.gov/117/bills/hr6580/BILLS-
117hr6580ih.pdf.
Q.2. Are there any ways in which existing statutes,
authorities, or resources fail to enable policymakers and
regulators to engage in oversight of novel software or models
using ML and AI? If so, what additional authorities are most
---------------------------------------------------------------------------
helpful? Please provide specific examples.
A.2. Resource constraints, regulatory bureaucracy, and
statutory restrictions explain why existing statutes,
authorities, or resources limit the ability of policymakers and
regulators to oversee novel software or models using ML and AI.
For example, under the Fair Housing Act and ECOA, HUD, and the
CFPB, respectively, have subpoena power and can make referrals
to the DOJ, but the DOJ is only able to make information
requests. States have subpoena power but must defer to Federal
regulators for cases against national banks. Congress could
increase the effectiveness of enforcement efforts by expanding
the subpoena power of the DOJ and the States.
A Federal judge recently dismissed a case filed by the CFPB
alleging that a mortgage company's communications would deter
prospective Black mortgage applicants, ruling that the ECOA
only applies to actual loan applicants, in contrast to the Fair
Housing Act, which covers potential applicants. \7\ While this
case did not relate to the use of AI/ML, it has implications
for the ability to enforce AI/ML marketing practices. A
legislative solution would be to align ECOA with the Fair
Housing Act.
---------------------------------------------------------------------------
\7\ Andreano, Jr., Richard (2023), ``CFPB Suffers Significant
Defeat in ECOA Lawsuit Against Townstone Mortgage'', Consumer Finance
Monitor, Feb. 6, https://www.consumerfinancemonitor.com/2023/02/06/
cfpb-suffers-significant-defeat-in-ecoa-lawsuit-against-townstone-
mortgage/.
---------------------------------------------------------------------------
The Fair Housing Act's ``disparate impact'' rule prohibits
the institution of lending policies, including the use of AI/ML
models, that have an unjustified discriminatory effect based on
protected characteristics. Similarly, ECOA prohibits the use of
protected demographic characteristics in credit decisions.
These regulations require lenders to legitimize actions that
deny access to credit and housing opportunities but would be
more impactful if expanded to stipulate an affirmative
obligation by federally regulated entities to expand access to
credit and housing opportunities.
------
RESPONSES TO WRITTEN QUESTIONS OF CHAIR SMITH
FROM NICHOLAS SCHMIDT
Q.1. How do we build incentives into housing and regulatory
systems so that we are maximizing fairness, accountability, and
transparency as we try to set guidelines both to encourage
innovation but also to make sure these systems are working for
the public? Can and should this be handled legislatively? Dr.
Perry underscored the importance of holding ``AI accountable
for expanding opportunity,'' and I welcome legislative and
regulatory recommendations for how to best incentivize this.
A.1. It is essential to recognize that innovation and the
principles of fairness, accountability, and transparency do not
conflict. In fact, these principles are the catalysts that
drive sustainable and impactful innovation. Rather than
hindering progress, these values guide the development of
technologies that will ultimately lead to successful
innovation, serve the public good, and ensure equitable
outcomes. Fairness is an integral component of successful
innovation. Innovations that exacerbate disparities based on
race, gender, age, or other categories cannot be deemed
successful. Implementing standards that mandate testing and
validation of the fairness of models mitigates risk and
promotes innovation that benefits all segments of society. The
importance of accountability in fostering successful innovation
becomes evident when we reflect on the causes and responses to
the 2008-2009 housing crisis. The lack of accountability by
individuals making investment decisions was a significant
factor in the crisis. However, the resilience of the lending
industry since then can be attributed, in part, to enhanced
regulatory standards for model risk management introduced by
the Federal Reserve and the Office of the Comptroller of the
Currency in 2011. These standards have significantly improved
the vetting, testing, and monitoring of algorithmic decision-
making systems, and our economy's success demonstrates that
accountability is a cornerstone of robust and responsible
innovation. Transparency, likewise, plays a pivotal role in
promoting responsible innovation within the housing sector. AI
is, without a doubt, overhyped. As a result, there has been a
tendency to implement AI systems without proper oversight in
many industries. By encouraging and requiring more
transparency, regulators and legislators will push companies to
consider whether an insufficiently tested system is worth the
potential for reputational harm that might result from
disclosure through transparency requirements. For example, a
modeler might want to use an unnecessarily and overly complex
algorithm to assess credit quality (I have seen unnecessarily
complex algorithms put into production far too frequently in
unregulated industries). However, suppose the company
understands that it has transparency requirements and must be
able to explain why the algorithm made a decision. If the
algorithm is too complex, the company cannot reasonably explain
the algorithm's decisions. This may drive business and
compliance leaders of the company to refuse to allow such a
complex model to be used. This is beneficial because overly
complex algorithms are often unfair and unsafe. Being
encouraged to push back against the desire to ``move fast and
break things'' will lead to a more considered and mature
adoption of AI.
While fairness, accountability, and transparency are
compatible with and necessary for successful innovation, it is
crucial to acknowledge the potential for legislative efforts to
inadvertently impede progress. Successful legislation should be
principled rather than prescriptive, offering a framework that
fosters innovation while ensuring it operates within ethical
boundaries. This approach allows flexibility and adaptation,
which are essential for nurturing innovation in a rapidly
evolving technological landscape.
Q.2. You underscored the bevy of existing laws that apply to
and cover most of the AI applications in housing. What are the
legislative gaps you see in this space, and how do you
recommend that Congress addresses them?
A.2. With exceptions related to the provision of transparent
explanations, standards around proxy discrimination, and
mandating that modelers search for less discriminatory
alternative models (which I outline in my responses to Senator
Warnock), large lending institutions are generally well-covered
by existing laws. However, smaller companies, including third-
party model providers, have generally not been subject to the
rigorous standards against which large institutions are
assessed in terms of model governance, model quality, fairness,
and safety. While it is likely unreasonable to require that
smaller companies meet the compliance standards of the top
banks, they should be subject to more rigorous standards and
expectations. Creating legislation that provides more funding
for existing regulators to understand these issues and
investigate companies that do not meet minimum standards would
help ensure that these models, which have a significant effect
on people's ability to rent apartments or buy homes, would be
fairer and more effective.
------
RESPONSES TO WRITTEN QUESTIONS OF SENATOR WARNOCK
FROM NICHOLAS SCHMIDT
Q.1. As credit underwriting models increasingly employ newer
and larger sets of data, I am concerned that historical
discrimination in housing may lead to biased training data.
What is your current assessment of the quality of training
data available for those building credit models?
A.1. The quality of training data used to construct credit
models is often inconsistent. Data quality is generally high
for individuals with sufficient historical credit history to be
evaluated through traditional credit scoring models, such as
FICO or Vantage. While this observation may diverge from
narratives frequently encountered in the press, credit bureaus
maintain robust standards for accuracy and offer mechanisms for
data dispute and correction. This achievement is, in part, a
testament to effective legislative frameworks that mandate
transparency and consumer rights to data access.
Conversely, the emergent forms of data increasingly
incorporated into credit assessments--particularly for those
lacking a traditional credit score--exhibit varying degrees of
quality. The issues stem from several factors, including the
nascent stage of integrating such data into credit algorithms,
the relative absence of mandates for data disclosure and
correction, and the usage of complex algorithms that may
obfuscate the impact of erroneous data on consumer credit
evaluations. The latter of these issues may require significant
efforts to uncover and address data inaccuracies, an effort
that is often not sufficiently addressed by some model builders
or users.
Further, the issue of data incompleteness is more
pronounced in these newer data streams. The dependency on
third-party data providers that may not consistently capture
information across all consumer segments exacerbates this
challenge. Notably, discrepancies in data completeness often
manifest along racial lines, underscoring the need for vigilant
review and monitoring to prevent systemic biases.
Despite these challenges, it is also crucial to recognize
the potential benefits of incorporating alternative data
sources. For individuals with limited or no traditional credit
history, access to such data enables financial institutions to
extend credit and housing opportunities that would otherwise be
inaccessible. This can significantly enhance financial equity
and provide opportunities to underserved populations.
Therefore, while maintaining vigilance and concern about data
quality and its associated fairness, it is essential to
consider that its use often promotes financial inclusion.
Q.2. What standards are currently used by industry to assess an
AI model's quality in terms of its data inputs as well as other
model characteristics (e.g., accuracy, bias and discrimination,
variance, performance on sparse data, etc.)?
A.2. Before machine learning (ML) and AI became widespread,
most major financial services companies had robust processes to
assess the fairness and reasonableness of decision making
(whether subjective or algorithmic). As algorithmic decisioning
has become more common, most companies have successfully
applied these processes to their machine learning and AI
algorithms. In particular, the legal framework of disparate
treatment, disparate impact, and proxy discrimination is the
most commonly used framework to assess fairness. This framework
assesses both the data used to build the algorithm and the
fairness of the algorithm's output.
This framework has a rich history. While originating in the
employment context, it has been extended to housing, lending,
and other industries, including insurance and technology. Its
advantages include its comprehensive yet flexible nature. It
also addresses how discrimination has manifested in our
society, and it provides an avenue to encourage inclusion
without being unnecessarily burdensome. Finally, it is well-
designed to be applied to questions of fairness and inclusion
in algorithmic decisioning by machine learning and AI.
AI, ML, and the data used for these algorithms present some
challenges to applying this framework, particularly in
assessing proxy discrimination and disparate impact.
Institutions would benefit from having more clarity around what
types of data they can use and what they are expected to do to
minimize any disparities in their models.
Proxy discrimination involves making a lending decision
based at least partially on a factor so closely related to
membership in a protected class that it is essentially just a
stand-in for that class. The classic example of proxy
discrimination is redlining, which is the refusal to lend in
certain neighborhoods that were almost always heavily minority
neighborhoods. Lenders justified this by claiming that these
neighborhoods were riskier to lend in than neighborhoods that
were majority White. This was a racist policy, and lenders go
to great lengths to avoid obvious proxies like this.
However, in machine learning and AI, modelers use far more
data than traditional credit bureau attributes, which are much
less likely to be proxies for a protected class. When building
these models, modelers may want to incorporate information
about such varied things as marketing preferences, shopping
preferences, and patterns of speech or writing that may cross
the line and become proxies for a protected class. While it may
seem clear that these variables should be avoided if they might
proxy for a protected class, the question becomes more
difficult for at least two reasons. First, using these
variables may increase financial inclusion for people without
sufficient traditional credit history--even for members of the
protected class proxied by the variable. The question may be
whether to deny a loan to everyone without sufficient credit
history or to use these variables, accept some people without
sufficient credit history, but accept relatively fewer members
of a protected class than truly should be accepted if better
data existed. The answer to this question is ultimately a
public policy question that individual companies in the housing
industry cannot answer on their own without encountering
significant regulatory and legal risks. Thus, having clearer
regulations around the use of potentially problematic variables
would be beneficial.
The second reason why this is a difficult question is that
there is not a clear dividing line between disparate impact and
proxy discrimination. While we can make some distinctions on
certain variables, others are less obvious. For example, a
variable that indicates where a person shops for clothing is
almost certainly a proxy for sex. However, a variable that
indicates the percent of disposable income a person spends on
clothing may not be a proxy. Suppose the second variable is
still correlated with sex, but it is also a robust predictor of
whatever outcome is being modeled. In that case, it might be
considered a source of disparate impact, not proxy
discrimination. As such, it is not necessarily illegal. If the
first variable does represent a proxy risk and the second does
not, then it is obvious that there is some line between the two
where a variable moves from being a cause of disparate impact
to being a cause of proxy discrimination. The difficulty is
that there is no clear guidance on where that line is. Industry
participants would benefit from regulations that set out
clearer guidance on what constitutes proxy discrimination.
Machine learning and AI also present new issues related to
disparate impact. While disparate treatment represents the
explicit inclusion of protected class status into a decision,
disparate impact is perhaps a subtler form of discrimination.
In a disparate impact analysis of a model, we start with a very
simple test. In the case of a yes/no outcome, we test whether a
protected group receives the favorable outcome less frequently
than a member of some reference group. For example, we test
whether Black or African American applicants receive loan
offers less frequently than non-Hispanic White applicants. In
the case of a continuous outcome, such as interest rates paid
for a loan, we assess whether the protected group has, on
average, received worse terms than the reference group. Using
interest rates as an example, we might test whether women pay
higher APRs than men.
Importantly, if we find evidence of unfavorable
differences, we do not necessarily assume that the algorithm is
illegally discriminatory. Instead, we move to the next step of
assessing whether the algorithm's use has sufficient business
justification and that none of the variables in the model
represent disparate treatment or proxy discrimination. For
example, a well-built credit underwriting model that has passed
rigorous model governance standards and uses traditional credit
bureau data would almost certainly pass this hurdle.
If the model passes this second stage, we then test whether
a similar process can still meet the institution's legitimate
business needs but have a smaller negative impact on protected
class members. Searching for such an alternative model is
called a ``search for less discriminatory alternative models''
or ``LDA search.''
Machine learning and AI have significantly extended the
ability to effectively search for LDA models. In traditional
models, the approach was to try adding or dropping a handful of
variables and see whether that might decrease disparities while
maintaining model quality. While sometimes effective, this
approach often did not result in significant changes. In the
new age of ML and AI, we have developed new techniques that
have greatly expanded the ability to search for LDA models.
Adopting such techniques can significantly increase equitable
and fair outcomes in housing.
While we have seen significant positive results from
implementing these techniques, there is a lack of regulatory
clarity around whether this is required, what techniques are
allowed, and what standards will meet regulatory expectations.
Legislation that clearly outlines that searching for less
discriminatory models is a requirement and allows regulators to
set the guidelines for what is acceptable will encourage
industry participants to adopt these approaches. This could
drive significant and positive change by making housing
outcomes fairer and more equitable.
Q.3. In your written testimony, you stated, ``Many . . . AI and
ML algorithms are described as `black-box' processes . . .
[W]hile we understand what data went into the algorithm, and we
can see what output of the algorithm is, it is very difficult
to understand how or why the algorithm made the prediction it
made.'' \1\ You also note that ``a lender might not be able to
provide a borrower with a reliable explanation for why the home
they wanted to buy received a low appraisal.'' \2\
---------------------------------------------------------------------------
\1\ Testimony of Nicholas Schmidt, U.S. Senate Committee on
Banking, Housing, and Urban Affairs, Subcommittee on Housing,
Transportation, and Community Development (Jan. 31, 2024), https://
www.banking.senate.gov/imo/media/doc/schmidt-testimony-1-31-24.pdf at
8.
\2\ Id.
---------------------------------------------------------------------------
In your experience, how commonly are ``black-box''
algorithms used in housing?
A.3. These algorithms are now quite commonly used in housing.
They are used throughout the industry, most notably for
marketing, appraisals, credit underwriting, loan pricing,
evaluating renters, and servicing loans.
Q.4. How can regulators better ensure that lenders and other
institutions use models capable of providing transparent
explanations for their decisions?
A.4. Regulators may be able to encourage lenders to use
appropriately transparent algorithms by assessing and judging
them based on the quality of their models' explanations. If a
lender knows it must give regulators evidence that its
algorithms provide reasonably accurate explanations, it will be
less likely to use an unnecessarily complex algorithm.
The difficulty is that, even with relatively simple
algorithms, all explanation methodologies have shortcomings;
even the most robust approach will not always provide the best
explanation for why an algorithm made a decision. This lack of
perfection should not be a reason to reject AI and machine
learning because well-built machine learning or AI algorithms
significantly benefit consumers and industry participants.
Regulators have not been clear about what constitutes a
sufficiently robust explanation methodology. While this lack of
clarity has been reasonable given the pace of technological
change, we are now at a point where better guidance would
benefit consumers and the industry. However, rather than
prescribing a particular methodology, effective regulation
would put forth methodologies for testing and evaluating the
robustness of explanations. A company could then be evaluated
on how well it tested its methodology and whether that
methodology passed the tests it constructed. The benefits of
this approach include that it would allow continued innovation
by not prescribing an approach that may quickly find itself out
of date while also providing companies with a clearer standard
against which they can judge themselves.
Additional Material Supplied for the Record
[GRAPHIC(S) NOT AVAILABLE IN TIFF FORMAT]