[Federal Register Volume 86, Number 13 (Friday, January 22, 2021)]
[Notices]
[Pages 6612-6626]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2021-01273]


 ========================================================================
 Notices
                                                 Federal Register
 ________________________________________________________________________
 
 This section of the FEDERAL REGISTER contains documents other than rules 
 or proposed rules that are applicable to the public. Notices of hearings 
 and investigations, committee meetings, agency decisions and rulings, 
 delegations of authority, filing of petitions and applications and agency 
 statements of organization and functions are examples of documents 
 appearing in this section.
 
 ========================================================================
 

  Federal Register / Vol. 86, No. 13 / Friday, January 22, 2021 / 
Notices  

[[Page 6612]]



ADMINISTRATIVE CONFERENCE OF THE UNITED STATES


Adoption of Recommendations

AGENCY: Administrative Conference of the United States.

ACTION: Notice.

-----------------------------------------------------------------------

SUMMARY: The Administrative Conference of the United States adopted six 
recommendations and one official statement at its virtual Seventy-third 
Plenary Session. The appended recommendations address: (a) Rules on 
Rulemakings; (b) Protected Materials in Public Rulemaking Dockets; (c) 
Agency Appellate Systems; (d) Government Contract Bid Protests Before 
Agencies; (e) Publication of Policies Governing Agency Adjudicators; 
and (f) Agency Litigation Webpages. The official statement addresses 
Agency use of Artificial Intelligence.

FOR FURTHER INFORMATION CONTACT: For Recommendations 2020-1 and 2020-2, 
Todd Rubin; for Recommendation 2020-3, Gavin Young; for Recommendations 
2020-4 and 2020-6, and Statement #20, Mark Thomson; and for 
Recommendation 2020-5, Leigh Anne Schriever. For each of these actions 
the address and telephone number are: Administrative Conference of the 
United States, Suite 706 South, 1120 20th Street NW, Washington, DC 
20036; Telephone 202-480-2080.

SUPPLEMENTARY INFORMATION: The Administrative Conference Act, 5 U.S.C. 
591-596, established the Administrative Conference of the United 
States. The Conference studies the efficiency, adequacy, and fairness 
of the administrative procedures used by Federal agencies and makes 
recommendations to agencies, the President, Congress, and the Judicial 
Conference of the United States for procedural improvements (5 U.S.C. 
594(1)). For further information about the Conference and its 
activities, see www.acus.gov. At its virtual Seventy-third Plenary 
Session on December 16-17, 2020, the Assembly of the Conference adopted 
six recommendations and one official statement.
    Recommendation 2020-1, Rules on Rulemakings. This recommendation 
encourages agencies to consider issuing rules governing their 
rulemaking procedures. It identifies subjects that agencies should 
consider addressing in their rules on rulemakings--without prescribing 
any particular procedures--and it urges agencies to solicit public 
input on these rules and make them publicly available.
    Recommendation 2020-2, Protected Materials in Public Rulemaking 
Dockets. This recommendation offers agencies best practices for 
protecting sensitive personal and confidential commercial information 
in public rulemaking dockets. It identifies, in particular, best 
practices for agencies to use when redacting, summarizing, and 
aggregating comments that contain such information. It also encourages 
agencies to provide public notices that discourage commenters from 
submitting such information in the first place.
    Recommendation 2020-3, Agency Appellate Systems. This 
recommendation offers agencies best practices to improve administrative 
review of hearing-level adjudicative decisions with respect to case 
selection, decision-making process and procedures, management 
oversight, and public disclosure and transparency. In doing so, it 
encourages agencies to identify the objectives of such review and 
structure their appellate systems to serve those objectives.
    Recommendation 2020-4, Government Contract Bid Protests Before 
Agencies. This recommendation suggests improvements to the procedures 
governing agency-level procurement contract disputes--commonly called 
bid protests--under the Federal Acquisition Regulation and agency-
specific regulations to make those procedures more simple, transparent, 
and predictable. It urges agencies to clarify what types of decisions 
can be the subjects of agency-level bid protests, what processes and 
deadlines will govern such protests, and who in the agency will decide 
such protests; make it easier for protesters to get information about 
the decisions they protest; and publish more data on agency-level 
protests.
    Recommendation 2020-5, Publication of Policies Governing Agency 
Adjudicators. This recommendation encourages agencies to disclose 
policies governing the appointment and oversight of adjudicators that 
bear on their impartiality and constitutional status. It offers best 
practices on how to provide descriptions of, and access to, such 
policies on agency websites.
    Recommendation 2020-6, Agency Litigation Webpages. This 
recommendation offers agencies best practices for making their federal 
court filings and relevant court opinions available to the public on 
their websites, with particular emphasis on materials from litigation 
dealing with agency regulatory programs. It provides guidance on the 
types of litigation materials that will be of greatest interest to the 
public and on how agencies can disseminate the materials in a way that 
makes them easy to find.
    Statement #20, Agency Use of Artificial Intelligence. This 
statement identifies issues agencies should consider when adopting, 
revamping, establishing policies and practices governing, and regularly 
monitoring artificial intelligence systems. Among the topics it 
addresses are transparency, harmful biases, technical capacity, 
procurement, privacy, security, decisional authority, and oversight.
    The Appendix below sets forth the full texts of these six 
recommendations and the official statement. The Conference will 
transmit the recommendations and statement to affected agencies, 
Congress, and the Judicial Conference of the United States, as 
appropriate. The recommendations and statement are not binding, so the 
entities to which they are addressed will make decisions on their 
implementation.
    The Conference based these recommendations and the statement on 
research reports that are posted at: https://www.acus.gov/meetings-and-events/plenary-meeting/73rd-plenary-session. Committee-proposed drafts 
of the recommendations and statement, and public comments received in 
advance of the plenary session, are also available using the same link.


[[Page 6613]]


    Dated: January 14, 2021.
Shawne C. McGibbon,
General Counsel.

Appendix--Recommendations and Statement of the Administrative 
Conference of the United States

Administrative Conference Recommendation 2020-1

Rules on Rulemakings

Adopted December 16, 2020

    Numerous agencies have promulgated rules setting forth the 
policies and procedures they will follow when conducting informal 
rulemakings under 5 U.S.C. 553.\1\ The rules can cover a variety of 
practices, including processes for initiating and seeking public 
input on new rules, coordinating with the Office of Management and 
Budget and other agencies as a rule is being formulated, and 
obtaining approval from agency leadership before a proposed rule is 
issued or finalized. Agencies refer to these rules by different 
names. This Recommendation calls them ``rules on rulemakings.''
---------------------------------------------------------------------------

    \1\ This Recommendation does not address rulemakings subject to 
the formal hearing requirements of the Administrative Procedure Act. 
See 5 U.S.C. 556-57.
---------------------------------------------------------------------------

    Rules on rulemakings vary--in terms of the particular matters 
they address, their scope and comprehensiveness, and other 
characteristics--but they share several common features. First, they 
authoritatively reflect the agency's position as to what procedures 
it will observe when adopting new rules. By ``authoritative,'' this 
Recommendation means that a rule on rulemakings sets forth the 
procedures that agency officials responsible for drafting and 
finalizing new rules will follow in at least most cases within the 
rule on rulemakings' scope, though it may contemplate the 
possibility that agency leadership could authorize an alternative 
set of procedures.\2\
---------------------------------------------------------------------------

    \2\ Cf. Admin. Conf. of the U.S., Recommendation 2019-1, Agency 
Guidance Through Interpretive Rules, 84 FR 38927 (Aug. 8, 2019); 
Admin. Conf. of the U.S., Recommendation 2017-5, Agency Guidance 
Through Policy Statements, 82 FR 61734 (Dec. 29, 2017).
---------------------------------------------------------------------------

    Second, rules on rulemakings do more than simply summarize or 
explain rulemaking requirements of the Administrative Procedure Act 
and other statutes, although they often serve an explanatory 
function at the same time that they set forth the procedures the 
agencies will follow in conducting rulemakings. Rules on rulemakings 
set forth additional commitments by an agency concerning how it will 
conduct rulemakings. And third, agencies disseminate rules on 
rulemakings publicly rather than only internally. They appear on 
agency websites and are often published not only in the Federal 
Register but also in the Code of Federal Regulations (CFR).
    Rules on rulemakings can serve at least four important 
objectives. First, they promote efficiency by ensuring that both 
agency officials and those outside the agency know where to go to 
find the agency's rulemaking policies. Second, they promote 
predictability by informing the public that the agency will follow 
particular procedures, thereby allowing the public to plan their 
participation in the rulemaking process accordingly. Third, they 
promote accountability by ensuring that agency leadership has 
approved the policies and procedures the agency will follow. And 
they can also provide accountability in connection with individual 
rulemakings by creating an internal approval process by which agency 
leadership reviews proposed and final rules. Finally, they promote 
transparency by affording the public access to the agency's internal 
procedures pertaining to its rulemaking process.
    In promulgating a rule on rulemakings, an agency may wish to 
solicit public input to inform the rule's development, even if such 
a rule is subject to 5 U.S.C. 553's exemption from notice-and-
comment procedures as a rule of procedure, general statement of 
policy, or otherwise. In soliciting public input, agencies may wish 
to use mechanisms that facilitate more robust participation, 
including by underrepresented communities.\3\ As the Administrative 
Conference has acknowledged in past recommendations, public comment 
can both provide valuable input from the public and enhance public 
acceptance of an agency's rules.\4\
---------------------------------------------------------------------------

    \3\ See, e.g., 2 U.S.C. 1534 (Unfunded Mandates Reform Act); 5 
U.S.C. 609 (Regulatory Flexibility Act); Exec. Order No. 13,175, 
Consultation and Coordination with Indian Tribal Governments, 65 FR 
67249 (Nov. 11, 2000).
    \4\ See Admin. Conf. of the U.S., Recommendation 92-1, The 
Procedural and Practice Rule Exemption from the APA Notice-and-
Comment Rulemaking Requirements, 57 FR 30102 (July 8, 1992); see 
also Recommendation 2019-1, supra note 2; Recommendation 2017-5, 
supra note 2.
---------------------------------------------------------------------------

    An agency may also wish to publish its rule on rulemakings in 
the CFR. Doing so can enhance transparency and facilitate 
accountability. Importantly, publishing a rule on rulemakings in the 
CFR does not, by itself, make the rule on rulemakings judicially 
enforceable.\5\
---------------------------------------------------------------------------

    \5\ See, e.g., Health Ins. Ass'n of Am. v. Shalala, 23 F.3d 412, 
423 (D.C. Cir. 1994) (stating that ``publication in the Code of 
Federal Regulations, or its absence'' is only ``a snippet of 
evidence of agency intent'' that the published pronouncement be 
given binding effect).
---------------------------------------------------------------------------

    This Recommendation does not seek to resolve whether, when, or 
on what legal bases a court might enforce a rule on rulemakings 
against an agency.\6\
---------------------------------------------------------------------------

    \6\ Some rules on rulemakings include a statement that they do 
not create any substantive or procedural rights or benefits. This 
Recommendation does not address whether such disclaimers should be 
included or what legal effect they may have on judicial review. 
These questions cannot be answered in isolation from the broader 
question of when a rule on rulemakings is judicially enforceable.
---------------------------------------------------------------------------

Recommendation

    1. Agencies should consider promulgating rules on rulemakings 
setting forth the policies and procedures they will follow in 
informal rulemaking under 5 U.S.C. 553.
    2. In issuing rules on rulemakings, agencies should consider 
including provisions addressing the following topics (which reflect 
topics frequently covered in existing rules on rulemakings):
    a. Procedures prior to the issuance of a notice of proposed 
rulemaking;
    b. Procedures connected with the notice-and-comment process;
    c. Procedures connected with the presidential review process, if 
applicable;
    d. Procedures for handling post-comment period communications;
    e. Internal approval procedures for issuing and finalizing 
rules; and
    f. Procedures for reassessing existing rules.
    The appendix gives examples of particular subtopics agencies may 
wish to consider under each of these topics.
    3. Agencies should make rules on rulemakings available in a 
prominent, easy-to-find place on the portion of their websites 
dealing with rulemaking matters. Additionally, agencies should 
consider publishing them in the Federal Register and the Code of 
Federal Regulations. When posting rules on rulemakings on their 
websites, agencies should use techniques like linked tabs, pull-down 
menus, indexing, tagging, and sorting tables to ensure that relevant 
documents are easily findable. Agencies should also design their 
search engines to allow people to easily identify relevant 
documents.
    4. In addition to issuing rules on rulemakings, agencies should 
consider explaining in accessible language how the rulemaking 
process works in order to educate the public. Such explanations 
might be integrated within a rule on rulemakings or might be 
contained in separate explanatory documents (e.g., documents 
identifying frequently asked questions). When providing such 
explanations, an agency should, to the extent practicable, 
distinguish between procedures it intends to follow and material 
provided purely by way of background.
    5. Agencies should consider a broad range of means of seeking 
public input on rules on rulemakings, even if the Administrative 
Procedure Act does not require it.
    6. Agencies should consider the extent to which procedures 
required by a rule on rulemakings should be made internally waivable 
and, if so, by whom. For example, they might consider drafting a 
rule on rulemakings in a way that allows high-level agency officials 
to permit other officials to use alternative procedures.

Appendix

Non-Exhaustive List of Topics for Agencies To Consider Including Within 
Their Rules on Rulemakings

(a) Procedures Prior to the Issuance of a Notice of Proposed 
Rulemaking

    Subtopic Examples:
    (1) Regulatory planning; \7\
---------------------------------------------------------------------------

    \7\ See Admin. Conf. of the U.S., Recommendation 2015-1, 
Promoting Accuracy and Transparency in the Unified Agenda, 80 FR 
36757 (June 26, 2015).
---------------------------------------------------------------------------

    (2) Issuing advance notices of proposed rulemaking and obtaining 
feedback from members of the public using means other than the 
notice-and-comment process, such

[[Page 6614]]

as requests for information and focus groups; \8\
---------------------------------------------------------------------------

    \8\ See Admin. Conf. of the U.S., Recommendation 2018-7, Public 
Engagement in Rulemaking, 84 FR 2146 (Feb. 6, 2019).
---------------------------------------------------------------------------

    (3) Accepting, reviewing, and responding to petitions for 
rulemaking; \9\
---------------------------------------------------------------------------

    \9\ See Admin. Conf. of the U.S., Recommendation 2014-6, 
Petitions for Rulemaking, 79 FR 75117 (Dec. 17, 2014).
---------------------------------------------------------------------------

    (4) Considering options besides rulemaking;
    (5) Performing ex ante regulatory analyses (e.g., benefit-cost 
analysis and regulatory flexibility analysis); \10\
---------------------------------------------------------------------------

    \10\ See Admin. Conf. of the U.S., Recommendation 2012-1, 
Regulatory Analysis Requirements, 77 FR 47801 (Aug. 10, 2012).
---------------------------------------------------------------------------

    (6) Using plain language in regulatory drafting; \11\
---------------------------------------------------------------------------

    \11\ See Admin. Conf. of the U.S., Recommendation 2017-3, Plain 
Language in Regulatory Drafting, 82 FR 61728 (Dec. 29, 2017).
---------------------------------------------------------------------------

    (7) Preparing for potential judicial review of rulemakings, 
including deciding whether to make any of the provisions of a rule 
severable; \12\
---------------------------------------------------------------------------

    \12\ See Admin. Conf. of the U.S., Recommendation 2018-2, 
Severability in Agency Rulemaking, 83 FR 30685 (June 29, 2018).
---------------------------------------------------------------------------

    (8) Conducting negotiated rulemaking; \13\ and
---------------------------------------------------------------------------

    \13\ See Admin. Conf. of the U.S., Recommendation 2017-2, 
Negotiated Rulemaking and Other Options for Public Engagement, 82 FR 
31040 (July 5, 2017).
---------------------------------------------------------------------------

    (9) Establishing an effective date for rules.

(b) Procedures Connected With the Notice-and-Comment Process

    Subtopic Examples:
    (1) Materials to be published on Regulations.gov with the 
notice; \14\
---------------------------------------------------------------------------

    \14\ See Admin. Conf. of the U.S., Recommendation 2018-6, 
Improving Access to Regulations.gov's Rulemaking Dockets, 84 FR 2143 
(Feb. 6, 2019).
---------------------------------------------------------------------------

    (2) Minimum comment periods to be allowed; \15\
---------------------------------------------------------------------------

    \15\ See Admin. Conf. of the U.S., Recommendation 2011-2, 
Rulemaking Comments, 76 FR 48791 (Aug. 9, 2011).
---------------------------------------------------------------------------

    (3) Policies on ex parte contacts; \16\
---------------------------------------------------------------------------

    \16\ See Admin. Conf. of the U.S., Recommendation 2014-4, ``Ex 
Parte'' Communications in Informal Rulemaking, 79 FR 35993 (June 25, 
2014).
---------------------------------------------------------------------------

    (4) Handling external merits communications not filed as 
comments;
    (5) Incorporating standards by reference; \17\
---------------------------------------------------------------------------

    \17\ See Admin. Conf. of the U.S., Recommendation 2011-5, 
Incorporation by Reference, 77 FR 2257 (Jan. 17, 2012).
---------------------------------------------------------------------------

    (6) Using social media to engage the public in rulemaking; \18\
---------------------------------------------------------------------------

    \18\ See Admin. Conf. of the U.S., Recommendation 2013-5, Social 
Media in Rulemaking, 78 FR 76269 (Dec. 17, 2013).
---------------------------------------------------------------------------

    (7) Obtaining feedback from American Indian tribes, other 
historically underrepresented or under-resourced groups, and state 
and local governments; \19\
---------------------------------------------------------------------------

    \19\ See Recommendation 2018-7, supra note 8.
---------------------------------------------------------------------------

    (8) Posting, analyzing, and responding to public comments, 
including comments that may contain confidential commercial 
information, protected personal information, or other kinds of 
sensitive submissions; \20\
---------------------------------------------------------------------------

    \20\ See Admin. Conf. of the U.S., Recommendation 2020-2, 
Protected Materials in Public Rulemaking Dockets, 86 FR (approved 
Dec. 16, 2020); Admin. Conf. of the U.S., Recommendation 2011-1, 
Legal Considerations in e-Rulemaking, 76 FR 48789 (Aug. 9, 2011).
---------------------------------------------------------------------------

    (9) Waiving or invoking of Administrative Procedure Act 
exemptions to notice and comment; \21\ and
---------------------------------------------------------------------------

    \21\ See Recommendation 92-1, supra note 4.
---------------------------------------------------------------------------

    (10) Using interim final rules or direct final rules.\22\
---------------------------------------------------------------------------

    \22\ See Admin. Conf. of the U.S., Recommendation 95-4, 
Procedures for Noncontroversial and Expedited Rulemakings, 60 FR 
43108 (Aug. 18, 1995).
---------------------------------------------------------------------------

(c) Procedures Connected With the Presidential Review Process, if 
Applicable

    Subtopic Examples:
    (1) Interacting with the Office of Information and Regulatory 
Affairs, the Office of the Federal Register, the Regulatory 
Information Service Center, the Small Business Administration's 
Office of Advocacy, and other offices with government-wide 
rulemaking responsibilities;
    (2) Participating in the interagency review process; and
    (3) Procedures related to international regulatory 
cooperation.\23\
---------------------------------------------------------------------------

    \23\ See Admin. Conf. of the U.S., Recommendation 2011-6, 
International Regulatory Cooperation, 77 FR 2259 (Jan. 17, 2012).
---------------------------------------------------------------------------

(d) Procedures for Handling Post-Comment Period Communications

    Subtopic Examples:
    (1) Provisions pertaining to reply comments \24\ and
---------------------------------------------------------------------------

    \24\ See Recommendation 2011-2, supra note 15.
---------------------------------------------------------------------------

    (2) Handling late-filed comments.\25\
---------------------------------------------------------------------------

    \25\ See id.
---------------------------------------------------------------------------

(e) Internal Approval Procedures for Issuing and Finalizing Rules

    Subtopic Examples:
    (1) Procedures for submitting rules to offices with legal, 
economic, and other responsibilities within the agency for review 
\26\ and
---------------------------------------------------------------------------

    \26\ See Admin. Conf. of the U.S., Recommendation 2019-5, Agency 
Economists, 84 FR 71349 (Dec. 27, 2019).
---------------------------------------------------------------------------

    (2) Procedures for submitting rules to the relevant agency 
official for final approval.

(f) Procedures for Reassessing Existing Rules

    Subtopic Examples:
    (1) Issuing regulatory waivers and exemptions; \27\
---------------------------------------------------------------------------

    \27\ See Admin. Conf. of the U.S., Recommendation 2017-7, 
Regulatory Waivers and Exemptions, 82 FR 61742 (Dec. 29, 2017).
---------------------------------------------------------------------------

    (2) Engaging in retrospective review of rules; \28\
---------------------------------------------------------------------------

    \28\ See Admin. Conf. of the U.S., Recommendation 2014-5, 
Retrospective Review of Agency Rules, 79 FR 75114 (Dec. 17, 2014).
---------------------------------------------------------------------------

    (3) Maintaining and preserving rulemaking records, including 
transparency of such records and the handling of confidential 
commercial information, protected personal information, or other 
kinds of sensitive information contained therein; \29\ and
---------------------------------------------------------------------------

    \29\ See Admin. Conf. of the U.S., Recommendation 2013-4, 
Administrative Record in Informal Rulemaking, 78 FR 41358 (July 10, 
2013).
---------------------------------------------------------------------------

    (4) Handling rules that have been vacated or remanded without 
vacatur.\30\
---------------------------------------------------------------------------

    \30\ See Admin. Conf. of the U.S., Recommendation 2013-6, Remand 
Without Vacatur, 78 FR 76272 (Dec. 17, 2013).
---------------------------------------------------------------------------

Administrative Conference Recommendation 2020-2

Protected Materials in Public Rulemaking Dockets

Adopted December 16, 2020

    As part of the rulemaking process, agencies create public 
rulemaking dockets, which consist of all rulemaking materials 
agencies have: (1) Proactively published online or (2) made 
available for public inspection in a reading room. Public rulemaking 
dockets include materials agencies generate themselves and comments 
agencies receive from the public. Their purpose is to provide the 
public with the information that informed agencies' rulemakings.\1\
---------------------------------------------------------------------------

    \1\ The public rulemaking docket is distinguished from ``the 
administrative record for judicial review,'' which is intended to 
provide courts with a record for evaluating challenges to the rule, 
and the ``rulemaking record,'' which means all comments and 
materials submitted to agencies during comment periods and any other 
materials agencies considered during the course of the rulemaking. 
See Admin. Conf. of the U.S., Recommendation 2013-4, The 
Administrative Record in Informal Rulemaking, 78 FR 41358 (July 10, 
2013).
---------------------------------------------------------------------------

    The Administrative Conference has issued several recommendations 
to help agencies balance the competing considerations of 
transparency and confidentiality in managing their public rulemaking 
dockets.\2\ This project builds on these recommendations.
---------------------------------------------------------------------------

    \2\ Recommendation 2011-1, Legal Considerations in e-Rulemaking, 
advises agencies to allow submitters to flag confidential 
information, including trade secrets, and advises agencies to devise 
procedures for reviewing and handling such information. Admin. Conf. 
of the U.S., Recommendation 2011-1, Legal Considerations in e-
Rulemaking, ] 1, 76 FR 48789, 48790 (Aug. 9, 2011). Recommendation 
2013-4, supra note 1, ] 11, advises agencies to develop guidance on 
managing and segregating protected information, such as confidential 
commercial information and sensitive personal information, while 
disclosing non-protected materials; see also Admin. Conf. of the 
U.S., Recommendation 89-7, Federal Regulation of Biotechnology, 54 
FR 53494 (Dec. 29, 1988); Admin. Conf. of the U.S., Recommendation 
82-1, Exemption (b)(4) of the Freedom of Information Act, 47 FR 
30702 (July 15, 1982); Admin. Conf. of the U.S., Recommendation 80-
6, Intragovernmental Communications in Informal Rulemaking 
Proceedings, 45 FR 86408 (Dec. 31, 1980).
---------------------------------------------------------------------------

    The scope of the Recommendation is limited to personal 
information and confidential commercial information that agencies 
have decided to withhold from their public rulemaking dockets, which 
this Recommendation calls ``protected material.'' The Recommendation 
specifies how agencies should consider handling protected material. 
For purposes of this Recommendation, personal information is 
information about an individual including his or her education, 
financial transactions, medical history, criminal or employment 
history, or similarly sensitive information, and that contains his 
or her name, or the identifying number, symbol, or other identifying 
particular

[[Page 6615]]

assigned to the individual.\3\ Confidential commercial information 
is commercial information that is customarily kept private, or at 
least closely held, by the person or business providing it.\4\ Other 
types of information, such as national security information and 
copyrighted materials, are beyond the Recommendation's scope. The 
Recommendation is also limited to addressing procedures for 
protecting materials that agencies decide warrant protection. It is 
not intended to define the universe of protected materials. In 
particular, the Recommendation does not address any issue that may 
arise if agencies choose to rely on protected material in explaining 
their rulemakings, whether in notices of proposed rulemaking, 
regulatory impact analyses, or otherwise.
---------------------------------------------------------------------------

    \3\ See Privacy Act of 1974 Sec.  3, 5 U.S.C. 552a(a)(4).
    \4\ See Food Mktg. Inst. v. Argus Leader Media, 139 S. Ct. 2356, 
2363 (2019); see also Exec. Order No. 12,600, Predisclosure 
Notification Procedures for Confidential Commercial Information, 52 
FR 23781 (June 23, 1987).
---------------------------------------------------------------------------

    Agencies accept public comments for their public rulemaking 
dockets primarily through Regulations.gov, their own websites, and 
email. Regulations.gov and many agency websites that accept comments 
expressly notify the public that agencies may publish the 
information submitted in public comments.\5\ When people submit 
comments to agencies, however, agencies typically do not immediately 
publish the comments. Instead, agencies generally take time to 
screen comments before publishing them. Most agencies perform at 
least some kind of screening during this period.
---------------------------------------------------------------------------

    \5\ See Christopher Yoo, Protected Materials in Public 
Rulemaking Dockets 24 (Nov. 24, 2020) (report to the Admin. Conf. of 
the U.S.), https://www.acus.gov/report/final-report-protected-materials-public-rulemaking-dockets.
---------------------------------------------------------------------------

    For all agencies, whether to withhold or disclose protected 
material is governed by various laws: Some mandate disclosure, some 
mandate withholding, and some leave agencies with substantial 
discretion in deciding whether to disclose. Although a full 
description of those laws is beyond the scope of this 
Recommendation, a brief overview of at least some of this body of 
law helps to identify the issues agencies face.
    The Administrative Procedure Act requires agencies to ``give 
interested persons an opportunity to participate in rulemaking 
through submission of written data, views, or arguments.'' \6\ The 
United States Court of Appeals for the D.C. Circuit has interpreted 
this provision to ordinarily require that agencies make publicly 
available the critical information--including studies, data, and 
methodologies--underlying proposed rules.\7\
---------------------------------------------------------------------------

    \6\ 5 U.S.C. 553(c).
    \7\ Portland Cement Ass'n v. Ruckelshaus, 486 F.2d 375, 393 
(D.C. Cir. 1973). In addition to these public transparency 
requirements, there are a number of federal record-retention 
requirements of which agencies should be aware. See, e.g., 44 U.S.C. 
3301.
---------------------------------------------------------------------------

    The Privacy Act and the Trade Secrets Act place limits on the 
disclosure norm discussed above. Generally, the Privacy Act prevents 
agencies from disclosing any information about a person, such as 
medical records, educational background, and employment history, 
contained in agencies' systems of records, without that person's 
written consent.\8\ The Trade Secrets Act generally prevents 
agencies from disclosing trade secrets and other kinds of 
confidential commercial information, such as corporate losses and 
profits.\9\
---------------------------------------------------------------------------

    \8\ 5 U.S.C. 552a(b).
    \9\ 18 U.S.C. 1905.
---------------------------------------------------------------------------

    Both the Privacy Act and the Trade Secrets Act have exceptions. 
For the Privacy Act, the main exception relevant to this 
Recommendation is for information required to be released under the 
Freedom of Information Act (FOIA).\10\ The Trade Secrets Act only 
has one exception, which covers any materials authorized to be 
disclosed by statute (including FOIA) or regulation.\11\ Whether a 
particular piece of personal or confidential commercial information 
meets one of these exceptions often involves a complex determination 
that depends upon the exact type of information at issue and its 
contemplated use, and agencies must determine the applicability of 
the exceptions on a case-by-case basis. For example, whether FOIA 
authorizes disclosure of confidential commercial information may 
turn in part on whether agencies in receipt of the information 
assured submitters that the information would be withheld from the 
public.\12\ If agencies offer assurances that they will not disclose 
confidential commercial information, agencies and submitters may 
rely on those assurances as a defense against compelled disclosure 
under FOIA. In many cases, agencies assure companies that they will 
not disclose such information in order to encourage companies to 
submit it.
---------------------------------------------------------------------------

    \10\ 5 U.S.C. 552a(b)(2).
    \11\ See CNA Fin. Corp. v. Donovan, 830 F.2d 1132, 1137-43 (D.C. 
Cir. 1987).
    \12\ See Food Mktg. Inst., 139 S. Ct. at 2361.
---------------------------------------------------------------------------

    Particular cases are governed by specific requirements of law, 
not broad categorical labels. But agencies often consider certain 
categories of personal information and confidential commercial 
information to be protected material (e.g., trade secrets, social 
security numbers, bank account numbers, passport numbers, addresses, 
email addresses, medical information, and information concerning a 
person's finances).
    There are many ways protected material may arrive at the agency 
in a rulemaking. A person might submit his or her own information, 
intentionally or unintentionally, and then ask the agency not to 
disclose it. A third party might submit another person's 
information, with or without that person's knowledge. A company 
might submit a document containing its own confidential commercial 
information, intentionally or unintentionally, with or without the 
agency's prior assurance of protection. Or a company might submit 
another company's or person's information. Depending on the 
information in question and the manner in which it was submitted, 
there may be issues of waiver of statutory protection. Such 
questions, like all questions regarding the substance of the laws 
governing protected material, are beyond this Recommendation's 
scope, but they illustrate the various considerations that agencies 
and the public often face in the submission and handling of such 
material.
    This Recommendation proposes steps agencies can take to withhold 
protected materials from their public rulemaking dockets while still 
providing the public with the information upon which agencies relied 
in formulating proposed rules.\13\
---------------------------------------------------------------------------

    \13\ Permitting the submission of anonymous and pseudonymous 
comments is one way that some agencies attempt to reduce the privacy 
risks commenters face when submitting protected material. Issues 
regarding the submission of anonymous and pseudonymous comments are 
being considered in an ongoing project of the Administrative 
Conference titled Mass, Computer-Generated, and Fraudulent Comments 
and are beyond the scope of this Recommendation.
---------------------------------------------------------------------------

Recommendation

Recommendations for All Agencies

    1. To reduce the risk that agencies will inadvertently disclose 
protected material, agencies should describe what kinds of personal 
and confidential commercial information qualify as protected 
material and should clearly notify the public about their treatment 
of protected material. An agency's notifications should:
    a. Inform members of the public that comments are generally 
subject to public disclosure, except when disclosure is limited by 
law;
    b. Inform members of the public whether the agency offers 
assurances of protection from disclosure for their confidential 
commercial information and, if so, how to identify such information 
for the agency;
    c. Provide guidance to the public concerning the submission of 
protected material that pertains to third parties, including 
instructions that the disclosure of some protected material may be 
prohibited by law;
    d. Advise members of the public to review their comments for the 
material identified above in (c) and, if they find such material, to 
remove any such material that is not essential to the comment;
    e. Inform members of the public that they may request, during 
the period between when a comment is received and when it is made 
public, that protected material they inadvertently submitted be 
withheld from the public rulemaking docket;
    f. Inform members of the public that they may request, after the 
agency has published any comment, that protected material pertaining 
to themselves or to their dependents within the comment be removed 
from the public rulemaking docket; and
    g. Inform members of the public that the agency reserves the 
right to redact or aggregate any part of a comment if the agency 
determines that it constitutes protected material, or may withhold a 
comment in its entirety if it determines that redaction or 
aggregation would insufficiently prevent the disclosure of this 
material.
    2. Agencies should include the notifications described in 
Paragraph 1, or a link to those notifications, in at least the 
following places:
    a. Within the rulemaking documents on which agencies request 
comments, such as a

[[Page 6616]]

notice of proposed rulemaking or an advance notice of proposed 
rulemaking;
    b. On agencies' own comment submission forms, if agencies have 
them;
    c. Within any automatic emails that agencies send acknowledging 
receipt of a comment;
    d. On any part of agencies' websites that describe their 
rulemaking process or within any rules on rulemakings they may have, 
as described in Recommendation 2020-1, Rules on Rulemakings; and
    e. Within any notices of public meetings pertaining to a rule.
    3. The General Services Administration's eRulemaking Program 
Management Office should work with agencies that participate in 
Regulations.gov to include or refer to the notifications described 
in Paragraph 1 within any automated emails Regulations.gov sends 
acknowledging receipt of a comment.
    4. If a submitter notifies an agency that the submitter 
inadvertently included protected material in the submitter's 
comment, the agency should act as promptly as possible to determine 
whether such material warrants withholding from the public 
rulemaking docket and, if so, withhold it from the public rulemaking 
docket, or, if already disclosed, remove it from the public 
rulemaking docket. If an agency determines that such material does 
not qualify as protected, it should promptly notify the submitter of 
this finding with a brief statement of reasons.
    5. Agencies should allow third parties to request that protected 
material pertaining to themselves or a dependent be removed from the 
public rulemaking docket. Agencies should review such requests and, 
upon determining that the material subject to the request qualifies 
as protected material, should remove it from the public rulemaking 
docket as promptly as possible. If an agency determines that the 
material does not qualify as protected, it should promptly notify 
the requestor of this finding with a brief statement of reasons.

Recommendations for Agencies That Screen Comments for Protected 
Material Before Publication in the Public Rulemaking Docket

    6. Agencies that screen comments for protected material before 
publication in the public rulemaking docket, either as required by 
law or as a matter of discretion, should redact the protected 
material and publish the rest of the comment. Redaction should be 
thorough enough to prevent the public from discerning the redacted 
material, but not so broad as to prevent the public from viewing 
non-protected material.
    7. If redaction is not feasible within a comment, agencies 
should consider presenting the data in a summarized form.
    8. If redaction is not feasible across multiple, similar 
comments, agencies should consider presenting any related 
information in an aggregated form. Agencies should work with data 
science experts and others in relevant disciplines to ensure that 
aggregation is thorough enough to prevent someone from 
disaggregating the information.
    9. If the approaches identified in Paragraphs 6-8 would still 
permit a member of the public to identify protected material, 
agencies should withhold the comment in its entirety. When doing so, 
they should describe the withheld material for the public in as much 
detail as possible without compromising its confidentiality.
    10. When deciding whether and how to redact, aggregate, or 
withhold protected material, agencies should explore using 
artificial intelligence-based tools to aid in identifying protected 
material. Agencies should consult with private sector experts and 
technology-focused agencies, such as the General Services 
Administration's Technology Transformation Service and the Office of 
Management and Budget's United States Digital Service, to determine 
which tools are most appropriate and how they can best be deployed 
given the agencies' resources.

Recommendations for Agencies That Offer Assurances of Protection From 
Disclosure of Confidential Commercial Information

    11. Agencies that offer assurances of protection from disclosure 
of confidential commercial information should decide how they will 
offer such assurances. Agencies can choose to inform submitters, 
directly upon submission, that they will withhold confidential 
commercial information from the public rulemaking docket; post a 
general notice informing submitters that confidential commercial 
information will be withheld from the public rulemaking docket; or 
both.
    12. Such agencies should adopt policies to help them identify 
such information. Agencies should consider including the following, 
either in tandem or as alternatives, as part of their policies, 
including within any rules on rulemakings they may have, as 
described in Recommendation 2020-1, Rules on Rulemakings:
    a. Instructing submitters to identify clearly that the document 
contains confidential commercial information;
    b. Instructing submitters to flag the particular text within the 
document that constitutes confidential commercial information; and
    c. Instructing submitters to submit both redacted and unredacted 
versions of a comment that contains confidential commercial 
information.

Administrative Conference Statement #20

Agency Use of Artificial Intelligence

Adopted December 16, 2020

    Artificial intelligence (AI) techniques are changing how 
government agencies do their work.\1\ Advances in AI hold out the 
promise of lowering the cost of completing government tasks and 
improving the quality, consistency, and predictability of agencies' 
decisions. But agencies' uses of AI also raise concerns about the 
full or partial displacement of human decision making and 
discretion.
---------------------------------------------------------------------------

    \1\ There is no universally accepted definition of ``artificial 
intelligence,'' and the rapid state of evolution in the field, as 
well as the proliferation of use cases, makes coalescing around any 
such definition difficult. See, e.g., John S. McCain National 
Defense Authorization Act for Fiscal Year 2019, Public Law 115-232, 
238(g), 132 Stat. 1636, 1697-98 (2018) (using one definition of AI); 
Nat'l Inst. of Standards & Tech., U.S. Leadership in AI: A Plan for 
Federal Engagement in Developing Technical Standards and Related 
Tools 7-8 (Aug. 9, 2019) (offering a different definition of AI). 
Generally speaking, AI systems tend to have characteristics such as 
the ability to learn to solve complex problems, make predictions, or 
undertake tasks that heretofore have relied on human decision making 
or intervention. There are many illustrative examples of AI that can 
help frame the issue for the purpose of this Statement. They 
include, but are not limited to, AI assistants, computer vision 
systems, biomedical research, unmanned vehicle systems, advanced 
game-playing software, and facial recognition systems as well as 
application of AI in both information technology and operational 
technology.
---------------------------------------------------------------------------

    Consistent with its statutory mission to promote efficiency, 
participation, and fairness in administrative processes,\2\ the 
Administrative Conference offers this Statement to identify issues 
agencies should consider when adopting or modifying AI systems and 
developing practices and procedures for their use and regular 
monitoring. The Statement draws on a pair of reports commissioned by 
the Administrative Conference,\3\ as well as the input of AI experts 
from government, academia, and the private sector (some ACUS 
members) provided at meetings of the ad hoc committee of the 
Administrative Conference that proposed this Statement.
---------------------------------------------------------------------------

    \2\ See 5 U.S.C. 591.
    \3\ David Freeman Engstrom, Daniel E. Ho, Catherine M. Sharkey, 
& Mariano-Florentino Cu[eacute]llar, Government by Algorithm: 
Artificial Intelligence in Federal Administrative Agencies (Feb. 
2020) (report to the Admin. Conf. of the U.S.), https://www.acus.gov/report/government-algorithm-artificial-intelligence-federal-administrative-agencies; Cary Coglianese, A Framework for 
Governmental Use of Machine Learning (Dec. 8, 2020) (report to the 
Admin. Conf. of the U.S.), https://www.acus.gov/report/framework-governmental-use-machine-learning-final-report.
---------------------------------------------------------------------------

    The issues addressed in this Statement implicate matters 
involving law, policy, finances, human resources, and technology. To 
minimize the risk of unforeseen problems involving an AI system, 
agencies should, throughout an AI system's lifespan, solicit input 
about the system from the offices that oversee these matters. 
Agencies should also keep in mind the need for public trust in their 
practices and procedures for use and regular monitoring of AI 
technologies.

1. Transparency

    Agencies' efforts to ensure transparency in connection with 
their AI systems can serve many valuable goals. When agencies set up 
processes to ensure transparency in their AI systems, they should 
consider publicly identifying the processes' goals and the 
rationales behind them. For example, agencies might prioritize 
transparency in the service of legitimizing its AI systems, 
facilitating internal or external review of its AI-based decision 
making, or coordinating its AI-based activities. Different AI 
systems are likely to satisfy some transparency goals more than 
others. When possible, agencies should use metrics to measure the 
performance of their AI-transparency processes.
    In setting transparency goals, agencies should consider to whom 
they should be

[[Page 6617]]

transparent. For instance, depending on the nature of their 
operations, agencies might prioritize transparency to the public, 
courts, Congress, or their own officials.
    The appropriate level or nature of transparency and 
interpretability in agencies' AI systems will also depend on 
context. In some contexts, such as adjudication, reason-giving 
requirements may call for a higher degree of transparency and 
interpretability from agencies regarding how their AI systems 
function. In other contexts, such as enforcement, agencies' 
legitimate interests in preventing gaming or adversarial learning by 
regulated parties could militate against providing too much 
information (or specific types of information) to the public about 
AI systems' processes. In every context, agencies should consider 
whether particular laws or policies governing disclosure of 
information apply.
    In selecting and using AI techniques, agencies should be 
cognizant of the degree to which a particular AI system can be made 
transparent to appropriate people and entities, including the 
general public. There may be tradeoffs between explainability and 
accuracy in AI systems, so that transparency and interpretability 
might sometimes weigh in favor of choosing simpler AI models. The 
appropriate balance between explainability and accuracy will depend 
on the specific context, including agencies' circumstances and 
priorities.
    The proprietary nature of some AI systems may also affect the 
extent to which they can be made transparent. When agencies' AI 
systems rely on proprietary technologies or algorithms the agencies 
do not own, the agencies and the public may have limited access to 
the information about the AI techniques. Agencies should strive to 
anticipate such circumstances and address them appropriately, such 
as by working with outside providers to ensure they will be able to 
share sufficient information about such a system. Agencies should 
not enter into contracts to use proprietary AI systems unless they 
are confident that actors both internal and external to the agencies 
will have adequate access to information about the systems.

2. Harmful Bias

    At their best, AI systems can help agencies identify and reduce 
the impact of harmful biases.\4\ Yet they can also unintentionally 
create or exacerbate those biases by encoding and deploying them at 
scale. In deciding whether and how to deploy an AI system, agencies 
should carefully evaluate the harmful biases that might result from 
the use of the AI system as well as the biases that might result 
from alternative systems (such as an incumbent system that the AI 
system would augment or replace). Because different types of bias 
pose different types of harms, the outcome of the evaluation will 
depend on agencies' unique circumstances and priorities and the 
consequences posed by those harms in those contexts.
---------------------------------------------------------------------------

    \4\ While the term bias has a technical, statistical meaning, 
the Administrative Conference here uses the term more generally, to 
refer to common or systematic errors in decision making.
---------------------------------------------------------------------------

    AI systems can be biased because of their reliance on data 
reflecting historical human biases or because of their designs. 
Biases in AI systems can increase over time through feedback. That 
can occur, for example, if the use of a biased AI system leads to 
systematic errors in categorizations, which are then reflected in 
the data set or data environment the system uses to make future 
predictions. Agencies should be mindful of the interdependence of 
the models, metrics, and data that underpin AI systems.
    Identifying harmful biases in AI systems can pose challenges. To 
identify and mitigate biases, agencies should, to the extent 
practical, consider whether other data or methods are available. 
Agencies should periodically examine and refresh AI algorithms and 
other protocols to ensure that they remain sufficiently current and 
reflect new information and circumstances relevant to the functions 
they perform.
    Data science techniques for identifying and mitigating harmful 
biases in AI systems are developing. Agencies should stay up to date 
on developments in the field of AI, particularly on algorithmic 
fairness; establish processes to ensure that personnel that reflect 
various disciplines and relevant perspectives are able to inspect AI 
systems and their decisions for indications of harmful bias; test AI 
systems in environments resembling the ones in which they will be 
used; and make use of internal and external processes for evaluating 
the risks of harmful bias in AI systems and for identifying such 
bias.

3. Technical Capacity

    AI systems can help agencies conserve resources, but they can 
also require substantial investments of human and financial capital. 
Agencies should carefully evaluate the short- and long-term costs 
and benefits of an AI system before committing significant resources 
to it. Agencies should also ensure they have access to the technical 
expertise required to make informed decisions about the type of AI 
systems they require; how to integrate those systems into their 
operations; and how to oversee, maintain, and update those systems.
    Given the data science field's ongoing and rapid development, 
agencies should consider cultivating an AI-ready workforce, 
including through recruitment and training efforts that emphasize AI 
skills. When agency personnel lack the skills to develop, procure, 
or maintain AI systems that meet agencies' needs, agencies should 
consider other means of expanding their technical expertise, 
including by relying on tools such as the Intergovernmental 
Personnel Act,\5\ prize competitions, cooperative research and 
development agreements with private institutions or universities, 
and consultation with external technical advisors and subject-matter 
experts.
---------------------------------------------------------------------------

    \5\ 5 U.S.C. 3371-76.
---------------------------------------------------------------------------

4. Obtaining AI Systems

    Decisions about whether to obtain an AI system can involve 
important trade-offs. Obtaining AI systems from external sources 
might allow agencies to acquire more sophisticated tools than they 
could design on their own, access those tools sooner, and save some 
of the up-front costs associated with developing the technical 
capacity needed to design AI systems.\6\ Creating AI tools within 
agencies, by contrast, might yield tools that are better tailored to 
the agencies' particular tasks and policy goals. Creating AI systems 
within agencies can also facilitate development of internal 
technical capability, which can yield benefits over the lifetime of 
the AI systems and in other technological tasks the agencies may 
confront.
---------------------------------------------------------------------------

    \6\ Agencies may also obtain AI systems that are embedded in 
commercial products. The considerations applicable to such embedded 
AI systems should reflect the fact that agencies may have less 
control over their design and development.
---------------------------------------------------------------------------

    Certain government offices are available to help agencies with 
decisions and actions related to technology.\7\ Agencies should make 
appropriate use of these resources when obtaining an AI system. 
Agencies should also consider the cost and availability of the 
technical support necessary to ensure that an AI system can be 
maintained and updated in a manner consistent with its expected life 
cycle and service mission.
---------------------------------------------------------------------------

    \7\ Within the General Services Administration, for example, the 
office called 18F routinely partners with government agencies to 
help them build and buy technologies. Similarly, the United States 
Digital Service (which is within the Executive Office of the 
President) has a staff of technologists whose job is to help 
agencies build better technological tools. While the two entities 
have different approaches--18F acts more like an information 
intermediary and the Digital Service serves as an alternative source 
for information technology contracts--both could aid agencies with 
obtaining, developing, and using different AI techniques.
---------------------------------------------------------------------------

5. Data

    AI systems require data, often in vast quantities. Agencies 
should consider whether they have, or can obtain, data that 
appropriately reflect conditions similar to the ones the agencies' 
AI systems will address in practice; whether the agencies have the 
resources to render the data into a format that can be used by the 
agencies' AI systems; and how the agencies will maintain the data 
and link them to their AI systems without compromising security or 
privacy. Agencies should also review and consider statutes and 
regulations that impact their uses of AI as a potential collector 
and consumer of data.\8\
---------------------------------------------------------------------------

    \8\ See, e.g., Paperwork Reduction Act, 44 U.S.C. 3501-20.
---------------------------------------------------------------------------

6. Privacy

    Agencies have a responsibility to protect privacy with respect 
to personally identifiable information in AI systems. In a narrow 
sense, this responsibility demands that agencies comply with 
requirements related to, for instance, transparency, due process, 
accountability, and information quality and integrity established by 
the Privacy Act of 1974, Section 208 of the E-Government Act of 
2002, and other applicable laws and policies.\9\ More broadly, 
agencies should recognize and appropriately manage privacy risks 
posed by an AI system. Agencies should consider privacy risks

[[Page 6618]]

throughout the entire life cycle of an AI system from development to 
retirement and assess those risks, as well as associated controls, 
on an ongoing basis. In designing and deploying AI systems, agencies 
should consider using relevant privacy risk management frameworks 
developed through open, multi-stakeholder processes.\10\
---------------------------------------------------------------------------

    \9\ See, e.g., 5 U.S.C. 552a(e), (g), & (p); 44 U.S.C. 3501 
note.
    \10\ See Nat'l Inst. of Standards & Tech. Special Publication 
SP-800-37 revision 2, Risk Management Framework for Information 
Systems and Organizations: A System Lifecycle Approach for Security 
and Privacy (Dec. 2018); Office of Mgmt. & Budget, Exec. Off. of the 
President, Circular A-130, Managing Information as a Strategic 
Resource (July 28, 2016); see also Nat'l Inst. of Standards & Tech., 
NIST Privacy Framework: A Tool for Improving Privacy Through 
Enterprise Risk Management, Version 1.0 (Jan. 16, 2020).
---------------------------------------------------------------------------

7. Security

    Agencies should consider the possibility that AI systems might 
be hacked, manipulated, fooled, evaded, or misled, including through 
manipulation of training data and exploitation of model 
sensitivities. Agencies must ensure not only that their data are 
secure, but also that their AI systems are trained on those data in 
a secure manner, make forecasts based on those data in a secure way, 
and otherwise operate in a secure manner. Agencies should regularly 
consider and evaluate the safety and security of AI systems, 
including resilience to vulnerabilities, manipulation, and other 
malicious exploitation. In designing and deploying AI systems, 
agencies should consider using relevant government guidance or 
voluntary consensus standards and frameworks developed through open, 
multi-stakeholder processes.\11\
---------------------------------------------------------------------------

    \11\ See supra note 10; see also Office of Mgmt. & Budget, Exec. 
Off. of the President, M-21-06, Guidance for Regulation of 
Artificial Intelligence Applications (Nov. 17, 2020); Nat'l Inst. 
for Standards & Tech., Framework for Improving Critical 
Infrastructure Cybersecurity (Apr. 16, 2018).
---------------------------------------------------------------------------

8. Decisional Authority

    Agencies should be mindful that most AI systems will involve 
human beings in a range of capacities--as operators, customers, 
overseers, policymakers, or interested members of the public. Human 
factors may sometimes undercut the value of using AI systems to make 
certain determinations. There is a risk, for example, that human 
operators will devolve too much responsibility to AI systems and 
fail to detect cases in which the AI systems yield inaccurate or 
unreliable determinations. That risk may be acceptable in some 
settings--such as when the AI system has recently been shown to 
perform significantly better than alternatives--but unacceptable in 
others.
    Similarly, if agency personnel come to rely reflexively on 
algorithmic results in exercising discretionary powers, use of an AI 
system could have the practical effect of curbing the exercise of 
agencies' discretion or shifting it from the person who is supposed 
to be exercising it to the system's designer. Agencies should beware 
of such potential shifts of practical authority and take steps to 
ensure that appropriate officials have the knowledge and power to be 
accountable for decisions made or aided by AI techniques.
    Finally, there may be some circumstances in which, for reasons 
wholly apart from decisional accuracy, agencies may wish to have 
decisions be made without reliance on AI techniques, even if the law 
does not require it. In some contexts, accuracy and fairness may not 
be the only relevant values at stake. In making decisions about 
their AI systems, agencies may wish to consider whether people will 
perceive the systems as unfair, inhumane, or otherwise 
unsatisfactory.\12\
---------------------------------------------------------------------------

    \12\ Cf. Admin. Conf. of the U.S., Recommendation 2018-3, 
Electronic Case Management in Federal Administrative Adjudication, 
83 FR 30,686 (June 29, 2018) (suggesting, in the context of case 
management systems, that agencies consider implementing electronic 
systems only when they conclude that doing so would lead to benefits 
without impairing either the objective ``fairness'' of the 
proceedings or the subjective ``satisfaction'' of those 
participating in those proceedings).
---------------------------------------------------------------------------

9. Oversight

    It is essential that agencies' AI systems be subject to 
appropriate and regular oversight throughout their lifespans. There 
are two general categories of oversight: External and internal. 
Agencies' mechanisms of internal oversight will be shaped by the 
demands of external oversight. Agencies should be cognizant of both 
forms of oversight in making decisions about their AI systems.
    External oversight of agencies' uses of AI systems can come from 
a variety of government sources, including inspectors general, 
externally facing ombuds, the Government Accountability Office, and 
Congress. In addition, because agencies' uses of AI systems might 
lead to litigation in a number of circumstances, courts can also 
play an important role in external oversight. Those affected by an 
agency's use of an AI system might, for example, allege that use of 
the system violates their right to procedural due process.\13\ Or 
they might allege that the AI system's determination violated the 
Administrative Procedure Act (APA) because it was arbitrary and 
capricious.\14\ When an AI system narrows the discretion of agency 
personnel, or fixes or alters the legal rights and obligations of 
people subject to the agency's action, affected people or entities 
might also sue on the ground that the AI system is a legislative 
rule adopted in violation of the APA's requirement that legislative 
rules go through the notice-and-comment process.\15\ Agencies should 
consider these different forms of potential external oversight as 
they are making and documenting decisions and the underlying 
processes for these AI systems.
---------------------------------------------------------------------------

    \13\ Courts would analyze such challenges under the three-part 
balancing framework from Mathews v. Eldridge, 424 U.S. 319, 335 
(1976).
    \14\ See 5 U.S.C. 706(2)(A). Courts would likely review such 
challenges under the standard set forth in Motor Vehicle 
Manufacturers Ass'n v. State Farm Mutual Automobile Insurance Co., 
463 U.S. 29, 43 (1983).
    \15\ See 5 U.S.C. 553(b)-(c).
---------------------------------------------------------------------------

    Agencies should also develop their own internal evaluation and 
oversight mechanisms for their AI systems, both for initial approval 
of an AI system and for regular oversight of the system, taking into 
account their system-level risk management, authorization to 
operate, regular monitoring responsibilities, and their broader 
enterprise risk management responsibilities.\16\ Successful internal 
oversight requires advance and ongoing planning and consultation 
with the various offices in an agency that will be affected by the 
agency's use of an AI system, including its legal, policy, 
financial, human resources, internally-facing ombuds, and technology 
offices. Agencies' oversight plans should address how the agencies 
will pay for their oversight mechanisms and how they will respond to 
what they learn from their oversight.
---------------------------------------------------------------------------

    \16\ See Office of Mgmt. & Budget, Circular A-130, supra note 
10; Office of Mgmt. & Budget, Exec. Office of the President, 
Circular A-123, Management's Responsibilities for Enterprise Risk 
Management and Internal Control (July 15, 2016).
---------------------------------------------------------------------------

    Agencies should establish a protocol for regularly evaluating AI 
systems throughout the systems' lifespans. That is particularly true 
if a system or the circumstances in which it is deployed are liable 
to change over time. In these instances, review and explanation of 
the system's functioning at one stage of development or use may 
become outdated due to changes in the system's underlying models. To 
enable that type of oversight, agencies should monitor and keep 
track of the data being used by their AI systems, as well as how the 
systems use those data. Agencies may also wish to secure input from 
members of the public or private evaluators to improve the 
likelihood that they will identify defects in their AI systems.
    To make their oversight systems more effective, agencies should 
clearly define goals for their AI systems. The relevant question for 
oversight purposes will often be whether the AI system outperforms 
alternatives, which may require agencies to benchmark their systems 
against the status quo or some hypothetical state of affairs.
    Finally, AI systems can affect how agencies' staffs do their 
jobs, particularly as agency personnel grow to trust and rely on the 
systems. In addition to evaluating and overseeing their AI systems, 
agencies should pay close attention to how agency personnel interact 
with those systems.

Administrative Conference Recommendation 2020-3

Agency Appellate Systems

Adopted December 16, 2020

    In Recommendation 2016-4,\1\ the Administrative Conference 
offered best practices for evidentiary hearings in administrative 
adjudications. Paragraph 26 recommended that agencies provide for 
``higher-level review'' (or ``agency appellate review'') of the 
decisions of hearing-level adjudicators.\2\ This Recommendation 
offers

[[Page 6619]]

best practices for such review. The Administrative Conference 
intends this Recommendation to cover appellate review of decisions 
resulting from (1) hearings governed by the formal hearing 
provisions of the Administrative Procedure Act (APA) and (2) 
evidentiary hearings that are not governed by those provisions but 
are required by statute, regulation, or executive order. Agencies 
may also decide to apply this Recommendation to appellate review of 
decisions arising from other hearings, depending on their level of 
formality.
---------------------------------------------------------------------------

    \1\ Admin. Conf. of the U.S., Recommendation 2016-4, Evidentiary 
Hearings Not Required by the Administrative Procedure Act, 81 FR 
94,314 (Dec. 23, 2016).
    \2\ Recommendation 2016-4 addressed agency adjudications in 
which an evidentiary hearing, though not governed by the formal 
hearing provisions of the Administrative Procedure Act (APA) (5 
U.S.C. 554, 556-57), is required by statute, regulation, or 
executive order. Those adjudications, which are often as formal as 
APA adjudications in practice, far outnumber so-called APA 
adjudications. Although Recommendation 2016-4 addresses only non-APA 
adjudications, most of its best practices are as applicable to APA 
adjudications as non-APA adjudications. Some such practices, in 
fact, are modeled on the APA's formal hearing provisions.
---------------------------------------------------------------------------

    Appellate review of hearing-level decisions can be structured in 
numerous ways. Two structures are most common. In the first, 
litigants appeal directly to the agency head, which may be a multi-
member board or commission. In the second, litigants appeal to an 
appellate adjudicator or group of adjudicators--often styled as a 
board or council--sitting below the agency head. The appellate 
decision may be the agency's final action or may be subject to 
further appeal within the agency (usually to the agency head).
    The Administrative Conference has twice before addressed agency 
appellate review. In Recommendations 68-6 and 83-3, it provided 
guidance to agencies when establishing new, and reviewing existing, 
organizational structures of appellate review.\3\ Both 
recommendations focused on the selection of ``delegates''--
individual adjudicators, review boards composed of multiple 
adjudicators, or panels composed of members of a multi-member 
agency--to exercise appellate review authority vested in agency 
heads (including boards and commissions). Recommendation 83-3 also 
addressed when agencies should consider providing appellate review 
as a matter of right and when as a matter of discretion, and, in the 
case of the latter, under what criteria.
---------------------------------------------------------------------------

    \3\ Admin. Conf. of the U.S., Recommendation 83-3, Agency 
Structures for Review of Decisions of Presiding Officers Under the 
Administrative Procedure Act, 48 FR 57,461 (Dec. 30, 1983); Admin. 
Conf. of the U.S., Recommendation 68-6, Delegation of Final 
Decisional Authority Subject to Discretionary Review by the Agency, 
38 FR 19,783 (July 23, 1973). Both recommendations concerned only 
the review of decisions in proceedings governed by the formal 
hearing provisions of the APA. Their principles, though, are not so 
confined.
---------------------------------------------------------------------------

    With the exception of the appropriate standard for granting 
review, this Recommendation's focus lies elsewhere. It addresses, 
and offers best practices with respect to, the following subjects: 
First, an agency's identification of the purpose or objective served 
by its appellate review; second, its selection of cases for 
appellate review, when review is not required by statute; third, its 
procedures for review; fourth, its appellate decision-making 
processes; fifth, its management, administration, and bureaucratic 
oversight of its appellate system; and sixth, its public disclosure 
of information about its appellate system.\4\
---------------------------------------------------------------------------

    \4\ Christopher J. Walker & Matthew Lee Wiener, Agency Appellate 
Systems (Dec. 14, 2020) (report to the Admin. Conf. of the U.S.), 
https://www.acus.gov/report/final-report-agency-appellate-systems.
---------------------------------------------------------------------------

    Most importantly, this Recommendation begins by suggesting that 
agencies identify, and publicly disclose, the purpose(s) or 
objective(s) of their appellate systems. Appellate systems may have 
different purposes, and any given appellate system may have multiple 
purposes. Purposes or objectives can include the correction of 
errors, inter-decisional consistency of decisions, policymaking, 
political accountability, management of the hearing-level 
adjudicative system, organizational effectiveness and systemic 
awareness, and the reduction of litigation in federal courts. The 
identification of purpose is important both because it dictates (or 
should dictate) how an agency administers its appellate system--
including what cases it hears and under what standards of review it 
decides them--and provides a standard against which an agency's 
performance can be evaluated.
    This Recommendation proceeds from the recognition that agency 
appellate systems vary enormously--as to their purposes or 
objectives, governing substantive law, size, and resources--and that 
what may be a best practice for one system may not always be the 
best practice for another. In offering the best practices that 
follow, moreover, the Administrative Conference recognizes that (1) 
an agency's procedural choices may sometimes be constrained by 
statute and (2) available resources and personnel policies may 
dictate an agency's decision as to whether and how to implement the 
best practices that follow. The Administrative Conference makes this 
Recommendation subject to these important qualifications.

Recommendation

Objectives of Appellate Review

    1. Agencies should identify the objective(s) of appellate 
review; disclose those objectives in procedural regulations; and 
design rules and processes, especially for scope and standard of 
review, to serve them.

Procedures for Appellate Review

    2. Agencies should promulgate and publish procedural regulations 
governing agency appellate review in the Federal Register and codify 
them in the Code of Federal Regulations. These regulations should 
cover all significant procedural matters pertaining to agency 
appellate review, including but not limited to the following:
    a. The objectives of the agency's appellate review system;
    b. The timing and procedures for initiating review, including 
any available interlocutory review;
    c. The standards for granting review, if review is 
discretionary;
    d. The standards for permitting participation by interested 
persons and amici;
    e. The standard of review;
    f. The allowable and required submissions by litigants and their 
required form and contents;
    g. The procedures and criteria for designating decisions as 
precedential and the legal effect of such designations;
    h. The record on review and the opportunity, if any, to submit 
new evidence;
    i. The availability of oral argument or other form of oral 
presentation;
    j. The standards of and procedures for reconsideration and 
reopening, if available;
    k. Any administrative or issue exhaustion requirements that must 
be satisfied before seeking agency appellate or judicial review, 
including whether agency appellate review is a mandatory 
prerequisite to judicial review;
    l. Openness of proceedings to the public and availability of 
video or audio streaming or recording;
    m. In the case of multi-member appellate boards, councils, and 
similar entities, the authority to assign decision-making authority 
to fewer than all members (e.g., panels); and
    n. Whether seeking agency appellate review automatically stays 
the effectiveness of the appealed agency action until the appeal is 
resolved (which may be necessary for appellate review to be 
mandatory, see 5 U.S.C. 704), and, if not, how a party seeking 
agency appellate review may request such a stay and the standards 
for deciding whether to grant it.
    3. Agencies should include in the procedural regulations 
governing their appellate programs: (a) A brief statement or 
explanation of each program's review authority, structure, and 
decision-making components; and (b) for each provision based on a 
statutory source, an accompanying citation to that source.
    4. When revising existing or adopting new appellate rules, 
agencies should consider the appellate rules (Rules 400-450) in the 
Administrative Conference's Model Rules of Agency Adjudication (rev. 
2018).
    5. When materially revising existing or adopting new appellate 
rules, agencies should use notice-and-comment procedures or other 
mechanisms for soliciting public input, notwithstanding the 
procedural rules exemption of 5 U.S.C. 553(b)(A), unless the costs 
clearly outweigh the benefits of doing so.

Case Selection for Appellate Review

    6. Based on the agency-specific objectives of appellate review, 
agencies should decide whether the granting of review should be 
mandatory or discretionary (assuming they have statutory authority 
to decide); if discretionary, the criteria for granting review 
should track the objectives of the appellate system, and they should 
be published in the procedural regulations.
    7. Agencies should consider implementing procedures for sua 
sponte appellate review of non-appealed hearing-level decisions, as 
well as for the referral of cases or issues by hearing-level 
adjudicators to the appellate entity for interlocutory review.

Appellate Decision-Making Processes and Decisions

    8. Whenever possible, agencies should consider maintaining 
electronic case

[[Page 6620]]

management systems that ensure that hearing records are easily 
accessible to appellate adjudicators. Such systems may include the 
capability for electronic filing.
    9. Although the randomized assignment of cases to appellate 
adjudicators is typically an appropriate docketing method for an 
agency appellate system, agencies should consider the potential 
benefits of sorting and grouping appeals on the appellate docket, 
such as reduced case processing times and more efficient use of 
adjudicators', staff attorneys', and law clerks' skills and time. 
Criteria for sorting and grouping cases may include the size of a 
case's record, complexity of a case's issues, subject matter of a 
case, and similarity of a case's legal issues to those of other 
pending cases.
    10. Consistent with the objectives of the agency's appellate 
system and in light of the costs of time and resources, agencies 
should consider adopting an appellate model of judicial review in 
which the standard of review is not de novo with respect to findings 
of fact and application of law to facts. For similar reasons, many 
agencies should consider limiting the introduction of new evidence 
on appeal that is not already in the administrative record from the 
hearing-level adjudication.
    11. Taking agency resources into account, agencies should 
emphasize concision, readability, and plain language in their 
appellate decisions and explore the use of decision templates, 
summary dispositions, and other quality-improving measures.
    12. Agencies should establish clear criteria and processes for 
identifying and selecting appellate decisions as precedential, 
especially for appellate systems with objectives of policymaking or 
inter-decisional consistency.
    13. Agencies should assess the value of oral argument and amicus 
participation in their appellate system based on the agencies' 
identified objectives for appellate review and should establish 
rules governing both. Criteria that may favor oral argument and 
amicus participation include issues of high public interest; issues 
of concern beyond the parties to the case; specialized or technical 
matters; and a novel or substantial question of law, policy, or 
discretion.

Administration, Management, and Bureaucratic Oversight

    14. Agency appellate systems should promptly transmit their 
precedential decisions to all appellate program adjudicators and, 
directly or through hearing-level programs, to hearing-level 
adjudicators (as appropriate). Appellate programs should include in 
their transmittals, when feasible, brief summaries of the decision.
    15. Agencies should notify their adjudicators of significant 
federal court decisions reviewing the agencies' decisions and, when 
providing notice, explain the significance of those decisions to the 
program. As appropriate, agencies should notify adjudicators if the 
agency will not acquiesce in a particular decision of the federal 
courts of appeals.
    16. Agencies in which decision making relies extensively on 
their own precedential decisions should consider preparing or having 
prepared indexes and digests--with annotations and comments, as 
appropriate--to identify those decisions and their significance.
    17. As appropriate, agency appellate systems should communicate 
with agency rule-writers and other agency policymakers--and 
institutionalize communication mechanisms--to address whether 
recurring issues in their decisions should be addressed by rule 
rather than precedential case-by-case adjudication.
    18. The Office of the Chairman of the Administrative Conference 
should provide for, as authorized by 5 U.S.C. 594(2), the 
``interchange among administrative agencies of information 
potentially useful in improving'' agency appellate systems. The 
subjects of interchange might include electronic case management 
systems, procedural innovations, quality-assurance reviews, and 
common management problems.

Public Disclosure and Transparency

    19. Agencies should disclose on their websites any rules 
(sometimes styled as ``orders''), and statutes authorizing such 
rules, by which an agency head has delegated review authority to 
appellate adjudicators.
    20. Regardless of whether the Government in the Sunshine Act (5 
U.S.C. 552b) governs their appellate review system, agencies should 
consider announcing, livestreaming, and maintaining video recordings 
on their websites of appellate proceedings (including oral argument) 
that present significant legal and policy issues likely to be of 
interest to regulated parties and other members of the public. Brief 
explanations of the issues to be addressed by oral argument may 
usefully be included in website notices of oral argument.
    21. Agencies should include on their websites brief and 
accessibly written explanations as to how their internal decision-
making processes work and, as appropriate, include links to 
explanatory documents appropriate for public disclosure. Specific 
subjects that agencies should consider addressing include: The 
process of assigning cases to adjudicators (when fewer than all of 
the programs' adjudicators participate in a case), the role of 
staff, and the order in which cases are decided.
    22. When posting decisions on their websites, agencies should 
distinguish between precedential and non-precedential decisions. 
Agencies should also include a brief explanation of the difference.
    23. When posting decisions on their websites, agencies should 
consider including, as much as practicable, brief summaries of 
precedential decisions and, for precedential decisions at least, 
citations to court decisions reviewing them.
    24. Agencies should include on their websites any digests and 
indexes of decisions they maintain. It may be appropriate to remove 
material exempt from disclosure under the Freedom of Information Act 
or other laws.
    25. Agencies should affirmatively solicit feedback concerning 
the functioning of their appellate systems and provide a means for 
doing so on their websites.

Administrative Conference Recommendation 2020-4

Government Contract Bid Protests Before Agencies

Adopted December 17, 2020

    Federal law establishes policies and procedures governing how 
federal executive agencies procure supplies and services.\1\ The 
primary source of these policies and procedures is the Federal 
Acquisition Regulation (FAR),\2\ which applies to all executive-
agency acquisitions of supplies and services with appropriated funds 
by and for the use of the federal government, unless expressly 
excluded. Other relevant policies and procedures are found in 
federal statutes and agencies' own procurement rules.
---------------------------------------------------------------------------

    \1\ See Federal Acquisition Regulation, 48 CFR ch. 1; see also 
Competition in Contracting Act of 1984, Public Law 98-369, div. B, 
tit. VII, 98 Stat. 494, 942-85 (codified, as amended, in various 
parts of the U.S. Code); Federal Acquisition Streamlining Act of 
1994, Public Law 103-355, 108 Stat. 3243; Federal Acquisition Reform 
Act of 1996, Public Law 104-106, 110 Stat. 186 (later renamed the 
Clinger-Cohen Act of 1996); Exec. Order No. 12,979, Agency 
Procurement Protests, 60 FR 55,171 (Oct. 25, 1995).
    \2\ See 48 CFR ch. 1.
---------------------------------------------------------------------------

    If a vendor believes a federal executive agency has not complied 
with the law or the terms of a solicitation, it may file what is 
called a bid protest--that is, a written objection to a government 
agency's conduct in acquiring supplies and services for its direct 
use or benefit.\3\ Responding to bid protests can require agencies 
to reevaluate their procurement processes and, sometimes, make 
improvements. That, in turn, results in more competitive, fairer, 
and more transparent procurement processes, benefiting vendors, 
agencies, and ultimately the public.
---------------------------------------------------------------------------

    \3\ See Admin. Conf. of the U.S., Recommendation 95-5, 
Government Contract Bid Protests, 60 FR 43,108, 43,113 (Aug. 18, 
1995).
---------------------------------------------------------------------------

    To file a bid protest, an actual or prospective vendor must show 
that it is an ``interested party''--meaning that its direct economic 
interest would be adversely affected by the award of, or failure to 
award, the contract in question \4\--and that it suffered prejudice 
because of an error in the procurement process. Ordinarily, vendors 
who meet those requirements may file bid protests in any of three 
forums: (1) The procuring agency,\5\ (2) the Government 
Accountability Office (GAO),\6\ or (3) the

[[Page 6621]]

United States Court of Federal Claims (COFC),\7\ and depending on 
where the protest is initiated, may be able to file protests in 
series. For example, a protest may be filed first at the agency, 
then (if unsuccessful at the agency) at GAO, and then (if again 
unsuccessful) at COFC.\8\ The procedural tools available in a given 
forum, along with other strategic and cost considerations, typically 
drive vendors' decisions about where to file their bid protests.
---------------------------------------------------------------------------

    \4\ See 4 CFR 21.0(a)(1) (defining ``interested party'' for 
purposes of bid protest proceedings before the Government 
Accountability Office); 48 CFR 33.101 (defining ``interested party'' 
for purposes of bid protest proceedings before procuring agencies); 
CliniComp Int'l, Inc. v. United States, 904 F.3d 1353, 1358 (Fed. 
Cir. 2018) (defining ``interested party'' for purposes of 28 U.S.C. 
1491(b), which covers actions in the Court of Federal Claims). There 
are some instances in which Congress has restricted the ability to 
file a protest, regardless of whether a vendor is an ``interested 
party.'' See, e.g., 41 U.S.C. 4106(f) (limiting the ability to 
protest the issuance or proposed issuance of a task or delivery 
order); 48 CFR 16.505(a)(10) (same).
    \5\ See 48 CFR 33.103.
    \6\ See 31 U.S.C. 3552(a), 3553(a). For civilian agencies, GAO 
has exclusive jurisdiction over protests of task and delivery orders 
in excess of $10 million, unless the protest is on the grounds that 
the order increases the scope, period, or maximum value of the 
contract. See 41 U.S.C. 4106(f); 48 CFR 16.505(a)(10).
    \7\ See 28 U.S.C. 1491(b).
    \8\ See Admin. Conf. of the U.S., Info. Interchange Bull. No. 
007, Agency Bid Protests (June 2020), https://www.acus.gov/fact-sheet/iib-007-agency-bid-protests.
---------------------------------------------------------------------------

    Bid protests filed with procuring agencies are commonly referred 
to as agency-level protests. Agency-level protests have important 
benefits for the public, contractors, procuring agencies, and COFC 
and GAO. By ``provid[ing] for inexpensive, informal, procedurally 
simple, and expeditious resolution of protests,'' \9\ agency-level 
protest mechanisms allow small businesses (among other vendors) to 
affordably contest agencies' procurement decisions. They also give 
procuring agencies the chance to review and improve their own 
procurement practices. And they funnel some protests away from COFC 
and GAO, reducing the likelihood that the number of protests will 
overwhelm those institutions.
---------------------------------------------------------------------------

    \9\ See Exec. Order. No. 12,979, Agency Procurement Protests, 60 
FR 55,171, 55,171 (Oct. 25, 1995).
---------------------------------------------------------------------------

    Vendors, however, seldom file agency-level protests. Although 
there is little data on the number of agency-level protests filed 
each year, available evidence suggests that substantially more 
protests are filed with COFC and GAO each year than with procuring 
agencies.\10\ There are several reasons why vendors may forego 
agency-level protests. Those reasons implicate the themes of 
transparency, predictability, and accountability.
---------------------------------------------------------------------------

    \10\ See Christopher Yukins, Stepping Stones to Reform: Making 
Agency-Level Bid Protests Effective for Agencies and Bidders by 
Building on Best Practices from Across the Federal Government 12-13 
(May 1, 2020) (report to the Admin. Conf. of the U.S.), https://www.acus.gov/report/agency-level-bid-protests-final-report.
---------------------------------------------------------------------------

    First, some vendors report shying away from agency-level 
protests because they perceive the agency as unlikely to change its 
decision.\11\ Sometimes, for instance, the official responsible for 
soliciting or awarding a procurement contract is also responsible 
for handling any agency-level protests that are filed regarding the 
procurement. This perception of a pre-judgment by the agency may 
cause some vendors to file their protests at GAO or COFC, rather 
than at the agency level.
---------------------------------------------------------------------------

    \11\ Id. at 23.
---------------------------------------------------------------------------

    Second, some vendors report that they view agency-level protest 
processes as opaque.\12\ Agencies do not publish or provide 
comprehensive data on their bid protest decisions. And the FAR and 
agency-specific bid protest rules establish few hard-and-fast 
requirements for the process. For example, although the FAR states 
that ``[a]gencies shall make their best efforts to resolve agency 
protests within 35 days after [an agency-level protest] is filed,'' 
\13\ that language is hortatory and does not establish any binding 
deadlines for agency decisions. Nothing in the FAR does. The failure 
to provide for any binding deadlines distinguishes the FAR from 
other federal procurement statutes, such as the Contract Disputes 
Act,\14\ which sets or requires contracting officers to set firm 
deadlines for deciding most claims \15\ and provides that the 
passage of the deadline for a claim means the claim is deemed 
denied.\16\
---------------------------------------------------------------------------

    \12\ Id. at 13.
    \13\ 48 CFR 33.103(g).
    \14\ 41 U.S.C. 7101-09.
    \15\ See id. Sec.  7103(f)(1)-(2).
    \16\ See id. Sec.  605(c)(5).
---------------------------------------------------------------------------

    Third, some vendors report being dissuaded by their inability to 
compel production of the procurement record as part of an agency-
level protest.\17\ The FAR gives disappointed offerors the right to 
an agency debriefing--a procedure whereby contracting personnel 
provide offerors with an explanation of the agency's evaluation 
process and an assessment of the offerors' proposals. But nothing in 
the FAR guarantees vendors the right to view the procurement record 
itself. The FAR provides only that agencies ``may exchange relevant 
information'' with agency-level protesters.\18\ By contrast, vendors 
who file bid protests at GAO may demand to see the entire record of 
the procurement, and procuring agencies must respond to such 
requests within twenty-five days and produce the responsive 
documents within thirty days (unless they are withheld for a valid 
reason).\19\
---------------------------------------------------------------------------

    \17\ Yukins, supra note 10, at 39.
    \18\ 48 CFR 33.103(g) (italics added).
    \19\ 4 CFR 21.3(c)-(d); 48 CFR 33.104(a).
---------------------------------------------------------------------------

    Finally, some vendors deem agency-level protests to be too 
risky.\20\ In many cases, vendors who do not obtain relief through 
an agency-level protest will seek relief from GAO by pursuing their 
protest in that forum. But GAO's deadline for filing such ``follow-
on protests'' often begins to run as soon as the vendor has actual 
or constructive notice of some ``adverse agency action,'' which can 
occur before a protester receives the decision in its agency-level 
protest.\21\ In this way, delayed notification about an agency's 
decision in a bid protest can seriously prejudice protesters' rights 
at GAO.\22\ This causes some vendors to forego agency-level protests 
altogether.\23\
---------------------------------------------------------------------------

    \20\ Yukins, supra note 10, at 31.
    \21\ See 4 CFR 21.0(e), 21.2.
    \22\ See Yukins, supra note 10, at 13-14, 18-19.
    \23\ See id. at 23.
---------------------------------------------------------------------------

    The perception that agency-level protests lack transparency, 
predictability, and accountability also makes it more likely that 
protesters who do file at the agency level and whose protests are 
denied will file follow-on protests with GAO or COFC. Such follow-on 
protests not only tax the limited resources of GAO and COFC, but 
also can disrupt activities at procuring agencies. For instance, 
just as the filing of an agency-level protest automatically 
prohibits the contract from being awarded or performed until the 
agency denies or dismisses the protest and takes some adverse 
action,\24\ a follow-on protest at GAO may automatically prevent the 
contract from being awarded or performed (if the requisite filing 
deadlines are met) until GAO denies or dismisses the protest.\25\ 
Thus, when an agency-level protest is followed by another protest at 
GAO, delays in procurements can be substantial.
---------------------------------------------------------------------------

    \24\ 48 CFR 33.103(f). Under certain circumstances, the agency 
can override the regulatory stay for agency-level protests. See 48 
CFR 33.103(f)(1), (f)(3).
    \25\ 31 U.S.C. 3553(c)(1), (d)(3). Under certain circumstances, 
the agency can override the statutory stay for protests to GAO. See 
31 U.S.C. 3553(c)-(d); 48 CFR 33.104(b)-(c).
---------------------------------------------------------------------------

    Protesters, agencies, and the public would all benefit from an 
improved agency-level protest system. Protesters would benefit 
because agency-level protests are typically the least formal and 
least costly types of bid protest procedures. Agencies would benefit 
from an improved agency-level protest system because greater use of 
agency-level protests means more agency control over the timing and 
conduct of protests and more opportunities for agencies to 
superintend their own procurement processes. And the public would 
benefit from more competitive, fairer, and more transparent agency 
procurements.
    Because an improved agency-level protest system is of 
significant value to contractors, agencies, and the public, this 
Recommendation identifies changes to make it more likely vendors 
will avail themselves of agency-level protest procedures. The 
recommended changes reflect three overarching principles--
transparency, simplicity, and predictability--meant to address 
contractors' principal concerns about agency-level protest systems.

Recommendation

Identification of Decisions Subject to Agency-Level Protests

    1. Agencies should clearly identify which categories of 
procurement decisions may or may not be made the subjects of agency-
level protests.

Transparency for the Process and Personnel for Agency-Level Protests

    2. Agencies should formalize and compile in a document that is 
publicly available online the procedures they apply in adjudicating 
agency-level protests. In so doing, they should be guided by the 
principles set out in Recommendation 2018-5, Public Availability of 
Adjudication Rules.
    3. Agencies should clearly identify who within the agency will 
adjudicate an agency-level protest. They should consider designating 
at least one Agency Protest Official (APO)--a person who specializes 
in handling agency-level protests--to oversee and coordinate agency-
level protests and hear protests brought to a level above the 
contracting officer. Agencies lacking the resources to designate 
their own APO might consider sharing an APO with other agencies.

Notice of the Timeline for Agency-Level Protests

    4. Agencies should consider adopting presumptive timelines for 
agency-level

[[Page 6622]]

protests, similar to the ones under the Contract Disputes Act. 
Agencies should also make best efforts to notify protesters of the 
timelines applicable to their agency-level protests.
    5. Agencies should clearly and immediately provide written 
notice to protesters of any adverse agency action affecting the 
rights of the protester under the challenged procurement. Agency 
rules should provide that protests are deemed denied after a 
specified number of days without a decision and that agencies may 
grant case-specific extensions based on identified criteria.

Compiling the Record and Making It Available

    6. Agencies should make available to protesters as much of the 
procurement record as is feasible. To address confidential 
information in the record, agencies should consider using tools such 
as enhanced debriefings.
    7. Agencies should consider adopting a thirty-day deadline, 
running from the date a protest is filed, for providing protesters 
with as much of the procurement record as is feasible.

Protecting Against Adverse Consequences

    8. Although the Federal Acquisition Regulation (FAR) prohibits 
the award of a contract or continued performance under an awarded 
contract during an agency-level protest, agencies should provide for 
a short extension of the stay after a final decision in an agency-
level bid protest as permitted by the FAR. The short extension 
should be of sufficient duration (e.g., five days) to give the 
protester time to bring a follow-on protest at the Government 
Accountability Office (GAO) or the United States Court of Federal 
Claims after the agency's decision.
    9. Congress should provide that, if a protester promptly files a 
GAO protest after an adverse decision in an agency-level protest, 
the agency shall not award the contract or commence performance 
under the contract during the pendency of the GAO protest, subject 
to potential override in urgent and compelling circumstances.
    10. GAO should amend its bid protest procedures to ensure that 
follow-on protests at GAO are handled on an expedited basis, to the 
extent feasible.

Publishing Data on Agency-Level Protests

    11. Agencies should collect and annually publish data about the 
bid protests they adjudicate. To the extent feasible, the data 
should at least include what the GAO currently provides in its 
annual reports about the bid protests it adjudicates (e.g., the 
number of bid protests filed with the agency; the effectiveness rate 
of agency-level bid protests (the ratio of protests sustained or in 
which corrective action is afforded versus total agency-level 
protests filed); the number of merits decisions by the agency; the 
number of decisions sustaining the protest; the number of decisions 
denying the protest; and the time required for bid protests to be 
resolved).

Administrative Conference Recommendation 2020-5

Publication of Policies Governing Agency Adjudicators

Adopted December 17, 2020

    [Note: Appendix B referenced in this Recommendation has been 
omitted from this notice because of the inaccessible images it 
contains. The full appendix may be found online at https://www.acus.gov/recommendation/publication-policies-governing-agency-adjudicators.]
    Federal agency officials throughout the country preside over 
hundreds of thousands of adjudications each year.\1\ As the 
Administrative Conference has previously observed, litigants, their 
lawyers, and other members of the public benefit from having ready 
online access to procedural rules, decisions, and other key 
materials associated with adjudications.\2\ They also benefit from 
having ready online access to the policies and practices by which 
agencies appoint and oversee administrative law judges and other 
adjudicators. The availability of these policies and practices helps 
inform the public about, among other things, any actions agencies 
have taken to ensure the impartiality of administrative adjudicators 
\3\ and promotes an understanding of adjudicators' constitutional 
status under the Appointments Clause and other constitutional 
provisions. The Administrative Conference acknowledges ongoing 
litigation regarding the constitutional status of many agency 
adjudicators and the continuing validity of the means and 
circumstances of their appointment and removal.\4\
---------------------------------------------------------------------------

    \1\ See Admin. Conf. of the U.S., Recommendation 2016-2, 
Aggregate Agency Adjudication, 81 FR 40,260, 40,260 (June 21, 2016).
    \2\ Admin. Conf. of the U.S., Recommendation 2018-5, Public 
Availability of Adjudication Rules, 84 FR 2142 (Feb. 6, 2019); 
Admin. Conf. of the U.S., Recommendation 2017-1, Adjudication 
Materials on Agency Websites, 82 FR 31,039 (July 5, 2017).
    \3\ Cf. Admin. Conf. of the U.S., Recommendation 2018-4, Recusal 
Rules for Administrative Adjudicators, 84 FR 2139 (Feb. 6, 2019).
    \4\ See, e.g., Lucia v. SEC, 138 S. Ct. 2044 (2018); Arthrex v. 
Smith & Nephew, 941 F.3d 1320 (Fed. Cir. 2019), cert. granted, __S. 
Ct. __(Oct. 13, 2020) (No. 19-1434).
---------------------------------------------------------------------------

    Agencies may benefit from disclosures about agency adjudicators 
because it allows them to compare their own policies with those made 
publicly available by other agencies. Agencies' proactive 
disclosures, which may sometimes already be required under the 
Freedom of Information Act and the E-Government Act, may also be 
more cost-effective than agencies' responding to individual requests 
for information.\5\
---------------------------------------------------------------------------

    \5\ FOIA Improvement Act of 2016, Public Law 114-185, 2, 130 
Stat. 538, 538 (amending 5 U.S.C. 552(a)(2)); E-Government Act of 
2002, Public Law 140-347, 206, 116 Stat. 2899, 2916 (amending 44 
U.S.C. 3501).
---------------------------------------------------------------------------

    Like other recent recommendations regarding adjudicators,\6\ 
this Recommendation pertains to officials who preside over (1) 
hearings governed by the formal hearing provisions of the 
Administrative Procedure Act (APA) \7\ and (2) hearings that are not 
governed by those provisions but are required by statute, 
regulation, or executive order. It also covers officials (agency 
heads excluded) who review hearing-level adjudicators' decisions on 
appeal. For ease of reference, this Recommendation refers to the 
covered adjudicators as either ``administrative law judges'' (ALJs) 
or ``administrative judges'' (AJs).\8\ Agencies may decide to 
include on their websites the disclosures identified in this 
Recommendation for other adjudicators, depending on the level of 
formality of the proceedings over which they preside and whether 
they serve as full-time adjudicators. Agencies may also decide to 
make similar disclosures with respect to agency heads if their 
websites do not already provide sufficient information.
---------------------------------------------------------------------------

    \6\ See, e.g., Admin. Conf. of the U.S., Recommendation 2018-4, 
Recusal Rules for Administrative Adjudicators, 84 FR 2139 (Feb. 6, 
2019).
    \7\ See 5 U.S.C. 554, 556-57.
    \8\ The vast majority of ALJs work at the Social Security 
Administration. AJs work at many different agencies under a variety 
of titles, including not only ``Administrative Judge'' but also, by 
way of example, ``Hearing Officer,'' ``Immigration Judge,'' 
``Veterans Law Judge,'' ``Administrative Patent Judge,'' and 
``Administrative Appeals Judge.''
---------------------------------------------------------------------------

    This Recommendation focuses on policies and practices relating 
to adjudicators that agencies should disclose, including those 
addressing appointment and qualifications; compensation (including 
salaries, bonuses, and performance incentives); duties and 
responsibilities; supervision and assignment of work; position 
within agencies' organizational hierarchies; methods of evaluating 
performance; limitations on ex parte communications and other 
policies ensuring separation between adjudicative and enforcement 
functions; recusal and disqualification; the process for review of 
adjudications; and discipline and removal.
    Many of the policies and practices applicable to ALJs governing 
these matters are already publicly available because they are in the 
APA, Office of Personnel Management rules, or other legal 
authorities.\9\ Nevertheless, agencies that employ ALJs can take 
steps to improve the public's access to this information.
---------------------------------------------------------------------------

    \9\ 5 U.S.C. 554, 557, 3105, 4301, 5372, 7521; 5 CFR pt. 930, 
subpt. B; Exec. Order No. 13,843, Executive Order Excepting 
Administrative Law Judges from the Competitive Service, 83 FR 32,755 
(July 13, 2018) (issued July 10, 2018).
---------------------------------------------------------------------------

    ALJs, in any case, make up a small portion of federal 
adjudicators. There are many more AJs than ALJs.\10\ AJs are 
regulated by a complex mix of statutory provisions, including civil 
service laws, agency rules codified in the Code of Federal 
Regulations, and agency-specific policies that take a variety of 
forms. Many types of information about AJs reside in these sources, 
but they may be difficult to find.\11\ Some relevant

[[Page 6623]]

sources may not be publicly available, including internal 
administrative and personnel manuals, position descriptions, and 
labor agreements. This is particularly true with respect to certain 
kinds of policies, such as those relating to compensation and 
performance incentives.\12\ Of course, the Administrative Conference 
recognizes that some of these agency policies and practices may 
qualify for an exemption under the Freedom of Information Act,\13\ 
Privacy Act,\14\ or other laws and executive-branch policies.
---------------------------------------------------------------------------

    \10\ Kent Barnett et al., Non-ALJ Adjudicators in Federal 
Agencies: Status, Selection, Oversight, and Removal 1 (Sept. 24, 
2018) (report to the Admin. Conf. of the U.S.), https://www.acus.gov/report/non-alj-adjudicators-federal-agencies-status-selection-oversight-and-removal-1.
    \11\ Leigh Anne Schriever, Public Availability of Information 
About Adjudicators 10 (Nov. 23, 2020) (report to the Admin. Conf. of 
the U.S.), https://www.acus.gov/report/final-report-public-availability-information-about-agency-adjudicators.
    \12\ Id. at 7.
    \13\ 5 U.S.C. 552.
    \14\ Id. Sec.  552a.
---------------------------------------------------------------------------

    Agency websites are the most helpful location for agencies to 
make relevant policies and practices publicly available. Individuals 
most naturally seek information about administrative policies and 
practices on agencies' websites. Agencies can situate information 
about their adjudicators in a logical and easily identifiable place 
on their websites and structure their websites to synthesize 
policies in plain language and link to information from many 
different sources.\15\
---------------------------------------------------------------------------

    \15\ Cf. Admin. Conf. of the U.S., Recommendation 2017-3, Plain 
Language in Regulatory Drafting, 82 FR 61,728 (Dec. 29, 2017).
---------------------------------------------------------------------------

    This Recommendation encourages agencies to post on their 
websites clear and readily accessible descriptions of the policies 
governing the appointment and oversight of ALJs and AJs, and to 
include links to relevant legal documents. How, exactly, they should 
do so will of course depend on the specific features of their 
adjudicative programs and their institutional needs.

Recommendation

    1. Each adjudicative agency should prominently display on its 
website a short, straightforward description of all generally 
applicable policies and practices, along with the legal authority, 
governing the appointment and oversight of Administrative Law Judges 
(ALJs) and Administrative Judges (AJs), including, as applicable, 
those that address:
    a. Procedures for assessing, selecting, and appointing 
candidates for adjudicator positions and the legal authority under 
which such appointments are made;
    b. Placement of adjudicators within agencies' organizational 
hierarchies;
    c. Compensation structure and performance incentives, such as 
bonuses, nonmonetary awards, and promotions;
    d. Procedures for assigning cases;
    e. Assignment, if any, of nonadjudicative duties to 
adjudicators;
    f. Limitations on ex parte communications, including between 
adjudicators and other agency officials, related to the disposition 
of individual cases, as well as other policies ensuring a separation 
of adjudication and enforcement functions;
    g. Standards for recusal by and disqualification of 
adjudicators;
    h. Administrative review of adjudicators' decisions;
    i. Supervision of adjudicators by higher-level officials;
    j. Evaluation of adjudicators, including quantitative and 
qualitative methods for appraising adjudicators' performances, such 
as case-processing goals, if any; and
    k. Discipline and removal of adjudicators.
    Agencies may choose not to provide access to policies covered by 
a Freedom of Information Act exemption.
    2. On the same web page as the information described in 
Paragraph 1 appears, each adjudicative agency should post links to 
key legal documents or, when links are not available, citations to 
such documents. These documents may include (a) federal statutes, 
including relevant provisions of the Administrative Procedure Act 
(APA) and other laws applicable to ALJs and AJs; (b) agency-
promulgated rules regarding adjudicators, including Office of 
Personnel Management rules applicable to ALJs; (c) publicly 
available agency-promulgated guidance documents relating to 
adjudicators, including manuals, bench books, and other explanatory 
materials; (d) delegations of authority; and (e) position 
descriptions. To the extent that some policies concerning 
adjudicators may be a matter of custom, such as assignment of 
nonadjudicative duties, each adjudicative agency should consider 
documenting those policies to make them publicly accessible to the 
extent practicable.
    3. The web page containing the information described in 
Paragraphs 1 and 2 should present the materials in a clear, logical, 
and comprehensive fashion. One possible method of presenting this 
information appears in Appendix A. The appendix gives one example 
for ALJs and another for AJs.
    4. If an agency's mission consists exclusively or almost 
exclusively of conducting adjudications, the agency should provide a 
link to the web page containing the information described in 
Paragraphs 1 and 2 on the agency's homepage. If conducting 
adjudications is one of an agency's many functions, the agency 
should provide a link to these materials from a location on the 
website that is both dedicated to adjudicative materials and logical 
in terms of a user's likelihood of finding the documents in the 
selected location. One example would be an enforcement or 
adjudication page or the homepage for the component in which a 
particular category of adjudicators works. Citations to agency web 
pages that currently provide this information in a way that makes it 
easy for the public to locate, as well as descriptions of how to 
find those pages on agency websites, appear in Appendix B.

Appendix A

Sample Website Text for Administrative Law Judges

About Our Administrative Law Judges

    Administrative Law Judges (ALJs) at [agency] conduct hearings 
and decide cases under [insert name of authorizing act]. They are 
part of the [agency component in which ALJs are located], which is 
directed by [title of office head] and has offices in [cities]. 
Visit [link to agency organization chart] to see how [office] 
relates to other offices at [agency].
    [Agency] is committed to ensuring that all hearings and appeals 
are conducted in a fair and equitable manner. Parties are entitled 
to a due process hearing presided over by an impartial, qualified 
ALJ. ALJs resolve cases involving [kinds of cases ALJs hear] in a 
fair, transparent, and accessible manner. Our ALJs are appointed by 
[agency official], and are [describe qualifications]. ALJs are paid 
according to the [pay scale for ALJs with link to the scale] scale 
set by statute under 5 U.S.C. 5372, subject to annual pay 
adjustments.
    Cases are assigned to ALJs [in each geographic office] in 
rotation so far as practicable. The ALJ assigned to your case is 
responsible for [job duties, like taking evidence, hearing 
objections, issuing decisions]. ALJs are required by statute to 
perform their functions impartially. 5 U.S.C. 556(b). To ensure 
impartiality, they do not take part in investigative or enforcement 
activities, nor do they report to officials in the [agency]'s 
investigative or enforcement components. 5 U.S.C. 554(d), 3105. The 
ALJ assigned to your case may not communicate privately about the 
facts of your case with other agency officials. [More details on 
[agency]'s rules about communicating with ALJs are available 
[location of agency-specific ex parte prohibitions]].
    By law, [agency] does not reward or discipline ALJs for their 
decisions. A federal statute provides that [agency] may remove, or 
take certain other disciplinary actions, against an ALJ it employs 
only for good cause established and determined by the Merit Systems 
Protection Board on the record after opportunity for hearing before 
the Board. 5 U.S.C. 7521.
    The agency has adopted rules of recusal [link] that allow a 
participant to request that the ALJ in charge of his or her case be 
disqualified if the participant believes the ALJ cannot fairly and 
impartially decide the case.
    If you are dissatisfied with an ALJ's decision, you can request 
reconsideration from the ALJ or appeal that decision to [agency 
office/official]. Visit [link] for information on appealing an ALJ 
decision. [Agency office/official] may also review your case on 
[its/his or her] own initiative if there is an issue with the ALJ's 
decision.
    For Further Information:

 Hiring process: [link]
 Pay rates: [link]
 How cases are assigned to ALJs: [link]
 Communicating with ALJs (ex parte communications): [link]
 Process for addressing allegations that an ALJ has a 
conflict of interest (recusal and disqualification procedures): 
[link]
 How to appeal an ALJ decision: [link]
 Case-processing goals: [link]
 Process for addressing allegations of ALJ misconduct: 
[link]

    See also:

 Statutory provisions governing ALJs: 5 U.S.C. 554, 557, 
3105, 4301, 5372, 7521
 OPM's regulations governing ALJs: 5 CFR 930.205-930.207, 
930.211
 MSPB's regulations governing ALJs: 5 CFR 1201.127-1201.142
 [Additional legal provisions governing ALJs]

[[Page 6624]]

 Executive Orders pertaining to ALJs: E.O. 13,843 (giving 
agencies control over the hiring process of ALJs) [add other 
pertinent EOs]

Sample Website Text for Administrative Judges

    If agencies have different kinds of adjudicators, they should 
consider providing a separate web page for each.

About Our [Insert Adjudicator Title]

    [Adjudicator title] at [agency] [conduct hearings and decide 
cases/review appeals] under [name of authorizing act(s)]. They are 
part of the [agency component in which adjudicators are located], 
which is directed by [title of office head] and has offices in 
[cities]. Visit [link to agency organization chart] to see how 
[office] relates to other offices at [agency].
    [Agency] is committed to ensuring that all hearings and appeals 
are conducted in a fair and equitable manner. Parties are entitled 
to a due process hearing presided over by an impartial, qualified 
[adjudicator title]. [Adjudicator title] resolve cases involving 
[kinds of cases] in a fair, transparent, and accessible manner. Our 
[adjudicator title] are appointed pursuant to [authorizing statute] 
by [agency official] [for terms of [number of years] years], and are 
[describe qualifications]. [Adjudicator title] are paid according to 
[[the pay scale for the adjudicator with link to the scale] or [the 
discretion of the agency head]].
    Cases are [describe how cases are assigned]. The [adjudicator 
title] assigned to your case is responsible for [job duties, like 
taking evidence, hearing objections, issuing decisions]. 
[Description of policies (if any exist) that ensure the agency 
component or adjudicators remain independent from investigative or 
enforcement activities]. [Description of rules about ex parte 
communications, if any exist].
    [Agency official or body] is responsible for evaluating the 
quality of [adjudicator title] decisions, and [agency official or 
body] conducts performance reviews of [adjudicator title]. [Agency 
official/entity from another agency] may remove the [adjudicator 
title] or [agency official or body/other entity] may discipline the 
[adjudicator title] by [kinds of discipline] when warranted.
    The agency has adopted rules of recusal [link] that allow a 
participant to request that the [adjudicator title] in charge of his 
or her case be disqualified if the participant believes the 
[adjudicator title] cannot fairly and impartially decide the case.
    If you are dissatisfied with an [adjudicator title] decision, 
you can request reconsideration from the [adjudicator title] or 
appeal that decision to [agency office/official]. Visit [link] for 
information on appealing an [adjudicator title] decision. [Agency 
office/official] may also review your case on [its/his or her] own 
initiative if there is an issue with the [adjudicator title]'s 
decision.
    For Further Information:

 Hiring process: [link]
 Pay rates: [link]
 Bonuses and performance incentives: [link]
 How cases are assigned to [adjudicator title]: [link]
 Communicating with [adjudicator title] (ex parte 
communications): [link]
 Process for addressing allegations that an [adjudicator 
title] has a conflict of interest (recusal and disqualification 
procedures): [link]
 How to appeal an [adjudicator title] decision: [link]
 Case-processing goals: [link]
 Process for addressing allegations of [adjudicator title] 
misconduct: [link]

    See also:

 Statutory provisions regarding [adjudicator title], 
including the appointment authority: [statutory citations]
 Agency regulations governing [adjudicator title]: [CFR 
provisions]

Appendix B

    [Note: Appendix B has been omitted from this notice because of 
the inaccessible images it contains. The full appendix may be found 
online at https://www.acus.gov/recommendation/publication-policies-governing-agency-adjudicators.]

Administrative Conference Recommendation 2020-6

Agency Litigation Web Pages

Adopted December 17, 2020

    Federal agencies and their component units \1\ participate in 
thousands of court cases every year. Most such cases result in 
``agency litigation materials,'' which this Recommendation defines 
as including agencies' publicly filed pleadings, briefs, and 
settlements, as well as court decisions, where such materials bear 
on agencies' regulatory or enforcement activities.
---------------------------------------------------------------------------

    \1\ The term ``component units'' encompasses an agency's sub-
units, which are often identified under terms like ``agency,'' 
``bureau,'' ``administration,'' ``office,'' ``division,'' or 
``service.'' For example, the United States Fish and Wildlife 
Service is a component unit of the Department of the Interior, and 
the Office of Water is a component unit of the United States 
Environmental Protection Agency.
---------------------------------------------------------------------------

    Public access to agency litigation materials is desirable for at 
least two reasons. First, because agency litigation materials often 
clarify how the federal government interprets and aims to enforce 
federal law, they can help people understand their legal 
obligations. Second, public access to agency litigation materials 
promotes accountable and transparent government. Those two reasons 
distinguish agency litigation materials from litigation filings by 
private parties.
    However valuable public access to agency litigation materials 
might be, federal law does little to mandate it. When it comes to 
agencies' own litigation filings, only the Freedom of Information 
Act (FOIA) requires disclosure, and then only when members of the 
public specify the materials in which they are interested (and no 
FOIA exception applies).\2\ In the same vein, the E-Government Act 
of 2002 requires federal courts to make their written opinions, 
including opinions in cases involving federal agencies, available on 
websites.\3\ But that requirement has not always made judicial 
opinions readily accessible to the public, partly because most 
courts' websites lack functions and features that would allow users 
to easily identify cases about specific topics or agencies.
---------------------------------------------------------------------------

    \2\ See 5 U.S.C. 552(a)(3).
    \3\ See 44 U.S.C. 3502(a).
---------------------------------------------------------------------------

    The most comprehensive source of agency litigation materials is 
the federal courts' Public Access to Court Electronic Records 
(PACER) service, which provides the public with instantaneous access 
to virtually every document filed in every federal court. But PACER 
searches often cost money, and the costs can add up quickly, 
especially when users are uncertain about what cases or documents 
they are trying to find. PACER's limited search functionality also 
makes it difficult to find cases involving particular agencies, 
statutes, regulations, or types of agency action. For example, a 
person interested in identifying ongoing cases to which the United 
States Fish and Wildlife Service (FWS) is a party would have to 
search for a host of terms--including ``United States Fish and 
Wildlife Service,'' ``U.S. Fish and Wildlife Service,'' and the 
names of FWS's recent directors--just to come close to identifying 
all such cases. Even after conducting all those searches, the person 
would still have to scroll through and eliminate search results 
involving state fish-and-wildlife agencies and private citizens with 
the same names as FWS's recent directors. Similarly, were a person 
interested in finding cases about FWS's listing of species under the 
Endangered Species Act (ESA), PACER would not afford that person any 
way to filter search results to include only cases about ESA 
listings. The person's only option would be to open and review 
documents in potentially thousands of cases.
    The cost and time involved in performing this type of research 
limit PACER's usefulness as a tool for locating and searching agency 
litigation materials. And although paid legal services, such as 
Westlaw and Lexis, have far greater search capabilities than PACER, 
their costs can dissuade many individuals and researchers.
    Agency litigation web pages, by contrast, can be a convenient 
way for the public to examine agency litigation materials. For 
purposes of this Recommendation, an agency litigation web page is a 
web page on an agency's website that systematically catalogs and 
links to agency litigation materials that may aid the public in 
understanding the agency's regulatory or enforcement activities. 
When agencies maintain up-to-date, search-friendly agency litigation 
web pages, the public can visit them and quickly find important 
filings in court cases concerning matters of interest. Agency 
litigation web pages thus make it easier for the public to learn 
about the law and to hold government accountable for agencies' 
actions.
    Several federal agencies already maintain agency litigation web 
pages.\4\ A survey of websites for twenty-five federal agencies 
revealed a range of practices regarding

[[Page 6625]]

agency litigation web pages.\5\ The survey suggests that most 
federal agencies do not maintain active agency litigation web pages. 
Among those that do, the quality of the agency litigation web pages 
varies appreciably. Some contain vast troves of agency litigation 
materials; others contain much more limited collections. Some are 
updated regularly; others are updated only sporadically. Some are 
easy to locate and search; others are not. In short, there appears 
to be no standard practice for publishing and maintaining agency 
litigation web pages, save that all the surveyed agency litigation 
web pages contained only the publicly filed versions of agency 
litigation materials, with all confidential material--such as trade 
secrets and personally identifiable information--redacted.
---------------------------------------------------------------------------

    \4\ See Mark Thomson, Report on Agency Litigation web pages 14-
16 (Nov. 24, 2020) (report to the Admin. Conf. of the U.S.), https://www.acus.gov/report/report-agency-litigation-web pages.
    \5\ See id. at 12-19 (identifying variations in agency 
practices). The survey conducted for this Recommendation covered all 
kinds of agencies--big and small, independent and not, regulatory 
and benefit-oriented, and so forth--with the aim of covering a broad 
and at least somewhat representative cross-section of federal 
agencies. In particular, the survey focused on agencies that are 
frequently in federal court or that are parties to a significant 
number of high-profile cases.
---------------------------------------------------------------------------

    An inspection of agencies' litigation web pages suggests four 
general features that make an agency litigation web page useful. 
First, an agency's litigation web page must be easy to find. Second, 
it must contain a representative and up-to-date collection of agency 
litigation materials. Third, those materials must be easy to search 
and sort. And fourth, the agency's litigation web page must give 
visitors the information they need to understand the materials on 
the web page, including information about materials the agency 
omitted from the web page and the criteria the agency employed to 
determine which materials to include on the web page.
    Agency litigation web pages can promote transparency and 
accountability. The Administrative Conference recognizes, however, 
that creating and maintaining a useful agency litigation web page 
takes time, money, and effort. An agency's decision to launch an 
agency litigation web page will necessarily be informed by 
considerations such as the agency's mission, litigation portfolio, 
existing technological capacity, budget, and the anticipated 
benefits--to the agency and the public--of creating an agency 
litigation web page.\6\ Further, an agency's decisions about what 
content to include on an agency litigation web page should be 
tailored to the agency's particular circumstances. An agency that 
litigates thousands of cases each year, for example, could choose to 
feature only a representative sample of agency litigation materials 
on its agency litigation web page.
---------------------------------------------------------------------------

    \6\ Most federal agencies do not have independent litigation 
authority but are represented in court by the Department of Justice 
(DOJ). In most cases, these agencies designate a DOJ liaison, who is 
then added as a recipient for all court filing notices, resulting in 
automatic access to all filings via PACER. This automatic access 
should enable implementation of this Recommendation by client 
agencies.
---------------------------------------------------------------------------

    Similarly, an agency that litigates many repetitive, fact-based 
cases could reasonably choose to post documents from just a few 
representative cases instead of posting documents from all of its 
cases.\7\ And an agency that litigates many different types of 
cases, some of obviously greater interest to the public than others, 
might appropriately restrict the contents of its agency litigation 
web page to agency litigation materials from the types of cases that 
are of greater public interest, particularly when the agency 
determines that the resources required to post more agency 
litigation materials can be better applied elsewhere.
---------------------------------------------------------------------------

    \7\ Cf. Admin. Conf. of the U.S., Recommendation 2017-1, 
Adjudication Materials on Agency websites, 82 FR 31,039, 31,040 
(July 5, 2017) (``Agencies that adjudicate large volumes of cases 
that do not vary considerably in terms of their factual contexts or 
the legal analyses employed in their dispositions should consider 
disclosing on their websites a representative sampling of actual 
cases and associated adjudication materials.'').
---------------------------------------------------------------------------

    Since the decision to create and maintain an agency litigation 
web page involves balancing factors that will differ from agency to 
agency, this Recommendation should not be read to suggest that 
agency litigation web pages be created and maintained by all 
agencies, especially those that litigate thousands of cases each 
year. Nor should this Recommendation be read as dictating the 
precise contents or structure of agency litigation web pages. While 
encouraging the creation and maintenance of agency litigation web 
pages, the Administrative Conference recognizes that an agency's 
particular circumstances might ultimately militate against creating 
an agency litigation web page or might support only the creation of 
a comparatively limited version.
    At bottom, this Recommendation simply offers best practices and 
factors for agencies to consider in making their agency litigation 
materials available on their websites, should the agencies choose to 
do so. The Recommendation leaves the weighing and balancing of those 
factors to the sound discretion of individual agencies.

Recommendation

Providing Access to Agency Litigation Materials

    1. Agencies should consider providing access on their websites 
to publicly filed pleadings, briefs, and settlements, as well as 
court decisions bearing on agencies' regulatory or enforcement 
activities (collectively ``agency litigation materials'').
    2. Should an agency choose to post such material, an agency with 
a large volume of court litigation could decide not to post 
documents from every case. The agency might, for instance, post 
examples of filings from routine litigation and all or a portion of 
the filings from cases raising important or unusual questions.
    3. In determining whether to provide access to agency litigation 
materials on their websites, and in determining which types of 
agency litigation materials to include on their websites, among the 
factors agencies should consider are the following:
    a. The public's interest in having ready access to certain 
categories of the agency's litigation materials;
    b. The extent to which providing access to agency litigation 
materials on the agency's website will advance the agency's mission;
    c. The internal benefits of maintaining a web page providing 
access to certain types of agency litigation materials;
    d. The costs of creating and maintaining a web page providing 
access to the types of agency litigation materials the agency sees 
fit to include;
    e. The nature of the agency's litigation portfolio, including 
the quantity of litigation materials the agency generates each year;
    f. The degree to which the agency's existing technological 
capacity can accommodate the creation and maintenance of a web page 
providing access to certain types of agency litigation materials;
    g. The availability and cost of other technological services 
that may more reliably and effectively give access to agency 
litigation material because of its scale or volume and the wide 
variety of issues and matters involved; and
    h. The risk of disclosure or increased dissemination of 
confidential or sensitive information of private litigants.
    4. In determining which agency litigation materials to include 
on their websites, agencies should ensure that they have implemented 
appropriate safeguards to protect relevant privacy or business 
interests implicated by the disclosure of agency litigation 
materials. Each agency should implement a protocol to ensure that, 
before a document is posted to the agency's litigation web page, the 
document has been reviewed and determined not to contain 
confidential information, such as trade secrets and personal 
identifying information.
    5. Agencies should disclose materials in a way that gives a full 
and accurate picture of their litigating positions. To provide 
proper context, agencies should:
    a. Use objective, clear, and publicly posted criteria to 
determine which agency litigation materials the agencies will 
publish on their websites;
    b. Regularly review their websites to ensure the agency 
litigation materials posted there (especially court opinions) are 
complete and up-to-date, and consider including notations regarding 
when material on the web page was last updated;
    c. Provide appropriate context for agency litigation materials, 
at least when failure to do so might confuse or mislead the public;
    d. Explain the types of litigation in which the agency is 
involved and other ways to search for any additional agency 
litigation materials not included on the agency's litigation web 
page, as well as opposing counsel's litigation filings;
    e. When resources permit, consider posting opposing parties' 
litigation filings when they are significant or important to 
understanding an issue;
    f. Neither present litigation materials as a means of setting 
policy, nor use those materials to circumvent rulemaking processes;
    g. Ensure that descriptions of agency litigation materials, if 
any, fairly reflect the litigation; and
    h. Recognize that some types of agency litigation materials may 
be of greater significance than others.
    6. Agencies that choose to post significant quantities of agency 
litigation materials on

[[Page 6626]]

their websites should consider grouping together links to those 
materials on a single, dedicated web page (an ``agency litigation 
web page''). If an agency is organized so that its component units 
have their own litigation portfolios, some or all of the component 
units may wish to have their own agency litigation web pages, or the 
agency may wish to maintain an agency litigation web page compiling 
litigation materials from or relating to the agency's component 
units.

Making It Easy To Locate Agency Litigation Web Pages

    7. Agencies that post agency litigation materials on their 
websites should make sure that website users can easily locate those 
materials. Agencies can accomplish this goal by:
    a. Displaying links to agency litigation web pages in readily 
visible locations on the homepage for the agency's website; and
    b. Maintaining a search engine and a site map or index, or both, 
on the agency's homepage.
    8. When an agency collects its component units' litigation 
materials on a single agency litigation web page, those component 
units' websites should clearly note that fact and include links to 
the agency's litigation web page. When an agency's component units 
maintain their own litigation web pages, the agency's website should 
clearly note that fact and include links to the component units' 
litigation web pages.

Making It Easy To Find Relevant Materials on Agency Litigation Web 
Pages

    9. Agencies and their component units should have substantial 
flexibility in organizing materials. Agencies should consider 
grouping together materials from the same and related cases on their 
agency litigation web pages. Agencies might, for example, consider 
providing a separate docket page for each case, with a link to the 
docket page on their agency litigation web pages. Agencies should 
also consider linking to the grouped-together materials when issuing 
press releases concerning a particular litigation.
    10. Agencies should consider offering general and advanced 
search and filtering options within their agency litigation web 
pages. The search and filtering options could, for instance, allow 
users to sort, narrow, or filter searches according to criteria such 
as action or case type, date, topic, case number, party name, a 
relevant statute or regulation, or specific words and phrases, along 
with any other criteria the agency decides are especially useful 
given its litigation activities.

[FR Doc. 2021-01273 Filed 1-21-21; 8:45 am]
BILLING CODE 6110-01-P