[Federal Register Volume 84, Number 84 (Wednesday, May 1, 2019)]
[Pages 18490-18492]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2019-08818]



National Institute of Standards and Technology

[Docket Number: [190312229-9229-01]]

Artificial Intelligence Standards

AGENCY: National Institute of Standards and Technology, U.S. Department 
of Commerce.

ACTION: Notice; Request for Information (RFI).


SUMMARY: The February 11, 2019, Executive Order on Maintaining American 
Leadership in Artificial Intelligence (AI) directs the National 
Institute of Standards and Technology (NIST) to create a plan for 
Federal engagement in the development of technical standards and 
related tools in support of reliable, robust, and trustworthy systems 
that use AI technologies (Plan). This notice requests information to 
help NIST understand the current state, plans, challenges, and 
opportunities regarding the development and availability of AI 
technical standards and related tools, as well as priority areas for 
federal involvement in AI standards-related activities. To assist in 
developing the Plan, NIST will consult with Federal agencies, the 
private sector, academia, non-governmental entities, and other 
stakeholders with interest in and expertise relating to AI.

DATES: Comments in response to this notice must be received May 31, 

ADDRESSES: Written comments in response to this RFI may be submitted by 
mail to AI-Standards, National Institute of Standards and Technology, 
100 Bureau Drive, Stop 2000, Gaithersburg, MD 20899. Online submissions 
in electronic form may be sent to [email protected]. Submissions 
may be in any of the following formats: HTML, ASCII, Word, RTF, or PDF. 
Please cite ``RFI: Developing a Federal AI Standards Engagement Plan'' 
in all correspondence. All relevant comments received by the deadline 
will be posted at https://www.nist.gov/topics/artificial-intelligence/ai-standards and regulations.gov without change or redaction, so 
commenters should not include information they do not wish to be posted 
(e.g., personal or confidential business information). Comments that 
contain profanity, vulgarity, threats, or other inappropriate language 
or content will not be posted or considered.

FOR FURTHER INFORMATION CONTACT: For questions about this RFI contact: 
Elham Tabassi, NIST, MS 8900, 100 Bureau Drive, Gaithersburg, MD 20899, 
telephone (301) 975-5292, email [email protected]. Please direct 
media inquiries to NIST's Public Affairs Office at (301) 975-NIST.


[[Page 18491]]

Genesis of the Plan for Federal Engagement in Artificial Intelligence 

    The Executive Order (E.O.) on AI \1\ states that ``[c]ontinued 
American leadership in AI is of paramount importance to maintaining the 
economic and national security of the United States and to shaping the 
global evolution of AI in a manner consistent with our Nation's values, 
policies, and priorities.'' Accordingly, Section 1 of the E.O. calls 
for a coordinated Federal Government strategy, the American AI 
Initiative, and notes that the U.S. must drive development of 
appropriate AI technical standards in order to enable the creation of 
new AI-related industries and the adoption of AI by today's industries. 
This can be achieved through the work and partnership of industry, 
academia, and government.

    \1\ https://www.whitehouse.gov/presidential-actions/executive-order-maintaining-american-leadership-artificial-intelligence/.

    Section 1(d) of the E.O. states that the U.S. must foster public 
trust and confidence in AI technologies and protect civil liberties, 
privacy, and American values in their application in order to fully 
realize the potential of AI technologies for the American people.
    Section 2(d) of the E.O. directs Federal agencies to ensure that 
technical standards minimize vulnerability to attacks from malicious 
actors and reflect Federal priorities for innovation, public trust, and 
public confidence, and to develop international standards to promote 
and protect those priorities.
    Section 6(d) of the E.O. directs the Secretary of Commerce, acting 
through the Director of NIST, to issue a Plan for Federal engagement in 
the development of technical standards and related tools in support of 
reliable, robust, and trustworthy systems that use AI technologies. It 
further directs NIST to lead the development of the Plan with 
participation from relevant agencies, as determined by the Secretary of 

Approach for Developing This Plan

    NIST will develop the Plan in a manner that fulfills the objectives 
of the E.O. and is consistent with relevant provisions of the Office of 
Management and Budget (OMB) Circular A-119, ``Federal Participation in 
the Development and Use of Voluntary Consensus Standards and in 
Conformity Assessment Activities,'' and NIST's mission to promote U.S. 
innovation and industrial competitiveness. NIST has a special interest 
in advancing the development and use of standards relied upon by all 
sectors of the economy and society, recognizing that the vast majority 
of standards are developed through a voluntary process led by the 
private sector.
    NIST will be informed through an open process that will include 
this RFI and other opportunities, such as a public workshop, to provide 
input. NIST expects to develop a draft Plan on which it will seek 
comment from the public and Federal agencies. Information about this 
effort, including ways to provide input, and future steps, will be 
available at https://www.nist.gov/topics/artificial-intelligence/ai-standards.

Goals of This Request for Information

    Timely and fit-for-purpose AI technical standards--whether 
developed by national or international organizations--will play a 
crucial role in the development and deployment of AI technologies, and 
will be essential in building trust and confidence about AI 
technologies and for achieving economies of scale.
    NIST seeks to understand the:
     Current status and plans regarding the availability, use, 
and development of AI technical standards and tools in support of 
reliable, robust, and trustworthy systems that use AI technologies;
     Needs and challenges regarding the existence, 
availability, use, and development of AI standards and tools; and
     The current and potential future role of Federal agencies 
regarding the existence, availability, use, and development of AI 
technical standards and tools in order to meet the nation's needs.
    For purposes of this Plan,\2\ AI technologies and systems are 
considered to be comprised of software and/or hardware that can learn 
to solve complex problems, make predictions or solve tasks that require 
human-like sensing (such as vision, speech, and touch), perception, 
cognition, planning, learning, communication, or physical action. 
Examples are wide-ranging and expanding rapidly. They include, but are 
not limited to, AI assistants, computer vision systems, automated 
vehicles, unmanned aerial systems, voicemail transcriptions, advanced 
game-playing software, facial recognition systems as well as 
application of AI in both Information Technology (IT) and Operational 
Technology (OT).

    \2\ This RFI is intended to be broadly directed to any and all 
technologies that might be considered AI by the US Government and 
other interested parties. AI systems have been defined in different 
ways, and this RFI is directed to any information that might fall 
within any of these definitions. See, for example, section 238(g) of 
the John S. McCain National Defense Authorization Act, 2019 (Pub. L. 
115-232), in which AI is defined to include the following:
    (1) Any artificial system that performs tasks under varying and 
unpredictable circumstances without significant human oversight, or 
that can learn from experience and improve performance when exposed 
to data sets;
    (2) An artificial system developed in computer software, 
physical hardware, or other context that solves tasks requiring 
human-like perception, cognition, planning, learning, communication, 
or physical action;
    (3) An artificial system designed to think or act like a human, 
including cognitive architectures and neural networks;
    (4) A set of techniques, including machine learning, that is 
designed to approximate a cognitive task; and
    (5) An artificial system designed to act rationally, including 
an intelligent software agent or embodied robot that achieves goals 
using perception, planning, reasoning, learning, communicating, 
decision making, and acting.

Responding to This Request for Information

    The scope of this RFI includes AI technical standards and related 
tools regardless of origin or use.\3\ Respondents may define 
``standards'' as they desire, indicating clearly what they mean when 
using the term. AI technical standards and related tools should include 
those necessary or helpful to reduce barriers to the safe testing and 
deployment of AI and to support reliable, robust, and trustworthy 
systems that use AI technologies.

    \3\ OMB Circular A-119 defines standards broadly to include: (1) 
Common and repeated use of rules, conditions, guidelines or 
characteristics for products or related processes and production 
methods, and related management systems practices; and (2) The 
definition of terms; classification of components; delineation of 
procedures; specification of dimensions, materials, performance, 
designs, or operations; measurement of quality and quantity in 
describing materials, processes, products, systems, services, or 
practices; test methods and sampling procedures; or descriptions of 
fit and measurements of size or strength.

    Respondents may define tools as broadly or as narrowly as they 
wish. They should indicate clearly what they mean when using specific 
terms (e.g., practices, datasets, guidelines). An illustrative, non-
exclusive list of standards-related tools includes:
     Test tools (e.g., executable test code) for conformance 
testing, performance testing, stress testing, interoperability testing, 
and other purposes;
     Use cases;
     Reference data and datasets;
     Reference implementations; and
     Training programs.
    Where this RFI uses the term ``organizations,'' it refers to 
private, public, and non-profit bodies, and

[[Page 18492]]

includes both national and international organizations. If desired, 
commenters may provide information about: The type, size, and location 
of their organization(s); and whether their organization develops AI 
technology and related tools; uses or potentially uses AI technology 
and related tools; and/or participates in the development of AI 
standards or related tools. Provision of such information is optional 
and will not affect NIST's full consideration of the comment.
    Comments containing references--including specific standards and 
related tools--studies, research, and other empirical data that are not 
widely published (e.g., available on the internet) should include paper 
or electronic copies of those materials, unless they are restricted due 
to copyright or are otherwise proprietary. In those cases, NIST 
encourages respondents to provide clear descriptions and designations 
of those references. Do not include in comments or otherwise submit any 
information deemed to be proprietary, private, or in any way 
confidential, as all comments relevant to this RFI topic area that are 
received by the deadline will be made available publicly at https://www.nist.gov/topics/artificial-intelligence/ai-standards and 
    The following list of topics covers the major areas about which 
NIST seeks information. This list is not intended to limit the topics 
that may be addressed by respondents, who may provide information about 
any topic which would inform the development of the Plan. Possible 
topics, subdivided by area, are:

AI Technical Standards and Related Tools Development: Status and Plans

    1. AI technical standards and tools that have been developed, and 
the developing organization, including the aspects of AI these 
standards and tools address, and whether they address sector-specific 
needs or are cross-sector in nature;
    2. Reliable sources of information about the availability and use 
of AI technical standards and tools;
    3. The needs for AI technical standards and related tools. How 
those needs should be determined, and challenges in identifying and 
developing those standards and tools;
    4. AI technical standards and related tools that are being 
developed, and the developing organization, including the aspects of AI 
these standards and tools address, and whether they address sector-
specific needs or are cross sector in nature;
    5. Any supporting roadmaps or similar documents about plans for 
developing AI technical standards and tools;
    6. Whether the need for AI technical standards and related tools is 
being met in a timely way by organizations; and
    7. Whether sector-specific AI technical standards needs are being 
addressed by sector-specific organizations, or whether those who need 
AI standards will rely on cross-sector standards which are intended to 
be useful across multiple sectors.
    8. Technical standards and guidance that are needed to establish 
and advance trustworthy aspects (e.g., accuracy, transparency, 
security, privacy, and robustness) of AI technologies.

Defining and Achieving U.S. AI Technical Standards Leadership

    9. The urgency of the U.S. need for AI technical standards and 
related tools, and what U.S. effectiveness and leadership in AI 
technical standards development should look like;
    10. Where the U.S. currently is effective and/or leads in AI 
technical standards development, and where it is lagging;
    11. Specific opportunities for, and challenges to, U.S. 
effectiveness and leadership in standardization related to AI 
technologies; and
    12. How the U.S. can achieve and maintain effectiveness and 
leadership in AI technical standards development.

Prioritizing Federal Government Engagement in AI Standardization

    13. The unique needs of the Federal government and individual 
agencies for AI technical standards and related tools, and whether they 
are important for broader portions of the U.S. economy and society, or 
strictly for Federal applications;
    14. The type and degree of Federal agencies' current and needed 
involvement in AI technical standards to address the needs of the 
Federal government;
    15. How the Federal government should prioritize its engagement in 
the development of AI technical standards and tools that have broad, 
cross-sectoral application versus sector- or application-specific 
standards and tools;
    16. The adequacy of the Federal government's current approach for 
government engagement in standards development,\4\ which emphasizes 
private sector leadership, and, more specifically, the appropriate role 
and activities for the Federal government to ensure the desired and 
timely development of AI standards for Federal and non-governmental 

    \4\ See the National Technology Transfer and Advancement Act, 
https://www.nist.gov/standardsgov/national-technology-transfer-and-advancement-act-1995, and OMB Circular A-119, https://www.whitehouse.gov/wp-content/uploads/2017/11/Circular-119-1.pdf.

    17. Examples of Federal involvement in the standards arena (e.g., 
via its role in communications, participation, and use) that could 
serve as models for the Plan, and why they are appropriate approaches; 
    18. What actions, if any, the Federal government should take to 
help ensure that desired AI technical standards are useful and 
incorporated into practice.

Kevin A. Kimball,
Chief of Staff.
[FR Doc. 2019-08818 Filed 4-30-19; 8:45 am]