[Federal Register Volume 79, Number 8 (Monday, January 13, 2014)]
[Notices]
[Pages 2201-2204]
From the Federal Register Online via the Government Publishing Office [www.gpo.gov]
[FR Doc No: 2014-00404]


=======================================================================
-----------------------------------------------------------------------

OFFICE OF SCIENCE AND TECHNOLOGY POLICY


Notice of Request for Information (RFI)

SUMMARY: The Office of Science and Technology Policy requests public 
comments to inform its policy development related to high-impact 
learning technologies. This Request for Information offers the 
opportunity for interested individuals and organizations to identify 
public and private actions that have the potential to accelerate the 
development, rigorous evaluation, and widespread adoption of high-
impact learning technologies. The focus of this RFI is on the design 
and implementation of ``pull mechanisms'' for technologies that 
significantly improve a given learning outcome. Comments must be 
received by 11:59 p.m. on March 7, 2014, to be considered. In your 
comments, please reference the question to which you are responding.

DATES: Comments must be received by 11:59 p.m. on March 7, 2014, to be 
considered.

ADDRESSES: Respondents are encouraged to submit their comments through 
one of the following methods. Email is the preferred method of 
submission. Please do not include in your comments information of a 
confidential nature, such as sensitive personal information or 
proprietary information. Responses to this notice are not offers and 
cannot be accepted by the Federal Government to form a binding contract 
or issue a grant. Information obtained as a result of this notice may 
be used by the Federal Government for program planning on a non-
attribution basis. Please be aware that your comments may be posted 
online.
     Email: [email protected]. Email submissions will receive 
an electronic confirmation acknowledging receipt of your response, but 
will not receive individualized feedback on any suggestions.
     Postal Mail: Office of Science and Technology Policy, 
Attn: Cristin Dorgelo, 1650 Pennsylvania Avenue NW., Washington, DC 
20504. Submissions by postal mail must be received by the deadline, and 
should allow sufficient time for security processing.
     Fax: 202.456.6021

SUPPLEMENTARY INFORMATION: This Request for Information (RFI) offers 
the oppm1unity for interested individuals and organizations to identify 
public and private actions that have the potential to accelerate the 
development, rigorous evaluation, and widespread adoption of high-
impact learning technologies. The focus of this RFI is on the design 
and implementation of ``pull mechanisms'' for technologies that 
significantly improve a given learning outcome. Pull mechanisms 
increase the incentives to develop specific products or services by 
committing to reward success. Examples of pull mechanisms include 
incentive prizes, Advance Market Commitments, milestone payments, ``pay 
for success'' bonds, and purchasing consm1ia. The public input provided 
through this notice will inform the deliberations of the Office of 
Science and Technology Policy (OSTP).

Background

    OSTP is interested in identifying policies and serving as a 
catalyst for public-private pat1nerships that have the potential to 
accelerate the development, rigorous evaluation, and widespread 
adoption of high-impact learning technologies. For example, imagine if 
learners in the United States had access to technologies that:
     Dramatically reduced the large and persistent gap in 
vocabulary size between children from wealthy and poor households.
     Allowed middle and high school students to outperform 
their international peers in math and science.
     Enabled English-language learners that are reading at 
several grade levels below average to catch up after only a year.
     Gave non-college bound students an industry skills 
ce1tification or set of cognitive skills (e.g. literacy, numeracy, 
ability to understand and apply chmis, graphs and diagrams) that are a 
ticket to a middle-class job, increasing their employability and their 
incomes by $10,000-$20,000 or more in less than a year.
     Doubled the percentage of community college students that 
pass remedial math, which is currently only 30 percent.

[[Page 2202]]

     Successfully delivered a ``growth mindset'' intervention 
to teachers and students.
     Were as effective as a personal tutor, were as engaging as 
the best video game, and improved the more students used them.
    Currently, there is a large gap between the relatively modest 
impact that technology has had on education, particularly in K-12, and 
the transformative impact that it has had in many aspects of our 
economic and social life. For example, businesses are using information 
and communications technologies to dramatically increase productivity, 
tap the expe1iise of their employees, slash the time needed to develop 
new products, tailor products and services to meet the needs of 
individual consumers, orchestrate global networks of suppliers, derive 
insights from huge volumes of transactional data, and improve their 
products and services by conducting rapid, low-cost experiments.
    Education, particularly K-12 education, remains relatively 
untouched by advances in our understanding of how people learn, how to 
design instruction that incorporates those insights, and the explosion 
in information technologies such as low-cost smartphones and tablets, 
cloud computing, broadband networks, speech recognition and speech 
synthesis, predictive analytics, data mining, machine learning, 
intelligent tutors, simulations, games, computer-suppmied collaborative 
work, and many other technologies. That is why President Obama has 
proposed ConnectED, a new initiative to connect 99 percent of America's 
students to the Internet through high-speed broadband and high-speed 
wireless within 5 years.
    Learning technologies will be much more effective if they informed 
by ``learning science''--advances in disciplines in fields such as 
neuroscience, cognitive science, educational psychology, and 
discipline-based education research that shed light on how people 
learn. This research can provide actionable insights on issues such as 
student motivation, the circumstances under which prior knowledge helps 
or hmis learning, how students can organize knowledge in rich and 
meaningful ways, and the ways in which students can progress from 
novice to expeti in a given domain.
    There are a number of reasons for the gap between the potential of 
learning science and technology and the cunent state-of-the-practice:
     The United States is investing 0.1 percent of K-12 
expenditures on R&D, compared to 2 percent in mature industries and 
18.7 percent in the pharmaceutical industry. This extremely low level 
of investment in educational R&D has clearly limited the pace of 
innovation.
     Entrepreneurs seeking to develop and market new products 
to the K-12 market face a number of challenges, including low per-pupil 
expenditures on software, lengthy adoption cycles, and a highly 
fragmented market. This in turn limits the amount that companies can 
spend on research and product development.
     It is difficult for companies to make authoritative claims 
about the impact of their products on learning outcomes assessed 
through rigorous third-party validation, which limits the premium that 
school districts and other consumers of learning technology are willing 
to pay for high-quality, effective products.
    This suggests that an effective national strategy for increasing 
the impact of learning science and technology should address both the 
``supply'' and ``demand'' for advanced learning technologies.
    To increase the ``supply'' of learning technology, the Federal 
government and philanthropists could increase funding for research and 
development and support training grants and scholarships in relevant 
disciplines such as educational psychology, cognitive science, 
instructional design, artificial intelligence, etc. The National 
Science Foundation is funding a program called ``Cyberlearning 
Transforming Education'' and the Depmiment of Defense is supporting 
research in advanced training technologies. The President FY14 Budget 
request includes funding for a ``DARPA for Education'' (ARPA-ED).

The Power

    However, there has been little discussion of the potential of what 
economists call ``pull mechanisms'' to accelerate the development, 
evaluation, and adoption of high-impact learning technologies.
    As economists have recently noted, governments and other funders 
can suppmt innovation using ``push'' programs (e.g. funding grants and 
contracts to universities and companies, providing tax incentives for 
R&D, or supporting government laboratories) and ``pull'' mechanisms 
that ``increase the rewards for developing specific products by 
committing to reward success.'' Push programs pay for research inputs; 
pull mechanisms pay for research outcomes.
    ``Pull mechanisms'' have been used successfully in the field of 
global health. In December 2010, children in developing countries began 
receiving a vaccine that will prevent deaths from ``pneumococcal'' 
diseases including pneumonia, meningitis, and sepsis. Nearly one 
million young children die every year from pneumococcal infections, 
with 90 percent of these deaths occurring in developing countries.
    The development of this vaccine was accelerated by a $1.5 billion 
``Advance Market Commitment'' backed by five governments and a private 
foundation. Pharmaceutical companies that have agreed to provide the 
vaccine at $3.50 per dose to low-income countries for the next 10 years 
will receive additional payments from the $1.5 billion in donor 
commitments. The AMC increased the size and predictability of the 
market for pneumococcal vaccines, which increased the willingness of 
companies to invest in high-volume production of these vaccines for 
developing country markets. Expe1ts predict that this AMC will save 7 
million lives over the next twenty years.
    Non-binding commitments to purchase products can also provide 
market pull, if there is both a clearly defined performance 
specification and a strong expression of interest from potential 
buyers. For example, in June 2013, the U.S. Department of Energy put 
together a coalition of the Federal government and over 200 major 
commercial building pmtners that issued a challenge to U.S. 
manufacturers: ``If you can build wireless sub-meters that cost less 
than $100 apiece and enable us to identify opportunities to save money 
by saving energy, we will buy them.'' At least 18 manufacturers agreed 
to take up the challenge. In 2011, the Department of Energy put 
together a similar and successful challenge for energy-efficient and 
cost-effective commercial air conditioners, with the first manufacturer 
meeting the challenge in May 2012.
    In addition, Federal agencies have offered almost 300 incentive 
prizes on Challenge.gov, providing opportunities for citizen solvers to 
offer novel solutions to tough problems, while minimizing risk to 
Federal agencies by only paying for success. More information about 
pull mechanisms can be found in this supplemental information document.
    OSTP is interested in stimulating a conversation about how pull 
mechanisms could be used to accelerate the development, evaluation, and 
adoption of learning technologies. Some of the advantages of pull 
mechanisms are that a funder can (a) pay only for

[[Page 2203]]

success; (b) set a goal without having to choose in advance which team 
or approach is most likely to be successful; and (c) increase the 
number and intellectual diversity of the teams that are working to 
solve a particular problem. Although there a variety of different types 
of pull mechanisms, they generally require establishing a clear goal 
and an agreed-upon set of metrics for evaluating progress towards that 
goal. If education is going to benefit from increased use of pull 
mechanisms, policy-makers and stakeholders have to identify some 
specific challenges that are important and measurable, and where it is 
plausible that learning technology can help improve student outcomes.

Using Pull Mechanisms for Learning Technologies

    Pull mechanisms can be used for social interventions that do not 
use technology. For example, the first ``social impact bond'' is being 
used by the United Kingdom to reduce recidivism among 3,000 prisoners. 
The United Kingdom's Depa11ment for International Development (DfiD) is 
supporting a ``Results-Based Aid'' approach to improving education in 
Ethiopia. Under this pilot, DfiD will make grant payments to the 
education ministry for the increase in the number of students above a 
baseline that sits for or passes the national grade 10 exam. There will 
be additional payments for students in the poorest regions, and for 
girls compared to boys.
    It may also make sense to experiment with pull mechanisms to 
accelerate the development and rigorous evaluation of learning 
technologies. Some of the potential advantages of learning technologies 
include:
     Low marginal cost: The marginal cost of making software or 
digital content and services available to more students is very low, 
although the fixed cost of R&D and rigorous evaluation may be high. 
This is why IT stmtups are able to grow rapidly--the cost of serving 
tens or hundreds of millions of customers does not increase 
arithmetically with the number of customers.
     Ability to maintain high levels of ``time on task'': For 
example, good game developers can keep users riveted for hours at a 
time. They can create experiences that are intrinsically motivating, 
and that offer an increasingly difficult set of challenges that keep 
users in the ``sweet spot'' between being bored and frustrated.
     Continuous improvement: The productivity of most public 
sector services is flat or negative. Researchers and entrepreneurs have 
ideas for developing online services that get better the more people 
use them by (a) conducting many low-cost experiments to discover what 
works; and (b) collect, analyze and act on the data that can be 
generated online.
     Learning anytime, anywhere: Mobile devices allow 
individuals to access digital content at a time, place, and pace that 
is convenient for them. This might be particularly impmiant for an 
adult who is trying to upgrade their skills while balancing the 
competing demands of work and family.
     Digital tutors: Research suggests that the average student 
tutored one-on-one using ``mastery learning'' techniques (students are 
helped to master each concept before proceeding to a more advanced 
learning task) performed better than 98 percent of the students that 
learn the same material using conventional instructional methods. 
Projects funded by DARPA and the Office of Naval Research suggest that 
it may be possible to develop ``digital tutors'' that model the one-on-
one interaction between a world-class subject matter expeti and a 
student. A pilot suppmied by the Veteran's Administration is allowing 
unemployed veterans that use the digital tutor for 6 months to get IT 
jobs that pay $40,000 to $80,000.
     Personalization: Researchers and firms are developing 
software and online services that are personalized to the needs, 
background, interests and skill levels of individuals.
     Interactive simulations that enable ``learning by doing'': 
Researchers have developed simulations in areas such as physics, 
chemistry, biology, emih science, and math. For example, an ``Energy 
Skate Park'' simulation allows students to explore energy conservation 
with multiple different variables (shape of the track, starting height 
and speed of the skater, mass of the skater, and friction). Students 
can quicldy repeat experiments and rapidly explore the effect of many 
different parameters.
     Embedded assessment: Technology can help provide 
continuous assessment of a given set of knowledge, skills and abilities 
if the designers know (a) what behaviors would constitute evidence that 
a student has mastered a given competency; and (b) which tasks can 
elicit those behaviors.

Questions

    To stimulate a national conversation on whether and how pull 
mechanisms might be used to accelerate the development of high-impact 
learning technologies, OSTP seeks public comment on the questions 
listed below:
    (1) What learning outcomes would be good candidates for the focus 
of a pull mechanism to catalyze the creation and use of new learning 
technology? These outcomes could be relevant to early childhood 
education, K-20, life-long learning, workforce readiness and skills, 
etc.
    (2) How are these learning outcomes currently measured and 
assessed?
    (3) What information exists about current U.S. performance relative 
to this learning outcome? What information exists about the presence 
(currently available or potential given current trends or 
breakthroughs) or absence of effective interventions (technology-based, 
offline, or hybrid) to improve this learning outcome?
    (4) Why would a pull mechanism in this area accelerate innovation 
in learning technology?
    (5) What role might different stakeholders (e.g. Federal agencies, 
state and local educational agencies, foundations, researchers, 
practitioners, companies, investors, or non-profit organizations) play 
in designing, funding, and implementing a pull mechanism for learning 
technology? What role would your organization be willing to play?
    (6) What changes in public policy would facilitate experimentation 
with pull mechanisms at different levels of government?
    Response to this RFI is voluntary. Responders are free to address 
any or all the above items, as well as provide additional information 
that they think is relevant to accelerating the development, rigorous 
evaluation and widespread adoption of high-impact learning 
technologies. Please note that the U.S. Government will not pay for 
response preparation or for the use of any information contained in the 
response.

Ted Waelder,
Deputy Chief of Staff and Assistant Director.

Supplementary Information: Overview of Pull Mechanisms

    Incentive prizes are one type of ``pull mechanism''--results-based 
market incentives designed to overcome market failures and catalyze 
itmovation. Experts often make a distinction between ``recognition'' 
prizes that honor past achievements and ``inducement'' or ``incentive'' 
prizes that encourage participants in the competition to achieve a 
particular goal. In a 2009 repot1, McKinsey identified six prize 
archetypes that provide a useful framework for identifying types of 
prizes that can best achieve different types of goals:

[[Page 2204]]

     Exemplar Prizes that define excellence within an area.
     Point Solution Prizes that aim to spur development of 
solutions for a pmiicular well-defined problem. Solutions can include 
software applications, algorithms, predictive models, ideas, business 
plans, policy proposals, designs, or prototypes.
     Market Stimulation Prizes that try to establish the 
viability of a market to address a potential market failure, mobilize 
additional human talent and financial capital to jumpstati the 
development of a new industry, or change public perceptions about what 
is possible.
     Exposition Prizes that are designed to highlight a broad 
range of promising ideas and practices, attract attention, and mobilize 
capital to further develop the winning innovations.
     Participation Prizes that create value during and after 
the competition- not through conferral of the prize award itself but 
through their role in encouraging contestants to change their behavior 
or develop new skills that may have beneficial effects during and 
beyond the competition.
     Network Prizes that build networks and strengthen 
communities by organizing winners into new problem-solving communities 
that can deliver more impact than individual effmis.
    Other types of pull mechanisms include:
     Advance Market Commitments: Binding commitments to 
purchase, or to subsidize purchase, of a ce1iain volume of a product at 
a fixed prize, if the product meets pre defined performance 
characteristics (pneumococcal vaccine and Department of Energy examples 
discussed above).
     Buyer's Consortia: Cooperative agreements between 
purchasers of products that leverage the combined buying power of those 
purchasers to drive down the price of products, such as a buyer's 
consmiium set up for Maine school districts to purchase specialized 
software and specific assistive technology devices.
     Pay-for-Success Bonds: Under a Pay for Success bond, also 
known as a social impact bond, the financing organization and the 
Federal, state, or local government enter into a contract that 
specifies the population to be served, the outcomes to be achieved, the 
measurement methodology to be used, and the schedule of payments to be 
made. The financing organization works with philanthropic and other 
investors to invest in innovative, data-driven service providers that 
can achieve results. One example of a pay for-success bond program is 
an initiative in New York targeted at reducing recidivism in adult 
males.
     Milestone-based Payments: Payment terms in a standard 
grant or contract in which the payment for each performance milestone 
established in the statement of work is not made until the milestone is 
proven to have been achieved. One example of this approach has been 
successfully demonstrated in NASA 's Commercial Orbital Transportation 
Services (COTS) program.
     Priority Review Vouchers: An accelerated regulatory review 
offered to products that meet certain performance or cost criteria, 
such as the FDA Innovation Pathway and USPTO 's Patents for Humanity.
     Patent Buyout: An offer to buy out the patent rights to a 
product that meets specified performance conditions at a set price 
(price for patent usually marked up over market value; followed by 
placing of the patent into the public domain to encourage competition 
for commercialization of the product). One example is the purchase of 
the patent for the Daguerreotype process by the French government in 
1839.

[FR Doc. 2014-00404 Filed 1-10-14; 8:45 am]
BILLING CODE P