Tax Administration: IRS Should Reassess the Level of Resources
for Testing Forms and Instructions (11-APR-03, GAO-03-486).
Taxpayers rated the Internal Revenue Service's (IRS) ability to
provide clear and easy-to-use forms and instructions among the
lowest of 27 indicators of service in 1993. Due to continuing
concerns about unclear forms and instructions, GAO was asked to
determine (1) whether and how often IRS tests the clarity of new
and revised individual income tax forms and instructions; (2) the
benefits, if any, of testing forms and instructions for clarity
prior to their use; and (3) whether any factors limit IRS's
ability to do more tests and if so, how they can be addressed.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-03-486
ACCNO: A06642
TITLE: Tax Administration: IRS Should Reassess the Level of
Resources for Testing Forms and Instructions
DATE: 04/11/2003
SUBJECT: First article testing
Forms (documents)
Income taxes
Internal controls
Performance measures
Tax administration
Earned Income Credit
Child Tax Credit
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-03-486
a
GAO United States General Accounting Office
Report to the Chairman and Ranking Minority Member, Committee on Finance,
U. S. Senate
April 2003 TAX ADMINISTRATION IRS Should Reassess the Level of Resources
for Testing Forms and Instructions
GAO- 03- 486
IRS used taxpayers and its employees to test revisions to five individual
income tax forms and instructions from July 1997 through June 2002.
According to IRS officials, they revised about 450 tax forms and
instructions in 2001, many of which were for individual income tax
returns.
Testing forms and instructions can help ensure their clarity and thereby
benefit taxpayers and IRS by, for instance, reducing taxpayers* time to
understand and complete tax forms, reducing calls to IRS for assistance,
and reducing taxpayer errors. Due to similar benefits, federal agencies we
contacted that routinely collect information from the public test their
questionnaires. Quantifying benefits due to testing is difficult, but
IRS*s experience in revising and testing Earned Income Credit and Child
Tax Credit forms and instructions suggests that benefits of testing in
some cases can considerably exceed the cost of testing. If taxpayers who
did their own tax returns needed 1 less minute to understand these two
credits due to testing, their time saved, valued at the minimum wage,
would be worth $1.2 million; IRS*s contracting cost for the two tests was
$56,000.
Although IRS officials recognized that testing could be beneficial, they
cited tight time frames and constrained resources as limiting their
ability to do more tests. While IRS faces time constraints when making
some changes to forms and instructions due to the passage of new laws, not
all changes are
time constrained. IRS does not have procedures specifying which versions
of draft forms and instructions should be tested with taxpayers or when in
its annual forms development process testing should occur. Resources
currently available for testing are limited but the office responsible for
testing has not developed data on missed testing opportunities and has
limited data on the benefits that have been realized when testing
occurred. IRS*s planning and budgeting process uses such data to support
resource allocation decisions.
Potential Benefits If Testing Helps Ensure Clarity of Tax Forms and
Instructions IRS*s benefits Taxpayers* benefits
Fewer errors needing correction
Fewer audits due to clarity- related taxpayer errors Less demand for
assistance at local
IRS offices may allow IRS to provide better service to other taxpayers
Less demand for IRS*s toll- free telephone assistance may enable IRS to
answer some calls that currently go unanswered
Knowledge gained from testing may lead to clearer forms and instructions
in the future
Reduction in time and expenses to prepare tax returns
Reduced burden from not having to deal with IRS notices, such as reduced
time to
-open and read IRS*s notices -decide what to do -research tax records
-prepare response to IRS -copy and mail response, if
necessary, to IRS -call IRS for assistance
Source: GAO TAX ADMINISTRATION
IRS Should Reassess the Level of Resources for Testing Forms and
Instructions
www. gao. gov/ cgi- bin/ getrpt? GAO- 03- 486. To view the full report,
including the scope and methodology, click on the link above. For more
information, contact Michael Brostek at (202) 512- 9110 or brostekm@ gao.
gov. Highlights of GAO- 03- 486, a report to
Chairman and Ranking Member of the Senate Committee on Finance
April 2003
Taxpayers rated the Internal Revenue Service*s (IRS) ability to provide
clear and easy- to- use forms and instructions among the lowest of 27
indicators of service in 1993.
Due to continuing concerns about unclear forms and instructions, GAO was
asked to determine whether and how often IRS
tests the clarity of new and revised individual income tax forms and
instructions; the benefits, if any, of testing
forms and instructions for clarity prior to their use; and whether any
factors limit IRS*s
ability to do more tests and if so, how they can be addressed. This report
makes a series of recommendations that, if fully implemented, would
improve IRS*s annual process for creating and revising individual income
tax
forms and instructions by helping to ensure that its scarce testing
resources are consistently applied to the highest priorities, impediments
to testing are lessened, and appropriate information is developed so IRS
management can better ensure that adequate resources are available to
support testing in view of the
potential benefits to taxpayers and IRS.
IRS agreed with our recommendations and plans to implement all but one of
them in time for the 2004 forms development cycle.
Page i GAO- 03- 486 Testing Forms and Instructions Letter 1 Results in
Brief 2 Background 4 IRS Tested Five Tax Forms and Instructions Over 5
Years 6 Testing Written Documents Helps Ensure Clarity and Benefits
Taxpayers and IRS 7 IRS Has Not Addressed Constraints to Increased Testing
20 Conclusions 25 Recommendations for Executive Action 26 Agency Comments
and Our Evaluation 26 Appendix I Objectives, Scope, and Methodology 30
Appendix II Use of Testing in NCHS, Census, and BLS 35
Appendix III Comments from the Internal Revenue Service 37
Tables
Table 1: Forms and Instructions IRS Tested Using Individual Taxpayers
Between July 1997 and June 2002 6 Table 2: Percentage Decrease in the
Number of EIC and Child Tax
Credit Errors before and after Revising and Testing 9 Table 3: Potential
Benefits if Testing Helps Ensure the Clarity of Tax Forms and Instructions
11 Table 4: Illustration of Potential Savings to IRS Alone in 2000 from
Testing EIC and Child Tax Credit Forms and Instructions 15 Table 5:
Illustration of Potential Savings to Taxpayers and IRS in
2000 from Error Reductions Alone from Testing EIC and Child Tax Credit
Forms and Instructions 16 Table 6: Illustration of Potential Benefit to
Taxpayers in 2000 if
Testing Reduced the Time Needed to Understand and Complete EIC and Child
Tax Credit Forms 17 Table 7: Illustration of Potential Cost Savings to IRS
in 2002 if the
Rate Reduction Credit Instructions Had Been Tested and Errors Were Reduced
18 Contents
Page ii GAO- 03- 486 Testing Forms and Instructions
Table 8: Illustration of Potential Savings to Taxpayers and IRS in 2002
from Potential Error Reductions if Testing the Rate Reduction Credit Had
Reduced Taxpayer Errors 19 Table 9: Illustration of Potential Benefit to
Taxpayers in 2002 if
Testing Reduced the Time Needed to Understand and Complete the Rate
Reduction Credit 19 Figures
Figure 1: Simplified Overview of IRS*s Annual Tax Forms Development
Process 5 Figure 2: Simplified Overview of Where Testing Can Occur in
IRS*s
Annual Tax Forms Development Process 22 Abbreviations
BLS Bureau of Labor Statistics EIC Earned Income Credit ERS Error
Resolution System FTE full- time equivalent IRS Internal Revenue Service
NCHS National Center for Health Statistics OMB Office of Management and
Budget TIGTA Treasury Inspector General for Tax Administration W& I Wage
and Investment
This is a work of the U. S. Government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. It may contain
copyrighted graphics, images or other materials. Permission from the
copyright holder may be necessary should you wish to reproduce copyrighted
materials separately from GAO*s product.
Page 1 GAO- 03- 486 Testing Forms and Instructions April 11, 2003 The
Honorable Charles E. Grassley Chairman,
Committee on Finance United States Senate
The Honorable Max Baucus Ranking Minority Member, Committee on Finance
United States Senate
Internal Revenue Service (IRS) tax forms and instructions that are not
clear and understandable have been a long- standing frustration of
taxpayers. 1 For example, respondents to a 1993 customer satisfaction
survey rated IRS*s ability to provide clear and easy- to- use tax forms
and instructions and to minimize taxpayer*s burden as the lowest of 27
indicators of service. 2 More recently, we cited the instructions for the
tax year 2001 rate reduction credit as confusing and one factor
contributing to millions of taxpayer errors. 3 To be fair, IRS faces a
challenge in clearly communicating with taxpayers
due to the complexity of the Internal Revenue Code. However the code*s
complexity also underscores the importance of tax forms and instructions
that are as clear and understandable as possible.
One approach with recognized potential for aiding in the development of
clearer forms and instructions is testing. By testing draft forms and
instructions with taxpayers, IRS might be able to detect and revise
language that is unnecessarily confusing. The Treasury Inspector General
for Tax Administration (TIGTA) and we have both previously 1 Tax forms and
instructions as referred to in this report also include related schedules
and worksheets. 2 1993 IRS Customer Satisfaction Survey: Public
Perceptions of IRS Service Quality, Contract number TIR 90- 0002 (Aug.
27,1993). 3 U. S. General Accounting Office, Tax Administration: Advance
Tax Refund Program Was a Major Accomplishment, but Not Problem Free, GAO-
02- 827 (Washington, D. C.: Aug. 2,
2002).
United States General Accounting Office Washington, DC 20548
Page 2 GAO- 03- 486 Testing Forms and Instructions recommended that IRS do
more to, for instance, identify what individual taxpayers find difficult
to understand about tax forms or publications. 4 Because of your interest
in ensuring that taxpayers have clearer, understandable tax forms and
instructions and because of the potential for
testing to contributing to that end, you asked us to determine: whether
and how often IRS used taxpayers to test the clarity of new or revised
individual income tax forms and instructions; the benefits, if any, of
having taxpayers test forms and instructions for
clarity prior to their use by the public; and whether any factors limit
IRS*s ability to test more forms and
instructions for clarity and if so, how these factors can be addressed. To
address these objectives, we interviewed officials (1) in IRS*s Wage and
Investment (W& I) Division*s Tax Forms and Publications Division, (2) from
three federal agencies that use private citizens extensively in testing
research and data collection activities, and (3) a private research firm
that
specializes in testing written documents, such as forms and surveys, for a
variety of public and private sector clients. We also developed
illustrations of the potential benefits to taxpayers and IRS of testing
forms and instructions for clarity. Our scope and methodology are
discussed in greater detail in appendix I to this report.
From July 1997 through June 2002, IRS used taxpayers or its employees,
primarily in focus groups, to test 5 individual income tax forms or
instructions. According to IRS officials, they revised about 450 forms and
instructions in 2001, many of which were for individual income tax
returns.
Testing written documents, such as forms and instructions, prior to their
use helps ensure that they communicate clearly. Testing thereby helps (1)
lessen the burden to respondents of understanding and completing forms and
(2) reduce errors made by respondents. To the extent that taxpayers
4 The Process of Developing Tax Forms for Individual Taxpayers Should be
Further Improved, TIGTA 2000- 40- 060 (March 2000); U. S. General
Accounting Office, Tax Administration: IRS Efforts to Improve Forms and
Publications, GAO/ GGD- 95- 34 (Washington, D. C.: Dec. 7, 1994). Results
in Brief
Page 3 GAO- 03- 486 Testing Forms and Instructions find IRS*s tax forms
and instructions that were tested easier to understand, IRS is less likely
to be contacted by taxpayers for assistance in fulfilling
their tax obligations and is likely to have fewer taxpayer errors to
detect and correct. Benefits like these may be difficult to quantify, but
form the basis for several agencies we contacted, as well as ourselves, to
make testing a standard practice when developing or modifying documents
used to collect information from the public. IRS*s recent, limited
experience with testing forms and instructions used by individuals
suggests that testing is beneficial. At least in some cases the potential
benefits of testing IRS*s forms and instructions are likely to be
substantially greater than the costs of testing to IRS.
Although IRS officials said that making greater use of testing to improve
clarity of forms and instructions could be beneficial, they cited tight
time frames for making changes and limited resources in the Tax Forms and
Publications Division as the primary factors that limit their ability to
conduct more tests. Although IRS does face time constraints when making
some changes, especially when changes are due to new tax laws, time
constraints do not apply to all changes IRS makes to forms and
instructions for individual taxpayers. Further, IRS does not have clear
procedures specifying which draft versions of forms and instructions
should be tested with taxpayers or when testing should occur during the
annual forms update process. Resources available within the division
responsible for developing forms and instructions are limited. For
example, one employee has responsibility for organizing testing efforts.
However, the division has not developed information that IRS uses when
making resource allocation decisions. The office has not developed data on
the universe of changes for which testing is most likely to yield
significant benefits to taxpayers and IRS and it has only limited data on
the benefits that have been realized when testing has occurred.
This report makes a series of recommendations to improve IRS*s annual
process for creating and revising individual income tax forms and
instructions through ensuring that (1) its scarce testing resources are
consistently applied to the highest priorities, (2) impediments to testing
are lessened, and (3) appropriate information is developed so IRS managers
can ensure that adequate resources are available for testing in view of
the potential benefits to taxpayers and IRS. IRS agreed with our
recommendations and plans to implement all but one of them in time for the
2004 forms development cycle. The remaining recommendation* to ensure that
an appropriate range of evaluations is conducted of tests* would take more
time to put into practice.
Page 4 GAO- 03- 486 Testing Forms and Instructions Although most
organizational components within IRS are involved, the Tax Forms and
Publications Division within IRS*s Media and Publications Division of the
W& I Division is primarily responsible for creating and improving tax
forms, instructions, and other documents. One goal of the
Tax Forms and Publications Division is to make tax forms and instructions
as clear and understandable as possible. It is divided into three
branches* Individual Forms and Publications, Business Forms and
Publications, and Tax Exempt/ Government Entities and Specialty Forms and
Publications.
As of late January 2003, 103 persons were assigned to the Tax Forms and
Publication Division, including about 15 persons whose primary
responsibility was creating, revising, and reviewing individual income tax
forms and instructions.
Many tax forms and instructions are revised annually, often with short
turnaround times and in response to tax law changes. IRS also periodically
reviews tax forms and, when appropriate, schedules them for revision.
According to IRS*s estimate, it revised about 450 tax forms and
instructions in 2001 that affected individual and business tax returns. In
addition to tax law changes, revisions to tax forms and instructions
generally reflected procedural changes, legal rulings, and feedback from
internal and external stakeholders about the understandability of forms
and instructions. As illustrated in figure 1, the annual tax forms
development process generally starts with a review of the current tax
forms. IRS*s tax law specialists review the existing forms and
instructions to determine what changes, if any, may be needed to reflect
tax law changes and other requirements. The tax law specialists consider
comments from a variety of sources both within and outside IRS. For
example, comments may be obtained from IRS customer service staff in toll-
free call centers who answer calls from taxpayers and have firsthand
knowledge of particular forms or instructions that were confusing to
taxpayers. IRS*s Taxpayer Advocate Service staff may also provide comments
useful to the tax law specialists. The Tax Forms Coordinating Committee,
comprised of representatives from all of IRS*s key components, the
Department of the Treasury, and IRS*s Chief Counsel, reviews draft forms
to help ensure that they are not overly burdensome and that they conform
to legal and technical requirements. Draft forms are generally posted to
IRS*s external website so that external stakeholders and taxpayers may
review and
comment on them. The Office of Management and Budget (OMB) is responsible
for approving each form once every 3 years. The purpose of OMB*s approval
is to assess Background
Page 5 GAO- 03- 486 Testing Forms and Instructions IRS*s compliance with
the Paperwork Reduction Act, which, among other things, requires agencies
to assess the extent of burden the information
they collect imposes on the public. Under the Paperwork Reduction Act, OMB
must approve new forms and major revisions to existing ones. After OMB*s
approval, the forms and instructions are sent to IRS*s vendors to be
printed. Generally, IRS needs to have approved forms ready for printing by
early October to ensure that they can be printed and distributed to the
public when the tax filing season starts the following January.
Figure 1: Simplified Overview of IRS*s Annual Tax Forms Development
Process
Note: GAO analysis of IRS data.
Page 6 GAO- 03- 486 Testing Forms and Instructions As shown in table 1,
IRS used taxpayers or IRS employees to test the clarity of five individual
income tax forms and instructions from July 1997
through June 2002. 5 IRS relied primarily on focus groups to do the
testing. Contractors using private citizens did three of the tests (i. e.,
Earned Income instructions, Child Tax Credit instructions, and Schedule D)
and IRS did the other two using IRS employees.
Table 1: Forms and Instructions IRS Tested Using Individual Taxpayers
Between July 1997 and June 2002
Forms and instructions Year tested Number of
participants Testing method
Earned Income Credit schedule 1998 50 5 focus groups Earned Income Credit
instructions 1999 104 10 focus groups Child Tax Credit instructions 1999
104 10 focus groups
Schedule D Form- Capital Gains and Losses 2001 48
6 focus groups 8 one- on- one
interviews Innocent Spouse Application form 2002 52 6 focus groups Source:
IRS.
Note: Forms and instructions may also refer to schedules and worksheets.
The two testing methods that IRS used* focus groups and one- on- one
interviews* are among the commonly used methods to obtain data from
individuals on whether documents such as forms and instructions are clear
and understandable. Focus groups generally consist of a small number of
participants* about 8 to 12 persons*- and are usually selected and
organized around the focus group topic. Focus groups are a form of group
interviewing that relies on interaction within the group to obtain the
impressions of a group of people but not necessarily the impressions of
each participant. One- on- one interviews, which aim to obtain individual
attitudes, beliefs, and feelings, are used to probe individuals about
specific
difficulties they may have with completing a form or reading instructions.
In some instances, these methods may be used in combination depending upon
the particular circumstances of the test. As noted in table 1, IRS used
5 Because our review was limited to forms and instructions used by
individual taxpayers, table 1 includes only tests IRS did on these
documents during this period. IRS conducted some other tests during this
period. For example, IRS used focus groups to test 5
publications. Of the 5 publications, 3 related to individual taxpayers, 1
to business taxpayers, and one related to both individual and business
taxpayers. IRS estimates that it will revise over 90 publications during
2003. IRS Tested Five Tax
Forms and Instructions Over 5 Years
Page 7 GAO- 03- 486 Testing Forms and Instructions both focus groups and
one- on- one interviews in testing the Schedule D form.
Testing written documents such as tax forms and instructions helps ensure
they are clear, thereby benefiting taxpayers and IRS. Researchers from
three federal agencies and a private research firm said testing leads to
clearer documents as well as more accurate responses. Our guidance on
developing and using questionnaires also recommends testing prior to
distribution. 6 IRS*s experience indicates that testing likely improves
the clarity of tax forms and instructions and may therefore help reduce
the number of errors taxpayers make. Although limited data were available
on the costs and benefits of testing IRS forms and instructions, recent
changes to Earned Income Credit (EIC) and Child Tax Credit forms
illustrate the potential for benefits to significantly exceed the costs of
testing.
Researchers from the National Center for Health Statistics (NCHS), the
Census Bureau, and the Bureau of Labor Statistics (BLS) said that testing
helps ensure that their documents are clear, and therefore users are more
likely to understand them and complete them correctly. Consequently,
researchers from the agencies said they routinely test written documents,
such as forms or surveys, prior to public distribution. A representative
of a private research firm also said that testing ensures that documents
communicate clearly and that his firm and many others perform such testing
for a wide variety of private and public clients.
The researchers from the federal agencies told us that the benefits of
testing are difficult to quantify and, in some instances, may not be
quantifiable. NCHS officials stated that the experience they have gained
over time from testing and revising documents has helped them develop
clearer forms from the outset, a benefit that may not be quantifiable. A
Census researcher also agreed that testing is beneficial but difficult to
quantify. Nevertheless, the Census researcher said that testing documents
prior to public use helps ensure that they are clear and understandable,
which gives the agency a greater chance of receiving accurate responses
and which lessens the need for follow- up interviews. Similarly, a BLS 6
U. S. General Accounting Office, Developing and Using Questionnaires, GAO/
PEMD10.1.7 (Washington, D. C.: October 1993). Testing Written
Documents Helps Ensure Clarity and Benefits Taxpayers and IRS
Experiences of Three Federal Agencies, a Private Research Firm, and IRS
Support Use of Testing
Page 8 GAO- 03- 486 Testing Forms and Instructions researcher noted that,
though difficult to quantify, testing benefits the agency by reducing
errors made by respondents and those reductions can
result in savings of time, money, and effort for BLS. See appendix II for
additional information on the use of testing in these three agencies.
The representative of the private research firm also provided some
perspective of its experiences in conducting tests for public and private
sector customers. This firm, like many similar companies, provides a
variety of testing and data collection services to public and private
sector customers. The firm arranges and conducts focus groups, one- on-
one
interviews, and other tests in order to ensure the clarity of forms,
instruction manuals, surveys, and Web sites among other things. Focus
groups generally involve 12 participants and cost around $6,700 to $7,600,
excluding costs to develop the item to be tested but including incentive
pay for participants that could range from $25 to $50 per participant.
These costs may be higher, the representative said, if the participants
come from special groups. For example, if the participants are medical
doctors, incentive pay could be as high as $250 per participant.
The researcher also said that private firms vary in how much they spend to
test their documents. He estimated that firms generally spend between
$300, 000 and $500,000 to ensure that a form or document is clear and will
meet the firm*s needs. In some cases, firms spend more than $500, 000 to
do a series of tests, making revisions between each test, before they
arrive at a final version of a form or document. He also added that most
firms do not spend at levels that would allow them to test all their forms
and
documents. While maintaining that testing is beneficial, researchers also
stated that testing is not fail- safe. It can help identify particular
parts of a form that are not clear, researchers said, but it cannot ensure
that subsequent changes to the form will entirely resolve the clarity
issue. In addition, testing may identify problems participants have in
completing written documents, but the participants* problems may be
related to other issues, such as poor math skills, rather than confusing
or unclear documents.
Testing questionnaires before distribution is also recommended as a
quality assurance measure in our guidance on developing and using
questionnaires. According to our guidance, testing questionnaires before
they are used is one of the best ways to ensure that the document actually
communicates what it was intended to communicate and that users will
uniformly interpret it. Testing increases the likelihood that respondents
will provide the information needed and helps to alleviate inaccurate
Page 9 GAO- 03- 486 Testing Forms and Instructions responses. Our guidance
is also consistent with professional literature on survey design.
According to professional literature, *reducing measurement error through
better question design is one of the least
costly ways to improve survey estimates. For any survey, it is important
to attend to careful question design and pretesting.* 7 IRS*s recent
limited experience with testing indicates that testing may help
ensure the clarity of tax forms and instructions. In 1999, IRS revised the
forms and instructions related to the EIC and the Child Tax Credit, tested
the revised forms and instructions, and revised them again based on the
test results. The following year, when the revised and tested forms and
instructions were used by taxpayers, the error rates for the EIC and the
Child Tax Credit decreased by 28 and 35 percent, respectively, as shown in
table 2. 8 Table 2: Percentage Decrease in the Number of EIC and Child Tax
Credit Errors
before and after Revising and Testing Forms/ Instructions
Number of errors on tax returns
filed in 1999 (before testing)
Number of errors on tax returns
filed in 2000 (after testing)
Percentage decrease in errors from
1999 to 2000
Earned Income Credit 1,797,162 1, 296,095 28 Child Tax Credit 1,430,394
934,289 35 Source: IRS data.
IRS officials told us they attribute part of the decrease in EIC errors to
a new approach officials developed for structuring EIC forms and
instructions and part to the improvements in the draft documents that
resulted from testing the revised forms and instructions. Before the EIC
forms and instructions were revised, the instructions included a
definition and example describing a qualifying child that taxpayers had to
interpret. 9 7 Floyd J Fowler, 2002; *Survey Research Methods*; Third
Edition; Sage Publications,
Applied Social Research Methods Series, volume 1 (2002). Also see R.
Tourangeau, L. J. Rips, and K. Rasinski, The Psychology of Survey Response
(2000).
8 As noted in table 1, IRS tested five forms during July 1997 through June
2002. IRS did not determine the effect of testing on error rates after
testing the EIC Schedule in 1998 according to IRS Research officials. IRS
tested Schedule D in 2001. Any change in error
rates would be based on the 2002 filing season, and IRS has not yet
analyzed 2002 figures. IRS tested the Innocent Spouse form in 2002 and any
change in error rates will not occur until the 2003 filing season.
9 Most taxpayers eligible for the EIC have a qualifying child.
Page 10 GAO- 03- 486 Testing Forms and Instructions Incorrectly claimed
qualifying children have been a major source of EIC errors. IRS revised
the instructions so that taxpayers would answer a
series of *yes/ no* questions to determine if they have a qualifying child
instead of relying on their interpretations of the definition or example
of a qualifying child. They then tested the old format and the new format.
The number of errors decreased substantially when taxpayers used the new
format. IRS then made some final changes to clarify the instructions based
on test results. In IRS officials* opinions, the *yes/ no* format made it
clearer for taxpayers to determine if they had a qualifying child. The
benefits of testing some changes to IRS*s forms and instructions can
considerably exceed IRS*s costs to do tests, especially because so many
taxpayers can be affected by improvements in clarity that may result from
testing. IRS*s contract costs, including travel, for testing changes to
the
EIC and Child Tax Credit forms and instructions were about $56,000 and
these costs may have been offset within IRS alone in the year that the
change was implemented. More significantly, if testing changes to forms
and instructions for these credits led to a 1- minute reduction, on
average, in the time taxpayers needed to understand and complete the forms
during the 2000 tax filing season, affected taxpayers would have saved
240,000 hours worth $1.2 million valued at minimum wage.
Testing has the potential to yield a wide range of benefits to taxpayers
and IRS. Table 3 summarizes some of the potential benefits that could
result if testing helps clarify tax forms and instructions. If the form or
instruction that has been revised and tested remains unchanged, some
potential benefits could recur annually for the life of the form or
instruction. Benefits of Testing Forms
and Instructions Can Far Exceed IRS*s Costs
Taxpayers and IRS Can Realize a Wide Range of Benefits and Costs From
Testing
Page 11 GAO- 03- 486 Testing Forms and Instructions Table 3: Potential
Benefits if Testing Helps Ensure the Clarity of Tax Forms and Instructions
IRS*s benefits Taxpayers* benefits
Fewer errors that need to be corrected before processing returns
Reduction in audits due to clarity- related taxpayer errors and
redirection of audit resources to other noncompliant taxpayers
Reduction in demand for taxpayer assistance at local IRS offices may
allow IRS to provide better service to other taxpayers
Reduction in demand for assistance via IRS*s toll- free telephone
service related to clarity issues may enable IRS to answer some calls that
would otherwise have not been answered
Experience and knowledge gained through testing may lead to creation of
clearer forms and instructions in the future
Increased clarity of IRS*s forms and instructions reduces
-time to complete tax forms -expenses associated with obtaining
assistance in completing tax forms
Reduced burden from not having to deal with IRS notices, such as reduced
time to
-open and read IRS*s notices -decide what to do -research tax records
-prepare response to IRS -copy and mail response, if
necessary, to IRS -call IRS for assistance
Source: GAO. From the taxpayer*s perspective, benefits from testing could
include avoiding the burdens associated with (1) interacting with IRS if
they make mistakes due to unclear forms and instructions and (2)
understanding and complying with unclear forms and instructions. From
IRS*s perspective, benefits are generally in the form of opportunities to
use its resources better serving other taxpayers and enforcing the tax
laws. 10 Because IRS makes many changes to forms and instructions
affecting
individual taxpayers every year, ranging from very simple to more complex
changes, the benefits can vary according to the type of changes made. Some
forms or instructions may change simply to update certain dollar
thresholds based on inflation and these changes may be unlikely to be
confusing or unclear to taxpayers. However, in other cases changes
may introduce new requirements or concepts to taxpayers, such as when 10
Generally, because IRS has more demand for assistance from taxpayers than
it can handle and identifies more potentially noncompliant tax returns
than it can address, benefits from testing are unlikely to include
reductions in IRS*s overall budget.
Page 12 GAO- 03- 486 Testing Forms and Instructions new rules are
established through legislation or regulation. Changes intended to address
situations like these may be more likely to be
confusing or unclear to taxpayers, which could result in a burden on
taxpayers to understand their obligations in preparing their tax forms
and, possibly, to errors that lead to subsequent interactions with IRS to
correct their returns.
To the extent that a form or instruction is unclear and the lack of
clarity leads to taxpayer errors, the method IRS uses to detect the errors
can affect the costs IRS incurs as a consequence. If a taxpayer*s error
can be detected by IRS and corrected under its *math error* procedures,
which rely extensively on automated processes, the cost to IRS to correct
the error is likely to be small. 11 On the other hand, if unclear forms or
instructions lead to compliance errors that are detected and addressed
through audits conducted through the mail, in IRS offices, or in the
taxpayer*s location, the costs to IRS are likely to be higher in part
because these processes are more labor intensive.
The burden and costs taxpayers might avoid if testing helps clarify forms
and instructions, and thereby helps taxpayers avoid errors, can vary
substantially just as IRS*s costs can vary. In general, because taxpayers
need only respond if they disagree with an IRS notice stating that it has
corrected an error under its math error procedures, the taxpayer*s burden
and cost are likely to be lower than if IRS contacts the taxpayer as part
of
an audit since audits require taxpayer responses and reviews of taxpayers*
books and records. 12 Illustrations we developed of the potential benefits
and costs of testing
forms and instructions show that at least in some cases benefits can be
substantially greater than the costs to IRS to do tests. The benefits of
testing to IRS alone can potentially exceed its testing costs in the first
year a change is implemented. But, primarily because a small change in the
time required of taxpayers to understand their tax obligations can total
to 11 In the math error program, IRS uses computer comparisons and
calculations to correct clerical and mathematical errors on tax returns.
These corrections involve addition and subtraction errors; incorrect
social security numbers, filing status and exemptions; and
missing schedules or forms. 12 Although in many cases taxpayers would not
need to respond to a math error notice, in some cases, such as when IRS
has recalculated EIC or other benefits due to its detection of an
apparently invalid social security number, the taxpayer may be able to
provide IRS additional information to substantiate the original return.
Testing Benefits Can Far
Exceed IRS*s Costs
Page 13 GAO- 03- 486 Testing Forms and Instructions a large aggregate
benefit, taking taxpayers* benefits into account can yield total benefits
substantially above IRS*s costs.
IRS officials have not attempted to develop quantitative estimates of the
benefits to taxpayers and IRS that may result from testing forms and
instructions and the costs IRS incurs to achieve those benefits. IRS
officials did believe that because taxpayers made fewer errors when using
the revised EIC and the Child Tax Credit forms and instructions as shown
in table 2, IRS spent less time and money correcting errors related to
them. 13 The officials said they could not quantify the cost savings
because IRS does not track error correction costs by type of error.
To provide some perspective on the potential magnitude of benefits and
costs that may be realized due to testing changes to forms and
instructions, we analyzed the changes IRS made to EIC and Child Tax Credit
forms and instructions. Our analyses are illustrations and not actual
assessments of benefits and costs that were associated with testing these
forms and instructions because complete data were not available on the
potential benefits and costs. Further, in constructing our illustrations
we sought to be conservative in estimating benefits, in part because we
did
not have information on the full range of costs IRS incurred to test forms
and instructions. Our illustrations focus on (1) a narrow set of benefits
to IRS alone due to potential reductions in taxpayer errors, (2) those
benefits plus certain benefits to the taxpayers from reduced errors, and
(3)
potential benefits to taxpayers in reduced time to do their taxes. See
appendix I for details on the methodology we used in developing our
illustrations.
Our first illustration quantifies a narrow set of benefits to IRS alone
from testing EIC and Child Tax Credit forms and instructions* that is, the
benefits IRS may have realized due to reduced numbers of errors that are
handled under its math error procedures. It is likely that to the extent
testing contributed to better taxpayer understanding of these two credits,
13 Error correction costs relate to the error resolution process. When a
taxpayer makes an error, in some cases, IRS will correct the error. For
example, if a taxpayer transposes information from the Schedule D to an
incorrect line on the form 1040, IRS can correct this mistake for the
taxpayer. IRS cannot correct certain errors made by taxpayers such as
claiming EIC for a child whose social security number on the tax return
does not match
Social Security Administration records. In this case, because it cannot
determine the correct social security number for the child, IRS will
disallow the claim and send an error notice to the taxpayer that explains
what IRS did and what the taxpayer can do if he/ she disagrees with IRS*s
actions.
Page 14 GAO- 03- 486 Testing Forms and Instructions IRS would have
obtained other benefits. For instance, because improperly claimed
qualifying children is one of the leading causes of the EIC*s high
noncompliance rate, 14 if clarified EIC forms and instructions lead fewer
taxpayers to improperly claim the EIC, IRS would likely be able to free
some of its EIC- related audit resources for other audits or to audit EIC
returns that it might otherwise have had insufficient resources to cover.
In fiscal year 2002, IRS used about 1,400 full- time equivalent (FTE)
staff
years 15 for correspondence audits of EIC issues. Our analysis in table 4
shows the amount by which IRS*s potential cost savings from not having to
correct EIC and Child Tax Credit errors may have exceeded its testing
costs given differing assumptions about how much testing alone may have
contributed to reduced taxpayer errors. As illustrated, IRS would have
saved more in cost avoidance (thereby freeing resources to work elsewhere)
in the first year of the change alone than it spent on the contracted
testing of the forms and instructions if half of the reduction in errors
was due to testing. 16 If only 10 or 25 percent of the reduction was due
to testing, then IRS would not have saved more than it spent on the
testing contract in the first year. However, some of the benefits of a
change in forms or instructions continue to be realized in future years.
Again, the illustration does not consider other benefits IRS may have
realized.
14 Department of the Treasury, Internal Revenue Service: Compliance
Estimates for Earned Income Tax Credit Claimed on 1999 Returns
(Washington, D. C.: Feb. 28, 2002). 15 An FTE consists of one or more
employees who collectively work 1 year. For example, one full- time
employee or two half- time employees equal one FTE. 16 Because IRS
officials attributed reduced errors both to their new approach to
structuring EIC guidance and to improvements to the new approach that
resulted from testing, we chose to illustrate reductions due to testing
alone that accounted for half or less of the decline in errors.
Page 15 GAO- 03- 486 Testing Forms and Instructions Table 4: Illustration
of Potential Savings to IRS Alone in 2000 from Testing EIC and Child Tax
Credit Forms and Instructions Form and
instruction Differing assumed
levels of errors avoided Assumed number
of errors eliminated due to
testing Estimated direct
labor cost to correct an error a
Estimated cost to correct eliminated
errors Potential IRS cost
savings assuming $56,000 cost to perform test b
50% 250,534 $0.36 $90,000 $34,000 25% 125,267 0.36 45,000 (11,000) EIC
10% 50,107 0.36 18,000 (38,000) 50% 248,053 0.36 89,000 33,000 25% 124,026
0.36 45,000 (11,000) Child tax credit 10% 49,611 0.36 18,000 (38,000)
Source: IRS and GAO. Note: GAO analysis of IRS data. a IRS officials
provided (1) the number of FTEs, including labor- related overhead costs
such as training and leave, for operating the Error Resolution System
(ERS) and (2) the number of errors corrected by ERS during fiscal year
2002. According to IRS officials, ERS personnel are hired at GS4 and the
journeyman level is GS- 6. Using the number of ERS FTEs, the number of
errors corrected by ERS staff, and the GS- 5, step 1 salary effective
January 2002, as the salary for ERS staff
members, we estimated that IRS incurred $0.36 in costs per error. (IRS
officials were unable to provide the actual costs for ERS staff. We chose
to use GS- 5, step 1, because beginning ERS staff are GS- 4s and more
experienced ERS staff are GS- 6s.) Total error correction costs may be
higher because the $0.36 estimate does not include various other costs
such as nonlabor- related overhead for equipment and supplies. b IRS
contracted with a firm to conduct focus groups to test EIC and Child Tax
Credit forms and
instructions. The contract was for $54,000 and IRS spent an additional
$2,000 on related travel costs. Because one contractor conducted the tests
for both credits, IRS officials said it would be difficult to allocate the
$56,000 between the tests for the two credits.
To provide some perspective on how the potential benefits to taxpayers
from testing EIC and Child Tax Credit forms could affect the overall
benefits and costs of testing, we next looked at potential reduced burden
from credit claimants receiving fewer notices due to reduced errors.
First, we assumed that on average all taxpayers receiving an error notice
from IRS take 2 or 5 minutes to deal with the notice. Based on those
assumptions, we calculated the value to taxpayers of the time saved (using
minimum wage levels) from not having to deal with IRS error notices. 17 We
used the same assumed reductions in errors due to testing that we made
17 We used minimum wage to value taxpayers* time to be conservative in
estimating potential benefits because EIC is targeted to low- income
taxpayers. Minimum wage is a conservative estimate of taxpayers* time even
for EIC in that some taxpayers can qualify for EIC at incomes that are
three times minimum wage. For purposes of the Paperwork Reduction Act, OMB
values time for dealing with paperwork requirements like tax forms at $30
per hour.
Page 16 GAO- 03- 486 Testing Forms and Instructions for table 4 and we
netted taxpayers* savings with the savings shown in table 4 for IRS alone.
As table 5 shows, including testing- related benefits to taxpayers from
decreased errors suggests that in the first year following testing of EIC
and Child Tax Credit forms and instructions, the net benefit to taxpayers
and IRS combined could have been positive except for our lowest assumption
about the degree to which testing may have reduced taxpayer errors* our 10
percent assumption.
Table 5: Illustration of Potential Savings to Taxpayers and IRS in 2000
from Error Reductions Alone from Testing EIC and Child Tax Credit Forms
and Instructions
Potential net savings to taxpayers and IRS from
error reductions Form and instruction
Assumed number of errors eliminated due to
testing Value to
taxpayers in time saved (2 minutes
per notice) Value to
taxpayers in time saved (5 minutes
per notice) Potential IRS cost
savings assuming $56,000 cost to perform test 2 minutes 5 minutes
250,534 $43,000 $108,000 $34,000 $77,000 $142,000 125,267 22,000 54,000
(11,000) 11,000 43,000 EIC
50,107 9,000 22,000 (38,000 (29,000) (16,000) 248,053 43,000 106,000
33,000 76,000 139,000 124,026 21,000 53,000 (11,000 10,000 42,000 Child
tax credit 49,611 9,000 21,000 (38,000) (29,000) (17,000)
Source: GAO. Note: GAO analysis.
Finally, to illustrate the potential benefits if testing EIC and Child Tax
Credit forms and instructions made them clearer and thereby reduced
taxpayers* time needed to understand and complete the credit forms, we
calculated the value of time saved by taxpayers (using minimum wage
levels) in understanding and completing EIC and Child Tax Credit forms
assuming the time saved was 1 minute. Unlike for tables 5 and 6, all
taxpayers who used the form or instructions to determine whether they
qualified for either credit may have saved time if testing contributed to
clearer EIC and Child Tax Credit forms and instructions. However, because
we did not know how many taxpayers might have used the forms and
instructions for this purpose, in calculating the value of time taxpayers
may have saved we used only the number of taxpayers who claimed these
credits and did not use paid preparers to prepare their tax returns.
Page 17 GAO- 03- 486 Testing Forms and Instructions Table 6 shows that if
testing the credits* forms and instructions helped clarify them and that
led taxpayers to take 1 minute less to understand and
complete the forms, credit claimants would have saved a total of about
240,000 hours worth $1.2 million at minimum wage levels.
Table 6: Illustration of Potential Benefit to Taxpayers in 2000 if Testing
Reduced the Time Needed to Understand and Complete EIC and Child Tax
Credit Forms
Form and Instruction
Number of returns claiming the credit
(millions) a Total hours saved
assuming 1 minute less per taxpayer
Potential value to taxpayers of time
saved (millions)
EIC 6.2 100,000 $0.5 Child Tax Credit 8.4 140,000 0.7
Totals 14.6 240,000 1.2
Source: GAO. Note: GAO analysis. a This is the number of tax returns
prepared by taxpayers claiming the credits. Since taxpayers who used a
preparer might not need to read and understand the forms and instructions,
we did not include them in this table. Both the number of EIC and Child
Tax Credit returns are based on the percentage of returns prepared by
taxpayers who claimed EIC since data on the percentage of returns prepared
by taxpayers who claimed the Child Tax Credit were not available.
To provide another perspective on the potential magnitude of benefits and
costs associated with testing changes to forms and instructions, we also
looked at IRS*s experience with the rate reduction credit. This one- time
credit was enacted in June 2001. 18 When 2001 tax returns were processed
during 2002, over 8 million returns had errors related to the credit. 19
IRS did not test the instructions for computing the rate reduction credit
that was included on the Form 1040 for tax year 2001. According to IRS
officials, they did not test the instructions because the credit was a
onetime event, and in their judgment, they had insufficient time to test
it. We reported that some of the taxpayers* errors were probably due to
18 Taxpayers who were eligible to receive an advance tax refund in 2001
but who (1) did not receive a check because IRS did not have their current
addresses or (2) did not have enough taxable income in 2000 to qualify for
the maximum amount allowable, may have been entitled to a rate reduction
credit when filing their tax year 2001 returns. In addition, taxpayers who
were not eligible for an advance tax refund, such as those who did not
have taxable income in 2000, may have been entitled to a rate reduction
credit provided they had taxable income in 2001.
19 This total only includes rate reduction credit errors corrected by IRS
under its math error authority as of September 27, 2002. Some
electronically filed tax returns that IRS rejected during screening for
processing also had rate reduction credit errors.
Page 18 GAO- 03- 486 Testing Forms and Instructions taxpayers not
understanding IRS*s instructions on how to compute the credit. We also
reported that the demand for telephone assistance related
to the credit was significant during the 2002 filing season, and that some
of these calls, based on Taxpayer Advocate Service information, were made
because taxpayers did not understand how to compute the credit. 20 Using
the same approach to illustrate whether IRS alone may have realized
benefits in excess of its testing costs as we did for EIC and Child Tax
Credit changes, we developed the illustration shown in table 7. 21 As
shown, considering only IRS*s cost and assuming that all errors were
corrected by IRS using its math error procedures and assuming IRS would
have spent
the same amount to test the rate reduction credit instructions as it did
for EIC and Child Tax Credit tests, IRS may have been able to save between
$233,000 and $666,000. Although this case is somewhat atypical since the
rate reduction credit affected essentially all individual taxpayers and
the number of errors related to the credit was unusually high, these
figures illustrate that the potential for savings to IRS alone from
testing instructions at times can substantially exceed its testing costs.
Table 7: Illustration of Potential Cost Savings to IRS in 2002 if the Rate
Reduction Credit Instructions Had Been Tested and Errors Were Reduced
Number of returns with rate reduction credit errors Differing assumed
levels of errors avoided Assumed number
of errors eliminated by
testing Estimated direct
labor cost to correct an error
Estimated cost to correct eliminated
errors Potential IRS cost
savings assuming $56,000 cost to perform test
25% 2,006,646 $0.36 $722,000 $666,000 15% 1,203,878 0.36 433,000 377,000
8,025,851
10% 802,585 0.36 289,000 233,000 Source: IRS and GAO. Note: GAO analysis
of IRS data. However, just as with EIC and Child Tax Credits, taxpayers
would have benefited if testing had been done, led to clearer instructions
and
consequently led to fewer taxpayer errors for the rate reduction credit.
Using the same approach we used for EIC and Child Tax Credits, table 8
20 GAO- 02- 827. 21 Consistent with our effort to be conservative in
estimating testing benefits, we assumed that testing would have reduced
errors by no more than 25 percent because many taxpayers only needed to
review a sentence or two of the instructions and did not need to do any
calculations.
Page 19 GAO- 03- 486 Testing Forms and Instructions shows the potential
taxpayers* savings from dealing with fewer rate reduction credit error
notices and the net savings to taxpayers and IRS.
Table 8: Illustration of Potential Savings to Taxpayers and IRS in 2002
from Potential Error Reductions if Testing the Rate Reduction Credit Had
Reduced Taxpayer Errors
Potential net savings to taxpayers and IRS from error reductions
Number of returns with rate reduction credit errors Assumed
number of errors eliminated by
testing Value to
taxpayers of time saved (2
minutes per notice)
Value to taxpayers of time saved (5
minutes per notice)
Potential IRS cost savings
assuming $56,000 cost to perform test 2 minutes 5 minutes
8,025,851 2,006,646 $344, 000 $861,000 $666,000 $1,010,000 $1,527,000
1,203,878 207,000 517,000 377,000 584,000 894,000 802,585 138,000 344,000
233,000 371,000 577,000 Source: GAO. Note: GAO analysis.
Table 9 shows that if testing had been done and it improved the clarity of
the instructions enough to save taxpayers, on average, 30 seconds in
understanding whether and how they needed to complete the credit line on
their tax returns, the savings would have been larger than savings to IRS
and taxpayers from avoided errors alone. 22 Table 9: Illustration of
Potential Benefit to Taxpayers in 2002 if Testing Reduced the
Time Needed to Understand and Complete the Rate Reduction Credit Number of
returns affected by the credit a Total hours saved
assuming 30 seconds less per taxpayer
Potential value to taxpayers of time saved
(millions)
50,000,000 415,000 $2.1 Source: GAO. Note: GAO analysis.
a This is the number of paper returns prepared by taxpayers. Since
taxpayers who used a preparer might not need to read and understand the
instructions, we did include them in this table. We only included the
number of paper returns because information was not readily available on
the number of returns filed electronically that were completed by
taxpayers rather than a preparer.
22 We assumed an average taxpayer time savings of 30 seconds because many
taxpayers only needed to review a sentence or two of the instructions and
did not need to do any calculations.
Page 20 GAO- 03- 486 Testing Forms and Instructions Although IRS officials
said that making greater use of testing to improve clarity of forms and
instructions could be beneficial, officials have not addressed the two
constraints* time and resources* that they state limit their ability to do
more testing of changes to forms and instructions. Time
constraints are not binding for some changes IRS considers to forms and
instructions, although IRS cannot realistically test the unknown portion
of the changes that are due to laws passed shortly before, or even after,
the effective dates for the forms. Also, IRS*s procedures for developing
and revising forms (1) do not clearly specify which draft version of forms
and instructions should be tested with taxpayers or (2) when in the annual
forms development cycle testing should occur. In addition to tight time
frames, officials also say that limited resources, such as only one person
responsible for coordinating all testing efforts in the Forms and
Publications Division, preclude them from increasing tests of forms and
instructions. However, IRS has not documented which changes to forms and
instructions likely would benefit from testing or demonstrated the
benefits that are gained when testing is done. IRS*s planning and
budgeting process uses such information in determining the level of
resources to be allocated to various units.
IRS officials told us that when new tax laws are enacted during the year
that require IRS to create or revise tax forms and instructions in time to
distribute them to taxpayers by January 1, the start of the tax- filing
season, they lack time to test the forms and instructions before
distributing them to taxpayers. However, not all changes to forms and
instructions are time constrained and IRS*s procedures lack a clear target
for which version of
forms and instructions should be tested with taxpayers. While sufficient
data were not available to determine the portion of changes IRS makes to
forms and instructions that cannot be tested due to time constraints, not
all changes are time constrained. Due to the
variability in the time that may be required to test a form or instruction
and in the amount of time IRS needs to develop the initial form or
instruction to be tested, we cannot say definitively when IRS may or may
not have sufficient time to conduct tests. In some cases, IRS likely could
have sufficient time to do testing when it identifies a needed change to
forms or instructions itself since it largely controls the scheduling of
this work. Similarly, when the Congress passes a law that is not effective
until a future tax year, or that contains provisions that are not
effective until a
future tax year, IRS may have sufficient time to conduct tests. For
example, the Economic Growth and Tax Reconciliation Act of 2001 was passed
on June 7, 2001, with some provisions effective for tax year 2001, IRS Has
Not
Addressed Constraints to Increased Testing
Tight Time Frames Do Not Always Exist and IRS*s Procedures Do Not Always
Facilitate Testing
Page 21 GAO- 03- 486 Testing Forms and Instructions but others with later
effective dates. The provisions modifying education Individual Retirement
Accounts were effective for taxable years beginning
after December 31, 2001. This gave IRS approximately 16 months to develop
and test any modifications to tax forms and instructions and make final
revisions before those forms and instructions needed to go to printing for
distribution by January 2003. When a law affects the current tax year, i.
e., changes how taxpayers will need to calculate their taxes in the next
tax- filing season, IRS is less likely to have sufficient time to test.
Even in such a case, however, the new law may be passed early enough to
allow testing.
IRS*s current procedures for developing and revising forms and
instructions do not clearly specify which draft version of forms and
instructions should be tested with taxpayers or when in the annual forms
development cycle testing should occur. Officials said that draft forms
may be tested with taxpayers either before or after they are posted to
IRS*s website for external comments by the public, tax practitioners,
software developers, and others. Tax Forms and Publications Division
officials said that they consider the particular circumstances surrounding
the development of each form and instruction when deciding which version
of a draft form or instruction they should test. However, because IRS does
not have a clear targeted time for testing, IRS*s ability to plan and
conduct tests maybe constrained. If IRS*s procedures defined a point in
the annual forms development cycle where a version of a draft form or
instruction would be available for testing, IRS would be able to establish
processes and deadlines designed to ensure that the opportunity for
testing is realized.
To the extent that a draft version of a form or instruction is available
for testing early in the process, it would give IRS a fuller range of
options for testing. For example, if IRS tested draft versions of forms
and instructions before or during the approximately 3- week period that
the form is available on its Web site, this would minimize any additional
calendar time that testing might otherwise add to IRS*s forms development
process.
Figure 2 shows the points in IRS*s annual forms development process where
testing can occur. As illustrated, testing may be conducted early in the
process and late in the process.
Page 22 GAO- 03- 486 Testing Forms and Instructions Figure 2: Simplified
Overview of Where Testing Can Occur in IRS*s Annual Tax Forms Development
Process
Note: GAO analysis of IRS data. Testing earlier drafts of forms and
instructions would also enable officials to select from various testing
alternatives depending on how early a draft is available for testing. We
did not find a uniform amount of time needed to test a change to a form or
instruction. At the low end of the spectrum, an official from NCHS said
that it takes about 7 weeks to test that agency*s questionnaires using
one- on- one interviews. IRS officials estimated that when IRS employees
are used as focus group participants it requires about 8 to 12 weeks to
schedule and conduct the tests, analyze the data, and prepare a report
summarizing the results. IRS officials estimated that when they contract
with a private firm to conduct focus groups using private citizens, 24 to
32 weeks are required to obtain a contract, recruit participants, conduct
the tests, analyze the results, and prepare a report. This time frame is
based on using regular contracting processes involving developing a
statement of work, soliciting bids, and selecting a contractor. Contract
options exist that enable agencies to identify a firm or group of firms
qualified to undertake work so that an expedited task order procedure can
be used to select a firm for when needs arise. According to
Page 23 GAO- 03- 486 Testing Forms and Instructions IRS officials, they
recently entered into a multiyear contract with two vendors that will
enable them to issue task orders when work is needed. Although IRS*s Tax
Forms and Publications Division officials believe
current resources are insufficient to support more testing of forms and
instructions, they do not have some of the information needed to determine
whether to allocate additional resources. This information is not
available at least in part because division guidelines and policies do not
require that it be gathered.
Officials said that because they have so few staff available to conduct
tests and have a limited budget to contract for testing, they could not
increase the number of tests they perform. According to the officials,
currently only 1 of 103 persons in the division is trained in testing
methods. In addition to other duties, this person coordinates the tests
for the division, such as the
test of EIC forms and instructions completed in 1999 by a private vendor
and the test of the innocent spouse application form completed by IRS in
2002. Officials also told us some staff who are primarily responsible for
creating and revising tax documents may occasionally assist in conducting
tests, such as the three persons involved in testing the innocent spouse
form. Officials also said the total budget for contract support for the
division was $150,000 in fiscal year 2002, $185,000 in fiscal year 2001,
and $130, 000 in fiscal year 2000.
As part of its annual planning and budgeting process, IRS management
determines what resources will be needed to accomplish strategies and
implement programs. IRS*s planning and budget guidance requires that each
operating unit prepare a business plan that, among other things, clearly
defines priorities and resource requirements. Requests in the business
plan for resources must be substantiated with evidence that allocating
additional resources is justified.
However, the division does not systematically identify when testing would
be beneficial and does not routinely demonstrate the benefits to taxpayers
and IRS that have been gained from such testing. Officials do not identify
which of the many changes it makes to forms and instructions each year
would most likely benefit from testing. Thus, the officials cannot tell
IRS management how many opportunities to improve forms and instructions
may be lost due to current resource levels. Further, when tests are
performed, officials do not identify, quantitatively or qualitatively, the
benefits that taxpayers and IRS may have realized. Data Are Not Collected
That Could Be Used by IRS Management to Determine the Proper Allocation of
Resources to Support Test i ng
Page 24 GAO- 03- 486 Testing Forms and Instructions One reason that IRS
does not have data on forgone testing opportunities is that the division
lacks formal, written guidelines and procedures for
determining when testing would be beneficial. Currently, testing is an
optional step in the process for developing forms. IRS*s Tax Forms and
Publications officials said that they decide which forms to test based on
informal guidelines and procedures and input from officials in IRS*s four
operating division program offices and the Taxpayer Advocate Service. The
informal guidelines and procedures call for officials to weigh, among
other things, whether a form or instruction (a) affects a large number of
taxpayers, (b) has a high error rate based on taxpayers* prior use of the
form, (c) is perceived as complex, and (d) will be used for several filing
seasons. Also, according to IRS, the amount of time available to perform
tests is factored into testing decisions.
These informal guidelines do not require officials to consider in all
cases whether testing would be beneficial and to document the decisions
made. Accordingly, even if the informal guidelines are applied, and
officials judge that some forms or instructions could benefit from testing
but cannot be tested due to scarce resources, those decisions are not made
systematically and documented. Further, although the factors the
guidelines suggest taking into account appear to have evolved from
officials* experience and therefore should be useful, they do not consider
some pertinent factors that could affect the benefits likely to be
realized from testing. For instance, the guidelines suggest taking the
number of affected taxpayers into account but not the likely amount of
burden they would face due to unclear forms or instructions. They also do
not clearly call for officials to consider the costs to test forms and
instructions and the benefits that may accrue throughout IRS, such as in
telephone service centers. In addition, these informal guidelines and
procedures automatically exclude testing forms and instructions that will
be used only one time. Also, according to IRS officials, the time frame
between the passage of new tax laws and when the newly created or revised
forms and instructions must be finalized may preclude some forms and
instructions from being tested. Even if one- timeuse forms meet other
testing criteria, such as affecting a large number of taxpayers who may
perceive them as complex, IRS will not consider testing them. As the rate
reduction credit situation discussed earlier illustrates, such automatic
exclusions may not be appropriate in all situations.
IRS officials do not have information on the results achieved when forms
and instructions are tested in part because the division does not have
Page 25 GAO- 03- 486 Testing Forms and Instructions policies that require
such evaluations. When IRS obtained information on the reduction in error
rates following testing of EIC and Child Tax Credit forms and
instructions, the studies did not include collecting other
information on the benefits that may have resulted for taxpayers and for
IRS. For instance, the studies did not estimate the savings IRS may have
realized in its telephone and walk- in service due to increased form
clarity.
Capturing fuller information on the results of testing would be consistent
with IRS*s strategic planning and budgeting process, which emphasizes
assessing the impact of current programs to efficiently allocate
resources. Further, by evaluating results of testing decisions, IRS
officials would be able to determine if their testing guidelines and
procedures lead to good decisions about when testing is most likely to be
beneficial. They may also be able to see if the methods they use to test*
for example, focus groups
formed by IRS employees or one- on- one interviews with individuals* yield
the most effective test results.
IRS continually faces the daunting task of developing and revising tax
forms and instructions to administer our ever- changing set of federal tax
laws. Taxpayers rely on IRS for forms and instructions that are as clear
and easy to understand as possible given the complexity of the tax laws
and providing clear materials is a key goal of IRS*s Tax Forms and
Publications Division.
In attempting to meet this goal, IRS has tested an average of one set of
forms and instructions each year over the last 5 years. In contrast,
officials from three federal agencies that routinely collect information
from the public say that testing documents for clarity before using them
is their standard practice. They do so because they believe testing will
ensure that their data collection documents are clear and that individuals
will
understand them and complete them accurately. Although it is difficult to
gauge how much testing alone contributes to the clarity of tax forms and
instructions, IRS officials believe testing has contributed to significant
declines in taxpayer errors. Illustrations we developed based on IRS*s
experience in testing forms and instructions suggest that IRS can
completely recover its testing costs in the first year following testing
in some circumstances and that when savings to taxpayers from more
understandable forms and instructions are considered, total benefits even
in the first year following tests can be several times IRS*s testing
costs. Conclusions
Page 26 GAO- 03- 486 Testing Forms and Instructions Although they
recognize that testing is beneficial, officials say time constraints and
limited resources preclude more testing. However, IRS*s
procedures do not clearly specify when draft versions of forms and
instructions should be available for testing. Having a clearly defined
point where testing would be performed would facilitate establishing
procedures and deadlines to better ensure that testing could be done even
within IRS*s annual forms update cycle. Further, IRS officials do not have
information that would help IRS management to determine whether to
allocate additional resources to support enhanced testing. Because IRS
lacks standard written procedures for testing, officials have not
documented cases where testing would likely be beneficial and have not
demonstrated the benefits that are gained from testing.
Because testing could potentially yield clearer and more understandable
tax forms and instructions, thereby producing benefits both to taxpayers
and IRS, we recommend that the Acting Commissioner of Internal Revenue
take the following actions.
Develop written criteria for determining which changes to tax forms and
instructions should be tested with taxpayers before publication.
Develop official written guidance that incorporates those criteria and
ensure that the guidance requires staff that develop new or revised forms
and instructions to document which changes would merit testing and why.
Clarify procedures by designating when in the annual forms development
process that a draft version of forms and instructions should be available
for testing with taxpayers.
Ensure that an appropriate range of evaluations are conducted of tests
that are performed to better establish the costs and benefits of
performing tests and to refine IRS*s approach to testing on the basis of
lessons learned.
Use information gained from documenting when changes to forms or
instructions likely would be beneficial and from evaluations of tests to
reassess an appropriate level of resources to perform testing.
The Acting Commissioner of Internal Revenue provided written comments on a
draft of this report in an April 7, 2003, letter, which is reprinted in
appendix III. The Acting Commissioner agreed with our recommendations.
Recommendations for
Executive Action Agency Comments and Our Evaluation
Page 27 GAO- 03- 486 Testing Forms and Instructions We are encouraged that
IRS plans to implement all but one of our recommendations in time for the
2004 forms development cycle.
Understandably, the remaining recommendation to ensure that an appropriate
range of evaluations is conducted of tests would take more time to put
into practice. The Acting Commissioner also provided additional comments
and observations on our draft report.
The Acting Commissioner commented that the crux of our report is that we
do not believe IRS has performed adequate testing on new and revised tax
forms and instructions due to a lack of resources. He said that resources
for testing forms and instructions have been adequate for the testing IRS
wanted to perform. While not questioning whether resources were adequate
for the testing IRS performed, we concluded that IRS officials do not have
information needed to determine the level of resources that should be
allocated to testing forms and instructions. Accordingly, we recommended
that IRS systematically identify opportunities to improve forms and
instructions through testing and to
evaluate the costs and benefits when testing is done. Although agreeing
that testing is beneficial, the Acting Commissioner also said that there
are significant staff costs associated with testing that are not included
in our cost analysis. We recognize that our analysis excluded
staff costs and as stated in our draft report we sought to be conservative
in estimating the benefits of testing, in part because we did not have
information on the full range of costs IRS incurs when undertaking
projects to tests forms and instructions. During the course of our work,
we requested estimates of staff costs for testing but IRS was unable to
provide them. Nevertheless, at least in the cases we illustrated the
potential benefits of testing were so much greater than the costs that
including staff costs likely would not have substantially changed the
results of our
illustrations. The Acting Commissioner expressed concern about whether IRS
could have forms and instructions ready for the filing season if testing
was done late in the forms development cycle as shown in our figure 2
depicting IRS*s process. We agree with the Acting Commissioner*s concern;
however, our figure shows the various points at which testing can occur in
IRS*s current processes based on interviews with IRS officials and the
documentation they provided us. As IRS implements our recommendation to
clarify when testing should be done, selecting a point as early as
possible would help maximize the number of changes that can be tested
during the annual forms update cycle.
Page 28 GAO- 03- 486 Testing Forms and Instructions The Acting
Commissioner also said that he disagreed with our conclusion that IRS*s
experience with obtaining feedback on its products is limited or recent.
He said IRS uses various methods to obtain customer feedback. We agree
that IRS uses methods other than testing to obtain feedback on its forms
and instructions. However, our report describes the potential
benefits and costs of testing as a feedback method. In terms of testing,
IRS has only tested five forms and instructions during July 1997 through
June 2002; in our view, this is a limited number of tests that were
conducted during the recent past.
The Acting Commissioner also disagreed that testing would result in
reduced demand for walk- in and toll- free assistance. He said that IRS
lacks data to support such a conclusion and, based on its experience, new
forms generate requests for assistance and error rates on them tend to be
higher. We recognize that there will always be a demand for taxpayer
customer assistance. However, we believe that reduced demand for
assistance is a potential benefit of testing. We note, for example, that
IRS officials seek input from telephone assistors when deciding which
forms or instructions need to be clarified, apparently believing that
clarifying the forms and instructions may help reduce calls to assistors.
Finally, testing is one means for ensuring that even for new forms and
instructions requests for assistance and errors made by taxpayers will be
minimized.
We are sending copies of this report to the Chairman and Ranking Minority
Member of the House Committee on Ways and Means and its Subcommittee on
Oversight; the Secretary of the Treasury; the Acting Commissioner of
Internal Revenue; the Director of the Office of Management and Budget; and
other interested parties. We will make copies available to others on
request. In addition, the report will be available at no charge on the GAO
Web site at http:// www. gao. gov.
Page 29 GAO- 03- 486 Testing Forms and Instructions This report was
prepared under the direction of Charlie Daniel, Assistant Director. If you
have any questions regarding this report, please contact
him or me at (202) 512- 9110. Key contributors to this report were David
Alexander, Christopher Currie, Ronald La Due Lake, Anne Laffoon, Veronica
Mayhand, Edward Nannenhorn, and Shellee Soliday.
Michael Brostek Director, Tax Issues
Appendix I: Objectives, Scope, and Methodology
Page 30 GAO- 03- 486 Testing Forms and Instructions To determine how often
IRS has used taxpayers to test the clarity of new or revised individual
income tax forms and instructions, we interviewed
officials in its W& I Division*s Tax Forms and Publications Division in
Washington, D. C. We sought to obtain (a) an understanding of the process
IRS used to develop and revise individual income tax forms and
instructions and (b) gather information on the forms and instructions IRS
tested using taxpayers for the 5- year period between July 1997 and June
2002. We requested information for a 5- year span to help ensure that the
information collected would reflect the amount of testing usually done by
IRS. Our work did not include assessing IRS*s processes for developing and
revising notices or publications or assessing the clarity of any specific
tax forms or instructions. To obtain insights on the benefits of testing
written documents, we interviewed officials from three federal agencies in
the metropolitan Washington, D. C., area that perform extensive research
and data collection using private citizens. We contacted NCHS, BLS, and
the Census Bureau because they have broad experience in conducting tests
of the clarity of written documents, such as forms, surveys, and
instructions. We also contacted a private research firm that specializes
in testing forms and
surveys for a variety of public and private sector clients. We also
reviewed our own guidance for developing and using questionnaires. 23 To
determine the benefits to taxpayers and IRS of testing tax forms and
instructions with taxpayers prior to their use by the public, we took
several steps. First, we interviewed IRS officials including Tax Forms and
Publications Division officials to obtain information on whether they
perceived testing as being beneficial to taxpayers and IRS. We also
interviewed IRS*s W& I Research Division officials in Indianapolis to
obtain their views on whether testing tax forms and instructions benefits
taxpayers and IRS. The research division officials, among other things,
collect information on taxpayer errors that the Tax Forms and Publications
Division uses when deciding which forms and instructions to test. These
officials provided us with data on changes in error rates for EIC and
Child Tax Credit forms and instructions before and after IRS revised and
tested them.
Second, we developed illustrations of the potential benefits and costs of
IRS*s testing EIC and Child Tax Credit forms and instructions by analyzing
23 GAO/ PEMD- 10.1.7. Appendix I: Objectives, Scope, and
Methodology
Appendix I: Objectives, Scope, and Methodology
Page 31 GAO- 03- 486 Testing Forms and Instructions the data on the
changes in error rates obtained from IRS research officials. To construct
our illustrations, we used readily available data and made
certain assumptions. Data were not available on many of the potential
benefits of the changes and on the full range of costs IRS incurred to
conduct the tests. We developed similar illustrations of potential
benefits to taxpayers and IRS if the rate reduction credit instructions
had been tested. In all cases, the illustrations we developed are not
actual
assessments of the costs and benefits that were associated with testing
forms and instructions, or that would have resulted if testing had
occurred. In developing the illustrations we sought to be conservative in
estimating benefits, in part because we did not have information on the
full range of costs IRS incurred to test forms and instructions. Because
we had to make various assumptions, our illustrations undoubtedly vary
from actual costs and benefits.
To determine the potential cost savings to IRS of testing these forms and
instructions, we first estimated IRS*s costs to correct a taxpayer error.
Our estimates of IRS*s cost to correct errors were limited to the labor
cost associated with correcting errors in IRS*s math error program. IRS
provided us the number of FTE staff years for operating the ERS that was
used to detect and correct math errors and the number of errors corrected
by ERS during fiscal year 2002. We used this information to calculate an
average labor cost to correct math errors that we then applied to
reductions in EIC and Child Tax Credit errors as well as to error
reductions that might have resulted from testing the rate reduction
credit. Total error correction costs may be higher because we did not
include a number of other costs associated with correcting errors. For
example, we excluded printing and postage costs for the notices sent to
taxpayers. IRS notices often cover more than one issue associated with a
tax return and data were not readily available to determine what portion
of the postage
cost might be attributable to EIC or Child Tax Credit issues alone. In
addition, the estimate does not include costs such as equipment and rent.
We did not specifically test the accuracy of the cost information
provided;
however, our audits of IRS*s annual financial statements have raised
concerns regarding IRS*s ability to identify all costs associated with a
given program or activity. 24 24 U. S. General Accounting Office, Major
Management Challenges and Program Risks:
Department of the Treasury, GAO- 03- 109 (Washington, D. C.: January
2003).
Appendix I: Objectives, Scope, and Methodology
Page 32 GAO- 03- 486 Testing Forms and Instructions Because data were not
available on the extent that testing reduced errors related to EIC and
Child Tax Credit forms and instructions, we assumed different percentage
reduction rates in errors due to testing these forms and instructions. Our
illustrations show potential savings based on
assumed percentage reduction in errors attributable to testing of 50, 25,
and 10 percent. Using our estimate of IRS*s costs to correct an error and
the assumed number of errors eliminated by testing, we arrived at the
costs that would have been incurred to correct those errors. The
difference between costs to correct eliminated errors and the costs to
test is the potential cost savings from testing. Cost savings due to
reduced errors likely would not mean reductions to IRS*s budget. Further,
savings likely would mean that IRS would provide services to other
taxpayers or would pursue other compliance or tax collection activities
that it would otherwise have been unable to do.
To determine IRS*s costs to test forms and instructions, we used IRS*s
actual contract costs for testing EIC and Child Tax Credits in 1999. IRS
spent a total of about $56,000 to test both EIC and Child Tax Credit forms
and instructions with focus groups. Although the cost of contracted
support for testing each form and instruction individually likely would
have been somewhat lower than this, we applied the total cost in each
case. 25 Because IRS could not provide data, we did not include the costs
associated with IRS letting and managing the contract or the cost for Tax
Forms and Publications Division staff to work with the contractor in
conducting and managing the tests or the cost of any other IRS staff that
were involved in these tests.
To determine the potential benefits to taxpayers from testing EIC and
Child Tax Credit forms and instructions, we calculated estimated values to
taxpayers of the time saved if testing improved form and instruction
clarity, thereby reducing taxpayer errors and the burden of dealing with
IRS error notices. We also estimated taxpayers* time saved if testing
reduced the time needed to understand and complete tax forms and
instructions. We developed similar illustrations for potential benefits to
taxpayers if the rate reduction credit had been tested. Our benefit
illustrations were based on a series of assumptions. For example, to
estimate the value to taxpayers of time saved from not having to deal with
25 IRS contracted with a firm to conduct focus groups to test EIC and
Child Tax Credit forms and instructions. The contract was for $54,000 and
IRS spent an additional $2,000 on related travel costs. Because one
contractor conducted both tests, IRS officials said it would be difficult
to allocate the $56,000 between the two credits.
Appendix I: Objectives, Scope, and Methodology
Page 33 GAO- 03- 486 Testing Forms and Instructions an IRS notice, we
applied the minimum wage rate that was in effect after IRS revised and
tested the forms and instructions to estimates of time
taxpayers might save by not having to deal with an IRS notice. 26 For
these illustrations we assumed, on average, that taxpayers who received an
IRS error notice might spend either 2 or 5 minutes to deal with it. Data
were
unavailable on how much time taxpayers actually spend dealing with IRS*s
notices; however, according to Taxpayer Advocate Service information,
IRS*s notices are difficult for taxpayers to understand. Further,
taxpayers who decide to contest an IRS notice may take time to call or
write letters to IRS or to contact and work with a tax preparer. Data were
not readily available to determine what portion of taxpayers who received
an EIC or Child Tax Credit error notice contested IRS*s change to their
tax returns.
For our illustrations of the estimated value of time saved by taxpayers if
testing reduced the time needed to understand and complete EIC and Child
Tax Credit forms and instructions, we assumed that taxpayers would save on
average 1 minute by using clearer forms and instructions. We excluded from
our illustrations those tax returns prepared by preparers because the
taxpayers might not have had to read and understand the forms and
instructions. Because data were not available on the number of tax returns
claiming the Child Tax Credit that were prepared by paid preparers, we
reduced the number of returns claiming the credit by the same percentage
of EIC returns that were prepared by preparers. The percentage of EIC
claimants using paid preparers exceeds the average for all taxpayers. For
the rate reduction credit, our illustration is based on a 30- second time
savings and the number of taxpayers who filed on paper and did not use
preparers. We chose a 30- second potential savings for this illustration
because many taxpayers would have had to read only part of the
instructions to determine what to do. We aggregated the times for all
taxpayers and multiplied the total hours saved by the prevailing minimum
wage rate to arrive at estimated benefits to taxpayers. To determine
whether any factors limited IRS*s ability to use individual taxpayers to
test forms and instructions and, if so, how these factors can
26 Our estimates of the value to taxpayers of time saved from using
clearer forms and instructions are conservative estimates. The minimum
wage rate of $5. 15 per hour that we used in our illustrations is
substantially less than OMB*s estimate of $30 per hour as the average
value of time for individuals and entities to provide information to the
government. Since EIC claimants are lower- income taxpayers, we used the
minimum wage rate. However, some taxpayers would qualify for EIC even at
three times the income of a full time minimum wage level employee.
Appendix I: Objectives, Scope, and Methodology
Page 34 GAO- 03- 486 Testing Forms and Instructions be addressed, we
interviewed IRS*s Tax Forms and Publications Division*s officials and
analyzed supporting data they provided us. Regarding
officials* view that they lacked sufficient time to do more testing, we
reviewed information on the amount of time IRS, NCHS, and the private
research firm we contacted took to perform various types of tests. We also
reviewed IRS*s process for developing new and revised forms and
instructions and determined how many weeks were available between the
dates that various laws were enacted or their provisions became effective
and IRS*s normal October 1st deadline for printing. Finally, regarding
officials* view that they lacked sufficient resources to do more testing,
we obtained information on the resources available within IRS for testing.
We performed our work from May 2002 through March 2003 in accordance with
generally accepted government auditing standards.
Appendix II: Use of Testing in NCHS, Census, and BLS
Page 35 GAO- 03- 486 Testing Forms and Instructions Like IRS, the three
federal agencies we contacted create written documents to be completed by
the public. The agencies create documents such as forms, surveys, and
questionnaires that they use to collect
information from the public to fulfill their missions. Unlike IRS,
described below, these agencies routinely test their forms, surveys, and
questionnaires prior to distribution to the public. The National Center
for Health Statistics, the nation*s principal health statistics agency,
compiles statistical information to guide actions and
policies relevant to public health and health policy. According to NCHS,
obtaining accurate and usable health information is crucial to
successfully fulfilling its mission to provide reliable information to the
Centers for Disease Control and Prevention and the Department of Health
and Human Services. NCHS collects information through various sources
including questionnaires that it develops and administers. Researchers at
NCHS told us that in support of their
research they administer surveys and questionnaires each year in addition
to developing questionnaires used by the Centers for Disease Control and
Prevention. They test each questionnaire using one- on- one interviews.
When documents pertain to a particular rather than a general population,
the researchers recruit participants with characteristics similar to those
persons who might be completing the forms or questionnaires. For example,
researchers recruited asthmatics to test a questionnaire related to
asthma. NCHS tests forms in one- onone settings in which a participant may
be asked to work through a form while a moderator observes and then later
interviews the participant. This approach allows the researcher to
identify specific points at which the forms were confusing or problematic
and learn why the participant had difficulty. NCHS prefers to use one- on-
one interviews when conducting tests because this method closely resembles
the ways in which individuals will be completing the documents since
individuals will likely complete the documents by themselves.
The Census Bureau is the principal agency responsible for collecting and
providing data about the people and the economy of the United States. An
accurate census is important because census results are used to
reapportion seats in the House of Representatives, redraw congressional
districts and other political boundaries, and address countless other
public and private data needs. The Census Bureau collects information
through short- form and long- form questionnaires that it develops, tests,
and administers. In preparation for the 2000 Census, the Congress budgeted
millions of dollars to develop and test questionnaires during the 1990s.
The Census Bureau*s policy requires Appendix II: Use of Testing in NCHS,
Census,
and BLS
Appendix II: Use of Testing in NCHS, Census, and BLS
Page 36 GAO- 03- 486 Testing Forms and Instructions that demographic
survey questionnaires be tested. It has used focus groups and one- on- one
interviews to test its questionnaires and forms.
For example, in fiscal year 1996, the Census Bureau decided to make
fundamental changes to the traditional census design such as shortening
census questionnaires. In that year, it budgeted funds to test, among
other things, respondents* understanding of race and ethnicity questions.
The Census Bureau has also conducted detailed cost- benefit analyses of
alternative designs; in 1992 it tested the simplified questionnaire in
order to gauge whether the new form would increase response rates and
reduce costly follow up with households that did not respond to the
census. 27 The Bureau of Labor Statistics, the principal fact- finding
agency for the
federal government in the broad field of labor economics and statistics,
also depends on clear and understandable written documents to collect
accurate information from the public. According to BLS*s policy, testing
documents such as forms and surveys prior to use by the general public
should be undertaken to help identify factors that may impede users*
ability to understand forms or surveys. Then these factors can be
addressed in order to improve the clarity of written documents and
increase the accuracy of responses. Testing should be done in the early
stages of document development so that any problems with clarity can be
identified early. BLS routinely tests its written documents using focus
groups and one- on- one interviews, and uses the results of the tests to
make improvements to the documents.
27 U. S. General Accounting Office, Census Reform: Questionnaire Test
Shows Simplification Holds Promise, GAO/ T- GGD- 92- 59 (Washington, D.
C.: July 1, 1992) and U. S. General Accounting Office, Decennial Census:
1990 Results Show Need for Fundamental Reform, GAO/ GGD- 92- 94
(Washington, D. C.: June 9, 1992) address recommendations that the Census
Bureau simplify its questionnaire in order to improve its response rate
and thereby reduce the number of follow- ups with nonrespondents and costs
associated with follow- up activities.
Appendix III: Comments from the Internal Revenue Service Page 37 GAO- 03-
486 Testing Forms and Instructions Appendix III: Comments from the
Internal Revenue Service
Appendix III: Comments from the Internal Revenue Service Page 38 GAO- 03-
486 Testing Forms and Instructions
Appendix III: Comments from the Internal Revenue Service Page 39 GAO- 03-
486 Testing Forms and Instructions
Appendix III: Comments from the Internal Revenue Service Page 40 GAO- 03-
486 Testing Forms and Instructions
Appendix III: Comments from the Internal Revenue Service Page 41 GAO- 03-
486 Testing Forms and Instructions (440129)
The General Accounting Office, the audit, evaluation and investigative arm
of Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability of
the federal government for the American people. GAO examines the use of
public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO*s commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents at no cost
is through the Internet. GAO*s Web site (www. gao. gov) contains abstracts
and fulltext files of current reports and testimony and an expanding
archive of older products. The Web site features a search engine to help
you locate documents using key words and phrases. You can print these
documents in their entirety, including charts and other graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as *Today*s Reports,* on its
Web site daily. The list contains links to the full- text document files.
To have GAO e- mail
this list to you every afternoon, go to www. gao. gov and select
*Subscribe to daily E- mail alert for newly released products* under the
GAO Reports heading.
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to: U. S. General Accounting Office 441 G Street NW, Room LM
Washington, D. C. 20548 To order by Phone: Voice: (202) 512- 6000
TDD: (202) 512- 2537 Fax: (202) 512- 6061
Contact: Web site: www. gao. gov/ fraudnet/ fraudnet. htm E- mail:
fraudnet@ gao. gov Automated answering system: (800) 424- 5454 or (202)
512- 7470 Jeff Nelligan, managing director, NelliganJ@ gao. gov (202) 512-
4800
U. S. General Accounting Office, 441 G Street NW, Room 7149 Washington, D.
C. 20548 GAO*s Mission Obtaining Copies of
GAO Reports and Testimony
Order by Mail or Phone To Report Fraud, Waste, and Abuse in Federal
Programs Public Affairs
*** End of document. ***