Military Education: DOD Needs to Enhance Performance Goals and
Measures to Improve Oversight of Military Academies (10-SEP-03,
GAO-03-1000).
Graduates of the service academies operated by the Army, Navy,
and Air Force currently make up approximately 18 percent of the
officer corps for the nation's armed services. The academies
represent the military's most expensive source of new officers.
The Department of Defense (DOD) pays the full cost of a student's
4-year education at the academies; and the related cost has
increased over the past 4 years. Admission to the academies is
highly competitive. The academies use a "whole person" method to
make admission decisions. Recent studies by the Air Force raised
questions about possible adverse effects of whole person
admissions policies on student quality. GAO was asked to review
all three service academies and specifically address the extent
to which (1) DOD oversees the service academies, (2) applicants
are granted waivers of academic standards, and (3) various groups
of students differ in admissions scores and academy performance.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-03-1000
ACCNO: A08371
TITLE: Military Education: DOD Needs to Enhance Performance
Goals and Measures to Improve Oversight of Military Academies
DATE: 09/10/2003
SUBJECT: Academic achievement
Comparative analysis
Education or training
Eligibility criteria
Evaluation criteria
Federal service academies
Higher education
Military training
Performance measures
Waivers
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-03-1000
Report to the Subcommittee on Defense, Committee on Appropriations, House
of Representatives
United States General Accounting Office
GAO
September 2003 MILITARY EDUCATION
DOD Needs to Enhance Performance Goals and Measures to Improve Oversight
of Military Academies
GAO- 03- 1000
The Office of the Under Secretary of Defense for Personnel and Readiness
(OUSD/ P& R), the services, and the academies* boards of visitors conduct
considerable oversight of the academies* operations and performance, but
they lack a complete oversight framework. A complete oversight framework
includes performance goals and measures against which the academies*
performance could be better assessed. OUSD/ P& R and the services use the
number and type of commissioned officers as the primary measure of
academy performance. OUSD/ P& R requires and receives reports on academy
performance from the services. While data submitted in these reports
provide perspective on current performance compared with past performance,
without stated performance goals and measures, these reports do not offer
OUSD/ P& R or the services as good an insight into the academies
performance as they could. Additionally, though the academy boards of
visitors serve as an external oversight mechanism to focus attention on a
wide range of issues, they also do not assess the academies* performance
against established performance goals and measures.
The academies do not grant waivers from academic criteria or have absolute
minimum scores for admission. However, under the whole person approach,
the academies can admit some applicants whose academic scores are lower
than might normally be competitive for admission, but who in their
totality (academics, physical aptitude, and leadership) are evaluated by
academy officials as being capable of succeeding at the academy. In our
review of the academy classes that started in 1998 (class of 2002), we
found that despite differences among various groups of students in their
admissions scores and similar differences in their performance while at
the academies, the differences in performance were not sizable. Some
groups, such as females, performed better in some categories than the
class as a whole and worse in others. Some groups (minorities, preparatory
school graduates, recruited athletes, and students in the lower 30 percent
of their class in terms of academic admissions scores) performed at lower
levels on average in all categories than the class as a whole.
Academy Operating Costs and Cost Per Graduate, Fiscal Years 1999- 2002
Academy Cost category FY 1999 FY 2000 FY 2001 FY 2002
Total operating costs $301,058,452 $330,603,820 $336,416,716 $364,971,975
Military Academy Cost per graduate 312,150 320,120 339,318 349,327
Total operating costs 245,749,679 253,817,467 273,809,865 292,696,358
Naval Academy Cost per graduate 254,983 256,931 266,033 275,001
Total operating costs 277,639,005 314,972,559 321,335,152 333,056,023 Air
Force Academy Cost per graduate 305,945 305,133 313,456 322,750
Source: DOD.
Graduates of the service academies operated by the Army, Navy, and Air
Force currently make up approximately 18 percent of the officer corps for
the nation*s armed services. The academies represent the military*s most
expensive
source of new officers. The Department of Defense (DOD) pays the full cost
of a student*s 4- year
education at the academies; and the related cost has increased over the
past 4 years. Admission to the
academies is highly competitive. The academies use a *whole person* method
to make admission decisions. Recent studies by the Air Force raised
questions about possible adverse effects of whole
person admissions policies on student quality. GAO was asked to review all
three service academies
and specifically address the extent to which (1) DOD oversees the service
academies, (2) applicants are granted waivers of academic standards, and
(3) various groups
of students differ in admissions scores and academy performance.
GAO recommends that the Secretary of Defense direct the Office of the
Under Secretary of Defense for Personnel and
Readiness (OUSD/ P& R), in concert with the services, to further enhance
performance goals and
measures to improve oversight of the operations and performance of the
service academies. In comments on a draft of this report, DOD concurred
with GAO*s recommendation.
www. gao. gov/ cgi- bin/ getrpt? GAO- 03- 1000. To view the full product,
including the scope and methodology, click on the link above. For more
information, contact Derek Stewart at (202) 512- 5559 or stewartd@ gao.
gov. Highlights of GAO- 03- 1000, a report to the
Subcommittee on Defense, Committee on Appropriations, House of
Representatives September 2003
MILITARY EDUCATION
DOD Needs to Enhance Performance Goals and Measures to Improve Oversight
of Military Academies
Page i GAO- 03- 1000 Military Education Letter 1 Results in Brief 3
Background 6 DOD Lacks a Complete Framework for Oversight of the Academies
15 Whole Person Approach Allows Academies Flexibility to Admit
Students with a Range of Qualifications 19 No Significant Differences in
Admissions and Academy Performance between Various Groups of Students 21
Conclusion 26 Recommendation for Executive Action 26 Agency Comments 26
Appendix I Scope and Methodology 28
Appendix II Results of Statistical Analysis of Class of 2002 Admissions
and Academy Performance Scores 31
Admissions and Performance Scores 31 Relationships between Admissions and
Performance Scores 34 Appendix III Comments from the Department of Defense
37
Appendix IV GAO Contact and Staff Acknowledgments 39
Related GAO Products 40
Tables
Table 1: Civilian and Military Faculty at the Service Academies 7 Table 2:
Academy Operating Costs and Cost Per Graduate, Fiscal Years 1999- 2002. 8
Table 3: Average Admissions Scores for the Selected Groups in the Class
That Started in 1998 at the U. S. Military Academy 22 Table 4: Average
Admissions Scores for the Selected Groups in the Class That Started in
1998 at the U. S. Naval Academy 22 Contents
Page ii GAO- 03- 1000 Military Education
Table 5: Average Admissions Scores for the Selected Groups in the Class
That Started in 1998 at the U. S. Air Force Academy 22 Table 6: Percentage
of the Selected Groups Making Up the Lower
30 percent of the Classes in Terms of Their Academic Admissions Scores, by
Academy 23 Table 7: Student Performance for Selected Groups at the U. S.
Military Academy 24 Table 8: Student Performance for Selected Groups at
the U. S. Naval Academy 24 Table 9: Student Performance for Selected
Groups at the U. S. Air Force Academy 25 Table 10: Admissions and Academy
Performance Scores for the Class of 2002 31 Table 11: Number of Students
Graduating and Graduation Rates for the Class of 2002 31 Table 12:
Admissions and Performance Scores for the Class of 2002 at the U. S.
Military Academy 32 Table 13: Admissions and Performance Scores for the
Class of 2002 at the U. S. Naval Academy 33 Table 14: Admissions and
Performance Scores for the Class of 2002 at the U. S. Air Force Academy 33
Table 15: Regression Coefficients (Standardized Coefficients) from Linear
Regression Models Testing Correlations between
Academic and Whole Person Admissions Scores with Cumulative GPA,
Cumulative MPA, and Order of Merit for the Class of 2002 at the Service
Academies 35 Table 16: Regression Coefficients (Standardized Coefficients)
from Logistic Regression Models Testing Correlations Between
Academic and Whole Person Admissions Scores and the Likelihood of
Graduation for the Class of 2002 at the Service Academies 36 Figures
Figure 1: Basic Steps in the Academy Admissions Process 9 Figure 2: Areas
and Their Weights Considered in the U. S. Military Academy*s Whole Person
Admissions Process 11 Figure 3: Areas and Their Weights Considered in the
U. S. Naval Academy*s Whole Person Admissions Process 12 Figure 4: Areas
and Their Weights Considered in the U. S. Air Force Academy*s Whole Person
Admissions Process 13 Figure 5: Categories of Academy Nominations 14
Page iii GAO- 03- 1000 Military Education Abbreviations
ACT American College Testing DOD Department of Defense GPA grade point
average MPA military performance average OUSD/ P& R Office of the Under
Secretary of Defense for Personnel
and Readiness ROTC Reserve Officer Training Corps SAT Scholastic Aptitude
Test This is a work of the U. S. government and is not subject to
copyright protection in the
United States. It may be reproduced and distributed in its entirety
without further permission from GAO. However, because this work may
contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
Page 1 GAO- 03- 1000 Military Education
September 10, 2003 The Honorable Jerry Lewis Chairman The Honorable John
P. Murtha Ranking Minority Member Subcommittee on Defense Committee on
Appropriations House of Representatives
Graduates of the service academies operated by the Army, Navy, and Air
Force make up approximately 18 percent of the officer corps for the
nation*s armed services. 1 The academies represent the most expensive
source of new officers, compared with other sources for officers, such as
Reserve Officer Training Corps (ROTC) programs at colleges and
universities or officer candidate/ training schools for individuals who
already have college degrees. The Department of Defense (DOD) pays the
full cost of providing the 4- year programs of academic education,
military training, physical conditioning, and pay for each student. 2 In
fiscal year 2002, DOD reported costs per graduate for the U. S. Military
Academy, the U. S. Naval Academy, and the U. S. Air Force Academy were
approximately $349, 000, $275,000, and $333, 000, respectively. These
costs have increased over the past 4 years. To ensure the best value for
the investment in the
academies, effective management principles are critical. Such principles
include a complete oversight framework, with clear roles and
responsibilities, as well as performance goals and measures against which
to objectively assess performance.
With each academy accepting about 1,200 of its more than 10,000 applicants
a year, admission to the academies is highly competitive. Applicants must
be selected or obtain a nomination, such as from a senator,
representative, the President, or the Vice President, based on the
1 The Marine Corps does not have its own academy. The Naval Academy
graduates both Navy and Marine Corps officers. 2 Students attending the U.
S. Military Academy at West Point, New York, and the U. S. Air Force
Academy in Colorado Springs, Colorado, are called *cadets,* while those
attending the U. S. Naval Academy, in Annapolis, Maryland, are called
*midshipmen.* We refer to
cadets and midshipmen collectively as *students.*
United States General Accounting Office Washington, DC 20548
Page 2 GAO- 03- 1000 Military Education
categories established by law. 3 Most nominations are reserved for
Congress, which, therefore, has a central role in admitting students to
the academies. In addition to basic age and medical qualifications, the
academies* admissions process involves an assessment of applicants*
academic achievement (e. g., Scholastic Aptitude Test* SAT* scores and
grade point averages), physical aptitude, and extracurricular activities
(i. e., leadership potential). Academy officials combine these assessments
into a *whole person* admissions score that is used to determine an
applicant*s potential to graduate from an academy and potential fitness as
a commissioned officer. Applicants compete for admission based on these
scores.
Air Force studies have raised questions about possible adverse effects of
whole person admissions policies on student quality. For example, the Air
Force found that its whole person assessments and resulting admissions
scores have led the Air Force Academy to admit an increasing number of
students whose academic qualifications are below academic minimums, as
well as to admit an increasing number of students recruited largely to
participate in varsity intercollegiate athletics.
The House of Representatives report on defense appropriations for fiscal
year 2003 4 directed that we perform reviews of all three service
academies and their respective preparatory schools. 5 As part of the
review of the service academies, we were also directed to obtain student
and faculty perceptions of various aspects of student life at the
academies. Issues associated with the academy preparatory schools and the
results of surveys on aspects of student life are addressed in separate
reports. 6 3 10 U. S. C. S:S: 4342, 6954, and 9342.
4 H. R. Rept. 107- 532, at 14- 15 (2002). 5 The academy preparatory
schools exist to prepare selected students who are not ready academically
to attend one of the academies. 6 U. S. General Accounting Office,
Military Education: DOD Needs to Align Academy Preparatory Schools*
Mission Statements with Overall Guidance and Establish Performance Goals,
GAO- 03- 1017 (Washington, D. C.: September 2003); and Military Education:
Student and Faculty Perceptions of Student Life at the Military Academies,
GAO- 03- 1001 (Washington, D. C.: September 2003).
Page 3 GAO- 03- 1000 Military Education
As agreed with your offices, this report addresses the following
questions, to what extent
(1) does DOD oversee the academies* operations and performance? (2) are
applicants granted waivers from academic criteria for admissions? (3) do
various groups of students differ in admissions scores and academy
performance? In addition to reviewing documents and interviewing officials
at all three academies, the service headquarters, the Office of the Under
Secretary of Defense for Personnel and Readiness (OUSD/ P& R), and the
academies* boards of visitors, we reviewed admissions policies and
procedures and observed their use by academy officials in evaluating
applications being considered for the incoming class of 2007. We also
obtained and analyzed admissions and performance data for the student
class that graduated in 2002. To compare student admissions qualifications
and performance at
the academies, we identified six major groups of students common to all
academies: females, minorities, academy preparatory school graduates,
recruited athletes, prior enlisted personnel, and students whose academic
admission scores fell in the lower 30 percent of the entering class. 7
Data on student performance included academic grade point average;
military performance average, which is similar to a performance evaluation
for commissioned officers; and class rank. 8 It also included graduation
rate. Other issues, such as recent controversies associated with alleged
sexual assault, did not fall within the scope of this review. Further
details on our scope and methodology are in appendix I. We conducted our
work from October 2002 through May 2003 in accordance with generally
accepted
government auditing standards. The Office of the Under Secretary of
Defense for Personnel and Readiness (OUSD/ P& R), the services, and the
academies* boards of visitors conduct considerable oversight of the
academies* operations and performance, but they lack a complete oversight
framework. In 1991, our report concluded that better oversight of the
academies was needed and made recommendations to improve DOD oversight.
Since then, DOD has taken
7 Each group may contain members of the other groups. 8 Class rank is
referred to as *order of merit* by the academies. Results in Brief
Page 4 GAO- 03- 1000 Military Education
measures to address these issues, including establishing guidance on
oversight of the academies and uniform academy cost reporting. However,
DOD has not established a complete oversight framework, which would
include not only clear roles and responsibilities, but also performance
goals and measures against which to objectively assess performance. OUSD/
P& R, the services, and the academies* boards of visitors have different
oversight roles, but largely conduct oversight activities without the
benefit of formalized performance goals and measures. OUSD/ P& R and the
services use the number and types of commissioned officers as the primary
measure of academy performance. OUSD/ P& R requires and receives reports
on academy performance from the services. While data submitted in these
reports provide perspective on such
performance measures as graduation rates, admissions trends for women and
minorities, and information on the quality of admitted students, without
stated performance goals and measures, these data do not offer OUSD/ P& R
or the services as good an insight into the academies* performance as they
could. For example, the data collected by the academies show that the
graduation rates have increased in the last 10 years; however, there is no
stated goal for graduation rate against which to
judge whether this rate of increase is adequate. Other data collected by
the academies indicate that the percentage of females and minorities has
fluctuated over the last 3 years, but apart from admissions targets used
by the Military Academy, there are no stated goals against which to
measure the adequacy of these admissions trends. Additionally, academy
officials regularly analyze data on student performance to determine the
extent to which admissions standards can be changed to improve overall
student
performance at the academies. However, there are no stated goals for
student body performance, apart from minimum graduation standards such as
the cumulative academic grade point average, that might help the academies
and other oversight bodies assess overall student performance.
Additionally, each academy*s board of visitors* an external oversight
mechanism* focuses attention and actions on a wide range of operational
and quality of life issues at the academies. However, the boards do not
evaluate academy performance against established performance goals and
measures. Without formal goals and measures that are, moreover, linked to
mission statements, oversight bodies do not have sufficient focus for
their efforts and cannot systematically assess an organization*s strengths
and weaknesses nor identify appropriate remedies that would help them
achieve the best value for the nation*s investment in the academies.
The academies do not grant waivers from academic criteria or have absolute
minimum scores for admission. Under the whole person
Page 5 GAO- 03- 1000 Military Education
approach, the academies can admit some applicants whose academic scores
are lower than might normally be competitive for admission, but who in
their totality (academics, physical aptitude, and leadership) are
evaluated by academy officials as being qualified and capable of
succeeding at the academy. The only admissions criteria with an absolute
minimum score for qualifying for admissions is physical aptitude. The
academic and leadership criteria have a range of qualifying scores based
on what general levels of ability are considered competitive during the
admissions process. If an applicant*s score is lower than the competitive
range in academics, then admission officials have some flexibility in (1)
further considering the applicant by re- examining the student*s record
for information that can produce further insight about his or her academic
achievement and (2) weighing the extent to which the leadership component
of the whole person score may offset the low component. It is possible for
students to be admitted whose academic scores were not as competitive as
some of their peers who may not have been admitted. The applicant is
considered a risk and is evaluated through a deliberative process by
academy officials on the basis of their judgment of whether the applicant
is fully qualified and capable of succeeding at that academy. The
subjective nature of this approach is consistent with the intent of the
whole person concept, by which the academies want to admit students who
also demonstrate leadership characteristics that cannot be quantified by
purely objective scoring methods. Academy officials do not consider
this approach to represent an academic waiver, but instead their judicious
assessment of the whole person.
In our review of the academy classes that started in 1998 (class of 2002),
we found differences among various groups of students in their admissions
scores and similar differences in their performance while at the
academies; the differences in performance were not sizable. For the class
data we reviewed, minorities, academy preparatory school graduates,
recruited athletes, and prior enlisted students 9 all had lower average
admissions scores than the average for the class as a whole. Of those
students in the lower 30 percent of the class in terms of academic
admissions scores, about 44 percent were recruited athletes, between 25
and 31 percent were minorities, and between 20 and 34 percent were
preparatory school graduates. Regarding performance, we found differences
at the academies between selected groups (i. e., females, minorities,
preparatory school graduates, recruited athletes, prior enlisted
9 Each of these groups can contain members from other groups.
Page 6 GAO- 03- 1000 Military Education
students, and students in the lower 30 percent of the class in terms of
academic admissions scores) and the class as a whole. Those differences
varied but were generally not sizable. For example, females at one academy
had a lower graduation rate than the class as a whole but a higher average
academic grade point average and a higher average class rank. Some groups
at all academies* such as minorities, preparatory school graduates,
recruited athletes, and students in the lower 30 percent of their class in
terms of academic admissions scores* performed at lower levels on average
in all categories than the class as a whole, but these differences were
not significant. For example, one of the lowest average academic grade
point averages among the groups we reviewed was 2.61, whereas the average
for the class as a whole at that academy was 2.93. A 2.0 grade point
average is required to graduate. The lowest graduation rate for the class
we reviewed was 65 percent for the students in the lower 30
percent of their class in terms of academic admissions scores at one
academy. The average graduation rate for the class as a whole at that
academy was 74 percent. We are making a recommendation to improve DOD*s
oversight of
operations and performance at the academies through the enhancement of
performance goals and measures. In comments on a draft of this report, DOD
concurred with our recommendation.
The Departments of the Army, the Navy, and the Air Force each have their
own educational institutions (academies) to produce a portion of each
branch*s officer corps: 10 U. S. Military Academy (West Point, N. Y.),
established in 1802;
U. S. Naval Academy (Annapolis, Md.), established in 1845; and U. S.
Air Force Academy (Colorado Springs, Colo.), established in 1954.
The academies are structured to provide a curriculum critical to the
development of successful future officers in academic, military, and
physical areas of achievement. Additionally, the academies emphasize the
moral and ethical development of students through their respective honor
codes and concepts.
10 Other sources for commissioned officers include ROTC programs at
colleges and universities and officer candidate/ training schools for
individuals who already have college degrees. Background
Page 7 GAO- 03- 1000 Military Education
There are approximately 4,000 students enrolled at each of the three
service academies at any given time, each comprising four classes. In
December 2002, Congress authorized an annual increase of up to 100
students until the total number reaches 4,400 for each academy. 11 In 2002
the Military Academy graduated 968 students; the Naval Academy 977
students; and the Air Force Academy 894 students. Faculty at the U. S.
Military Academy and the U. S. Air Force Academy are comprised
predominantly of military officers (79 and 75 percent, respectively),
while at the U. S. Naval Academy 59 percent of the faculty are civilians.
Table 1 shows the composition of the faculty at the service academies.
Table 1: Civilian and Military Faculty at the Service Academies Service
academy
Total number of faculty
Total number of civilian faculty
(% of faculty) Total number of
military faculty (% of faculty)
U. S. Military Academy 622 131 (21%) 491 (79%) U. S. Naval Academy 555 326
(59%) 229 (41%) U. S. Air Force Academy 490 123 (25%) 367 (75%)
Total 1,667 580 (35%) 1, 087 (65%)
Source: DOD. Note: Faculty information is based on a snapshot of each
academy in February 2003.
DOD reports that the total cost to operate all three academies in fiscal
year 2002 was $990.7 million. Table 2 shows the reported operating costs
and cost per graduate for each academy from fiscal year 1999 through
fiscal year 2002. We did not independently verify these costs.
11 Pub. L. 107- 314, Bob Stump National Defense Authorization Act for
Fiscal Year 2003, S: 532, December 2, 2002.
Page 8 GAO- 03- 1000 Military Education
Table 2: Academy Operating Costs and Cost Per Graduate, Fiscal Years 1999-
2002 Academy Cost Category Fiscal year 1999 Fiscal year
2000 Fiscal year 2001 Fiscal year
2002
Total operating costs $301,058,452 $330,603,820 $336,416,716 $364,971,975
Military Academy Cost per graduate 312,150 320,120 339,318 349,327 Total
operating costs 245,749,679 253,817,467 273,809,865 292,696,358 Naval
Academy Cost per graduate 254,983 256,931 266,033 275,001 Total operating
costs 277,639,005 314,972,559 321,335,152 333,056,023 Air Force Academy
Cost per graduate 305,945 305,133 313,456 322,750 Source: DOD.
Prospective students must meet basic eligibility requirements for
appointment to an academy. They must (1) be unmarried, (2) be a U. S.
citizen, (3) be at least 17 years of age and must not have passed their
twenty- third birthday on July 1 of the year they enter an academy, (4)
have no dependents, and (5) be of good moral character. 12 After
determining eligibility, a candidate submits an application to a
preferred academy or academies. Each submitted application is required to
include information such as, but not limited to, the candidate*s (1) SAT
scores (or American College Testing* ACT* examination scores); (2) high
school grade point average (and class rank, if possible); (3) physical
aptitude scores; (4) medical examination results; and (5) extracurricular
activities. The academies admit those candidates that have secured a
nomination and who represent, in the opinion of academy officials, the
best mixture of attributes (academic, physical, and leadership) necessary
to ensure success at the academies and as military officers.
The military academies use a *whole person* method to assess potential
candidates in three major areas: (1) academics, (2) physical aptitude, and
(3) leadership potential. Each academy uses the same basic approach.
Admissions assessments are weighted toward academic scores that include
objective tests and high school performance. Leadership potential is
measured by assessing athletic and non- athletic extracurricular
activities. Subjective assessments of potential candidates in these major
areas also contribute to final admissions *scores.* Such assessments
12 10 U. S. C. S:S: 4346, 6958, and 9346; and Department of Defense,
Directive 1322.22, Service Academies, S: 4. 3, August 24, 1994.
Page 9 GAO- 03- 1000 Military Education
include interviews with prospective candidates, teacher/ coach
evaluations, and analyses of writing samples. Though medical criteria
differ between services, the medical examinations are conducted according
to the same standards, under a joint DOD Medical Examination Review Board
that manages the medical examination process and records for applicants to
all academies. 13 Each academy is authorized to permit up to 60 foreign
students to attend
at any given time on a reimbursable basis by their country of origin. 14
This number does not count against the authorized student strength of the
academies. The admission of foreign students is covered by separate
policies and procedures. Our review was limited to the policies and
procedures for admitting U. S. citizens to the academies. Figure 1 shows
the basic steps in the admissions process for all U. S. applicants.
Figure 1: Basic Steps in the Academy Admissions Process
13 See Department of Defense, Directive 5154.25, DOD Medical Examination
Review Board, June 11, 1981; Directive 6130.3, Physical Standards for
Appointment, Enlistment, or Induction, December 15, 2000; and Instruction
6130.4, Criteria and Procedure Requirements for Physical Standards for
Appointment, Enlistment, or Induction in the Armed Forces, December 14,
2000.
14 10 U. S. C. S:S: 4344, 6957, and 9344.
Page 10 GAO- 03- 1000 Military Education
Students who are disenrolled from an academy after the start of their
third year may be required to complete a period of active duty enlisted
service of up to 4 years or may be required to reimburse the federal
government for the cost of their education. Those who are disenrolled in
their first 2 years do not incur an active service or reimbursement
obligation. 15 The United States Military Academy*s admissions evaluation
considers
academics, leadership, and physical aptitude. Academic considerations
include above- average high school or college academic records as well as
strong performance on SAT/ ACT. Additionally, the Military Academy
considers recommendations from English, mathematics, and science teachers.
The leadership potential considers demonstrations of leadership and
initiative in sports, school, community, or church activities and strong
recommendations from faculty and community leadership and is a more
subjective assessment of character. Physical aptitude is based on a scored
standardized test. This test is made up of pull- ups for men or the
flexedarm hang for women, push- ups, standing long jump, basketball throw,
and shuttle run. Figure 2 shows the areas considered and the weights
assigned to each area in the U. S. Military Academy*s whole person
admissions process.
15 Department of Defense, Directive 1332.23, Service Academy
Disenrollment, S:S: 6. 1 and 6.2, February 19, 1988. United States
Military
Academy Admissions Process
Page 11 GAO- 03- 1000 Military Education
Figure 2: Areas and Their Weights Considered in the U. S. Military
Academy*s Whole Person Admissions Process The United States Naval
Academy*s admissions evaluation considers
academics, leadership, physical aptitude, and technical interest. Academic
considerations include above- average high school or college academic
records as well as strong performance on SAT/ ACT. Additionally, the Naval
Academy considers recommendations from English and mathematics teachers.
Assessment of leadership potential represents a subjective evaluation of
character in which the academy considers demonstrations of leadership in
terms of extracurricular activities in sports, school, community, or
church and strong recommendations from faculty and community leadership.
Physical aptitude is based on a scored, standardized test consisting of
pull- ups for men or the flexed- arm hang for
women, push- ups, standing long jump, basketball throw, and shuttle run.
Additionally, the Naval Academy considers the technical interest of a
prospective student, which is measured through a questionnaire in the
application packet and used to gauge interest in pursuing a technical
degree. The intent of this requirement is to admit students that are
interested in pursuing technical degrees, specifically nuclear and
maritime engineering. The admissions board can also apply further points
to an applicant*s overall whole person score based on further
consideration of an applicant*s record, including such things as the
results of the evaluation form filled out by the Naval Academy
representative who interviewed the United States Naval
Academy Admissions Process
Page 12 GAO- 03- 1000 Military Education
applicant. Figure 3 shows the areas considered and the weights assigned to
each area in the U. S. Naval Academy*s whole person admissions process.
Figure 3: Areas and Their Weights Considered in the U. S. Naval Academy*s
Whole Person Admissions Process The United States Air Force Academy*s
admissions evaluation considers
academics, leadership, and an assessment by the selections panel. Academic
considerations include above- average high school or college academic
records as well as strong performance on SAT/ ACT. Additionally, the Air
Force Academy considers recommendations from English and mathematics
teachers. Under leadership potential, the academy considers
extracurricular activities in sports, school, community, or church and
strong recommendations from faculty and community leadership. Finally, the
Air Force Academy Selections Panel makes an assessment of all potential
students. This assessment is composed of a pass/ fail score from the
physical aptitude examination and the evaluation of the academy*s liaison
officer evaluation, made after interviewing the applicant. The physical
aptitude examination is made up of pull- ups for men or the flexed- arm
hang for women, push- ups, standing long jump, basketball throw, and
shuttle run. The leadership potential area and the admissions board
include the more subjective assessments of a potential United States Air
Force
Academy Admissions Process
Page 13 GAO- 03- 1000 Military Education
student. Figure 4 shows the areas considered and the weights assigned to
each area in the U. S. Air Force Academy*s whole person admissions
process.
Figure 4: Areas and Their Weights Considered in the U. S. Air Force
Academy*s Whole Person Admissions Process
The President of the United States alone appoints candidates to the
academies. 16 Before receiving an appointment, all candidates must secure
one or more nominations according to the following categories: 17
congressional (including a U. S. senator, representative, delegate, or the
Vice President); service- connected (including, among others, children
of disabled
veterans, enlisted personnel in the active or reserve components, and
students from ROTC programs or other designated honor school graduates);
and
16 10 U. S. C. S:S: 4341a, 6953, and 9341a. 17 10 U. S. C. S:S: 4342,
6954, and 9342. Nomination and
Appointment of Candidates
Page 14 GAO- 03- 1000 Military Education
other (including the academy superintendents* nominees and other
nominees to bring the incoming class to full strength).
Figure 5 shows the approximate distribution of categories of academy
nominations, based on the types and numbers of nominees per category
allowed by law.
Figure 5: Categories of Academy Nominations
Note: Percentages do not add to 100 due to rounding.
Oversight of the academies is the responsibility of three principal
organizations: OUSD/ P& R, the service headquarters, and the board of
visitors of each academy. According to Department of Defense Directive
1322. 22 (Service Academies), 18 OUSD/ P& R serves as the DOD focal point
for matters affecting the academies and has responsibility to assess
academy operations and establish policy and guidance for uniform oversight
and management of the military academies. The military
departments perform the primary DOD oversight function for their
respective academies. The superintendent of each academy reports
18 Department of Defense, Directive 1322.22, Service Academies S: 5.1,
August 24, 1994. DOD is currently revising this directive. Oversight of
the
Academies
Page 15 GAO- 03- 1000 Military Education
directly to the uniformed head of his respective service (the Chiefs of
Staff for the Army and the Air Force and the Chief of Naval Operations for
the Navy), in accordance with the chain of command for each service. Each
academy also has a board of visitors, mandated by law, 19 that is
comprised of congressional members and presidential appointees. These
boards focus attention and action on a wide range of operational and
quality of life issues at the academies.
As educational institutions, the service academies are also overseen by
several nongovernmental organizations that are outside DOD purview. Each
academy undergoes periodic review by a higher- education accreditation
body associated with its region of the country, 20 usually involving a
full review every 10 years with an interim review every 5 years. The
accreditation bodies review such areas as core curriculum, strategic
planning, self- assessments, diversity of faculty and students, and
faculty credentials. The athletic programs of the academies are also
subject to periodic certification by the National Collegiate Athletic
Association. This body reviews academy athletics in terms of such issues
as finances and impact on the education mission of the academies. We
limited our review
of oversight of the academies to DOD organizations and the boards of
visitors.
The OUSD/ P& R, the services, and the academies* boards of visitors
conduct many oversight activities, but they lack a complete oversight
framework. A complete oversight framework includes not only clear roles
and responsibilities, but also performance goals and measures against
which to objectively assess performance. Such elements embody the
principles of effective management in which achievements are tracked in
comparison with plans, goals, and objectives and the differences between
actual performance and planned results are analyzed. Without formal goals
and measures, oversight bodies do not have sufficient focus for their
efforts and cannot systematically assess an organization*s strengths and
weaknesses nor identify appropriate remedies that would permit DOD to
achieve the best value for the investment in the academies. In a prior
19 10 U. S. C. S:S: 4355, 6968, and 9355. 20 Commission on Higher
Education of the Middle States Association of Colleges and Schools
(Military and Naval Academies) and Commission of Institutions of Higher
Education of the North Central Association of Colleges and Schools (Air
Force Academy). DOD Lacks a
Complete Framework for Oversight of the Academies
Page 16 GAO- 03- 1000 Military Education
report, 21 GAO concluded that better external oversight of the academies
was needed to provide useful guidance and suggestions for improvement. The
report recommended that DOD improve oversight of the academies through
such measures as establishing a focal point for monitoring academy issues
in the Office of the Secretary of Defense and establishing guidance on
uniform cost reporting.
OUSD/ P& R and the services have established clear roles and
responsibilities for oversight of the academies, with the former serving
as the focal point for issues affecting all academies and the latter
having direct oversight authority over their respective academies. DOD
established guidance in 1994 for the oversight of the academies 22 and for
uniform reporting of costs and resources. 23 OUSD/ P& R is directly
involved in those policy issues that affect all academies and require DOD-
level attention and legislative matters. For example, the office was
recently the DOD focal point on the issue of increasing authorized
enrollment at the academies from 4,000 to 4,400. With respect to the
academies, the office is chiefly concerned with monitoring the degree to
which the services are
meeting their goals for the accession of new officers. 24 The office also
coordinates major studies that affect the academies, such as a November
1999 report on the career progression of minority and women officers.
The services are responsible for direct oversight of their respective
academies; and the academies are treated similarly to major military
commands. The superintendents of the academies are general/ flag officers
who report directly to the uniformed heads of their services (the Chiefs
of Staff for the Army and the Air Force and the Chief of Naval Operations
for the Navy). In addition to overseeing the academies* budget through the
same approval process as a major command activity, the services oversee
the academies* operations and performance primarily through the academies*
goal of meeting service officer accession targets. The
21 U. S. General Accounting Office, DOD Service Academies: Improved Cost
and Performance Monitoring Needed, GAO/ NSIAD- 91- 79 (Washington, D. C.:
July 16, 1991). 22 Department of Defense, Directive 1322.22, Service
Academies, August 24, 1994. DOD is currently revising this directive. 23
Department of Defense, Instruction 1025. 4, Service Academy Resources
Report, October 18, 1994. 24 The academies are one of the sources for
officers. The others include reserve officer training programs at colleges
and universities, officer candidate/ training schools, and direct
commissioning programs.
Page 17 GAO- 03- 1000 Military Education
superintendents are responsible for meeting those targets and, in so
doing, are given wide discretion in such areas as modifying their specific
admissions objectives and the process for matching graduates with service
assignments. The service headquarters use a number of mechanisms to
oversee academy performance. For example, each service headquarters
provides officer accession targets to the academies so that the assignment
of graduates and the make up of incoming student classes can be modified
as necessary. In addition to general numbers of officers, each service
also has a number of specialty officer fields that need to be filled, and
the services also monitor the extent to which the academies will be able
to meet those accession goals.
The services also directly oversee the academies by requiring the
superintendents to report on and discuss their operations. For example,
the Air Force uses an annual forum of the most senior Air Force officers
to focus on the Air Force Academy with respect to how it is meeting the
needs of the operational Air Force. The Navy uses similar senior officer
conferences and frequent interaction between the superintendent and Navy
headquarters to conduct oversight. The Army uses the U. S. Military
Academy Forum, comprised of senior Army officers, to address academy
operations issues. The superintendents of the three academies also hold
annual meetings to discuss issues common to all academies. These
mechanisms have resulted in such academy actions as curriculum changes to
increase the number of technical degree majors, increasing language
requirements, and increasing the number of students attending the
academies.
While OUSD/ P& R and the services conduct a wide variety of oversight
activity, there are few stated performance goals against which to measure
academy operations and performance. Each of the academies has a strategic
plan that is focused on providing quality military and professional
training and education in order to commission highly capable junior
officers. These plans are approved by the service headquarters but are not
generally used by the services as benchmarks against which to measure
academy performance, and they do not contain specific goals against which
to measure student performance. OUSD/ P& R is required to assess and
monitor academy operations based on the information provided in annual
reports it requires from the service secretaries. 25 These reports provide
data on various aspects of performance, such as student 25 DOD Directive
1322.22 S:S: 5.1. 2 and 6.1.
Page 18 GAO- 03- 1000 Military Education
demographics and trends, student quality, admissions and attrition trends,
compensation for students and faculty, leadership and honor systems, and
incidents of indiscipline. The reports provide OUSD/ P& R and the services
with information on
current and past performance for academy operations, but apart from
officer accession goals, neither OUSD/ P& R nor the services have specific
stated performance goals against which to compare the information provided
in the assessment reports, thus they do not have an explicit basis for
judging the adequacy of their performance. For example, the data collected
by the academies show that graduation rates have increased in the last 10
years; however, there is no stated goal for a graduation rate against
which to judge whether this rate of increase is adequate. Other data
collected by the academies indicate that the percentage of females and
minorities has fluctuated over the last 3 years, but apart from admissions
targets used by the U. S. Military Academy, there are no stated goals
against which to assess these trends. Additionally, academy officials
regularly analyze data on student body performance to determine the
extent to which admissions standards can be changed to affect student body
performance. However, there are no stated goals for student body
performance, apart from minimum graduation standards, that might help the
academies and other oversight bodies assess overall student
performance. The oversight efforts of each academy*s board of visitors are
similarly limited by the absence of sufficient performance goals and
measures. Each of the academies has a board of visitors, mandated by law
26 and comprised
of Members of Congress and presidential appointees, that is outside the
DOD chain of command. The boards have a broad legal mandate to inquire
into all aspects of academy operations. 27 The boards meet several times a
year to be briefed on and discuss academy operations and must conduct an
annual visit to their respective academies. During these visits, the
boards are briefed by academy staff on such issues as admissions,
curriculum, recruiting, athletics, morale and welfare, and construction
programs; they also interview students to obtain their perceptions of life
at the academies. The boards also address inquiries to academy staff,
which are usually followed up at subsequent meetings, and they make
suggestions to improve operations or quality of life at the academies. For
26 10 U. S. C. S:S: 4355, 6968, and 9355. 27 10 U. S. C. S:S: 4355, 6968,
and 9355.
Page 19 GAO- 03- 1000 Military Education
example, boards of visitors have recommended increased recruiting of
qualified minority applicants from various congressional districts and
increased surveying of students on quality of life issues.
The boards submit annual reports to the President on the status of and
issues at the academies but do not evaluate academy operations and
performance against established performance goals. The boards of visitors
do not have dedicated staffs to conduct their work, and though board
members may inquire into any aspect of academy operations, the agenda is
set largely by the briefings presented to the boards by academy officials.
Academy officials with whom we spoke were generally satisfied with the
oversight provided by the boards of visitors, though there were concerns
at the Air Force Academy about poor attendance by board members during
annual visits to the academy.
The academies do not grant waivers from academic criteria but do not have
absolute minimum scores for admission. Under the whole person approach,
the academies can admit some applicants whose academic scores are lower
than might normally be competitive for admission, but who in their
totality (academics, physical aptitude, and leadership potential) are
deemed an acceptable risk and qualified to attend an academy. This
admissions approach is consistent with the intent of the academies to
admit students who also demonstrate leadership and initiative
characteristics, which cannot be quantified by purely objective scoring
methods.
When conducting their admissions processes, the academies do not set
absolute minimum scores for academic ability. Rather, they establish a
range of scores that would be considered competitive, based on past
incoming class performance and academy research on the overall quality of
the applicant pool. Prior to 2002, the Air Force Academy set absolute
minimum academic scores, and a waiver was required to further consider an
applicant who fell below that minimum, no matter how high his or her
scores in the leadership area. However, the Air Force Academy no longer
has absolute minimums and uses the same competitive range approach as the
other academies. Under this approach, if an applicant*s academic score is
lower than the competitive range guidelines, academy officials have some
flexibility to further consider the applicant. Academy officials will
re- examine the applicant*s record for information that might provide
further insight about his or her academic achievement. For example,
officials may contact high school teachers to inquire about the types and
difficulty of the classes the applicant has been taking and his or her
Whole Person
Approach Allows Academies Flexibility to Admit Students with a Range of
Qualifications
Page 20 GAO- 03- 1000 Military Education
performance in those classes. Academy officials will also weigh the extent
to which the leadership component of the applicant*s whole person score
offset the low component. The applicant is considered a risk and is
evaluated through a deliberative process by academy officials on the basis
of their judgment of whether the applicant is fully qualified and capable
of succeeding at that academy. The subjective nature of this approach is
consistent with the intent of the whole person concept, by which the
academies want to admit students who also demonstrate leadership
characteristics that cannot be quantified by purely objective scoring
methods. Academy officials do not consider these judgments to constitute a
waiver of academic standards, but rather a judicious assessment of the
whole person. The process for assessing those applicants whose academic
scores are lower than might normally be competitive is nonetheless similar
to the former Air Force Academy process for granting waivers.
With over 10, 000 applicants 28 for each academy each year and about 1,200
students admitted, the academic standards are high. Academy data show that
the academic quality of the applicants has remained high over the past 4
years, and the competitive ranges for academic scores used by the
academies have remained the same or have increased during this time.
However, it is possible for students to be admitted whose academic scores
were not as competitive as some other applicants who may not have been
admitted. Senators, representatives, and delegates may submit up to 10
nominees for each student vacancy available to him or her per academy.
They may choose to designate one as a principal nominee. 29 If an
applicant receives a principal nomination and is in all other respects
qualified, the
academies must admit that applicant, even over an applicant on the same
senator*s, delegate*s, or representative*s nomination list with higher
academic and/ or whole person scores. The other nominated names become
alternates for possible admission later in the admissions process.
Though some academies award credit for the extent to which an applicant
surpasses the standards of the physical aptitude examination, there are
minimum standards for the physical test that must be met. None of the
academies uses a system of *waivers,* except for medical conditions. An
applicant can be waived for a medical condition, based on the deliberation
and judgment of DOD medical personnel and the academy superintendent.
28 This includes the total number of students who applied and not the
number that received a nomination. 29 10 U. S. C. S:S: 4342, 6954, and
9342.
Page 21 GAO- 03- 1000 Military Education
For example, an applicant who is disqualified due to a vision condition
may apply for and receive a waiver, based on subsequent surgical vision
correction or determination by the academy superintendent that the
applicant would be able to serve on active duty without the vision
condition being a problem.
In our review of the academy classes that started in 1998 (class of 2002),
we found differences among various groups of students in their admissions
scores and similar differences in their performance while at the
academies, but the differences were not significant in magnitude. In terms
of performance after admission to the academies, 30 differences between
these student groups and the class as a whole were also not sizable. We
reviewed data for the following distinct groups: 31 overall class,
females, minorities, academy preparatory school graduates,
recruited athletes, prior enlisted, and lower 30 percent of class by
academic admissions scores.
For the class data we reviewed, minorities, academy preparatory school
graduates, recruited athletes, and prior enlisted students all had lower
average admissions scores than the average for the class as a whole,
though these differences varied. The differences between groups and the
class as a whole were not sizable, generally falling within 5 percent.
Those differences that were statistically significant and outside the 5
percent range were still generally less than 10 percent of the class as a
whole. Tables 3, 4, and 5, show the average admissions scores for the
selected groups in the class that started in 1998 at the Military, Naval,
and Air Force Academies, respectively. Although each academy uses the same
fundamental whole person approach, they use different scales to calculate
scores. Therefore, the academic and whole person scores cannot be compared
across academies.
30 We used the following performance factors to measure student
performance at the academies: cumulative grade point average, cumulative
military performance average, order of merit (class rank), and graduation
rate (for each group of students). 31 Each of these groups can contain
members from other groups. No Significant
Differences in Admissions and Academy Performance between Various Groups
of Students
Page 22 GAO- 03- 1000 Military Education
Table 3: Average Admissions Scores for the Selected Groups in the Class
That Started in 1998 at the U. S. Military Academy Average admissions
score Overall
(1,246) Females (192) Minorities
(269) Preparatory
school graduates
(184) Recruited
athletes (279) Prior enlisted
(31) Lower
30 percent of admissions
class (377)
Academic score 600 603 583 546 b 558 a 594 532 b Whole person score 6,006
6,022 5,865 5,645 a 5,814 5,861 5,609 a Source: GAO analysis, from
Military Academy sources. a Denotes a group average or percentage with a
statistically significant difference greater than 5% from the overall
average or percentage. b Denotes a group average or percentage with a
statistically significant difference greater than 10%
from the overall average or percentage.
Table 4: Average Admissions Scores for the Selected Groups in the Class
That Started in 1998 at the U. S. Naval Academy Average admissions score
Overall
(1,226) Females (190) Minorities
(221) Preparatory
school graduates
(146) Recruited
athletes (380) Prior enlisted
(76) Lower
30 percent of admissions class
(368)
Academic score 618 624 594 545 b 596 570 a 544 b Whole person Score 65,732
65,719 63,769 61,254 a 64,233 62,256 a 61,404 a Source: GAO analysis, from
Naval Academy sources. a Denotes a group average or percentage with a
statistically significant difference greater than 5% from the overall
average or percentage. b Denotes a group average or percentage with a
statistically significant difference greater than 10% from the overall
average or percentage.
Table 5: Average Admissions Scores for the Selected Groups in the Class
That Started in 1998 at the U. S. Air Force Academy Average admissions
score Overall
(1,216) Females (190) Minorities
(229) Preparatory
school graduates
(157) Recruited
athletes (312) Prior enlisted
(44) Lower
30 percent of admissions
class (366)
Academic Score 3,202 3,216 3,123 3,112 3,043 3,188 2,863 b Whole person
Score 798 805 782 774 773 792 751 a Source: GAO analysis, from Air Force
Academy sources. a Denotes a group average or percentage with a
statistically significant difference greater than 5% from the overall
average or percentage. b Denotes a group average or percentage with a
statistically significant difference greater than 10% from the overall
average or percentage.
Page 23 GAO- 03- 1000 Military Education
Of those students in the lower 30 percent of the class in terms of
academic admissions scores, about 44 percent were recruited athletes,
between 25 and 31 percent were minorities, and between 20 and 34 percent
were preparatory school graduates. Table 6 shows the percentage of the
selected groups making up the lower 30 percent of the classes in terms of
their academic admissions scores, by academy. Table 6: Percentage of the
Selected Groups Making Up the Lower 30 percent of the Classes in Terms of
Their Academic
Admissions Scores, by Academy
Numbers in percent Academy Females Minorities Preparatory school graduates
Recruited
athletes Prior enlisted
Military Academy 16 31 34 44 3 Naval Academy 13 29 32 45 13 Air Force
Academy 14 25 20 44 5 Source: GAO analysis, from DOD sources. Note:
Numbers do not add to 100 percent because each of the groups can contain
members of
another group.
We also found differences in performance after admission to the academies
between selected groups and the class as a whole. For example, females at
the Naval Academy had a lower graduation rate than the class as a whole,
but they had a higher average academic grade point average (cumulative
GPA) than the class as a whole and higher average class rank (order of
merit). The differences in performance between the selected groups and the
class as a whole were not sizable, generally falling within 5 percent.
Those differences that were statistically significant and outside the 5
percent range were still generally less than 10 percent of the
class as a whole. Tables 7, 8, and 9 show how the selected groups
performed at the Military, Naval, and Air Force Academies, respectively.
See appendix II for further information on comparisons of performance by
defined student groups.
Page 24 GAO- 03- 1000 Military Education
Table 7: Student Performance for Selected Groups at the U. S. Military
Academy Performance score Overall
(1,246) Females (192) Minorities
(269) Preparatory
school graduates
(184) Recruited
athletes (279) Prior enlisted
(31) Lower
30 percent of admissions
class (377)
Average cumulative GPA 2.99 2.99 2.82 2.61 a 2.81 3.14 2.66 a Average
cumulative
MPA 3.28 3.26 3.21 3.26 3.20 3.37 3.21 Average order of merit 3.03 3.04
2.86 a 2.75 a 2.90 3.06 2.78 a Graduation rate 78% 76% 71% a 72% 76% 71%
71% a Source: GAO analysis, from Military Academy sources.
a Denotes a group average or percentage with a statistically significant
difference greater than 5% from the overall average or percentage. b
Denotes a group average or percentage with a statistically significant
difference greater than 10% from the overall average or percentage.
Table 8: Student Performance for Selected Groups at the U. S. Naval
Academy Performance score Overall
(1,226) Females (190) Minorities
(221) Preparatory
school graduates
(146) Recruited
athletes (380) Prior enlisted
(76) Lower
30 percent of admissions
class (368)
Average cumulative GPA 2.97 3.01 2.82 a 2.67 a 2.86 3.02 2.67 a Average
cumulative
MPA 3.12 3.16 3.02 2.99 3.08 3.19 3.00 Average order of merit 489 456 a
590 b 658 b 551 b 453 661 b Graduation rate 80% 71% b 75% 77% 79% 72% 76%
a Source: GAO analysis, from Naval Academy sources.
a Denotes a group average or percentage with a statistically significant
difference greater than 5% from the overall average or percentage. b
Denotes a group average or percentage with a statistically significant
difference greater than 10% from the overall average or percentage.
Page 25 GAO- 03- 1000 Military Education
Table 9: Student Performance for Selected Groups at the U. S. Air Force
Academy Performance score Overall
(1,216) Females (190) Minorities
(229) Preparatory
school graduates
(157) Recruited
athletes (312) Prior enlisted
(44) Lower
30 percent of admissions
class (366)
Average cumulative GPA 2.93 2.97 2.78 a 2.61 b 2.79 2.89 2.64 a Average
cumulative
MPA 2.90 2.93 2.89 2.83 2.81 2.93 2.84 Average order of merit 469 440 545
b 663 b 568 b 499 646 b Graduation rate 74% 75% 71% 69% 71% 66% 65% b
Source: GAO analysis, from Air Force Academy sources.
a Denotes a group average or percentage with a statistically significant
difference greater than 5% from the overall average or percentage. b
Denotes a group average or percentage with a statistically significant
difference greater than 10% from the overall average or percentage.
Some groups* such as minorities, preparatory school graduates, recruited
athletes, and students in the lower 30 percent of their class in terms of
academic admissions scores* performed at lower levels on average in all
categories than the class as a whole, but these differences varied between
academies and by category and were not sizable. For example, one of the
lowest average academic grade point averages for the groups we reviewed
was 2.61 and the average for the class as a whole at that academy was
2.93. A 2.0 grade point average is required to graduate for academic and
military averages. Similarly, the lowest graduation rate for the class we
reviewed was 65 percent for the students in the lower 30 percent of their
class in terms of academic admissions scores at one academy. The average
graduation rate for the class as a whole was 74 percent.
Our analysis of data for the students who entered the academies in 1998
(class of 2002) indicates that admissions scores are generally good
predictors of performance at the academies. Of the admissions scores, the
academic component of the whole person scores was often the best predictor
of academic performance at the academies, and the whole person scores in
their entirety were often the best predictors of military performance at
the academies. Both academic and whole person admissions scores were good
predictors of class rank. In general, whole person admissions scores were
better predictors of graduation rate than the academic admissions scores
alone.
Page 26 GAO- 03- 1000 Military Education
Although the service academies receive oversight from a number of
organizations and have established guidance for that oversight that
includes the reporting of a wide range of data on academy operations,
without clear and agreed- upon performance goals, there is no objective
yardstick against which to fully measure academy performance and
operations, apart from the officer accessions goals currently used.
Establishment of such performance goals is consistent with the principles
of effective management and would enhance the quality of oversight already
performed by OUSD/ P& R, the services, and the academy boards of visitors,
permitting them to more clearly note those areas in which the academies
excel, highlight areas where improvement is warranted, and achieve the
best value for the nation*s investment in the academies.
To improve DOD oversight of the operations and performance of the service
academies, we recommend that the Secretary of Defense direct the OUSD/ P&
R, in concert with the services, to further enhance performance
goals and measures whereby the information required in annual assessment
reports can be better evaluated. These performance goals should be
developed for each academy and, where appropriate, in common for all
academies. The specific goals should coincide with performance elements
agreed upon by the services and OUSD/ P& R and might include such things
as graduation rates, demographic composition of student classes,
assessments of officer performance after graduation, and other performance
information already collected by the academies, including performance
characteristics of various groups of students.
In comments on a draft of this report, DOD agreed with our recommendation
to further enhance performance goals and measures for the service
academies whereby the information required in annual assessment reports
can be better evaluated. DOD further stated that the Office of the Under
Secretary of Defense for Personnel and Readiness OUSD/ P& R will (1)
monitor development of improved goals and measures by the service
academies, to include facilitating the development of common performance
goals where appropriate and (2) update DOD Conclusion
Recommendation for Executive Action
Agency Comments
Page 27 GAO- 03- 1000 Military Education
Directive 1322.22, Service Academies, as required. DOD*s written comments
are included in their entirety in appendix III.
We are sending copies of this report to the appropriate congressional
committees; the Secretaries of Defense, the Army, the Navy, and the Air
Force; and the Director, Office of Management and Budget. We will also
make copies available to others upon request. In addition, the report will
be available at no charge on the GAO Web site at http:// www. gao. gov.
Please contact me on (202) 512- 5559 if you or your staff have any
questions concerning this report. Key contributors are listed in appendix
V. Derek B. Stewart
Director Defense Capabilities and Management
Appendix I: Scope and Methodology Page 28 GAO- 03- 1000 Military Education
To assess the extent to which DOD oversees the service academies*
operations and performance, we interviewed officials at the Office of the
Under Secretary of Defense for Personnel and Readiness; the Army, Navy,
and Air Force headquarters; and the U. S. Military, U. S. Naval, and U. S.
Air Force Academies. We reviewed documents on service and DOD oversight
criteria and structures, reporting mechanisms, academy strategic plans,
academy annual reports on operations and performance, boards of visitors*
minutes and reports, and superintendents conference reports. We also
attended a U. S. Naval Academy Board of Visitors meeting at the Naval
Academy in December 2002 and a U. S. Military Academy Board of Visitors
meeting in Washington, D. C., in March 2003. Additionally, we reviewed
criteria on the principles of effective management, such as those found in
Standards for Internal Control in the Federal Government. 1 To assess the
extent to which academy applicants are granted waivers from academic
admissions criteria, we interviewed officials from the Military, Naval,
and Air Force Academies and reviewed documents on admissions policies,
standards, and practices. We discussed with academy officials their
execution of the whole person approach, including how they assess
applicants* records, the weights applied to the various components of the
whole person score (academic, leadership, and physical aptitude), and the
justification for points given to various aspects of an applicant*s
scores. We also reviewed data from each academy on trends in academic
admissions scores. During site visits to each academy, we observed the
evaluation of applicant packages for the incoming class of 2007 by academy
officials, including how the whole person approach was applied for
admissions scores. We also observed meetings of senior officials at each
academy where applicants* records were evaluated and final admissions
decisions were made.
To assess the extent to which admissions and academy performance scores
differ between various groups of students, we analyzed admissions scores
and academy performance scores for all students who started at the three
academies in 1998 and should have graduated in 2002. This represented the
most recent group of students for which complete data were available. We
requested and received from each academy a database that included data on
both admission scores and information about
students* performance while attending the academy. We did not 1 U. S.
General Accounting Office, Standards for Internal Control in the Federal
Government, GAO/ AIMD- 00- 21. 3.1 (Washington, D. C.: November 1999).
Appendix I: Scope and Methodology
Appendix I: Scope and Methodology Page 29 GAO- 03- 1000 Military Education
independently assess data reliability, but we obtained assurances about
data completeness, accuracy, and reliability from academy officials
responsible for maintaining data at each academy. We analyzed these data
separately for each academy since each academy calculated admission
scores or performance scores somewhat differently. We identified six major
groups of students common to all academies: females, minorities, academy
preparatory school graduates, recruited athletes, prior enlisted
personnel, and students whose academic admission scores fell in the lower
30 percent of the entering class (we chose the latter group in order to
capture information on students whose academic admissions scores may have
been lower than might normally be competitive). Information specifying a
student*s membership in each of these groups was provided in the databases
from the academies. To assess differences, we first
compared the mean performance scores for each group to the overall mean
for each performance measure for the entire class. See appendix II for
details on the results of our analysis of the relationships between
admissions and performance scores. In addition, we assessed the
relationship between admissions scores and
performance at the academies by using the whole person admission score and
the academic component of the admissions score. We estimated the effects
of those scores on four measures of performance for students at the
academies: (1) cumulative grade point average (GPA), (2) cumulative
military performance average (MPA), (3) order of merit (class standing),
and (4) graduation rate. We used cumulative GPA upon graduation as an
indicator of academic performance at the academies and military
performance averages upon graduation as an indicator of military
performance at the academies. Order of merit is a measure of class
standing at each academy that combines academic and military grade
performance and is a final rank for each graduating student. At both the
Air Force Academy and the Naval Academy, order of merit is an actual class
rank number. At the Military Academy, however, order of merit could range
between 0 and 4.0 and was given on the same scale as grade point averages.
For each academy, we analyzed the association of both the
academic component scores and whole person admission scores with each of
the performance scores using regression models. Relationships between the
admissions scores and cumulative GPA, cumulative MPA, and order of merit
were estimated using linear regression models. The relationships between
these two admissions scores and the likelihood of graduating were
estimated using logistic regression models. See appendix II for more
details on the results of those analyses.
Appendix I: Scope and Methodology Page 30 GAO- 03- 1000 Military Education
Issues related to alleged sexual assaults at the academies fell outside
the scope of our objectives. We conducted our work from October 2002
through May 2003 in accordance with generally accepted government auditing
standards.
Appendix II: Results of Statistical Analysis of Class of 2002 Admissions
and Academy Performance Scores
Page 31 GAO- 03- 1000 Military Education
This appendix provides the results of our analyses of both admissions and
performance scores for the class of 2002 at the U. S. Military Academy,
the U. S. Naval Academy, and the U. S. Air Force Academy.
We obtained data from all three service academies that included
information on admissions scores (academic and whole person), performance
scores while at the academy (cumulative academic grade point average,
military performance average, and order of merit), attrition information
where applicable, and various demographic characteristics for all students
entering each academy in 1998. Table 10 shows the minimum, maximum and
average admissions and performance scores for students at each academy.
Table 11 shows graduation rates at each academy.
Table 10: Admissions and Academy Performance Scores for the Class of 2002
Military Academy Naval Academy Air Force Academy Average Min Max Average
Min Max Average Min Max
Academic Admissions score 600 430 791 618 440 788 3,202 2,492 4,005
Whole person score 6,006 4,587 7,188 65,732 51,651 82,250 798 655 931 Cum.
GPA 2.99 1.97 4.19 2.97 2.03 4.00 2.93 2.06 3.97 Cum. MPA 3.28 2.09 3.99
3.12 2.17 3.85 2.90 2.32 3.92 Order of merit 3.03 1.30 3.92 489 1 977 469
1 929 Source: GAO analysis, from DOD sources. Note: For the U. S. Air
Force Academy, an additional step during the selection panel process
results in
a lower whole person score than the component parts.
Table 11: Number of Students Graduating and Graduation Rates for the Class
of 2002 Military Academy Naval Academy Air Force Academy Number Percent
Number Percent Number Percent
Graduation rate 968 78% 977 80% 894 74% Source: GAO analysis, from DOD
sources. Next, we compared the average admissions scores, performance
scores,
and graduation rates of the six student groups to these overall scores and
rates. Tables 12, 13, and 14 show the average admission scores and the
four measures of student performance for the overall sample, and for the
six student groups, for each of the academies. Because we have data for
the population of students in this class and there is no sampling error,
the Appendix II: Results of Statistical Analysis of
Class of 2002 Admissions and Academy Performance Scores
Admissions and Performance Scores
Appendix II: Results of Statistical Analysis of Class of 2002 Admissions
and Academy Performance Scores
Page 32 GAO- 03- 1000 Military Education
standard error of these estimates are small and differences that could be
considered small in magnitude may in fact be statistically significant. In
the tables below, differences that are statistically significant (p<. 05)
and exceed 5 percent are considered meaningful and noted, though such
differences may not be practically significant when compared with class
performance requirements overall. For example, at the Naval Academy the
overall average academic admissions score is 618, 5 percent of 618 is
about 31. Only those group average academic admissions scores that are
statistically significant and more than 31 points below 618 are noted with
an *a.* Differences that are greater than 10 percent are marked with a
*b.*
Table 12: Admissions and Performance Scores for the Class of 2002 at the
U. S. Military Academy Overall (1,246)
Females (192)
Minorities (269)
Prep school graduates
(184) Recruited
athletes (279)
Prior enlisted personnel
(31) Lower 30
percent of admissions
class (377)
Academic admissions 600 603 583 546 b 558 a 594 532 b Whole person
admissions score 6,006 6,022 5,865 5,645 a 5,814 5,861 5,609 a
Four performance measures 1. Cumulative GPA 2.99 2.99 2.82 2.61 a 2.81
3.14 2.66 a 2. Cumulative MPA 3.28 3.26 3.21 3.26 3.20 3.37 3.21 3. Order
of merit 3.03 3.04 2.86 a 2.75 a 2.90 3.06 2.78 a 4. Graduation rate 78%
76% 71% a 72% 76% 71% 71% a Source: GAO analysis, from Military Academy
sources.
a Denotes a group average or percentage with a statistically significant
(p<. 05) difference greater than 5% from the overall average or
percentage. b Denotes a group average or percentage with a statistically
significant (p<. 05) difference greater than
10% from the overall average or percentage.
Appendix II: Results of Statistical Analysis of Class of 2002 Admissions
and Academy Performance Scores
Page 33 GAO- 03- 1000 Military Education
Table 13: Admissions and Performance Scores for the Class of 2002 at the
U. S. Naval Academy Overall (1,226)
Females (190)
Minorities (221)
Prep school graduates
(146) Recruited
athletes (380)
Prior enlisted personnel
(76) Lower 30
percent of admissions
class (368)
Academic admissions 618 624 594 545 b 596 570 a 544 b Whole Person
admissions score 65,732 65,719 63,769 61,254 a 64,233 62,256 a 61,404 a
Four performance measures 1. Cumulative GPA 2.97 3.01 2.82 a 2.67 a 2.86
3.02 2.67 a 2. Cumulative MPA 3.12 3.16 3.02 2.99 3.08 3.19 3.00 3. Order
of merit 489 456 a 590 b 658 b 551 b 453 661 b 4. Graduation rate 80% 71%
b 75% 77% 79% 72% 76% a Source: GAO analysis, from Naval Academy sources.
a Denotes a group average or percentage with a statistically significant
(p<. 05) difference greater than 5% from the overall average or
percentage. b Denotes a group average or percentage with a statistically
significant (p<. 05) difference greater than
10% from the overall average or percentage.
Table 14: Admissions and Performance Scores for the Class of 2002 at the
U. S. Air Force Academy Overall (1,216)
Females (190)
Minorities (229)
Prep school graduates
(157) Recruited
athletes (312)
Prior enlisted personnel
(44) Lower 30
percent of admissions
class (366)
Academic admissions 3,202 3,216 3,123 3,112 3,043 3,188 2,863 b Whole
Person admissions score 798 805 782 774 773 792 751 a
Four performance measures 1. Cumulative GPA 2.93 2.97 2.78 a 2.61 b 2.79
2.89 2.64 a 2. Cumulative MPA 2.90 2.93 2.89 2.83 2.81 2.93 2.84 3. Order
of merit 469 440 545 b 663 b 568 b 499 646 b 4. Graduation rate 74% 75%
71% 69% 71% 66% 65% b Source: GAO analysis, from Air Force Academy
sources.
a Denotes a group average or percentage with a statistically significant
(p<. 05) difference greater than 5% from the overall average or
percentage. b Denotes a group average or percentage with a statistically
significant (p<. 05) difference greater than
10% from the overall average or percentage.
Appendix II: Results of Statistical Analysis of Class of 2002 Admissions
and Academy Performance Scores
Page 34 GAO- 03- 1000 Military Education
Regression models were used to assess the relationship between admission
scores and performance at the three academies. We used linear regression
models to examine relationships between admission scores and GPA, MPA, and
order of merit. To examine the relationship between admission scores and
the likelihood of graduating we used a logistic regression model. Both the
academic admission score and the whole person score were included as
independent variables in each model. We estimated separate regression
models for each academy. The results of these regressions are shown in
tables 15 and 16. The tables show both regression coefficients and
standardized
coefficients. In general, regression coefficients are interpreted as the
predicted change in the dependent variable for every unit change in the
independent variables. Here, we have scaled the admissions scores so that
the regression coefficients in the table can be interpreted as the
predicted change in the relevant measure of success for every 100- point
increase in the academic or *whole person* admission score. For example,
overall at
the U. S. Air Force Academy, for every 100- point increase in the academic
admission score we expect to see a 0.06 increase in GPA. For every 100-
point increase in the *whole person* score, we expect to see a 0.18
increase in GPA. Both relationships are statistically significant, meaning
that both the academic score and the *whole person* score are significant
predictors of cumulative GPA at the academy.
We cannot compare the size of these coefficients across the three
academies, though, because the academic and *whole person* scores are on
different scales. Because the size of the unstandardized regression
coefficients is affected by the scale of the independent variables (the
admissions scores), we use standardized regression coefficients to compare
them. These appear in parentheses in the tables. To estimate these
coefficients, all of the coefficients are standardized by dividing the
regression coefficient by the ratio of the standard deviation of the
success measure to standard deviation of the admission score. The
standardized regression coefficients, therefore, represent the change in
the measure of success for each change of one standard deviation in
admission scores. Using standardized coefficients, one can conclude that
the coefficient that is larger in magnitude has a greater effect on the
measures of success.
Using the same U. S. Air Force Academy example, we see that while the
relationships between both academic and *whole person* scores and GPA are
significant, the relationship between academic scores and GPA is actually
a stronger one than the relationship between the *whole person* score and
GPA. Overall, while the academic scores are often a better predictor of
academic performance at the academies (GPA), the *whole Relationships
between Admissions and Performance Scores
Appendix II: Results of Statistical Analysis of Class of 2002 Admissions
and Academy Performance Scores
Page 35 GAO- 03- 1000 Military Education
person* scores are often better predictors of military performance (MPA).
The academic admissions scores have no effect on MPA at the Military and
Air Force Academies and the whole person scores, not the academic
admissions scores, predict likelihood of graduating at all three
academies.
We also used the R 2 statistic to estimate how much of the variation in
each performance score can be explained by both academic and whole person
admission scores. The admission scores explained about 30 percent of the
variation in GPAs at both the Naval and Air Force Academies and about 40
percent of the variation in GPAs at the Military Academy. The admission
scores explained between a quarter and a third of the variation in order
of merit across the three academies. However, admission scores did not
explain as much of the variation in either military performance scores or
graduation rates. Therefore, while both types of admission scores are
significant predictors of performance at the academy, they only explain
between 7 and 40 percent of the variation in performance at the academies,
and only a very small percentage of the variability in the likelihood of
graduating. Other factors not studied here, such as the military training
and academic environment students experience at the academies, may
contribute to performance more than just students* admissions scores do.
Table 15: Regression Coefficients (Standardized Coefficients) from Linear
Regression Models Testing Correlations between Academic and Whole Person
Admissions Scores with Cumulative GPA, Cumulative MPA, and Order of Merit
for the Class of 2002 at the Service Academies
Cumulative GPA Cumulative MPA Order of Merit
Academic admission score Whole person
score Academic admission score Whole person
score Academic admission score Whole person score
Military Academy class overall .42
(. 56) a .01 (. 09)
-. 04 (-. 11)
.03 (. 44) a .18
(. 34) a .02 (. 29) a R 2 = .42 R 2 = .12 R 2 = .37 Naval Academy class
overall .28
(. 38) a .002 (. 18)
.09 (. 18) a .001
(. 19) a -157.62 (-. 35) a -1.13 (-. 17) a R 2 = .30 R 2 = .13 R 2 = .26
Air Force Academy class overall .06
(. 38) a .18 (. 20) a -. 01
(-. 14) .21
(. 37) a -27.89 (-. 30) a -147.29 (-. 25) a R 2 = .31 R 2 = .07 R 2 = .29
Source: GAO analysis, from DOD sources. Note: Because of the difference in
scales for admissions scores between academies, the size of the
coefficients cannot be compared across academies. a Denotes statistically
significant (p<. 05) relationships.
Appendix II: Results of Statistical Analysis of Class of 2002 Admissions
and Academy Performance Scores
Page 36 GAO- 03- 1000 Military Education
Table 16: Regression Coefficients (Standardized Coefficients) from
Logistic Regression Models Testing Correlations Between Academic and Whole
Person Admissions Scores and the Likelihood of Graduation for the Class of
2002 at the Service Academies
Graduation Academic admission score Whole person score -. 11 (-. 03)
.11 (. 24) a Military Academy class overall
R 2 = .02 -. 36 (-. 13)
.01 (. 23) a Naval Academy class overall
R 2 = .01 .01 (. 02)
.75 (. 20) a Air Force Academy class overall
R 2 = .03 Source: GAO analysis, from DOD sources. Note: Because of the
difference in scales for admissions scores between academies, the size of
the coefficients cannot be compared across academies.
a Denotes statistically significant (p<. 05) relationships.
Appendix III: Comments from the Department of Defense
Page 37 GAO- 03- 1000 Military Education
Appendix III: Comments from the Department of Defense
Appendix III: Comments from the Department of Defense
Page 38 GAO- 03- 1000 Military Education
Appendix IV: GAO Contact and Staff Acknowledgments
Page 39 GAO- 03- 1000 Military Education
Sandra F. Bell (202) 512- 8981 In addition to the individual named above,
Gabrielle M. Anderson, Herbert I. Dunn, Brian G. Hackett, Joseph W.
Kirschbaum, Wendy M. Turenne, and Susan K. Woodward also made key
contributions to this report. Appendix IV: GAO Contact and Staff
Acknowledgments GAO Contact Staff Acknowledgments
Related GAO Products Page 40 GAO- 03- 1000 Military Education
Military Education: DOD Needs to Align Academy Preparatory Schools*
Mission Statements with Overall Guidance and Establish Performance Goals.
GAO- 03- 1017. Washington, D. C.: September 2003.
Military Education: Student and Faculty Perceptions of Student Life at the
Military Academies. GAO- 03- 1001. Washington, D. C.: September 2003.
DOD Service Academies: Problems Limit Feasibility of Graduates Directly
Entering the Reserves. GAO/ NSIAD- 97- 89. Washington, D. C.: March 24,
1997.
DOD Service Academies: Comparison of Honor and Conduct Adjudicatory
Processes. GAO/ NSIAD- 95- 49. Washington, D. C.: April 25, 1995.
DOD Service Academies: Academic Review Processes. GAO/ NSIAD- 95- 57.
Washington, D. C.: April 5, 1995.
DOD Service Academies: Update on Extent of Sexual Harassment.
GAO/ NSIAD- 95- 58. Washington, D. C.: March 31, 1995.
Coast Guard: Cost for the Naval Academy Preparatory School and Profile of
Minority Enrollment. GAO/ RCED- 94- 131. Washington, D. C.: April 12,
1994.
Military Academy: Gender and Racial Disparities. GAO/ NSIAD- 94- 95.
Washington, D. C.: March 17, 1994.
DOD Service Academies: Further Efforts Needed to Eradicate Sexual
Harassment. GAO/ T- NSIAD- 94- 111. Washington, D. C.: February 3, 1994.
DOD Service Academies: More Actions Needed to Eliminate Sexual Harassment.
GAO/ NSIAD- 94- 6. Washington, D. C.: January 31, 1994.
Academy Preparatory Schools. GAO/ NSIAD- 94- 56R. Washington, D. C.:
October 5, 1993.
Air Force Academy: Gender and Racial Disparities. GAO/ NSIAD- 93- 244.
Washington, D. C.: September 24, 1993.
Military Education: Information on Service Academies and Schools.
GAO/ NSIAD- 93- 264BR. Washington, D. C.: September 22, 1993. Related GAO
Products
Related GAO Products Page 41 GAO- 03- 1000 Military Education
Naval Academy: Gender and Racial Disparities. GAO/ NSIAD- 93- 54.
Washington, D. C.: April 30, 1993.
DOD Service Academies: More Changes Needed to Eliminate Hazing.
GAO/ NSIAD- 93- 36. Washington, D. C.: November 16, 1992.
DOD Service Academies: Status Report on Reviews of Student Treatment. GAO/
T- NSIAD- 92- 41. Washington, D. C.: June 2, 1992.
Service Academies: Historical Proportion of New Officers During Benchmark
Periods. GAO/ NSIAD- 92- 90. Washington, D. C.: March 19, 1992. DOD
Service Academies: Academy Preparatory Schools Need a Clearer
Mission and Better Oversight. GAO/ NSIAD- 92- 57. Washington, D. C.: March
13, 1992.
Naval Academy: Low Grades in Electrical Engineering Courses Surface
Broader Issues. GAO/ NSIAD- 91- 187. Washington, D. C.: July 22, 1991.
DOD Service Academies: Improved Cost and Performance Monitoring Needed.
GAO/ NSIAD- 91- 79. Washington, D. C.: July 16, 1991.
Review of the Cost and Operations of DOD*s Service Academies. GAO/ TNSIAD-
90- 28. Washington, D. C.: April 4, 1990.
(350270)
The General Accounting Office, the audit, evaluation and investigative arm
of Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve the performance and accountability of
the federal government for the American people. GAO examines the use of
public funds; evaluates federal programs and policies; and provides
analyses, recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO*s commitment to
good government is reflected in its core values of accountability,
integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents at no cost
is through the Internet. GAO*s Web site (www. gao. gov) contains abstracts
and fulltext files of current reports and testimony and an expanding
archive of older products. The Web site features a search engine to help
you locate documents using key words and phrases. You can print these
documents in their entirety, including charts and other graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as *Today*s Reports,* on its
Web site daily. The list contains links to the full- text document files.
To have GAO e- mail
this list to you every afternoon, go to www. gao. gov and select
*Subscribe to e- mail alerts* under the *Order GAO Products* heading.
The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to: U. S. General Accounting Office 441 G Street NW, Room LM
Washington, D. C. 20548 To order by Phone: Voice: (202) 512- 6000
TDD: (202) 512- 2537 Fax: (202) 512- 6061
Contact: Web site: www. gao. gov/ fraudnet/ fraudnet. htm E- mail:
fraudnet@ gao. gov Automated answering system: (800) 424- 5454 or (202)
512- 7470 Jeff Nelligan, Managing Director, NelliganJ@ gao. gov (202) 512-
4800
U. S. General Accounting Office, 441 G Street NW, Room 7149 Washington, D.
C. 20548 GAO*s Mission Obtaining Copies of
GAO Reports and Testimony
Order by Mail or Phone To Report Fraud, Waste, and Abuse in Federal
Programs Public Affairs
*** End of document. ***