Darfur Crisis: Death Estimates Demonstrates Severity of Crisis,  
but Their Accuracy and Credibility Could Be Enhanced (09-NOV-06, 
GAO-07-24).							 
                                                                 
In 2003, violent conflict in Darfur, Sudan, broke out between	 
rebel groups and government troops and government-supported Arab 
militias. While few would dispute that many thousands of Darfur  
civilians have died, less consensus exists about the total number
of deaths attributable to the crisis. Estimates by the Department
of State (State) and other parties report death tolls up to about
400,000 for varying populations and periods of time between	 
February 2003 and August 2005. Based on the views of experts	 
convened by GAO and the National Academy of Sciences, interviews 
with estimate authors, and a review of relevant literature, this 
report (1) evaluates six Darfur death estimates, (2) identifies  
general challenges to estimating deaths in such crises, and (3)  
discusses measures to improve death estimates.			 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-07-24						        
    ACCNO:   A63187						        
  TITLE:     Darfur Crisis: Death Estimates Demonstrates Severity of  
Crisis, but Their Accuracy and Credibility Could Be Enhanced	 
     DATE:   11/09/2006 
  SUBJECT:   Data collection					 
	     Data integrity					 
	     Documentation					 
	     Evaluation 					 
	     Evaluation criteria				 
	     Evaluation methods 				 
	     Foreign governments				 
	     Genocide						 
	     International relations				 
	     Mortality						 
	     Statistical methods				 
	     Strategic planning 				 
	     Surveys						 
	     Systems evaluation 				 
	     Death						 
	     Transparency					 
	     Sudan						 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-07-24

   

     * [1]Results in Brief
     * [2]Background

          * [3]Conflict in Darfur
          * [4]U.S. Response to Darfur Crisis
          * [5]Data Sources and Methods for Darfur Death Estimates

     * [6]Some Death Estimates Judged More Accurate and Methodological

          * [7]No Estimate Consistently Considered Accurate
          * [8]Experts Found Methodological Shortcomings in Each Estimate

               * [9]CRED Estimates
               * [10]State Estimate
               * [11]WHO Estimate
               * [12]Three Other Estimates

     * [13]Estimating Deaths in Humanitarian Crises Involves Many Chall

          * [14]Challenges in Collecting Source Data Affected Data Quality a
          * [15]Gaps in Data Leads to Reliance on Assumptions and Limited In
          * [16]Limitations in Population Data Affect Reliability of Death E
          * [17]Varying Use of Baseline Mortality Rates Can Affect Estimates

     * [18]Wide Range of Measures Could Improve Death Estimates for Dar

          * [19]Proposed Measures Cover Wide Range of Activities
          * [20]Existing U.S. Initiatives May Improve Some Aspects of Data a

     * [21]Conclusions
     * [22]Recommendations for Executive Action
     * [23]Agency Comments and Our Evaluation
     * [24]Appendix I: Objectives, Scope, and Methodology
     * [25]Appendix II: List of Experts and Meeting Agenda

          * [26]List of Experts

               * [27]Summary of Meeting Discussion Questions

     * [28]Appendix III: List of Death Estimates and Mortality Surveys

          * [29]Death Estimates

               * [30]Surveys Used in Estimates as Data Sources
               * [31]Additional Bibliographical References

     * [32]Appendix IV: Summary Description of Death Estimates Reviewed
     * [33]Appendix V: Additional Follow-Up Survey Results
     * [34]Appendix VI: Comments from the Department of State

          * [35]GAO Comments

     * [36]Appendix VII: Comments from the U.S. Agency for Internationa
     * [37]Appendix VIII: Summary of Authors' Comments
     * [38]Appendix IX: GAO Contacts and Staff Acknowledgments

          * [39]GAO Contacts
          * [40]Staff Acknowledgments

               * [41]Order by Mail or Phone

Contents

Letter 1

Results in Brief 3
Background 5
Some Death Estimates Judged More Accurate and Methodologically Stronger,
Despite Shortcomings 19
Estimating Deaths in Humanitarian Crises Involves Many Challenges 27
Wide Range of Measures Could Improve Death Estimates for Darfur and Future
Humanitarian Crises 34
Conclusions 41
Recommendations for Executive Action 42
Agency Comments and Our Evaluation 42
Appendix I Objectives, Scope, and Methodology 44
Appendix II List of Experts and Meeting Agenda 48
Appendix III List of Death Estimates and Mortality Surveys Provided to
Experts and Additional Bibliographical References 50
Appendix IV Summary Description of Death Estimates Reviewed 56
Appendix V Additional Follow-Up Survey Results 62
Appendix VI Comments from the Department of State 64
GAO Comments 67
Appendix VII Comments from the U.S. Agency for International Development
68
Appendix VIII Summary of Authors' Comments 69
Appendix IX GAO Contacts and Staff Acknowledgments 71

Tables

Table 1: Accuracy of Darfur Death Estimates Rated by Experts 20
Table 2: Measures Rated by Experts as Likely to Greatly Improve Death
Estimates for Future Crises, in Order of Ranking and Number of
Endorsements 35

Figures

Figure 1: Map of Sudan 6
Figure 2: Map of Destroyed and Damaged Darfur Villages, as of February
2005 8
Figure 3: Example of Generation of a Death Estimate 14
Figure 4: Death Estimates Based on Reported Figures and Time Included 17

This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.

Abbreviations

CDC Centers for Disease Control and Prevention CE-DAT Complex Emergency
Database CIJ Coalition for International Justice CMR crude mortality rate
CRED Centre for Research on the Epidemiology of Disasters IDP internally
displaced persons JEM Justice and Equality Movement NGO nongovernmental
organization SLM/A Sudan Liberation Movement/Army SMART Standardized
Monitoring and Assessment of Relief and Transitions USAID U.S. Agency for
International Development UN United Nations WHO World Health Organization

United States Government Accountability Office

Washington, DC 20548

November 9, 2006

The Honorable Tom Lantos Ranking Minority Member Committee on
International Relations House of Representatives

The Honorable Mike DeWine The Honorable Richard J. Durbin United States
Senate

In 2003, violent conflict broke out in the Darfur region of western Sudan
when rebel groups, believing that the region was marginalized by the
Sudanese government, led attacks against the government. In response, the
government armed and supported local Arab and tribal militias, commonly
known as the Janjaweed, to defeat the rebels. Attacks on the civilian
population by these militias, sometimes in conjunction with the Sudanese
army, have resulted in widespread death and disruption. The crisis has
affected an estimated 3.76 million people in Darfur, including
approximately 1.85 million--"internally displaced persons" (IDPs)--who now
live in camps. To aid these populations, the United States has provided
more than $1 billion in assistance for Darfur since fiscal year 2004,
largely through the U.S. Agency for International Development (USAID).1

While few would dispute that many civilians have died in Darfur owing to
violence, disease, and malnutrition, less consensus exists about the total
number of deaths during, or directly resulting from, the crisis. The U.S.
Department of State (State) reported that a total of 98,000 to 181,000
people died between March 2003 and January 2005. Five other
studies--conducted for varying purposes by international institutions,
academics, and individual researchers--have produced estimates ranging up
to about 400,000 deaths for various periods of time between February 2003
and August 2005.2

1Simultaneously with the release of this report, we are issuing a separate
report on U.S. humanitarian assistance to relieve the crisis in Darfur;
challenges that have affected the delivery of the assistance; the African
Union's efforts to fulfill a mandate to support peace in Darfur; and
factors that have affected its implementation of this mandate. See GAO,
Darfur Crisis: Progress in Aid and Peace Monitoring Threatened by Ongoing
Violence and Operational Challenges, [42]GAO-07-9 (Washington, D.C.: Nov.
9, 2006).

Because policymakers require an accurate estimate of the death toll in
Darfur to understand the dimensions of the crisis and determine the U.S.
response, we were asked to examine the estimates, the methods used to
produce them, and their relative accuracy. This report (1) evaluates the
relative accuracy and methodological strengths and shortcomings of the six
death estimates for Darfur;3 (2) identifies general challenges to
estimating the total death toll in Darfur and similar humanitarian crises;
and (3) discusses measures that the U.S. government could take to improve
its death estimates for Darfur and any similar, future crisis.

To evaluate the estimates, we reviewed and analyzed public information on
the estimates and interviewed the estimate authors regarding their
studies' data, methods, and objectives. We provided this information and
summaries of the interviews to a group of 12 experts in epidemiology,
demography, statistics, and the Darfur crisis convened in April 2006 in
collaboration with the National Academy of Sciences.4 These experts
discussed their review of this information and evaluation of the estimates
during an all-day session and also assessed the estimates in a follow-up
survey. State's Bureau of Intelligence and Research, which conducted the
department's death estimate for Darfur, declined to speak with us or
provide additional information, limiting the experts' ability to fully
understand State's methods of analysis. However, despite this limitation,
the experts were able to discuss State's estimate in detail and assess its
accuracy and methodologies. To identify challenges of estimating total
deaths in Darfur, we asked the group of experts to highlight key problems,
and we reviewed literature related to death estimates and mortality data
in crises such as the one in Darfur. To identify ways in which the U.S.
government could improve death estimates for that and any future
humanitarian crisis, we asked the experts to provide suggestions during
their discussion, and we solicited further opinions on these suggestions
in the follow-up survey. Additionally, to learn about current practices
related to the experts' suggestions for improving death estimates, we
spoke with officials from USAID, State, and the Centers for Disease
Control and Prevention (CDC) in the U.S. Department of Health and Human
Services. Appendix I provides additional details of our objectives, scope
and methodology. We conducted our work from September 2005 to November
2006 in accordance with generally accepted government auditing standards.

2The five estimates that were publicly available in March 2006 were
conducted by, respectively, (1) Jan Coebergh, a medical doctor who has
worked in Darfur; (2) the Centre for Research on the Epidemiology of
Disasters (CRED) in Brussels, Belgium; (3) John Hagan, Wynnona
Rymond-Richmond, and Patricia Parker (released by the Coalition of
International Justice); (4) Eric Reeves, an researcher and advocate for
Sudan-related issues; and (5) the World Health Organization (WHO).

3This report's assessment of the estimates' methodological strengths and
shortcomings is based on experts' opinions of the estimates' data,
methods, objectivity, and sufficiency of reporting.

4Two additional experts participated by phone for parts of the day.

Results in Brief

The experts we consulted did not consistently rate any of the death
estimates as having a high level of accuracy and noted that all of the
studies had methodological strengths and shortcomings. Most of the experts
had the highest overall confidence in the estimates by the Centre for
Research on the Epidemiology of Disasters (CRED), which relied primarily
on a statistical analysis of about 30 mortality surveys, and they rated
the CRED estimates' accuracy and methodological strengths highest among
the six.5 The experts had a slightly lower level of confidence in the
State estimate and gave it slightly lower ratings for accuracy and
methodological strengths. Further, many experts believed that the lower
end of State's estimate was too low and found that published documents
describing State's estimate lacked sufficient information about its data
and methods to allow it to be replicated and verified by external
researchers. Most experts rated an estimate by the World Health
Organization (WHO), which primarily extrapolated findings from its 2004
mortality survey, somewhat lower than the estimates by CRED and State.
Finally, most of the experts expressed the least confidence in three
estimates that reported the highest number of deaths. They cited several
shortcomings in these estimates, such as a reliance on unrealistic
assumptions regarding fixed levels of mortality for all populations and
time periods included in the estimate.6

Estimating deaths in a humanitarian crisis such as that in Darfur involves
numerous challenges. In Darfur, difficulties related to the collection of
survey data, including a lack of access to particular geographical
regions, challenging survey conditions, and limited resources and training
for research staff, impacted the data's quality and resulted in data gaps.
Because of the lack of complete mortality data, some of the estimates rely
on potentially risky assumptions and limited contextual information. For
example, some estimates assume that mortality rates in accessible areas
can be applied to inaccessible areas, without accounting for differences
between the two due to factors such as patterns of violence and
humanitarian relief efforts. In addition, limitations in estimates of
Darfur's population before and during the crisis may have led to over- or
underestimations of the death toll. Finally, varying use of baseline
mortality rates--the rate of deaths that would have occurred regardless of
the crisis--may also have led to overly high or low death estimates.

5CRED conducted two death estimates: one for the period of September
2003-January 2005 and another for the period of February-June 2005.

6The three highest estimates reviewed by the experts ranged from 253,573
to 396,563 deaths over varying periods of time.

The group of experts proposed and rated a wide range of measures that U.S.
agencies could take, directly or by supporting other organizations, to
improve the quality and reliability of death estimates and relevant data
for Darfur and any future such humanitarian crises. The most highly rated
measure was ensuring that publicly available documentation of U.S.
government estimates contains sufficient information on data and methods,
so that external researchers can replicate the estimates and verify their
credibility and objectivity. Other very highly rated measures that the
government could consider included collecting and maintaining temporal and
spatial data (i.e., data covering specific periods of time and geographic
areas); housing the responsibility for making estimates in a reputable
independent body; improving the training of nongovernmental organization
(NGO) staff who collect mortality survey data; and promoting an
interdisciplinary approach to estimating mortality. Through various
initiatives already under way, U.S. agencies are carrying out efforts that
may address some of the experts' suggestions, for example, providing
guidance on the design and implementation of survey instruments and
supporting the collection and maintenance of temporal and spatial data.

To safeguard State's credibility as a source of accurate and reliable
death estimates, we recommend that the Secretary of State provide, for
this and any future estimates of mortality that State conducts,
sufficiently detailed descriptions of its data and methodology to enable
other parties to assess and replicate its findings.7 Additionally, to
enhance the U.S. government's capacity to assess and respond to any future
humanitarian crisis, we recommend that the Secretary of State and the
Director of U.S. Foreign Assistance and USAID Administrator consider the
experts' other suggestions to help address gaps in data and improve any
future death estimates.

7We recognize that when such estimates draw from classified data, external
researchers reviewing these data would require appropriate levels of
security clearance.

We provided a draft of this report to the Department of State and USAID.
State and USAID responded with formal comments, agreeing with our
recommendations, and State provided additional perspectives on reporting
and documentation regarding its death estimate. Reproductions of these
letters, as well as our responses to the letters, can be found in
appendixes VI and VII. We also provided a draft to the CDC for technical
review, and we received technical comments from both the CDC and State,
which we incorporated in the report as appropriate. Finally, we provided
the authors of the other five estimates the portions of the report
pertaining to their individual estimates. They provided comments, which we
incorporated in the report as appropriate. These authors' comments and our
responses are summarized in appendix VIII.

Background

Sudan is the largest country in Africa, with a population estimated at
about 40 million (see fig. 1). Darfur is in the western region of Sudan
and comprises three states, with an estimated preconflict population of
around 6 million. 8

8Estimates for the preconflict population of Darfur range from about 4
million to close to 7 million. Experts we interviewed have noted
uncertainty regarding the population estimate for Darfur due to the lack
of a current census and the fact that migration in this region occurs even
during nonconflict times.

Figure 1: Map of Sudan

Conflict in Darfur

In early 2003, Darfur rebels attacked Sudanese police stations and the
airport in El Fasher, the capital of North Darfur. In El Fasher, the
rebels destroyed numerous military aircraft, killed many Sudanese
soldiers, and kidnapped a Sudanese general. In response, the government
armed and supported the local tribal and Arab militias (the Janjaweed).
Fighting between the principal rebel groups--the Sudan Liberation
Movement/Army (SLM/A) and the Justice and Equality Movement (JEM)--and the
Sudan military and Janjaweed intensified during late 2003. In addition to
displacing approximately 1.85 million Darfurians in the region, attacks on
civilians by the Sudan military and Janjaweed have forced an estimated
220,000 Darfur refugees to take shelter in Chad9 and so affected
approximately 1.91 million additional Darfur residents that they require
humanitarian assistance. (Fig. 2 shows the locations of Darfur villages
destroyed or damaged in the conflict.)

9U.S. law considers any person who is outside the country of such person's
nationality and who is unable or unwilling to return to, and is unable or
unwilling to avail himself or herself of the protection of, that country
because of persecution or a well-founded fear of persecution on account of
race, religion, nationality, membership in a particular social group, or
political opinion to be a refugee. 8 U.S.C. S 1101 (a) (42)A.

Figure 2: Map of Destroyed and Damaged Darfur Villages, as of February
2005

Peace negotiations under the mediation of the Chad government in 2003 led
to a September 2003 cease-fire agreement between the SLM/A and the Sudan
government; however, the agreement collapsed in December of that year. A
second cease-fire agreement was signed by the Sudanese government, the
SLM/A, and the JEM in April 2004.10 At this point, the African Union was
brought in to monitor compliance with the cease-fire agreement between the
three parties, and peace negotiations continued under African Union
auspices with Chadian participation.11 After a relatively calm 2005,
cease-fire violations and violent incidents reportedly began to increase
in the final months of the year and into 2006.

On May 5, 2006, the government of Sudan and one faction of the SLM/A
signed the Darfur Peace Agreement, which establishes agreements in key
areas such as power sharing, wealth sharing, and security arrangements.12
The U.S. government and other parties support a transition of peacekeeping
responsibilities from the African Union to the UN. In August 2006, the UN
Security Council adopted a resolution expanding the mandate of the UN
Mission in the Sudan and calling for the mission's deployment to Darfur.13
As of October 2006, the Sudanese president had rejected a UN transition
but expressed support for a September offer by the UN Secretary-General to
assist the African Union Mission in Sudan by providing equipment and
dedicated personnel. Meanwhile, the environment in Darfur remained
insecure, with attacks and displacement continuing and, during some
periods, worsening over time.

10"Agreement on Humanitarian Ceasefire on the Conflict in Darfur," signed
on April 8, 2004, in N'Djamena, Chad.

11Additional interim agreements were also reached, including the July 5,
2005, "Declaration of Principles for the Resolution of the Sudanese
Conflict in Darfur" signed by the Sudanese government, the SLM/A, and the
JEM. This declaration contains 17 principles to guide future
deliberations, such as respect for the diversity of the people of the
Sudan, democracy, a federal system of government, effective representation
in all national government institutions, and equitable distribution of
national wealth.

12Darfur Peace Agreement" signed on May 5, 2006, at Abuja, Nigeria.

13UN Security Council Resolution 1706, adopted August 31, 2006. The
expanded UN Mission in the Sudan mandate includes, among other things,
supporting the implementation of the 2006 Darfur Peace Agreement and the
2004 Humanitarian Ceasefire Agreement.

U.S. Response to Darfur Crisis

The U.S. government has been active in addressing the conflict. On July
22, 2004, the U.S. House and the Senate each passed separate resolutions
citing events in Darfur as acts of genocide.14 Further, on September 9,
2004, in testimony before the Senate Foreign Relations Committee, the U.S.
Secretary of State announced that genocide had been committed in Darfur
and that the Sudanese government had supported the Janjaweed, directly and
indirectly, as they carried out a "scorched earth" policy toward the
rebels and the African civilian population in Darfur.15

According to State officials, the administration's declaration of genocide
was influenced by findings from interviews with 1,136 Darfur refugees in
eastern Chad in July and August 2004, which demonstrated a pattern of
abuse against members of Darfur's non-Arab communities. The interviews
were conducted by an "Atrocities Documentation Team" assembled by State
and consisting of officials from State and USAID, as well as members of
the Coalition for International Justice16 and the American Bar
Association. State reported that 61 percent of those interviewed said that
they had directly witnessed the killing of a family member in addition to
other crimes.17

For fiscal years 2004 through 2006, the United States obligated $996
million in humanitarian assistance for Darfur.18 Although more than 68
percent of this assistance has consisted of food aid, U.S. assistance to
Darfur has also supported other vital needs, such as water and sanitation,
shelter, and primary health care services. Partly as a result of U.S.
assistance, NGOs and UN agencies have made significant progress in
increasing the number of internally displaced persons and affected
residents receiving aid. The number of international and national
humanitarian aid workers in Darfur expanded from 202 in April 2004 to
13,500 staff members of 84 NGOs and 13 UN agencies in July 2006. In 2005
and early 2006, malnutrition and mortality rates in Darfur had dropped
since 2004, a trend that U.S. and other officials attribute in part to
international humanitarian assistance efforts. Further, according to State
sources, the U.S. government, via private contractors, provided about $280
million, between June 2004 through September 2006, primarily to build and
maintain facilities throughout Darfur to house African Union troops
assigned to monitor compliance with the April 2004 cease-fire agreement.19

14H. Con. Res. 467, 108th Cong. (2004); S. Con. Res. 133, 108th Cong.
(2004).

15The Convention on the Prevention and Punishment of the Crime of Genocide
of 1948 defines genocide as any of the following acts committed with
intent to destroy, in whole or in part, a national, ethnical, racial or
religious group, as such: (a) killing members of the group, (b) causing
serious bodily or mental harm to members of the group, (c) deliberately
inflicting on the group conditions of life calculated to bring about its
physical destruction in whole or in part,(d) imposing measures intended to
prevent births within the group, or (e) forcibly transferring children of
the group to another group. The U.S. government ratified the convention in
1988. According to a State official, the key factor in the U.S.
government's genocide determination was the intent of the Sudanese
government regarding its actions in Darfur (i.e., its intent to destroy,
in whole or in part, a specific group of people); the number of deaths
attributable to the crisis was not a critical factor.

16The Coalition for International Justice (CIJ) was an international,
nonprofit organization that supported international war crimes tribunals
and justice initiatives. It closed its operations in March 2006.

17Department of State, Documenting Atrocities in Darfur, Publication 11182
(Washington, D.C.: Bureau of Democracy, Human Rights, and Labor and the
Bureau of Intelligence and Research, September 2004). Available at
[43]http://www.state.gov/g/drl/rls/36028.htm .

18In addition to the U.S. assistance provided for Darfur, an additional
$197 million has supported Darfur refugees located in Eastern Chad for
fiscal years 2004 through 2006.

Data Sources and Methods for Darfur Death Estimates

A key data source for the Darfur death estimates is health, nutrition, and
mortality surveys conducted in the field by NGOs delivering humanitarian
services (e.g., Medecins Sans Frontieres, known in English as Doctors
without Borders, and Save the Children)20 as well as by UN or governmental
agencies (e.g., the WHO, the World Food Program, and the CDC). These
surveys are discrete data collection exercises carried out at a specific
time with a particular sample of the affected population, such as people
in a certain IDP camp.

Surveys that ask about mortality are often combined with those collecting
health and nutrition data. "Retrospective" mortality data are collected by
asking a sample of respondents to recall the number of deaths that
occurred in their household during an earlier defined period of time.21
Interviewers may also ask respondents questions that allow them to
categorize the cause of death (e.g., deaths due to violence, disease, or
malnutrition). Households surveyed may be located in a single area (e.g.,
displacement camp) or multiple areas (e.g., multiple camps or sites within
a region.) Because an absolute number of deaths is difficult to interpret,
organizations conducting surveys calculate mortality rates for the
population sampled and for the time included in the "recall period." These
mortality rates allow for comparison among different population groups and
with different time periods, such as when no humanitarian crisis is
occurring. In acute emergencies, when mortality may change significantly
during a short time interval, mortality rates are often expressed as the
number of deaths per 10,000 people per day. Typically, a crude mortality
rate--that is, the rate of death for the entire population, including both
sexes and all ages22--is reported, as well as mortality rates for specific
groups (such as those younger than 5 years or of a specific sex).

19See [44]GAO-07-9 .

20 The estimates by Coebergh, Hagan, and Reeves used the Atrocities
Documentation Team's survey of Chad refugees as an additional key source
of data.

21Respondents are also asked to report the number of births and the
numbers of people in their household during the recall period.

Data from CRED's Complex Emergency Database (CE-DAT) show that at least 68
surveys conducted with IDPs and affected residents reporting crude
mortality rates were conducted in Darfur between 2004 and 2006.23 Two
retrospective mortality studies used in most Darfur death estimates were
conducted, respectively, by Medecins Sans Frontieres and the WHO.

           o Medecins Sans Frontieres Mortality Surveys in West Darfur. The
           organization conducted retrospective mortality surveys with 3,175
           households in four sites in West Darfur.24 The surveys, conducted
           between April and June 2004, asked respondents about the number
           and cause of deaths in their household up to 6 months earlier,25
           before they fled their villages, as well as after they arrived at
           their IDP camps or settlements. Medecins Sans Frontieres reported
           crude mortality rates ranging from 5.9 to 9.5 deaths per 10,000
           per day during the period when respondents were in their villages
           and in flight and rates ranging from 1.2 to 5.6 deaths per 10,000
           per day during the camp period. Medecins Sans Frontieres also
           reported that violence caused 68 to 93 percent of deaths during
           the village and flight periods.

           o 2004 World Health Organization Retrospective Mortality Survey.
           The WHO surveyed 3,140 households in accessible IDP settlements in
           the three regions of Darfur to determine mortality rates between
           June and August of 2004.26 The organization reported crude
           mortality rates of 1.5 per 10,000 per day for North Darfur; 2.9
           per 10,000 per day in West Darfur; and 3.8 per 10,000 per day for
           Kalma camp in South Darfur for the defined recall period. Diarrhea
           accounted for most of the reported deaths in all three regions,
           while violence or injury accounted for a smaller percentage of
           deaths, ranging from 10 to 21 percent.

           Researchers estimating death tolls in humanitarian crises such as
           that in Darfur generally extrapolate mortality surveys' various
           mortality rates to longer time periods and larger populations. In
           addition, to estimate an "excess" number of deaths directly
           attributable to the conflict, some researchers subtract a baseline
           mortality rate--that is, an expected number of deaths that would
           have occurred absent the conflict--from the total number of deaths
           estimated for the time period and population. (Fig. 3 illustrates
           the generation of a death estimate.)

           Figure 3: Example of Generation of a Death Estimate

           The death estimates that we and the group of experts reviewed were
           produced for varying purposes, according to their authors.

                        o The authors of the CRED estimates27 said that their
                        aim in conducting their estimate was to develop a
                        method that, rather than extrapolating mortality from
                        a single survey to the entire region and conflict
                        period, took into account variations over time and
                        space.

                        o Dr. Coebergh28 noted that his estimate was intended
                        as a political statement to increase public awareness
                        of the crisis. He also stated that the estimate was
                        produced as a response to earlier total death tolls
                        quoted in the media, which he believed were too low
                        and did not adequately capture violent deaths.

                        o According to Dr. Hagan,29 his estimate in the
                        spring of 2005 responded to earlier total death
                        figures, quoted in the media, that he believed were
                        too low and did not adequately capture violent
                        deaths. He stated that one of his goals was to
                        provide greater transparency about the available
                        estimates.

                        o Dr. Reeves30 said that he produced his first
                        estimate of the Darfur death toll, in June 2004,
                        because he believed that the figures being quoted by
                        the UN significantly understated the Darfur death
                        toll and were not supported by the data.

                        o According to State, its purpose was to provide
                        information for internal policymakers.

                        o A WHO31 official told us that the organization
                        sought to assess the order of magnitude of deaths in
                        Darfur to assist it in planning humanitarian relief
                        for IDPs in camps.

           The estimates that we and the experts reviewed vary quantitatively
           and in the time that they cover--from 35,000 excess deaths or
           45,000 total deaths for IDPs in camps over 7 months of the
           conflict, to almost 400,000 total deaths for Darfur over 26 months
           of the conflict. (See fig. 4 for a summary description of the
           estimates and their findings, and see app. IV for more details.)

22According to the Sphere Project, the crude mortality rate can be
expressed with different standard population denominators and for
different time periods; however, the daily crude mortality rate is the
most specific and useful health indicator to monitor in a disaster
situation. [See The Sphere Project, Humanitarian Charter and Minimum
Standards in Disaster Response (Geneva: Switzerland, 2004); also known as
the Sphere Handbook.] Sphere, launched in 1997 by a group of humanitarian
NGOs and the Red Cross, developed minimum standards to be attained in
disaster assistance in five sectors: water supply and sanitation,
nutrition, food aid, shelter, and health services. The Sphere Handbook
reports that a doubling of the baseline crude mortality rate indicates a
significant public health emergency, requiring an immediate response. If
the baseline rate is unknown, health agencies should aim to maintain the
crude mortality rate at below 1.0 per 10,000 per day. According to data
from the U.S. National Center for Health Statistics, the normal crude
mortality rate for 2003 in the United States is about 0.23 deaths per
10,000 per day.

23CE-DAT is a searchable database of complex emergencies that includes
information on health and mortality indicators. The database which
receives funds from the State's Bureau of Population, Refugees, and
Migration, is housed in CRED, a WHO Collaborating Center located within
the School of Public Health of the Universite Catholique de Louvain in
Brussels. (See [45]http://www.cred.be/cedat/index.htm .) According to
CRED, the database contains 1,155 surveys covering 36 countries; half of
these surveys are drawn from original reports, and the other half are
drawn from secondary sources. Research staff from CE-DAT we interviewed
have told us that, because they rely on voluntary reporting from
organizations conducting surveys to populate their database, not all
surveys that have been conducted in Darfur may be included in their
database.

24Across the four sites, Medecins Sans Frontieres surveyed 3,175
households representing a total of about 17,500 people and drawn from an
estimated IDP population of 215,400. See Depoortere, Evelyn et al.
"Violence and Mortality in West Darfur, Sudan (2003-2004): Epidemiological
Evidence from Four Surveys." Lancet, 364 (2004): 1,315-1,320. Medecins
Sans Frontieres produced this study in collaboration with Epicentre, a
nonprofit organization created in 1987 by Medecins Sans Frontieres, which
groups health professionals specialized in public health and epidemiology.
(See http://www.epicentre.msf.org/.)

25The recall period varied in the surveys conducted across the four sites.
The longest was 183 days for the survey conducted at Zalingei, and the
shortest was 39 days for the survey at El Geneina. Medecins Sans
Frontieres reported using a calendar of locally important events to
facilitate recall.

26The WHO reported surveying a total of 3,140 households representing
about 21,000 people. The WHO surveyed IDPs from 43 locations in North
Darfur and 43 locations in West Darfur, but due to security problems
surveyed only IDPs residing in Kalma camp in South Darfur. See World
Health Organization. Retrospective Mortality Survey among the Internally
Displaced Population Greater Darfur, Sudan 2004, 2004.
www.who.int/disasters/repo/14656.pdf

27CRED is a nonprofit research institution and a World Health Organization
Collaborating Centre based in the School of Public Health of the
Universite Catholique de Louvain in Brussels.

28Jan Coebergh is a medical doctor in the Netherlands who has worked in
Darfur.

29John Hagan, a professor of sociology at Northwestern University,
conducted his estimate with his colleagues, Wynnona Rymond-Richmond and
Patricia Parker; the estimate was released by the Coalition of
International Justice. Dr. Hagan stated that the 2005 estimate used an
assumption of constant levels of mortality over time, because he believed
other estimates had relied on that assumption. He also noted that he
subsequently modified his assumptions and estimating methods in his
estimate with Alberto Palloni. [See John Hagan and Alberto Palloni,
"Social Science: Death in Darfur," Science, vol. 313 (2006): 1,578-1,579.]

30Eric Reeves is a professor of English language and literature at Smith
College and has spent the past 7 years working full-time on research and
advocacy issues related to Sudan. When the experts convened in April 2006,
they reviewed and discussed Reeves's latest available estimate of 375,000
through August 2005. At the end of April 2006, he reported that total
excess mortality in Darfur was greater than 450,000 deaths.

31The WHO estimate was presented by David Nabarro, a senior WHO official.

Figure 4: Death Estimates Based on Reported Figures and Time Included

aCRED conducted two death estimates: one for the period of September
2003-January 2005 and another for the period of February-June 2005. The
bar on the left for "estimated deaths" is the combined total of these two
estimates.

bThis estimate was conducted by John Hagan and colleagues and released by
the CIJ.

We and the group of experts also reviewed a preliminary version of a death
estimate for West Darfur by John Hagan and Alberto Palloni; however, this
estimate is not discussed in our report, because the version that the
experts reviewed was preliminary and not publicly available when they
convened in April 2006. In the estimate's final version, which became
publicly available in September 2006, Hagan and Palloni estimated that a
range of 57,506 to 85,346 deaths had occurred in West Darfur over 31
months. Using the same ratio of deaths to displacement, they estimated a
range of 170,000 to 255,000 deaths in all three states of Darfur over the
same time period. 32

In addition, the estimates differ methodologically, incorporating varying
source data, mortality rates and causes of death, assumptions regarding
variable or fixed mortality rates, estimates of the affected population,
consideration of baseline mortality rates, and reporting methods. (See
app. IV for more information.)

           o Source data. The estimates' respective sources of data range
           from about 30 mortality surveys to a single mortality survey. The
           estimates also use nonsurvey or contextual information, such as
           the retrospective reporting of attacks or displacement patterns,
           to varying degrees. In addition, the amount of source data used
           varies according to the availability of such data when the
           estimates were conducted. For example, fewer data were available
           for estimates conducted in 200433 than for those conducted in
           2005.

           o Mortality rates and cause of death. The estimates differ in
           whether they applied mortality rates that include all causes of
           death or calculated and applied mortality rates due to violence
           and nonviolence separately. However, most of the estimates, as
           well as the mortality surveys we reviewed, express mortality rates
           in deaths per 10,000 people per day.

           o Assumptions of variable or fixed mortality rates. The estimates
           include different assumptions regarding whether mortality rates
           varied over the period of the conflict or according to the
           affected region or remained constant for all populations in all
           areas over the entire period of conflict.

           o Affected population estimates. The estimates rely primarily on
           information from the UN Humanitarian Profiles in determining the
           total affected population, but they differ in their assumptions
           about whether the affected population changed over time or
           remained constant. Additionally, one estimate also includes only
           IDPs in camps and excludes affected residents in Darfur and
           refugees in Chad.

           o Baseline mortality rates. The estimates varied in their use of a
           baseline mortality rate--that is, the number of expected deaths
           absent the conflict--from using no baseline to using a baseline of
           0.5 deaths per 10,000 per day.

           o Reporting methods. Some estimates report a range of possible
           total deaths, others report point estimates -- that is, single
           numbers. Additionally, some estimates are precise figures, such as
           396,563, while others are rounded to fewer significant digits,
           such as 134,000.
			  
			  Some Death Estimates Judged More Accurate and Methodologically
			  Stronger, Despite Shortcomings

           Although none of the death estimates was consistently considered
           accurate or methodologically strong, the experts we consulted
           rated some of the estimates more highly than others. Overall, the
           experts expressed the highest level of confidence in CRED's
           estimates and slightly lower levels of confidence in State's and
           the WHO's estimates. They expressed the lowest level of confidence
           in the three estimates that report the highest number of deaths,
           citing multiple shortcomings, such as a reliance on unrealistic
           assumptions about populations' level of risk over periods of time.
			  
			  No Estimate Consistently Considered Accurate

           Although the experts clearly had greater confidence in some
           estimates than in others, they did not consistently express a high
           level of confidence in any of the estimates or consistently rate
           any of them as accurate.34

           The experts expressed a slightly higher level of confidence in the
           CRED estimates than the State estimate. However, only 2 of the 12
           experts expressed a high level of confidence, and most had a
           moderate level of confidence, in the CRED estimates. Experts had a
           moderate to low level of confidence in the WHO estimate. Almost
           all experts expressed a low level of confidence in the estimates
           by Coebergh, Hagan, and Reeves.35 (See app. V for additional
           survey results.)

           Similarly, the experts did not consistently rate any of the
           estimates as accurate; the majority of experts rated almost all of
           the reported figures as either too high or too low. Only CRED's
           second estimate--36,000 total deaths for February-June 2005--was
           viewed by half of the experts as "about right." (See table 1.)

           Table 1: Accuracy of Darfur Death Estimates Rated by Experts

           Source: GAO.

           Note: We asked experts to rate the accuracy of total deaths when
           these were reported in the estimate. Coebergh and Reeves reported
           excess deaths only. Additionally, in responding to this question,
           one expert said that there was no basis to judge or that he or she
           was not sure about the accuracy of the estimates; one expert chose
           not to respond to this question; another expert chose two
           responses for the second CRED estimate, and his or her responses
           are excluded.

           In some cases, experts tended to agree as to whether the reported
           figures were too high or too low; in other cases, there was less
           agreement. For example, 9 of 10 experts rated the lower-end of
           State's estimate as too low, and the majority of the experts
           viewed the estimates by Coebergh, Hagan, and Reeves as too high.
           In contrast, experts did not agree as to whether State's higher
           estimate was too high, too low, or about right. For example, a few
           experts believed the higher end of State's estimate was likely to
           be closer to a reasonable midrange estimate. These experts also
           thought that the mortality surveys State used for the estimate may
           have been conducted at places with higher levels of aid and
           subsequently lower levels of mortality, or that disease outbreaks
           may have been missed. However, one expert believed that State may
           have overestimated mortality by applying elevated mortality rates
           for too many months of the crisis.
			  
			  Experts Found Methodological Shortcomings in Each Estimate

           The experts' overall assessment of the estimates' methodological
           strengths in terms of their data, methods, objectivity, and
           reporting did not produce any consistently high ratings, and
           experts noted shortcomings in each estimate.36

			  CRED Estimates

           Overall, the experts rated CRED's estimates most highly in terms
           of data, methods, objectivity, and reporting of limitations.
           However, several experts found shortcomings in the CRED estimates'
           data and methods and thought that CRED could have provided more
           information and clarity in its reporting.

           o Source data. Most experts said that the data CRED used for its
           two estimates (drawing from a total of about 30 mortality surveys
           in Darfur and Chad37) were generally sound.38 CRED reported
           checking the reliability and validity of the surveys included in
           its estimates, which experts found to be a strength. However,
           several experts cited some shortcomings in CRED's data sources.
           For example, some said that CRED could have better articulated the
           criteria used to select the survey data or weighted the surveys it
           used by sample size. A few experts also thought that CRED should
           have considered other sources besides mortality surveys, such as
           surveillance data on morbidity or nutrition.

           o Methods used, including extrapolations and assumptions. Some
           experts found CRED's method of using disparate data sources to
           estimate total deaths to be innovative and logical.39
           Additionally, more than half of the experts rated CRED's
           assumptions and extrapolations as somewhat appropriate or
           reasonable.40 For example, several experts found the Sudan
           baseline mortality rate that CRED used more accurate than the
           baseline mortality rates derived from a larger region of
           sub-Saharan Africa used in some of the other estimates. However,
           some assumptions and extrapolations were questioned. For example,
           several experts thought that CRED's assumption regarding a
           generally stable rate for nonviolent mortality during much of 2004
           may have been inaccurate, owing to possible changes in the
           affected population, camp formations, the level of aid, and the
           outbreak of disease. Additionally, some experts considered the
           method that CRED used to estimate deaths among refugees in Chad to
           be somewhat unsystematic.

           o Level of objectivity. Overall, experts viewed CRED's death
           estimates as having the highest level of objectivity.41 Two
           authors of other estimates also concluded that the CRED estimates
           were likely to be more reliable and more scientific than other
           Darfur death estimates, including their own.

           o Sufficiency of reporting. Experts noted that, among the
           estimates they evaluated, CRED most sufficiently reported the
           limitations and potential sources of over- or underestimation in
           its estimates. However, several experts believed that better
           descriptions of the methods used, including information on
           specific formulas and calculations, could have been provided. An
           author of another estimate also noted that the mortality surveys
           used in CRED's estimates lacked complete citation information. In
           our review of CRED's first estimate, we were able to replicate it
           to some degree only after the authors provided a substantial
           amount of information, such as specific mortality rates and
           formulas used and citations for source studies, in addition to the
           information in the published document.
			  
			  State Estimate

           In assessing State's estimate, the experts identified
           methodological strengths related to each of the four elements but
           also noted some shortcomings. Strengths included its use of
           multiple types of information, including contextual data from
           other sources besides surveys, such as reporting of attacks. Many
           experts also believed the estimate had a high level of
           objectivity. However, the experts, as well as authors of other
           estimates, cited several shortcomings in State's estimate. For
           example, many believed that the lower end of the estimate was too
           low, owing to several factors including the use of some data that
           underestimated mortality rates. Additionally, experts and other
           authors thought that the published documents containing the
           estimate lacked sufficient information, such as a clear
           description of the mortality rates used for all populations and
           time periods included in the estimate.

           o Source data. Many experts cited as one strength the estimate's
           use of different types of data, including mortality survey data
           and contextual information, to triangulate findings and estimate
           mortality, and one expert deemed this approach a "pioneering
           attempt" in the field of death estimates in humanitarian crises.42
           However, just over half of the experts thought that the data used
           were methodologically sound.43 Some experts said that several of
           the mortality surveys used in State's estimate may have had
           methodological limitations in areas such as survey design,
           implementation, or accessibility to insecure regions, resulting in
           unrealistically low mortality rates. These experts believed that
           such limitations in source data, in addition to other
           problems--for example, the estimate's lack of clarity regarding
           how missing populations are accounted for and use of a relatively
           higher baseline mortality rate--may have pulled down State's
           estimate, in particular, its lower end.

           o Methods used, including extrapolations and assumptions. About
           half of the experts felt that State's estimate applied somewhat
           appropriate extrapolations, and a similar proportion thought that
           it made reasonable assumptions. A few experts thought that its
           depiction of varying levels of mortality over time and affected
           regions was appropriate to estimate total deaths throughout the
           Darfur crisis. However, several experts thought that the
           assumptions used, in some cases, were based on insufficient
           rationale and evidence and that additional sensitivity analysis
           could have been conducted. For example, the State estimate assumes
           that mortality rates in Darfur are 20 percent lower for affected
           residents than for IDPs, an assumption that some experts believed
           lacked sufficient rationale.

           o Level of objectivity. Nine of the 12 experts rated the State
           estimate's level of objectivity as high.44 Several experts
           generally believed that the estimate represented a "good faith
           effort" to use available evidence in an unbiased way.

           o Sufficiency of reporting. Many of the experts found that the
           published documents containing State's estimate lacked sufficient
           information to allow them to replicate the estimate and verify the
           accuracy and reliability of the data and methods. For example,
           some experts noted that the data used were not sufficiently
           described and that more specific citations, as well as a
           description of the criteria applied to select the data, would have
           been useful. Additionally, some experts felt that they did not
           have a good sense of the reliability and validity of the
           contextual data used in the estimate and the application of these
           data to determine mortality rates or total deaths. Moreover,
           several experts said that State should have included the specific
           formulas or mortality rates used for all populations, time
           periods, and regions. Similar comments were made by several
           authors of other estimates. Our review of the State estimate also
           showed that it could not be replicated with the information
           contained in the report. Further, one expert noted that the
           published document available on State's Web site particularly
           lacked sufficient description of the estimate's methods, data, and
           potential limitations.
			  
			  WHO Estimate

           Several experts found strengths in the data and level of
           objectivity of the WHO's estimate, which it presented in a short
           briefing in October 2004. However, several experts observed
           shortcomings in the WHO's reporting of its estimate.

           o Source data. The WHO estimate of IDP deaths in Darfur for 7
           months in 2004 relied primarily on findings from the
           organization's 2004 mortality survey. Several experts noted that
           this survey followed standard methods and was generally reliable.
           However, a few experts and estimate authors said that the 2004 WHO
           survey may have underestimated mortality, owing to local
           government restrictions that prevented researchers from asking
           respondents detailed questions about mortality.

           o Methods used, including extrapolations and assumptions. Experts
           provided mixed ratings on the appropriateness of the
           extrapolations and the reasonableness of the assumptions used in
           the WHO estimate. For example, one expert believed that because
           the methodology of the organization's 2004 survey was appropriate
           to gauge levels of mortality among a limited IDP population for 2
           months, but not aggregate mortality, the survey findings should
           not have been extrapolated to generate a death estimate for the
           total IDP population for 7 months.

           o Level of objectivity. Half of the experts rated the level of
           objectivity of the WHO estimate as high, three rated it as
           moderate, and two rated it as low.45 About half of the experts
           thought that the WHO estimate's level of objectivity was equal to
           CRED's and State's.
           o Sufficiency of reporting. Several experts thought that the WHO's
           reporting of its estimate had several shortcomings. For example,
           they thought that the briefing document describing the death
           estimate did not make it sufficiently clear that the estimate
           included deaths of IDPs from all causes--both violent and
           nonviolent--but excluded deaths of affected residents in Darfur
           and refugees in Chad. The experts found that the lack of a clear
           description of such issues allowed for misinterpretation by
           readers, including the media and academia.
			  
			  Three Other Estimates

           The experts cited several methodological shortcomings in the
           Coebergh, Hagan, and Reeves estimates, including the use of
           problematic data and application of unrealistic assumptions about
           the levels of mortality over time and affected populations.

           o Source data. Many experts found shortcomings in each of the
           three estimates' use of certain survey data.46 A number of experts
           noted problems in the design, sampling, and data collection in the
           Atrocities Documentation Team's survey of Chad refugees on which
           all three estimates based, at least in part, their numbers of
           violent deaths. Experts also pointed out that, because the
           survey's intended purpose was to document levels and types of
           victimization, the estimates by Dr. Coebergh, Dr. Hagan, and Dr.
           Reeves should not have extrapolated the survey findings to a
           broader population or time period in order to estimate total
           deaths. In addition, many experts observed that the estimates by
           Coebergh and Hagan inappropriately used findings from the 2004 WHO
           survey to calculate only nonviolent deaths without taking into
           account the fact that some deaths reported by the WHO were due to
           violence or injury.47

           o Methods used, including extrapolations and assumptions. Most
           experts found that the Coebergh, Hagan, and Reeves estimates used
           unrealistic extrapolations and assumptions to fill information
           gaps and estimate total deaths. For example, many experts thought
           that each of the three estimates relied on too few data points
           extrapolated to an excessive degree. As a result of this type of
           extrapolation, the experts observed, a sensitivity analysis
           changing one or two assumptions could swing the total number of
           deaths from 100,000 to half a million, making the estimates
           unreliable. Moreover, several experts believed that some of the
           authors had inappropriately assumed constant rates of mortality
           for different population groups in Darfur at different periods in
           the conflict.48 Two of the estimates (Coebergh's and Hagan's) also
           used "fixed" estimates of the affected population over time, a
           method that some experts thought was inappropriate because the
           affected population grew over the course of the conflict.
           Additionally, the three estimates involve assumptions that some
           experts viewed as questionable, such as using unsupported numbers
           to estimate the number of deaths among populations inaccessible to
           aid or assuming all missing persons were likely to be dead.

           o Level of objectivity. Most experts rated the level of
           objectivity of the three estimates as low, particularly those by
           Drs. Coebergh and Reeves. The experts thought that the estimates
           were more characteristic of advocacy or journalistic material than
           of objective analysis.

           o Sufficiency of reporting. One expert noted that the estimate by
           Dr. Hagan was very straightforward and one could follow the logic
           of the data and methods applied. However, most of the experts
           found that the three estimates did not sufficiently describe
           limitations that may have resulted in under- or overestimation of
           total deaths. In reviewing the estimates, we found we were able to
           replicate Dr. Hagan's entire estimate based on its description in
           public documents. We were also generally able to replicate Reeves'
           estimate after receiving additional information about, among other
           things, his calculation of some numbers and the sources of his
           data. We had more difficulty in replicating Dr. Coebergh's
           estimate based on the information in the published article and
           were able to do so only after the author provided, at our request,
           details about the specific data, methods, and formulas that were
           used.
			  
			  Estimating Deaths in Humanitarian Crises Involves Many Challenges

           Estimating the total deaths in a humanitarian crisis such as that
           in Darfur involves a number of challenges, most notably related to
           collecting data in the field and extrapolating from limited data.
           Difficulties in collecting sound, consistent survey
           data--including lack of access to particular geographical regions,
           the conditions under which the surveys are conducted, and limited
           resources and training for field staff conducting surveys--affect
           the quality of the data collected and result in data gaps. Because
           of such limitations, as well as the unavailability of data from
           other sources, the death estimates that we reviewed rely on
           potentially risky assumptions and limited contextual information.
           Limitations in estimates of Darfur's population before and during
           the crisis also impacted the quality of the death estimates.
           Additionally, varying approaches to the use of baseline mortality
           rates may have somewhat affected their accuracy.
			  
			  Challenges in Collecting Source Data Affected Data Quality and Led
			  to Data Gaps

           Difficulties in conducting health, nutrition, and mortality
           surveys in a crisis such as Darfur's can affect survey data's
           quality and completeness and, thus, the soundness of death
           estimates based on the data. In Darfur, these difficulties include
           lack of access to certain geographical regions, difficult survey
           conditions, researchers' limited resources and training, and lack
           of consensus over sampling methods. In addition, the findings from
           the surveys in Darfur are not always publicly available, and few
           other reliable sources of mortality data exist in Darfur.

           o Lack of access to some geographical regions. Researchers' lack
           of access to some geographic areas in Darfur raises concerns about
           the completeness of the data collected. Because of security
           problems, humanitarian aid organizations that typically conduct
           the surveys, such as Medecins Sans Frontieres, and other
           researchers found it difficult to access all areas of Darfur,
           particularly South Darfur, according to several experts. In one
           instance, a survey conducted by the WHO in 2005 in South Darfur
           was suspended because of security concerns. Other surveys were
           also curtailed because of security concerns or attacks on NGO
           staff. In general, surveys were conducted primarily in camps where
           humanitarian relief was being provided, which could lead to
           underestimates of actual mortality. For example, the WHO and World
           Food Program note in their respective reports that their 2004
           surveys may underestimate mortality, owing to their samples'
           exclusion of people in inaccessible areas. Additionally, one
           expert reported that the Sudanese government at times placed
           restrictions on the relief organizations collecting data and
           limited their access to particular camps and regions.

           o Challenging survey conditions. Challenges in conducting
           household-based mortality surveys during and after humanitarian
           crises such as that in Darfur can affect the data's accuracy,
           consistency, and completeness and lead to over- or underreporting
           of mortality. Such challenges include linguistic and other
           cultural factors, difficulties resulting from the surveys'
           conflict or postconflict environment, and issues of recall and
           precision.

                        o Accurate translation of surveys into foreign
                        languages can be difficult.

                        o Surveys might not be conducted in all local
                        languages.49

                        o Definitions of a household vary.50

                        o Some people are reluctant to talk about death.

                        o Some cultures will not report the deaths of
                        infants.51

                        o If all household members die, none remain to be
                        surveyed.52

                        o The presence of government employees or parties to
                        the conflict can lead to over- or underreporting.

                        o A reluctance to forgo food rations may lead to
                        underreporting of deaths in the household.

                        o Dating deaths that occurred months prior to survey
                        can be difficult.53

                        o The length of the survey's recall period may lead
                        to under- or overreporting of deaths and affect the
                        precision of estimated mortality rates.54

                        o Identifying some causes of death can be
                        difficult.55

           o Insufficient training and resources among research staff.
           Difficulties in training staff and assembling resources may also
           have affected the quality of the surveys conducted. Mortality
           surveys require staff who are proficient in matters such as
           designing questionnaires, selecting samples of the local
           population,56 and conducting interviews in local languages.
           Although larger organizations that routinely conduct surveys
           generally have staff that are experienced in designing and
           implementing surveys, other groups may not have as much knowledge
           and experience in collecting data, according to experts and the
           research literature. Although the NGOs provide some training, high
           turnover rates make it difficult to retain the levels of knowledge
           and skill that are required.

           o Challenges related to sampling methods. The research literature
           notes that samples drawn in IDP or refugee camps may not provide
           an accurate count of deaths that occurred in attacks on villages
           and when IDPs were fleeing to the camps.57 However, although it
           recognizes the difficulty of selecting and implementing an
           appropriate sampling method to measure violent deaths outside
           camps, the literature does not offer any definitive solutions.

           In addition, the NGOs that conduct mortality surveys in Darfur do
           so primarily to monitor conditions in the camps they serve, and
           they generally do not disseminate the survey results. To address
           this problem, CE-DAT was established in 2003, under the
           Standardized Monitoring and Assessment of Relief and Transitions
           (SMART) initiative,58 to provide quick access to accurate and
           reliable data needed by humanitarian decision makers. However, the
           usefulness of the database is limited, because NGOs are not
           reporting the survey results systematically and because the
           quality of many of the reported surveys is problematic.

           Moreover, mortality data from other sources, such as prospective
           or ongoing surveillance systems59 that systematically record
           deaths, are generally unavailable in Darfur. The research
           literature notes that existing systems for registering or
           reporting deaths generally collapse when crises occur and that
           prospective surveillance systems established in crisis situations
           have limitations.60 However, the literature also notes that data
           from graveyard counts and regular reporting and surveillance
           systems have been used in some crises, despite such limitations,
           in conjunction with survey-based data to estimate mortality in
           crisis situations.61 In Darfur, data from surveillance systems
           were generally not available, although one expert reported that
           systems had been set up in some of the larger camps by the middle
           of 2004. However, the expert also reported that these systems were
           capturing data in ways that would allow them to be used only
           qualitatively, rather than quantitatively, for death toll
           estimates.
			  
			  Gaps in Data Leads to Reliance on Assumptions and Limited Information

           Gaps in data on mortality can lead to reliance on extrapolations
           based on potentially risky assumptions and limited contextual
           information. Because the available data for Darfur cover certain
           geographic areas and time periods, the estimates that we examined
           assume that mortality rates for surveyed locations and specific
           periods can apply to unsurveyed locations and longer periods. Most
           of the experts we consulted voiced concerns about such
           extrapolations. Some experts noted that factors such as patterns
           of attack and displacement, humanitarian relief efforts, and the
           incidence of disease might cause mortality rates in the surveyed
           areas to differ from rates in the unsurveyed areas. One expert
           stated that he could only speculate on conditions and mortality
           rates in the areas that had not been surveyed. In addition,
           several experts expressed concern about extrapolations from
           limited time periods to longer periods, noting that mortality
           rates can change rapidly.

           Some of the estimates' assumptions are informed by contextual
           information in the absence of data; however, this information also
           had limitations. For example, some estimates relied on anecdotal
           reports of conditions in Darfur, satellite imagery on attacks of
           villages, and information about weather conditions to determine
           appropriate mortality rates. While the experts generally approved
           of the use of contextual information, they did not believe that
           the information used could compensate for the gaps in the data.
			  
			  Limitations in Population Data Affect Reliability of Death Estimates

           Limitations in the population data before and during a crisis such
           as Darfur's can also impact researchers' ability to produce
           reliable death estimates. No definitive estimate of Darfur's
           pre-crisis population exists, and estimates of the current
           population vary considerably, from around 4 to 7 million. The
           difficulty of estimating the region's population is compounded by
           the fact that migration was widespread in Darfur before the
           crisis, making it difficult to get accurate counts. Because the
           death estimates essentially extrapolate the mortality rates from
           the surveys to the entire population affected by the crisis, an
           estimate of the affected population that is too high or too low
           will lead to an over- or underestimate of the death toll. The
           estimates we reviewed generally used data for the affected
           population that were reported in the UN Humanitarian Profiles;
           however, the collection and reliability of the profile data had
           limitations.62 For example, the profiles estimated a population of
           more than 160,000 IDPs in Kalma camp, the largest in Darfur, prior
           to October 2005. Yet, a subsequent count by a relief agency in
           October 2005 reported only 87,000 IDPs in the camp. An expert also
           noted that figures based on registration can be unreliable and
           depend greatly on the agency responsible for registration. He
           stated that sources of bias include poor coverage by the agency
           (underestimation), as well as problems with populations' being
           "double-counted" to increase the amount of aid delivered
           (overestimation).
			  
			  Varying Use of Baseline Mortality Rates Can Affect Estimatesï¿½
			  Accuracy

           Varying approaches to the selection of a baseline for normal
           mortality can raise or lower death estimates, possibly making the
           totals overly high or low. All but one of the Darfur death
           estimates that we examined subtract baselines for normal or
           expected mortality from the total deaths, based on the assumption
           that some deaths from disease, old age, or malnutrition would have
           occurred without the crisis. For example, CRED's use of a baseline
           mortality rate of 0.3 subtracted about 16,000 deaths from the
           total estimate of 134,000, resulting in 118,000 "excess" deaths.
           If CRED had used a baseline of 0.5, it would likely have
           subtracted about 26,000 deaths, obtaining a somewhat lower
           estimate of deaths from the crisis.63

           The experts we convened proposed various baseline mortality rates,
           ranging from 0.3 to 0.75 deaths per 10,000 affected persons per
           day.64 One reason for the difference in the suggested rates is the
           range of methods that the experts used to select them--for
           example, a rate that prevailed in the country before the crisis, a
           standard based on prior humanitarian crises, or a rate from a
           comparable country in the region. However, each of these methods
           has acknowledged limitations that could result in inaccurate
           estimates of expected rates and, therefore, in over- or
           underestimations of mortality due to the crisis.

           In addition, the experts debated whether a baseline of any sort
           was justified for a humanitarian crisis such as Darfur, arguing
           ethical and philosophical, rather than technical, considerations.
           About half of the experts said that deaths that would have
           occurred regardless of the crisis should be subtracted from the
           death toll attributed to the crisis. However, two experts took a
           contrary position, arguing that the concept of expected or normal
           levels of mortality was not appropriate in the presence of
           genocide or ethnic cleansing because the perpetrators of those
           crimes against humanity should be considered culpable for all
           deaths that resulted from the crises they instigated.65 Using a
           baseline to estimate mortality would lead to a somewhat smaller
           excess death toll than not using a baseline. For example, State's
           estimate of total deaths ranged from 98,000 to 181,000, minus
           35,000 expected deaths; thus, State estimated 63,000 to 146,000
           excess deaths directly resulting from the crisis.
			  
			  Wide Range of Measures Could Improve Death Estimates for Darfur
			  and Future Humanitarian Crises

           The group of experts proposed and rated a wide range of measures
           that U.S. agencies could take directly, or support other entities
           in taking, to improve the quality and reliability of death
           estimates and relevant data for Darfur and future humanitarian
           crises. The measure rated most likely to produce the most
           improvements was ensuring sufficient public documentation of
           estimates' data and methods to allow replication of the methods,
           verification of the findings, and confirmation of the estimates'
           credibility and objectivity. Other highly rated measures included
           collecting and maintaining temporal and spatial data, housing
           responsibility for making the estimates in a reputable independent
           body, improving the training of nongovernmental organizations'
           staff who collect survey mortality data, and promoting an
           interdisciplinary approach to estimating mortality. U.S. agencies
           are engaged in several initiatives that address some of these
           measures.
			  
			  Proposed Measures Cover Wide Range of Activities

           The experts suggested a series of measures covering a broad range
           of activities that U.S. agencies or other, U.S.-funded or
           -supported entities could take to improve death estimates for
           future humanitarian crises.66 (See table 2.) Although the experts
           acknowledged the importance and necessity of estimating
           mortality--for example, to help hold perpetrators accountable and
           to have a complete historical record--they generally believed that
           death estimates should be conducted with caution. To assist in
           this endeavor, the experts proposed and rated 19 measures as
           likely to produce some improvements in data collection and
           mortality estimation. Some experts differed as to whether
           government agencies or other entities would be best placed to
           implement particular suggestions.

32See John Hagan and Alberto Palloni, "Social Science: Death in Darfur,"
Science, vol. 313 (2006): 1578-1579. In addition, the experts did not
review a study of the Darfur conflict by Bloodhound, a Danish advocacy
group, because it was published after the experts met in April 2006. The
study estimates that deaths in Darfur ranged from 57,000 to 128,000 deaths
between April 2003 and September 2005 due to attacks on villages
throughout the region. [See Andreas Hoefer Petersen and Lise-Lotte Tullin,
The Scorched Earth of Darfur: Patterns in Death and Destruction Reported
by the People of Darfur. January 2001-September 2005 (Copenhagen:
Bloodhound, 2006). Available at http://www.bloodhound.se/rap_uk.html.]

33The WHO conducted an initial estimate in July 2004 and a subsequent
estimate in October 2004. Eric Reeves began his estimates in June 2004 and
has reported ongoing "mortality updates" since that time. As of October
2006, his last update was conducted at the end of April 2006.

Estimate (number of months in estimate)       Too high About right Too low 
Coebergh                                                                   
218,449 excess deaths (21)                           7           1       2 
253,573 excess deaths (21)                           8           0       2 
306,130 excess deaths (21)                           8           1       1 
CRED's two estimates                                                       
Sept. 2003-Jan. 2005: 134,000 total deaths           1           3       6 
(17)                                                                       
Feb. 2005-July 2005: 36,237 total deaths (5)         2           5       2 
Hagan et al.                                                               
396,563 total deaths (26)                           10           0       0 
Reeves                                                                     
Over 370,000 excess deaths (31)                     10           0       0 
State's lower- and higher-end estimates                                    
Lower-end estimate: 98,000 total deaths (23)         0           1       9 
Higher-end estimate: 181,000 total deaths            4           4       2 
(23)                                                                       
WHO's lower- and higher-end estimates                                      
Lower-end estimate: 45,000 total deaths (7)          1           2       7 
Higher-end estimate: 80,000 total deaths (7)         3           1       6 

34Although they questioned the accuracy of the estimates and had differing
views about the actual number of deaths that have occurred in Darfur, the
experts did not question the severity of the crisis. In discussing the
importance of the estimates' accuracy, several experts noted that
estimates can be used in war crime proceedings, and one stated that it was
important to account for all those who died so that this could become part
of the historical record.

35One expert rated having a moderate level of confidence in the estimate
by Hagan.

36In evaluating each estimate's methodological strength, the experts rated
the source data and whether such data were methodologically sound; the
methods, including whether extrapolations were appropriate, assumptions
were reasonable, and shortcomings that could result in over- or
underestimation were sufficiently described; the level of objectivity,
based on whether a particular bias appears to be part of the estimating
procedure; and the sufficiency of reporting and information contained in
published documents describing the estimate.

37CRED selected or excluded surveys for reasons such as whether findings
from a regional survey differed statistically from findings for statewide
surveys or whether a survey's findings were viewed as outliers. CRED used
the selected surveys to analyze violent or nonviolent mortality rates
depending on, for example, whether the surveys reported the proportion of
deaths due to violence. Although CRED primarily relied on mortality survey
data, it also used contextual information to refine its estimates and make
some assumptions.

38Nine experts rated the data CRED used as generally sound, two rated the
data as equally sound and unsound, and two rated the data as generally
unsound.

39CRED analyzed findings from the mortality surveys mentioned above to
separately estimate nonviolent and violent mortality rates and combined
these to get overall mortality rates for different regions and periods of
the conflict.

40Eight experts rated CRED's extrapolations as very or somewhat
appropriate, two experts rated them about as appropriate as inappropriate,
one expert rated them as somewhat inappropriate, and one expert found no
basis to judge or was not sure. Additionally, 10 experts rated CRED's
assumptions as very or somewhat reasonable, and two experts rated them as
somewhat or very unreasonable.

41Ten experts rated CRED's level of objectivity as very high or high, one
expert rated it as moderate, and one expert indicated that he or she had
no basis to judge or was not sure.

42State reports using various sources of information from sources other
than surveys, such as displacement patterns, village destruction, and
retrospective reporting of attacks to, among other things, fill in data
gaps and determine whether mortality rates derived from existing surveys
could be applied as an overall rate to a broader population or if higher
or lower rates were warranted.

43Seven of the 12 experts rated the State's data as generally
methodologically sound, 2 rated the data as equally sound and unsound, 2
rated the data as generally not sound, and 1 indicated that he or she was
not sure or had no basis to judge.

44Two experts rated the objectivity as moderate, and one said that he or
she had no basis to judge or was not sure.

45One indicated no basis to judge or not sure.

46Each of the three estimates based, at least in part, its numbers of
violent deaths on the "Atrocities Documentation Team's" survey of Chad
refugees and its numbers of nonviolent deaths on the 2004 WHO survey. The
Coebergh and Reeves estimates also used additional data sources.

47The 2004 WHO survey reported that violence or injury accounted for 10 to
21 percent of deaths in North and West Darfur and Kalma camp in South
Darfur for a 2-month recall period in the summer of 2004. Jan Coebergh and
John Hagan told us they knew that the WHO survey reported a percentage of
deaths due to violence but believed the survey did not sufficiently
capture violent deaths, particularly during the period prior to camp
arrival.

48Reeves's estimate assumes some change in mortality levels over time.

49In Darfur, for example, some surveys were conducted in Arabic and not in
other local languages.

50One expert stated that in Darfur, a household was defined as everyone
who ate at the same table the previous night. This definition is important
because household size is a factor in calculating crude mortality rates.

51One expert noted that certain cultures do not report the deaths of
unnamed infants and, in some of these cultures, naming does not occur
until an infant has lived for 40 days. Because the first 40 days of life
present the highest risk of mortality, unreported infant deaths could
compromise mortality estimates.

52Known as "survivor bias" in the research literature. For example, see
World Health Organization, "Module 4: Studying Health Status and Health
Needs." (Available at http://
www.who.int/hac/techguidance/tools/disrupted_sectors/module_04/en/index2.html
p.3)

53Known as "recall bias" in the research literature. For example, see
World Health Organization, "Module 4: Studying Health Status and Health
Needs." (Available at
http://www.who.int/hac/techguidance/tools/disrupted_sectors/module_04/en/index2.html
p.3.)

54For example, a survey conducted by the World Food Program in the fall of
2004 had a recall period of 7 months. However, because the average length
of displacement was 7.5 months, it is likely that the survey did not
capture all mortality that occurred prior to displacement, some of which
could have been due to violence.

55For example, identifying death from particular illnesses and diseases
can be difficult for respondents.

56The research literature also questions whether appropriate guidance and
training in sampling are routinely provided to field staff and whether
guidance is always followed when provided. See Paul B. Spiegel, Peter
Salama, Susan Maloney, and Albertine van der Veen, "Quality of
Malnutrition Assessment Surveys Conducted During Famine in Ethiopia,"
JAMA, vol. 292, no. 5 (2004).

57See Jennifer Leaning and Michael VanRooyen, "An Assessment of Mortality
Studies in Darfur, 2004-2005," Humanitarian, vol. 30 (June 2005). Also see
Michel Thieren, "Health Information Systems in Humanitarian Emergencies,"
Bulletin of the World Heath Organization, vol. 83, no. 8 (2005). For a
discussion of possible approaches to sampling in these situations, see
Holly Reed (Rapporteur), "Demographic Assessment Techniques in Complex
Humanitarian Emergencies," Summary of a Workshop (Washington, D.C.:
National Academy Press, 2002).

58SMART coordinated by USAID, involves experts from U.S. government
agencies, multilateral organizations, universities, and nongovernmental
organizations. SMART has issued guidance on how to measure mortality,
nutrition, and food security in crisis situations. CE-DAT serves as
SMART's primary data source for trend analysis and monitoring and
reporting.

59Mortality surveillance systems require teams of trained home visitors to
record deaths in the population as they occur. The home visitors are
assigned a specific sector of the camp or village and instructed to visit
their assigned areas regularly. During their visits, they record the
population size of the area and all deaths that have occurred.

60See World Health Organization, pp. 2-3, and Bradley Woodruff, "Violence
and Mortality in West Darfur," The Lancet, vol. 364 (2004), p. 1,290.

61See Bradley Woodruff. "Interpreting mortality data in humanitarian
emergencies," The Lancet, vol. 367 (2006), p. 9. Also see Romesh Silva and
Patrick Ball, "The Demography of Conflict-Related Mortality in Timor-Leste
(1974-1999): Empirical Quantitative Measurement of Civilian Killings,
Disappearances and Famine-related Deaths," in Human Rights and Statistical
Objectivity, J. Asher, D. Banks, F. Schueren, eds. New York: Springer
(forthcoming). Also see Francesco Checchi and Les Roberts, Interpreting
and Using Mortality Data in Humanitarian Emergencies: A Primer for
Non-Epidemiologists, Network Paper no. 52 (London: Humanitarian Practice
Network, 2005).

62Estimates of the affected population reported in the UN Humanitarian
Profiles are based on data provided by international humanitarian agencies
and their implementing partners; the majority of information comes from
the UN World Food Program food registration. These estimates do not
include residents in the three state capitals of Darfur, Nyala, El Fasher,
and Geneina because their number is relatively large compared with the IDP
population they are hosting. Officials from the UN Office for the
Coordination of Humanitarian Affairs told us that they had had difficulty
verifying the reliability of data reported and insuring the uniformity of
such data.

63The Darfur death estimates used different baseline mortality rates. For
example, CRED used a baseline of 0.3 because UNICEF reported that as a
national average prior to the conflict. State used 0.5 because that was
the WHO baseline mortality rate for sub-Saharan Africa.

64In addition, some experts felt that it was appropriate to use two rates,
one for the "normal" period and one for the "emergency" period of the
crisis. For example, one expert proposed 0.3 for the normal period and 0.6
for the emergency period.

65The remaining experts either presented views for or against or did not
comment on this issue. An expert who made this argument also noted that
the countries with higher rates of normal mortality would attribute
relatively more deaths to "normal" factors than in countries with lower
mortality rates.

Table 2: Measures Rated by Experts as Likely to Greatly Improve Death
Estimates for Future Crises, in Order of Ranking and Number of
Endorsements

Measure                                             Number of endorsements 
Ensure that publicly available documents on U.S.                           
government estimates provide sufficient information                        
on methods, data, assumptions, and limitations.                         11 
Support the collection and maintenance of temporal                         
and spatial data.                                                       10 
House responsibility for mortality estimates in a                          
reputable, independent body or group.                                    9 
Improve training of NGO staff who collect mortality                        
survey data.                                                             9 
Promote an interdisciplinary approach to estimating                        
mortality (include epidemiologists and                                     
demographers).                                                           9 
Create technical teams, under the auspices of an                           
international body, that can conduct mortality                             
estimates as needed.                                                     8 
Report mortality and morbidity information more                            
routinely and systematically to provide an ongoing                         
sense of the situation.                                                  8 
Promote data collection by NGOs on the ground at                           
routine service points in addition to periodic                             
assessments.                                                             7 
Promote the use of other measures of a conflict's                          
severity (e.g., displaced persons, number of                               
attacks) in addition to mortality estimates.                             7 
Improve existing surveying techniques (e.g.,                               
cluster sampling) by incorporating spatial or                              
temporal information.                                                    7 
Provide guidance (minimum standards) on how to                             
design and implement survey instruments in the                             
affected region (e.g., pretesting or translation                           
techniques).                                                             6 
Provide guidance on amalgamating existing                                  
mortality/morbidity surveys.                                             5 
Define criteria for selecting and using data.                            4 
Make satellite tapes and imagery available to                              
researchers.                                                             3 
Provide guidance on triangulating different types                          
of data.                                                                 3 
Provide assistance to local statistical agencies to                        
improve data collection.                                                 3 
Develop algorithms to track population change over                         
time.                                                                    3 
Create a statistical unit under appropriate agency                         
(possibly under auspices of the Committee on                               
National Statistics) to be responsible for these                           
types of estimates.                                                      3 
Tap other resources, such as pro bono groups of                            
statisticians, that could provide assistance.                            1 

66The experts suggested that the U.S. agencies best able to implement
these suggestions are the CDC, USAID, and State. Other, nongovernment
entities include academic institutions, such as the Johns Hopkins
University; multilateral organizations, such as the WHO; and NGOs such as
Medecins Sans Frontieres.

Source: GAO.

Note: The table is based on analysis of 12 experts in epidemiology,
demography, statistics, and the Darfur crisis. Numbers shown under "Number
of endorsements" represent the number of experts who rated the measure as
likely to "very greatly" or "greatly" to improve the estimates. In several
instances, one or two experts noted problems with suggestions that had
been highly rated by others. The experts rated an additional measure for
the Darfur crisis, namely, whether an independent agency should conduct a
retroactive assessment.

Majorities of the experts rated 10 of the 19 measures, and 9 of the 12
experts rated 5 of the measures, as likely to "very greatly" or "greatly"
improve the estimates for future crises.67 Following are the 10 most
highly rated measures:

           o Ensure that publicly available documents on the U.S. government
           estimates provide sufficient information on methodology.
           Sufficient documentation and transparency of data and methods are
           needed to allow independent researchers to verify the reliability
           and validity of estimates. Although this suggestion would not
           improve the estimates per se, experts felt that it was important
           for establishing the credibility of the estimates.

           o Support the collection and maintenance of temporal and spatial
           data. Temporal and spatial databases would allow researchers to
           track mortality over time and across regions and could improve
           researchers' ability to estimate mortality. Experts suggested that
           temporal and spatial data could be collected in a number of ways,
           including through the use of satellite imagery.

           o House responsibility for mortality estimates in a reputable,
           independent entity or group. Experts indicated that an independent
           entity could be perceived as free of bias, political goals, or
           both. However, such an entity would need adequate funding, and
           experts had differing views as to where it should be housed. One
           expert felt it could be placed under a U.S. organization such as
           the CDC, while another felt it should be under an international
           organization such as the WHO.

           o Improve training of NGO staff who collect mortality survey data.
           Because NGO staff conduct most of the surveys, improving their
           training would help improve the quality of data collected. Several
           experts reported the existence of initiatives to improve and
           standardize NGOs' data collection in emergencies, such as the
           SMART initiative, and of guidance on the topic from the World Food
           Program and the WHO. Some experts suggested expanding these
           efforts with additional U.S. government support.

           o Promote an interdisciplinary approach to estimating mortality
           (include epidemiologists and demographers). Because
           epidemiologists and demographers bring different expertise and
           perspectives to mortality estimation, collaboration between the
           disciplines could lead to greater understanding of each other's
           approaches and, ultimately, to more integrated methodologies for
           death estimates.

           o Create technical teams, under the auspices of an international
           body, that can conduct mortality estimates as needed. Technical
           teams, assembled as needed and operating under independent
           organizations, could--rather than a single, permanent independent
           entity--be responsible for conducting death estimates. Such teams
           might be perceived as free of bias and political goals, which
           would increase the credibility of their findings. However, some
           experts reported that such technical teams already exist, citing
           as examples the WHO, the International Rescue Committee, and
           Medecins Sans Frontieres, among others.68

           o Report mortality and morbidity information more routinely and
           systematically to provide an ongoing sense of the situation. This
           suggestion is intended to allow researchers and governments to
           monitor crises as they develop. Although this suggestion was
           relatively highly rated, experts had some questions about who
           would do the reporting, how the data would be gathered, and how
           the estimates would be made. One expert noted that groups like
           CRED are currently reporting such mortality and morbidity
           information somewhat routinely and systematically.

           o Promote data collection by NGOs on the ground at routine service
           points, in addition to periodic assessments. Proponents stated
           that humanitarian relief organizations could collect data on
           famine-related deaths and that human rights organizations could
           collect narrative testimonies about political violence and
           conflict-related displacement. Some experts believed that the NGOs
           could collect these data with little additional time or expense
           and that the data collection could help create an early warning
           system for famine and diseases. However, several experts were
           concerned that data collected at routine service points would
           provide biased estimates because certain segments of the affected
           population would not be likely to pass through these points.

           o Promote the use of other measures of a conflict's severity
           (e.g., displaced persons, number of attacks) in addition to
           mortality estimates. Such alternative estimates could provide
           insights into crises and help check the accuracy of the mortality
           data. These estimates could include conflict-related displacement,
           property destruction, property looting, exposure to violence as a
           marker for psychological trauma, incidence of sexual and
           gender-based violence, forced displacement, and data from security
           incidence reports. One expert indicated that some of these data
           are probably being collected by various human rights agencies and
           organizations and suggested creating a body to collate and report
           their data.

           o Improve existing surveying techniques (e.g., cluster sampling)
           by incorporating spatial or temporal information. A few experts
           felt that improvements in surveying techniques were needed because
           of limitations in certain aspects of cluster sampling, the most
           commonly used technique; however, another expert stated that
           cluster sampling69 was extremely useful. One expert argued that
           existing surveying techniques should take into account the spatial
           and temporal distribution of the affected population.
			  
			  Existing U.S. Initiatives May Improve Some Aspects of Data and Death
			  Estimates

           Several current U.S. initiatives may enhance the availability and
           quality of mortality data and produce more accurate death
           estimates for Darfur and other similar crises in the future. These
           initiatives embody several of the measures rated by the group of
           experts as likely to improve death estimates; however, several of
           the experts, as well as U.S. officials knowledgeable about the
           initiatives, whom we spoke with indicated that more can be done.

           o CE-DAT may improve death estimates through its promotion of more
           routine reporting of mortality and morbidity information and
           defining some criteria based on methodological requirements for
           assessing the quality of data. In addition, CE-DAT includes a
           cartographical database to link indicators to maps, which aligns
           with the experts' recommendation for collecting and maintaining
           spatial and temporal data. CE-DAT staff discussed some
           limitations, with respect to the data in the database, including
           the fact that information is not provided to CE-DAT routinely and
           CE-DAT is not widely known. CE-DAT staff also said that there is
           currently no systematic means of verifying data reliability and
           validity of data and making data publicly available in a timely
           manner.

           o SMART, an interagency initiative coordinated by USAID, may
           improve death estimates through its promotion of a standard
           methodology to design and implement survey instruments that
           measure crude mortality, as well as other indicators of need.70
           According to a USAID official, SMART is also intended to help
           build the technical capacity of NGO field workers and host
           government partners to collect and report more reliable data. To
           this end, the SMART initiative aligns with some of the
           recommendations made by our group of experts, including, improving
           the training of NGO staff that collect mortality data and
           reporting mortality and morbidity data more routinely and
           systematically. Experts and officials whom we spoke with generally
           agreed that the idea behind SMART represents positive steps toward
           improving mortality data collection. However, they also discussed
           various limitations of the initiative. For example, an official
           involved with the effort noted that that funding has been at times
           sporadic and that limited resources have stymied outreach efforts.
           Others stated that some of SMART's methodological principles need
           further research and testing in the field and that no consensus
           currently exists regarding some of these principles. Officials
           from USAID indicated that SMART's major components, including
           mortality, nutrition, and food security, have been piloted in
           several countries and that the SMART methodology is meant to be
           iterative with continuous upgrading based on further research and
           best practices.

           o The Humanitarian Information Unit, housed in State's Bureau of
           Intelligence and Research, may enhance the quality and
           availability of spatial and temporal information used in future
           death estimates through its current efforts to develop of maps and
           other visual images of humanitarian crises. For example, for the
           Darfur crisis, it has developed a series of maps showing the
           number of destroyed villages or reported attacks in the region at
           different points in time based on data from satellite imagery or
           reports from on the ground organizations such as the African
           Union, the UN, and USAID. The unit's work also addresses two
           additional suggested measures--promoting the use of other measures
           of conflict and improving existing surveying techniques by
           incorporating spatial and temporal data. However, the Humanitarian
           Information Unit, in some cases, has had difficulty obtaining
           standardized, reliable, or complete data. Additionally, the unit
           does not consistently and systematically collect data, such as
           preconflict population estimates, in part because the unit
           performs its work upon request.

           o The International Emergency and Refugee Health Branch of the CDC
           may improve death estimates and the collection of mortality data
           through its provision of technical assistance to UN and U.S.
           agencies collecting data in complex emergencies and its support of
           the collection of spatial and temporal data.71 However, officials
           from the International Emergency and Refugee Health Branch stated
           that because it often works as a consultative body and its
           resources are constrained, the scope of its work is limited.

           Several international initiatives also address problems with data
           collection and analysis. The WHO plans to implement a Health and
           Nutrition Tracking Service that would routinely monitor mortality
           and malnutrition during major crises and provide a central help
           desk, remotely located, for field staff conducting surveys.
           Additionally, the World Food Program has created a manual on
           survey techniques to measure health and nutrition indicators,
           including crude mortality.

           The group of experts we convened were generally aware of, or
           involved, with these U.S. government and international
           initiatives, as well as some NGO initiatives, that addressed
           certain aspects of their suggestions. For example, three experts
           assisted in the development of the SMART guidelines, and two of
           the three were also technical advisors for the CE-DAT initiative.
           Some experts felt that certain suggestions could be met by
           investing more in these existing initiatives rather than by
           creating new ones. U.S. officials responsible or aware of these
           initiatives also acknowledged limitations with several of these or
           thought that more could be done with respect to the measures
           suggested by the experts.
			  
			  Conclusions

           Despite variance among the death estimates we reviewed, each of
           the estimates shows, tragically and unquestionably, that many
           thousands of civilians died in Darfur between February 2003 and
           August 2005. Given the continuing conflict, insecurity, and
           displacement in the region, many more have undoubtedly died since
           the estimates were conducted, as a direct result of violence or
           because of increased vulnerability to disease and malnutrition.
           However, despite the importance of the death estimates in showing
           the severity of the crisis, none of the estimates consistently
           received high ratings in terms of accuracy or methodological
           strengths from the experts we convened. Some of the shortcomings
           in the estimates' source data, methods, and objectivity may be
           attributed to challenges in mortality data collection and
           extrapolation, characteristic of a humanitarian crisis such as
           Darfur. However, in certain cases, a lack of transparent reporting
           of an estimate's data, methods, assumptions, or limitations
           hindered the experts from replicating it and thus verifying its
           accuracy and credibility. In particular, many experts noted the
           unavailability of these published details--and the resulting lack
           of transparency--for State's estimate.

           Several ongoing U.S. initiatives may produce enhancements that
           align with the experts' suggested measures for addressing gaps in
           data and improving death estimates conducted for Darfur and any
           similar humanitarian crises in the future. For example, the SMART
           initiative and CE-DAT, respectively, provide guidance on how to
           design and implement surveys and promote more routine reporting of
           mortality and morbidity data. However, experts and U.S. government
           officials observed that gaps in these areas still exist. Among the
           measures that have not been addressed, the one that the experts
           rated most highly--ensuring the public availability of information
           on estimate methods, data, assumptions, and limitations--is
           essential to protect the credibility of U.S. government death
           estimates for Darfur and any future humanitarian crises.
			  
			  Recommendations for Executive Action

           To safeguard the Department of State's credibility as a source of
           accurate and reliable death estimates, we recommend that the
           Secretary of State promote greater transparency in any of its
           future death estimates for Darfur or other humanitarian crises by
           ensuring that publicly available documents contain sufficient
           detail on the estimates' data, methods, assumptions, and
           limitations to allow external researchers to replicate and verify
           the estimates.

           Additionally, to enhance the U.S. government's capacity to assess
           the dimensions of, and respond appropriately to, any future
           humanitarian crises, we recommend that the Secretary of State and
           the Director of U.S. Foreign Assistance and USAID Administrator
           consider the experts' other suggested measures to help address
           gaps in data and improve the quality and reliability of any future
           death estimates.
			  
			  Agency Comments and Our Evaluation

           We provided a draft of this report to the Department of State and
           USAID. State and USAID responded with formal comments, agreeing
           with our recommendations, and State provided additional
           perspectives on reporting and documentation regarding its death
           estimate. Reproductions of these letters, as well as our responses
           to the letters, can be found in appendixes VI and VII. We also
           provided a draft to the CDC for technical review, and we received
           technical comments from both the CDC and State, which we
           incorporated in the report as appropriate.

           We provided the authors of the other five estimates the portions
           of the report pertaining to their individual estimates. The
           authors of the CRED estimates agreed with the experts' evaluation.
           Jan Coebergh did not provide any major comments regarding the
           relevant portion of the report and indicated that he was aware of
           the limitations of his estimate. John Hagan expressed concerns
           regarding the experts' relatively positive evaluation of the
           estimates by CRED and State and believed we should have included
           the estimate he did with Alberto Palloni in our evaluation. David
           Nabarro from the WHO disagreed with our inclusion of his estimate
           with the others in the report and said that we had not clearly
           conveyed the estimate's purpose. Eric Reeves disagreed with the
           experts' criticism of the Atrocities Documentation Team's survey
           of Chad refugees, stating that the survey was well conducted and a
           critical source of data to estimate violent deaths, particularly
           early on in the conflict when little data was available. More
           details on the authors' comments and our responses can be found in
           appendix VIII.

           As agreed with your offices, unless you publicly announce the
           contents of this report earlier, we plan no further distribution
           until 30 days from the report date. At that time, we will send
           copies of this report to the Secretary of State, the Director of
           U.S. Foreign Assistance and USAID Administrator, the Director of
           the CDC, relevant congressional committees, and other interested
           parties. We will also make copies available to others on request.
           In addition, the report will be available on GAO's Web site at
           http://www.gao.gov.

           If you or your staff have any questions about this report, please
           contact either David Gootnick at (202) 512-3149 or
           [email protected], or Nancy Kingsbury at (202) 512-2700 or
           [email protected]. Contact points for our Offices of
           Congressional Relations and Public Affairs may be found on the
           last page of this report. Key contributors are listed in appendix
           IX.

           David Gootnick, Director
			  International Affairs and Trade

           Nancy Kingsbury, Managing Director			  
			  Applied Research and Methods
			  
			  Appendix I: Objectives, Scope, and Methodology

           This report (1) evaluates the relative accuracy and methodological
           strengths and shortcomings of six death estimates for Darfur,
           including one by the Department of State (State); (2) identifies
           general challenges to estimating the total death toll in Darfur
           and similar humanitarian crises; and (3) discusses measures that
           the U.S. government could take to improve its death estimates for
           Darfur and any similar, future crises.

           To evaluate the relative accuracy and methodological strengths and
           shortcomings of the Darfur death estimates, we selected estimates
           that had been made public prior to March 2006. We imposed this
           cutoff point so that we could interview the authors, prepare
           materials for the experts, and give the experts time to review the
           materials. Because our cutoff point was March 2006, the convening
           experts' discussion was of death estimates that spanned various
           time periods between February 2003 and August 2005. Our findings
           do not, therefore, discuss any estimates that were made after
           March 2006.

           Additionally, to address these objectives, we convened, in
           collaboration with the National Academy of Sciences, 12 experts
           for an all-day meeting on April 7, 2006, in Washington, D.C., to
           discuss the six death estimates. (Two additional experts also
           participated by phone for parts of the day.) The academy proposed
           lists of experts, and we approved their final selection. The
           selected experts had extensive knowledge of estimating mortality
           rates and death totals in conflict and postconflict situations or
           other types of humanitarian crises and were familiar with the
           sorts of data and estimates that have been produced for the Darfur
           crisis or other such crises in Africa. Additionally, the selected
           experts represented a range of professional experiences and
           backgrounds, including epidemiologists, demographers,
           statisticians, and directors of aid programs currently in Darfur.
           They were affiliated with various organizations, including
           universities, U.S. government and United Nations (UN) agencies,
           and humanitarian aid and nonprofit organizations based in the
           United States, as well as overseas. All of the experts signed a
           form from the National Academy of Sciences certifying that they
           had no conflicts of interest that could compromise their ability
           to assess the death estimates objectively. Some experts reported
           they had had professional contact with several of the authors.

           During the meeting, the experts discussed their evaluation of each
           of the Darfur death estimates, as well as challenges to estimating
           total deaths for Darfur and similar crises and measures to improve
           such estimates. The meeting was recorded and transcribed to ensure
           that we accurately captured the experts' statements, and we
           reviewed the transcripts as a source of evidence. (See app. II for
           a list of the experts' names and affiliations and a summary of the
           meeting agenda and discussion questions.)

           In addition, prior to the meeting, we reviewed and analyzed public
           information on the estimates, including documents describing the
           estimates and source data for the estimates and provided this
           information to the experts.1 The source data documents that we
           reviewed included 15 reports of health, nutrition, and mortality
           surveys; State publication describing the July-August 2004 survey
           with refugees in Chad; and volumes of the UN Nutrition Information
           in Crisis Situations containing descriptions of additional health,
           nutrition, and mortality surveys of which we were unable to obtain
           full reports. (See app. III for a complete list of published
           documents that were provided to the experts.) We also provided the
           experts supplemental information on the design, sampling and
           implementation of the Atrocities Documentation Team's survey of
           Chad refugees and a copy of the survey instrument, which we
           obtained from representatives of the Coalition of International
           Justice. Additionally, we interviewed the estimates' authors
           regarding the data, methods, and objectives of their work and
           replicated their estimates when we had sufficient information. We
           provided summaries of these interviews to the experts, as well as
           replications of the estimates, reviewed and approved by the
           authors, in advance of the meeting. We spoke with all of the
           estimate authors except State's Bureau of Intelligence and
           Research, which declined to speak with us or provide additional
           information. Although this limited the ability of our experts to
           fully understand State's methods of analysis as noted during their
           discussion, the experts were able to discuss State's estimate in
           detail and provide assessments of its data, methods, objectivity,
           limitations, and accuracy.

           Following the meeting, we also asked the experts to answer
           additional questions on each estimate, as well as questions on
           suggested measures to improve estimates through a follow-up data
           collection instrument. We developed the instrument with the help
           of survey specialists and based the questions in the instrument on
           the meeting agenda and points that arose during the meeting
           deliberations. We pretested the instrument with two experts and
           made changes based on their input. We administered the instrument
           via e-mail and received responses from all 12 of the experts who
           attended the meeting in Washington, D.C. In some instances, we
           contacted the respondents by e-mail or phone to obtain greater
           clarity or details regarding their answers.

           Further, to identify challenges involved in estimating total
           deaths in humanitarian crises, such as that in Darfur, we asked
           the group of experts to highlight key challenges during their
           discussion, and we reviewed literature related to death estimates
           and mortality data for humanitarian crises. We summarized the
           parts of the experts' discussion that most directly addressed
           challenges, and we identified themes that were raised by the
           experts. The literature we reviewed included articles that we
           identified through databases, such as ProQuest, Lexis Nexis, and
           Medline, using various search terms, such as mortality estimates,
           death estimates, humanitarian crise(s), conflict(s), or (complex)
           emergency/ emergencies. We also reviewed literature cited in these
           articles and on Web sites related to humanitarian assistance or
           data analysis in humanitarian crises, as well as literature
           recommended by the group of experts2 and the authors of the
           estimates. In total, we reviewed about 20 articles. We organized
           the individual challenges identified by the experts and in the
           literature according to the two overarching themes we identified
           from the experts' discussion and from the literature: (1)
           challenges affecting the source data used for the death estimates
           and (2) challenges affecting the generation of the death
           estimates. We also provided portions of the draft pertaining to
           the challenges to conducting death estimates as well as the
           background regarding mortality surveys to several of the experts
           we convened and incorporated their technical comments into the
           final version of the report.

           Finally, to identify measures that the U.S. government could take
           to improve death estimates for Darfur and in future humanitarian
           crises, we asked the experts to provide suggestions during the
           meeting and solicited further opinions on these suggestions in the
           follow-up instrument. The instrument listed 19 measures that the
           experts had suggested as likely to improve death estimates for
           Darfur and such crises that may occur in the future and asked the
           experts to rate them with a five-point scale, ranging from "Very
           greatly improve" to "Not improve." 3 We ranked the measures
           according to the numbers of respondents that rated them "Very
           greatly improve" and "Greatly improve." The instrument also asked
           for experts' comments on each of the suggested measures, and we
           followed-up with some experts to get additional information
           regarding comments on suggested measures, particularly with
           respect to current efforts. Additionally, we spoke with officials
           from the U.S. Agency for International Development, State, and the
           Department of Health and Human Services' Centers for Disease
           Control and Prevention to learn of any current U.S. government
           initiatives related to the suggested measures. We also reviewed
           public information and documents provided by the officials and
           available on the Internet that describe efforts by the U.S.
           government, as well as other international initiatives.

           We conducted our work in San Francisco and Washington, D.C., from
           September 2005 to November 2006 in accordance with generally
           accepted government auditing standards.
			  
1As noted in the report, we and the group of experts also reviewed a
preliminary death estimate for West Darfur by John Hagan and Alberto
Palloni, but we do not discuss the estimate in this report because the
estimate had not been published. The authors told us that the version of
the estimate they gave us was preliminary and that they were working on a
more refined version of the estimate using different methods.

2Over half of our experts were also authors or coauthors on some of the
articles we reviewed.

3The experts also rated an additional measure for the Darfur crisis
regarding whether an independent agency should conduct a retrospective
assessment.
			  
			  Appendix II: List of Experts and Meeting Agenda

           This appendix provides the names and affiliation of the experts
           participating in the meeting held on April 7, 2006, in Washington,
           D.C., and a summary of the day's discussion questions.
			  
			  List of Experts

           The following experts attended the meeting:

           o Jana Asher, American Association for the Advancement of Science

           o Richard Brennan, Health Unit, International Rescue Committee

           o Francesco Checchi, London School of Hygiene and Tropical
           Medicine

           o Allan Hill, Harvard School of Public Health, Harvard University

           o Arif Husain, Vulnerability Analysis and Mapping Unit, United
           Nations World Food Program

           o Mark Myatt, University College of London

           o W. Courtland Robinson, Bloomberg School of Public Health, Johns
           Hopkins University

           o William Seltzer, Department of Sociology and Anthropology,
           Fordham University

           o Romesh Silva, Human Rights Data Analysis Group, The Benetech
           Initiative

           o Michael VanRooyen, Program on Humanitarian Crises and Human
           Rights, Harvard University

           o Ronald Waldman, Mailman School of Public Health, Columbia
           University

           o Bradley Woodruff, Maternal and Child Nutrition Branch, U.S.
           Centers for Disease Control and Prevention

           The following experts participated in the meeting by phone for
           parts of the day:

           o Bushra Gamar Hussein, Darfur Region, Sudan Social Development
           Organization

           o Jennifer Leaning, Harvard School of Public Health, Harvard
           University
			  
			  Summary of Meeting Discussion Questions

           Darfur Death Estimates

           o Are the source data used in each of the estimates sufficiently
           representative and reliable?

           o To what extent do the methods used in each of the estimates
           follow principles that are generally accepted in the field?

           o Do the key assumptions made in the estimates seem reasonable?

           o How accurate or inaccurate do you think each of these estimates
           are in terms of representing the actual number of deaths that
           occurred in Darfur during the time period and regions under
           consideration?

           o For each of the estimates, is sufficient information presented
           in the reports to adequately assess the strengths and weaknesses
           and/or the reasonableness of the estimates? Do you understand the
           processes by which the estimates were derived?

           o What are the major strengths and limitations of each of these
           estimates for Darfur?

           Challenges to Estimating Total Deaths for Darfur and Similar
           Crisis

           o In your opinion, what sources of data for what time periods and
           locations would be necessary to produce reasonably precise
           estimates of death totals for Darfur?

           o What general observations do you have concerning the challenges
           of estimating total deaths in Darfur?

           Implications for the U.S. Government: Ways to Improve Death
           Estimates for Darfur and Future Crises

           o In your view, are there any lessons learned about conducting
           death estimates in Darfur for the U.S. government? If so, what are
           these?

           o Are there measures the U.S. government could take or promote to
           improve estimates for Darfur or similar crises in the future?
			  
			  Appendix III: List of Death Estimates and Mortality Surveys Provided
			  to Experts and Additional Bibliographical References
			  
			  Death Estimates1

           Estimate by the U.S. Department of State

           o State Department's Fact Sheet, "Sudan: Death Toll in Darfur,"
           March 25, 2005. [46]www.state.gov/s/inr/rls/fs/2005/45105.htm
           (Viewed, August 24, 2006)

           o M. Phelan in D. Guha-Sapir and O. Degomme, "Darfur: Counting the
           Deaths" (Method 2). Centre for Research on the Epidemiology of
           Disasters (CRED). Brussels: May 26, 2005.
           [47]www.cred.be/docs/cedat/DarfurCountingTheDeaths-withClarifications.pdf
           (Viewed October 23, 2006)

           Estimate by the Centre for Research on the Epidemiology of
           Disasters: Debarati Guha-Sapir and Olivier DeGomme

           o Guha-Sapir, D., O. Degomme, and M. Phelan, "Darfur: Counting the
           Deaths" (Method 1). Centre for Research on the Epidemiology of
           Disasters (CRED). Brussels: May 26, 2005.
           www.cred.be/docs/cedat/DarfurCountingTheDeaths-withClarifications.pdf
           (Viewed October 23, 2006)

           o Guha-Sapir, D. and O. Degomme, "Darfur: Counting the Deaths (2):
           What are the trends?" Centre for Research on the Epidemiology of
           Disasters (CRED). Brussels: December 15, 2005.
           [48]www.cred.be/docs/cedat/DarfurCountingTheDeaths2.pdf (Viewed
           October 23, 2006)

           Estimate by Jan Coebergh

           o Coebergh, J. "Sudan: genocide has killed more than the tsunami."
           Parliamentary Brief, vol. 9, no. 7, pp. 5-6 (February 2005).
           [49]www.thepolitician.org/ (Viewed, August 24, 2006)

           Estimate by John Hagan, Wenona Rymond-Richmond, and Patricia
           Parker and announced by the Coalition for International Justice

           o Coalition for International Justice Press Release, "New Analysis
           Claims Darfur Deaths Near 400,000." April 21, 2005.

           o Hagan, J., W. Rymond-Richmond, and P. Parker, "The Criminology
           of Genocide: The Death and Rape of Darfur," Criminology, vol. 43,
           no. 3, pp. 525-561 (July 2005).

           Estimate by Eric Reeves

           o Portions of mortality updates from [50]www.sudanreeves.org .
           (Reviewed by author.) Dr. Reeves began reporting his mortality
           updates in June 11, 2004. The most recent update reviewed by the
           experts was from August 31, 2005.

           Estimate by the World Health Organization

           o Nabarro, D. "Mortality Projections for Darfur," World Health
           Organization Briefing presented October 15, 2004.
           [51]www.who.int/mediacentre/news/briefings/2004/en (Viewed, August
           24, 2006)

           Surveys Used in Estimates as Data Sources
			  
			  State or regionwide estimates

           o U.S. Centers for Disease Control and Prevention and UN World
           Food Program, "Emergency Nutrition Assessment of Crisis Affected
           Populations Darfur Region, Sudan, August-September 2004."
           [52]www.reliefweb.int/library/documents/2004/wfp-sdn-26ocr.pdf .
           (Viewed, August 24, 2006)

           o UN World Food Program, "Emergency Food Security and Nutrition
           Assessment in Darfur, Sudan." (Rome, Italy: World Food Program,
           October 2004).
           [53]www.wfp.org/country_brief/africa/sudan/assessments/041025_food_assessment.pdf
           (Viewed, August 24, 2006)

           o World Health Organization, "Retrospective Mortality Survey Among
           the Internally Displaced Population Greater Darfur, Sudan 2004."
           September 15, 2004.
           [54]www.reliefweb.int/library/documents/2004/who-sdn-15sep.pdf
           (Viewed, August 24, 2006)

           o World Health Organization, "Mortality Survey among Internally
           Displaced Persons and Other Affected Populations in Greater
           Darfur, Sudan." September 2005.
           www.emro.who.int/sudan/pdf/CMS%20Darfur%202005%20final%20report_11%2010%2005.pdf
           (Viewed, August 24, 2006)

           North Darfur

           o Dubray, C., "Health assessment in emergencies: Serif Umra, North
           Darfur, Sudan." Epicentre and Medecins Sans Frontiers. November
           2004.

           o Sibson, V., "Findings of a nutrition survey of Kutum town, Kasab
           camp and Fata Borno, North Darfur." GOAL. May 2005.2

           West Darfur

           o Abadallah, S., "Report of rapid baseline survey Azirni, Sanidadi
           and Um Tagouk." Center for International Emergency, Refugee and
           Disaster Studies, Johns Hopkins University and World Relief.
           October 12, 2004.

           o Deconinick, H. and O. Karouri, "Nutrition assessment; Fur
           Baranga, Administrative Unit, Habila Locality, West Darfur,
           Sudan." Save the Children, USA. January 14-17, 2005.
           www.humanitarianinfo.org/darfur/uploads/assessments/fb%20nutrition%20survey%20report%20jan%2005.doc
           (Viewed, August 24, 2006)

           o Depoortere, E., F. Checchi, F. Broillet, S. Gerstl, A. Minettia,
           O. Gayraud et al., "Violence and mortality in West Darfur, Sudan
           (2003-2004): epidemiological evidence from four surveys" Lancet,
           vol. 364, no. 9442, pp. 1,315-1,320 (2004).
           [55]www.msf.fr/documents/base/2004-10-01-Depoortere.pdf (Viewed,
           August 24, 2006)

           o Depoortere, E., "Health assessment in emergencies: Murnei and
           Zalingei, West Darfur, Sudan." Epicentre and Medecins Sans
           Frontieres. (Paris, France: Medecins Sans Frontieres, June 2004).
           [56]www.doctorswithoutborders.org/publications/reports/2004/epicentre_
           report_darfur_05-2004.pdf (Viewed, August 24, 2006)

           o Hearns, A. and A. Agar, "Nutritional Survey, Mornei Camp,
           January-February 2005." Concern Worldwide in collaboration with
           State Ministry of Health and UNICEF.

           o Rio, D., "Nutrition and Food Security Assessment, Wade Saleh and
           Mukjar Provinces, West Darfur, North-Sudan." Medecins Sans
           Frontieres Holland. April 2004.

           o West Darfur State, Ministry of Health in collaboration with
           UNICEF, Concern and Save the Children-US. "Nutrition and Mortality
           Survey in Ardamat, Dorti, Riyad and Abu-zar IDP camps." June
           14-18, 2005.

           South Darfur

           o Grandesso, F., F. Sanderson, J. Kruijit, T. Koene, and V. Brown,
           "Mortality and Malnutrition among Populations Living in South
           Darfur, Sudan." JAMA, vol. 293, no. 12, pp. 1,490-1,494 (March
           2005).
           [57]http://jama.ama-assn.org/cgi/content/abstract/293/12/1490
           (Viewed, August 24, 2006)

           Chad

           o United Nations High Commissioner for Refugees et al., "Emergency
           Nutrition and Mortality Surveys Conducted among Sudanese Refugees
           and Chadian Villagers, Northeast Chad, June 2004."
           [58]http://www.cdc.gov/nceh/ierh/ResearchandSurvey/Chad_report04.pdf
           (Viewed, August 24, 2006)

           Other Information Provided

           o UN Nutrition Information in Crisis Situations, volumes 3, 4, 6,
           7 from 2004 and 2005.
           [59]http://www.unsystem.org/scn/publications/RNIS/countries/sudan.htm
           (Viewed, August 24, 2006)

           o Bureau of Democracy, Human Rights and Labor and Bureau of
           Intelligence and Research, Department of State, "Documenting
           Atrocities in Darfur," Publication 11182 (Washington, D.C.: Bureau
           of Democracy, Human Rights, and Labor and the Bureau of
           Intelligence and Research, September 2004).
           [60]http://www.state.gov/g/drl/rls/36028.htm (Viewed, August 24,
           2006)
			  
			  Additional Bibliographical References

           Boss, L., M. Toole, and R. Yip, "Assessment of Mortality,
           Morbidity and Nutritional Status in Somalia During the 1991-1992
           Famine: Recommendations for Standardization of Methods," JAMA,
           vol. 272, no. 5 (1994).

           Checchi, F. and L. Roberts. "Interpreting and using mortality data
           in humanitarian emergencies: A primer for non-epidemiologists,"
           Humanitarian Practice Network, Network Paper no. 52. (London, UK,
           2005).

           Coghlan, B., R. Brennan, P. Ngay, D. Dofara, B. Otto, M. Clements,
           and T. Stewart, "Mortality in the Democratic Republic of Congo: a
           nationwide survey," The Lancet, vol. 367, pp. 44-51 (2006).

           Guha-Sapir, D. and W. van Panhuis, "Conflict Mortality: Comparing
           the data." CE-DAT Working Paper, series no. CD/2004/001. (2004).

           Hofman, C., L. Roberts, J. Shoham, P. Harvey, "Measuring the
           impact of humanitarian aid: A review of current practice,"
           Humanitarian Policy Group. (London, UK, 2004).

           Leaning, J. and M. VanRooyen. "An assessment of mortality studies
           in Darfur, 2004-2005," Humanitarian Exchange, vol. 30, pp. 23-26
           (2005).

           Myatt M., A. Taylor, and W. Courtland Robinson, "A method for
           estimating mortality rates using previous birth history," Field
           Exchange, p. 13 (2002).

           Reed, H. (Rapporteur),"Demographic Assessment Techniques in
           Complex Humanitarian Emergencies, Summary of a Workshop." National
           Academy Press, Washington, D.C. (2002).

           Reed, H. and C. B. Keeley (eds.), Forced Migration and Mortality.
           National Academy Press, Washington, D.C. (2001).

           Salama, P., P. Spiegel, L. Talley, and R. Waldman, "Lessons
           Learned from complex emergencies over past decade," The Lancet,
           vol. 364, pp.1,801-1,813 (2004).p

           Silva, R. and P. Ball, "The Demography of Conflict-Related
           Mortality in Timor-Leste (1974-1999): Empirical Quantitative
           Measurement of Civilian Killings, Disappearances & Famine-Related
           Deaths." In Human Rights and Statistical Objectivity. J. Asher, D.
           Banks, F. Schueren, eds. New York: Springer (forthcoming).

           Spiegel, P., P. Salama, S. Maloney, and A. van der Veen, "Quality
           of Malnutrition Assessment Surveys Conducted During Famine in
           Ethiopia," JAMA, vol. 292, no. 5 (2004).

           Thieren, M., "Health information systems in humanitarian
           emergencies," Bulletin of the World Heath Organization, vol. 83,
           no. 8 (2005).

           Toole, M. and R. Waldman, "The Public Health Aspects of Complex
           Emergencies and Refugee Situations," Annual Review of Public
           Health, vol. 18, pp. 283-312 (1997).

           U.S. Centers for Disease Control and Prevention and The World Food
           Program, A Manual: Measuring and Interpreting Malnutrition and
           Mortality. Washington and Rome. (2005).

           Woodruff, B., "Violence and mortality in West Darfur." The Lancet,
           vol. 364, pg. 1,290 (2004).

           Woodruff, B., "Interpreting mortality data in humanitarian
           emergencies," The Lancet, vol. 367, pg. 9 (2006).

           World Food Program, "Emergency Food Security and nutrition
           assessment in Darfur, Sudan 2005." Rome. (December 2005).
           documents.wfp.org/stellent/groups/public/documents/ena/wfp089682.pdf
           (Viewed October 23, 2006)

           World Health Organization, "Module 4. Studying Health Status and
           Health Needs."
           www.who.int/hac/techguidance/tools/disrupted_sectors/module_04/en/index2.html
           (Viewed October 23, 2006)

67Experts rated the measures using the following 5-point scale: "Very
greatly improve," "Greatly improve," Moderately Improve," "Somewhat
Improve," and "Not improve."

68One expert felt that it would be preferable for the U.S. government to
support the existing technical teams.

69Cluster sampling is a simple method that can be used to draw a
representative sample even where there is no listing, or known total
number, of households (that is, no individual household sampling frame)
and where households are arranged in a chaotic pattern, as if often the
case in IDP camps. See Francesco Checchi and Les Roberts, Interpreting and
Using Mortality Data in Humanitarian Emergencies: A Primer for
Non-Epidemiologists, Network Paper no. 52 (London: Humanitarian Practice
Network, 2005).

70SMART initiative is aimed at improving the monitoring, reporting, and
evaluation of humanitarian assistance interventions. (See
[61]www.smartindicators.org .)

71For example, members from the International Emergency and Refugee Health
Branch provided technical training on data collection to the World Food
Program for two mortality surveys conducted in Darfur.

1As noted in the report, the experts reviewed a death estimate of West
Darfur by John Hagan and Alberto Palloni, but this estimate is not
discussed in this report because it was not publicly available.

2Survey included in CRED December 2005 update. Note: the report included
here was only publicly available in HTML format. Therefore, some of the
appendixes and formatting did not come through.

Appendix IV: Summary Description of Death Estimates Reviewed

Estimate by Jan Coebergh

Reported figures            o Three possible point estimates for excess    
                               deaths: 218,449; 253,573;or 306,130 represent  
                               a range of death estimates all above 200,000.  
Time period covered         o April 2003 through December 2004.            
Total number of months      o 21                                           
Data sources                o Uses varied sources including Atrocities     
                               Documentation Team's survey of Chad refugees   
                               based on number of people who reported seeing  
                               family member killed, 2004 mortality surveys   
                               by World Health Organization (WHO) and         
                               Medicins Sans Frontiers and estimates by WHO   
                               officials.                                     
                               o Does not use any contextual or nonsurvey     
                               information.                                   
Mortality rates applied     o Calculates violent and nonviolent deaths     
and cause of death          separately.                                    
                               o 306,130: Includes nonviolent and violent     
                               deaths. About 170,000 from violence, 108,000   
                               from health causes, and 25,000 in inaccessible 
                               areas.                                         
                               o 218,449: Includes nonviolent and violent     
                               deaths. About 73,700 from violence and 126,000 
                               from health causes                             
                               o 253,573: Includes nonviolent and violent     
                               deaths. About 111,000 from health causes and   
                               about 143,000 from violence                    
                               o Applies some daily mortality rates; other    
                               mortality rates are monthly or for longer      
                               periods of time.                               
Mortality changes over      o Assumes the same mortality levels over time  
time and/or region          and per region.                                
Baseline crude mortality    o 0.5                                          
rate assumed                                                               
Affected population         o Uses one "fixed" population estimate of      
included in estimate        400,000 which is the average of an assumed     
                               starting point of 0 in February 2003 and       
                               800,000 in April 2004. (Affected population    
                               does not change over time.)                    
                               o Estimate includes affected population from   
                               all regions in Darfur and refugees in Chad.    

Centre for Research on the Epidemiology of Disasters (CRED): Two Estimates

Reported figures    o 134,000 total deaths.       o 36,237 total deaths.   
                       o 118,142 excess deaths.      o 23,658 excess deaths.  
Time period         o September 2003 through      o February 2005 through  
covered             January 2005.                 June 2005.               
Number of months    o 17                          o 5                      
Data sources        o Examines data from about    o Examines about 10      
                       20 mortality surveys.         surveys conducted        
                       About half of these were      between January and      
                       used to estimate              August 2005; however,    
                       nonviolent mortality          primarily uses findings  
                       rates, and half were used     from the 2005 statewide  
                       to analyze the proportion     WHO survey because these 
                       due to violence. (Surveys     results did not differ   
                       used depended on what         with other smaller       
                       surveys reported the          surveys that covered the 
                       proportion of deaths due      same period.             
                       to violence.)                                          
                       o Some steps were taken to                             
                       examine the reliability of                             
                       methods used in the                                    
                       surveys and exclude                                    
                       outliers.                                              
                       o Relies mainly on mortality survey findings and uses
                       other sources of contextual or nonsurvey information
                       to refine both estimates.  
Mortality rates     o Estimates nonviolent and violent deaths separately.
applied and         Total crude mortality is nonviolence crude mortality
cause of death      divided by (1 minus the proportion due to violence).
                       o Nonviolent mortality is     o 2005 WHO survey        
                       generally assumed stable      findings include crude   
                       except for the hunger (or     mortality rates and      
                       rainy season) during June     proportion of deaths due 
                       through August 2004.          to violence or injury.   
                       o Proportion due to                                    
                       violence is based on a                                 
                       regression analysis of                                 
                       survey data.                                           
Mortality           o Nonviolent mortality        o Assumes different      
changes over        rate is assumed stable at     mortality levels per     
time and/or         1.1 for all regions except    region but no difference 
region              June through August 2004      over time from February  
                       for which rates are higher    to June 2005.            
                       due to impact of rainy        o Shows increase in      
                       season. (Rates used are       violence for North       
                       based on results from the     Darfur from last         
                       2004 WHO survey for each      estimate.                
                       region.)                                               
                       o Based on regression                                  
                       analysis, proportion of                                
                       violence generally                                     
                       decreases over time                                    
                       beginning December 2003 to                             
                       January 2005. Prior to                                 
                       December 2003, assumes                                 
                       steady rate of violence.                               
                       o Assumes some difference                              
                       in violence between                                    
                       regions.                                               
Baseline crude      o 0.3 based on national    
mortality rate      average from the United    
assumed             Nations Children's Fund    
                       (UNICEF).                  
Affected            o Assumes different           o Assumes different      
population          monthly estimates of the      monthly estimates        
included in         affected population based     separated by internally  
estimate            on information from the       displaced persons (IDP)  
                       United Nations                and affected residents   
                       Humanitarian Profiles for     based on information     
                       each Darfur state from        from the United Nations  
                       September 2003 till           Humanitarian Profiles.   
                       January 2005.                 o Includes all regions   
                       o Excludes deaths from        in Darfur.               
                       isolated areas with high      o Applies different      
                       violence rates in South       mortality rates to       
                       Darfur after September        groups of affected       
                       2004.                         residents, IDPs in       
                       o Adds additional 200,000     camps, IDPs outside      
                       for Chad refugees; their      camps (did not do this   
                       deaths estimated as 10% of    in first estimate).      
                       the total excess deaths                                
                       estimated.                                             

Estimate by John Hagan and Others and Released by the Coalition for
International Justice

Reported figures            o 396,563 total deaths.                        
Time period covered         o February 2003 through March 2005.            
Total number of months      o 26                                           
Data sources                o Uses Atrocities Documentation Team's survey  
                               of Chad refugees to estimate violent deaths    
                               based on number of people who reported seeing  
                               a family member killed.                        
                               o Uses 2004 WHO mortality survey findings for  
                               North and West Darfur to estimate nonviolent   
                               deaths.                                        
                               o Does not use any contextual or nonsurvey     
                               information.                                   
Mortality rates applied     o Nonviolent mortality rate is on a            
and cause of death          combination of findings from North and West    
                               Darfur WHO survey.                             
                               o Violent mortality rate is calculated based   
                               on survey of Chad refugees and assumptions     
                               include that all missing persons have died and 
                               that the average family size is five.          
Mortality changes over      o Assumes the same mortality levels over time  
time and/or region          and per region.                                
Baseline crude mortality    o 0 (Did not apply a baseline mortality rate   
rate assumed                because believed it was not appropriate.)      
Affected population         o Uses a single estimate of 1.5 million, as of 
included in estimate        April 2005, to include IDP population from all 
                               regions in Darfur and refugees in Chad. 1.5    
                               million is the midpoint between 1.2 and 1.8    
                               million. (1.2 comes from State's Documenting   
                               Atrocities Report, and 1.8 was stated as       
                               widely cited as the affected population in     
                               early 2005.)                                   

Estimate by Eric Reeves

Reported figures            o Over 370,000 excess deaths.1                 
Time period covered         o February 2003 through August 2005.           
Number of months            o 31                                           
Data sources                o Uses varied sources such as the Atrocities   
                               Documentation Team's survey of refugees in     
                               Chad, the 2005 WHO mortality survey,           
                               communication from David Nabarro, and          
                               projections from the Department of U.S. Agency 
                               for International Development.                 
                               o Uses contextual or nonsurvey data in various 
                               calculations, such as mortality rates for      
                               inaccessible populations.                      
Mortality rates applied     o Generally estimates nonviolent and violent   
and cause of death          deaths separately.                             
                               o Survey of Chad refugees to estimate violent  
                               deaths based on number of people who reported  
                               seeing family member killed.                   
                               o Estimates some mortality rates per day;      
                               estimates other rates per month or longer      
                               periods of time.                               
Mortality changes over      o Assumes some changes in mortality over time. 
time and/or region          Generally does not assume different mortality  
                               rates per region except for populations in     
                               inaccessible areas.                            
Baseline crude mortality    o 0.3 based on UNICEF.                         
rates                                                                      
Affected population         o Assumes affected population changes somewhat 
included in estimate        over time, although not on a monthly basis.    
                               Uses information from UN profiles, as well as  
                               other sources. For example, in some cases,     
                               assumes some groups are excluded from UN       
                               profiles and increases numbers accordingly.    
                               o Includes affected population from all        
                               regions in Darfur and Chad refugees.           

1This figure is from an August 2005 estimate that was the latest available
at the time the group of experts convened, and this was reviewed. At the
end of April 2006, Eric Reeves provided a new estimate of over 450,000
people have died from violence, disease, and malnutrition.

Department of State Estimate

Reported figures        o 98,000 to 181,000 total deaths.                  
                           o 63,000 to 146,000 excess deaths.                 
Time period covered     o March 2003 through January 2005.                 
Total number of         o 23                                               
months                                                                     
Data sources            o Uses data deemed sound from available mortality  
                           surveys in the Complex Emergency Database at that  
                           time for Darfur and refugees in Chad (refers to    
                           list of about 20 surveys included in report).      
                           o Provides some information regarding which        
                           surveys and contextual data sources were used and  
                           why.                                               
                           o Applies contextual information to fill in data   
                           gaps and determine if existing mortality rates     
                           should be applied to a broader population within   
                           that region or if a higher or lower rate was       
                           warranted. Refers to information such as           
                           displacement patterns, fire mapping remote sensing 
                           data, reporting of attacks, historical trends on   
                           humanitarian intervention, and disease outbreaks.  
Mortality rates         o Estimates high and low mortality rates that are  
applied and cause of    assumed to include mortality due to all causes     
death                   (nonviolent and violent).                          
                           o Reports that some mortality rates were adjusted  
                           based on contextual information (e.g., report of   
                           attacks). For example, reports using West Darfur   
                           survey data as a basis for estimating mortality in 
                           North Darfur are based on nonsurvey data showing   
                           similar patterns of conflict in 2003 and early     
                           2004, but estimates of higher total deaths in      
                           North Darfur are due to larger displaced           
                           population and more incidents of fighting.         
Mortality changes       o Assumes differences in mortality over time and   
over time and/or        per region.                                        
region                  o Divides conflict into four time periods: (1) the 
                           initial outbreak of violence (March through        
                           September 2003), (2) the breakdown of              
                           cease-fire/escalation of conflict (October 2003    
                           through March 2004), (3) the second cease-fire     
                           (April through June 2004) and (4) increased        
                           international humanitarian response (July 2004     
                           through January 2005).                             
                           o Reports that highest mortality and level of      
                           violence occurred during December 2003 to February 
                           2004. Mortality began falling in March 2004.       
Baseline crude          o Uses 0.5 deaths per 10,000 per based on          
mortality rate          estimates from the WHO of sub-Saharan African      
assumed                 populations.                                       
Affected population     o Assumes different monthly estimates (separated   
included in estimate    by displaced and affected populations) based on UN 
                           profiles for each Darfur state from September 2003 
                           until January 2005.                                
                           o Assumes an incremental increase prior to         
                           September 2003 because no source of reliable       
                           information was available. Uses data from the      
                           United Nations High Commissioner for Refugees      
                           (UNHCR) on spontaneous settlements and camps to    
                           estimate Chad population.                          
                           o Assumes affected residents experienced mortality 
                           rates that were 20% lower than displaced           
                           populations.                                       

WHO Estimate

Reported figures            o 45,000 to 80,000 total deaths.               
                               o 35,000 to 70,000 excess deaths.              
Time period covered         o March 2004 through September 2004.           
Total number of months      o 7                                            
Data sources                o Primarily uses findings from 2004 WHO        
                               mortality survey, although looked at other     
                               surveys conducted.                             
                               o Uses contextual information, such as the     
                               occurrence of disease outbreaks and the level  
                               of humanitarian assistance, to help refine     
                               estimates and define best and worst case       
                               scenarios.                                     
Mortality rates applied     o Estimates mortality rates include all causes 
and cause of death          of death (violent and nonviolent).             
Mortality changes over      o Estimates an average mortality rate for the  
time and/or region          entire region of Darfur. Does not assume       
                               different mortality rates over time or per     
                               region.                                        
Baseline crude mortality    o 0.5                                          
rate assumed                                                               
Affected population         o Assumes different monthly estimates of IDPs  
included in estimate        based on UN profile reports.                   
                               o Estimate includes only IDPs in camps and     
                               excludes affected residents from all Darfur    
                               regions and refugees in Chad.                  

Source: GAO analysis based on published documents of estimates and
interviews with estimate authors.

Note: The summaries of these estimates are based on the published
information, as well as interviews with and additional information
provided by estimate authors with the exception of State who declined to
speak with us. Additionally, mortality rates discussed in table are crude
mortality rates per 10,000 per day.

Appendix V: Additional Follow-Up Survey Results

This is a summary of responses to additional questions from our follow-up
survey from the 12 experts we convened with the National Academy of
Sciences in April 2006.

Appendix VI: Comments from the Department of State

Note: GAO comments supplementing those in the report text appear at the
end of this appendix.

See comment 2.

See comment 1.

Following are GAO's comments on the Department of State's letter dated
October 4, 2006.

GAO Comments

           1. State asserts that its death estimate was intended for internal
           purposes. We maintain that because State's estimate was publicly
           available and discussed by State officials, sufficient detail on
           this and any future such estimates is necessary to safeguard
           State's credibility as a source of accurate and reliable death
           estimates, particularly where such a serious topic is concerned.
           2. When asked, the CRED authors indicated that State provided them
           with the text on the methodology as included in the published
           report, but did not provide source information on the data used in
           State's estimate.

Appendix VII: Comments from the U.S. Agency for International Development

Appendix VIII: Summary of Authors' Comments

To the five other estimate authors, we provided portions of the report
pertaining to their individual estimates. We summarize their comments and
our responses below.

           o The authors of the Centre for Research on the Epidemiology of
           Disasters (CRED) estimates agreed with the experts' evaluation.

           o Jan Coebergh did not provide any major comments regarding the
           relevant portion of the report and indicated that he was aware of
           the limitations of his estimate.

           o John Hagan expressed concern regarding the experts' relatively
           positive evaluations of the estimates by CRED and the Department
           of State (State), stating that the respective estimates' published
           documentation does not provide sufficient information on the data
           and methodologies used. As discussed in the report, the experts
           noted limitations in the sufficiency of reporting for both these
           estimates, but they were able to discuss these estimates in detail
           and assess their respective levels of accuracy and methodological
           strengths and shortcomings. In addition to providing the experts
           the published reports containing the estimates, we provided them
           information regarding the source mortality surveys used in the
           estimates, as listed in appendix III. For the CRED estimate, we
           also provided the experts with additional details about the
           methodology based on our interviews with the authors. Dr. Hagan
           also believed that we should have included in our evaluation the
           estimate he did with Alberto Palloni published in the September
           2006 issue of Science. We added information on the key findings of
           this estimate to the report, but as we note in the report, it does
           not include an analysis of the estimate because a final publicly
           available version was not available when the experts convened in
           April 2006.

           o Eric Reeves disagreed with the experts' criticism of the
           Atrocities Documentation Team's survey of Chad refugees. He stated
           this survey was well conducted and a critical source of data to
           estimate violent deaths especially early in the conflict when
           little information was available. In the report, we note that the
           literature acknowledges difficulties in capturing deaths due to
           violence. Nevertheless, some of the estimates reviewed by the
           experts used other techniques to account for violent deaths, such
           as statistical analysis or contextual information. Moreover, many
           experts felt that Atrocities Documentation Team's survey was not a
           reliable or appropriate source of data to estimate violent deaths
           for a cumulative death estimate on Darfur, based on public
           documentation on the survey, as well as supplemental information
           on the survey's design, implementation, and sampling we obtained
           from representatives at the Coalition for International Justice.
           In addition, Dr. Reeves also expressed a concern regarding the
           2004 survey by WHO and believed it did not sufficiently capture
           violent mortality in areas outside camps. To address this concern,
           we added information regarding the issue of accessibility as a
           potential source for underestimation of mortality and specifically
           discuss this survey. Dr. Reeves also stated that in a draft
           version of the report, we had incorrectly characterized his
           estimate's use of the 2004 WHO survey to account for only
           nonviolent deaths and, accordingly, we removed this
           characterization from the report.
           o Regarding the WHO estimate, David Nabarro emphasized in his
           comments that the estimate was undertaken to provide a rough order
           of magnitude of deaths in order to facilitate humanitarian relief
           efforts. He also pointed out that it applied to a shorter time
           period than the other estimates and was restricted to IDPs located
           in camps, without attempting to account for deaths in the larger
           population. We provided additional clarification in the report to
           emphasize the more restricted scope and coverage of the WHO
           estimate. Further, we stated that the purpose of the WHO estimate
           was to provide an order of magnitude estimate to assist in
           humanitarian relief planning.

Appendix IX: GAO Contacts and Staff Acknowledgments

GAO Contacts

David Gootnick, (202) 512-3149, [email protected] Nancy Kingsbury,
(202) 512-2700, [email protected]

Staff Acknowledgments

In addition to the individuals named above, Emil Friberg, Assistant
Director; Jim Ashley; Martin de Alteriis; Etana Finkler; Leslie Holen;
Theresa Lo; Reid Lowe; Grace Lui; John F. Miller; and Chhandasi Pandya
made key contributions to this report.

(320420)

GAO's Mission

The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.

Obtaining Copies of GAO Reports and Testimony

The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site ( www.gao.gov ). Each weekday, GAO posts
newly released reports, testimony, and correspondence on its Web site. To
have GAO e-mail you a list of newly posted products every afternoon, go to
www.gao.gov and select "Subscribe to Updates."

Order by Mail or Phone

The first copy of each printed report is free. Additional copies are $2
each. A check or money order should be made out to the Superintendent of
Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more
copies mailed to a single address are discounted 25 percent. Orders should
be sent to:

U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548

To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061

To Report Fraud, Waste, and Abuse in Federal Programs

Contact:

Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail:
[email protected] Automated answering system: (800) 424-5454 or (202)
512-7470

Congressional Relations

Gloria Jarmon, Managing Director, [email protected] (202) 512-4400 U.S.
Government Accountability Office, 441 G Street NW, Room 7125 Washington,
D.C. 20548

Public Affairs

Paul Anderson, Managing Director, [email protected] (202) 512-4800
U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548

www.gao.gov/cgi-bin/getrpt?GAO-07-24 .

To view the full product, including the scope and methodology, click on the link above.

For more information, contact David Gootnick at (202) 512-3190 or
[email protected].

Highlights of [70]GAO-07-24 , a report to congressional requesters

November2006

DARFUR CRISIS

Death Estimates Demonstrate Severity of Crisis, but Their Accuracy and
Credibility Could Be Enhanced

In 2003, violent conflict in Darfur, Sudan, broke out between rebel groups
and government troops and government-supported Arab militias. While few
would dispute that many thousands of Darfur civilians have died, less
consensus exists about the total number of deaths attributable to the
crisis. Estimates by the Department of State (State) and other parties
report death tolls up to about 400,000 for varying populations and periods
of time between February 2003 and August 2005. Based on the views of
experts convened by GAO and the National Academy of Sciences, interviews
with estimate authors, and a review of relevant literature, this report
(1) evaluates six Darfur death estimates, (2) identifies general
challenges to estimating deaths in such crises, and (3) discusses measures
to improve death estimates.

[71]What GAO Recommends

To safeguard the U.S. government's credibility as a source of reliable
death estimates, GAO recommends ensuring greater transparency regarding
the data and methods used for such estimates. GAO also recommends that the
U.S. government consider other measures suggested by the experts to help
address gaps in data and improve the quality of any future death
estimates. State and the U.S. Agency for International Development agreed
with GAO's recommendations.

The experts GAO consulted did not consistently rate any Darfur death
estimate as having a high level of accuracy; moreover, they noted that all
the studies had methodological strengths and shortcomings. Most of the
experts had the highest overall confidence in estimates by the Centre for
Research on the Epidemiology of Disasters (CRED) and had a slightly lower
level of confidence in State's estimate. Many experts believed State's
lower-end estimate was too low. Additionally, the published documents
describing State's estimate lacked sufficient information about its data
and methods to allow it to be replicated and verified by external parties.

Estimating deaths in a humanitarian crisis such as that in Darfur involves
numerous challenges. For example, in Darfur, difficulties in collecting
mortality data, such as lack of access to particular geographical regions,
impacted the data's quality and led to data gaps. Because of such data
gaps, some Darfur death estimates relied on potentially risky assumptions
and limited contextual information. Further, limitations in estimates of
Darfur's population before and during the crisis may have led to over- or
underestimates of the death toll. Finally, varying use of baseline
mortality rates--the rate of deaths that would have occurred without the
crisis--may have led to overly high or low death estimates.

The experts proposed and rated a wide range of measures that U.S. agencies
could take to improve the quality and reliability of death estimates for
Darfur and future humanitarian crises. Among these measures, the most
highly rated was ensuring that public documentation of the data and
methods used contain sufficient information to enable external replication
and verification of the estimates. Other very highly rated measures
include collecting and maintaining data for specific periods of time and
geographic areas and housing the responsibility for making estimates in a
reputable independent body.

Darfur Village Burning

References

Visible links
  42. http://www.gao.gov/cgi-bin/getrpt?GAO-07-9
  43. http://www.state.gov/g/drl/rls/36028.htm
  44. http://www.gao.gov/cgi-bin/getrpt?GAO-07-9
  45. http://www.cred.be/cedat/index.htm
  46. http://www.state.gov/s/inr/rls/fs/2005/45105.htm
  47. http://www.cred.be/docs/cedat/DarfurCountingTheDeaths-withClarifications.pdf
  48. http://www.cred.be/docs/cedat/DarfurCountingTheDeaths2.pdf
  49. http://www.thepolitician.org/
  50. http://www.sudanreeves.org/
  51. http://www.who.int/mediacentre/news/briefings/2004/en
  52. http://www.reliefweb.int/library/documents/2004/wfp-sdn-26ocr.pdf
  53. http://www.wfp.org/country_brief/africa/sudan/assessments/041025_food_assessment.pdf
  54. http://www.reliefweb.int/library/documents/2004/who-sdn-15sep.pdf
  55. http://www.msf.fr/documents/base/2004-10-01-Depoortere.pdf
  56. http://www.doctorswithoutborders.org/publications/reports/2004/epicentre_report_darfur_05-2004.pdf
  57. http://jama.ama-assn.org/cgi/content/abstract/293/12/1490
  58. http://www.cdc.gov/nceh/ierh/ResearchandSurvey/Chad_report04.pdf
  59. http://www.unsystem.org/scn/publications/RNIS/countries/sudan.htm
  60. http://www.state.gov/g/drl/rls/36028.htm
  61. http://www.smartindicators.org/

  70. http://www.gao.gov/cgi-bin/getrpt?GAO-07-24
*** End of document. ***