Legal Services Corporation: More Needs to Be Done to

Legal Services Corporation: More Needs to Be Done to Correct Case Service
Reporting Problems (Letter Report, 09/20/1999, GAO/GGD-99-183).

Pursuant to a congressional request, GAO determined: (1) what efforts
the Legal Services Corporation (LSC) and its grantees have made to
correct problems with case service reporting; and (2) whether these
efforts are likely to resolve the case reporting problems that occurred
in 1997.

GAO noted that: (1) LSC revised its written guidance and issued a new
handbook to its grantees to clarify case reporting requirements; (2)
based on telephone interviews with a sample of 79 LSC grantee executive
directors, GAO estimates that 90 percent of grantees viewed the new
guidance as having clarified reporting requirements, overall; (3)
virtually, all grantees said they responded to the new requirements by
making or planning to make one or more changes to their program
operations; (4) however, many grantees indicated that they were unclear
about certain aspects of LSC's reporting requirements, particularly
regarding: (a) the specific information required on client assets; (b)
the information required for documenting citizenship/alien eligibility
for services provided over the telephone; (c) the criteria for avoiding
duplicate counts of cases; and (d) who can provide legal assistance to
clients in order for the service to be counted as a case; (5) LSC
initiated a self-inspection procedure in which grantees were required to
review their 1998 case data and submit certification letters to LSC if
they found that the extent of error in their data was 5 percent or less;
(6) grantees who could not certify their 1998 data were required to
develop corrective actions that would address the problems identified;
(7) about 75 percent of the grantees submitted letters to LSC certifying
that the error rate in their 1998 data was 5 percent or less, while
about 25 percent of the grantees submitted letters to LSC indicating
that they could not certify their 1998 data; (8) according to LSC, about
30 of the 50 grantees with the largest caseloads were unable to certify
their 1998 case data; (9) GAO could not assess whether the number of
certified and uncertified programs that LSC obtained for 1998 was
correct, lower, or higher than it should be; (10) this is because LSC
did not provide grantees with a standardized way of reporting their
self-inspection results, and LSC's instructions on how to conduct the
self-inspections may have led some of the smaller grantees to select too
few cases to reliably assess the amount of error in their case data;
(11) some grantees did not correctly interpret LSC's case reporting
requirements; and (12) for these reasons, GAO do not believe that LSC's
efforts to date have been sufficient to fully resolve the case reporting
problems that occurred in 1997.

--------------------------- Indexing Terms -----------------------------

REPORTNUM: GGD-99-183
TITLE: Legal Services Corporation: More Needs to Be Done to
Correct Case Service Reporting Problems
DATE: 09/20/1999
SUBJECT: Legal aid
Internal controls
Federal grants
Statistical data
Grant administration
Reporting requirements
Eligibility criteria
Resident aliens
IDENTIFIER: LSC Case Service Reporting System

******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. This text was extracted from a PDF file. **
** Delineations within the text indicating chapter titles, **
** headings, and bullets have not been preserved, and in some **
** cases heading text has been incorrectly merged into **
** body text in the adjacent column. Graphic images have **
** not been reproduced, but figure captions are included. **
** Tables are included, but column deliniations have not been **
** preserved. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************

LEGAL SERVICES CORPORATION: More Needs to Be Done to Correct Case Service Reporting Problems (GAO/GGD-99-183)
LEGAL SERVICES CORPORATION
More Needs to Be Done to Correct Case Service Reporting Problems
United States General Accounting Office
GAO Report to Congressional Requesters
September 1999

GAO/GGD-99-183

United States General Accounting Office General Government Division Washington, D. C. 20548
Page 1 GAO/ GGD- 99- 183 LSC Case Reporting Problems
B- 283551 September 20, 1999 The Honorable Dick Armey The Honorable Dan Burton The Honorable Tom Latham The Honorable Dan Miller The Honorable Charles Taylor House of Representatives
The Legal Services Corporation (LSC), operating through grantees, provides legal assistance in civil matters to low- income individuals. In the past year, LSC's Office of Inspector General (OIG) and we have identified misreporting by grantees on both the number of cases they closed during calendar year 1997 and the number they had open at the end of that year. The accuracy of the data is important because LSC has used case statistics to seek increased funding for LSC, and Congress has considered these case statistics in determining funding for LSC.
As agreed with your offices, we determined (1) what efforts LSC and its grantees have made to correct problems with case service reporting and (2) whether these efforts are likely to resolve the case reporting problems that occurred in 1997.
LSC revised its written guidance and issued a new handbook to its grantees to clarify case reporting requirements. Based on telephone interviews with a sample of 79 LSC grantee executive directors, we estimate that 90 percent of grantees viewed the new guidance as having clarified reporting requirements, overall. Virtually all grantees said they responded to the new requirements by making or planning to make one or more changes to their program operations. However, many grantees indicated that they were unclear about certain aspects of LSC's reporting requirements, particularly regarding (1) the specific information required on client assets, (2) the information required for documenting citizenship/ alien eligibility for services provided over the telephone, (3) the criteria for avoiding duplicate counts of cases, and (4) who can provide legal assistance to clients in order for the service to be counted as a case.
LSC initiated a self- inspection procedure in which grantees were required to review their 1998 case data and submit certification letters to LSC if they found that the extent of error in their data was 5 percent or less. Grantees who could not certify their 1998 data were required to develop Results in Brief
B- 283551 Page 2 GAO/ GGD- 99- 183 LSC Case Reporting Problems
corrective actions that would address the problems identified. About 75 percent of the grantees submitted letters to LSC certifying that the error rate in their 1998 data was 5 percent or less, while about 25 percent of the grantees submitted letters to LSC indicating that they could not certify their 1998 data. According to LSC, about 30 of the 50 grantees with the largest caseloads were unable to certify their 1998 case data.
We could not assess whether the number of certified and uncertified programs that LSC obtained for 1998 was correct, lower, or higher than it should be. This is because LSC did not provide grantees with a standardized way of reporting their self- inspection results, and LSC's instructions on how to conduct the self- inspections may have led some of the smaller grantees to select too few cases to reliably assess the amount of error in their case data. In addition, some grantees did not correctly interpret LSC's case reporting requirements. LSC is planning to consider alternative ways of training grantee staff so that they gain a better understanding of the reporting requirements.
For these reasons, we do not believe that LSC's efforts to date have been sufficient to fully resolve the case reporting problems that occurred in 1997. This report includes recommendations to LSC for clarifying its case reporting guidance to grantees and improving its procedures for conducting future self- inspections.
LSC was established in 1974 as a private, nonprofit, federally funded corporation to provide legal assistance to low- income people in civil matters. LSC provides the assistance indirectly through grants to competitively selected local programs. LSC distributes funds to the grantees on the basis of the number of low- income persons living within a service area. Grantees may receive additional funding from non- LSC sources.
During 1998, LSC funded 262 local grantees that operated through approximately 900 neighborhood law offices employing about 3,600 attorneys and 1,400 paralegals. Each local program is governed by its own board of directors and is required to spend at least 12.5 percent of its LSC grant to encourage private attorney involvement (PAI) in delivering legal services to low- income clients. In fiscal years 1998 and 1999, LSC received appropriations of $283 million and $300 million, respectively.
LSC's authorizing legislation restricts it from engaging in lobbying; political activities; class actions except under certain conditions; and cases involving abortion, school desegregation, and draft registration or Background
B- 283551 Page 3 GAO/ GGD- 99- 183 LSC Case Reporting Problems
desertion from the military. Annual appropriations laws have placed additional restrictions on the activities in which LSC grantees can engage, even with non- LSC funds. In 1996, for example, grantees were prohibited from engaging in challenges to welfare reform, litigation on behalf of prisoners, representation in drug- related public housing evictions, and representation of certain categories of aliens.
Grantees must serve only those clients who meet financial and citizenship/ alien eligibility requirements. With respect to client financial eligibility, local programs are to establish their own criteria, which, in general, require that clients' income not exceed 125 percent of the federal poverty guidelines. With appropriate documentation of the grantee's decision, clients who are between 125 percent and 187.5 percent of the federal poverty level may be found eligible. LSC regulations require that grantees (1) adopt a form and procedure to obtain eligibility information and (2) preserve that information for audit by LSC. With regard to citizenship/ alien eligibility, only citizens and certain categories of aliens are eligible for services. For clients who are provided services in person, a citizen attestation form or documentation of eligible alien status is required. For clients who are provided services on the telephone, grantees must document that they made inquiries regarding the individuals' citizenship/ alien eligibility.
LSC uses a Case Service Reporting (CSR) system to gather quantifiable information from grantees on the services they provide that meet LSC's definition of a case. The CSR Handbook is LSC's primary official guidance to grantees on how to record and report cases. According to the 1999 CSR Handbook, which revised and expanded the guidance in LSC's 1993 Handbook, information about cases is an important indicator of the number of legal problems that programs address each year, and LSC relies on such case information in its annual request for federal funding for legal services.
Audit reports on five grantees issued by LSC's OIG between October 1998 and July 1999 reported that all five grantees misreported the number of cases they had closed during calendar year 1997 and the number of cases that remained open at the end of that year. The OIG found that all five grantees overstated the number of closed cases, while four overstated and one understated open cases. The OIG attributed the overreporting to such factors as (1) counting as cases telephone calls in which individuals were not provided any legal assistance and only partial eligibility determinations were made; (2) counting wholly non- LSC funded cases as LSC cases; (3) double counting of the same cases; (4) reporting cases as closed during
B- 283551 Page 4 GAO/ GGD- 99- 183 LSC Case Reporting Problems
1997, or as still open at the end of 1997, when service ceased in prior years; and (5) counting as cases the provision of services to over- income clients. In June 1999, in response to Congress' request for information on whether the 1997 case data of other LSC programs had problems similar to those reported by LSC's OIG, we issued a report on our audit of five of LSC's largest grantees: Baltimore, Chicago, Los Angeles, New York City, and Puerto Rico. 1 We found similar types of reporting errors at the five grantees and estimated that, overall, 75,000 of the 221,000 open and closed cases that the five grantees reported to LSC were questionable. 2 Interviews that we conducted with LSC officials and executive directors of the 5 audited grantees indicated that they had taken or were planning to take steps to correct the causes of these case- reporting problems.
Our objectives were to determine (1) what efforts LSC and its grantees have made to correct problems with case service reporting, and (2) whether these efforts are likely to resolve the case reporting problems that occurred in 1997.
To address the objectives, we reviewed documents that contained LSC case reporting guidance, interviewed cognizant officials at LSC headquarters, and conducted structured telephone interviews with a random sample of grantee executive directors. Specifically, we reviewed LSC regulations and LSC's 1993 and 1999 CSR Handbook, as well as supplemental guidance that LSC distributed to grantees in the form of program letters and frequently asked questions and answers about case reporting. We also collected documentation and interviewed LSC officials and grantee executive directors to gather information about a selfinspection process that LSC required all grantees to undertake in order to determine the accuracy of their 1998 case data. In our telephone interviews with executive directors, we asked if the directors viewed LSC's case reporting requirements as being clear, what changes they have made or planned to make as a result of the requirements, and the results of their self- inspection of 1998 case data. We also interviewed LSC officials and grantee executive directors about areas of case reporting that they felt needed further clarification.
1 Legal Services Corporation: Substantial Problems in 1997 Case Reporting by Five Grantees (GAO/ GGD- 99- 135R, June 25, 1999). 2 Our estimate was based on file reviews of random samples of cases that the five grantees reported as closed during 1997 and open at the end of 1997. In addition, we reviewed random samples of potentially duplicate cases for four grantees. We estimated that, overall, 75, 000 (+/- 6,100) cases were questionable. Objectives, Scope, and
Methodology
B- 283551 Page 5 GAO/ GGD- 99- 183 LSC Case Reporting Problems
To identify the universe of LSC grantees, an LSC official referred us to the list of LSC programs on the corporation's Internet Web site. As of July 15, 1999, the Internet site listed 256 programs. From this list, we randomly selected 80 programs for our sample. We developed the interview instrument, pretested it with executive directors from two grantees, revised the instrument based on the pretest results, and completed approximately 1- hour- long structured interviews by telephone with the executive directors of 79 of the 80 programs, for a 99 percent response rate. (After numerous attempts over several days, we were unable to contact the executive director of one LSC grantee.) Our sample was designed so that we could generalize our findings with 95- percent confidence and a maximum 10- percent margin of error to the universe of LSC grantees.
We performed our work from July through September 1999 in accordance with generally accepted government auditing standards. We requested comments on a draft of this report from the President of LSC. LSC's comments are discussed at the end of this letter and included as appendix I.
LSC issued a new CSR Handbook and distributed other written communications intended to clarify reporting requirements to its grantees. Most grantees indicated that the new guidance helped clarify LSC's reporting requirements, and virtually all of them indicated that they had or planned to make program changes as a result of the requirements. Many grantees, however, identified areas of case reporting that remained unclear to them.
To address specific problems identified by the OIG and LSC's own internal program reviews, LSC has issued revised reporting guidance. In November 1998, LSC issued the 1999 CSR Handbook, which replaced the 1993 CSR Handbook. The 1999 handbook instituted changes to some of LSC's reporting requirements and provided more detailed information on other requirements than previously existed. LSC also distributed program letters and a list of frequently asked CSR questions and answers that further elaborated on points made in the 1999 handbook.
The 1999 CSR Handbook instituted several notable changes to case reporting requirements. These included (1) procedures for timely closing LSC Clarified CSR
Requirements and Grantees Have Been Making Program Changes to Comply With the Requirements, but Some Grantees Remain Unclear About Certain Requirements
LSC Issued New CSR Guidance
B- 283551 Page 6 GAO/ GGD- 99- 183 LSC Case Reporting Problems
of cases; (2) procedures for management review of case service reports; (3) procedures for ensuring single recording of cases; (4) requirements to report LSC- eligible cases, regardless of funding source; and (5) requirements for reporting PAI cases separately. In addition, grantees were required to use automated case management systems and procedures that would ensure that program managers had timely access to accurate information on cases and the capacity to meet their reporting requirements.
On November 24, 1998, LSC informed its grantees that two of the changes in the 1999 CSR Handbook were to be applied to the 1998 case data. The two changes pertained to timely closing of cases and management review of case service reports. The timely closing provision required grantees to ensure that cases in which legal assistance had ceased in 1998, and was not likely to resume, would be closed prior to grantees' submission of case service reports to LSC in March 1999. To the extent practicable, cases in which the only assistance provided to the client was counsel and advice, brief service, or referral after legal assessment were to be closed in the year in which these types of service were provided. Cases involving other types of service were to be closed in the year in which program staff determined that further legal assistance was unnecessary, not possible, or inadvisable and a closing memorandum or other case- closing notation was prepared. The management review provision required the executive director, or a designee, to review the program's case service reports prior to their submission to LSC in order to ensure their accuracy and completeness.
The remaining new provisions of the 1999 CSR Handbook were not applicable to 1998 cases. For example, for 1998, there was no requirement for grantees to ensure that cases were not double counted. For 1999, LSC is requiring the use of automated case management systems and procedures to ensure that cases involving the same client and specific legal problem are not reported to LSC more than once. For 1998, grantees could report only those cases that were at least partially supported by LSC funds. For 1999, LSC is requiring grantees to report all LSC- eligible cases, regardless of funding source. LSC intends to estimate the percentage of activity spent on LSC service by applying a formula that incorporates the amount of funds grantees receive from other funding sources compared with the amount they receive from LSC. For 1998, grantees were required to report their LSC- funded PAI cases together with non- PAI cases. For 1999, PAI cases are to be reported separately.
B- 283551 Page 7 GAO/ GGD- 99- 183 LSC Case Reporting Problems
In addition to changing certain reporting requirements, the 1999 handbook also provides more detailed guidance to grantees than the 1993 handbook. For example, the 1999 handbook provides more specific definitions of what constitutes a case and a client for CSR purposes. The 1999 handbook also addresses documentation requirements that were not discussed in the 1993 handbook. For example, the 1999 handbook indicates that, except for telephone service cases, the client's file must contain an attestation of citizenship or documentation of alien eligibility in order for the case to be reported to LSC. 3 The 1999 handbook also imposes requirements for documentation of information on client income and assets. The handbook states that, for all cases reported to LSC, the eligibility documentation must include specific information about income and assets. The 1999 handbook also contains a requirement that legal assistance, to be counted as a case, must be provided by an attorney or paralegal.
On the basis of our survey, we estimate that over 90 percent 4 of grantee executive directors viewed the changes in the 1999 CSR Handbook as being clear overall, and virtually all of them planned to or had made at least one change to their program operations as a result of the revised case reporting requirements. Program changes that were cited included revising policies and procedures, providing staff training, modifying forms and/ or procedures used during client intake, implementing computer hardware and software changes, and increasing review of cases.
Nearly half of the grantees indicated that they planned to or had revised their policies and/ or procedures to comply with the 1999 handbook requirements, and slightly less than a third planned to or had conducted staff training on the requirements. Respondents told us that the focus of their training was on such issues as the current definition of a case, how to determine and document clients' financial eligibility, how to determine and document citizenship/ alien eligibility, timely closing of cases, and prevention of duplicate case reporting.
More than half of the grantees indicated that they planned to or had changed their intake forms and/ or procedures. Of the 41 respondents who
3 The requirement for documentation is contained in LSC regulations, at 45 C. F. R. 1626. 6( a) and 1626. 7( a). However, the 1999 CSR Handbook specified for the first time that the case could not be reported to LSC if the documentation was missing.
4 All percentage estimates presented in this report have 95 percent confidence intervals with a margin of error of 10 percent or less. Grantee Directors Reported
That They Are Implementing Changes to Comply With Reporting Requirements
B- 283551 Page 8 GAO/ GGD- 99- 183 LSC Case Reporting Problems
made this comment, 26 said they were making these changes in order to document client income and assets.
Slightly over 40 percent of the grantees reported that they planned to or had made computer changes. The computer changes included such actions as adding new fields to their automated case management (e. g., to ensure that client eligibility and acceptance information is recorded), making programming changes (e. g., to identify duplicate cases, ensure that cases are not assigned separate numbers for the same client with the same legal problem, and better document client assets), and installing new software (e. g., to generate reports so that they can track how long cases are open).
About 10 percent of grantee executive directors indicated that they could better comply with CSR requirements if they had uniform case management software. LSC, too, believes that grantees' case management systems should provide for more consistency in the collection and processing of CSR information. According to an LSC official, LSC is developing a strategy for modifying grantees' case management systems so that data errors can be detected and prevented. As part of the strategy, LSC has hired an expert in case management systems and is working to develop standard input requirements for these systems. LSC intends to work with case management system vendors to modify the systems so that they prevent cases from being accepted if grantees do not record the required compliance information. LSC expects modifications to the case management systems to be implemented in calendar year 2000 and available for application to grantees' 2000 CSR data. In the nearer term, LSC plans to work with a contractor to develop customized case management queries that would enable grantees to detect and remedy errors in their 1999 case data. In 1999, LSC plans to pilot test the computer queries at five programs that have the most commonly used case management systems.
Nearly three- fourths of the grantees indicated that they planned to or had increased their review of cases to comply with LSC requirements. Respondents cited such review activities as more intense monitoring of open cases, to ensure that they are closed in a timely manner, and more thorough monitoring of PAI cases. Some grantees said they have directed increased attention to reporting requirements during routine reviews of cases, while others said they instituted more frequent reviews of cases. One executive director reported to LSC that managing attorneys would meet with all case- handling staff and reemphasize the importance of keeping accurate activity records. These managing attorneys are to monitor compliance with the requirement initially on a weekly basis and
B- 283551 Page 9 GAO/ GGD- 99- 183 LSC Case Reporting Problems
then on a random basis. If deficiencies are discovered, the managing attorney is to direct the case handler to correct them immediately, and the managing attorney is to recheck the file within 48 hours to ensure compliance with the directive. Another executive director told us that his program had not previously reviewed CSR data. He said that they now are reviewing reports, along with listings of open and closed cases, to ensure their compliance with LSC guidelines. Some executive directors said that they planned to become more involved in reviewing case files and case management reports.
Many grantees reported making other efforts to comply with reporting requirements, such as disseminating the new handbook to case handlers, developing compliance checklists, emphasizing the importance of compliance to staff, sending written instructions or memos to staff, holding meetings and discussions, providing more feedback to case handlers, and more stringently enforcing requirements.
Although most of the grantee executive directors reported that the new LSC guidance helped clarify requirements, many of them also indicated that they were still unclear about certain requirements and that additional clarification was needed. Among the areas of confusion or uncertainty that executive directors identified were requirements pertaining to asset and citizenship/ alien eligibility documentation, single recording of cases, and who can provide legal services.
Asset documentation: The 1999 CSR Handbook states that, for all cases to be reported to LSC, the eligibility documentation should include specific information about income and assets. About 30 percent of the executive directors indicated that LSC's requirements for documenting client assets were clear only to some, to little, or to no extent. Of the 24 respondents who gave this response, 23 made comments to the effect that LSC should clarify what it means by assets and asset limits and/ or clarify its documentation requirements for client assets. In a July 14, 1999, program letter to grantees, LSC noted that although many grantees inquire about applicants' assets, they do not consistently document either the inquiries or the applicants' responses. In its program letter, LSC sought to clarify its requirements for documenting and preserving the asset information obtained from each applicant. Because we conducted our telephone survey in late July and early August, we do not know how many of the executive directors with whom we spoke had reviewed LSC's July 14 guidance and felt that it sufficiently clarified their questions concerning asset documentation. However, two respondents told us that LSC's July 14 guidance was still unclear. One respondent said, for example, that the Many Grantees Remain
Unclear About Certain Reporting Requirements
B- 283551 Page 10 GAO/ GGD- 99- 183 LSC Case Reporting Problems
program letter left unclear the meaning of the term household goods, whether exempted assets had to be documented, who was to value the goods, and who was to determine what to count and what not to count as part of assets. Our own analysis of LSC guidance on the asset issue indicated that LSC has not been consistent in its directives to grantees about the specificity of asset information they must have in order to comply with CSR reporting requirements. For example, in its March 24, 1999, communication on frequently asked CSR questions and answers, LSC instructed grantees to document, at minimum, the total amount of household assets accessible to the client. In its July 14, 1999, program letter, LSC instructed grantees to identify and document all of the liquid and nonliquid assets of all persons who are resident members of a family unit.
Citizenship/ alien eligibility documentation requirements for telephone cases: The 1999 CSR Handbook does not explicitly address the citizenship or eligible alien status documentation requirements for situations where assistance is provided only over the telephone. Nearly one- fourth of the executive directors indicated that LSC's documentation requirements in this area were clear only to some, to little, or to no extent. Of 19 respondents who gave this response in our survey, 15 said that more clarification was needed on the grantees' documentation responsibilities. Respondents said, for example, that they were confused about exactly when they needed to obtain a written attestation, whether it was sufficient simply to record that questions about citizenship/ alien eligibility had been asked, whether certain types of service required documentation while others did not, and whether the requirement changed if an individual receiving assistance over the telephone came into the office to drop off documents. In its July 14, 1999, program letter to grantees, LSC sought to clarify its requirements for documenting citizenship/ alien eligibility information in telephone cases. LSC stated that it requires recipients to make appropriate inquiry of every telephone applicant and record the inquiry and response. All such documentation is to be maintained in the client file. We do not know how many of the respondents to our survey were familiar with LSC's July 14 guidance when we interviewed them in late July and early August.
Single recording of cases: The 1999 CSR Handbook requires that programs ensure that cases involving the same client and specific legal problem are not recorded and reported to LSC more than once. Over one- fourth of the executive directors indicated that the requirement for preventing duplicate case reporting was clear only to some, to little, or to no extent. Of the 22 respondents who gave this response in our survey, 17 said that the
B- 283551 Page 11 GAO/ GGD- 99- 183 LSC Case Reporting Problems
distinction between specific and related legal problems is difficult to determine. Several respondents cited examples in family law cases where more than one problem can arise within one family case. Depending on the situation, it was not clear to them at what point legally related issues become separate enough to count as separate cases. One respondent indicated that attorneys in his program have a hard time interpreting this requirement, and that he got calls every week about this issue.
Provider of legal assistance: The CSR handbook requires that legal assistance be provided by an attorney or paralegal in order for a service to be called a case. 5 Slightly over 40 percent of executive directors indicated that LSC was clear only to some, to little, or to no extent about who can provide such legal assistance. Respondents noted in our telephone survey, for example, that the terms case handler, and paralegal are not clearly defined, and that they did not know whether nonlawyers (e. g., intake workers) who are supervised by lawyers can provide legal assistance. Comments made to us by the executive directors revealed that grantees held varying views about who can provide legal assistance to clients. Several respondents specifically stated that this is an important issue that LSC should address in its handbook. Our own analysis of LSC guidance on this issue indicates that LSC has not been consistent in its advice to grantees. For example, in its communication with grantees on frequently asked CSR questions and answers, LSC stated that a telephone conversation between an intake specialist and a caller who was accepted for service can be counted as a case if the caller received some advice that addressed a specific legal problem. LSC officials told us that LSC's current position on this issue is that the person giving legal advice must be someone who is authorized to practice law or is under the supervision of an attorney in accordance with local rules of practice. Therefore, legal advice can be given by a (1) lawyer, (2) paralegal, or (3) intake specialist or law student under the supervision of a lawyer, as long as the assistance does not violate local rules of practice.
5 The 1999 CSR handbook does not explicitly state that legal assistance must be provided by an attorney or paralegal. However, section VIII of the handbook states that, to be reported as a case, the legal assistance must meet the definition of case in 45 CFR 1635. 2( a). This regulation defines a case as a form of program service in which an attorney or paralegal provides legal services.
B- 283551 Page 12 GAO/ GGD- 99- 183 LSC Case Reporting Problems
LSC sought to determine the accuracy of grantees' case data by requiring that grantees do self- inspections of their open and closed caseload data for 1998. Grantees were required to determine whether the error rate in their data exceeded 5 percent. If the error rate was 5 percent or less, they could certify that their data were substantially correct. If the error rate was higher than 5 percent, they were to determine how the problems identified could be addressed. LSC found that about three- fourths of the grantees were able to certify to the substantial accuracy of their data. LSC used the results of the self- inspections to estimate the total number of case closings in 1998. Our review of LSC's self- inspection process raised concerns about the accuracy and interpretation of the results, and what the correct number of certifying programs should be.
On May 14, 1999, LSC issued a memo to all grantees instructing them to complete a self- inspection procedure by July 1, 1999. The purpose of the self- inspection was to ensure that (1) grantees were properly applying instructions in the 1999 edition of the CSR Handbook that were applicable to the 1998 data, and (2) LSC had accurate case statistical information to report to Congress for calendar year 1998.
LSC provided detailed guidance to grantees on the procedures for the selfinspection. Each grantee was to select and separately test random samples of open and closed cases to determine whether the number of cases it reported to LSC earlier in the year was correct. Grantees were to verify that the case file contained a notation of the type of assistance provided, the date on which the assistance was provided, and the name of the case handler providing the assistance. Grantees were also to determine whether assistance had ceased prior to January 1, 1998; was within certain service categories as defined by the 1999 handbook; was provided by an attorney or paralegal; and was not prohibited or restricted. Finally, grantees were to verify that each case had eligibility information on household income, household size, household assets, citizenship attestation for in- person cases, and indication of citizenship/ alien status for telephone- only cases. According to LSC officials, the self- inspection was a single procedure that was undertaken within a limited time period, and LSC did not expect the self- inspection to resolve all case- reporting problems.
In requiring grantees to verify that their 1998 case files contained information on client assets and an indication of citizenship/ alien status for telephone- only cases, LSC imposed stringent criteria on the selfinspections. The criteria were stringent in that LSC had not promulgated explicit documentation requirements related to these issues until it released the 1999 CSR Handbook and July 14 program letter. Grantees Most Grantees
Certified to the Accuracy of Their 1998 CSR Data, but Questions About Data Accuracy and Interpretation Remain
LSC Grantees Conducted Self- Inspections of 1998 CSR Data
B- 283551 Page 13 GAO/ GGD- 99- 183 LSC Case Reporting Problems
were not required to apply these new documentation requirements to their 1998 case data. Recognizing that in 1998 many grantees did not keep sufficient documentation on assets or citizenship/ alien eligibility to comply with the stringent self- inspection requirements, LSC allowed grantees some latitude in determining whether they were in fundamental compliance with the requirement that legal assistance only be provided to eligible individuals. In conversations with grantee executive directors during the self- inspection, LSC officials advised them that if they had a level of certainty that their program staff had asked questions about assets, and if their certainty was sufficient for them to sign a form attesting to this, that was an acceptable basis for asserting compliance with the reporting requirement for assets. Similarly, if grantees were sufficiently certain that, for telephone cases, their program staff had asked questions about citizenship/ alien status but had not documented the inquiry, that too was acceptable. Finally, LSC allowed grantees some latitude with respect to the requirement that an attorney or paralegal must be the provider of legal assistance. LSC advised grantees during the self- inspection that it was acceptable for them to count as valid cases instances in which legal advice was given by a (1) lawyer, (2) paralegal, or (3) intake specialist or law student under the supervision of a lawyer, as long as the assistance did not violate local rules of practice.
If any single aspect of a case failed to meet LSC's requirements, the case was to be classified as an error for reporting purposes. If the grantees found that their CSR case sampling had an error rate of 5 percent or less, the program directors and policy board chairs were to sign a certification form and return it to LSC. Grantees who could not certify to the correctness of their 1998 CSR data were to submit a letter to LSC describing (1) the problems they had identified during the self- inspection process and (2) the corrective actions they had instituted to address the problems. Grantees could resubmit their 1998 CSR data to LSC if they identified one or more problems in the random sample and corrected their entire 1998 database so that the problems no longer appeared. If, by correcting the problems, the error rate in the data was reduced to 5 percent or less, the grantees could resubmit their 1998 data along with a signed certification attesting to the substantial accuracy of the resubmitted data. In this way, grantees who were unable to certify at one point in time could certify at a later point in time. As of July 29, 1999, 26 grantees had resubmitted their 1998 CSR data after having made corrections to the data, and 20 of them had certified their data.
B- 283551 Page 14 GAO/ GGD- 99- 183 LSC Case Reporting Problems
According to LSC officials, about three- fourths of the grantees certified the accuracy of their 1998 case data. As of August 26, 1999, LSC documents indicated that 199 of 261 grantees 6 (76 percent) reported substantially correct CSR data to LSC. The remaining 62 grantees (24 percent) did not certify to LSC that their CSR data were substantially correct. On the basis of the self- inspection results, LSC estimated that grantees closed 1. 1 million cases in 1998.
LSC officials told us that they were surprised that such a large number of grantees were able to certify their 1998 CSR data. They attributed the lower- than- expected error rates to the following factors:
The self- inspection did not attempt to identify duplicate cases. LSC officials explained that, during most of 1998, LSC did not have a standard regarding duplicate cases, and that they believed it would have been too burdensome on grantees for LSC to require them to apply the new standard retroactively. According to an LSC official, duplicate cases are best caught at the time of intake. The official also noted that duplicate cases had not in the past been found to be a major problem in comparison with other identified problems.
Grantees received the new 1999 CSR Handbook in November1998, and a number of them had implemented the requirements that applied to their 1998 data by the time they submitted their 1998 case statistics.
Grantees were aware of the problem that the OIG had identified with cases coded as referral after legal assessment. That is, some programs had inappropriately reported as CSR cases numerous instances of making a telephone referral without providing legal advice and/ or without documenting an individual's eligibility. Because grantees had been sensitized to this issue, LSC officials believed that they were less likely to count these referrals as cases in 1998.
According to LSC officials, CSR problems were more common at larger grantees that had heavier case loads and multiple branches. They noted that approximately 30 of the 50 grantees with the largest caseloads were unable to certify their 1998 data. They hypothesized that larger programs may have difficulty certifying if such programs have (1) numerous branch offices, one or more of which are out of compliance with regulations and causing reporting problems for the entire program, and/ or (2) numerous
6 LSC funded 262 programs in 1998. Funding for one program was discontinued in 1999, and LSC has no self- inspection results for this program. Most Grantees Certified
Their 1998 CSR Data
B- 283551 Page 15 GAO/ GGD- 99- 183 LSC Case Reporting Problems
sources of funding and more compliance requirements, which could add to the complexity of their reporting process. The officials acknowledged that these hypotheses needed further exploration.
LSC officials also reported that noncertifying grantees identified two principal problems. One pertained to the lack of citizenship attestations in case files. The second pertained to the lack of information on client assets. LSC officials also noted that some grantees had reported matters, such as referrals, as cases, and some had reported cases as open when they did not meet the timely closing requirement. In our telephone interviews with 79 executive directors, 24 reported that they did not certify their 1998 data to LSC. Factors that the executive directors of noncertifying programs believed greatly affected the errors in their 1998 data included lack of clear guidance from LSC (cited by 11 respondents), computer problems (cited by 7 respondents), and insufficient attention by their programs to administrative matters (cited by 5 respondents).
Computer problems that were noted included difficulties merging databases and using several different software packages, and problems with upgrading the computer system.
Based on the self- inspection results, LSC estimated that its grantees closed about 1.1 million cases in 1998. LSC arrived at this figure by subtracting the total number of closed cases estimated to be in error (135,498) from the total number of cases that grantees reported to LSC (1,260,351). 7 As of July 29, 1999, LSC estimated that its grantees' total closed caseload for 1998 was 1,124,853. 8
LSC intends to report only the estimated number of cases closed in 1998, even though the self- inspections were to encompass both open and closed cases. According to LSC officials, LSC has less confidence in the open- case numbers than the closed- case numbers. One reason for this is that they believe open cases are more likely than closed cases to have timely closing
7 LSC performed this calculation separately for certifying and noncertifying programs. For noncertifying programs, LSC determined that 96, 273 cases were reported in error out of grantees' original submission of 475,856 closed cases. For certifying programs, LSC subtracted 5 percent, or 39, 225 cases, from grantees' original submission of 784, 495 closed cases. LSC reduced the case statistics of certifying programs by 5 percent to ensure that there would be no overstatement of cases for these programs.
8 In April 1999, prior to the self- inspection, LSC's CSR database indicated that grantees closed 1,338,976 cases in 1998. The difference between the April 1999 figure and LSC's July 1999 estimate of 1,260,351 closed cases that is, the number reported closed prior to adjusting for the self- inspection results is attributable to the fact that some grantees who found data errors corrected them and resubmitted their statistical results to LSC. Those who initially did not certify their data may have later certified them if their corrections reduced their error rate to 5 percent or less. LSC Used Self- Inspection
Results to Estimate Number of Cases Closed
B- 283551 Page 16 GAO/ GGD- 99- 183 LSC Case Reporting Problems
problems. A second reason is that some grantees experienced problems when converting to a new computer system. System conversions sometimes caused dates to be lost or cases to be mistakenly coded as being open.
Our review of LSC's self- inspection results raised some concerns about LSC's interpretation of the results and about the accuracy of the data provided to LSC by grantees. As a result, we could not assess whether the number of certified programs and case closures that LSC estimated for 1998 is correct, lower, or higher than it should be.
Although LSC provided instructions to grantees on how they should select test samples and what case information they should verify, LSC did not issue standardized procedures for grantees to use in reporting the results of their self- inspections. Grantees that could not certify their data wrote letters to LSC in which they described the errors they uncovered. The letters contained varying degrees of detail about the errors. Some programs provided an overall error rate and did not separately report how much error they found in open and closed cases. Others provided detailed information on both the number of errors found in open and closed cases, respectively, as well as the number of errors broken out by type. Since LSC did not have a standard protocol for collecting the results of the selfinspections, in some cases LSC officials had to rely on interpreting grantee letters that described the problems that were discovered. LSC officials believe that their numerous contacts with grantees who called them to ask questions about the self- inspections, combined with their analysis of the grantee letters, enabled them to correctly determine the number of certifying programs and estimate the number of closed cases.
We are uncertain how many programs should have been counted as certified because we are uncertain if LSC applied a consistent definition of certification. Most programs that were on LSC's certification list determined that they had error rates of 5 percent or less for both open and closed cases. However, LSC placed some programs on the certified list if the program's overall error rate for closed cases was 5 percent or less, even if the overall error rate actually was higher than 5 percent. We encountered this situation in two instances in which executive directors told us in telephone interviews that they did not certify their CSR data because their overall error rate exceeded 5 percent. However, these programs appeared on LSC's list of certified programs. When we asked an LSC official about this, he told us that they advised grantees that if their closed case error rate did not exceed 5 percent, they should partially certify their data. In response to our inquiry, the official reviewed the Self- Inspection Results
Raised Concerns
B- 283551 Page 17 GAO/ GGD- 99- 183 LSC Case Reporting Problems
certification letters submitted by nearly 200 grantees, and he identified 5 certified programs whose error rates for open cases exceeded 5 percent. Given that some grantees submitted only an overall estimate of data error, we do not know how many programs qualified to be certified overall, just for closed cases, or just for open cases.
In another instance, an executive director told us that she did not certify her program because 8 of the 14 case- handling offices had error rates exceeding 5 percent. Nonetheless, this program appeared on LSC's certified list. An LSC official explained that, after reviewing the information provided by this program, the official agreed that the program should not have been classified as certified. In a fourth instance, in which an executive director reported to us that he did not certify his program, an LSC official said that, although they thought the grantee's data had been corrected, the program had not yet certified its data and might need to do additional sampling.
We are also concerned that LSC's instructions to grantees on how to conduct the self- inspections may have led some of the smaller grantees to select too few test cases to make a reliable assessment of the proportion of error in their case data. For example, LSC instructed grantees to select every tenth case for review if the program handled less than 1,000 cases in 1998. This was to be done separately for open and closed cases. Based on 1998 case statistics that grantees submitted to LSC between January and March 1999, several programs would have based their certification determinations on reviews of a relatively small number of cases. Seven grantees had fewer than 300 closed cases, and 43 grantees had fewer than 300 open cases. Three programs had fewer than 300 cases combined. The smallest program would have based its self- inspection on 3 closed cases and 1 open case. We believe that, in general, samples of 30 or fewer cases are too small to provide reliable estimates of the total number of case data errors. 9 Because these were smaller grantees, this limitation would have had little effect on LSC's estimate of total closed caseload. However, it could have affected by either overstating or understating LSC's count of the number of certified programs.
LSC does not know how well grantees conducted the self- inspection process, nor how accurate the results are. We spoke with several executive
9 To illustrate, a program that reviewed 30 closed cases and found a single error would have had a 3 percent error rate. However, the margin of error based on a sample size of 30 would range from about 0.1 percent to 17 percent at the 95 percent statistical confidence level. If the program found two errors, it would have had a 7 percent error rate, with a margin of error ranging from 1 percent to 22 percent. Statistically, the smaller the number of cases sampled, the larger the margin of error.
B- 283551 Page 18 GAO/ GGD- 99- 183 LSC Case Reporting Problems
directors who did not correctly follow LSC's reporting requirements. In one case, the executive director sought clarification from LSC headquarters about the use of a CSR closure code and was given incorrect verbal guidance. The situation concerned whether applicant files could be closed as client withdrew cases when individuals who were accepted for service and assigned to staff attorneys did not subsequently appear for a meeting with the attorney. According to both the executive director and an LSC official, the executive director was told that if the program attempted to contact these individuals, by telephone or letter, to determine whether they were still interested in obtaining legal assistance, they could be counted as cases. The executive director told us that she would review her entire database for this type of error and make corrections. This program was included on LSC's certified list, but we do not know whether the program would stay on that list if all the errors were identified.
Another executive director told us that he was concerned that LSC did not want grantees to count assistance over the telephone as a case. This is not an entirely correct interpretation of LSC guidance since, under certain conditions, LSC permits legal assistance over the telephone to be counted as a case. Although this program was included on LSC's certified list, any valid telephone cases that were not counted would have erroneously increased its error rate.
A third executive director told us that the timely closing rule required him to close new cases in December. If the person sought assistance with a similar problem in January, it would be treated as a new case because a new reporting year had begun. In a March 1999 written communication intended to supplement the 1999 handbook, LSC advised grantees to exercise discretion about when to close cases that were opened near the end of the year. LSC did not require grantees to close all cases at year- end. These examples illustrate the possibility that incorrect interpretations of LSC guidance may have resulted in some programs certifying their 1998 data when they shouldn't have, and other programs not certifying their 1998 data when they should have. An LSC official told us that, although they have conducted CSR training sessions for grantee executive directors, there are thousands of case handlers in grantee offices who have not received similar training. The official acknowledged that written guidance and telephone contacts with grantees may not be sufficient to ensure correct and consistent understanding of reporting requirements, and that LSC may consider alternative ways of providing training to staff, such as through videos.
B- 283551 Page 19 GAO/ GGD- 99- 183 LSC Case Reporting Problems
Incorrect interpretations of LSC guidance could also have affected the accuracy of LSC's estimate of closed cases. The Inspector General told us that his office has completed audits of six grantees' 1998 case data. One additional audit of a grantee's CSR data had not yet been completed. The six completed audits included both certifying and noncertifying grantees. According to the Inspector General, the results of the six completed audits would be provided to Congress by September 29, 1999.
LSC officials told us that the self- inspection was valuable and that LSC plans to have grantees complete self- inspections again early next year as part of the 1999 CSR reporting process. We agree that a self- inspection process can be valuable, provided that grantees have clear, consistent, and appropriate guidance on the procedures for reviewing their case data, determining whether to certify the data, correcting errors, and providing their results to LSC in a standardized way that facilitates validation of the results.
LSC's 1999 CSR Handbook and other written communications have improved the clarity of reporting requirements for its grantees. However, many grantees remain unclear about and/ or misunderstood certain aspects of the reporting requirements. LSC's practice of disseminating guidance primarily by written or telephone communications may not be sufficient to ensure that grantees correctly and consistently interpret the requirements.
LSC sought to determine the accuracy of grantees' 1998 case statistics by requiring grantees to conduct self- inspections. However, we do not know the extent to which the results of the self- inspection process are accurate. The validity of the results are difficult to determine because LSC did not standardize the way that grantees were to report their results, some of the grantees used samples that were too small to assess the proportion of error in their data, some grantees did not correctly follow LSC's reporting guidance, and LSC had done no verification of the grantees' self- inspection procedures.
We do not believe that LSC's efforts, to date, have been sufficient to fully resolve the case reporting problems that occurred in 1997.
We recommend that the President of LSC
clarify and disseminate information on the specific information on client assets that grantees must obtain, record, and maintain; Conclusions
Recommendations
B- 283551 Page 20 GAO/ GGD- 99- 183 LSC Case Reporting Problems
clarify and disseminate information on the types of citizenship/ alien eligibility information grantees must obtain, record, and maintain for clients who receive legal assistance only over the telephone;
clarify and disseminate LSC's criteria for single recording of cases;
clarify and disseminate LSC's policy concerning who can provide legal assistance to clients for the service to be counted as a case;
explore options for facilitating correct and consistent understanding of reporting requirements, such as developing and disseminating a training video for grantee staff;
develop a standard protocol for future self- inspections to ensure that grantees systematically and consistently report their results for open and closed cases;
direct grantees to select samples for future self- inspections that are sufficient to draw reliable conclusions about magnitude of case data errors; and finally,
ensure that procedures are in place to validate the results of LSC's 1998 self- inspection, as well as of any future self- inspections.
The President of LSC provided written comments on a draft of this report, which are printed in full in appendix I. LSC generally agreed with our findings and stated that it intends to implement our recommendations.
Concerning the issue of the clarity of case reporting guidance, LSC's letter stated that LSC is troubled by our finding that some grantees are continuing to have difficulty in this area. LSC's letter reiterated a point that we made in the report; namely, that the overlap in time between our data collection effort and LSC's distribution to grantees of a July 1999 program letter intended to clarify client eligibility documentation requirements may have caused some of the executive directors not to factor in the new guidance when they responded to our telephone interview questions. Although, as we state on pages 9 and 10 of the report, some respondents may not have been familiar with LSC's July 1999 guidance, two respondents told us that they were familiar with the new guidance and that they were still unclear about the requirements dealing with assets. We also state on page 10 of the report that our own analysis indicated that LSC was not consistent in its guidance to grantees on what asset information they needed to obtain from clients. Therefore, we do not believe that grantees' Agency Comments
B- 283551 Page 21 GAO/ GGD- 99- 183 LSC Case Reporting Problems
lack of clarity concerning reporting requirements on client eligibility was due solely to the overlap in the time period of our data collection and LSC's distribution of the July 1999 program letter.
With respect to grantees' lack of clarity about how to determine duplicate case counting, LSC indicated that it will consider providing additional guidance to grantees through further revisions to the CSR handbook. LSC also indicated that it will revise the handbook to clarify the issue of when legal assistance by nonlawyers can be reported as a case.
LSC also reiterated that it is developing additional methods to supplement written guidance, including (1) conducting training and technical assistance, (2) developing case management standards that would detect and prevent cases from being accepted and reported if the required eligibility documentation was not obtained, and (3) developing database queries for grantees to apply to their case management systems to identify instances where cases lacked required compliance documentation.
Concerning the self- inspection results, LSC noted that its principal purpose in requiring grantees to self- inspect their 1998 case data was to ensure that case- reporting guidance was being properly applied. LSC stated that assessing the accuracy of the 1998 case statistics that grantees submitted to LSC in March 1999 was a secondary purpose of the self- inspection. LSC believes that the self- inspection increased grantees' awareness of reporting requirements and prompted them to make changes in their case reporting practices. Indeed, in LSC's view, the self- inspection requirement resulted in more program improvements than did the 1999 CSR handbook and other written guidance issued by LSC because grantees that identified significant problems were required to take corrective actions. LSC notes correctly that, in relation to the self- inspection, our report focuses on the accuracy of LSC's determination of the number of grantees that certified their data, and the factors that affected the accuracy of LSC's estimate of the total number of closed cases in 1998. Our review focused on the stated goals of the self- assessment. Although we did not assess whether, relative to LSC's other efforts, the self- inspection had a greater effect on grantees' awareness of and compliance with CSR reporting, we do report that 26 grantees made corrections to their 1998 data as a result of their selfinspections.
As arranged with your offices, unless you publicly announce the contents of this letter earlier, we plan no further distribution until 7 days after the date of this letter. At that time, we will send a copy of the report to the Chairmen and Ranking Minority Members of LSC's appropriations and
B- 283551 Page 22 GAO/ GGD- 99- 183 LSC Case Reporting Problems
legislative committees and to Mr. John McKay, the President of LSC. The major contributors to this report are acknowledged in appendix II. If you or your staff have any questions concerning this report, please contact me or Evi L. Rezmovic on 202- 512- 8777.
Laurie E. Ekstrand Director, Administration of Justice Issues
Page 23 GAO/ GGD- 99- 183 LSC Case Reporting Problems
Appendix I Comments From the Legal Services Corporation
Page 24 GAO/ GGD- 99- 183 LSC Case Reporting Problems
Appendix I Comments From the Legal Services Corporation
Page 25 GAO/ GGD- 99- 183 LSC Case Reporting Problems
Now on pp. 9- 11.
Appendix I Comments From the Legal Services Corporation
Page 26 GAO/ GGD- 99- 183 LSC Case Reporting Problems
Appendix II GAO Contacts and Staff Acknowledgments
Page 27 GAO/ GGD- 99- 183 LSC Case Reporting Problems
Laurie E. Ekstrand, 202- 512- 8777 Evi L. Rezmovic, 202- 512- 8777
In addition to those named above, Mark Tremba, Kristeen McLain, Jan Montgomery, David Alexander, Barry Seltser, Lemuel Jackson, Brian Lipman, and Walter Vance made key contributions to this report. GAO Contacts
Acknowledgments
Page 28 GAO/ GGD- 99- 183 LSC Case Reporting Problems
Ordering Information The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.
Order by mail: U. S. General Accounting Office P. O. Box 37050 Washington, DC 20013
or visit: Room 1100 700 4 th St. NW (corner of 4 th and G Sts. NW) U. S. General Accounting Office Washington, DC
Orders may also be placed by calling (202) 512- 6000 or by using fax number (202) 512- 6061, or TDD (202) 512- 2537.
Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512- 6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.
For information on how to access GAO reports on the INTERNET, send e- mail message with info in the body to:
info@ www. gao. gov or visit GAO's World Wide Web Home Page at: http:// www. gao. gov
United States General Accounting Office Washington, D. C. 20548- 0001
Official Business Penalty for Private Use $300
Address Correction Requested Bulk Rate
Postage & Fees Paid GAO Permit No. G100
(182080)
LEGAL SERVICES CORPORATION: More Needs to Be Done to Correct Case
Service Reporting Problems (GAO/GGD-99-183) LEGAL SERVICES
CORPORATION More Needs to Be Done to Correct Case Service
Reporting Problems United States General Accounting Office GAO
Report to Congressional Requesters September 1999 GAO/GGD-99-183
United States General Accounting Office General Government
Division Washington, D. C. 20548 Page 1 GAO/GGD-99-183 LSC Case
Reporting Problems B-283551 September 20, 1999 The Honorable Dick
Armey The Honorable Dan Burton The Honorable Tom Latham The
Honorable Dan Miller The Honorable Charles Taylor House of
Representatives The Legal Services Corporation (LSC), operating
through grantees, provides legal assistance in civil matters to
low- income individuals. In the past year, LSC's Office of
Inspector General (OIG) and we have identified misreporting by
grantees on both the number of cases they closed during calendar
year 1997 and the number they had open at the end of that year.
The accuracy of the data is important because LSC has used case
statistics to seek increased funding for LSC, and Congress has
considered these case statistics in determining funding for LSC.
As agreed with your offices, we determined (1) what efforts LSC
and its grantees have made to correct problems with case service
reporting and (2) whether these efforts are likely to resolve the
case reporting problems that occurred in 1997. LSC revised its
written guidance and issued a new handbook to its grantees to
clarify case reporting requirements. Based on telephone interviews
with a sample of 79 LSC grantee executive directors, we estimate
that 90 percent of grantees viewed the new guidance as having
clarified reporting requirements, overall. Virtually all grantees
said they responded to the new requirements by making or planning
to make one or more changes to their program operations. However,
many grantees indicated that they were unclear about certain
aspects of LSC's reporting requirements, particularly regarding
(1) the specific information required on client assets, (2) the
information required for documenting citizenship/ alien
eligibility for services provided over the telephone, (3) the
criteria for avoiding duplicate counts of cases, and (4) who can
provide legal assistance to clients in order for the service to be
counted as a case. LSC initiated a self- inspection procedure in
which grantees were required to review their 1998 case data and
submit certification letters to LSC if they found that the extent
of error in their data was 5 percent or less. Grantees who could
not certify their 1998 data were required to develop Results in
Brief B-283551 Page 2 GAO/GGD-99-183 LSC Case Reporting Problems
corrective actions that would address the problems identified.
About 75 percent of the grantees submitted letters to LSC
certifying that the error rate in their 1998 data was 5 percent or
less, while about 25 percent of the grantees submitted letters to
LSC indicating that they could not certify their 1998 data.
According to LSC, about 30 of the 50 grantees with the largest
caseloads were unable to certify their 1998 case data. We could
not assess whether the number of certified and uncertified
programs that LSC obtained for 1998 was correct, lower, or higher
than it should be. This is because LSC did not provide grantees
with a standardized way of reporting their self- inspection
results, and LSC's instructions on how to conduct the self-
inspections may have led some of the smaller grantees to select
too few cases to reliably assess the amount of error in their case
data. In addition, some grantees did not correctly interpret LSC's
case reporting requirements. LSC is planning to consider
alternative ways of training grantee staff so that they gain a
better understanding of the reporting requirements. For these
reasons, we do not believe that LSC's efforts to date have been
sufficient to fully resolve the case reporting problems that
occurred in 1997. This report includes recommendations to LSC for
clarifying its case reporting guidance to grantees and improving
its procedures for conducting future self- inspections. LSC was
established in 1974 as a private, nonprofit, federally funded
corporation to provide legal assistance to low- income people in
civil matters. LSC provides the assistance indirectly through
grants to competitively selected local programs. LSC distributes
funds to the grantees on the basis of the number of low- income
persons living within a service area. Grantees may receive
additional funding from non- LSC sources. During 1998, LSC funded
262 local grantees that operated through approximately 900
neighborhood law offices employing about 3,600 attorneys and 1,400
paralegals. Each local program is governed by its own board of
directors and is required to spend at least 12.5 percent of its
LSC grant to encourage private attorney involvement (PAI) in
delivering legal services to low- income clients. In fiscal years
1998 and 1999, LSC received appropriations of $283 million and
$300 million, respectively. LSC's authorizing legislation
restricts it from engaging in lobbying; political activities;
class actions except under certain conditions; and cases involving
abortion, school desegregation, and draft registration or
Background B-283551 Page 3 GAO/GGD-99-183 LSC Case Reporting
Problems desertion from the military. Annual appropriations laws
have placed additional restrictions on the activities in which LSC
grantees can engage, even with non- LSC funds. In 1996, for
example, grantees were prohibited from engaging in challenges to
welfare reform, litigation on behalf of prisoners, representation
in drug- related public housing evictions, and representation of
certain categories of aliens. Grantees must serve only those
clients who meet financial and citizenship/ alien eligibility
requirements. With respect to client financial eligibility, local
programs are to establish their own criteria, which, in general,
require that clients' income not exceed 125 percent of the federal
poverty guidelines. With appropriate documentation of the
grantee's decision, clients who are between 125 percent and 187.5
percent of the federal poverty level may be found eligible. LSC
regulations require that grantees (1) adopt a form and procedure
to obtain eligibility information and (2) preserve that
information for audit by LSC. With regard to citizenship/ alien
eligibility, only citizens and certain categories of aliens are
eligible for services. For clients who are provided services in
person, a citizen attestation form or documentation of eligible
alien status is required. For clients who are provided services on
the telephone, grantees must document that they made inquiries
regarding the individuals' citizenship/ alien eligibility. LSC
uses a Case Service Reporting (CSR) system to gather quantifiable
information from grantees on the services they provide that meet
LSC's definition of a case. The CSR Handbook is LSC's primary
official guidance to grantees on how to record and report cases.
According to the 1999 CSR Handbook, which revised and expanded the
guidance in LSC's 1993 Handbook, information about cases is an
important indicator of the number of legal problems that programs
address each year, and LSC relies on such case information in its
annual request for federal funding for legal services. Audit
reports on five grantees issued by LSC's OIG between October 1998
and July 1999 reported that all five grantees misreported the
number of cases they had closed during calendar year 1997 and the
number of cases that remained open at the end of that year. The
OIG found that all five grantees overstated the number of closed
cases, while four overstated and one understated open cases. The
OIG attributed the overreporting to such factors as (1) counting
as cases telephone calls in which individuals were not provided
any legal assistance and only partial eligibility determinations
were made; (2) counting wholly non- LSC funded cases as LSC cases;
(3) double counting of the same cases; (4) reporting cases as
closed during B-283551 Page 4 GAO/GGD-99-183 LSC Case Reporting
Problems 1997, or as still open at the end of 1997, when service
ceased in prior years; and (5) counting as cases the provision of
services to over- income clients. In June 1999, in response to
Congress' request for information on whether the 1997 case data of
other LSC programs had problems similar to those reported by LSC's
OIG, we issued a report on our audit of five of LSC's largest
grantees: Baltimore, Chicago, Los Angeles, New York City, and
Puerto Rico. 1 We found similar types of reporting errors at the
five grantees and estimated that, overall, 75,000 of the 221,000
open and closed cases that the five grantees reported to LSC were
questionable. 2 Interviews that we conducted with LSC officials
and executive directors of the 5 audited grantees indicated that
they had taken or were planning to take steps to correct the
causes of these case- reporting problems. Our objectives were to
determine (1) what efforts LSC and its grantees have made to
correct problems with case service reporting, and (2) whether
these efforts are likely to resolve the case reporting problems
that occurred in 1997. To address the objectives, we reviewed
documents that contained LSC case reporting guidance, interviewed
cognizant officials at LSC headquarters, and conducted structured
telephone interviews with a random sample of grantee executive
directors. Specifically, we reviewed LSC regulations and LSC's
1993 and 1999 CSR Handbook, as well as supplemental guidance that
LSC distributed to grantees in the form of program letters and
frequently asked questions and answers about case reporting. We
also collected documentation and interviewed LSC officials and
grantee executive directors to gather information about a
selfinspection process that LSC required all grantees to undertake
in order to determine the accuracy of their 1998 case data. In our
telephone interviews with executive directors, we asked if the
directors viewed LSC's case reporting requirements as being clear,
what changes they have made or planned to make as a result of the
requirements, and the results of their self- inspection of 1998
case data. We also interviewed LSC officials and grantee executive
directors about areas of case reporting that they felt needed
further clarification. 1 Legal Services Corporation: Substantial
Problems in 1997 Case Reporting by Five Grantees (GAO/GGD-99-135R,
June 25, 1999). 2 Our estimate was based on file reviews of random
samples of cases that the five grantees reported as closed during
1997 and open at the end of 1997. In addition, we reviewed random
samples of potentially duplicate cases for four grantees. We
estimated that, overall, 75, 000 (+/- 6,100) cases were
questionable. Objectives, Scope, and Methodology B-283551 Page 5
GAO/GGD-99-183 LSC Case Reporting Problems To identify the
universe of LSC grantees, an LSC official referred us to the list
of LSC programs on the corporation's Internet Web site. As of July
15, 1999, the Internet site listed 256 programs. From this list,
we randomly selected 80 programs for our sample. We developed the
interview instrument, pretested it with executive directors from
two grantees, revised the instrument based on the pretest results,
and completed approximately 1- hour- long structured interviews by
telephone with the executive directors of 79 of the 80 programs,
for a 99 percent response rate. (After numerous attempts over
several days, we were unable to contact the executive director of
one LSC grantee.) Our sample was designed so that we could
generalize our findings with 95- percent confidence and a maximum
10- percent margin of error to the universe of LSC grantees. We
performed our work from July through September 1999 in accordance
with generally accepted government auditing standards. We
requested comments on a draft of this report from the President of
LSC. LSC's comments are discussed at the end of this letter and
included as appendix I. LSC issued a new CSR Handbook and
distributed other written communications intended to clarify
reporting requirements to its grantees. Most grantees indicated
that the new guidance helped clarify LSC's reporting requirements,
and virtually all of them indicated that they had or planned to
make program changes as a result of the requirements. Many
grantees, however, identified areas of case reporting that
remained unclear to them. To address specific problems identified
by the OIG and LSC's own internal program reviews, LSC has issued
revised reporting guidance. In November 1998, LSC issued the 1999
CSR Handbook, which replaced the 1993 CSR Handbook. The 1999
handbook instituted changes to some of LSC's reporting
requirements and provided more detailed information on other
requirements than previously existed. LSC also distributed program
letters and a list of frequently asked CSR questions and answers
that further elaborated on points made in the 1999 handbook. The
1999 CSR Handbook instituted several notable changes to case
reporting requirements. These included (1) procedures for timely
closing LSC Clarified CSR Requirements and Grantees Have Been
Making Program Changes to Comply With the Requirements, but Some
Grantees Remain Unclear About Certain Requirements LSC Issued New
CSR Guidance B-283551 Page 6 GAO/GGD-99-183 LSC Case Reporting
Problems of cases; (2) procedures for management review of case
service reports; (3) procedures for ensuring single recording of
cases; (4) requirements to report LSC- eligible cases, regardless
of funding source; and (5) requirements for reporting PAI cases
separately. In addition, grantees were required to use automated
case management systems and procedures that would ensure that
program managers had timely access to accurate information on
cases and the capacity to meet their reporting requirements. On
November 24, 1998, LSC informed its grantees that two of the
changes in the 1999 CSR Handbook were to be applied to the 1998
case data. The two changes pertained to timely closing of cases
and management review of case service reports. The timely closing
provision required grantees to ensure that cases in which legal
assistance had ceased in 1998, and was not likely to resume, would
be closed prior to grantees' submission of case service reports to
LSC in March 1999. To the extent practicable, cases in which the
only assistance provided to the client was counsel and advice,
brief service, or referral after legal assessment were to be
closed in the year in which these types of service were provided.
Cases involving other types of service were to be closed in the
year in which program staff determined that further legal
assistance was unnecessary, not possible, or inadvisable and a
closing memorandum or other case- closing notation was prepared.
The management review provision required the executive director,
or a designee, to review the program's case service reports prior
to their submission to LSC in order to ensure their accuracy and
completeness. The remaining new provisions of the 1999 CSR
Handbook were not applicable to 1998 cases. For example, for 1998,
there was no requirement for grantees to ensure that cases were
not double counted. For 1999, LSC is requiring the use of
automated case management systems and procedures to ensure that
cases involving the same client and specific legal problem are not
reported to LSC more than once. For 1998, grantees could report
only those cases that were at least partially supported by LSC
funds. For 1999, LSC is requiring grantees to report all LSC-
eligible cases, regardless of funding source. LSC intends to
estimate the percentage of activity spent on LSC service by
applying a formula that incorporates the amount of funds grantees
receive from other funding sources compared with the amount they
receive from LSC. For 1998, grantees were required to report their
LSC- funded PAI cases together with non- PAI cases. For 1999, PAI
cases are to be reported separately. B-283551 Page 7 GAO/GGD-99-
183 LSC Case Reporting Problems In addition to changing certain
reporting requirements, the 1999 handbook also provides more
detailed guidance to grantees than the 1993 handbook. For example,
the 1999 handbook provides more specific definitions of what
constitutes a case and a client for CSR purposes. The 1999
handbook also addresses documentation requirements that were not
discussed in the 1993 handbook. For example, the 1999 handbook
indicates that, except for telephone service cases, the client's
file must contain an attestation of citizenship or documentation
of alien eligibility in order for the case to be reported to LSC.
3 The 1999 handbook also imposes requirements for documentation of
information on client income and assets. The handbook states that,
for all cases reported to LSC, the eligibility documentation must
include specific information about income and assets. The 1999
handbook also contains a requirement that legal assistance, to be
counted as a case, must be provided by an attorney or paralegal.
On the basis of our survey, we estimate that over 90 percent 4 of
grantee executive directors viewed the changes in the 1999 CSR
Handbook as being clear overall, and virtually all of them planned
to or had made at least one change to their program operations as
a result of the revised case reporting requirements. Program
changes that were cited included revising policies and procedures,
providing staff training, modifying forms and/ or procedures used
during client intake, implementing computer hardware and software
changes, and increasing review of cases. Nearly half of the
grantees indicated that they planned to or had revised their
policies and/ or procedures to comply with the 1999 handbook
requirements, and slightly less than a third planned to or had
conducted staff training on the requirements. Respondents told us
that the focus of their training was on such issues as the current
definition of a case, how to determine and document clients'
financial eligibility, how to determine and document citizenship/
alien eligibility, timely closing of cases, and prevention of
duplicate case reporting. More than half of the grantees indicated
that they planned to or had changed their intake forms and/ or
procedures. Of the 41 respondents who 3 The requirement for
documentation is contained in LSC regulations, at 45 C. F. R.
1626. 6( a) and 1626. 7( a). However, the 1999 CSR Handbook
specified for the first time that the case could not be reported
to LSC if the documentation was missing. 4 All percentage
estimates presented in this report have 95 percent confidence
intervals with a margin of error of 10 percent or less. Grantee
Directors Reported That They Are Implementing Changes to Comply
With Reporting Requirements B-283551 Page 8 GAO/GGD-99-183 LSC
Case Reporting Problems made this comment, 26 said they were
making these changes in order to document client income and
assets. Slightly over 40 percent of the grantees reported that
they planned to or had made computer changes. The computer changes
included such actions as adding new fields to their automated case
management (e. g., to ensure that client eligibility and
acceptance information is recorded), making programming changes
(e. g., to identify duplicate cases, ensure that cases are not
assigned separate numbers for the same client with the same legal
problem, and better document client assets), and installing new
software (e. g., to generate reports so that they can track how
long cases are open). About 10 percent of grantee executive
directors indicated that they could better comply with CSR
requirements if they had uniform case management software. LSC,
too, believes that grantees' case management systems should
provide for more consistency in the collection and processing of
CSR information. According to an LSC official, LSC is developing a
strategy for modifying grantees' case management systems so that
data errors can be detected and prevented. As part of the
strategy, LSC has hired an expert in case management systems and
is working to develop standard input requirements for these
systems. LSC intends to work with case management system vendors
to modify the systems so that they prevent cases from being
accepted if grantees do not record the required compliance
information. LSC expects modifications to the case management
systems to be implemented in calendar year 2000 and available for
application to grantees' 2000 CSR data. In the nearer term, LSC
plans to work with a contractor to develop customized case
management queries that would enable grantees to detect and remedy
errors in their 1999 case data. In 1999, LSC plans to pilot test
the computer queries at five programs that have the most commonly
used case management systems. Nearly three- fourths of the
grantees indicated that they planned to or had increased their
review of cases to comply with LSC requirements. Respondents cited
such review activities as more intense monitoring of open cases,
to ensure that they are closed in a timely manner, and more
thorough monitoring of PAI cases. Some grantees said they have
directed increased attention to reporting requirements during
routine reviews of cases, while others said they instituted more
frequent reviews of cases. One executive director reported to LSC
that managing attorneys would meet with all case- handling staff
and reemphasize the importance of keeping accurate activity
records. These managing attorneys are to monitor compliance with
the requirement initially on a weekly basis and B-283551 Page 9
GAO/GGD-99-183 LSC Case Reporting Problems then on a random basis.
If deficiencies are discovered, the managing attorney is to direct
the case handler to correct them immediately, and the managing
attorney is to recheck the file within 48 hours to ensure
compliance with the directive. Another executive director told us
that his program had not previously reviewed CSR data. He said
that they now are reviewing reports, along with listings of open
and closed cases, to ensure their compliance with LSC guidelines.
Some executive directors said that they planned to become more
involved in reviewing case files and case management reports. Many
grantees reported making other efforts to comply with reporting
requirements, such as disseminating the new handbook to case
handlers, developing compliance checklists, emphasizing the
importance of compliance to staff, sending written instructions or
memos to staff, holding meetings and discussions, providing more
feedback to case handlers, and more stringently enforcing
requirements. Although most of the grantee executive directors
reported that the new LSC guidance helped clarify requirements,
many of them also indicated that they were still unclear about
certain requirements and that additional clarification was needed.
Among the areas of confusion or uncertainty that executive
directors identified were requirements pertaining to asset and
citizenship/ alien eligibility documentation, single recording of
cases, and who can provide legal services. Asset documentation:
The 1999 CSR Handbook states that, for all cases to be reported to
LSC, the eligibility documentation should include specific
information about income and assets. About 30 percent of the
executive directors indicated that LSC's requirements for
documenting client assets were clear only to some, to little, or
to no extent. Of the 24 respondents who gave this response, 23
made comments to the effect that LSC should clarify what it means
by assets and asset limits and/ or clarify its documentation
requirements for client assets. In a July 14, 1999, program letter
to grantees, LSC noted that although many grantees inquire about
applicants' assets, they do not consistently document either the
inquiries or the applicants' responses. In its program letter, LSC
sought to clarify its requirements for documenting and preserving
the asset information obtained from each applicant. Because we
conducted our telephone survey in late July and early August, we
do not know how many of the executive directors with whom we spoke
had reviewed LSC's July 14 guidance and felt that it sufficiently
clarified their questions concerning asset documentation. However,
two respondents told us that LSC's July 14 guidance was still
unclear. One respondent said, for example, that the Many Grantees
Remain Unclear About Certain Reporting Requirements B-283551 Page
10 GAO/GGD-99-183 LSC Case Reporting Problems program letter left
unclear the meaning of the term household goods, whether exempted
assets had to be documented, who was to value the goods, and who
was to determine what to count and what not to count as part of
assets. Our own analysis of LSC guidance on the asset issue
indicated that LSC has not been consistent in its directives to
grantees about the specificity of asset information they must have
in order to comply with CSR reporting requirements. For example,
in its March 24, 1999, communication on frequently asked CSR
questions and answers, LSC instructed grantees to document, at
minimum, the total amount of household assets accessible to the
client. In its July 14, 1999, program letter, LSC instructed
grantees to identify and document all of the liquid and nonliquid
assets of all persons who are resident members of a family unit.
Citizenship/ alien eligibility documentation requirements for
telephone cases: The 1999 CSR Handbook does not explicitly address
the citizenship or eligible alien status documentation
requirements for situations where assistance is provided only over
the telephone. Nearly one- fourth of the executive directors
indicated that LSC's documentation requirements in this area were
clear only to some, to little, or to no extent. Of 19 respondents
who gave this response in our survey, 15 said that more
clarification was needed on the grantees' documentation
responsibilities. Respondents said, for example, that they were
confused about exactly when they needed to obtain a written
attestation, whether it was sufficient simply to record that
questions about citizenship/ alien eligibility had been asked,
whether certain types of service required documentation while
others did not, and whether the requirement changed if an
individual receiving assistance over the telephone came into the
office to drop off documents. In its July 14, 1999, program letter
to grantees, LSC sought to clarify its requirements for
documenting citizenship/ alien eligibility information in
telephone cases. LSC stated that it requires recipients to make
appropriate inquiry of every telephone applicant and record the
inquiry and response. All such documentation is to be maintained
in the client file. We do not know how many of the respondents to
our survey were familiar with LSC's July 14 guidance when we
interviewed them in late July and early August. Single recording
of cases: The 1999 CSR Handbook requires that programs ensure that
cases involving the same client and specific legal problem are not
recorded and reported to LSC more than once. Over one- fourth of
the executive directors indicated that the requirement for
preventing duplicate case reporting was clear only to some, to
little, or to no extent. Of the 22 respondents who gave this
response in our survey, 17 said that the B-283551 Page 11 GAO/GGD-
99-183 LSC Case Reporting Problems distinction between specific
and related legal problems is difficult to determine. Several
respondents cited examples in family law cases where more than one
problem can arise within one family case. Depending on the
situation, it was not clear to them at what point legally related
issues become separate enough to count as separate cases. One
respondent indicated that attorneys in his program have a hard
time interpreting this requirement, and that he got calls every
week about this issue. Provider of legal assistance: The CSR
handbook requires that legal assistance be provided by an attorney
or paralegal in order for a service to be called a case. 5
Slightly over 40 percent of executive directors indicated that LSC
was clear only to some, to little, or to no extent about who can
provide such legal assistance. Respondents noted in our telephone
survey, for example, that the terms case handler, and paralegal
are not clearly defined, and that they did not know whether
nonlawyers (e. g., intake workers) who are supervised by lawyers
can provide legal assistance. Comments made to us by the executive
directors revealed that grantees held varying views about who can
provide legal assistance to clients. Several respondents
specifically stated that this is an important issue that LSC
should address in its handbook. Our own analysis of LSC guidance
on this issue indicates that LSC has not been consistent in its
advice to grantees. For example, in its communication with
grantees on frequently asked CSR questions and answers, LSC stated
that a telephone conversation between an intake specialist and a
caller who was accepted for service can be counted as a case if
the caller received some advice that addressed a specific legal
problem. LSC officials told us that LSC's current position on this
issue is that the person giving legal advice must be someone who
is authorized to practice law or is under the supervision of an
attorney in accordance with local rules of practice. Therefore,
legal advice can be given by a (1) lawyer, (2) paralegal, or (3)
intake specialist or law student under the supervision of a
lawyer, as long as the assistance does not violate local rules of
practice. 5 The 1999 CSR handbook does not explicitly state that
legal assistance must be provided by an attorney or paralegal.
However, section VIII of the handbook states that, to be reported
as a case, the legal assistance must meet the definition of case
in 45 CFR 1635. 2( a). This regulation defines a case as a form of
program service in which an attorney or paralegal provides legal
services. B-283551 Page 12 GAO/GGD-99-183 LSC Case Reporting
Problems LSC sought to determine the accuracy of grantees' case
data by requiring that grantees do self- inspections of their open
and closed caseload data for 1998. Grantees were required to
determine whether the error rate in their data exceeded 5 percent.
If the error rate was 5 percent or less, they could certify that
their data were substantially correct. If the error rate was
higher than 5 percent, they were to determine how the problems
identified could be addressed. LSC found that about three- fourths
of the grantees were able to certify to the substantial accuracy
of their data. LSC used the results of the self- inspections to
estimate the total number of case closings in 1998. Our review of
LSC's self- inspection process raised concerns about the accuracy
and interpretation of the results, and what the correct number of
certifying programs should be. On May 14, 1999, LSC issued a memo
to all grantees instructing them to complete a self- inspection
procedure by July 1, 1999. The purpose of the self- inspection was
to ensure that (1) grantees were properly applying instructions in
the 1999 edition of the CSR Handbook that were applicable to the
1998 data, and (2) LSC had accurate case statistical information
to report to Congress for calendar year 1998. LSC provided
detailed guidance to grantees on the procedures for the
selfinspection. Each grantee was to select and separately test
random samples of open and closed cases to determine whether the
number of cases it reported to LSC earlier in the year was
correct. Grantees were to verify that the case file contained a
notation of the type of assistance provided, the date on which the
assistance was provided, and the name of the case handler
providing the assistance. Grantees were also to determine whether
assistance had ceased prior to January 1, 1998; was within certain
service categories as defined by the 1999 handbook; was provided
by an attorney or paralegal; and was not prohibited or restricted.
Finally, grantees were to verify that each case had eligibility
information on household income, household size, household assets,
citizenship attestation for in- person cases, and indication of
citizenship/ alien status for telephone- only cases. According to
LSC officials, the self- inspection was a single procedure that
was undertaken within a limited time period, and LSC did not
expect the self- inspection to resolve all case- reporting
problems. In requiring grantees to verify that their 1998 case
files contained information on client assets and an indication of
citizenship/ alien status for telephone- only cases, LSC imposed
stringent criteria on the selfinspections. The criteria were
stringent in that LSC had not promulgated explicit documentation
requirements related to these issues until it released the 1999
CSR Handbook and July 14 program letter. Grantees Most Grantees
Certified to the Accuracy of Their 1998 CSR Data, but Questions
About Data Accuracy and Interpretation Remain LSC Grantees
Conducted Self- Inspections of 1998 CSR Data B-283551 Page 13
GAO/GGD-99-183 LSC Case Reporting Problems were not required to
apply these new documentation requirements to their 1998 case
data. Recognizing that in 1998 many grantees did not keep
sufficient documentation on assets or citizenship/ alien
eligibility to comply with the stringent self- inspection
requirements, LSC allowed grantees some latitude in determining
whether they were in fundamental compliance with the requirement
that legal assistance only be provided to eligible individuals. In
conversations with grantee executive directors during the self-
inspection, LSC officials advised them that if they had a level of
certainty that their program staff had asked questions about
assets, and if their certainty was sufficient for them to sign a
form attesting to this, that was an acceptable basis for asserting
compliance with the reporting requirement for assets. Similarly,
if grantees were sufficiently certain that, for telephone cases,
their program staff had asked questions about citizenship/ alien
status but had not documented the inquiry, that too was
acceptable. Finally, LSC allowed grantees some latitude with
respect to the requirement that an attorney or paralegal must be
the provider of legal assistance. LSC advised grantees during the
self- inspection that it was acceptable for them to count as valid
cases instances in which legal advice was given by a (1) lawyer,
(2) paralegal, or (3) intake specialist or law student under the
supervision of a lawyer, as long as the assistance did not violate
local rules of practice. If any single aspect of a case failed to
meet LSC's requirements, the case was to be classified as an error
for reporting purposes. If the grantees found that their CSR case
sampling had an error rate of 5 percent or less, the program
directors and policy board chairs were to sign a certification
form and return it to LSC. Grantees who could not certify to the
correctness of their 1998 CSR data were to submit a letter to LSC
describing (1) the problems they had identified during the self-
inspection process and (2) the corrective actions they had
instituted to address the problems. Grantees could resubmit their
1998 CSR data to LSC if they identified one or more problems in
the random sample and corrected their entire 1998 database so that
the problems no longer appeared. If, by correcting the problems,
the error rate in the data was reduced to 5 percent or less, the
grantees could resubmit their 1998 data along with a signed
certification attesting to the substantial accuracy of the
resubmitted data. In this way, grantees who were unable to certify
at one point in time could certify at a later point in time. As of
July 29, 1999, 26 grantees had resubmitted their 1998 CSR data
after having made corrections to the data, and 20 of them had
certified their data. B-283551 Page 14 GAO/GGD-99-183 LSC Case
Reporting Problems According to LSC officials, about three-
fourths of the grantees certified the accuracy of their 1998 case
data. As of August 26, 1999, LSC documents indicated that 199 of
261 grantees 6 (76 percent) reported substantially correct CSR
data to LSC. The remaining 62 grantees (24 percent) did not
certify to LSC that their CSR data were substantially correct. On
the basis of the self- inspection results, LSC estimated that
grantees closed 1. 1 million cases in 1998. LSC officials told us
that they were surprised that such a large number of grantees were
able to certify their 1998 CSR data. They attributed the lower-
than- expected error rates to the following factors: The self-
inspection did not attempt to identify duplicate cases. LSC
officials explained that, during most of 1998, LSC did not have a
standard regarding duplicate cases, and that they believed it
would have been too burdensome on grantees for LSC to require them
to apply the new standard retroactively. According to an LSC
official, duplicate cases are best caught at the time of intake.
The official also noted that duplicate cases had not in the past
been found to be a major problem in comparison with other
identified problems. Grantees received the new 1999 CSR Handbook
in November1998, and a number of them had implemented the
requirements that applied to their 1998 data by the time they
submitted their 1998 case statistics. Grantees were aware of the
problem that the OIG had identified with cases coded as referral
after legal assessment. That is, some programs had inappropriately
reported as CSR cases numerous instances of making a telephone
referral without providing legal advice and/ or without
documenting an individual's eligibility. Because grantees had been
sensitized to this issue, LSC officials believed that they were
less likely to count these referrals as cases in 1998. According
to LSC officials, CSR problems were more common at larger grantees
that had heavier case loads and multiple branches. They noted that
approximately 30 of the 50 grantees with the largest caseloads
were unable to certify their 1998 data. They hypothesized that
larger programs may have difficulty certifying if such programs
have (1) numerous branch offices, one or more of which are out of
compliance with regulations and causing reporting problems for the
entire program, and/ or (2) numerous 6 LSC funded 262 programs in
1998. Funding for one program was discontinued in 1999, and LSC
has no self- inspection results for this program. Most Grantees
Certified Their 1998 CSR Data B-283551 Page 15 GAO/GGD-99-183 LSC
Case Reporting Problems sources of funding and more compliance
requirements, which could add to the complexity of their reporting
process. The officials acknowledged that these hypotheses needed
further exploration. LSC officials also reported that
noncertifying grantees identified two principal problems. One
pertained to the lack of citizenship attestations in case files.
The second pertained to the lack of information on client assets.
LSC officials also noted that some grantees had reported matters,
such as referrals, as cases, and some had reported cases as open
when they did not meet the timely closing requirement. In our
telephone interviews with 79 executive directors, 24 reported that
they did not certify their 1998 data to LSC. Factors that the
executive directors of noncertifying programs believed greatly
affected the errors in their 1998 data included lack of clear
guidance from LSC (cited by 11 respondents), computer problems
(cited by 7 respondents), and insufficient attention by their
programs to administrative matters (cited by 5 respondents).
Computer problems that were noted included difficulties merging
databases and using several different software packages, and
problems with upgrading the computer system. Based on the self-
inspection results, LSC estimated that its grantees closed about
1.1 million cases in 1998. LSC arrived at this figure by
subtracting the total number of closed cases estimated to be in
error (135,498) from the total number of cases that grantees
reported to LSC (1,260,351). 7 As of July 29, 1999, LSC estimated
that its grantees' total closed caseload for 1998 was 1,124,853. 8
LSC intends to report only the estimated number of cases closed in
1998, even though the self- inspections were to encompass both
open and closed cases. According to LSC officials, LSC has less
confidence in the open- case numbers than the closed- case
numbers. One reason for this is that they believe open cases are
more likely than closed cases to have timely closing 7 LSC
performed this calculation separately for certifying and
noncertifying programs. For noncertifying programs, LSC determined
that 96, 273 cases were reported in error out of grantees'
original submission of 475,856 closed cases. For certifying
programs, LSC subtracted 5 percent, or 39, 225 cases, from
grantees' original submission of 784, 495 closed cases. LSC
reduced the case statistics of certifying programs by 5 percent to
ensure that there would be no overstatement of cases for these
programs. 8 In April 1999, prior to the self- inspection, LSC's
CSR database indicated that grantees closed 1,338,976 cases in
1998. The difference between the April 1999 figure and LSC's July
1999 estimate of 1,260,351 closed cases that is, the number
reported closed prior to adjusting for the self- inspection
results is attributable to the fact that some grantees who found
data errors corrected them and resubmitted their statistical
results to LSC. Those who initially did not certify their data may
have later certified them if their corrections reduced their error
rate to 5 percent or less. LSC Used Self- Inspection Results to
Estimate Number of Cases Closed B-283551 Page 16 GAO/GGD-99-183
LSC Case Reporting Problems problems. A second reason is that some
grantees experienced problems when converting to a new computer
system. System conversions sometimes caused dates to be lost or
cases to be mistakenly coded as being open. Our review of LSC's
self- inspection results raised some concerns about LSC's
interpretation of the results and about the accuracy of the data
provided to LSC by grantees. As a result, we could not assess
whether the number of certified programs and case closures that
LSC estimated for 1998 is correct, lower, or higher than it should
be. Although LSC provided instructions to grantees on how they
should select test samples and what case information they should
verify, LSC did not issue standardized procedures for grantees to
use in reporting the results of their self- inspections. Grantees
that could not certify their data wrote letters to LSC in which
they described the errors they uncovered. The letters contained
varying degrees of detail about the errors. Some programs provided
an overall error rate and did not separately report how much error
they found in open and closed cases. Others provided detailed
information on both the number of errors found in open and closed
cases, respectively, as well as the number of errors broken out by
type. Since LSC did not have a standard protocol for collecting
the results of the selfinspections, in some cases LSC officials
had to rely on interpreting grantee letters that described the
problems that were discovered. LSC officials believe that their
numerous contacts with grantees who called them to ask questions
about the self- inspections, combined with their analysis of the
grantee letters, enabled them to correctly determine the number of
certifying programs and estimate the number of closed cases. We
are uncertain how many programs should have been counted as
certified because we are uncertain if LSC applied a consistent
definition of certification. Most programs that were on LSC's
certification list determined that they had error rates of 5
percent or less for both open and closed cases. However, LSC
placed some programs on the certified list if the program's
overall error rate for closed cases was 5 percent or less, even if
the overall error rate actually was higher than 5 percent. We
encountered this situation in two instances in which executive
directors told us in telephone interviews that they did not
certify their CSR data because their overall error rate exceeded 5
percent. However, these programs appeared on LSC's list of
certified programs. When we asked an LSC official about this, he
told us that they advised grantees that if their closed case error
rate did not exceed 5 percent, they should partially certify their
data. In response to our inquiry, the official reviewed the Self-
Inspection Results Raised Concerns B-283551 Page 17 GAO/GGD-99-183
LSC Case Reporting Problems certification letters submitted by
nearly 200 grantees, and he identified 5 certified programs whose
error rates for open cases exceeded 5 percent. Given that some
grantees submitted only an overall estimate of data error, we do
not know how many programs qualified to be certified overall, just
for closed cases, or just for open cases. In another instance, an
executive director told us that she did not certify her program
because 8 of the 14 case- handling offices had error rates
exceeding 5 percent. Nonetheless, this program appeared on LSC's
certified list. An LSC official explained that, after reviewing
the information provided by this program, the official agreed that
the program should not have been classified as certified. In a
fourth instance, in which an executive director reported to us
that he did not certify his program, an LSC official said that,
although they thought the grantee's data had been corrected, the
program had not yet certified its data and might need to do
additional sampling. We are also concerned that LSC's instructions
to grantees on how to conduct the self- inspections may have led
some of the smaller grantees to select too few test cases to make
a reliable assessment of the proportion of error in their case
data. For example, LSC instructed grantees to select every tenth
case for review if the program handled less than 1,000 cases in
1998. This was to be done separately for open and closed cases.
Based on 1998 case statistics that grantees submitted to LSC
between January and March 1999, several programs would have based
their certification determinations on reviews of a relatively
small number of cases. Seven grantees had fewer than 300 closed
cases, and 43 grantees had fewer than 300 open cases. Three
programs had fewer than 300 cases combined. The smallest program
would have based its self- inspection on 3 closed cases and 1 open
case. We believe that, in general, samples of 30 or fewer cases
are too small to provide reliable estimates of the total number of
case data errors. 9 Because these were smaller grantees, this
limitation would have had little effect on LSC's estimate of total
closed caseload. However, it could have affected by either
overstating or understating LSC's count of the number of certified
programs. LSC does not know how well grantees conducted the self-
inspection process, nor how accurate the results are. We spoke
with several executive 9 To illustrate, a program that reviewed 30
closed cases and found a single error would have had a 3 percent
error rate. However, the margin of error based on a sample size of
30 would range from about 0.1 percent to 17 percent at the 95
percent statistical confidence level. If the program found two
errors, it would have had a 7 percent error rate, with a margin of
error ranging from 1 percent to 22 percent. Statistically, the
smaller the number of cases sampled, the larger the margin of
error. B-283551 Page 18 GAO/GGD-99-183 LSC Case Reporting Problems
directors who did not correctly follow LSC's reporting
requirements. In one case, the executive director sought
clarification from LSC headquarters about the use of a CSR closure
code and was given incorrect verbal guidance. The situation
concerned whether applicant files could be closed as client
withdrew cases when individuals who were accepted for service and
assigned to staff attorneys did not subsequently appear for a
meeting with the attorney. According to both the executive
director and an LSC official, the executive director was told that
if the program attempted to contact these individuals, by
telephone or letter, to determine whether they were still
interested in obtaining legal assistance, they could be counted as
cases. The executive director told us that she would review her
entire database for this type of error and make corrections. This
program was included on LSC's certified list, but we do not know
whether the program would stay on that list if all the errors were
identified. Another executive director told us that he was
concerned that LSC did not want grantees to count assistance over
the telephone as a case. This is not an entirely correct
interpretation of LSC guidance since, under certain conditions,
LSC permits legal assistance over the telephone to be counted as a
case. Although this program was included on LSC's certified list,
any valid telephone cases that were not counted would have
erroneously increased its error rate. A third executive director
told us that the timely closing rule required him to close new
cases in December. If the person sought assistance with a similar
problem in January, it would be treated as a new case because a
new reporting year had begun. In a March 1999 written
communication intended to supplement the 1999 handbook, LSC
advised grantees to exercise discretion about when to close cases
that were opened near the end of the year. LSC did not require
grantees to close all cases at year- end. These examples
illustrate the possibility that incorrect interpretations of LSC
guidance may have resulted in some programs certifying their 1998
data when they shouldn't have, and other programs not certifying
their 1998 data when they should have. An LSC official told us
that, although they have conducted CSR training sessions for
grantee executive directors, there are thousands of case handlers
in grantee offices who have not received similar training. The
official acknowledged that written guidance and telephone contacts
with grantees may not be sufficient to ensure correct and
consistent understanding of reporting requirements, and that LSC
may consider alternative ways of providing training to staff, such
as through videos. B-283551 Page 19 GAO/GGD-99-183 LSC Case
Reporting Problems Incorrect interpretations of LSC guidance could
also have affected the accuracy of LSC's estimate of closed cases.
The Inspector General told us that his office has completed audits
of six grantees' 1998 case data. One additional audit of a
grantee's CSR data had not yet been completed. The six completed
audits included both certifying and noncertifying grantees.
According to the Inspector General, the results of the six
completed audits would be provided to Congress by September 29,
1999. LSC officials told us that the self- inspection was valuable
and that LSC plans to have grantees complete self- inspections
again early next year as part of the 1999 CSR reporting process.
We agree that a self- inspection process can be valuable, provided
that grantees have clear, consistent, and appropriate guidance on
the procedures for reviewing their case data, determining whether
to certify the data, correcting errors, and providing their
results to LSC in a standardized way that facilitates validation
of the results. LSC's 1999 CSR Handbook and other written
communications have improved the clarity of reporting requirements
for its grantees. However, many grantees remain unclear about and/
or misunderstood certain aspects of the reporting requirements.
LSC's practice of disseminating guidance primarily by written or
telephone communications may not be sufficient to ensure that
grantees correctly and consistently interpret the requirements.
LSC sought to determine the accuracy of grantees' 1998 case
statistics by requiring grantees to conduct self- inspections.
However, we do not know the extent to which the results of the
self- inspection process are accurate. The validity of the results
are difficult to determine because LSC did not standardize the way
that grantees were to report their results, some of the grantees
used samples that were too small to assess the proportion of error
in their data, some grantees did not correctly follow LSC's
reporting guidance, and LSC had done no verification of the
grantees' self- inspection procedures. We do not believe that
LSC's efforts, to date, have been sufficient to fully resolve the
case reporting problems that occurred in 1997. We recommend that
the President of LSC clarify and disseminate information on the
specific information on client assets that grantees must obtain,
record, and maintain; Conclusions Recommendations B-283551 Page 20
GAO/GGD-99-183 LSC Case Reporting Problems clarify and
disseminate information on the types of citizenship/ alien
eligibility information grantees must obtain, record, and maintain
for clients who receive legal assistance only over the telephone;
clarify and disseminate LSC's criteria for single recording of
cases; clarify and disseminate LSC's policy concerning who can
provide legal assistance to clients for the service to be counted
as a case; explore options for facilitating correct and
consistent understanding of reporting requirements, such as
developing and disseminating a training video for grantee staff;
develop a standard protocol for future self- inspections to ensure
that grantees systematically and consistently report their results
for open and closed cases; direct grantees to select samples for
future self- inspections that are sufficient to draw reliable
conclusions about magnitude of case data errors; and finally,
ensure that procedures are in place to validate the results of
LSC's 1998 self- inspection, as well as of any future self-
inspections. The President of LSC provided written comments on a
draft of this report, which are printed in full in appendix I. LSC
generally agreed with our findings and stated that it intends to
implement our recommendations. Concerning the issue of the clarity
of case reporting guidance, LSC's letter stated that LSC is
troubled by our finding that some grantees are continuing to have
difficulty in this area. LSC's letter reiterated a point that we
made in the report; namely, that the overlap in time between our
data collection effort and LSC's distribution to grantees of a
July 1999 program letter intended to clarify client eligibility
documentation requirements may have caused some of the executive
directors not to factor in the new guidance when they responded to
our telephone interview questions. Although, as we state on pages
9 and 10 of the report, some respondents may not have been
familiar with LSC's July 1999 guidance, two respondents told us
that they were familiar with the new guidance and that they were
still unclear about the requirements dealing with assets. We also
state on page 10 of the report that our own analysis indicated
that LSC was not consistent in its guidance to grantees on what
asset information they needed to obtain from clients. Therefore,
we do not believe that grantees' Agency Comments B-283551 Page 21
GAO/GGD-99-183 LSC Case Reporting Problems lack of clarity
concerning reporting requirements on client eligibility was due
solely to the overlap in the time period of our data collection
and LSC's distribution of the July 1999 program letter. With
respect to grantees' lack of clarity about how to determine
duplicate case counting, LSC indicated that it will consider
providing additional guidance to grantees through further
revisions to the CSR handbook. LSC also indicated that it will
revise the handbook to clarify the issue of when legal assistance
by nonlawyers can be reported as a case. LSC also reiterated that
it is developing additional methods to supplement written
guidance, including (1) conducting training and technical
assistance, (2) developing case management standards that would
detect and prevent cases from being accepted and reported if the
required eligibility documentation was not obtained, and (3)
developing database queries for grantees to apply to their case
management systems to identify instances where cases lacked
required compliance documentation. Concerning the self- inspection
results, LSC noted that its principal purpose in requiring
grantees to self- inspect their 1998 case data was to ensure that
case- reporting guidance was being properly applied. LSC stated
that assessing the accuracy of the 1998 case statistics that
grantees submitted to LSC in March 1999 was a secondary purpose of
the self- inspection. LSC believes that the self- inspection
increased grantees' awareness of reporting requirements and
prompted them to make changes in their case reporting practices.
Indeed, in LSC's view, the self- inspection requirement resulted
in more program improvements than did the 1999 CSR handbook and
other written guidance issued by LSC because grantees that
identified significant problems were required to take corrective
actions. LSC notes correctly that, in relation to the self-
inspection, our report focuses on the accuracy of LSC's
determination of the number of grantees that certified their data,
and the factors that affected the accuracy of LSC's estimate of
the total number of closed cases in 1998. Our review focused on
the stated goals of the self- assessment. Although we did not
assess whether, relative to LSC's other efforts, the self-
inspection had a greater effect on grantees' awareness of and
compliance with CSR reporting, we do report that 26 grantees made
corrections to their 1998 data as a result of their
selfinspections. As arranged with your offices, unless you
publicly announce the contents of this letter earlier, we plan no
further distribution until 7 days after the date of this letter.
At that time, we will send a copy of the report to the Chairmen
and Ranking Minority Members of LSC's appropriations and B-283551
Page 22 GAO/GGD-99-183 LSC Case Reporting Problems legislative
committees and to Mr. John McKay, the President of LSC. The major
contributors to this report are acknowledged in appendix II. If
you or your staff have any questions concerning this report,
please contact me or Evi L. Rezmovic on 202- 512- 8777. Laurie E.
Ekstrand Director, Administration of Justice Issues Page 23
GAO/GGD-99-183 LSC Case Reporting Problems Appendix I Comments
From the Legal Services Corporation Page 24 GAO/GGD-99-183 LSC
Case Reporting Problems Appendix I Comments From the Legal
Services Corporation Page 25 GAO/GGD-99-183 LSC Case Reporting
Problems Now on pp. 9- 11. Appendix I Comments From the Legal
Services Corporation Page 26 GAO/GGD-99-183 LSC Case Reporting
Problems Appendix II GAO Contacts and Staff Acknowledgments Page
27 GAO/GGD-99-183 LSC Case Reporting Problems Laurie E. Ekstrand,
202- 512- 8777 Evi L. Rezmovic, 202- 512- 8777 In addition to
those named above, Mark Tremba, Kristeen McLain, Jan Montgomery,
David Alexander, Barry Seltser, Lemuel Jackson, Brian Lipman, and
Walter Vance made key contributions to this report. GAO Contacts
Acknowledgments Page 28 GAO/GGD-99-183 LSC Case Reporting Problems
Ordering Information The first copy of each GAO report and
testimony is free. Additional copies are $2 each. Orders should be
sent to the following address, accompanied by a check or money
order made out to the Superintendent of Documents, when necessary.
VISA and MasterCard credit cards are accepted, also. Orders for
100 or more copies to be mailed to a single address are discounted
25 percent. Order by mail: U. S. General Accounting Office P. O.
Box 37050 Washington, DC 20013 or visit: Room 1100 700 4 th St. NW
(corner of 4 th and G Sts. NW) U. S. General Accounting Office
Washington, DC Orders may also be placed by calling (202) 512-
6000 or by using fax number (202) 512- 6061, or TDD (202) 512-
2537. Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any
list from the past 30 days, please call (202) 512- 6000 using a
touchtone phone. A recorded menu will provide information on how
to obtain these lists. For information on how to access GAO
reports on the INTERNET, send e- mail message with info in the
body to: info@ www. gao. gov or visit GAO's World Wide Web Home
Page at: http:// www. gao. gov United States General Accounting
Office Washington, D. C. 20548- 0001 Official Business Penalty for
Private Use $300 Address Correction Requested Bulk Rate Postage &
Fees Paid GAO Permit No. G100 (182080)

*** End of document. ***