Mutual Fund Industry: SEC's Revised Examination Approach Offers
Potential Benefits, but Significant Oversight Challenges Remain
(17-AUG-05, GAO-05-415).
As the frontline regulator of mutual funds, the Securities and
Exchange Commission (SEC) plays a key role in protecting the
nearly half of all U.S. households owning mutual funds, valued
around $8 trillion in 2005. Mutual fund abuses raised questions
about the integrity of the industry and quality of oversight
provided by SEC and self-regulatory organizations (SRO) that
regulate broker-dealers selling funds. This report assesses (1)
changes SEC has made to, or is planning for, its mutual fund exam
program; (2) key aspects of SEC's quality control framework for
routine fund exams; and (3) the adequacy of SEC's oversight of
NASD and the New York Stock Exchange in protecting shareholders
from mutual fund sales abuses.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-05-415
ACCNO: A33317
TITLE: Mutual Fund Industry: SEC's Revised Examination Approach
Offers Potential Benefits, but Significant Oversight Challenges
Remain
DATE: 08/17/2005
SUBJECT: Brokerage industry
Mutual funds
Program abuses
Quality control
Regulatory agencies
Securities regulation
Self-regulatory organizations
Monitoring
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-05-415
United States Government Accountability Office
GAO Report to Congressional Requesters
August 2005
MUTUAL FUND INDUSTRY
SEC's Revised Examination Approach Offers Potential Benefits, but Significant
Oversight Challenges Remain
a
GAO-05-415
[IMG]
August 2005
MUTUAL FUND INDUSTRY
SEC's Revised Examination Approach Offers Potential Benefits, but Significant
Oversight Challenges Remain
What GAO Found
SEC is initiating several changes intended to strengthen its mutual fund
exam program but faces challenges overseeing the fund industry. In the
wake of the fund abuses, SEC has revised its past approach of primarily
conducting routine exams of all funds on a regular schedule. It concluded
these exams were not the best tool for identifying emerging problems,
since funds were not selected for examination based on risk. To quickly
identify problems, SEC is shifting resources away from routine exams to
targeted exams that focus on specific risks. It will conduct routine exams
on a regular schedule but only of funds deemed high risk. SEC also is
forming teams to monitor some of the largest groups of advisers and funds.
Although SEC is seeking to focus its resources on higher risk funds and
activities, the resource tradeoffs it made in revising its oversight
approach raise significant challenges. The tradeoffs may limit SEC's
capacity not only to examine funds considered lower risk within a 10-year
period but also to accurately identify which funds pose higher risk and
effectively target them for routine examination. Potentially taxing its
resources further, SEC recently adopted a rule to require advisers to
hedge funds (investment vehicles generally not widely available to the
public) to register with it. This rule is expected to increase SEC's exam
workload, but the precise extent is not yet known.
SEC has integrated some quality controls into its routine exams, but
certain aspects of its framework could be improved. It relies on
experienced staff to oversee all exam stages but does not expressly
require supervisors to review work papers or document their review. GAO
found deficiencies in key SEC exam work papers, raising questions about
the quality of supervisory review. SEC also does not require examiners to
prepare written exam plans, though they use considerable judgment in
customizing each exam. Written plans could serve as a guide for conducting
exams and reviewing whether exams were completed as planned. As done by
other regulators, SEC also could review a sample of work papers to test
compliance with its standards.
A primary tool that SEC uses to assess the adequacy of SRO oversight of
broker-dealers offering mutual funds provides limited information for
achieving its objective and imposes duplicative costs on firms. To assess
SRO oversight, SEC reviews SRO exam programs and conducts oversight exams
of broker-dealers, including their mutual fund sales practices. SEC's
oversight exams take place 6 to 12 months after SROs conduct their exams
and serve to assess the quality of SRO exams. However, GAO reported in
1991 that SEC's oversight exams provided limited information in helping
SROs to improve their exam quality, because SEC and the SROs used
different exam guidelines and their exams often covered different periods.
GAO found that these problems remain, raising questions about the
considerable resources SEC devotes to oversight exams. GAO also found that
SEC has not developed an automated system to track the full scope of work
done during its oversight exams. Thus, SEC cannot readily determine the
extent to which these exams assess mutual fund sales practices.
United States Government Accountability Office
Contents
Letter
Results in Brief
Background
SEC's Revised Mutual Fund Examination Program Offers Potential
Benefits but also Poses Significant Oversight Challenges SEC Can Improve
Certain MutualFund Examination Quality Control Measures SEC's Oversight
Examinations of Broker-Dealers Provide Limited
Information on the Adequacy of SRO Oversight Conclusions Recommendations
for Executive Action Agency Comments and Our Evaluation
1 4 7
9
19
28 35 36 37
Appendixes
Appendix I: Scope and Methodology 41
Appendix II: Securities and Exchange Commission's (SEC) Broker-Dealer
Examination Guidance and Training 43
Appendix III: Comments from the Securities and Exchange Commission 48
Appendix IV: GAO Contact and Staff Acknowledgments 52
Table Table 1: Strategic Areas Covered by SEC's Risk Scorecards as of
April 2005
Figures Figure 1: Number of Examiners and Entities Subject to
Examination by SEC and Federal Bank Regulators in
2004 16
Figure 2: Violations Found by SEC during Oversight Examinations
of NASD but Not Found by NASD, and SEC Oversight
Examinations Conducted of NASD, Fiscal Years
2002-2004 32
Contents
Abbreviations
FDIC Federal Deposit Insurance Corporation
MRO Midwest Regional Office
NERO Northeast Regional Office
NFA National Futures Association
NYSE New York Stock Exchange
OCC Office of the Comptroller of the Currency
OCIE Office of Compliance Inspections and Examinations
ORA Office of Risk Assessment
PDO Philadelphia District Office
SEC Securities and Exchange Commission
SRO self-regulatory organization
STARS super tracking and reporting system
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
A
United States Government Accountability Office Washington, D.C. 20548
August 17, 2005
The Honorable Barney Frank Ranking Member Committee on Financial Services
House of Representatives
The Honorable Paul E. Kanjorski
Ranking Member
Subcommittee on Capital Markets, Insurance and Government Sponsored
Enterprises Committee on Financial Services House of Representatives
Nearly half of all U.S. households own mutual funds, with total assets of
about $8 trillion as of February 2005, and roughly one-third of the total
assets are held in retirement accounts.1 Recent trading abuses implicating
well-known mutual funds have called into question the integrity of the
mutual fund industry and quality of oversight provided by the Securities
and Exchange Commission (SEC), which is the industry's frontline
regulator. State regulators rather than SEC were the first to uncover in
the summer of 2003 abuses involving market timing, which typically
involves the frequent buying and selling of mutual fund shares by
sophisticated investors seeking opportunities to profit from differences
in prices between overseas and U.S. markets. Although market timing is not
itself illegal, it can constitute illegal conduct if, for example,
investment advisers (firms that manage mutual funds) enter into
undisclosed agreements with favored customers, such as hedge funds,
permitting them to trade frequently and in contravention of fund
prospectuses-as certain advisers
1The term "mutual fund" refers to an open-end management company, which is
a type of investment company and comprises the largest segment of the
investment company industry.
did before September 2003.2 Another type of abuse commonly referred to as
late trading was significant but less widespread than market timing
abuses. Late trading occurs when investors place orders to buy or sell
mutual fund shares after the mutual fund has calculated the price of its
shares, usually once daily at the 4:00 p.m. Eastern Time (ET) market
close, but receive that day's fund share price.3 Investors who are
permitted to late trade can profit from their knowledge of events in the
financial markets that take place after 4:00 p.m., an opportunity that
other fund shareholders do not have. Although late trading can involve
fund personnel, late trading violations typically have occurred at
intermediaries, such as broker-dealers that offer mutual funds to their
customers, before these institutions submit their daily aggregate orders
to mutual funds for final settlement.
SEC's initial inability to detect the market timing abuses before late
2003 raised questions about the agency's mutual fund examination program,
which is the agency's primary means of detecting deficiencies and
violations and thereby protecting investors.4 However, following the
detection of the mutual fund trading abuses, SEC initiated a series of
examinations to determine the extent of the abuses, vigorously pursued
enforcement actions against violators of securities laws, and issued new
rules to overhaul the regulatory framework in which funds operate. The
agency also initiated changes to its examination program that are intended
to make it more focused on detecting abuses and emerging problems more
quickly.
2The term "hedge fund" generally identifies an entity that holds a pool of
securities and perhaps other assets that is not required to register its
securities offerings under the Securities Act and is excluded from the
definition of an investment company under the Investment Company Act of
1940. Hedge funds are also characterized by their fee structure, which
compensates an adviser based upon a percentage of the hedge fund's capital
gains and capital appreciation. Pursuant to a new rule recently adopted by
SEC, advisers of certain hedge funds are required to register with SEC
under the Investment Advisers Act of 1940. See Registration Under the
Advisers Act of Certain Hedge Fund Advisers, 69 Fed. Reg. 72054 (2004) (to
be codified in various sections of 17 C.F.R. Parts 275 and 279).
3Unlike market timing, late trading is illegal. Under SEC rules, mutual
funds accept orders to sell and redeem fund shares at a price based on the
current net asset value, which most funds calculate once a day at the 4:00
p.m. ET close of the U.S. securities markets.
4We discuss the reasons that SEC did not detect the market timing and late
trading abuses in a recently issued report. See GAO, Mutual Fund Trading
Abuses: Lessons Can Be Learned from SEC Not Having Detected Violations at
an Earlier Stage, GAO-05-313 (Washington, D.C.: Apr. 20, 2005).
In addition, the involvement of some broker-dealers in the recent mutual
fund trading abuses has raised concerns about regulatory oversight of that
industry. NASD and the New York Stock Exchange (NYSE) are selfregulatory
organizations (SRO) and have primary responsibility for regulating and
examining their member broker-dealers, including their mutual fund sales
practices. Although NASD is primarily responsible for assessing
broker-dealer mutual fund sales practices, it did not detect the trading
abuses through examinations or other means.5 SEC is responsible for
overseeing the quality of SRO regulation of broker-dealers and does so
through on-site inspections of SRO regulatory programs. SEC also conducts
oversight examinations of broker-dealers recently examined by SROs to
assess the quality of their examination programs.
This report responds to your request that we review various issues
concerning SEC's oversight of the mutual fund industry. Specifically, you
asked us to (1) identify and assess changes SEC has made to, or is
planning for, its mutual fund examination program; (2) assess key aspects
of the quality control framework of SEC's routine mutual fund
examinations; and (3) determine the adequacy of SEC's oversight of NASD
and NYSE, particularly in regard to the SROs' oversight of mutual fund
sales practices.
To accomplish our reporting objectives, we reviewed policies, procedures,
and other guidance applicable to SEC's mutual fund examination program as
well as laws and regulations related to mutual funds. We also reviewed
policies, procedures, and other guidance applicable to SEC's oversight of
NASD's and NYSE's broker-dealer examination programs. At three SEC field
offices that accounted for the largest number of completed routine fund
examinations in fiscal year 2004, we reviewed all routine examinations of
funds completed that year. In addition, we interviewed officials at SEC
headquarters and four field offices, NASD, NYSE, the Investment Company
Institute, and other industry participants about SEC's oversight of mutual
funds or broker-dealers. Finally, we interviewed officials from the
Federal Deposit Insurance Corporation (FDIC), the Board of Governors of
the Federal Reserve System, the Office of the Comptroller of the Currency
(OCC), NASD, NYSE, and the National Futures Association (NFA) about their
examination programs. We also
5Although NYSE is also responsible for regulating its member
broker-dealers, NASD typically conducts the sales practice portions of
examinations for firms that are dually registered with it and NYSE. As a
result, NYSE generally plays a lesser role in examining broker-dealers for
matters involving mutual fund sales.
reviewed guidance related to those programs. We performed our work in
Boston, Massachusetts; Chicago, Illinois; New York, New York;
Philadelphia, Pennsylvania; and Washington, D.C. We conducted our work
between February 2004 and July 2005 in accordance with generally accepted
government auditing standards. Appendix I provides a detailed description
of our scope and methodology.
Results in Brief Although SEC is initiating several significant changes
intended to strengthen its mutual fund examination program, it faces
challenges in effectively overseeing the mutual fund industry. In the wake
of the recent mutual fund trading abuses, SEC has revised its traditional
approach of primarily conducting routine examinations of all funds
generally within a 5year period to oversee the industry. Agency staff
concluded that these examinations were not the best tool for identifying
emerging compliance problems, because funds were selected for examination
based on the passage of time and not based on risk. To more proactively
identify and address compliance risks, SEC is shifting resources away from
routine examinations to targeted examinations that narrowly focus on
specific risks (e.g., market timing) at individual or groups of funds
based on tips or other information. SEC is continuing to conduct routine
examinations on a regular schedule but only of funds it perceives to pose
higher risk. It will randomly select a sample of lower risk advisers and
funds for routine examination each year. In addition, SEC plans to
implement a pilot program to assign examination teams to continuously
monitor some of the largest mutual funds, an approach modeled after the
use of similar teams by federal bank regulators to supervise large banks.
Although SEC is seeking to focus its resources on higher risk advisers and
funds as well as higher risk activities, the resource tradeoffs it made in
revising its oversight approach raise significant challenges.
Specifically, the tradeoffs may limit SEC's capacity to examine funds
considered lower risk within a 10-year period. In turn, this outcome could
limit SEC's capacity to accurately identify which mutual funds pose
relatively higher or lower risk and effectively target higher risk funds
for routine examination. Potentially taxing its examination resources
further, SEC recently adopted a rule to require hedge fund advisers-some
of which were involved in the recent mutual fund abuses-to register with
the agency. This rule is expected to increase SEC's examination workload,
but the precise extent is not yet known.
SEC has integrated some quality controls into its routine mutual fund
examinations, but quality control improvements could further ensure that
examinations are conducted in a thorough and consistent manner throughout
SEC field offices. Under its revised examination priorities, SEC's routine
examinations are continuing to serve as the primary regulatory tool for
testing whether higher risk mutual funds are complying with the federal
securities laws. As part of its quality controls, SEC relies extensively
on experienced supervisory examiners to oversee all stages of the routine
examination process, but three controls could be enhanced to facilitate
and ensure the adequacy of supervisory review. First, although SEC has
standards for preparing examination work papers, it does not expressly
require supervisors to review work papers or document their review.
Requiring documented supervisory review could further ensure that work is
reviewed and meets SEC standards. Deficiencies we found raise questions
about the adequacy or completeness of supervisory review of completed risk
scorecards-work papers that play a key role in determining the scope of a
fund's routine examination and the timing of the fund's next routine
examination. Second, SEC standards do not require examiners to prepare
written examination plans for supervisory review, even though examiners
use considerable judgment in customizing the examination scope to the
particular risks of a mutual fund. Written plans could serve as a guide
for conducting examinations, coordinating examination work, and reviewing
whether examinations were completed as planned. Third, while SEC uses
several methods to ensure examination quality and consistency, federal
bank and other regulators take the additional step of reviewing some
completed examinations and work papers to test compliance with and
evaluate the effectiveness of applicable policies and procedures. SEC
officials cited staff resource constraints for not reviewing completed
examinations and work papers. While reviews of a sample of completed
examinations and work papers involve resource tradeoffs, they can yield
important benefits and are an integral part of an effective quality
control system.
A primary tool that SEC uses to assess the adequacy of SRO oversight of
broker-dealers offering mutual funds to customers provides limited
information for achieving its objective and imposes duplicative regulatory
costs on the securities industry. To assess SRO oversight, SEC reviews SRO
examination programs and conducts oversight examinations of brokerdealers,
including their mutual fund sales practices. SEC's oversight examinations
take place 6 to 12 months after SROs conduct their examinations of
broker-dealers and are intended to assess the quality of SRO examinations.
However, we reported in 1991 that SEC's broker-dealer
examination program had significant problems.6 For example, we reported
that the way in which SEC conducted these broker-dealer oversight
examinations provided limited information in helping SROs to improve the
quality of their examination programs. This was because SEC and the SROs
used different broker-dealer examination guidelines, and their
examinations often covered different periods of time. Our recent work
found that these problems still remain, which raises questions about SEC's
goal of conducting about 40 percent of its broker-dealer examinations as
oversight examinations. Another deficiency we found during our review is
that SEC has not developed an automated system to track the full scope of
work performed during broker-dealer examinations. Absent such an automated
system, SEC managers cannot readily determine the extent to which the
agency's broker-dealer examinations assess mutual fund sales practices or
other issues. Given the resource challenges that SEC faces in its role as
the frontline regulator of mutual funds, SEC's current commitment of staff
to broker-dealer oversight examinations may need to be reexamined.
This report makes four recommendations to the SEC Chairman for improving
oversight of mutual funds and SRO oversight of broker-dealers that sell
mutual funds. First, we recommend that SEC periodically assess the level
of resources allocated to the various types of examinations in light of
their regulatory benefits to help ensure that the agency is using its
resources efficiently and effectively to oversee the mutual fund industry,
including broker-dealers that offer mutual funds. As part of this
assessment, SEC should seek to ensure that it allocates sufficient
resources to mitigate any regulatory gaps that may currently exist
concerning the timely examination of mutual funds perceived to represent
lower risk; complete fund risk assessments within a more reasonable
period; and fulfill its new oversight responsibilities of the hedge fund
industry. Second, in so doing, we recommend that SEC assess its
methodology for conducting broker-dealer oversight examinations and
whether some portion of the resources currently devoted to these
examinations could be better utilized to perform mutual fund examinations.
Third, to strengthen SEC's approach to mutual fund examinations, we
recommend that SEC establish additional policies or procedures for
improving its controls to ensure examination quality and consistency
throughout SEC field offices. Fourth, we recommend that SEC
6GAO, Securities Industry: Strengthening Sales Practice Oversight,
GAO/GGD-91-52 (Washington, D.C.: Apr. 25, 1991).
electronically track information about the full scope of work performed
during broker-dealer oversight examinations.
We received comments on a draft of this report from SEC, which are
included in appendix III. SEC also provided technical comments on a draft
of the report, which were incorporated into the final report, as
appropriate. SEC focused most of its comments on providing further
elaboration on the potential benefits of its examination strategy for
overseeing mutual funds and investment advisers and on the benefits
obtained from its brokerdealer oversight examinations. In addition, SEC
briefly commented that it will consider our recommendation directed at
improving its quality controls for routine fund examinations and that it
has formed a working group to explore ways to enhance the value of its
broker-dealer oversight examinations.
Background SEC oversees mutual funds primarily through its Office of
Compliance Inspections and Examinations (OCIE), Division of Investment
Management, and Division of Enforcement. OCIE examines mutual funds to
evaluate their compliance with the federal securities laws, to determine
if they are operating in accordance with disclosures made to investors,
and to assess the effectiveness of their compliance control systems. The
Division of Investment Management administers the securities laws
affecting funds and advisers. It reviews disclosure documents that mutual
funds are required to file with SEC and engages in other regulatory
activities, such as rulemaking, responding to requests for exemptions from
federal securities laws, and providing interpretation of those laws.
Finally, SEC's Division of Enforcement investigates and prosecutes
violations of securities laws related to mutual funds.
SEC regulates mutual funds under the Investment Company Act of 1940, the
Investment Advisers Act of 1940, the Securities Act of 1933, and the
Securities Exchange Act of 1934. The Investment Company Act was passed
specifically to regulate mutual funds and other types of investment
companies. Under the act, mutual funds are required to register with SEC,
subjecting their activities to SEC regulation. The act also imposes
requirements on the operation and structure of mutual funds. Its core
objectives are to
o ensure that investors receive adequate and accurate information about
mutual funds,
o protect the integrity of fund assets,
o prohibit abusive forms of self-dealing,
o prevent the issuance of securities that have inequitable or
discriminatory provisions, and
o ensure the fair valuation of investor purchases and redemptions.
The Investment Advisers Act requires mutual fund advisers to register with
SEC, imposes reporting requirements on them, and prohibits them from
engaging in fraudulent, deceptive, or manipulative practices.
TheSecurities Act requires fund shares offered tothe public to be
registered with SEC and regulates mutual fund advertising. Under the
Securities Act and Investment Company Act, SEC has adopted rules to
require mutual funds to make extensive disclosures in their prospectuses.
The Securities Exchange Act, among other things, regulates how funds are
sold and requires persons distributing funds or executing fund
transactions to be registered with SEC as broker-dealers.
SEC, NASD, and NYSE regulate broker-dealers, including their mutual fund
sales practices, by examining their operations and reviewing customer
complaints. Broker-dealers that are members of NYSE and do business with
the public are typically also required to be members of NASD.
Historically, NASD has conducted the mutual fund sales practice portions
of examinations for firms that are dually registered with it and NYSE. As
a result, NYSE generally plays a lesser role in examining broker-dealers
for mutual fund sales practices. NASD has established specific rules of
conduct for its members that provide, among other things, standards for
advertising and sales literature, including filing requirements, review
procedures, approval and recordkeeping obligations, and general standards.
NASD also tests members to certify their qualifications as registered
representatives.
SEC evaluates the quality of NASD and NYSE oversight in enforcing their
member compliance with federal securities laws through SRO oversight
inspections and broker-dealer oversight examinations. SROs are private
organizations with statutory responsibility to regulate their own members
through the adoption and enforcement of rules of conduct for fair,
ethical, and efficient practices. As part of this responsibility, they
conduct examinations of the sales practices of their broker-dealer
members. SEC's SRO oversight inspections cover all aspects of an SRO's
compliance,
examination, and enforcement programs. The inspections determine whether
an SRO is (1) adequately assessing risks and targeting its examinations to
address those risks, (2) following its examination procedures and
documenting its work, and (3) referring cases to enforcement authorities
when appropriate. Under its broker-dealer oversight examinations, SEC
examines some of the broker-dealers that SROs recently examined. SEC
conducts these examinations to assess the adequacy of the SRO examination
programs. In addition to its oversight examinations, SEC conducts cause,
special, and surveillance examinations of broker-dealers, but these
examinations do not serve to assess the quality of SRO examinations.
SEC's Revised Mutual Fund Examination Program Offers Potential Benefits but
also Poses Significant Oversight Challenges
Since the detection of the mutual fund trading abuses in the summer of
2003, SEC has made significant changes to its traditional examination
approach, which generally focused on conducting routine examinations of
all funds on an established schedule. To better detect potential
violations, SEC has reallocated or plans to reallocate its staff to
conducting targeted examinations focusing on specific risks and monitoring
larger funds on a continuous basis. SEC's revised examination approach
offers the potential for the agency to more quickly identify emerging
risks and better understand the operations of large and complex funds,
although it is too soon to reach definitive judgments. However, due to the
limited number of SEC's examination staff relative to the number of mutual
funds and advisers for which the agency has oversight responsibility, the
decision to focus examination resources on particular areas involved
tradeoffs that raise regulatory challenges. In particular, SEC's capacity
to examine lower risk advisers and funds within a reasonable time period
and develop industry risk ratings has been limited.
SEC Has Revised Its Historically, routine examinations of mutual fund
complexes-groups or Traditional Mutual Fund families of funds sharing the
same adviser or underwriter-have served as Examination Approach in the
cornerstone of SEC's mutual fund oversight, accounting for 85 percent
of the total fund examinations done from 1998 through 2003. During thatthe
Wake of the Mutual period, SEC generally tried to examine each complex at
least once every 5Fund Trading Abuses
years.7 Due to resource constraints, SEC examinations typically focused on
discrete areas that staff viewed as representing the highest risks of
presenting compliance problems that could harm investors. Major areas of
review have included portfolio management, order execution, allocation of
trades, and advertising returns. In late 2002, SEC implemented a revised
approach to conducting routine examinations that included a systematic
process for documenting and assessing risks and controls for managing
those risks in a range of areas related to the asset management function.8
Besides routine examinations, SEC conducts sweep examinations to probe
specific activities of a sample of funds identified through tips,
complaints, the media, or other information. The agency also conducts
cause examinations when it has reason to believe something is wrong at a
particular fund. Sweep and cause examinations accounted for about 5 and 10
percent, respectively, of the total examinations done during 1998 through
2003.
After the detection of the market timing and late trading abuses in the
summer of 2003, SEC officials concluded that the agency's traditional
focus on routine examinations had limitations. In particular, SEC staff
said that routine examinations were not the best tool for broadly
identifying emerging compliance problems, since funds were selected for
examination based largely on the passage of time, not based on their
particular risk characteristics.9 In addition, SEC officials stated that
they concluded the growth in the number of mutual fund companies and the
breadth of their operations, combined with the need to perform more
in-depth examinations of discrete areas, did not allow SEC to maintain its
existing routine examination cycle.
7In late 2003, SEC established a 2 or 4-year examination cycle based on
the size or risk level of the fund complex. However, this cycle was not
fully implemented before SEC made significant changes to its mutual fund
examination program as described in this section.
8Under this process, examiners use a set of standardized work papers
called control or risk scorecards to guide and document their assessment
of the effectiveness of a fund's compliance controls designed to prevent
or detect violations of the federal securities laws. Based on that
assessment, examiners assign the fund an overall compliance risk rating of
low, medium, or high.
9In an earlier report, we found that SEC's focus on areas traditionally
considered to be high risk hindered its capacity to detect violations not
traditionally considered to be high risk, such as market timing abuses. We
concluded that SEC needed to test controls in a variety of areas at least
at a sample of companies to validate its assumptions about risks and
verify the adequacy of controls in place to mitigate them.
To focus its resources on issues and funds presenting the greatest risk of
having compliance problems that may harm investors, SEC has made
significant revisions to its examination priorities and oversight
processes as described below:
o First, SEC is placing a higher priority on sweep and cause examinations
and a lower priority on routine examinations. SEC has directed its 10
field offices that conduct fund examinations to give priority to
initiating, as warranted, sweep examinations of funds or advisers,
focusing particularly on operational or compliance issues.10 To address
the market timing and late trading abuses surfacing in late 2003, SEC
shifted resources away from routine examinations to support sweep and
cause examinations, according to SEC officials. As a result, sweep and
cause examinations accounted for 87 percent of the 690 fund examinations
completed in fiscal year 2004. SEC officials said that about 17 percent of
these examinations resulted in referrals to the agency's Division of
Enforcement for potential violations of securities laws and regulations.
(We note that the large increase in the number of sweep and cause
examinations in fiscal year 2004 as well as the number of referrals was
likely due to SEC's focusing a substantial amount of resources on
detecting market timing and late trading abuses.)
o Second, SEC no longer will routinely examine all funds and advisers on
a regular basis, but it will conduct routine examinations of funds and
advisers perceived to be high risk, once every 2 to 3 years. In addition,
SEC will randomly select a sample of advisers and their affiliated funds
perceived to be low risk for routine examination each year. Because these
firms will be selected randomly, each firm will have an equal chance of
being examined each year. According to SEC officials, the random selection
process will enable agency staff to project the examination findings to
the population of firms deemed low risk and assess the possible existence
of problems within the population.
o Third, SEC plans to provide more continuous and in-depth oversight of
the largest mutual funds. Specifically, SEC is creating teams of examiners
dedicated to regularly interacting with and closely monitoring and
examining the activities of firms in the largest and most complex groups
of affiliated advisers and mutual funds. SEC initially plans to form teams
under a pilot program to monitor 10 large advisory
10SEC has 11 field offices, but 1 office does not have fund examination
staff.
groups. Any decision to form additional monitoring teams will depend on
how the pilot program develops, according to an SEC official. SEC
officials said that the monitoring teams are loosely modeled on the
federal bank regulators' use of on-site teams to continuously monitor
operations of large banks. However,unlike the bank regulator approach, SEC
staff said the monitoring teams would not be located on-site at large
mutual fund companies.
o Fourth, an SEC task force is considering the development of a
surveillance program to support the agency's oversight of all funds and
advisers. The purpose of this program is to obtain from firms information
that would enable examiners to identify aberrant patterns in fund and
adviser activities and the possible existence of fraud or abusive schemes
that require follow-up through examinations. In its fiscal 2006 budget
request, SEC reported that the agency expects the surveillance system to
begin operations during the second half of 2006.11
o Fifth, SEC has promulgated rules that require investment advisers and
investment companies to appoint independent chief compliance officers
(CCO) who are responsible for ensuring that their companies adopt policies
and procedures designed to prevent violations of federal securities laws
and regulations.12 Fund CCOs are also responsible for preparing annual
reports that must, among other things, identify any material compliance
matter at the company since the date of the last report. SEC staff said
that they plan to review such annual compliance reports while conducting
examinations to assist in identifying problems at mutual funds and
determine whether the funds have taken corrective actions. (As described
later in this report, SEC is missing opportunities to take full advantage
of CCO compliance reports to detect potential violations in the mutual
fund industry.)
o Finally, SEC has established the Office of Risk Assessment (ORA) to
assist the agency in carrying out its overall oversight responsibilities,
including mutual fund oversight. ORA's director reports directly to the
SEC Chairman. According to SEC staff, ORA will enable the agency to
analyze risk across divisional boundaries, focusing on new or resurgent
11SEC, In Brief: Fiscal 2006 Congressional Budget Request (Feb. 2005).
12SEC, Final Rule: Compliance Programs of Investment Companies and
Investment Advisers, 68 Fed. Reg. 74714 (Dec. 24, 2003).
forms of fraudulent, illegal, or questionable behavior or products. ORA's
duties include (1) gathering and maintaining data on new trends and risks
from external experts, domestic and foreign agencies, surveys, focus
groups, and other market data; (2) analyzing data to identify and assess
new areas of concern across professions, industries, and markets; and (3)
preparing assessments of the agency's risk environment. ORA is to work in
coordination with internal risk teams established in each of the agency's
major program areas-including OCIE-and a Risk Management Committee
responsible for reviewing implications of identified risks and
recommending appropriate courses of action.
SEC's Revised Oversight Approach Reflects Some of the Lessons Learned from
the Recent Mutual Fund Scandals
As we recently reported, the market timing and late trading abuses that
surfaced in 2003 revealed weaknesses in SEC's mutual fund oversight
approach.13 We noted in the report that lessons can be learned from SEC
not having detected market timing arrangements at an earlier stage. The
key initiatives that SEC is taking to strengthen its mutual fund oversight
program are largely intended to focus the agency's resources on the
largest and highest risk funds and activities. Although it is too soon to
assess the effectiveness of the initiatives in light of their recent or
planned implementation, the initiatives are consistent with some of the
lessons learned concerning the importance of (1) conducting independent
assessments of the adequacy of controls over areas such as market timing,
(2) developing the institutional capability to identify and analyze
evidence of potential risks, and (3) ensuring the independence and
effectiveness of company compliance staff and potentially using their work
to benefit the agency's oversight program.
By placing greater priority on sweep examinations, SEC may be better
positioned to independently assess, as needed, the adequacy of fund
controls designed to prevent and detect abusive practices. As we reported,
SEC staff did not examine mutual funds for market timing abuses before
late 2003, because they viewed market timing as a relatively lower risk
area since agency staff believed that funds had adequate financial
incentives to establish effective controls for it. In that regard, we
noted the importance for SEC to conduct independent assessments of
controls at a sample of funds, at a minimum, to verify that areas viewed
as low risk, such as market
13GAO-05-313.
timing, are in fact low risk and effective controls are in place. SEC's
revised examination priorities, particularly their emphasis on initiating
sweep examinations that focus on operational or compliance issues, may
provide the agency with greater opportunity to conduct independent
assessments of controls for emerging risks, in part to validate critical
assumptions about such risks and confirm the adequacy of controls in place
to address those risks.
By forming examiner teams dedicated to monitoring the largest and most
complex groups of affiliated advisers and funds, SEC may have the
opportunity to more efficiently or effectively use its resources and help
ensure the independence and effectiveness of the monitored firms'
compliance staff. SEC estimates that the 100 largest advisory groups of
affiliate advisers and funds accounted for about $7.1 trillion, or 85
percent, of the fund assets under management as of the end of September
2004. Thus, focusing on the largest advisory groups may enable SEC to
attain the greatest dollar coverage with its limited examination
resources. Focusing on the largest advisory groups may also be appropriate
due to the control deficiencies that have been found at such companies.
For example, SEC determined that nearly 50 percent of the 80 largest
mutual funds had entered into undisclosed arrangements permitting certain
shareholders to engage in market timing that appeared to be inconsistent
with the funds' policies, prospectus disclosures, or fiduciary
obligations. In our earlier mutual fund work, we also found that
compliance staff at some funds identified market timing but lacked the
independence or authority necessary to control it. This finding suggested
that routine communications with fund compliance staff could enhance SEC's
capacity to detect potential violations at an earlier stage, if compliance
staff are effective and forthcoming about the problems they detect. SEC's
monitoring teams will provide agency staff with the opportunity to be in
routine communication with fund compliance staff, including CCOs.
Furthermore, such communications, combined with examinations, could help
SEC ensure that fund CCOs, as required under SEC's compliance rules, are
in a position of authority to compel others to adhere to applicable
compliance policies and procedures.
By creating ORA, SEC is laying an important part of the foundation for
developing the institutional capability to identify and analyze evidence
of potential risks. SEC staff said that ORA will seek to ensure that SEC
will have the information necessary to make better, more informed
decisions on regulation. Working with other SEC offices, ORA staff expect
to identify new technologies, such as data mining systems, that can help
agency staff
detect and track risks. SEC's compliance rules create opportunities for
ORA to leverage the knowledge of fund CCOs, including their annual
compliance reports. Although ORA may help SEC be more proactive and better
identify emerging risks, it is too soon to assess its effectiveness. In
this regard, we note that as of February 2005, ORA had established an
executive team of 5 individuals but still planned to hire an additional 10
staff to assist in carrying out its responsibilities.
Finally, SEC's fund and adviser surveillance system is in the exploratory
stage but, if properly designed and implemented, may help the agency to
leverage its limited resources to augment its examinations and oversee
funds and advisers. Federal bank and other regulators use off-site
surveillance programs to complement their on-site examinations. Each
federal bank regulator has an off-site surveillance program to monitor the
financial condition of banks between examinations. Information from
offsite monitoring is used in setting bank examination schedules and
determining the allocation of examiner resources for higher risk banks.
Similarly, a recently deployed NASD surveillance program is used to
analyze trends in broker-dealer activities and identify unusual patterns
that indicate potential problems.14 NASD uses surveillance analyses to
initiate cause examinations and to help its examiners focus on high-risk
areas during their routine broker-dealers examinations.
SEC's Revised Examination Approach Raises Oversight Challenges
SEC's planned changes to its mutual fund examination program offer
potential advantages, but they also involve significant tradeoffs that
raise important regulatory challenges for the agency. In comparison to
federal bank regulators, SEC has significantly less examiners relative to
the number of entities it regulates (see fig. 1), although bank and mutual
fund regulatory regimes, including their examinations, differ from each
other.15 As reflected in SEC's revised oversight approach, any decision by
SEC to focus additional examination resources on one or more fund areas
involves tradeoffs that could result in less oversight of, or create a
regulatory gap in, other areas. We are particularly concerned about SEC's
capacity going
14NASD's surveillance program is called Integrated National Surveillance
and Information Technology Enhancements.
15SEC recently reported that mutual funds and other investment companies
managed roughly $8 trillion in assets at the start of fiscal year 2005,
nearly double the $4.5 trillion in insured deposits at commercial banks
and about equal to the $8 trillion of financial assets at commercial
banks.
forward to review the operations of firms considered to be lower risk,
conduct risk assessments of the industry, and potentially oversee the
hedge fund industry.
Figure 1: Number of Examiners and Entities Subject to Examination by SEC and
Federal Bank Regulators in 2004
Sources: SEC, the Federal Reserve, FDIC, and OCC.
By shifting examination resources to targeted sweep and cause examinations
as well as monitoring teams for larger funds, SEC may be limiting its
capacity to examine the operations of funds perceived to pose lower risk
(generally smaller funds) within a reasonable period. As stated
previously, between 1998 and 2003, SEC generally sought to conduct routine
examinations of all funds once every 5 years and shortened the cycle to 2
or 4 years in fiscal year 2004 following an increase in resources.16
However, under SEC's revised examination program, some mutual funds may
not be examined within a 10-year period. This is because SEC plans to
annually review the operations of 10 percent of the funds deemed lower
risk on a random basis. While reviewing funds on a random basis means each
firm will have an equal chance of being reviewed annually, it is not clear
that this approach will have more of an effect in deterring abuses than if
each fund wasassured ofbeing examined every 5 years or less. Moreover, if
SEC lacks sufficient resources to annually examine 10 percent of the funds
deemed lower risk, its approach would have less of a deterrent effect. We
recognize that through sweep examinations, SEC may review particular
facets of funds deemed lower risk much more frequently than
16In fiscal year 2004, SEC sought to conduct routine examinations of (1)
the 20 largest funds as well as funds and advisers posing high risk every
2 years and (2) all other funds, including their advisers, every 4 years.
every 10 years or more. At the same time, sweep examinations are much more
narrowly scoped than routine examinations and may exclude other potential
areas of noncompliance at individual firms.
Similarly, SEC's inability to conduct examinations of all mutual funds
within a reasonable period may limit its capacity to accurately
distinguish relatively higher risk funds from lower risk funds and
effectively conduct routine examinations of higher risk funds. Between
late 2002 and October 2004, SEC routinely examined 297, or 30 percent, of
the existing fund complexes and used its revised examination guidelines to
assess the effectiveness of the funds' compliance controls in deterring
and preventing abuses and to assign the funds risk ratings of low, medium,
or high. Had SEC not decided in late 2003 and 2004 to shift examination
resources to sweep and cause examinations, it might have been able to
assign risk ratings to all 982 fund complexes within the following 3 years
in accordance with its routine examination cycle. Completing risk ratings
for all fund complexes would have provided SEC with an additional basis
for allocating resources to the highest risk firms.17 Over time, SEC's
risk ratings can become outdated, or stale, raising the possibility for
funds deemed lower risk to become higher risk. For example, changes in a
fund's management, such as CCO, could lead to changes that weaken the
fund's compliance culture and controls. However, because SEC may not
examine all fund complexes within a 10-year period under its revised
examination program, its ability to assign risk ratings to all fund
complexes and routinely examine all higher risk funds may be limited.18
In a previous report, we found that SEC may be missing opportunities to
obtain useful information about the compliance controls of mutual funds,
including those perceived to represent lower risks and may not be
17Absent a compliance risk rating for a fund complex, SEC officials stated
that an alternative risk rating assigned to the fund's adviser will be
used to determine when the fund will be routinely examined. The
alternative rating captures risks inherent in the adviser's business such
as conflicts of interests but does not measure the effectiveness of the
adviser's compliance controls designed to mitigate conflicts of interest
or other risks that could harm mutual fund shareholders.
18We note that, as described later in this report, our work has identified
deficiencies in SEC's implementation of its revised mutual fund
examination guidelines, which raise questions about the quality of risk
assessments made between 2002 and 2004. Further, risk ratings completed in
2002 and much of 2003 do not reflect the quality of fund controls over
market timing and late trading as SEC did not view these as high-risk
areas. SEC subsequently implemented revised procedures to test these areas
at each mutual fund that it examined.
examined within a reasonable period of time.19 While SEC plans to review
investment company CCO annual compliance reports during examinations, the
agency has not developed a plan to receive and review the reports on an
ongoing basis. Obtaining access to such annual reports and reviewing them
on an annual basis could provide SEC examiners with insights into the
operations of all mutual funds, including those perceived to represent
lower risks, and could serve as a basis for initiating examinations to
correct potential deficiencies or violations. SEC noted that it is
considering how best to utilize the annual reports but noted any required
filing of the reports with SEC would require rulemaking by SEC.
A final oversight challenge facing SEC's mutual fund examination program
involves a new rule requiring hedge fund advisers to register with the
agency.20 Issued in December 2004, the new rule requires hedge fund
advisers to register with SEC as investment advisers by February 2006. The
rule is designed, in part, to enhance SEC's ability to deter or detect
fraud by unregistered hedge fund advisers, some of which were involved in
the recent mutual fund abuses. Once hedge fund advisers register, SEC will
have the authority to examine their activities. The rule is expected to
increase SEC's examination workload, but because of data limitations the
precise extent will notbe known until hedge fund advisers actually
register. Currently, comprehensive information on the number of hedge
funds and advisers is not available, but SEC estimates that from 690 to
1,260 additional hedge fund advisers may be required to register under the
new rule, increasing the pool of registered advisers by 8 to 15 percent.21
SEC officials estimate that at least 1,000 hedge fund advisers have
previously registered as investment advisers with SEC to meet client needs
or requirements. Under its examination program, SEC has examined these
hedge fund advisers in the same way it has examined all other registered
advisers. According to SEC officials, it is anticipated that the
additional hedge fund advisers that register with SEC will be treated the
same as all other registered advisers under SEC's examination program. SEC
has
19GAO-05-313.
20SEC, Final Rule: Registration Under the Advisers Act of Certain Hedge
Fund Advisers, 69 Fed. Reg. 72054 (Dec. 18, 2004).
21For additional detail on how SEC arrived at its estimates, see Proposed
Rule: Registration Under the Advisers Act of Certain Hedge Fund Advisers,
Release No. IA-2266, File No. S730-04 (July 20, 2004).
recognized that providing oversight of the additional registered hedge
fund advisers will pose a resource challenge and has identified options
for addressing the challenge. It could require fewer hedge fund advisers
to register with SEC by raising the threshold level of assets under
management required for adviser registration. It also has the option of
seeking additional resources from Congress for the increased workload
resulting from an increased number of registered advisers. Whatever
approach is ultimately taken, SEC will have to consider the potential
resource implications of the new rule for its oversight of mutual funds.
SEC Can Improve Certain Mutual Fund Examination Quality Control Measures
SEC has integrated quality controls into its routine examinations but
could benefit from additional controls to ensure that policies and
procedures are being implemented effectively and consistently throughout
SEC field offices. Under its new initiatives, SEC's routine examinations
will continue to be the primary regulatory tool for determining whether
all funds and advisers are complying with the federal securities laws.
Examination quality controls provide, among other things, assurances that
important documents are provided supervisory review, and examinations are
conducted according to agency policies, procedures, and individual
examination plans.22 SEC could improve its quality control measures in
three areas: supervisory review of risk scorecards, preparation of written
examination plans, and review of completed examinations and work papers.
Bank and other financial regulators have quality control measures that
provide assurances above and beyond those measures used by SEC.
SEC Standards for Reviewing Mutual Fund Risk Scorecards Do Not Ensure
Accuracy or Completeness
The risk scorecards prepared by SEC during each mutual fund examination
are critical work papers, providing the basis for determining areas to
review in depth and an overall risk rating for a fund. A set of individual
scorecards has been developed to assist examiners in assessing and
documenting a fund's compliance controls in 13 strategic areas and to
determine the amount of additional testing examiners will do.23 (See table
1.)
22GAO, An Audit Quality Control System: Essential Elements, GAO/OP-4.1.6
(Washington, D.C.: August 1993).
23When first implemented in October 2002, the risk scorecards covered 10
areas related to the asset management function. In July 2003, OCIE
developed three additional risk scorecards.
Table 1: Strategic Areas Covered by SEC's Risk Scorecards as of April 2005
Name of scorecard area
Firm Maintains a Strong Compliance Culture
Minimize Ability of Dominant Individual to Override Control System
Consistency of Portfolio Management with Clients' Mandates
Order Placement Practices Consistent with Seeking Best Execution and Disclosures
Personal Trading of Access Persons Is Consistent with Code of Ethics
Fair Allocation of Blocked and Initial Public Offering Trades
Fund/Advisory Clients' Assets Are Priced and Fund Net Asset Values Are
Calculated Accordingly
Accuracy and Fairness of Performance Information
Information That Is Created, Recorded, Maintained, and Reported Is
Protected from Unauthorized Alterations
Safety of Clients' Funds and Assets
Third Party Sends Periodic Account Statement to Clients
Fund/Shareholder Order Processing and Cash-Book Reconciliations Fund Corporate
Governance
Source: SEC.
If controls in an area are strong, examiners may do limited or no
additional testing to detect potential abuses, but if weak, additional
testing is expected to be performed. Collectively, the 13 areas reviewed
with the set of individual scorecards provides the basis for determining a
mutual fund's overall risk rating, which OCIE uses to determine how
frequently the fund will be examined. While the risk scorecards currently
cover 13 areas, SEC officials stated that each scorecard serves, in
concept, as a model for assessing controls in a particular area of a
firm's activities. As such, SEC staff could create additional scorecards
to assist them in their review of areas not covered by existing scorecards
or modify existing scorecards not suitable for reviewing the controls used
by a firm in a critical area. OCIE and field office officials told us that
all applicable risk scorecards generally should be completed during
routine examinations, but if there are time constraints due to extenuating
factors, all scorecards may not be completed.
Even though risk scorecards are important work papers for documenting and
assessing fund compliance controls, SEC standards do not expressly require
that they receive supervisory review. Current OCIE standards for preparing
examination work papers, including scorecards, specify thatthey should be
prepared in an organized manner facilitating supervisory review
and examination reporting. The standards do not provide further
supervisory review requirements such as who should do the review, how, or
when. While the review of scorecards is not expressly required, OCIE
headquarters and SEC field office officials stated that supervisors do
review scorecards and other examination work papers but typically do not
sign or initial them to document that they have been reviewed. In
addition, we were told that lead examiners and branch chiefs review work
papers throughout the examination process. These officials also review
risk scorecards and other work papers when reviewing final examination
reports, making sure that all findings are adequately supported and
summaries of the scorecard findings included in the examination reports
are accurate. After completing their review of examination reports, branch
chiefs sign a form to document their review.
In contrast to OCIE, federal bank and other regulators have standards
requiring supervisors to document that they have reviewed examination work
papers. Examples of the work paper standards include:
o Federal Reserve guidance requires examiners-in-charge or other
experienced examiners to review all work papers as soon as practicable and
to sign or initial the applicable documents to evidence their review.
o OCC guidance requires examiners-in-charge or other experienced
examiners to sign or initial work paper cover sheets to evidence their
review. The guidance allows reviewers to tailor the thoroughness of their
review based on the experience of the examiner preparing the work paper.
o According to NYSE and NFA officials, the organizations require senior
staff to review and sign work papers. NFA officials said that their work
papers are electronic, so staff mark a checkbox to evidence their work
paper review.
While SEC officials stated that the review of the scorecards is documented
indirectly by the supervisor's signature on the examination report,
without the supervisor's signature or initials on the scorecards
themselves, there is no way to readily verify that the scorecards were
reviewed. Our review of 546 scorecards from 66 routine examinations of
funds completed in fiscal year 2004 by SEC's Midwest Regional Office
(MRO), Northeast Regional Office (NERO), and Philadelphia District Office
(PDO) disclosed a number of deficiencies potentially stemming from quality
control weaknesses. Most of the scorecards did not contain evidence of
supervisory review as
expected, based on statements by SEC officials, but 34 scorecards, or
about 6 percent, were signed or initialed as evidence of review.
Regardless of whether the completed scorecards were signed or initialed,
we found deficiencies in four areas that raise questions about the
adequacy or completeness of supervisory review.
o First, each scorecard should be marked as to whether examiners rated
the compliance controls in the area as highly effective, effective, or
ineffective. We found 32, or about 6 percent, of the total scorecards
where the control rating was not marked.
o Second, copies of scorecards should be included with the work papers to
facilitate supervisory review, but we found 11, or about 17 percent, of
the 66 examinations lacked any scorecards and 15, or about 23 percent,
were missing one or more scorecards.24
o Third, documentary evidence should be cited on scorecards to support
effective and highly effective ratings, but we found 25, or about 5
percent, of the total scorecards did not cite documentary evidence
supporting such ratings.
o Fourth, scorecard ratings are included in examination reports, but we
found the ratings marked on 21, or about 4 percent, of the total
scorecards had ratings that differed from the ones in the examination
reports.
SEC supervisors document their review of examination reports, which
include a summary of the risk scorecard findings. Nonetheless, without
documenting that the scorecards themselves were reviewed, SEC does not
know if deficiencies resulted from a lack of or inadequate supervisory
review. The systematic supervisory review of work papers, particularly
risk scorecards, is essential for ensuring examination quality. Such
reviews help to ensure that the work is adequate and complete to support
the assessment of fund compliance controls as well as report findings and
conclusions. Likewise, documentation of the review is important to ensure
24Of the 11 examinations lacking any risk scorecards, 8 of them covered
funds organized as unit investment trusts. MRO and NERO officials told us
that the risk scorecards were not designed for unit investment trusts and,
thus, staff did not always complete the scorecards for such types of
funds. In one of the other examinations, staff noted on the scorecards
that they found extensive violations at the fund and did not have time to
complete the scorecards.
that all critical areas are reviewed. The reviewer's initials or signature
are written verification that a specific employee checked the work.
Written Examination Plans for Fund Examinations Are Not Required but Would
Be Useful for Documenting Agreements Reached on Review Areas
Written examination plans that document the scope and objectives of
routine examinations are not required by OCIE. Instead, OCIE officials
stated that written examination plans are optional. OCIE allows branch
chiefs and lead examiners to decide whether to prepare written plans, with
branch chiefs typically meeting with examination teams to discuss the
preliminary scope of examinations. Each routine examination is somewhat
different because of the risk-based approach used by OCIE. Under this
approach, all areas of compliance or fund business activities are not
reviewed and instead review areas are judgmentally selected based on their
degree of risk to shareholders. As a result, each examination is
customized to the activities of the particular fund under examination,
with the success of routine examinations depending, in part, on proper
planning. The documentation of this planning is important for tracking
agreements reached on examination scope and objectives and can be used as
a guide for the examination team. Furthermore, the plan can be used to
determine whether the examination was completed in accordance with the
planned scope. According to OCIE officials, written plans may be helpful
in planning examinations of large fund complexes, but many of the
examinations conducted are of small firms that have five or fewer
employees. For these small firms, the officials said that it may not be
necessary to prepare a written examination plan, especially if the
examination team conducting the work consists of one or two persons.
While OCIE does not require the preparation of written examination plans,
we found that SEC's NERO requires examiners to prepare a planning
memorandum to document examination scope and objectives, including firms
to be examined within a fund complex, areas considered high risk, and
areas to be reviewed. NERO branch chiefs approve the memorandums before
the on-site work begins, and the memorandums effectively serve as
examination plans. In contrast, SEC's MRO and PDO do not require planning
memorandums or examination plans. Instead, branch chiefs in these two
offices meet with the examination teams to discuss the scope of
examinations and then let the staff decide whether to prepare a written
plan, according to MRO and PDO officials. MRO officials said that some
branch chiefs will recommend that for large funds, teams prepare written
examination plans since it helps coordinate the work. For 66 routine
examinations we reviewed at these three offices, about half, or 53
percent, had written planning memorandums or examination plans.
Examinations
of the larger fund complexes that were managing more than $1 billion in
assets also had examination plans for about half, or 54 percent.
In contrast to OCIE, federal bank and other regulators require their staff
to prepare written examination plans before conducting examinations.
Examples of examination plan requirements include:
o FDIC guidance requires the examiner-in-charge to prepare a scope
memorandum to document, among other things, the preliminary examination
scope; areas to be reviewed, including the reasons why; and areas not to
be included in the examination scope, including the reasons why.
o Federal Reserve guidance requires that a comprehensive risk-focused
supervisory plan be prepared annually for each banking organization. The
guidance also requires the examiner-in-charge, before going on-site, to
prepare a scope memorandum to document, among other things, the objectives
of the examination and activities and risks to be evaluated; level of
reliance on internal risk management systems and internal and external
audit findings; and the procedures that are to be performed. To ensure
consistency, the guidance requires the scope memorandum to be reviewed and
approved by Reserve Bank management.
o OCC guidance requires the examiner-in-charge or portfolio manager to
develop and document a supervisory strategy for the bank that integrates
all examination areas and is tailored to the bank's complexity and risk
profile. The strategy includes an estimate of resources that will be
needed to effectively supervise the bank and outlines the specific
strategy and examination activities that are planned for that supervisory
cycle. The strategies are reviewed and approved by the examiner-incharge's
or portfolio manager's supervisor.
o NYSE and NFA officials told us that staff are required to prepare
written examination or audit plans. NYSE officials said that staff meet
with examination directors to reach agreement on the scope of their
examination plans. NFA officials said that staff complete a planning
module that includes a series of questions that staff answer to determine
the scope of the audit, and the completed planning module serves as the
audit plan.
Examination planning meetings between SEC branch chiefs and examination
teams are important for providing the opportunity to discuss
and reach decisions about critical areas of examination scope and
objectives. These discussions by themselves, however, do not provide a
record of the agreements reached and may not result in a clear and
complete understanding for examiners about the scope and objectives of a
particular examination. A written examination plan would provide such a
record-potentially enabling branch chiefs to better supervise examinations
and assisting lead examiners to better communicate the examination
strategy to the examination team. Such quality control is especially
important given that staff must exercise considerable judgment for
examination scope under SEC's risk-based approach.
SEC Efforts to Ensure Quality Do Not Include Review of Work Papers of
Completed Mutual Fund Examinations
SEC uses several methods to ensure the quality of its examinations but
does not review completed examinations and work papers as done by other
regulators to determine whether the examinations were conducted according
to procedures or done consistently across field offices. OCIE has issued
various policies and procedures to promote examination quality and
consistency across the 10 SEC field offices that conduct the majority of
its examinations. To help ensure that these policies and procedures are
followed, SEC relies on experienced supervisors in its field offices to
oversee all stages of routine examinations. Specifically, branch chiefs
meet with examination staff to discuss the preliminary scope of
examinations, advise staff during the fieldwork, and review all
examination reports. Assistant directors in SEC field offices also assist
in overseeing examinations and review all examination reports. Also,
associate directors and regional or district administrators in SEC field
offices may review examination reports. In addition, SEC field offices
send each report and deficiency letter, if any, to an OCIE liaison, who
reviews them. Finally, OCIE annually evaluates each field office
examination program based on factors such as the overall quality of the
office's examination selection and findings; new initiatives and special
projects; use of novel or effective risk assessment approaches; and
overall productivity, including achievement of numerical examination
goals.
In contrast to OCIE, we were told that federal bank and other regulators
have quality assurance programs that include reviews of completed
examinations or other activities. Examples of such reviews include:
o FDIC guidance states that the agency reviews each regional office's
compliance examination program every 2 years, in part, to evaluate the
consistency of supervision across the regions and compliance with
policies and procedures. According to the guidance, evaluations include a
review of examination reports and work papers.
o Federal Reserve officials said that the agency conducts on-site
operations reviews of the banking supervision function of individual
Reserve Banks at least every 3 years. The review targets each Reserve
Bank's risk-focused supervision program and includes a review of a sample
of examination reports, work papers, and other supporting documentation.
It also encompasses the bank's ongoing quality management function, or the
processes, procedures, and activities the bank uses to ensure that
examination reports and related documents are of high quality and comply
with established policy.
o OCC officials told us that the agency reviews its large bank
examination program, including specific examination procedures. It
conducts reviews to determine whether lead examiners are supervising banks
according to plans. It also assesses specific examination procedures
across samples of banks. Agency officials said that teams periodically
review how examiners are conducting certain procedures to ensure that they
are being implemented consistently throughout all field offices.
o NASD conducts quality and peer reviews to improve the quality,
consistency, and effectiveness of its examination program. Under quality
reviews, each NASD district office annually evaluates its performance in
two or three areas. Under peer reviews, staff go on-site to district
offices to evaluate particular program areas.
o NFA officials told us that the organization randomly selects completed
audits for review on a quarterly basis and, as part of the review,
supervisory teams review work papers to determine whether the audits
complied with established policies and procedures.
While OCIE staff evaluate all completed examinations by reviewing the
final examination report, they do not review a sample of completed
examinations and work papers to periodically assess examination quality
and consistency across SEC's field offices. SEC officials stated that
afterthe-fact reviews of underlying work papers may not be a
cost-effective use of resources, given that key findings and evidentiary
materials should be discussed and described in the examination report
itself, which is reviewed by OCIE staff. Further, it would be difficult to
second-guess decisions made by examiners when on-site, since reviewers
would not have access to the same information. Finally, agency officials
said that OCIE resources are
limited, and time spent reviewing completed examination work papers would
result in less time spent on conducting examinations. While reviewing
completed examination work papers involves resource tradeoffs, it may
yield important benefits. OCIE may be able to better determine whether its
examiners are complying with established policies and procedures and
whether its built-in quality controls are working. A review of underlying
work papers also may allow OCIE to better assess the consistency of
examination quality within and across SEC's field offices as well as the
extent to which existing quality controls are helping to ensure that
quality is maintained.
According to SEC officials, the agency is implementing a computer-based
document management system. Under this system, it is anticipated that
most, if not all, of the work papers created during examinations will be
converted into electronic files, and these files will be maintained in a
consistent manner online for a number of years. SEC officials said that
when the system is fully operational, estimated to be some time in 2006,
all work papers created during an examination will be available
electronically to OCIE staff. At that point, OCIE liaisons could review
electronic examination work papers on a sample basis in conjunction with
their review of examination reports. In addition, electronic work papers
would eliminate the need to be on-site to review underlying examination
documentation and work papers across SEC's examination program.
Importantly, deficiencies we found during our review of risk scorecards
highlight the need for OCIE to periodically assess the consistency of
examination staff's use of scorecards and other steps being taken during
examinations. While the requirement to complete risk scorecards became
effective in October 2002, SEC has not yet evaluated, for instance,
whether the risk scorecards are being completed according to the guidance
provided, whether changes to the design of the scorecards are needed, and
whether additional guidance or training is needed. In March 2003, OCIE
provided one training course on the scorecards, which was attended by 98
examiners, or about 20 percent of the SEC examiners devoted to fund and
adviser examinations. According to SEC officials, two senior OCIE staff
visited each field office during the spring and summer of 2003 and
provided a full day of training on the scorecards to all examination
staff. Nevertheless, the scorecard deficiencies we found during our review
may indicate that additional training is needed.
In addition, the scorecards may have design weaknesses that result in
inconsistencies across SEC field offices. For example, field office
officials
stated that scorecards are designed for investment companies organized as
mutual funds and do not readily apply to investment companies organized as
unit investment trusts.25 NERO examiners did not complete scorecards for
unit investment trusts, but MRO examiners did by modifying the scorecards
as needed. Similarly, SEC field office officials stated that while the
scorecards are designed to cover a broad range of fund compliance
controls, fund controls for detecting and preventing market timing do not
fall squarely under any of the 13 areas covered by the scorecards. As a
result, staff have used work papers other than the risk scorecards to
document their assessment of market timing controls. SEC officials said
that the scorecards are models created to assist examiners in assessing
fund controls. As such, scorecards are not intended to exist necessarily
for every conceivable control and examiners have the flexibility to modify
the scorecards as necessary. Moreover, the officials said that some
inconsistencies in the preparation of risk scorecards are expected because
not all funds and advisers are the same. In that regard, SEC officials
told us that the approach taken by MRO staff in modifying a scorecard to
fit the circumstances of an examination appears to be consistent with the
approach to scorecard use expected by OCIE.
SEC's Oversight Examinations of Broker-Dealers Provide Limited Information on
the Adequacy of SRO Oversight
To assess SRO oversight of broker-dealers, including their mutual fund
sales practices, SEC conducts examinations of broker-dealers shortly after
they have been examined by SROs. However, these SEC broker-dealer
examinations, which involve a significant commitment of agency examination
resources, provide limited information on the adequacy of SRO oversight
and impose duplicative regulatory costs on the securities industry. SEC
and SROs' broker-dealer examinations often cover different time periods,
and generally employ different sampling methodologies and use different
examination guidelines. Consequently, SEC cannot reliably determine
whether its examination findings are due to weaknesses in SRO examination
procedures or some other factor. Another deficiency we found regarding
SEC's SRO oversight of broker-dealer mutual fund sales practices is that
the agency does not have automated information on the full scope of areas
reviewed during its broker-dealer oversight examinations and, therefore,
cannot readily and reliably track useful examination information.
25A unit investment trust is an investment company that (1) is organized
under a trust indenture, (2) does not have a board of directors, and (3)
issues only redeemable securities, each of which represents an undivided
interest in a unit of specified securities.
SEC Often Cannot Attribute Broker-Dealer Oversight Violations It Finds to
Weaknesses in SRO Examination Programs, Because Different Examination
Procedures Are Used
SEC performs two types of activities to review the quality of SRO
oversight of broker-dealers, including their sales of mutual funds. First,
SEC conducts inspections of NASD and NYSE on a 3-year cycle that cover
various aspects of their compliance, examination, and enforcement
programs. These SRO oversight inspections are designed to determine
whether an SRO is (1) adequately assessing risks and targeting its
examinations to address those risks, (2) following its examination
procedures and documenting its work, and (3) referring cases to
enforcement authorities when appropriate. When conducting these
inspections, SEC reviews a sample of the SRO's examination reports and
work papers to identify problems in examination scope or methods. As a
result of these inspections, SEC has identified deficiencies in SRO
examinations, including ones related to the SROs' examinations of mutual
fund sales practices, and communicated those to the SRO to remedy the
problem. Second, SEC conducts broker-dealer oversight examinations, during
which it examines some broker-dealers from 6 to 12 months after an SRO
examines the firms. The purpose of broker-dealer oversight examinations is
to help the SROs improve their examination programs by identifying
violations that the SROs did not find and also by assisting them in
evaluating improvements in how SRO examiners perform their work. SEC
officials told us that a secondary goal of these examinations is to
supplement the SROs' enforcement of broker-dealer compliance with federal
securities laws and regulations.
SEC's broker-dealer oversight examinations involve a significant
commitment of agency resources and expose firms to duplicative
examinations and costs. In addition to conducting broker-dealer
examinations for the purposes of assessing SRO oversight (including for
mutual fund sales practices), SEC conducts cause, special, and
surveillance examinations of broker-dealers to directly assess
broker-dealer compliance with federal securities laws and regulations,
including those related to mutual fund sales. SEC currently has an
internal goal of having oversight examinations account for 40 percent of
all broker-dealer examinations each year. In 2004, 250, or 34 percent, of
its 736 broker-dealer examinations were oversight examinations.26
Broker-dealers that are subject to similar SEC and SRO examinations that
may take place within a 6 to 12 month
26As of October 2004, about 40 percent of OCIE's total examination staff
of 810 individuals was assigned to broker-dealer, transfer agent, and
clearing agency examinations (315 examiners dedicated to broker-dealer,
transfer agent, and clearing agency examinations and 495 individuals
assigned to mutual fund and investment adviser examinations).
period incur the costs associated with assigning staff to respond to
examiner inquiries and to make available relevant records as requested.
Although SEC broker-dealer oversight examinations involve a significant
commitment of agency examination resources and impose costs on securities
firms, our past work questioned their cost-effectiveness. In a 1991
report, we found that the way SEC conducted oversight examinations of
broker-dealers provided limited information to help SROs improve the
quality of their broker-dealer examination programs.27 Specifically,
during its oversight examinations of broker-dealers, SEC often found
violations not identified by SROs and frequently could not attribute the
violations it found to weaknesses in SRO examination programs. Because SEC
and SROs used different examination procedures or covered different time
periods of broker-dealer activity, SEC examiners often could not determine
whether the violations they found resulted from the improper
implementation of procedures by SRO examiners or differences between the
procedures used or the activity period covered. We previously recommended
that SEC directly test SRO examination methods and results. However, based
on its efforts to replicate some examinations conducted by SROs, the
agency concluded that this was unproductive because it only confirmed
findings identified by SROs during their examinations.
Our current review has shown that despite our 1991 findings, SEC continues
to conduct oversight examinations in a similar manner-by using different
examination guidelines and time periods. First, SEC continues to review
firm activities during the time between the completion of the SRO
examination and its own examination. Next, when SEC is reviewing a firm's
transactions or customer accounts to identify potential abuses, it
generally does not duplicate the sampling technique used by the SRO, but
instead selects its own sample of transactions or customer accounts based
on its own procedures. Finally, SEC examiners ask different questions to
identify potential abuses. For example, although SEC and NASD both direct
their examiners to ask questions to assess potential weaknesses in a
firm's internal controls to prevent market timing and late trading, their
procedures call for examiners to ask about different potential internal
control weaknesses.
27GAO/GGD-91-52.
According to SEC officials, its examiners do not use the same procedures
as SROs because using different procedures allows them to find violations
that would not otherwise be found if they just duplicated the SRO
procedures. Also, SEC officials stated that SEC has an obligation to
review the broker-dealer's activities at the time of the SEC examination
to ensure compliance with securities laws at that time. However, as a
result, SEC often cannot determine the specific reason why the SRO did not
find the violations, limiting its ability to suggest improvements to SRO
programs. SEC routinely provides SROs copies of deficiency letters it
sends to brokerdealers as a result of oversight examinations. These
deficiency letters sometimes include oversight comments that include steps
the SRO can take toenhance itsprogram.SRO officials stated they can often
identify the reasons why SEC found the violations, but in many cases the
reason is due to SEC's use of different procedures, such as different
review periods or samples. Consequently, SEC often cannot attribute a
violation it finds to a problem with the SRO's examination program. SEC
officials said in some cases when SEC identifies a violation, it is able
to determine whether the violation was occurring at the time of the
original examination and should have been detected by the SRO. For
example, in some cases when SEC finds an error in a broker-dealer's net
capital calculation, it is able to trace the error to previous
calculations and determine whether it existed during the SRO examination.
Even in cases when SEC can attribute a violation it found to a weakness in
the SRO examination, it does not track this information in its automated
examination tracking system and, as a result, cannot use it to identify
trends in SRO problems it discovered during oversight examinations. SEC
officials stated that they have a staff committee conducting a
comprehensive review of oversight examination procedures and plan to add a
feature to SEC's examination tracking system to allow it to more
systematically track identified weaknesses in SROs' examination programs.
Although SEC's oversight examinations continue to find violations at
broker-dealers and, thus, provide investor protection benefits, the
violations provide limited information for assessing the quality of the
SRO program. This information is particularly important given that the
number of violations that SEC has found during its oversight examinations
and determined as not found by NASD has increased in recent years. As
shown in figure 2, the number of these violations that SEC found but has
categorized as not found by NASD more than doubled between fiscal years
2002 and 2004.
Figure 2: Violations Found by SEC during Oversight Examinations of NASD
but Not Found by NASD, and SEC Oversight Examinations Conducted of NASD,
Fiscal Years 2002-2004
Number
250
235
200
150
100
50
0 2002 2003 2004
Year
Violations found by SEC during oversight examinations of NASD but not
found by NASD
SEC oversight examinations conducted of NASD
Source: SEC.
Despite this significant increase, SEC officials could not explain why the
number of these violations increased but stated that the increase did not
necessarily represent a decrease in the quality of NASD's examination
program. They said some of the increase is due to a significant increase
in the number of rules applicable to broker-dealers. SEC officials told us
that SRO officials have noted, and they agree, that the number of these
violations, alone, is not always an appropriate measure of the quality of
SRO examination programs. Accordingly, SEC officials told us that the
agency recently began tracking findings deemed to be significant to allow
it to better assess the materiality of an increase in the number of missed
violations. If SEC had tested NASD's examination methods or better tracked
the reasons why NASD did not find a violation, SEC would have more
information to assess the quality of NASD's examination program.
SEC Does Not Track the Full Scope of Work Performed during Its Oversight
Examinations
Another deficiency we found regarding SEC's SRO oversight is that the
agency cannot readily and reliably track key examination information. In
assessing the quality of SEC's oversight of broker-dealer sales of mutual
funds, we asked SEC to provide data on which of its broker-dealer
oversight examinations in recent years included reviews of mutual fund
sales practices. The data would help determine the extent that SEC has
reviewed mutual fund sales practices. SEC was not able to provide this
information because it does not have automated information on the full
scope of areas reviewed during its broker-dealer oversight examinations.
SEC maintains a broad range of automated information about its
examinations in its Super Tracking and Reporting System (STARS), including
basic information about the firm, SEC staff assigned to conduct the
examination, and the deficiencies and violations found during the
examination. STARS identifies examinations that reviewed specific areas of
special interest to SEC, called "focus areas," as identified by senior SEC
staff in headquarters, and new areas are added in part based on the
emergence of new abuses. For example, SEC added breakpoints as a focus
area in January 2003 and market timing and late trading in 2004.28
Although focus area designations provide useful information about how
often SEC reviews some areas, focus areas do not cover all areas
potentially reviewed by SEC during its examinations. Without methodically
tracking the full scope of work performed during oversight examinations,
SEC lacks information for determining how effective its oversight is in
two important areas.
First, because SEC does not know how often it has reviewed particular
areas such as mutual fund sales practices during its oversight
examinations, it cannot ensure that it has adequately reviewed all areas
it considers important. When SEC reviews particular areas, its examiners
generally refer to a set of written procedures, known as examination
modules that provide information to guide examiners' work. STARS does not
include data fields to track whether staff use the module on mutual funds
during an examination. Therefore, the extent of coverage of mutual funds
is unknown. As a result, SEC officials could not determine how many of the
approximately 1,400 broker-dealer oversight examinations conducted between
2000 and 2004 included a review of mutual fund sales
28Breakpoints are discounts offered to investors on up-front sales charges
on certain mutual fund shares when an investor makes a large purchase.
practices. SEC officials stated that they have a separate database
containing examination reports that can be electronically searched to
identify relevant examinations containing a search term such as "mutual
fund," which would yield an estimate of the number of examinations that
reviewed broker-dealer mutual fund sales practices. However, according to
an official, not all examinations covering mutual fund sales practices
would be captured because some examination reports that included reviews
of mutual fund sales practices would not necessarily include any mention
of mutual funds, especially if SEC identified no deficiencies or
violations in that area.
In contrast to SEC, both NASD and NYSE have systems with capability to
track the full scope of examinations including the use of mutual fund and
other examination modules. For NASD, some of its offices are able to track
which of its broker-dealerexaminations were followed by an SEC oversight
examination. At 8 of its 15 district offices, which account for 55 percent
of its examinations, NASD tracked this information and SEC conducted
oversight examinations of approximately 5 percent of the 2,602 NASD
examinations conducted between January 1999 and August 2004 that reviewed
mutual fund sales practices. The remaining seven offices were not able to
track this information because, according to an NASD official, the SEC
field office conducting oversight examinations did not always provide a
letter informing them that an oversight examination was conducted. With
mutual fund sales practices being a regulatory priority, the percentage of
SEC examinations reviewing these practices would be a useful measure for
ensuring that the agency is addressing this priority.
Second, because SEC does not track the full scope of work performed during
its oversight examinations, it is limited in its ability to assess the
significance of deficiencies and violations it finds. Because SEC does not
know how often it has reviewed a particular area, the data it tracks on
the number of deficiencies and violations it finds in a particular area
are less meaningful. For example, it would be less significant if SEC
found violations in a particular area during 5 out of 100 examinations as
opposed to finding violations during 5 out of 5 examinations during which
it reviewed the area. Without knowing the full scope of each oversight
examination and therefore the number of times a particular area was
reviewed, data tracked by SEC on the number of deficiencies and violations
it finds are less meaningful.
In addition to conducting broker-dealer oversight examinations to evaluate
the adequacy of SRO activities, SEC conducts other types of examinations,
including cause and sweep examinations, which are designed to directly
assess broker-dealer compliance with the law. SEC tracks the number of
firms it targets during its examination sweeps along with the number of
violations and deficiencies it finds. SEC officials told us that the
agency tracks the number of findings from these examinations as a
percentage of the number of firms examined, and that tracking such
information helps SEC assess the prevalence of the findings relative to
the number of firms. However, without tracking the scope of work performed
during its oversight examinations, SEC is unable to make similar
assessments about the prevalence of violations and deficiencies identified
during those reviews.
Appendix II provides information you requested about (1) how SEC, NASD,
and NYSE share information, including written examination guidance,
related to their review of mutual fund sales practices and other
examination priorities; (2) how SEC distributes and stores examination
guidance for use by its broker-dealer examiners; and (3) what training SEC
has provided to broker-dealer examiners on mutual funds and other topics
and how it tracks and assesses such training.
Conclusions In the wake of the market timing and late trading abuses, SEC
staff implemented significant changes to the agency's mutual fund
examination program in the view that doing so would help ensure the
earlier detection and correction of violations. These changes-including
conducting additional sweep examinations and continuously monitoring large
companies-reflect a practical approach designed to focus SEC's limited
resources on higher risk funds and activities and have the potential to
strengthen SEC's oversight practices in certain regards. Nonetheless, the
changes also involve tradeoffs, such as limiting the agency's capacity to
review funds perceived to be lower risk and conduct risk assessments of
all funds in a timely manner. Moreover, SEC's capacity to effectively
monitor the hedge fund industry is open to question, given the tradeoffs
that the agency has had to make in overseeing the mutual fund industry.
While we recognize that SEC at some point may need to request additional
resources from Congress to carryout its mutual fund and other oversight
responsibilities, such requests should only occur after the agency has
explored and achieved all available efficiencies within its existing
resource limitations. Whether SEC's utilization of resources under its
revised examination program will provide effective oversight remains to be
seen. Future adjustments by SEC to resources devoted to various oversight
activities, such as sweep examinations and randomly selected lower risk
fund examinations, are likely to occur as the agency gains experience
through conducting these oversight activities and changing conditions in
the mutual fund industry. However, SEC has had extensive experience with
its broker-dealer oversight examinations, and the effectiveness of these
examinations for improving the quality of SRO oversight remains unclear.
This situation raises concern, particularly in light of the significant
level of resources devoted to oversight examinations and the resource
challenges faced by SEC's fund and adviser examination program.
We also identified basic weaknesses in SEC's approaches to conducting
mutual fund and broker-dealer examinations. For mutual fund examinations,
SEC does not require staff to document their examination plans to
facilitate supervisory review. Second, SEC has issued work paper standards
but lacks guidance on their supervisory review. Moreover, despite the
importance of risk scorecards in determining the depth of work done during
examinations, SEC has not yet assessed whether they are prepared according
to standards since implementing the scorecards in 2002. For broker-dealer
examinations, SEC has not developed an automated system to track the full
scope of work completed during examinations and therefore lacks useful
information about SRO oversight. Without addressing these deficiencies,
SEC's capacity to effectively oversee the mutual fund industry and SROs is
reduced.
Recommendations for Executive Action
To improve SEC's oversight of mutual funds and SRO oversight of
brokerdealers that sell mutual funds, we are making four recommendations
to the SEC Chairman. First, we recommend that SEC periodically assess the
level of resources allocated to the various types of examinations in light
of their regulatory benefits to help ensure that the agency is using its
resources efficiently and effectively to oversee the mutual fund industry,
including broker-dealers that offer mutual funds. As part of this
assessment, SEC should seek to ensure that it allocates sufficient
resources to mitigate any regulatory gaps that may currently exist
concerning the timely examination of mutual funds perceived to represent
lower risk, complete mutual fund risk assessments within a more reasonable
period, and fulfill its new oversight responsibilities for the hedge fund
industry. Second, in so doing, we recommend that the agency assess its
methodology for conducting broker-dealer oversight examinations and
whether some portion of the resources currently devoted to these
examinations could be better utilized to perform mutual fund examinations.
Third, to strengthen SEC's approach to mutual fund examinations, we
recommend that SEC
o establish a policy or procedure for supervisory review of work papers
prepared during routine examinations and for documenting such reviews;
o establish a policy or procedure for preparing a written plan for each
routine examination, documenting at a minimum the preliminary objectives
and scope of the examination; and
o consider reviewing on a sample basis completed routine examinations and
work papers to assess the quality and consistency of work within and
across the field offices conducting examinations.
Fourth, to assess and improve the effectiveness of SEC's oversight of SRO
broker-dealer examination programs, we recommend that the Chairman, SEC,
electronically track information about the full scope of work performed
during broker-dealer oversight examinations, including all major areas
reviewed, to determine whether areas are receiving adequate review and to
more fully assess the significance of deficiencies and violations found.
Agency Comments and Our Evaluation
SEC provided written comments on a draft of this report, which are
reprinted in appendix III. SEC also provided technical comments that we
incorporated into the final report, as appropriate. SEC focused most of
its comments on providing further elaboration on the potential benefits of
its examination strategy for overseeing mutual funds and investment
advisers and on the benefits obtained from its broker-dealer oversight
examinations. In addition, SEC briefly commented that it will consider our
recommendation directed at improving its quality controls for routine fund
examinations and that it has formed a working group to explore ways to
enhance the value of its broker-dealer oversight examinations, including
their ability to identify the reasons that violations may have been missed
by SRO examinations.
First, SEC stated that it is not possible for the agency to conduct
timely, comprehensive routine examinations of every mutual fund and
adviser, given the size of the industry and agency resources. Further, it
expects its risk-targeted examinations to provide an effective means of
addressing risks in the securities industry. Specifically, it believes
that looking at the
same type of risk at a number of different firms is a better approach than
examining a single firm in depth. According to SEC, this approach will
provide benefits by promptly identifying emerging trends and compliance
problems, and individual firms can be compared to their industry peers.
The agency believes this approach has already yielded benefits in
identifying and addressing significant compliance problems before becoming
major crises. In addition, SEC stated that the program it is developing to
randomly select a sample of lower risk firms for routine examination will
address our concern that such firms may not be given sufficient attention
under its revised oversight strategy. According to SEC, this approach will
provide a deterrent effect, enable the agency to test assumptions and
techniques used throughout its examination program, and allow the agency
to draw inferences about compliance in the adviser community, based on
statistically valid sampling techniques.
We recognize that SEC's revised examination strategy for mutual funds and
advisers offers potential benefits, including focusing its limited
resources on firms and activities that are perceived to pose higher risks.
Nonetheless, we continue to be concerned about SEC's ability to examine
all mutual funds within a reasonable period and accurately assess the
relative risk of each fund on a timely basis. Unlike broker-dealers,
mutual funds are regulated and examined solely by SEC. Under SEC's current
plans to randomly sample 10 percent of the firms perceived to be lower
risk for routine examination each year, it is possible that up to a third
of the total number of firms would not be selected for examination within
a 10-year period. We believe that this is a lengthy time period for firms
to conduct business without being examined. Similarly, SEC's inability to
conduct examinations of all mutual funds within a reasonable period will
limit its capacity to accurately distinguish relatively higher risk funds
from lower risk funds and effectively target its limited examination
resources on those funds posing the highest risks. Therefore, we continue
to believe that, as recommended, SEC should periodically assess the level
of resources allocated to its various types of examinations and in so
doing ensure that it allocates sufficient resources to mitigate any
regulatory gaps that currently exist in the timely examination of funds
perceived to represent lower risks and to ensure that it completes mutual
fund risk assessments within a more reasonable time period.
Second, SEC stated that its broker-dealer oversight examinations provide
quality control over SRO examinations and serve other important goals. For
example, SEC stated that oversight examinations allow it to detect
violations that otherwise might not be detected, conduct routine
examinations of new products or services, and test and validate
assumptions and techniques used throughout the broker-dealer examination
program. In addition, SEC expressed concern about our suggestion that it
should reproduce SRO examinations if its oversight examinations are to
provide accurate quality control information. SEC stated that this
suggested approach would result in redundancies for broker-dealers being
examined and limit the agency's ability to reach conclusions about SRO
examination programs. By conducting its examinations as independent
compliance reviews, SEC stated that it can assess whether SRO procedures
were followed and whether SRO procedures need to be modified or enhanced.
The agency stated that through its oversight program it has identified SRO
procedures that need to be modified or enhanced and its examiners meet
regularly with SRO examiners to review the results of oversight
examinations. Finally, SEC commented that it has formed a working group to
explore ways to gain additional value from its broker-dealer oversight
examinations, such as by better identifying the reasons that a violation
may not have been detected by an SRO examination, aiding the SRO in
improving its program, and minimizing burden on the firm examined.
We recognize that SEC's oversight examinations serve more than one goal
and provide investor protection benefits. While such examinations serve a
variety of purposes, one of their primary purposes is to assess the
quality of SRO examinations. In fulfilling this purpose, we remain
concerned that SEC's approach provides limited ability to identify the
reasons why an SRO did not find violations that SEC found and, in turn,
provide suggestions for improving SRO examinations. SEC is responsible for
overseeing SROs that examine broker-dealers on a regular basis, and it
conducts oversight examinations of only a small percentage of the total
number of brokerdealers. Thus, it is critical for SEC to ensure that SROs
conduct effective examinations. As discussed, SEC has formed a working
group to evaluate its oversight examinations. We believe this is a step in
the right direction and also provides the agency with the opportunity to
evaluate its approach and level of resources devoted to broker-dealer
oversight examinations.
Finally, regarding our recommendation that SEC strengthen three aspects of
its quality control framework for routine fund examinations, the agency
stated it will fully consider the recommendation. Specifically, in 2006,
the agency plans to deploy an electronic system for work papers. In
preparation for this effort, it plans to review how new technology can be
used to improve the quality of examinations and it will consider our
recommendation in its review. While SEC did not directly comment on our
recommendation that it electronically track information about the full
scope of work performed during its broker-dealer oversight examinations,
we believe that this would provide SECimportant information to determine
whether areas are receiving adequate review and the relative significance
of violations found in each area.
As agreed with your office, unless you publicly announce the contents of
this report earlier, we plan no further distribution of this report until
30 days from the report date. At that time, we will provide copies of this
report to the Chairman of the House Committee on Financial Services; the
Chairman of the Subcommittee on Capital Markets, Insurance and Government
Sponsored Enterprises, House Committee on Financial Services; and the
Chairman and Ranking Minority Member of the Senate Committee on Banking,
Housing, and Urban Affairs. We also will provide copies of the report to
SEC, FDIC, the Federal Reserve Board of Governors, NASD, NYSE, and OCC and
will make copies available to others upon request. In addition, the report
will be available at no cost on GAO's Web site at http://www.gao.gov.
If you or your staff have any questions about this report, please contact
me at (202) 512-8678 or [email protected]. Contact points for our Offices
of Congressional Relations and Public Affairs may be found on the last
page of this report. GAO staff who made key contributions to this report
are listed in appendix IV.
Richard J. Hillman Director, Financial Markets and
Community Investment
Appendix I
Scope and Methodology
To identify and assess the changes SEC has made to or is planning for its
mutual fund examination program, we reviewed SEC testimony, speeches,
reports, and other documents related to the agency's mutual fund
examination program. We also reviewed federal securities laws and
regulations applicable to mutual funds and analyzed SEC data on the
number, types, and results of its fund and adviser examinations. We also
interviewed officials from SEC's Office of Compliance Inspections and
Examinations (OCIE), Division of Investment Management, and Office of Risk
Assessment and representatives from the Investment Company Institute to
obtain information on the significance of planned changes. In addition, we
interviewed federal bank regulatory officials from the Federal Deposit
Insurance Corporation (FDIC), Board of Governors of the Federal Reserve
System, and Office of the Comptroller of the Currency (OCC) and
self-regulatory organization (SRO) officials from NASD, the New York Stock
Exchange (NYSE), and the National Futures Association (NFA) to discuss
their examination programs and supervisory tools.
To assess key aspects of the quality control framework of SEC's routine
mutual fund examinations, we reviewed policies, procedures, and other
guidance applicable to those examinations. We also reviewed routine fund
examinations completed in fiscal year 2004 by SEC's Midwest Regional
Office (MRO), Northeast Regional Office (NERO), and Philadelphia District
Office (PDO). We selected these field offices because they were the three
largest in the number of completed routine fund examinations in fiscal
year 2004. The three offices completed 66 routine fund examinations,
accounting for about 72 percent of all routine fund examinations completed
in fiscal year 2004.1 Where appropriate, we also reviewed examinations of
advisers to the funds we reviewed.2 We used a standardized data collection
instrument to document the methods examiners used to conduct examinations
and areas examiners reviewed during examinations. In addition, we
interviewed officials from OCIE and three SEC field offices-MRO, NERO, and
PDO-about their examination policies and procedures and representatives
from a mutual fund company and consulting firm about fund examinations. To
gather information and compare SEC examinations with those of other
regulators, we interviewed
1MRO, NERO, and PDO completed 40, 17, and 9 routine fund examinations,
respectively, in fiscal year 2004.
2SEC focuses its examinations on fund complexes, or groups of funds that
generally share the same adviser. SEC officials told us they examine both
the fund complex and affiliated adviser at the same time when both are
located at the same site.
Appendix I Scope and Methodology
officials from FDIC, the Board of Governors of the Federal Reserve System,
OCC, NYSE, and NFA about their quality controls and reviewed some of their
quality control policies and procedures.
To determine the adequacy of SEC's oversight of NASD and NYSE in
protecting shareholders from mutual fund sales practice abuses, we
reviewed SEC policies, procedures, and other guidance related to its
broker-dealer oversight examinations and inspections and interviewed
officials from SEC's OCIE and Boston District Office, NASD, and NYSE. We
also reviewed judgmentally selected SEC broker-dealer oversight
examinations conducted by SEC's Boston District Office in 2003 and 2004,
and reviewed all reports of SRO inspections conducted of NASD and NYSE
between 2001 and 2003. Togather informationon SEC's automated tracking
system, Super Tracking and Reporting System, we interviewed SEC staff
responsible for the system in headquarters and received an overview of the
system and its capabilities at the Boston District Office. In addition, we
reviewed reports generated from the system and training documents for the
system. To help assess the extent to which SEC, NASD, and NYSE have shared
written guidance, we compared and contrasted the examination modules they
used to examine for certain mutual fund sales practice abuses. As part of
our assessment of the training received by broker-dealer examiners, we
obtained and analyzed SEC's training attendance rosters and list of
examiners employed by SEC since 1999.
To ensure that data provided about the number, nature, and results of
examinations conducted by SEC, NASD, and NYSE were reliable, we reviewed
written materials describing these systems and reviewed the data provided
to check for missing or inaccurate entries. We also interviewed agency
staff responsible for maintaining the information systems that track such
data. We determined that the data were sufficiently reliable for use in
this report.
We performed our work in Boston, Massachusetts; Chicago, Illinois; New
York, New York; Philadelphia, Pennsylvania; and Washington, D.C. We
conducted our work between February 2004 and July 2005 in accordance with
generally accepted government auditing standards.
Appendix II
Securities and Exchange Commission's (SEC) Broker-Dealer Examination Guidance
and Training
You asked us to provide information about aspects of SEC's oversight of
the broker-dealer industry, including (1) how SEC, NASD, and the New York
Stock Exchange (NYSE) share information, including written examination
guidance, related to their review of mutual fund sales practices and other
examination priorities; (2) how SEC distributes and stores examination
guidance for use by its broker-dealer examiners; and (3) what training SEC
has provided to its broker-dealer examiners on mutual funds and other
topics, and how it tracks and assesses such training.
Sharing of Written Mutual Fund Examination Guidance among Regulators
SEC, NASD, and NYSE have developed guidance for examiners to use in
assessing compliance by broker-dealers with mutual fund sales practice
rules. Each regulator has developed its own examination module, or set of
procedures, covering various topics related to mutual fund sales.
Moreover, all three regulators recently revised their modules to include
procedures to detect market timing and late trading abuses. In addition,
the regulators periodically have provided their staff with other written
guidance related to mutual fund sales. For example, SEC issued internal
memorandums in 1997 and 2001 to inform staff about abuses related to
breakpoints1 and other mutual fund sales practices and provide them with
procedures for detecting such abuses.
Through its oversight role, SEC reviews aspects of the self-regulatory
organization (SRO) examination modules, including the mutual fund sales
practice module. First, SEC officials told us that NYSE and NASD e-mail
SEC copies of their examination modules when they make material changes to
them. Second, during SEC's on-site inspections of SRO examination
programs, staff generally review SRO examination modules in connection
with their review of completed SRO broker-dealer examinations and work
papers. Third, as part of their broker-dealer oversight examinations, SEC
staff generally review the SRO broker-dealer examinations and applicable
examination modules before going on-site to conduct examinations of such
broker-dealers.
SEC and the SRO officials meet at least semiannually to discuss
significant examination findings, customer complaints, trends in the
industry, enforcement cases, and examination guidance. SEC officials told
us that
1Breakpoints are discounts offered to investors on up-front sales charges
on certain mutual fund shares when an investor makes a large purchase.
Appendix II Securities and Exchange Commission's (SEC) Broker-Dealer
Examination Guidance and Training
agency staff have met with NASD and NYSE officials semiannually and
quarterly, respectively, to discuss, among other things, examination
findings and guidance. The officials also said that they hold frequent
telephone conversations to coordinate their examination efforts. For
example, SEC, NASD, and NYSE staff talked with each other immediately
following NASD's discovery of breakpoint abuses in 2002, and established a
joint task force to determine the extent of the abuses by conducting
examinations of firms designed to identify failures to provide breakpoint
discounts. Similarly, SEC, NASD, and NYSE staff talked with each other in
their efforts to respond to the late trading and market timing abuses
uncovered in 2003. In addition, SEC and SRO staff jointly attended
conferences and training that included examination guidance as a topic of
discussion. Finally, SEC, NASD, and NYSE have jointly developed a number
of examination modules to enforce recent changes in laws and rules
applicable to broker-dealers.
Although SEC, NASD, and NYSE coordinate in these ways to oversee
broker-dealers, they generally do not provide copies of their written
examination materials to each other. That is, SEC typically does not
provide copies of its modules or other internal written guidance to the
SROs, nor do NASD and NYSE generally provide copies of such guidance to
each other. Officials at these agencies shared benefits and drawbacks of
providing written copies of examination materials to each other. The
regulators agreed that sharing information about their examination
approaches and outcomes is overall a positive way to more effectively
oversee the broker-dealer industry. They cautioned, however, that certain
drawbacks should be considered regarding the sharing of written
examination materials. SEC officials said that sharing SEC examination
modules could compromise its supervision of the SROs. According to the
officials, if SEC shared its modules, the SROs may be less innovative and
motivated to improve their methods. They said, for example, that the SROs
may view SEC's procedures as the most that they would need to do. NASD
officials strongly disagreed with SEC's assertions about the sharing of
examination modules, saying they always seek the most effective
examination procedures, regardless of those used by SEC; and an NYSE
official said that while NYSE understands the SEC's position in this
regard, the sharing of SEC's examination module would only enhance NYSE's
preexisting examination procedures related to mutual funds. NASD and NYSE
officials said it would be helpful if SEC shared copies of its modules and
other guidance it shares with its own examiners. However, SEC and NASD
officials said that NASD and NYSE may not want to share their examination
modules with each other because of competitive reasons. For example, if
Appendix II Securities and Exchange Commission's (SEC) Broker-Dealer Examination
Guidance and Training
one SRO shared its modules with another SRO, it would run the risk that
its competitor could be able to adopt similar procedures without the cost
of developing them. Finally, NASD officials told us that differences exist
between NASD's and NYSE'smembership, culture, priorities, and strategies
that can lead to differences in examination procedures, and the same is
true for financial institutions overseen by the banking regulators.
Distribution and Storage of Broker-Dealer Examination Guidance
SEC's Office of Compliance Inspections and Examinations (OCIE) oversees
and directs SEC's broker-dealer examination program, but SEC's 11 field
offices conduct the vast majority of the broker-dealer examinations. Among
other things, OCIE creates and updates brokerdealer examination modules,
or policies and procedures; issues other examination guidance; and reviews
broker-dealer examination reports. Currently, when OCIE develops and
issues policy changes and examination guidance, it typically distributes
such guidance to the field offices by e-mail. In turn, each field office
separately stores the guidance on one of its shared computer drives or in
some other way to provide its examiners access to the information. Field
office examiners generally are responsible for keeping abreast of changes
in guidance and reviewing it as needed in performing examinations.
To better ensure that SEC examiners across all field offices have access
to current and complete broker-dealer examination guidance, OCIE is
developing an internal Web site to serve as a central repository for all
broker-dealer examination guidance. According to agency officials, OCIE
launched its internal Web site in April 2005 on a pilot basis to select
brokerdealer examiners nationwide to obtain their comments about its
organization and comprehensiveness. Subsequently, SEC made the Web site
available to all examiners in July 2005. According to SEC officials, the
Web site will allow broker-dealer examiners to access not only all
guidance at one location but also links to databases and numerous other
examiner tools.
Broker-Dealer Examiner SEC's OCIE has a training branch that provides
routine and specialized
Training and Tracking training to its broker-dealer examiners, with some
of the training related to mutual funds. More specifically, OCIE's
training branch provides a twophased training program for broker-dealer
examiners that is designed to teach examiners how to handle increasingly
complex examination issues. According to an SEC official, the phase-one
course is designed for new examiners and includes some training on mutual
fund operations and
Appendix II Securities and Exchange Commission's (SEC) Broker-Dealer
Examination Guidance and Training
mutual fund sales practices of broker-dealers. OCIE's training branch also
offers a range of specialized training delivered in a variety of formats.
For example, it offers classroom training sessions and videoconferences
taught by senior examiners or vendors, such as NASD, as well as training
videos that examiners can view when convenient. An SEC official told us
that since 1999 the training branch has offered over 25 training sessions
that have included mutual fund topics, such as breakpoints.
In addition, SECperiodically has coordinated its training efforts with
SROs, including NASD and NYSE. For example, examiners representing SEC,
NASD, NYSE, and other SROs, as well asstate regulators have metannually
for a 3-day joint regulatory seminar to receive training about emerging
and recurring regulatory issues. In 2003 and 2004, the seminars provided
training on mutual fund sales practice abuses, including late trading,
market timing, and failure to provide breakpoint discounts. Finally, SEC
examiners attend or participate in external training, such as industry
conferences.
Separate from OCIE's training branch, SEC has a central training center
called SEC University that oversees the agency's training programs.2 SEC
University uses an electronic database to track training received by SEC
staff. According to SEC officials, the database has a number of weaknesses
that limits its usefulness in helping SEC to track and assess the training
received by examiners. For example, the database cannot be used to
generate reports on which examiners have taken or not taken a particular
course. Also, the database is not directly accessible to examiners or
their supervisors and, thus, does not allow them to review their training
records or enter external training they may have taken. Because of these
weaknesses, OCIE's training branch uses training rosters as needed to
manually track which examiners have taken particular courses. SEC training
staff said that they are requesting that the agency purchase a learning
management system that would better enable it, including OCIE, to track
and assess all training and other developmental opportunities. According
to one of the officials, the initiative is currently tabled and may or may
not receive funding this year.
2As part of its 2004-2009 Strategic Plan, SEC is implementing SEC
University-a comprehensive redesign of the agency's training and
orientation programs-to help the agency develop and reinforce a strong
operating culture, enhance employee performance, and broaden staff
knowledge.
Appendix II Securities and Exchange Commission's (SEC) Broker-Dealer
Examination Guidance and Training
Despite challenges in its ability to track training in an automated way,
OCIE takes some steps to evaluate the training needs of its examiners. It
gathers and evaluates training participants' reactions to and satisfaction
with training programs and uses that information to decide on what
training to offer in the future. Training branch staff told us that at the
end of each course, they hand out course evaluation forms to participants.
These forms include closed-ended questions about the extent to which
participants found the course helpful and open-ended questions about what
additional training needs they have. The training branch uses the
information to improve individual classes and the program as a whole. In
addition, training staff attend monthly meetings with management and staff
from all field offices, in part, to identify training needs and
opportunities, and they also attend yearly meetings with examination
program managers to discuss the examiners' training needs.
Appendix III
Comments from the Securities and Exchange Commission
Appendix III Comments from the Securities and Exchange Commission Appendix
III Comments from the Securities and Exchange Commission Appendix III
Comments from the Securities and Exchange Commission
Appendix IV
GAO Contact and Staff Acknowledgments
GAO Contact Richard J. Hillman (202) 512-8678
Staff In addition to the contact named above, John Wanska, Randall
Fasnacht, Joel Grossman, Christine Houle, Marc Molino, Wesley Phillips,
David
Acknowledgments Pittman, Paul Thompson, Richard Tsuhara, and Mijo Vodopic
made key contributions to this report.
GAO's Mission
Obtaining Copies of GAO Reports and Testimony
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting its
constitutional responsibilities and to help improve the performance and
accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents at no cost
is through GAO's Web site (www.gao.gov). Each weekday, GAO posts newly
released reports, testimony, and correspondence on its Web site. To have
GAO e-mail you a list of newly posted products every afternoon, go to
www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out
to the Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are discounted 25
percent. Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
To Report Fraud, Contact:
Waste, and Abuse in Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: [email protected] Programs Automated answering system: (800)
424-5454 or (202) 512-7470
Gloria Jarmon, Managing Director, [email protected] (202)
512-4400Congressional U.S. Government Accountability Office, 441 G Street
NW, Room 7125 Relations Washington, D.C. 20548
Public Affairs Paul Anderson, Managing Director, [email protected] (202)
512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
PRINTED ON RECYCLED PAPER
*** End of document. ***