Financial Markets: Stronger System Controls and Oversight Needed to
Prevent NASD Computer Outages (Letter Report, 12/21/94, GAO/AIMD-95-22).

Outages experienced by the National Association of Securities Dealers'
(NASD) automated quotation and trading systems during July and August
1994 were caused by unrelated software and hardware malfunctions.  These
outages had little effect on individual investors and derivatives
markets but hampered the ability of broker-dealers to perform the best
and the most efficient trades. Although NASD takes the reliability of
its systems very seriously, these recent outages and malfunctions point
to areas, such as testing, in which further improvement is needed to
prevent recurrences.  In addition, although NASD has a separate backup
computer facility in case of emergencies, control weaknesses at this
facility and in NASD's contingency and disaster plan could make it hard
for NASD to recover quickly when emergencies occur.  Finally, NASD's
oversight of systems is limited by the fact that its internal audit
function generally does not include the review of market systems in the
scope of its work.  Compounding these problems is the fact that although
the Securities and Exchange Commission has strengthened oversight of
market automation in such areas as contingency planning, gaps exist in
its oversight program.

--------------------------- Indexing Terms -----------------------------

     TITLE:  Financial Markets: Stronger System Controls and Oversight 
             Needed to Prevent NASD Computer Outages
      DATE:  12/21/94
   SUBJECT:  Securities regulation
             Brokerage industry
             Internal audits
             Independent regulatory commissions
             Systems management
             Emergency preparedness
             Computer software
             Computer security
             Information systems
IDENTIFIER:  National Association of Securities Dealers Automated 
             Quotation System
*   This file contains an ASCII representation of the text of a GAO           *
*   report.  Delineations within the text indicating chapter titles,          *
*   headings, and bullets are preserved.  Major divisions and subdivisions    *
*   of the text, such as Chapters, Sections, and Appendixes, are              *
*   identified by double and single lines.  The numbers on the right end      *
*   of these lines indicate the position of each of the subsections in the    *
*   document outline.  These numbers do NOT correspond with the page          *
*   numbers of the printed product.                                           *
*                                                                             *
*   No attempt has been made to display graphic images, although figure       *
*   captions are reproduced. Tables are included, but may not resemble        *
*   those in the printed version.                                             *
*                                                                             *
*   A printed copy of this report may be obtained from the GAO Document       *
*   Distribution Facility by calling (202) 512-6000, by faxing your           *
*   request to (301) 258-4066, or by writing to P.O. Box 6015,                *
*   Gaithersburg, MD 20884-6015. We are unable to accept electronic orders    *
*   for printed documents at this time.                                       *

========================================================================== COVER

Report to the Chairman and Ranking Minority Member, Subcommittee on
Telecommunications and Finance, Committee on Energy and Commerce, House of

December 1994



NASDAQ Controls and Oversight

========================================================================= ABBREV

  NASD - National Association of Securities Dealers
  NASDAQ - National Association of Securities Dealers Automated Quotation and
     Trading Systems
  SEC - the Securities and Exchange Commission

========================================================================= LETTER


December 21, 1994

The Honorable Edward J.  Markey
The Honorable Jack Fields
Ranking Minority Member
Subcommittee on Telecommunications
 and Finance
Committee on Energy and Commerce
House of Representatives

This report responds to your August 2, 1994, letter requesting that we review
recent outages experienced by the National Association of Securities Dealers
(NASD) automated quotation and trading systems--commonly called NASDAQ. 
Specifically, in your letter and in subsequent meetings with your office, you
asked us to determine (1) the nature and causes of the outages of July 14 and
15, and August 1, 1994, (2) the impact of the outages on market participants,
(3) the adequacy of NASD's approach to respond to contingencies and disasters,
(4) how well NASD oversees its automated systems and facilities, and (5) how
well the Securities and Exchange Commission (SEC) is ensuring that the
securities markets are prepared for contingencies and disasters. 

---------------------------------------------------------------------- Letter :1

The NASDAQ system outages on July 14 and 15, and August 1, 1994, were caused by
unrelated software and hardware malfunctions.  These outages had limited impact
on individual investors and derivatives markets but hampered the ability of
broker-dealers to perform best and efficient trade executions.  While NASD
takes the reliability of its systems very seriously, these recent outages and
associated malfunctions point to areas, such as testing, where further
improvement is needed to guard against the risk of recurrence.  In addition,
while NASD has a separate, backup computer facility in case of contingencies,
control weaknesses at this facility and in NASD's contingency and disaster plan
could make it difficult for NASD to recover quickly when exigencies occur. 
Finally, NASD's oversight of systems is limited by the fact that its internal
audit function generally does not include the review of market systems in the
scope of its work. 

Compounding these problems is the fact that while SEC has strengthened
oversight of market automation in such areas as contingency planning, gaps
exist in its oversight program.  For example, SEC does not always follow up to
ensure auditors' recommendations are carried out.  Until SEC fills these gaps,
it cannot ensure that it is adequately overseeing the rapid growth of
automation in the securities industry. 

---------------------------------------------------------------------- Letter :2

Established in 1939, NASD regulates (1) over-the-counter securities trading
(that is, trading that does not occur on the floor of a stock exchange) and (2)
all brokers and dealers conducting securities business with the public.  NASD
owns and operates NASDAQ, a computerized communication system that provides
quotation information on and facilitates trade executions for 5,700 securities. 
Implemented in 1971, NASDAQ links a nationwide network of about 500 brokerage
firms, called market makers.  These firms maintain inventories of securities
which they buy from or sell to investors. 

During 1993, 66.5 billion shares of stock--totaling $1.35 trillion--were traded
in this market.  These volumes represent 43.6 percent of the total shares
traded on U.S.  stock markets, or about 32.6 percent of the total dollar value

NASD's headquarters is located in Washington, D.C.  The Association's automated
quotation and trading systems are located in and operated from its primary data
processing facility in Trumbull, Connecticut.  Its backup systems are located
at NASD's data processing facility in Rockville, Maryland, which also houses
automated administrative systems such as payroll, personnel, and market

The U.S.  securities markets are primarily governed by self-regulatory
organizations, such as NASD, which, in turn, are overseen by SEC.  While
self-regulatory organizations are responsible for maintaining smooth and
dependable operations with their automated systems, SEC is responsible for
overseeing overall market operations, including systems used to support such

---------------------------------------------------------------------- Letter :3

To determine the nature and causes of the outages and to better learn how NASD
develops, tests, and operates systems, we interviewed NASD senior officials,
including the Executive Vice President and Chief Technology Officer, the Senior
Vice President for Production Services, the Vice President for Computer
Operations, the Director for Quality Assurance, and the Director for
Performance Measurement.  In addition, these officials provided us with a
minute-by-minute chronology of events as they occurred on July 14 and 15 and
August 1.  We also obtained and reviewed NASD's policies and procedures for
quality assurance and stress testing.  Finally, we visited the primary data
processing facility in Trumbull, Connecticut, to observe the systems and the
controls used to safeguard them. 

We determined the impact of the recent NASD system outages on three categories
of market participants--market makers, derivatives markets, and individual
investors.  For the market makers, we used a structured questionnaire to
collect information from the top 12 market makers--Merrill Lynch; Smith Barney
Shearson; Herzog, Heine, Geduld; Mayer & Schweitzer; Troster Singer
Corporation; Goldman, Sachs & Company; Lehman Brothers; Morgan Stanley &
Company; Bear, Stearns & Company; The First Boston Corporation; PaineWebber;
and Sherwood Securities Corporation.  Together, these 12 represent over 50
percent of NASD's total trading volume. 

Our questionnaire included inquiries on how the outages impacted the market
makers' ability to obtain information and execute trades, as well as questions
on the impact of the outages on confidence in NASD systems and the market, and
on future participation in this stock market.  We met with six of the market
makers and mailed the questionnaire to the others. 

For the derivatives markets, we interviewed NASD officials including the Chief
Operating Officer, Chief Technology Officer, and Chief Economist, as well as
senior officials from the Chicago Board Options Exchange and the Chicago
Mercantile Exchange. 

Finally, to assess the impact of the outages on individual investors, we
interviewed market makers, senior NASD officials, and officials from the
National Association of Investors Corporation and the American Association of
Individual Investors--representing about 440,000 members combined. 

In assessing the adequacy of NASD's plans to respond to contingencies and
disasters, we conducted a walk-through of NASD's backup facility in Rockville,
Maryland.  We also interviewed those NASD officials responsible for preparing,
maintaining, and testing the Association's contingency and disaster recovery
plan.  In addition, we reviewed NASD's contingency and disaster recovery plan
and processes, including examinations of the 1993-94 test schedules and

To determine how well NASD oversees its automated market systems and
facilities, we examined the role of NASD's Internal Review office and discussed
the work it has done in the past and is planning to undertake in the future. 
We interviewed the Director of Internal Review and reviewed position
descriptions for the auditors who review NASD's systems and computer
facilities.  We also obtained and examined Internal Review's audit plan
detailing the scope of work to be performed through April 1995.  Finally, we
interviewed systems managers at both the primary and backup computing
facilities to determine the extent of their involvement with Internal Review. 

To determine how SEC generally oversees markets' preparedness for contingencies
and disasters, we interviewed senior officials in SEC's Division of Market
Regulation and obtained a chronology of events and supporting documentation
regarding the Commission's role and response to the NASD outages experienced
during July 14 and 15 and August 1.  In addition, we reviewed SEC's automation
review policy, the Commission's report of its most recent inspection at NASD
which occurred in 1992, and the audit report of the most recent review of
automated NASD systems conducted by an independent public accountant in 1992. 

We conducted our review from August through October 1994, in accordance with
generally accepted government auditing standards.  We discussed the contents of
this report with senior officials from NASD and SEC's Division of Market
Regulation and incorporated their comments where appropriate. 

---------------------------------------------------------------------- Letter :4

The system outages experienced by NASD in July and August were due to
malfunctioning software and hardware.  Specifically, on July 14, new
communications software being implemented as part of NASD's efforts to upgrade
its system did not operate as intended and caused the system to fail.  When
NASD staff restarted the system, the communications software experienced
additional problems. 

Consequently, NASD shut down the system and reconfigured it to use the old
communications software.  NASD operated its system this way for the remainder
of the trading day with only minor problems.  In total, the outages caused the
system to be down for about 14 minutes.  According to NASD systems officials,
they corrected one problem with the new communications software that evening. 
These officials also told us that they disabled a function of the new
communications software that was causing a second problem that could not be
fixed immediately, and reconfigured the system to use the old software for this

On July 15, before the normal market opening (9:30 a.m.  EST), the system's
response time slowed to unacceptable levels while processing routine tasks,
prompting NASD to delay opening the market.  At about 11:00 a.m., NASD
diagnosed the problem as a faulty hardware component (used to manage disk
access and storage devices), took it off-line, and opened the market at 11:55
a.m., approximately 2-1/2 hours late.  According to NASD systems officials, it
took them about 1-1/2 hours after normal opening time to locate this problem
because their focus was on the new software, while the problem was actually
caused by an intermittent hardware failure. 

After the market opened, NASD then opened a market function for exchange-listed
securities--the Consolidated Quotation Service.  Because this function had been
closed for the morning, the transaction rate surged.  Software controls in the
communications software that were designed to manage (limit) the number of
transactions the system would accept did not fully protect the system from this
surge and transaction backlogs began to build, resulting in the system's
response time increasing.  NASD responded by turning off selected automated
services to reduce the processing workload.  The system functioned with only
minor problems for the rest of the trading day. 

The outage on August 1, which lasted 34 minutes, was caused by a faulty circuit
board in NASD's backup electrical system.  The backup electrical system, which
consists of commercial-grade batteries and generators owned and operated by
NASD, was activated because the power from the local utility company dipped to
an unacceptable level.  As designed, the backup battery system operated until
the facility could be switched over to the backup generators; however, during
the switchover, the circuit board responsible for monitoring the conversion
malfunctioned.  This resulted in a total loss of power to the data center.  At
this point, NASD switched operations to the backup data processing facility in
Maryland and continued operations for the rest of the trading day. 

To address this problem, NASD (1) replaced the circuit board, (2) had the
contractor who supplied the backup electrical system determine why the board
malfunctioned, (3) is considering purchasing a second backup electrical system
of batteries, generators, and circuit board to supplement its existing backup
electrical system, and (4) hired a contractor to assess other single points of
failure in the backup electrical system. 

---------------------------------------------------------------------- Letter :5

The malfunctions that caused the July outages might have been avoided had NASD
more thoroughly tested its software.  Testing systems to assess their ability
to operate as intended and process unusually large workloads--commonly referred
to as quality assurance and stress testing, respectively--helps identify and
correct system weaknesses before they cause data processing disruptions in a
live operating environment. 

NASD performs quality assurance and stress tests on its systems.  For instance,
quality assurance personnel test software to determine whether it meets
established business requirements.  However, NASD's quality assurance testing
was limited in scope.  Specifically, quality assurance did not (1) test all
requirements and (2) verify that the system would not operate in inappropriate
ways.  For example, one requirement of NASD's communications software was to
limit the total number of transactions the system could accept into its
processing queue; however, quality assurance did not test this software
function.  On July 15 when NASD opened the market, the system accepted more
transactions than it was designed to handle without having to re-queue
transactions, which slowed system processing speed to unacceptable levels. 

In addition, NASD has a Performance Measurement Unit responsible for stress
testing.  This unit tests systems to determine how they behave under high
workloads and demanding conditions.  However, these tests were also limited in
scope.  For example, NASD did not test the system with sufficient volume to
drive the system beyond the point where it begins to re-queue transactions, nor
with a heavy backlog of transactions, such as occurred on July 15. 

NASD systems officials said that their quality assurance testing program is
rigorous enough to catch most problems, but acknowledged that problems
sometimes can go undetected.  In addition, while these officials said that
their stress testing is adequate, they also agreed that their stress tests
could be expanded to include transaction backlog conditions similar to those
experienced on July 15. 

We also discussed with systems officials why they installed new communications
software on Friday, July 15, a "double witching" day.  On such Fridays, the
market is potentially volatile because options and other related financial
instruments expire and market participants may need to buy or sell stocks to
meet obligations.  Installing new software on such potentially volatile days
increases the risk that system problems could worsen market conditions and
therefore, should be avoided.  SEC has also found this practice to be
undesirable and has recommended since the July and August outages that NASD
avoid installing new software on such days and that systems managers coordinate
changes with top management at NASD headquarters who are knowledgeable about
market conditions. 

NASD systems officials told us that it is NASD's policy not to install system
changes on Fridays.  While the installation of the new communications software
carried over to July 15, due to the problems on July 14, these officials
believed that there was minimal risk of a malfunction because (1) the software
had been tested, (2) it was installed in a phased approach over a 2-week
period, and (3) they were confident in the systems and the personnel who
developed and operate the software.  Additionally, the systems officials told
us that they do discuss system changes with business managers at the primary

During the course of our work, NASD systems managers began to inform top
business managers via electronic mail of all upcoming system changes and
installation schedules.  This notwithstanding, unless NASD strictly adheres to
its policy of avoiding the installation of new software changes on potentially
volatile days, it risks having system malfunctions exacerbate market
conditions.  This risk could be made greater by the fact that NASD will be
making numerous changes as it upgrades its system. 

---------------------------------------------------------------------- Letter :6

The recent outages experienced by NASD had varying effects on market
participants and derivatives markets.  For example, individual investors were
not significantly affected and did not report complaints regarding the outages. 
Conversely, market makers were impacted because they did not have the benefit
of NASD's automated quotation and trading system to conduct business.  In
general, the impact was greatest on July 15 when the system was out for 2-1/2
hours.  Nonetheless, 213 million shares--79 percent of the average daily volume
for July--were traded that day. 

-------------------------------------------------------------------- Letter :6.1

Market makers surveyed characterized the impact of the outages as being very
great because they could not obtain updated price quotations from NASD. 
Without this information, market makers were unable to facilitate the best and
efficient execution of trades.  Market makers stated that the July 15 outage
was particularly severe because they could not get quote information for the
first 2-1/2 hours.  Their frustration was heightened by the fact that they were
uncertain when the market would reopen. 

However, to serve customers who were willing to buy or sell without updated
quotes, market makers relied on other means, such as using broker-dealer owned
trading systems (Instinet,\1 for example) to execute trades.  Market makers
generally told us that the 2-1/2-hour delay of July 15 resulted in lost
opportunities to do business.  In addition, seven market makers stated that
they lost revenue they collect for executing trades on a normal day due to the
outage.  Of these seven, three estimated that they lost 20 to 25 percent of
such fees, while the remaining four firms had not estimated the extent of their
monetary losses attributable to the outage. 

\1 Instinet is a network of computer terminals that facilitates the trading
process by matching buyers with sellers.  Instinet is registered with the SEC
as a broker-dealer. 

-------------------------------------------------------------------- Letter :6.2

According to securities industry officials we interviewed, the outages had
little impact on individual investors, who hold about 55 percent of all NASD
market stocks.  First, officials from two nonprofit associations, representing
about 440,000 individual investors, told us that while their members generally
report events that affect them, no complaints were reported regarding the NASD
outages.  One official stated that individual investors tend to make long-term
investments so that outages of 1 day would probably not affect them. 

Second, according to NASD, the majority of individual investors who participate
in the stock markets do so through mutual funds, which generally price their
funds using end-of-day stock quotes.  Since NASD provided end-of-day stock
quotes on the days the system experienced outages, mutual funds, and thus most
individual investors, were unaffected.  Finally, one of the market makers whose
business caters to individual investors told us that all of its trades were
executed, although not immediately, given the unavailability of updated quotes
to guarantee best price execution of trades. 

-------------------------------------------------------------------- Letter :6.3

The derivatives markets--such as the options and futures markets--trade
products that derive their value from NASD and other markets' stocks.  The
Chicago Board Options Exchange, which trades the largest number of NASD stock
options, had to stop trading these instruments on July 15 and August 1 because
quotation information, which is used to derive the price of options, was not
available.  In addition, according to options exchange officials, when NASD
opened its market 2-1/2 hours later on July 15 and began transmitting quote
information, the exchange encountered a large volume of orders that had to be
processed in a relatively short time frame. 

We also interviewed officials at the Chicago Mercantile Exchange, which trades
the largest number of futures on stock indices whose values are derived from
the value of stocks traded in NASD's market.  These officials stated that the
outages had no discernable impact on their trading operations because the
vendors who price the indices continued to do so using last available quote
information from NASD. 

---------------------------------------------------------------------- Letter :7

NASD has taken significant steps to prepare its systems for contingencies and
disasters.  It operates a backup computer facility to be used if there are
problems or outages at the primary computer site.  In addition, NASD has
prepared a detailed plan that identifies critical operations and the key
individuals responsible for carrying out specified procedures during
emergencies, such as power outages and natural disasters.  The Association also
conducts tests to gauge staff preparedness. 

However, there are management and control weaknesses in NASD's contingency and
disaster recovery activities.  While these weaknesses did not contribute to the
problems experienced on July 14 and 15 and August 1, they make NASD vulnerable
to problems should emergencies occur.  For example, the contingency and
disaster recovery plan is incomplete and out of date.  Certain contingency
scenarios have not yet been drafted and incorporated into the plan and names of
some emergency personnel, who are no longer in such positions, have not been
updated.  In addition, the plan does not clearly delineate who is responsible
for making systems decisions during contingencies and disasters.  During our
limited scope review at the backup site, we also identified certain internal
control weaknesses.  For example, the data center is located over a storage
room of paper products, posing a potential fire hazard. 

During the course of our work, we brought these management and control
weaknesses to NASD's attention and they attributed the weaknesses to oversights
on their part.  NASD agreed to correct them immediately. 

---------------------------------------------------------------------- Letter :8

We examined the role of NASD's internal audit function in identifying system
and control weaknesses.  We found that despite NASD's extensive reliance on
automated systems to accomplish its mission, until recently it had only one
auditor with computer expertise reviewing automated systems.  In addition, the
scope of the auditor's work was generally limited to reviewing administrative
systems located at the backup site. 

According to NASD officials, while internal audit's focus has been on
administrative systems, it has performed some work on market-related systems. 
Specifically, internal audit was involved during the development and
implementation of the Fixed Income Pricing System and has reviewed Small Order
Execution System outages.  In addition, the internal audit function was
established 2 years ago and is still in the process of establishing a program
to ensure adequate audit coverage.  Finally, internal audit focused its work on
administrative systems at the backup site because these systems were judged to
be more at risk than the market systems.  This decision was based in part on
the fact that the market systems had been reviewed by an external auditor in
1992 and internal audit believed it could rely on this work. 

Because regular external and internal reviews are complementary management
control practices used to oversee the use of automated systems, reviews by an
external auditor are not a complete substitute for the day-to-day audit
coverage provided by internal audit.  Recognizing this, NASD officials stated
that they (1) recently hired a second internal auditor with computer expertise,
(2) plan to expand coverage of market systems in the audit work plan for the
upcoming year, and (3) are discussing with SEC the frequency of external

---------------------------------------------------------------------- Letter :9

Our past reviews of automated stock market systems have identified the need for
SEC to establish the capability to address such technical issues as contingency
and disaster recovery planning.\2 SEC has subsequently taken steps to improve
its oversight of the markets' use of automation.  For example, the Commission
established an Office of Automation and International Markets and issued an
automation review policy that encourages the securities markets to perform
independent reviews of their automated systems and operations in such areas as
contingency and disaster planning.\3 The Commission planned to measure
compliance with the policy by conducting inspections on a periodic basis. 

However, gaps exist in SEC's oversight program.  First, it is unclear how often
SEC expects the markets to perform independent automation reviews because the
Commission's policy does not state a specific frequency requirement.  For
example, the last such review at NASD was performed in 1992.  In addition, SEC
has not established how frequently it will perform its inspections.  In
practice, its inspections have been limited to about every 3 years because the
Commission only has four technical staff members capable of conducting this
work.  Further, the scope of SEC's inspections has been limited to reviewing
other auditors' work, rather than conducting first-hand reviews of system

SEC also has not always followed up to ensure auditors' recommendations are
carried out.  For example, in 1992, NASD had an external auditor review its
systems.  This auditor identified a serious control weakness in the way NASD
modified software on the production system during emergencies.  As part of its
1992 inspection, SEC reviewed the audit report, agreed with the auditor's
finding, and recommended that NASD take countermeasures to mitigate this
weakness.  However, at the time of our work, NASD had not yet corrected this
weakness.  While SEC officials told us that it is their goal to discuss all
unresolved audit findings with NASD and the other markets during periodic
briefings on market automation developments, these officials acknowledged that
they had not taken action to ensure NASD had implemented the recommendation. 

Officials in SEC's Market Regulation Division stated that the automation review
policy is still evolving and for this reason, they have not yet finalized all
of its requirements.  For example, SEC staff are currently negotiating with the
securities markets to determine how often the external reviews will be
performed and expect to reach agreement soon.  In addition, the Market
Regulation officials advised us that they are now including first-hand reviews
of selected system safeguards as part of their inspections. 

However, these officials told us that they are unclear what the optimal
frequency for inspections should be.  They also told us that SEC would be
unable to conduct more frequent inspections and other oversight activities
because it only has four computer specialists to oversee market automation at
over 19 markets and other related organizations, such as clearing agencies and
depositories.  In addition, they said that hiring additional staff has been
deferred by the Division due to other priorities.  On November 17, 1994, these
officials told us that they had recently received authority to hire two
additional technical staff and were in the process of advertising the
positions.  Until SEC determines the appropriate frequency of inspections, it
cannot be sure it has the correct number of technical staff to oversee
automated market systems. 

\2 Financial Markets:  Computer Security Controls at Five Stock Exchanges Need
Strengthening (GAO/IMTEC-91-56, August 28, 1991) and Financial Markets:  Active
Oversight of Market Automation by SEC and CFTC Needed (GAO/IMTEC-91-21, April
2, 1991). 

\3 Securities and Exchange Commission Release No.  34-27445, 54 Fed.  Reg. 
48703 (1989), and No.  34-29185, 56 Fed.  Reg.  22490 (1991). 

--------------------------------------------------------------------- Letter :10

NASD is aware of the importance of maintaining reliable systems and providing
backup in the case of emergencies, and is taking action to correct the
weaknesses identified in this report.  Addressing these weaknesses will lower
the risk of future outages and enable NASD to respond more quickly and
appropriately to future contingencies and disasters. 

While SEC has made progress in strengthening oversight of market automation,
gaps still exist in its oversight program.  Until SEC fills these gaps, the
Commission cannot ensure that it is adequately overseeing the rapid growth of
automation in the securities industry. 

--------------------------------------------------------------------- Letter :11

We recommend that the Chairman, SEC, ensure that NASD

expands testing processes for its market systems to better detect problems;

performs a thorough assessment of its existing systems environment to identify

avoids implementing software changes on potentially volatile trading days;

corrects weaknesses in its contingency and disaster recovery plan and backup
data processing facility; and

regularly schedules and conducts audits of its market systems. 

In addition, we recommend that SEC's Chairman (1) reach agreement with
securities markets on the frequency of independent reviews, (2) determine SEC
inspection frequency needed to ensure adequate oversight of market systems and
facilities, and (3) follow up on systems auditors' recommendations and ensure
that the recommendations are adequately resolved.  Given that the gaps in
Commission oversight are attributable in part to a lack of technical staff, the
Chairman should also determine the number of staff needed to adequately oversee
the rapid growth of market automation and report this information to the
Commission's congressional appropriations and authorization committees in time
for consideration in next year's budget. 

--------------------------------------------------------------------- Letter :12

We discussed the contents of this report with senior officials from NASD and
SEC's Division of Market Regulation.  We incorporated their comments where
appropriate.  SEC officials agreed with our findings, conclusions, and
recommendations.  Except as noted below, NASD officials also agreed with the

NASD officials disagreed with our characterization that their software testing
approach is limited or incomplete in scope.  They said that they have adopted a
rigorous approach to testing.  In addition, NASD said that while its approach
may differ from other approaches, it is successful, as demonstrated by the
significant number of changes that have been introduced over the years without
problems.  Nevertheless, NASD officials told us that as an act of caution, they
will engage an independent reviewer to assess the testing function and will
respond appropriately to the reviewer's recommendations.  We believe that this
is a prudent step. 

------------------------------------------------------------------- Letter :12.1

We are sending copies of this report to interested congressional committees,
the Chairman of the Securities and Exchange Commission, the President and Chief
Executive Officer of the National Association of Securities Dealers, and to
other interested parties.  Copies will also be made available to others upon
request.  Please call me at (202) 512-6418 if you or your staffs have questions
about this report.  Other major contributors are listed in appendix I. 

Hazel E.  Edwards
Director, Information Resources Management/
 General Government Issues

===================================================================== Appendix I

------------------------------------------------------------------- Appendix I:1

Linda D.  Koontz, Associate Director
Gary N.  Mountjoy, Senior Evaluator
William D.  Hadesty, Technical Assistant Director
Kevin G.  McCarthy, Senior Evaluator
Sabine R.  Paul, Senior Information Systems Analyst

*** End of document. ***