ONCOCIN's task is to assist oncologists with of computer-based decision aids by physicians. It the management of cancer patients undergoing was clear that diagnostic accuracy was not enough structured (protocol) treatment for their disease.

'ince much of cancer chemotherapy research this reasoning system with a high speed interface today is dependent upon rigorously designed and to help make the system acceptable to oncologists.

During the quarter century since the birth of "artificial intelligence" (Al), attempts to develop symbolic models of human reasoning processes have been a major focus of the ongoing research. It is only in the last half-dozen years or so, however, that several related Al research themes have come together in the formation of what is now known as "expert systems researoh" CI], In this brief paper I would 1.ke to review the key aspects of A: and expert syste-.s

Reserved Function Names 109 Appendix Ili.

Programs Based on the Blackboard Model 8 2.2.1.

"Learning", the incorporation of additional knowledge into expert systems, ranges from human data entry (learning by being told), to data gathering (learning by observing), to full-fledged theory formation (learning by discovery). One important kind of learning is the compiling of descriptive meta-knowledge into strategic form, recasting it into a form in which it can be evaluated efficiently. Much of what we earlier called strategic meta-knowledge may be seen to be operationalized "caches" of descriptive or systemic meta-knowledge. For instance, R9 and R10 can be converted from systemic to strategic form by slight rewordings of their actions.

Search 4. Methods for assigning credit and blame--obtaining a local Performance Standard

This manual describes a domain-independent system, called EMYCIN, for constructing one class of expert computer programs: rule-based consultants. The resulting programs use knowledge specific to a problem domain to provide consultative advice to a client. The system-building tool, EMYCIN, is based on the domain-independent core of the MYCIN program. Domain kno,./ledge is represented in EMYCIN systems primarily as production rules, which are applied by a goal-directed backwardchaining control structure. Rules and consultation data may have associated measures of certainty, and incomplete data entry is allowed. The system includes an explanation facility that can display the line of reasoning followed by the consultation program, and answer questions from the client about the contents of its knowledge base. To aid the system designer in producing a knowledge base for a domain quickly and accurately, EMYCIN provides the following features: (1) a terse.

This paper presents an "expert system" devised to aid organic chemists in determining the structure (i.e. the arrangement of atoms and bonds) of newly isolated, naturally occurring compounds. The system exploits a data base of rules for analyzing.013 C nuclear magnetic resonance spectra" [2. 13 C spectroscopy is a relatively new technique; only in the last ten years, at most, has this analytic approach been practical. Currently, no general interpretive schemes exist for thoroughly analyzing 13C spectra. A few limited classes of compounds have been investigated in detail, and highly specific schemes for interpreting their 13C spectra have been developed. The only generally applicabl- "interpretation rules" rely on correlation J of spectral and substructural features.

This paper is the author's contribution to Chapter 6. in the volume BU:LDIN3 EXPERT SYSTEMS, edited by R. Hayes-Roth, D. Lenat, and D. Waterman:4 The full article is entitled "Evaluation of expert systems: issues and case studies", and is authored by J. Gaschnig, P. Klahr, H. Pople, E. Shortliffe. The volume is the result of a Workshop on Expert Systems held in San Diego in August 1980 and sponsored by the Rand Corporation, ARPA, and the NSF. Parts of Chapters 7 & 8. Reprinted with permission. In this section we define ma,ly of the parameters that determine an appropriate design for an evaluation experiment. When one examines the literature on computer performance evaluation, it is clear that the term is used with a variety of meanings depending upon the;.-c.3pective of tne authors. Each tends to focus on the specific performance issues that have been most central to the design of the system in question. Utner aspects warranting formal evaluation are often ignored.