Report 81 09 Evaluating Expert Systems . Stanford Edward H. Jul 1981

This paper is the author's contribution to Chapter 6. in the volume BU:LDIN3 EXPERT SYSTEMS, edited by R. Hayes-Roth, D. Lenat, and D. Waterman:4 The full article is entitled "Evaluation of expert systems: issues and case studies", and is authored by J. Gaschnig, P. Klahr, H. Pople, E. Shortliffe. The volume is the result of a Workshop on Expert Systems held in San Diego in August 1980 and sponsored by the Rand Corporation, ARPA, and the NSF. Parts of Chapters 7 & 8. Reprinted with permission. In this section we define ma,ly of the parameters that determine an appropriate design for an evaluation experiment. When one examines the literature on computer performance evaluation, it is clear that the term is used with a variety of meanings depending upon the;.-c.3pective of tne authors. Each tends to focus on the specific performance issues that have been most central to the design of the system in question. Utner aspects warranting formal evaluation are often ignored.