Towards Validation of Rule-Based Systems - The Loop is Closed

AAAI Conferences

Thus, the only way to validate these systems is to confront the system with representative test cases and to compare the system's answers with the answers of human experts.


Utilizing Validation Experience for System Validation

AAAI Conferences

This paper adopts the idea of using knowledge gained by various validation sessions over time with a validation technology developed previously. The work is designed to reduce the human involvement needed to apply this technology. It introduces the reuse of test cases with the "best solution", discovered in previous validation sessions. This reduces the number of test cases to be solved and rated by the experts within the validation process. By reducing the workload of the involved experts, the costs of validation can be reduced. Moreover, this approach may compensate for possible shortages of expertise available for the validation process.


The Rule Retranslation Problem and the Validation Interface

AAAI Conferences

Current rule base maintenance is wasting refinement and inference performance. There are only few maintenance concepts, which enjoy both (1) formal rule refinement and (2) utilizing topical knowledge provided by experts within the refinement process. The current state of the art in rule base validation and refinement reveals that there is no generic validation interface and no optimal rule trace refinement. This paper characterizes two different retranslation approaches for reduced rule bases and proposes a two-step validation process, which combines a case-based approach with a rule trace validation approach.


Compiling Experience into Knowledge

AAAI Conferences

Typical application fields of Knowledge Based Systems are a usually characterized by having human expertise as the only one source to specify their desired behavior. Their design, evaluation and refinement has to make effective use of this valuable source. After sketching the concept of collecting validation experience in a Validation Knowledge Base (VKB), the paper introduces an estimation of the significance of the cases collected in the VKB. A high significance signalizes that a VKB should not longer serve as a case-based source of external knowledge, but compiled into the Knowledge Base instead. A technology to compile well-selected cases into the Knowledge Base of the system under evaluation is shown.


Flairs04-060.pdf

AAAI Conferences

Because of the character of their typical application fields, intelligent systems are validated and refined on the basis of human expertise. Experts have different beliefs, experiences, learning capabilities and are not free of mistakes. Their opinions about the desired system's behavior differ from each other and change over time. Their opinions differ from their previous ones, even in the same context, as a result of misinterpretations, mistakes or new insights. Furthermore, experts are too busy and too expensive to spend that much time for system validation and adjustment. Thus, the experts' workload for system validation is a serious issue. To make validation results less dependent on the experts' opinions and to decrease the workload of the experts, the importance of storing and using historical validation results