Expert system tester
First Claim
1. A test system for testing an expert system having rules, said test system comprising:
- test storage means for storing a complete normal test set, separate from the rules, for testing the rules and a complete specialized test set, separate from the rules, for testing the rules, the normal test set comprising data values inside a diagnostic level which will not cause the expert system to perform a diagnosis and the specialized test set comprising data values which will cause the expert system to perform the diagnosis; and
testing means for testing all the expert system rules to automatically detect malfunctioning rules using the complete normal test set and the complete specialized test set to produce a complete output data set.
1 Assignment
0 Petitions
Accused Products
Abstract
The present invention is an expert system test system which has the capability of exercising all combinations of data input types using specialized data test sets. The specialized data test sets include a set of sensor values for each level of an operating system. All levels of diagnosis can be tested without testing every possible sensor value and combination of sensor values. The test sets can be combined to produce various orders of testing allowing complex relationships between sensors and rules to be tested. The system includes a test plan which is used by the testing system to iteratively apply the normal and specialized test data to the expert system. Changing the sensor values produces outputs for each iteration which are compared to expected results or to a baseline. Each test produces a log file which can be examined by the knowledge engineer. If exceptions are produced, the system produces an exception report which allows the knowledge engineer to determine whether the exception is abnormal or unexpected. An output log is also stored as historical data and used with future tests to obtain a complete test regression comparison using conventional database comparison tools. The system includes a test language which allows the user to formulate a test plan without the need for complicated programming associated therewith by designating the rulebase to be used, and by defining the order of the test and the tests performed therein, along with exceptions which include exceptions which include simple, set and sufficiency type exceptions.
30 Citations
15 Claims
-
1. A test system for testing an expert system having rules, said test system comprising:
-
test storage means for storing a complete normal test set, separate from the rules, for testing the rules and a complete specialized test set, separate from the rules, for testing the rules, the normal test set comprising data values inside a diagnostic level which will not cause the expert system to perform a diagnosis and the specialized test set comprising data values which will cause the expert system to perform the diagnosis; and testing means for testing all the expert system rules to automatically detect malfunctioning rules using the complete normal test set and the complete specialized test set to produce a complete output data set. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8)
-
-
9. A test system for testing an expert system for a power plant, the expert system having rules and said test system comprising:
-
test storage means for storing an array of test sets for testing the rules, the test sets including a complete normal sensor value set, separate from the rules, and complete specialized sensor value sets separate from the rules and including sensor failure, predictive and diagnostic values, the normal sensor value set comprising data values inside a diagnostic level which will normally not cause the expert system to perform a diagnosis and the specialized sensor value sets comprising data values which will normally cause the expert system to perform the diagnosis; testing means for testing all the expert system rules cyclically to automatically detect malfunctioning rules using the normal sensor value set and the specialized sensor value set while iteratively applying the specialized sensor value sets to produce a complete output data set, said testing means including; reading means for reading a test input in a test plan language and converting the test inputs into a test plan, the test plan specifying an order of the test and exceptions to be tested; exception means for comparing expert system outputs with the expected results and reporting when a match occurs; logging means for logging sensors tested, sensor data, malfunctions indicated, malfunction descriptions, the test plan and the test cases comprising the specialized sensor value sets; and comparison means for comparing logs before and after changes to the expert system rules.
-
-
10. A method, initiated by a user, of testing an expert system having rules, comprising the steps of:
-
(a) applying, by the computer, a complete normal test set to the rules of the expert system, the normal test set being separate from the rules and comprising normal data values inside a diagnostic level which will normally not cause the expert system to perform a diagnosis and; (b) applying, by the computer, a complete specialized test set to the rules of the expert system, the specialized test set being separate from the rules and comprising test data values which normally will cause the expert system to perform the diagnosis; and (c) recording, by the computer, abnormal rule executions produced during step (b). - View Dependent Claims (11, 12, 13, 14)
-
-
15. A method, initiated by a user, of testing an expert system for a power plant, comprising the steps of:
-
(a) reading, by the computer, test requirements input by a user specifying test type, test order and test exceptions; (b) converting, by the computer, the test requirements into a test plan; (c) initializing, by the computer, the expert system; (d) setting, by the computer, all sensor input values to normal values test values being separate from the rules and comprising data values inside a diagnostic level which will normally not cause the expert system to perform a diagnosis; (e) updating and firing rules of the expert system; (f) setting, by the computer, sensor values to test values responsive to the test plan, the test values being separate from the rules and comprising data values which will normally cause the expert system to perform the diagnosis; (g) updating and firing the rules of the expert system; (h) logging, by the computer, all outputs of the expert system; (i) comparing, by the computer, the outputs of the expert system to the test exceptions; (j) producing, by the computer, an exception report when a match occurs; (k) repeating steps (d)-(j) in accordance with the test plan for all test values; (l) performing steps (a)-(k) after the user makes changes to the rules in the expert system; and (m) comparing, by the computer, the logging outputs produced before and after the changes to the rules.
-
Specification