System and method for automatically verifying the performance of a virtual robot

• US 6,259,969 B1
• Filed: 02/03/1998
• Issued: 07/10/2001
• Est. Priority Date: 06/04/1997
• Status: Expired due to Term

• Current Assignee: Micro Focus LLC (Open Text Corporation)
• Original Assignee: NativeMinds, Inc. (Healthstream Incorporated)
• Inventors: Benson, Scott S., Tackett, Walter A.
• Primary Examiner(s): Gordon, Paul P.
• Application Number:
  US09/017,760
• Time in Patent Office: 1,253 Days
• Field of Search: 704/9, 702/182, 702/188, 714/46, 700/264, 700/9, 700/17, 700/86, 318/567
• US Class Current: 700/264
• CPC Class Codes: G06F 40/35 Discourse or dialogue representation
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/006 based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
• CPC Class Codes: Y10S 706/924 Medical

Abstract

In an automated interface program designed to interact and communicate with users, said program executing actions when a category among a predefined set of categories is activated, a method is disclosed for automatically verifying the performance of said program, the steps of said method comprising: specifying inputs under which the program should be tested; associating said inputs with conditions within categories in the program, each said condition comprising at least one response which could be given if said condition is satisfied; executing said program under at least one said input; determining whether the associated condition is satisfied upon said input; and determining whether the response associated with said condition is given upon said input. In another aspect of the present invention, the test inputs are embedded within the script itself, and specifically, within categories that can be automatically listed upon compilation of the script. Such list of test inputs can then be automatically executed to test the program. The execution of a test input can be used to check whether the test input activated the category in which the test input is embedded. The response given upon execution of a test input can then determine whether other categories are erroneously activated; or whether inputs, other than the test input, erroneously activate the category associated with the test input.

Claims

1. A method of verifying the performance of an automated interface program adapted for interacting with a user, the method comprising:

(a) defining within the automated interface program at least one category associated with actions executable by the automated interface program during interaction with a user, wherein the at least one category includes at least one condition;

(b) defining at least one example statement associated with the at least one category in the automated interface program, wherein the example statement includes at least one example input;

(c) selecting at least one example statement for examination, wherein the example statement selected for examination includes at least one example input;

(d) executing at least one category using at least one example input of the example statement selected for examination as an input, wherein the category is activated if the condition of the category is satisfied; and

(e) determining whether an activated category contains the example statement selected for examination.

2. The method of verifying the performance of an automated interface program as recited in claim 1, wherein the example input comprises natural language input.

3. The method of verifying the performance of an automated interface program as recited in claim 1, wherein the automated interface program further comprises a listing of all example inputs in the automated interface program.

4. The method of verifying the performance of an automated interface program as recited in claim 1, wherein a first example input included within an example statement may be associated with an equivalent example statement including one or more example inputs equivalent to the first example input.

5. The method of verifying the performance of an automated interface program as recited in claim 4, wherein an equivalent example input may comprise a context dependent input, wherein a context dependent input generates a match with the example input when the automated interface program is focused on the category associated with the example statement that is associated with the context dependent input.

6. The method of verifying the performance of an automated interface program as recited in claim 4, wherein the automated interface program further comprises a list of all equivalent example inputs in the automated interface program, wherein each equivalent example input is associated with an example input of an example statement associated with a category in the automated interface program.

7. The method of verifying the performance of an automated interface program as recited in claim 6, wherein the list of all equivalent example inputs in the automated interface program further includes context dependency information for each equivalent example input.

8. The method of verifying the performance of an automated interface program as recited in claim 1, further comprising an automated interface program testing process wherein one or more example inputs are examined during execution of the automated interface program.

9. The method of verifying the performance of an automated interface program as recited in claim 8, further comprising the step of defining one or more initial example inputs, wherein, during execution of the automated interface program testing process, an initial example input is examined prior to example inputs not defined as initial example inputs.

10. The method of verifying the performance of an automated interface program as recited in claim 9, wherein the initial example inputs are examined in a predefined order.

11. The method of verifying the performance of an automated interface program as recited in claim 8, further comprising saving a user state after executing an action associated with a condition if the condition matches with the example input, wherein the current program state of the automated interface program may be restored to the saved user state during the automated interface program testing process prior to examination of additional example inputs.

12. The method of verifying the performance of an automated interface program as recited in claim 1, further comprising the step of defining a sequence of one or more examples inputs based on a series of order-dependent interactions with the automated interface program.

13. The method of verifying the performance of an automated interface program as recited in claim 12 wherein a unique index is appended to each example input of the sequence appearing in example statements in the automated interface program.

14. The method of verifying the performance of an automated interface program as recited in claim 1, further comprising identifying a correct output state if the current example input matches the category and a condition associated with an example statement that matches the example input.

15. The method of verifying the performance of an automated interface program as recited in claim 1, further comprising the step of identifying a warning state if the current example input does not match the example statement associated with a category matched by the current example input.

16. The method of verifying the performance of an automated interface program as recited in claim 1, further comprising the step of identifying an incorrect output state if the current example input does not match the category associated with an example statement matched by the current example input.

17. A system for testing a program adapted for interacting with a user in natural language, the system comprising:

an interface for receiving natural language input from a user; and

a processor comprising a computer useable medium including computer readable program code embodied in said medium for interacting with a user in natural language, said computer readable program code including;

computer readable program code configured to define at least one category associated with actions executable by the automated interface program during interaction with a user, wherein the at least one category includes at least one condition;

computer readable program code configured to define at least one example statement associated with the at least one category, wherein the example statement includes at least one example input;

computer readable program code configured to select at least one example statement for examination, wherein the example statement selected for examination includes at least one example input;

computer readable program code configured to execute at least one category using the at least one example input of the example statement selected for examination as an input;

computer readable program code configured to activate the category if the condition of the category is satisfied; and

computer readable program code configured to determine whether an activated category contains the example statement selected for examination.

18. A computer program product for interacting with a user in natural language, said computer program product comprising:

computer readable program code configured to define within the automated interface program at least one category associated with actions executable by the automated interface program during interaction with a user, wherein said at least one category includes at least one condition;

computer readable program code configured to define at least one example statement associated with the at least one category, wherein the example statement includes at least one example input;

computer readable program code configured to select at least one example statement for examination, wherein the example statement selected for examination includes at least one example input;

computer readable program code configured to execute at least one category using the at least one example input of the example statement selected for examination as an input, wherein the category is activated if the condition of the category is satisfied; and

computer readable program code configured to determine whether an activated category contains the example statement selected for examination.

19. The computer program product for interacting with a user in natural language as recited in claim 18, wherein the at least one example input comprises natural language input.