Interactive complex task teaching system
First Claim
1. An interactive method for learning and executing executable tasks using language and demonstration inputs from a user, comprising:
- a. providing a computational device including a graphical user interface (GUI);
b. providing software running on said computational device and supported GUI-based interaction with said user;
c. wherein said user performs tasks using the GUId. wherein said software includes a natural dialog-based interface whereby said user can communicate with said software using natural dialog-based language;
e. for each of said executable tasks, recognizing said user'"'"'s overall intent;
f. for each of said executable tasks, identifying a plurality of steps needed to complete said task;
g. for each of said plurality of steps, identifying and generalizing a step objective;
h. learning to execute each of said steps from demonstrations provided by said user;
i. providing incremental execution and interaction with said user using said natural dialog-based interface;
j. providing a database for storing a task definition for each of said tasks, wherein said task definition includes said steps comprising said task and said step objectives;
k. storing said task definitions in said database;
l. for each of said tasks, learning semantic characterization of said task for later retrieval from said database;
m. retrieving a particular task definition from said database using said semantic characterization; and
n. improving said task definition for each of said tasks through practice, with instruction being provided by said user.
1 Assignment
0 Petitions
Accused Products
Abstract
A system which allows a user to teach a computational device how to perform complex, repetitive tasks that the user usually would perform using the device'"'"'s graphical user interface (GUI) often but not limited to being a web browser. The system includes software running on a user'"'"'s computational device. The user “teaches” task steps by inputting natural language and demonstrating actions with the GUI. The system uses a semantic ontology and natural language processing to create an explicit representation of the task that is stored on the computer. After a complete task has been taught, the system is able to automatically execute the task in new situations. Because the task is represented in terms of the ontology and user'"'"'s intentions, the system is able to adapt to changes in the computer code while still pursuing the objectives taught by the user.
-
Citations
20 Claims
-
1. An interactive method for learning and executing executable tasks using language and demonstration inputs from a user, comprising:
-
a. providing a computational device including a graphical user interface (GUI); b. providing software running on said computational device and supported GUI-based interaction with said user; c. wherein said user performs tasks using the GUI d. wherein said software includes a natural dialog-based interface whereby said user can communicate with said software using natural dialog-based language; e. for each of said executable tasks, recognizing said user'"'"'s overall intent; f. for each of said executable tasks, identifying a plurality of steps needed to complete said task; g. for each of said plurality of steps, identifying and generalizing a step objective; h. learning to execute each of said steps from demonstrations provided by said user; i. providing incremental execution and interaction with said user using said natural dialog-based interface; j. providing a database for storing a task definition for each of said tasks, wherein said task definition includes said steps comprising said task and said step objectives; k. storing said task definitions in said database; l. for each of said tasks, learning semantic characterization of said task for later retrieval from said database; m. retrieving a particular task definition from said database using said semantic characterization; and n. improving said task definition for each of said tasks through practice, with instruction being provided by said user. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 15, 16, 17, 18, 19, 20)
-
Specification