×

Interactive complex task teaching system that allows for natural language input, recognizes a user's intent, and automatically performs tasks in document object model (DOM) nodes

  • US 7,983,997 B2
  • Filed: 11/02/2007
  • Issued: 07/19/2011
  • Est. Priority Date: 11/02/2007
  • Status: Active Grant
First Claim
Patent Images

1. An interactive method for learning and executing executable tasks using language and demonstration inputs from a user, comprising:

  • a. providing a computational device including a graphical user interface (GUI);

    b. providing software running on said computational device and supported GUI-based interaction with said user;

    c. wherein said user performs tasks using the GUId. wherein said software includes a natural dialog-based interface whereby said user can communicate with said software using natural dialog-based language;

    e. for each of said executable tasks, recognizing said user'"'"'s overall intent;

    f. for each of said executable tasks, identifying a plurality of steps needed to complete said task;

    g. for each of said plurality of steps, identifying and generalizing a step objective;

    h. learning to execute each of said steps from demonstrations provided by said user;

    i. providing incremental execution and interaction with said user using said natural dialog-based interface;

    j. providing a database for storing a task definition for each of said tasks, wherein said task definition includes said steps comprising said task and said step objectives;

    k. storing said task definitions in said database;

    l. for each of said tasks, learning semantic characterization of said task for later retrieval from said database;

    m. retrieving a particular task definition from said database using said semantic characterization;

    n. improving said task definition for each of said tasks through practice, with instruction being provided by said user;

    o. wherein one of said executable tasks returns a list of results;

    p. displaying said list of results in a first configuration of said GUI wherein said GUI displays a list consisting of multiple Document Object Model nodes;

    q. wherein said user provides natural language to said software running on said computational device indicating that iteration should be performed;

    r. automatically creating a second configuration of said GUI wherein said GUI displays a plurality of cells arranged into columns and rows, with each row representing a single Document Object Model node from said displayed list;

    s. wherein said user demonstrates a first task to be performed in a first Document Object Model node; and

    t. wherein thereafter said software running on said computation device automatically performs said first task demonstrate by said user in all other Document Object Model nodes and displays a result of said performance in said second configuration of said GUI.

View all claims
  • 1 Assignment
Timeline View
Assignment View
    ×
    ×