Method for integrating processes with a multi-faceted human centered interface

• US 7,188,067 B2
• Filed: 07/14/2003
• Issued: 03/06/2007
• Est. Priority Date: 12/23/1998
• Status: Expired due to Term

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Original Assignee: Eastern Investments LLC
• Inventors: Grant, Richard, McGregor, Pedro E.
• Primary Examiner(s): Chawan, Vijay B.
• Application Number:
  US10/619,204
• Publication Number:
  US 20040249640A1
• Time in Patent Office: 1,331 Days
• Field of Search: 704270-277, 704/231, 704/251
• US Class Current: 704/275
• CPC Class Codes: G10L 15/193 Formal grammars, e.g. finite state automata, context free grammars or word networks
• CPC Class Codes: G10L 2015/228 of application context

Abstract

According to the present invention, a method for integrating processes with a multi-faceted human centered interface is provided. The interface is facilitated to implement a hands free, voice driven environment to control processes and applications. A natural language model is used to parse voice initiated commands and data, and to route those voice initiated inputs to the required applications or processes. The use of an intelligent context based parser allows the system to intelligently determine what processes are required to complete a task which is initiated using natural language. A single window environment provides an interface which is comfortable to the user by preventing the occurrence of distracting windows from appearing. The single window has a plurality of facets which allow distinct viewing areas. Each facet has an independent process routing its outputs thereto. As other processes are activated, each facet can reshape itself to bring a new process into one of the viewing areas. All activated processes are executed simultaneously to provide true multitasking.

Claims

1. A method for integrating processes within a voice command human interface, comprising:

receiving a voice signal comprising a contextually defined command sequence;

determining a context;

processing the voice signal using a context-dependent natural language model;

parsing executable commands from the processed voice signal;

determining a set of processes appropriate for execution of the parsed commands;

routing the parsed commands to the appropriate processes for execution; and

executing the contextually defined command sequence, wherein a plurality of processes execute concurrently.

2. The method according to claim 1, wherein said processing step comprises converting received speech to a text representation thereof.

3. The method according to claim 1, wherein said parsing step comprises comparing a set of text representation grammars to a known vocabulary table.

4. The method according to claim 1, further comprising the step of registering a set of valid command statements in a process registration database.

5. The method according to claim 1, wherein at least one process is a system process and at least one process is an application process, the system process being continuously available and the application process being invoked by the system process as required to execute commands.

6. The method according to claim 1, further comprising the step of providing a response to the command through the human interface.

7. The method according to claim 1, further comprising the step of providing an error message response through the human interface.

8. The method according to claim 1, further comprising the step of presenting an output from a plurality of processes through respectively different windows of a graphic user interface.

9. The method according to claim 1, further comprising the step of presenting an output from a plurality of processes through respectively different facets of a graphic user interface.

10. An apparatus for integrating processes within a voice command human interface, comprising:

an input for receiving a voice signal comprising a contextually defined command sequence;

at least one memory location storing information defining a natural language model;

at least one memory location storing an application process;

an input for receiving context information;

a processor for processing the voice signal using a the natural language model in context sensitive manner, parsing executable commands from the processed voice signal, and executing the parsed commands by employing the respectively appropriate process; and

an output for producing a signal in dependence on the execution of the parsed commands.

11. The apparatus according to claim 10, wherein said processor converts received voice to a text representation of words represented therein.

12. The apparatus according to claim 10, further comprising at least one memory location storing a known vocabulary table, wherein the parsed executable commands are compared with known vocabularies stored in the known vocabulary table.

13. The apparatus according to claim 10, further comprising at least one memory location storing a process registration database, wherein a process for executing a parsed executable commands is determined by reference to the process registration database.

14. The apparatus according to claim 10, wherein the output comprises a graphic user interface having a plurality of windows, each of a plurality of concurrently executing processes presenting a respective output through respectively different windows of the graphic user interface.

15. A computer readable storage medium for controlling a general purpose computer to perform a method to provide a natural language voice command human interface, the method comprising:

receiving a voice signal comprising a contextually defined command sequence;

determining a context;

processing the voice signal using a context-dependent natural language model;

parsing executable commands from the processed voice signal;

determining a set of processes appropriate for execution of the parsed commands;

routing the parsed commands to the appropriate processes for execution; and

executing the contextually defined command sequence, wherein a plurality of processes execute concurrently.

16. The computer readable medium according to claim 15, wherein the method further comprises the step of determining received non-command voice signals, determining a word content thereof, and presenting the word content as data for at least one process.

17. The computer readable medium according to claim 15, further comprising the step of presenting an output from a plurality of processes through respectively different windows of a graphic user interface.

18. A method for controlling a plurality of processes and performing at least one operation based on a voice input stream, comprising the steps of:

processing the voice input stream to produce at least one grammar, using a context-dependent natural language model;

parsing the grammar to extract at least one command therefrom;

employing a plurality of concurrently active process handlers in to handle the at least one command; and

outputting information associated with at least a portion of the process handlers.

19. The method according to claim 18, further comprising the steps of:

displaying the outputted information within a facet of a graphic user interface for each respective process handler; and

updating and displaying stored information associated with the process handler in the respective facet.

20. A method for performing multiple step operations with a single spoken command phrase using natural language modeling, comprising the steps of:

using a context-dependent natural language model to produce a machine-interpretable command structure from a spoken command phrase stream; and

in response to the machine-interpretable command structure, simultaneously initiating multiple process to be executed in an appropriate sequence for controlling appropriately responding to the spoken command phrase.

21. A method for controlling a multi-faceted graphical user interface display using a natural language model for performing multiple step operations with a single spoken command phrase comprising the steps of:

using a natural language model in response to a user initiating a single spoken command phrase;

activating and controlling at least one of said multiple facets on said graphic user interface display area through the use of said user'"'"'s single spoken command phrase based on said natural language model; and

displaying information associated with execution of said process in at least one display facet.

22. The method according to claim 21, further comprising the step of replacing displayed information associated with previously executed processes with display of information associated with execution of a newly executing process.