Natural-language voice-activated personal assistant

• US 7,216,080 B2
• Filed: 09/26/2001
• Issued: 05/08/2007
• Est. Priority Date: 09/29/2000
• Status: Expired due to Term

• Current Assignee: Nuance Communications, Inc. (Microsoft Corporation), Vlingo Corporation (Microsoft Corporation)
• Original Assignee: Mindfabric Holdings LLC
• Inventors: Tong, Peter P., Chao, David Yinkai, Tsiao, James Chi-Shun
• Primary Examiner(s): Hudspeth; David
• Assistant Examiner(s): Sked; Matthew J
• Application Number:
  US09/964,947
• Publication Number:
  US 20020040297A1
• Time in Patent Office: 2,050 Days
• Field of Search: 704/235, 704/260, 704/257, 704/246, 704/275
• US Class Current: 704/257
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

A handheld personal assistant including a voice-recognizer and a natural-language processor is disclosed. The recognizer is configured to transform a verbal expression from a person into a different mode of information (e.g., text). The natural-language processor is configured to process the mode of information to extract, from a database, a piece of information that is personal to the person. In processing, the processor analyzes the expression grammatically and semantically to transform at least a part of the mode of operation into at least one instruction. This piece of information can be a to-do list, information in the person'"'"'s calendar or information from the person'"'"'s address book, such as a phone number. The processor can still extract the piece of information when the person declares the expression differently.

Claims

1. A handheld personal assistant operated by a user comprising:

a categorizer configured to store a plurality of data items in a database organized by categories of information that are grouped into a calendar category, an address book category and a to-do-list category wherein each category of information includes one or more subcategories of information that are linked in a hierarchical structure;

a voice-recognizer configured to recognize the user'"'"'s voice and transform an expression input by the user into a different mode of information;

a context processor configured to determine the category of information corresponding to the expression input by the user, based on a first inferred context of the expression input and maintain the storage of the data items by the categorizer in the determined category of information until a new category of information is determined based upon a second inferred context; and

a natural-language processor configured to process the mode of information to extract a piece of information from the determined category of the database;

wherein if there is ambiguity with the mode of information to extract from a first category of information, the natural-language processor provides a first response to the user having a plurality of options, the natural-language processor then provides a second response based upon the user'"'"'s selection from the options.

2. A handheld personal assistant as recited in claim 1, wherein the processor analyzes the expression grammatically and semantically to transform at least a part of the expression into at least one instruction.

3. A handheld personal assistant as recited in claim 1, wherein the piece of information is selected from the address book category, the to-do-list category and the calendar category.

4. A handheld personal assistant as recited in claim 3, wherein said personal assistant further includes a display to display the piece of information.

5. A handheld personal assistant as recited in claim 3, wherein said personal assistant further includes a voice synthesizer that transforms the piece of information into sound to communicate to the person.

6. A handheld personal assistant as recited in claim 3,

wherein the piece of information was entered into the assistant by the user, and

wherein the personal assistant further includes a formatting process that transforms the expression input by the user into a question with one or more phrases corresponding to the one or more categories of information, and a transformation process that converts the question into an instruction comprising a query to the database.

7. A handheld personal assistant as recited in claim 6, wherein the piece of information was entered through voice.

8. A handheld personal assistant as recited in claim 3, wherein if the assistant cannot resolve an ambiguity in the expression, the personal assistant provides the person with a number of alternatives to resolve the ambiguity.

9. A handheld personal assistant as recited in claim 2,

wherein the personal assistant only allows the user to access the piece of information that is personal to the user if the natural-language processor recognizes of the users'"'"'s voice.

10. A method for obtaining information for a requestor interacting with a handheld computing device, said method comprising:

storing a plurality of data items in a database organized categories of information that are grouped into a calendar category, an address book category and a to-do-list category wherein each category of information includes one or more subcategories of information that are linked together in a hierarchical structure;

receiving an input voice expression from the requestor;

converting the input voice expression into a text string;

processing the text string using grammatical and semantic processing to determine a natural language meaning for the text string;

determining a category of information corresponding to the input by the requestor based on a first inferred context based on the expression input;

extracting a piece of information from the determined category of information based upon the input voice expression;

determining if there is an ambiguity with information extracted from the category of information;

transmitting a plurality of choices to the requestor if an ambiguity is determined;

resolving the ambiguity based upon the requestor'"'"'s response to the plurality of choices and repeating the extracting step;

maintaining the storing of the data items and the extracting of the piece of information in the determined category of information; and

changing the category of information for storing the data items to a new category of information that is determined based upon a second inferred context.

11. A method as recited in claim 10, further comprising the steps of:

transforming the input expression into a question format; and

converting the resulting question into an instruction comprising a query to the database.

12. A method as recited in claim 11, wherein said performing further comprises:

presenting the retrieved information to the requestor.