SYSTEMS AND METHODS FOR RESPONDING TO NATURAL LANGUAGE SPEECH UTTERANCE

• US 20080319751A1
• Filed: 07/07/2008
• Published: 12/25/2008
• Est. Priority Date: 06/03/2002
• Status: Active Grant

• Current Assignee: Dialect, LLC
• Original Assignee: VoiceBox Technologies Corporation (Microsoft Corporation)
• Inventors: Kennewick, Robert A., Locke, David, Kennewick, Richard, Freeman, Tom, Kennewick, Michael R. JR., Kennewick, Michael R. SR.
• Granted Patent: US 7,809,570 B2
• Time in Patent Office: Days
• US Class Current: 704/257
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/228 of application context
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching

Abstract

Systems and methods for receiving natural language queries and/or commands and execute the queries and/or commands. The systems and methods overcomes the deficiencies of prior art speech query and response systems through the application of a complete speech-based information query, retrieval, presentation and command environment. This environment makes significant use of context, prior information, domain knowledge, and user specific profile data to achieve a natural environment for one or more users making queries or commands in multiple domains. Through this integrated approach, a complete speech-based natural language query and response environment can be created. The systems and methods creates, stores and uses extensive personal profile information for each user, thereby improving the reliability of determining the context and presenting the expected results for a particular question or command.

Claims

1. A method responsive to a user generated natural language speech utterance, comprising:

receiving the user generated natural language speech utterance, the received user utterance containing at least one request;

determining a meaning of the received user utterance, wherein determining the meaning includes determining a context for the at least one request contained in the user utterance;

selecting at least one domain agent based on the determined meaning, the selected domain agent being an autonomous executable that receives, processes, and responds to requests associated with the determined context;

formulating the at least one request contained in the utterance in accordance with a grammar used by the selected domain agent to process requests associated with the determined context; and

invoking the selected domain agent to process the formulated request.

2. The method according to claim 1, wherein determining the context includes applying prior probabilities or fuzzy possibilities to keywords contained in the utterance, a profile of the user, and a dialog history relating to one or more prior utterances.

3. The method according to claim 1, wherein the selected domain agent processes requests associated with the determined context by submitting one or more queries to local or network information sources.

4. The method according to claim 1, wherein the selected domain agent processes requests associated with the determined context by directing one or more commands to local or remote processing devices.

5. A method responsive to a user generated natural language speech utterance, comprising:

receiving the user generated natural language speech utterance, the received user utterance containing at least one question;

determining a meaning of the received user utterance, wherein determining the meaning includes determining a context for the at least one question contained in the user utterance;

selecting at least one domain agent based on the determined meaning, the selected domain agent being an autonomous executable that receives, processes, and responds to questions associated with the determined context;

formulating the at least one question contained in the utterance in accordance with a grammar used by the selected domain agent to process requests associated with the determined context; and

invoking the selected domain agent to process the formulated question, wherein the invoked domain agent processes the formulated question by;

submitting a plurality of asynchronous queries to a plurality of information sources;

asynchronously receiving a plurality of results of the queries from the plurality of information sources; and

scoring the asynchronously received results to determine one or more best responses to the at least one question contained in the utterance.

6. The method according to claim 5, wherein the invoked domain agent formats the one or more best responses using a format template associated with the determined context, criteria identified from the user utterance, and a profile of the user.

7. The method according to claim 6, wherein the invoked domain agent formats the one or more best responses by performing variable substitutions and transformations to produce a response best understood and most natural to the user.

15. The method according to claim 5, wherein the invoked domain agent scores the asynchronously received results by applying probabilistic or fuzzy scoring techniques based on previously received asynchronous results, the determined context, criteria identified from the user utterance, tokens associated with a desired response, a dialog history relating to one or more prior utterances, a profile of the user, and information specific to the invoked domain agent.

16. The method according to claim 7, wherein the invoked domain agent produces the response best understood and most natural to the user using probabilistic or fuzzy set decision methods to vary one or more of a personality for presenting the response, an order for presenting tokens contained in the response, or a terminology for presenting the tokens contained in the response.

8. A system responsive to a user generated natural language speech utterance, the system comprising one or more processing units collectively operable to:

receive the user generated natural language speech utterance, the received user utterance containing at least one request;

determine a meaning of the received user utterance, wherein determining the meaning includes determining a context for the at least one request contained in the user utterance;

select at least one domain agent based on the determined meaning, the selected domain agent being an autonomous executable that receives, processes, and responds to requests associated with the determined context;

formulate the at least one request contained in the utterance in accordance with a grammar used by the selected domain agent to process requests associated with the determined context; and

invoke the select domain agent to process the formulated request.

9. The system according to claim 8, the processing units operable to determine the context by applying prior probabilities or fuzzy possibilities to keywords contained in the utterance, a profile of the user, and a dialog history relating to one or more prior utterances.

10. The system according to claim 8, wherein the selected domain agent processes requests associated with the determined context by:

submitting a plurality of asynchronous queries to a plurality of local or network information sources;

asynchronously receiving a plurality of results of the queries from the plurality of information sources; and

scoring the asynchronously received results to determine one or more best responses to the at least one request contained in the utterance.

11. The system according to claim 8, wherein the selected domain agent processes requests associated with the determined context by directing one or more commands to local or remote processing devices.

12. The system according to claim 10, wherein the selected domain agent scores the asynchronously received results by applying probabilistic or fuzzy scoring techniques based on previously received asynchronous results, the determined context, criteria identified from the user utterance, tokens associated with a desired response, a dialog history relating to one or more prior utterances, a profile of the user, and information specific to the selected domain agent.

13. The system according to claim 10, wherein the selected domain agent formats the one or more best responses using a format template associated with the determined context, criteria identified from the user utterance, and a profile of the user.

14. The system according to claim 13, wherein the selected domain agent formats the one or more best responses by performing variable substitutions and transformations to produce a response best understood and most natural to the user.

17. The system according to claim 14, wherein the selected domain agent produces the response best understood and most natural to the user using probabilistic or fuzzy set decision methods to vary one or more of a personality for presenting the response, an order for presenting tokens contained in the response, or a terminology for presenting the tokens contained in the response.

18. A method for recognizing natural language speech utterances, comprising:

receiving a natural language speech utterance from a user;

determining an identity of the user by comparing voice characteristics of the received utterance to one or more user profiles;

recognizing one or more words or phrases contained in the received utterance using at least one dictionary and phrase table, wherein the at least one dictionary and phrase table includes a plurality of entries associated with a dynamic set of prior probabilities or fuzzy possibilities;

updating the dynamic set of prior probabilities or fuzzy possibilities based on a confidence level for the recognized words or phrases;

requesting, based on the confidence level for the recognized words or phrases, a phonetic alphabet spelling for unrecognized or out-of-vocabulary words or phrases in the received utterance, the phonetic alphabet spelling provided by the user in one or more subsequent natural language utterances; and

associating the identity of the user with a recognition of the received natural language utterance, wherein the recognition includes the recognized words or phrases, and further includes the phonetic alphabet spelling for the unrecognized or out-of-vocabulary words or phrases.

19. A method for filtering and eliminating noise from natural language speech utterances, comprising:

receiving a natural language utterance from a user at a microphone array, wherein the microphone array directs a beam pattern in a direction of the user, the beam pattern including one or more nulls that notch out point or limited area noise sources;

establishing parameters for a band shaping and notch filtering mechanism, the parameters established by comparing environmental noise to the signal representing the utterance;

removing narrow-band noise from the signal representing the utterance using the band shaping and notch filtering mechanism, the band shaping and notch filtering mechanism optimized according to the established parameters;

suppressing cross-talk and environmentally caused echoes in the signal representing the utterance using adaptive echo cancellation;

removing momentary gaps from the signal representing the utterance using adaptive lossy compression; and

storing a digitized representation of the signal representing the utterance in an output buffer.