Controlling the listening horizon of an automatic speech recognition system for use in handsfree conversational dialogue

• US 7,240,011 B2
• Filed: 10/24/2005
• Issued: 07/03/2007
• Est. Priority Date: 05/17/1999
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Horvitz, Eric J.
• Primary Examiner(s): Storm; Donald L.
• Application Number:
  US11/257,249
• Publication Number:
  US 20060036445A1
• Time in Patent Office: 617 Days
• Field of Search: None
• US Class Current: 704/275
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

Conversational dialog with a computer or other processor-based device without requiring push-to-talk functionality. In one embodiment, a computer-implemented method first determines that a user desires to engage in a dialog. Based thereon the method turns on a speech recognition functionality for a period of time referred to as a listening horizon. Upon the listening horizon expiring, the method turns off the speech recognition functionality.

Claims

1. A computer-implemented method comprising:

determining that a user desires to engage in a dialog based at least upon inferring a probability that the user desires an automated service;

upon determining that the user desires to engage in a dialog, turning on a speech recognition functionality for a listening horizon, wherein the length of the listening horizon is determined as a function of at least an inferred probability that the user desires automated service; and

,

turning off the speech recognition functionality after the listening horizon has expired.

2. The method of claim 1, wherein determining that a user desires to engage in a dialog comprises performing a cost-benefit analysis to determine whether engaging in a dialog comprises a highest expected utility action.

3. The method of claim 2, wherein inferring a probability that the user desires an automated service is based upon at least one of text, contextual information, and recent user activity.

4. The method of claim 1, further comprising prior to turning on a speech recognition functionality, engaging the user with a question.

5. The method of claim 4, wherein engaging the user with a question comprises engaging the user with a question regarding a desire for an automated service.

6. The method of claim 4, wherein engaging the user with a question comprises displaying an automated assistant asking the question.

7. The method of claim 1, wherein determining a length of the listening horizon comprises determining the length of the listening horizon as a function of at least an inferred probability that the user desires automated service and an acute listening history.

8. The method of claim 1, wherein turning on a speech recognition functionality comprises displaying an automated assistant having listening-for-user-utterances indications.

9. The method of claim 1, further comprising prior to turning off the speech recognition functionality,

detecting an utterance from the user during the listening horizon;

and,

determining a confidence level of the utterance.

10. The method of claim 9, further comprising prior to turning off the speech recognition functionality, upon determining that the confidence level of the utterance is greater than a predetermined threshold, displaying an automated assistant indicating understanding and proceeding to turning off the speech recognition functionality.

11. The method of claim 9, further comprising prior to turning off the speech recognition functionality, upon determining that the confidence level of the utterance is greater than a predetermined threshold and the utterance indicates deliberation, displaying an automated assistant indicating increased attentiveness and continuing to detecting an utterance from the user during the listening horizon.

12. The method of claim 9, further comprising prior to turning off the speech recognition functionality, upon determining that the confidence level of the utterance is less than a predetermined threshold, displaying an automated assistant indicating hearing difficulty and continuing to detecting an utterance from the user during the listening horizon.

13. The method of claim 9, further comprising prior to turning off the speech recognition functionality, upon determining that the confidence level of the utterance is less than a predetermined threshold, and based on continued hearing difficulty, displaying an automated assistant indicating failure to hear and proceeding to turning off the speech recognition functionality.

14. The method of claim 1, further comprising prior to turning off the speech recognition functionality, upon failure to detect an utterance from the user and upon expiration of the listening horizon, displaying an automated assistant indicating sensitivity that the user is busy and proceeding to turning off the speech recognition functionality.

15. A computer-implemented method comprising:

determining that a user desires to engage in a dialog based at least upon inferring a probability that the user desires an automated assistant;

upon determining that the user desires to engage in a dialog,

engaging the user with a question;

displaying an automated assistant asking the question;

turning on a speech recognition functionality for a listening horizon;

during the listening horizon,

detecting an utterance from the user;

determining a confidence level of the utterance, wherein the automated assistant indicates understanding of the utterance if the confidence level is above a predetermined threshold; and

,

no later than after expiration of the listening horizon,

removing the automated assistant;

turning off the speech recognition functionality.

16. The method of claim 15, wherein determining that a user desires to engage in a dialog further comprises performing a cost-benefit analysis to determine whether engaging in a dialog comprises a highest expected utility action.

17. The method of claim 15, wherein engaging the user with a question comprises engaging the user with a question regarding a desire for an automated service.

18. A machine-readable medium having instructions stored thereon for execution by a processor to cause performance of a method comprising:

determining that a user desires to engage in a dialog based at least upon inferring a probability that the user desires an automated agent;

upon determining that the user desires to engage in a dialog, turning on a speech recognition functionality for a listening horizon, wherein the length of the listening horizon is determined as a function of at least an inferred probability that the user desires automated service; and

,

turning off the speech recognition functionality after the listening horizon has expired.