SYSTEM AND METHOD FOR MACHINE-MEDIATED HUMAN-HUMAN CONVERSATION

• US 20130144616A1
• Filed: 12/06/2011
• Published: 06/06/2013
• Est. Priority Date: 12/06/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: BANGALORE, Srinivas
• Granted Patent: US 9,214,157 B2
• Time in Patent Office: Days
• US Class Current: 704/226
• CPC Class Codes: G10L 15/075 supervised, i.e. under machine guidance
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/197 Probabilistic grammars, e.g. word n-grams
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence
• CPC Class Codes: G10L 15/25 using position of the lips, movement of the lips or face analysis
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/228 of application context
• CPC Class Codes: G10L 25/84 for discriminating voice from noise

Abstract

Disclosed herein are systems, methods, and non-transitory computer-readable storage media for processing speech. A system configured to practice the method monitors user utterances to generate a conversation context. Then the system receives a current user utterance independent of non-natural language input intended to trigger speech processing. The system compares the current user utterance to the conversation context to generate a context similarity score, and if the context similarity score is above a threshold, incorporates the current user utterance into the conversation context. If the context similarity score is below the threshold, the system discards the current user utterance. The system can compare the current user utterance to the conversation context based on an n-gram distribution, a perplexity score, and a perplexity threshold. Alternately, the system can use a task model to compare the current user utterance to the conversation context.

Claims

1. A method comprising:

monitoring user utterances to generate a conversation context;

receiving a current user utterance independent of non-natural language input intended to trigger speech processing;

comparing the current user utterance to the conversation context to generate a context similarity score;

if the context similarity score is above a threshold, incorporating the current user utterance into the conversation context; and

if the context similarity score is below the threshold, discarding the current user utterance.

2. The method of claim 1, further comprising:

computing an n-gram distribution for the user utterances in the conversation context;

computing a perplexity of the current user utterance based on the n-gram distribution;

if, based on a perplexity threshold, the current user utterance is a continuation of the conversation context, incorporating the current user utterance into the conversation context and updating the n-gram distribution based on the current user utterance; and

if, based on the perplexity threshold, the current user utterance is not a continuation of the conversation context, discarding the current user utterance.

3. The method of claim 1, wherein comparing the current user utterance to the conversation context is based on a task model associated with a specific task.

4. The method of claim 3, wherein the task model indicates at least one of a conversation structure, a grammar, and a dictionary.

5. The method of claim 1, wherein the conversation context is generated using speech recognition.

6. The method of claim 1, wherein user utterances are monitored without being triggered by a “

push-to-talk”

event.

7. The method of claim 1, wherein the user utterances originate from at least one user.

8. The method of claim 1, wherein the conversation context describes at least part of a human-human dialog.

9. The method of claim 1, wherein the conversation context describes at least part of a human-machine dialog.

10. The method of claim 1, further comprising applying a noise suppression mechanism when monitoring the user utterances.

11. The method of claim 1, wherein the threshold is dynamic.

12. The method of claim 11, wherein the threshold is based on at least one of a user, a topic, the conversation context, confidence scores, and background noise.

13. A system comprising:

a processor;

a memory having stored therein instructions for controlling the processor to perform steps comprising;

monitoring user utterances to generate a conversation context;

receiving a current user utterance independent of non-natural language input intended to trigger speech processing;

comparing the current user utterance to the conversation context to generate a context similarity score;

if the context similarity score is above a threshold, incorporating the current user utterance into the conversation context; and

if the context similarity score is below the threshold, discarding the current user utterance.

14. The system of claim 13, the instructions further comprising:

computing an n-gram distribution for the user utterances in the conversation context;

computing a perplexity of the current user utterance based on the n-gram distribution;

if, based on a perplexity threshold, the current user utterance is a continuation of the conversation context, incorporating the current user utterance into the conversation context and updating the n-gram distribution based on the current user utterance; and

if, based on the perplexity threshold, the current user utterance is not a continuation of the conversation context, discarding the current user utterance.

15. The system of claim 13, wherein comparing the current user utterance to the conversation context is based on context modeling.

16. The system of claim 15, wherein the context modeling is based on at least one other multimodal feature.

17. The system of claim 16, wherein the at least one other multimodal feature comprises at least one of gaze, head orientation, and a visual feature.

18. The system of claim 13, wherein user utterances are monitored without being triggered by a “

push-to-talk”

event.

19. The system of claim 13, wherein the user utterances originate from at least one user.

20. A non-transitory computer-readable storage medium having stored therein instructions which, when executed by a computing device, cause the computing device to perform steps comprising:

monitoring user utterances to generate a conversation context;

receiving a current user utterance independent of non-natural language input intended to trigger speech processing;

comparing the current user utterance to the conversation context to generate a context similarity score;

if the context similarity score is above a threshold, incorporating the current user utterance into the conversation context; and

if the context similarity score is below the threshold, discarding the current user utterance.