Method for using pauses detected in speech input to assist in interpreting the input during conversational interaction for information retrieval

• US 10,140,982 B2
• Filed: 08/31/2017
• Issued: 11/27/2018
• Est. Priority Date: 08/03/2012
• Status: Active Grant

• Current Assignee: Veveo, Inc. (Adeia Inc.)
• Original Assignee: Veveo, Inc. (Adeia Inc.)
• Inventors: Aravamudan, Murali, Gill, Daren, Venkataraman, Sashikumar, Agarwal, Vineet, Ramamoorthy, Ganesh
• Primary Examiner(s): Zhu, Richard
• Application Number:
  US15/693,162
• Publication Number:
  US 20170365254A1
• Time in Patent Office: 453 Days
• Field of Search: None
• CPC Class Codes: G06F 16/433 using audio data
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
• CPC Class Codes: G10L 15/19 Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

A method for using speech disfluencies detected in speech input to assist in interpreting the input is provided. The method includes providing access to a set of content items with metadata describing the content items, and receiving a speech input intended to identify a desired content item. The method further includes detecting a speech disfluency in the speech input and determining a measure of confidence of a user in a portion of the speech input following the speech disfluency. If the confidence measure is lower than a threshold value, the method includes determining an alternative query input based on replacing the portion of the speech input following the speech disfluency with another word or phrase. The method further includes selecting content items based on comparing the speech input, the alternative query input (when the confidence measure is low), and the metadata associated with the content items.

Claims

1. A method for using speech disfluencies detected in speech input to assist in interpreting the input, the method comprising:

providing access to a set of content items, each of the content items being associated with metadata that describes the corresponding content item;

receiving a speech input from a user, the input intended by the user to identify at least one desired content item;

detecting a speech disfluency in the speech input;

determining a confidence measure of the user in a portion of the speech input following the speech disfluency based on a manner by which the user utters the portion of the speech input following the speech disfluency;

upon a condition in which the confidence measure does not exceed a threshold value;

retrieving from memory preferences of the user for particular content items of the set of content items;

comparing the portion of the speech input to metadata associated with the particular content items of the set of content items;

determining, based on the comparing, whether metadata associated with a content item of the particular content items at least partially matches the portion of the speech input;

in response to determining that metadata associated with the content item of the particular content items at least partially matches the portion of the speech input;

replacing the portion of the speech input with the metadata associated with the content item to generate a query comprising a modified speech input for searching a database associated with the set of content items;

selecting from the database a subset of content items from the set of content items based on comparing the query and metadata associated with the subset of content items;

upon a condition in which the confidence measure exceeds a threshold value, selecting the subset of content items from the set of content items based on comparing the speech input and the metadata associated with the subset of content items; and

presenting the subset of content items to the user.

2. The method of claim 1, further comprising measuring a duration of the speech disfluency, wherein the determination of the confidence measure is based on the duration of the speech disfluency.

3. The method of claim 1, wherein the subset of content items is a first subset of content items, and wherein the determination of the confidence measure is based on a second subset of the content items.

4. The method of claim 1, further comprising offering assistance when the user engages in a speech disfluency.

5. The method of claim 4, wherein the assistance is inferring a word or phrase following the speech disfluency and presenting the word or phrase to the user.

6. The method of claim 1, wherein the speech disfluency is a pause or an auditory time filler.

7. A system for using speech disfluencies detected in speech input to assist in interpreting the input, the system comprising control circuitry configured to:

provide access to a set of content items, each of the content items being associated with metadata that describes the corresponding content item;

receive a speech input from a user, the input intended by the user to identify at least one desired content item;

detect a speech disfluency in the speech input;

determine a confidence measure of the user in a portion of the speech input following the speech disfluency based on a manner by which the user utters the portion of the speech input following the speech disfluency;

upon a condition in which the confidence measure does not exceed a threshold value;

retrieve from memory preferences of the user for particular content items of the set of content items;

compare the portion of the speech input to metadata associated with a content item of the particular content items at least partially matches the portion of the speech input;

in response to determining that the metadata associated with the content item of the particular content items at least partially matches the portion of the speech input;

replacing the portion of the speech input with the metadata associated with the content item to generate a query comprising a modified speech input for searching a database associated with the set of content items;

select from the database a subset of content items from the set of content items based on comparing the query and metadata associated with the subset of content items;

upon a condition in which the confidence measure exceeds a threshold value, select the subset of content items from the set of content items based on comparing the speech input and the metadata associated with the subset of content items; and

present the subset of content items to the user.

8. The system of claim 7, wherein the control circuitry is further configured to measure a duration of the speech disfluency and wherein the determination of the confidence measure is based on the duration of the speech disfluency.

9. The system of claim 7, wherein the subset of content items is a first subset of content items, and wherein the determination of the confidence measure is based on a second subset of the content items.

10. The system of claim 7, wherein the control circuitry is further configured to offer assistance when the user engages in a speech disfluency.

11. The system of claim 10, wherein the assistance is inferring a word or phrase following the speech disfluency and presenting the word or phrase to the user.

12. The system of claim 7, wherein the speech disfluency is a pause or an auditory time filler.