METHOD FOR USING PAUSES DETECTED IN SPEECH INPUT TO ASSIST IN INTERPRETING THE INPUT DURING CONVERSATIONAL INTERACTION FOR INFORMATION RETRIEVAL

• US 20170365254A1
• Filed: 08/31/2017
• Published: 12/21/2017
• Est. Priority Date: 08/03/2012
• Status: Active Grant

• Current Assignee: Veveo, Inc. (Adeia Inc.)
• Original Assignee: Veveo, Inc. (Adeia Inc.)
• Inventors: Aravamudan, Murali, Gill, Daren, Venkataraman, Sashikumar, Agarwal, Vineet, Ramamoorthy, Ganesh
• Granted Patent: US 10,140,982 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/433 using audio data
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/187 Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
• CPC Class Codes: G10L 15/19 Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

A method for using speech disfluencies detected in speech input to assist in interpreting the input is provided. The method includes providing access to a set of content items with metadata describing the content items, and receiving a speech input intended to identify a desired content item. The method further includes detecting a speech disfluency in the speech input and determining a measure of confidence of a user in a portion of the speech input following the speech disfluency. If the confidence measure is lower than a threshold value, the method includes determining an alternative query input based on replacing the portion of the speech input following the speech disfluency with another word or phrase. The method further includes selecting content items based on comparing the speech input, the alternative query input (when the confidence measure is low), and the metadata associated with the content items.

Claims

1-28. -28. (canceled)

29. A method for using speech disfluencies detected in speech input to assist in interpreting the input, the method comprising:

providing access to a set of content items, each of the content items being associated with metadata that describes the corresponding content item;

receiving a speech input from a user, the input intended by the user to identify at least one desired content item;

detecting a speech disfluency in the speech input;

determining a measure of confidence of the user in a portion of the speech input following the speech disfluency based on a manner by which the user utters the portion of the speech input following the speech disfluency;

upon a condition in which the confidence measure does not exceed a threshold value, determining an alternative query input by automatically replacing the portion of the speech input following the speech disfluency with another word or phrase and selecting a subset of content items from the set of content items based on comparing the speech input, the alternative query input, and the metadata associated with the subset of content items;

upon a condition in which the confidence measure exceeds a threshold value, selecting the subset of content items from the set of content items based on comparing the speech input and the metadata associated with the subset of content items; and

presenting the subset of content items to the user.

30. The method of claim 29, further comprising measuring a duration of the speech disfluency, wherein the determination of the confidence measure is based on the duration of the speech disfluency.

31. The method of claim 29, wherein the subset of content items is a first subset of content items, and wherein the determination of the confidence measure is based on a second subset of the content items.

32. The method of claim 29, further comprising offering assistance when the user engages in a speech disfluency.

33. The method of claim 32, wherein the assistance is inferring a word or phrase following the speech disfluency and presenting the word or phrase to the user.

34. The method of claim 29, wherein the speech disfluency is a pause or an auditory time filler.

35. The method of claim 29, further comprising providing a user preference signature, the user preference signature describing preferences of the user for at least one of (i) particular content items and (ii) metadata associated with the content items, wherein each of the content items is associated with metadata that describes the corresponding content items and wherein the portion of the speech input that is replaced is selected based on the user preference signature.

36. A method for using speech disfluencies detected in speech input to assist in interpreting the input, the method comprising:

providing access to a set of content items, each of the content items being associated with metadata that describes the corresponding content item;

receiving a speech input from a user, the input intended by the user to identify at least one desired content item;

detecting front-end clipping of a first portion of a first word in a beginning of the speech input based on an absence of a period of silence in the beginning of the speech input, wherein the first portion is not detected in the speech input, the front-end clipping resulting in incomplete detection of the first word;

in response to detecting the front-end clipping, identifying a second portion of the first word detected in the received speech input;

identifying a plurality of whole words having a suffix matching the second portion detected in the received speech input;

constructing a plurality of query inputs using the plurality of whole words;

selecting a subset of content items from the set of content items based on comparing the plurality of query inputs and the metadata associated with the subset of content items; and

presenting the subset of content items to the user.

37. The method of claim 36, further comprising providing a user preference signature, the user preference signature describing preferences of the user for at least one of (i) particular content items and (ii) metadata associated with the content items, wherein each of the content items is associated with metadata that describes the corresponding content items and wherein the words having a suffix matching the second portion detected in the received speech input are further selected based on the user preference signature.

38. The method of claim 36, further comprising determining that a confidence of the user in the second portion of the word surpasses a threshold minimum confidence measure.

39. A system for using speech disfluencies detected in speech input to assist in interpreting the input, the system comprising control circuitry configured to:

provide access to a set of content items, each of the content items being associated with metadata that describes the corresponding content item;

receive a speech input from a user, the input intended by the user to identify at least one desired content item;

detect a speech disfluency in the speech input;

determine a measure of confidence of the user in a portion of the speech input following the speech disfluency based on a manner by which the user utters the portion of the speech input following the speech disfluency;

upon a condition in which the confidence measure does not exceed a threshold value, determine an alternative query input by automatically replacing the portion of the speech input following the speech disfluency with another word or phrase and select a subset of content items from the set of content items based on comparing the speech input, the alternative query input, and the metadata associated with the subset of content items;

upon a condition in which the confidence measure exceeds a threshold value, select the subset of content items from the set of content items based on comparing the speech input and the metadata associated with the subset of content items; and

present the subset of content items to the user.

40. The system of claim 39, wherein the control circuitry is further configured to measure a duration of the speech disfluency and wherein the determination of the confidence measure is based on the duration of the speech disfluency.

41. The system of claim 39, wherein the subset of content items is a first subset of content items, and wherein the determination of the confidence measure is based on a second subset of the content items.

42. The system of claim 39, wherein the control circuitry is further configured to offer assistance when the user engages in a speech disfluency.

43. The system of claim 42, wherein the assistance is inferring a word or phrase following the speech disfluency and presenting the word or phrase to the user.

44. The system of claim 39, wherein the speech disfluency is a pause or an auditory time filler.

45. The system of claim 39, wherein the control circuitry is further configured to provide a user preference signature, the user preference signature describing preferences of the user for at least one of (i) particular content items and (ii) metadata associated with the content items and wherein each of the content items is associated with metadata that describes the corresponding content items and wherein the portion of the speech input that is replaced is selected based on the user preference signature.

46. A system for using speech disfluencies detected in speech input to assist in interpreting the input, the comprising control circuitry configured to:

provide access to a set of content items, each of the content items being associated with metadata that describes the corresponding content item;

receive a speech input from a user, the input intended by the user to identify at least one desired content item;

detect front-end clipping of a first portion of a first word in a beginning of the speech input based on an absence of a period of silence in the beginning of the speech input, wherein the first portion is not detected in the speech input, the front-end clipping resulting in incomplete detection of the first word;

in response to detecting the front-end clipping, identify a second portion of the first word detected in the received speech input;

identify a plurality of whole words having a suffix matching the second portion detected in the received speech input;

construct a plurality of query inputs using the plurality of whole words;

select a subset of content items from the set of content items based on comparing the plurality of query inputs and the metadata associated with the subset of content items; and

present the subset of content items to the user.

47. The system of claim 46, wherein the control circuitry is further configured to provide a user preference signature, the user preference signature describing preferences of the user for at least one of (i) particular content items and (ii) metadata associated with the content items, wherein each of the content items is associated with metadata that describes the corresponding content items and wherein the words having a suffix matching the second portion detected in the received speech input are further selected based on the user preference signature.

48. The system of claim 46, wherein the control circuitry is further configured to determine that a confidence of the user in the second portion of the word surpasses a threshold minimum confidence measure.