Speaker and Call Characteristic Sensitive Open Voice Search

• US 20150294669A1
• Filed: 06/25/2015
• Published: 10/15/2015
• Est. Priority Date: 03/03/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Zhang, Shilei, Bao, Shenghua, Liu, Wen, Qin, Yong, Shuang, Zhiwei, Chen, Jian, Su, Zhong, Shi, Qin, Ganong, William F. III
• Granted Patent: US 10,032,454 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/35 Clustering; Classification
• CPC Class Codes: G06F 16/433 using audio data
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/1807 using prosody or stress
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

Techniques disclosed herein include systems and methods for open-domain voice-enabled searching that is speaker sensitive. Techniques include using speech information, speaker information, and information associated with a spoken query to enhance open voice search results. This includes integrating a textual index with a voice index to support the entire search cycle. Given a voice query, the system can execute two matching processes simultaneously. This can include a text matching process based on the output of speech recognition, as well as a voice matching process based on characteristics of a caller or user voicing a query. Characteristics of the caller can include output of voice feature extraction and metadata about the call. The system clusters callers according to these characteristics. The system can use specific voice and text clusters to modify speech recognition results, as well as modifying search results.

Claims

1. A method comprising:

classifying, by a computing device, speech into at least one voice cluster based on identified acoustic features of the speech, the at least one voice cluster corresponding to a text cluster and a customized language model that reflects characteristics of a speaker of the speech;

determining, by the computing device, a text query based on the customized language model and one or more text strings determined based on the speech;

receiving, by the computing device, search results based on the text query, each of the search results having a ranking indicating a measure of importance relative to other of the search results; and

re-ranking, by the computing device, the search results based on re-scoring the search results using the text cluster.

2. The method of claim 1, comprising:

re-assigning scores to the one or more text strings based on evaluating the one or more text strings using the customized language model.

3. The method of claim 2, comprising:

updating the at least one voice cluster based on an log of interaction with the search results.

4. The method of claim 1, comprising:

receiving, by the computing device, the speech; and

converting, by the computing device, the speech into the one or more text strings using a speech recognition process.

5. The method of claim 4, comprising:

determining, using an initial language model, a score for each of the one or more text strings, the score of each of the one or more text strings being used to compute a probability of correct conversion of the speech into the one or more text strings.

6. The method of claim 1, comprising:

receiving metadata that corresponds to the speech,

wherein classifying the speech into the at least one voice cluster based on the identified acoustic features of the speech comprises classifying the speech into the at least one voice cluster based on the identified acoustic features of the speech and the metadata that corresponds to the speech.

7. The method of claim 6, wherein the metadata that corresponds to the speech comprises a time of day when the speech was captured.

8. The method of claim 6, wherein the metadata that corresponds to the speech comprises a telephone number of a telephone that captured the speech.

9. The method of claim 8, comprising:

retrieving previous spoken queries requested by the telephone using the telephone number; and

modifying rankings of the search results based on the previous spoken queries.

10. The method of claim 8, wherein the metadata that corresponds to the speech comprises an area code of the telephone number of the telephone that captured the speech.

11. The method of claim 10, comprising:

using the area code to determine regional word choices and interests; and

modifying rankings of the search results based on the regional word choices and interests.

12. The method of claim 6, wherein the metadata comprises a location of a mobile device that captured the speech, wherein the location of the mobile device comprises a moving vehicle.

13. The method of claim 6, wherein the metadata comprises model information of a mobile device that captured the speech.

14. The method of claim 6, wherein the metadata comprises hardware information of a mobile device that captured the speech, and profile information of a profile of the speaker of the speech on the mobile device that captured the speech.

15. The method of claim 1, wherein the acoustic features of the speech comprise pitch, frequency, and speed.

16. The method of claim 1, wherein classifying the speech into the at least one voice cluster based on the identified acoustic features of the speech comprises determining that the identified acoustic features are associated with an age.

17. The method of claim 1, wherein classifying the speech into the at least one voice cluster based on the identified acoustic features of the speech comprises determining that the identified acoustic features are associated with an accent of a geographic region.

18. The method of claim 1, comprising:

receiving the speech from a wireless mobile device,

wherein receiving the search results based on the text query comprises receiving the search results from an open domain search executed by a search engine based on the one or more text strings.

19. Non-transitory computer-readable media storing executable instructions that, when executed by one or more processors, cause a system to:

classify speech into at least one voice cluster based on identified acoustic features of the speech, the at least one voice cluster corresponding to a text cluster and a customized language model that reflects characteristics of a speaker of the speech;

determine a text query based on the customized language model and one or more text strings determined based on the speech;

receive search results based on the text query, each of the search results having a ranking indicating a measure of importance relative to other of the search results; and

re-rank the search results based on re-scoring the search results using the text cluster.

20. A system comprising:

one or more processors; and

non-transitory memory storing instructions that, when executed by the one or more processors, cause the system to;

classify speech into at least one voice cluster based on identified acoustic features of the speech, the at least one voice cluster corresponding to a text cluster and a customized language model that reflects characteristics of a speaker of the speech;

determine a text query based on the customized language model and one or more text strings determined based on the speech;

receive search results based on the text query, each of the search results having a ranking indicating a measure of importance relative to other of the search results; and

re-rank the search results based on re-scoring the search results using the text cluster.