Search results using intonation nuances

• US 9,378,741 B2
• Filed: 03/12/2013
• Issued: 06/28/2016
• Est. Priority Date: 03/12/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Coussemaeker, Didier, Deniel, Loic, Kong, Hoon, Mai, Nicolas, Krishnamoorthy, Narayan, Na, Ty Long
• Primary Examiner(s): Zhu, Richard
• Application Number:
  US13/795,748
• Publication Number:
  US 20140278400A1
• Time in Patent Office: 1,204 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/243 Natural language query formulation
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/636 by using biological or physiological data
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06Q 30/0251 Targeted advertisements
• CPC Class Codes: G06Q 30/0269 based on user profile or attribute
• CPC Class Codes: G10L 15/06 Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice G10L15/14 takes precedence
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 15/32 Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices
• CPC Class Codes: G10L 2015/221 Announcement of recognition results
• CPC Class Codes: G10L 25/00 Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 muting semiconductor-based amplifiers when some special characteristics of a signal are sensed by a speech detector, e.g. sensing when no signal is present, H03G3/34
• CPC Class Codes: G10L 25/63 for estimating an emotional state
• CPC Class Codes: G10L 25/75 for modelling vocal tract parameters

Abstract

Systems and methods for responding to an audio query are presented. More particularly, vocalization nuances of a vocalized search query (audio query) are identified are utilized in responding to the audio query. In addition to converting the audio query to a textual representation, vocalization nuances of the audio query are identified. Search results are identified according to the textual representation of the audio query and in light of the vocalization nuances. A search results presentation is prepared in response to the audio query, where the search results presentation is based on the identified search results and also based on the vocalization nuances. The search results presentation is returned in response to the audio query.

Claims

1. A computer implemented method for responding to an audio query from a computer user, the method comprising:

receiving an audio query from a computer user;

translating the audio query to a textual representation;

identifying a vocalization nuance corresponding to the computer user from the audio query;

determining a confidence value associated with the identified vocalization nuance, the confidence value indicating a confidence score that the identified vocalization nuance correctly reflects the corresponding aspect of the computer user;

determining a risk value associated with the identified vocalization nuance, the risk value corresponding to the risk of providing offensive content to the computer user of search results selected in conjunction with the identified vocalization nuance irrespective of whether the vocalization nuance accurately reflects some attribute of the computer user;

identifying a plurality of search results corresponding to the textual representation of the audio query based, in part, on the identified vocalization nuance according to the confidence value and the risk value associated with the identified vocalization nuance;

generating a search results presentation based on the identified plurality of search results; and

returning the search results presentation to the computer user in response to receiving the audio query.

2. The computer implemented method of claim 1, wherein identifying a plurality of search results corresponding to the textual representation of the audio query in conjunction with the identified vocalization nuance associated with the audio query comprises:

determining that the confidence value and the risk value associated with the identified vocalization nuance fall within predetermined acceptable thresholds;

updating the textual representation of the audio query according to the identified vocalization nuance; and

identifying a plurality of search results corresponding to the updated textual representation of the audio query in conjunction with the identified vocalization nuance.

3. The computer implemented method of claim 1, wherein generating a search results presentation based on the identified plurality of search results comprises generating an audio presentation of at least some of the identified search results.

4. The computer implemented method of claim 1 further comprising identifying an advertisement to be included with the search results presentation, the advertisement being selected according to the identified vocalization nuance;

and

wherein generating a search results presentation based on the identified plurality of search results comprises generating a visual presentation based on the identified plurality of search results and the identified advertisement.

5. The computer implemented method of claim 1, wherein the identified vocalization nuance comprises a nuance corresponding to any one of gender, age, mood, emotion, urgency, ethnicity, country or region of origin, and social class.

6. The computer implemented method of claim 1, wherein identifying a vocalization nuance associated with the audio query comprises obtaining the vocalization nuance associated with the audio query from a third party service.

7. A computer readable medium bearing computer executable instructions which, when executed on a computing system comprising at least a processor executing the instructions retrieved from the medium, carry out a method for responding to an audio query from a computer user, the method comprising:

receiving an audio query from a computer user;

obtaining a textual representation of the audio query and vocalization nuances associated with the audio query;

obtaining confidence/risk value pairs corresponding the vocalization nuances, wherein the confidence value of a confidence/risk value pair identifies a confidence that the corresponding vocalization nuance correctly reflects a corresponding aspect of the computer user, and wherein the risk value of a confidence/risk value pair identifies a risk of providing offensive content to the computer user of search results selected in conjunction with the identified vocalization nuance irrespective of whether the vocalization nuance accurately reflects some attribute of the computer user;

identifying a plurality of search results corresponding to the textual representation of the audio query based, in part, on the identified vocalization nuance according to the confidence/risk value pairs associated with the identified vocalization nuance;

generating a search results presentation based on the identified plurality of search results; and

returning the search results presentation to the computer user in response to receiving the audio query.

8. The computer readable medium of claim 7, wherein obtaining a textual translation of the audio query comprises accessing an external translation service to obtain the textual translation of the audio query.

9. The computer readable medium of claim 7, wherein obtaining vocalization nuances associated with the audio query comprises accessing an external service to obtain the vocalization nuances associated with the audio query.

10. The computer readable medium of claim 7, wherein obtaining confidence/risk value pairs corresponding the vocalization nuances comprises accessing an external service to obtain the confidence/risk value pairs corresponding the vocalization nuances.

11. The computer readable medium of claim 7, wherein identifying a plurality of search results corresponding to the textual representation of the audio query in conjunction with the vocalization nuances associated with the audio query comprises:

determining that a confidence/risk value pair associated with at least one vocalization nuance falls within predetermined acceptable thresholds;

updating the textual representation of the audio query according to the at least one vocalization nuance; and

identifying a plurality of search results corresponding to the updated textual representation of the audio query in conjunction with the vocalization nuances.

12. The computer readable medium of claim 7, wherein generating a search results presentation based on the identified plurality of search results comprises generating an audio presentation of at least some of the identified search results.

13. The computer readable medium of claim 7, wherein the method further comprises:

identifying an advertisement to be included with the search results presentation, the advertisement being selected according to at least one of the vocalization nuances; and

wherein generating a search results presentation based on the identified plurality of search results comprises generating a visual presentation based on the identified plurality of search results and the identified advertisement.

14. A computer system for responding to an audio query, the system comprising a processor and a memory, wherein the processor executes instructions stored in the memory as part of or in conjunction with additional components to respond to receiving an audio query from a computer user, and causes the processor to:

enable a network communication component to communicate between the computer system and other computing devices over a network;

translate an audio query to a textual representation of the audio query;

identify vocalization nuances of the audio query;

generate a confidence value for each vocalization nuance of the identified vocalization nuances, wherein each confidence value is a part of a confidence/risk values pair corresponding to an vocalization nuance, and wherein each confidence value represents a confidence that a corresponding vocalization nuance correctly reflects a corresponding aspect of the computer user; and

provide a risk value corresponding to each vocalization nuance of the identified vocalization nuances, wherein each risk value is a part of a confidence/risk values pair corresponding to a vocalization nuance irrespective of whether the vocalization nuance accurately reflects some attribute of the computer user, and wherein each risk value represents a risk that content selected based on a corresponding vocalization nuance may be offensive;

identify a plurality of search results in response to receiving the audio query from a content data store and based, in part, on the vocalization nuances of the audio query and the corresponding confidence/risk value pairs of the vocalization nuances; and

generate a search results presentation for the computer user based on the identified plurality of search results and in light of the vocalization nuances of the audio query.

15. The computer system of claim 14, wherein the executed instructions further cause the processor to identify a plurality of search results in response to receiving the audio query from a content data store and in light of the vocalization nuances of the audio query whose confidence/risk value pairs fall within predetermined thresholds.