Background audio identification for speech disambiguation

• US 9,123,338 B1
• Filed: 03/14/2013
• Issued: 09/01/2015
• Est. Priority Date: 06/01/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Taubman, Gabriel, Lee, John J., Sanders, Jason
• Primary Examiner(s): Chawan, Vijay B
• Application Number:
  US13/804,986
• Time in Patent Office: 901 Days
• Field of Search: 704/233, 704/235, 340/474, 340/463, 340/460, 340/815.69, 379/93.17, 379 8801- 8804
• US Class Current: 1/1
• CPC Class Codes: G06F 16/685 using automatically derived transcript of audio data, e.g. lyrics speech recognition G10L15/00
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/225 Feedback of the input speech
• CPC Class Codes: G10L 21/0208 Noise filtering
• CPC Class Codes: G10L 21/0272 Voice signal separating
• CPC Class Codes: G10L 25/48 specially adapted for particular use
• CPC Class Codes: H04M 2201/40 using speech recognition
• CPC Class Codes: H04M 2203/352 In-call/conference information service
• CPC Class Codes: H04M 3/4936 Speech interaction details speech recognition per se G10L15/00

Abstract

Implementations relate to techniques for providing context-dependent search results. A computer-implemented method includes receiving an audio stream at a computing device during a time interval, the audio stream comprising user speech data and background audio, separating the audio stream into a first substream that includes the user speech data and a second substream that includes the background audio, identifying concepts related to the background audio, generating a set of terms related to the identified concepts, influencing a speech recognizer based on at least one of the terms related to the background audio, and obtaining a recognized version of the user speech data using the speech recognizer.

Claims

1. A computer-implemented method, the method comprising:

receiving an audio stream at a computing device during a time interval, the audio stream comprising user speech data and background audio;

separating the audio stream into a first substream that comprises the user speech data and a second substream that comprises the background audio;

identifying concepts related to the background audio;

generating a set of terms related to the identified concepts;

influencing a speech recognizer based on at least one of the terms related to the background audio; and

obtaining a recognized version of the user speech data using the speech recognizer.

2. The method of claim 1, further comprising:

recognizing voice pattern information associated with a user; and

using the voice pattern information to separate the user speech data from the background audio.

3. The method of claim 1, wherein identifying concepts related to the background audio comprises:

recognizing at least a portion of the background audio by matching it to an acoustic fingerprint; and

identifying concepts related to the background audio comprising concepts associated with the acoustic fingerprint.

4. The method of claim 3, wherein the acoustic fingerprint is an acoustic fingerprint for an audio sample from a media recording.

5. The method of claim 1, wherein generating a set of terms related to the background audio comprises:

generating a set of terms based on querying a conceptual expansion database based on the concepts related to the background audio.

6. The method of claim 5, wherein the conceptual expansion database contains a plurality of documents related to a plurality of concepts and the generated set of terms comprises terms that are determined to be keywords for the documents in the conceptual expansion database.

7. The method of claim 1, wherein the first substream only comprises audio from the audio stream obtained during a portion of the time interval.

8. The method of claim 1, wherein influencing the language model comprises increasing the probability that at least one of the terms related to the background audio will be obtained.

9. The method of claim 8, further comprising:

measuring the relevance of a term related to the background audio; and

increasing the probability that at least one of the terms related to the background audio based on the measured relevance.

10. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

receiving an audio stream during a time interval, the audio stream comprising user speech data and background audio;

separating the audio stream into a first substream that comprises the user speech data and a second substream that comprises the background audio;

identifying concepts related to the background audio;

generating a set of terms related to the identified concepts;

influencing a speech recognizer based on at least one of the terms related to the background audio; and

obtaining a recognized version of the user speech data using the speech recognizer.

11. The system of claim 10, the operations further comprising:

recognizing voice pattern information associated with a user; and

using the voice pattern information to separate the user speech data from the background audio.

12. The system of claim 10, wherein identifying concepts related to the background audio comprises:

recognizing at least a portion of the background audio by matching it to an acoustic fingerprint; and

identifying concepts related to the background audio comprising concepts associated with the acoustic fingerprint.

13. The system of claim 12, wherein the acoustic fingerprint is an acoustic fingerprint for an audio sample from a media recording.

14. The system of claim 10, wherein generating a set of terms related to the background audio comprises:

generating a set of terms based on querying a conceptual expansion database based on the concepts related to the background audio.

15. The system of claim 14, wherein the conceptual expansion database contains a plurality of documents related to a plurality of concepts and the generated set of terms comprises terms that are determined to be keywords for the documents in the conceptual expansion database.

16. The system of claim 10, wherein the first substream only comprises audio from the audio stream obtained during a portion of the time interval.

17. The system of claim 10, wherein influencing the language model comprises increasing the probability that at least one of the terms related to the background audio will be obtained.

18. The system of claim 17, the operations further comprising:

measuring the relevance of a term related to the background audio; and

increasing the probability that at least one of the terms related to the background audio based on the measured relevance.

19. A computer-readable storage device storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

receiving an audio stream at a computing device during a time interval, the audio stream comprising user speech data and background audio;

separating the audio stream into a first substream that comprises the user speech data and a second substream that comprises the background audio;

identifying concepts related to the background audio;

generating a set of terms related to the identified concepts;

influencing a speech recognizer based on at least one of the terms related to the background audio; and

obtaining a recognized version of the user speech data using the speech recognizer.

20. The computer-readable storage device of claim 19, the operations further comprising:

recognizing voice pattern information associated with a user; and

using the voice pattern information to separate the user speech data from the background audio.