Metadata-based weighting of geotagged environmental audio for enhanced speech recognition accuracy

• US 8,428,940 B2
• Filed: 08/01/2012
• Issued: 04/23/2013
• Est. Priority Date: 04/14/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Kristjansson, Trausti T., Lloyd, Matthew I.
• Primary Examiner(s): Smits, Talivaldis Ivars
• Application Number:
  US13/564,636
• Publication Number:
  US 20120296643A1
• Time in Patent Office: 265 Days
• Field of Search: None
• US Class Current: 704/227
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence
• CPC Class Codes: G10L 21/0208 Noise filtering

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for enhancing speech recognition accuracy. In one aspect, a method includes receiving an audio signal that corresponds to an utterance recorded by a mobile device, determining a geographic location associated with the mobile device, identifying a set of geotagged audio signals that correspond to environmental audio associated with the geographic location, weighting each geotagged audio signal of the set of geotagged audio signals based on metadata associated with the respective geotagged audio signal, and using the set of weighted geotagged audio signals to perform noise compensation on the audio signal that corresponds to the utterance.

Claims

1. A system comprising:

one or more computers; and

a computer-readable medium coupled to the one or more computers having instructions stored thereon which, when executed by the one or more computers, cause the one or more computers to perform operations comprising;

receiving an audio signal that corresponds to an utterance recorded by a mobile device;

determining a geographic location associated with the mobile device;

identifying a set of geotagged audio signals that correspond to environmental audio associated with the geographic location;

weighting each geotagged audio signal of the set of geotagged audio signals based on metadata associated with the respective geotagged audio signal; and

using the set of weighted geotagged audio signals to perform noise compensation on the audio signal that corresponds to the utterance.

2. The system of claim 1, wherein the operations further comprise receiving geotagged audio signals, each geotagged audio signal including the metadata associated with the respective geotagged audio signal.

3. The system of claim 1, wherein the operations further comprise:

receiving geotagged audio signals;

generating the metadata for each geotagged audio signal; and

associating the metadata with the respective geotagged audio signal.

4. The system of claim 1, wherein weighting each geotagged audio signal comprises weighting each geotagged audio signal of the set of geotagged audio signals based on the metadata associated with the respective geotagged audio signal and metadata associated with the audio signal that corresponds to an utterance.

5. The system of claim 1, wherein the metadata indicates whether background voices are present in the respective geotagged audio signal, and weighting each geotagged audio signal comprises weighting each geotagged audio signal based on whether background voices are present in the respective geotagged audio signal.

6. The system of claim 1, wherein the metadata indicates an accuracy of geographic location information associated with the respective geotagged audio signal, and weighting each geotagged audio signal comprises weighting each geotagged audio signal based on the accuracy of the geographic location information associated with the respective geotagged audio signal.

7. The system of claim 1, wherein the metadata indicates a length of the respective geotagged audio signal, and weighting each geotagged audio signal comprises weighting each geotagged audio signal based on the length of the respective geotagged audio signal.

8. The system of claim 1, wherein the metadata indicates a quality of the respective geotagged audio signal, and weighting each geotagged audio signal comprises weighting each geotagged audio signal based on the quality of the respective geotagged audio signal.

9. The system of claim 1, wherein the metadata indicates a geographic location associated with the respective geotagged audio signal, and weighting each geotagged audio signal comprises weighting each geotagged audio signal based on proximity of the geographic location associated with the respective geotagged audio signal to the geographic location associated with the mobile device.

10. The system of claim 1, wherein the metadata indicates a geographic location associated with the respective geotagged audio signal, and weighting each geotagged audio signal comprises weighting each geotagged audio signal based on a level of detail defining the geographic location associated with the respective geotagged audio signal.

11. The system of claim 1, wherein the metadata indicates a geographic location type associated with the respective geotagged audio signal, and weighting each geotagged audio signal comprises weighting each geotagged audio signal based on the geographic location type associated with the respective geotagged audio signal.

12. The system of claim 1, wherein the metadata indicates a type of mobile device associated with the respective geotagged audio signal, and weighting each geotagged audio signal comprises weighting each geotagged audio signal based on the type of mobile device associated with the geotagged audio signal.

13. The system of claim 1, wherein the metadata indicates a time of day associated with the respective geotagged audio signal, and weighting each geotagged audio signal comprises weighting each geotagged audio signal based on the time of day associated with the respective geotagged audio signal.

14. The system of claim 1, wherein the metadata indicates a day of a week associated with the respective geotagged audio signal, and weighting each geotagged audio signal comprises weighting each geotagged audio signal based on the day of the week associated with the respective geotagged audio signal.

15. The system of claim 1, wherein weighting each geotagged audio signal further comprises weighting each geotagged audio signal after receiving the audio signal that corresponds to the utterance.

16. The system of claim 1, wherein using the set of weighted geotagged audio signals to perform noise compensation comprises:

using the set of weighted geotagged audio signals to generate a noise model for the geographic location; and

performing noise compensation on the audio signal that corresponds to the utterance using the generated noise model.

17. The system of claim 1, wherein the operations further comprise performing speech recognition on the utterance using the noise-compensated audio signal.

18. The system of claim 1, wherein using the set of weighted geotagged audio signals to perform noise compensation further comprises performing noise compensation on the audio signal that corresponds to the utterance using the set of weighted geotagged audio signals and using an environmental audio portion of the audio signal that corresponds to the utterance.

19. A computer storage medium encoded with a computer program, the program comprising instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

receiving an audio signal that corresponds to an utterance recorded by a mobile device;

determining a geographic location associated with the mobile device;

identifying a set of geotagged audio signals that correspond to environmental audio associated with the geographic location;

weighting each geotagged audio signal of the set of geotagged audio signals based on metadata associated with the respective geotagged audio signal; and

using the set of weighted geotagged audio signals to perform noise compensation on the audio signal that corresponds to the utterance.

20. A computer-implemented method comprising:

receiving an audio signal that corresponds to an utterance recorded by a mobile device;

determining a geographic location associated with the mobile device;

identifying a set of geotagged audio signals that correspond to environmental audio associated with the geographic location;

weighting each geotagged audio signal of the set of geotagged audio signals based on metadata associated with the respective geotagged audio signal; and

using the set of weighted geotagged audio signals to perform noise compensation on the audio signal that corresponds to the utterance.

21. The computer-implemented method of claim 20, wherein the operations further comprise receiving geotagged audio signals, each geotagged audio signal including the metadata associated with the respective geotagged audio signal.

22. The computer-implemented method of claim 20, wherein the operations further comprise:

receiving geotagged audio signals;

generating the metadata for each geotagged audio signal; and

associating the metadata with the respective geotagged audio signal.

23. The computer-implemented method of claim 20, wherein the metadata indicates whether background voices are present in the respective geotagged audio signal, and weighting each geotagged audio signal comprises weighting each geotagged audio signal based on whether background voices are present in the respective geotagged audio signal.

24. The computer-implemented method of claim 20, wherein the metadata indicates an accuracy of geographic location information associated with the respective geotagged audio signal, and weighting each geotagged audio signal comprises weighting each geotagged audio signal based on the accuracy of the geographic location information associated with the respective geotagged audio signal.

25. The computer-implemented method of claim 20, wherein the metadata indicates a length of the respective geotagged audio signal, and weighting each geotagged audio signal comprises weighting each geotagged audio signal based on the length of the respective geotagged audio signal.

26. The computer-implemented method of claim 20, wherein the metadata indicates a quality of the respective geotagged audio signal, and weighting each geotagged audio signal comprises weighting each geotagged audio signal based on the quality of the respective geotagged audio signal.

27. The computer-implemented method of claim 20, wherein the metadata indicates a geographic location associated with the respective geotagged audio signal, and weighting each geotagged audio signal comprises weighting each geotagged audio signal based on proximity of the geographic location associated with the respective geotagged audio signal to the geographic location associated with the mobile device.

28. The computer-implemented method of claim 20, wherein the metadata indicates a geographic location associated with the respective geotagged audio signal, and weighting each geotagged audio signal comprises weighting each geotagged audio signal based on a level of detail defining the geographic location associated with the respective geotagged audio signal.

29. The computer-implemented method of claim 20, wherein the metadata indicates a geographic location type associated with the respective geotagged audio signal, and weighting each geotagged audio signal comprises weighting each geotagged audio signal based on the geographic location type associated with the respective geotagged audio signal.

30. The computer-implemented method of claim 20, wherein the metadata indicates a type of mobile device associated with the respective geotagged audio signal, and weighting each geotagged audio signal comprises weighting each geotagged audio signal based on the type of mobile device associated with the geotagged audio signal.