Third-party audio subsystem enhancement

• US 8,983,845 B1
• Filed: 03/26/2010
• Issued: 03/17/2015
• Est. Priority Date: 03/26/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Kristjansson, Trausti, Talkin, David
• Primary Examiner(s): Godbold, Douglas
• Assistant Examiner(s): ORTIZ SANCHEZ, MICHAEL
• Application Number:
  US12/732,788
• Time in Patent Office: 1,817 Days
• Field of Search: 704/231, 704/233, 704/243, 704/246, 704/251, 704/257, 704/270, 704/275, 381/58
• US Class Current: 704/275
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 21/00 Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility G10L19/00 takes precedence
• CPC Class Codes: H04L 41/0803 Configuration setting
• CPC Class Codes: H04M 1/24 Arrangements for testing

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing audio subsystem enhancement. In one aspect, a method includes: receiving a voice search query by an automatic speech recognition (ASR) engine that processes voice search queries for a search engine, wherein the voice search query includes an audio signal that corresponds to an utterance, and a test flag that indicates that an audio test is being performed; performing speech recognition on the audio signal to select one or more textual, candidate transcriptions that match the utterance; generating, in response to receiving the test flag, one or more audio quality metrics using the audio signal; and generating a response to the voice search query by the ASR engine, wherein the response references one or more of the candidate transcriptions and one or more of the audio quality metrics.

Claims

1. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

receiving, from a mobile device or a mockup of the mobile device and by an automatic speech recognition (ASR) engine that processes voice search queries for a search engine, (i) a voice search query and (ii) data indicating that the voice search query represents audio test data for a test of an audio quality of an audio signal output by a test audio subsystem configuration of the mobile device or the mockup of the mobile device, wherein the voice search query includes a pre-recorded test utterance used for testing mobile devices or mockups of mobile devices;

generating, in response to receiving the data indicating that the voice search query represents audio test data for a test of the audio quality of the audio signal output by a test audio subsystem configuration of the mobile device or on the mockup of the mobile device, one or more audio quality metrics that reflect the audio quality of the audio signal output by the test audio subsystem configuration of the mobile device or the mockup of the mobile device; and

generating a response to the voice search query by the ASR engine, wherein the response references at least one of the one or more audio quality metrics.

2. The system of claim 1, wherein generating one or more audio quality metrics that reflect the audio quality of the audio signal output by the test audio subsystem configuration of the mobile device or the mockup of the mobile device further comprises comparing the audio signal to the pre-recorded test utterance.

3. The system of claim 2, wherein the operations further comprise:

receiving audio test data including the pre-recorded test utterance and a transcription associated with the pre-recorded test utterance.

4. The system of claim 1, comprising:

performing speech recognition on the audio signal output by a test audio subsystem configuration of the mobile device or the mockup of the mobile device to select one or more textual, candidate transcriptions that match the pre-recorded test utterance; and

establishing a speech recognition confidence value for each candidate transcription.

5. The system of claim 4, wherein performing speech recognition on the audio signal output by a test audio subsystem configuration of the mobile device or the mockup of the mobile device to select one or more textual, candidate transcriptions that match the pre-recorded test utterance further comprises performing speech recognition on the audio signal output by a test audio subsystem configuration of the mobile device or the mockup of the mobile device to select an n-best of the candidate transcriptions that have the n-highest speech recognition confidence values.

6. The system of claim 1, wherein generating one or more audio quality metrics that reflect the audio quality of the audio signal output by the test audio subsystem configuration of the mobile device or the mockup of the mobile device further comprises determining an amount of clipping, a gain, a signal-to-noise-ratio (SNR), an onset point, or an offset point of the audio signal.

7. The system of claim 1, wherein the voice search query further includes data that references a third party who is performing the audio test.

8. The system of claim 1, wherein the voice search query further includes data that references a type of audio test being performed.

9. The system of claim 1, wherein:

the voice search query further includes data that references a type of a mobile device; and

the operations further comprise updating an acoustic model that is specific to the type of the mobile device, using the audio signal output by a test audio subsystem configuration of the mobile device or the mockup of the mobile device.

10. The system of claim 1, wherein:

the voice search query further includes data that references a term that is actually being uttered by the pre-recorded test utterance.

11. The system of claim 1, wherein the operations further comprise:

providing the response to a mobile device from which the voice search query originated.

12. The system of claim 1, the operations further comprising based on the one or more audio quality metrics, determining adjustments for the a test audio subsystem configuration of the mobile device or the mockup of the mobile device.

13. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

receiving, from a mobile device or a mockup of the mobile device and by an automatic speech recognition (ASR) engine that processes voice search queries for a search engine, (i) a voice search query and (ii) data indicating that the voice search query represents audio test data for a test of an audio quality of an audio signal output by a test audio subsystem configuration of the mobile device or the mockup of the mobile device, wherein the voice search query includes a pre-recorded test utterance used for testing mobile devices or mockups of mobile devices;

generating, in response to receiving the data indicating that the voice search query represents audio test data for a test of the audio quality of the audio signal output by a test audio subsystem configuration of the mobile device or on the mockup of the mobile device, one or more audio quality metrics that reflect the audio quality of the audio signal output by the test audio subsystem configuration of the mobile device or the mockup of the mobile device; and

generating a response to the voice search query by the ASR engine, wherein the response references at least one of the one or more audio quality metrics.

14. The medium of claim 13, wherein generating one or more audio quality metrics that reflect the audio quality of the audio signal output by the test audio subsystem configuration of the mobile device or the mockup of the mobile device further comprises comparing the audio signal to the pre-recorded test utterance.

15. The medium of claim 13, wherein generating one or more quality metrics using the audio signal further comprises determining an amount of clipping, a gain, a signal-to-noise-ratio (SNR), an onset point, or an offset point of the audio signal.

16. The medium of claim 13, wherein the voice search query further includes data that references a term that is actually being uttered by the pre-recorded test utterance.

17. A computer-implemented method comprising:

receiving, from a mobile device or a mockup of the mobile device and by an automatic speech recognition (ASR) engine that processes voice search queries for a search engine, (i) a voice search query and (ii) data indicating that the voice search query represents audio test data for a test of an audio quality of an audio signal output by a test audio subsystem configuration of the mobile device or the mockup of the mobile device, wherein the voice search query includes a pre-recorded test utterance used for testing mobile devices or mockups of mobile devices;

generating, in response to receiving the data indicating that the voice search query represents audio test data for a test of the audio quality of the audio signal output by a test audio subsystem configuration of the mobile device or on the mockup of the mobile device, one or more audio quality metrics that reflect the audio quality of the audio signal output by the test audio subsystem configuration of the mobile device or the mockup of the mobile device; and

generating a response to the voice search query by the ASR engine, wherein the response references at least one of the one or more audio quality metrics.

18. The method of claim 17, wherein the voice search query is being performed by a manufacturer of the mobile device.

19. The method of claim 17, wherein generating one or more audio quality metrics that reflect the audio quality of the audio signal output by the test audio subsystem configuration of the mobile device or the mockup of the mobile device further comprises comparing the audio signal to the pre-recorded test utterance.

20. The method of claim 17, wherein the voice search query further includes data that references a term that is actually being uttered by the pre-recorded test utterance.