Recognizing speech in the presence of additional audio

• US 9,601,116 B2
• Filed: 04/07/2016
• Issued: 03/21/2017
• Est. Priority Date: 02/14/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Melendo Casado, Diego, Moreno, Ignacio Lopez, Gonzalez-Dominguez, Javier
• Primary Examiner(s): SINGH, SATWANT K
• Application Number:
  US15/093,309
• Publication Number:
  US 20160225373A1
• Time in Patent Office: 348 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence
• CPC Class Codes: G10L 15/222 Barge in, i.e. overridable guidance for interrupting prompts
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 17/00 Speaker identification or verification techniques
• CPC Class Codes: G10L 17/06 Decision making techniques; Pattern matching strategies
• CPC Class Codes: G10L 21/034 Automatic adjustment
• CPC Class Codes: G10L 25/84 for discriminating voice from noise
• CPC Class Codes: H03G 3/3005 in amplifiers suitable for low-frequencies, e.g. audio amplifiers H03G3/32, H03G3/34 take precedence

Abstract

The technology described in this document can be embodied in a computer-implemented method that includes receiving, at a processing system, a first signal including an output of a speaker device and an additional audio signal. The method also includes determining, by the processing system, based at least in part on a model trained to identify the output of the speaker device, that the additional audio signal corresponds to an utterance of a user. The method further includes initiating a reduction in an audio output level of the speaker device based on determining that the additional audio signal corresponds to the utterance of the user.

Claims

1. A computer-implemented method comprising:

receiving, by a mobile device, an audio signal;

determining, by the mobile device and using a model that is trained to detect a presence of a synthesized voice and a model that is trained to detect a presence of a user'"'"'s voice, that the audio signal likely includes both the synthesized voice and the user'"'"'s voice;

in response to determining, by the mobile device and using a model that is trained to detect a presence of a synthesized voice and a model that is trained to detect a presence of a user'"'"'s voice, that the audio signal likely includes both the synthesized voice and the user'"'"'s voice, suppressing, by the mobile device, operation of a speech synthesis module implemented by the mobile device;

after suppressing operation of the speech synthesis module, obtaining, by the mobile device, a transcription corresponding to the audio signal from an automated speech recognizer; and

providing, by the mobile device, the transcription for output.

2. The method of claim 1, wherein suppressing, by the mobile device, operation of a speech synthesis module implemented by the mobile device comprises initiating a reduction in an audio output level of the speech synthesis module.

3. The method of claim 2, wherein initiating a reduction in an audio output level of the speech synthesis module comprises interrupting output of the speech synthesis module.

4. The method of claim 1, further comprising:

obtaining a first vector corresponding to at least a portion of the audio signal;

comparing the first vector to a second vector corresponding to the model that is trained to detect a presence of a synthesized voice; and

determining that the audio signal comprises additional audio other than the synthesized voice based on a result of the comparison satisfying a threshold.

5. The method of claim 1, further comprising:

obtaining a first vector corresponding to at least a portion of the audio signal; and

determining that the audio signal comprises additional audio other than the synthesized voice based on the first vector satisfying a threshold.

6. The method of claim 1, wherein each of the model that is trained to detect a presence of a synthesized voice and the model that is trained to detect a presence of a user'"'"'s voice is an i-vector based model.

7. The method of claim 1, wherein each of the model that is trained to detect a presence of a synthesized voice and the model that is trained to detect a presence of a user'"'"'s voice is a neural network based model.

8. A non-transitory computer readable storage device storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

receiving, by a mobile device, an audio signal;

determining, by the mobile device and using a model that is trained to detect a presence of a synthesized voice and a model that is trained to detect a presence of a user'"'"'s voice, that the audio signal likely includes both the synthesized voice and the user'"'"'s voice;

in response to determining, by the mobile device and using a model that is trained to detect a presence of a synthesized voice and a model that is trained to detect a presence of a user'"'"'s voice, that the audio signal likely includes both the synthesized voice and the user'"'"'s voice, suppressing, by the mobile device, operation of a speech synthesis module implemented by the mobile device;

after suppressing operation of the speech synthesis module, obtaining, by the mobile device, a transcription corresponding to the audio signal from an automated speech recognizer; and

providing, by the mobile device, the transcription for output.

9. The computer readable storage device of claim 8, wherein suppressing, by the mobile device, operation of a speech synthesis module implemented by the mobile device comprises initiating a reduction in an audio output level of the speech synthesis module.

10. The computer readable storage device of claim 9, wherein initiating a reduction in an audio output level of the speech synthesis module comprises interrupting output of the speech synthesis module.

11. The computer readable storage device of claim 8, further comprising:

obtaining a first vector corresponding to at least a portion of the audio signal;

comparing the first vector to a second vector corresponding to the model that is trained to detect a presence of a synthesized voice; and

determining that the audio signal comprises additional audio other than the synthesized voice based on a result of the comparison satisfying a threshold.

12. The computer readable storage device of claim 8, further comprising:

obtaining a first vector corresponding to at least a portion of the audio signal; and

determining that the audio signal comprises additional audio other than the synthesized voice based on the first vector satisfying a threshold.

13. The computer readable storage device of claim 8, wherein each of the model that is trained to detect a presence of a synthesized voice and the model that is trained to detect a presence of a user'"'"'s voice is an i-vector based model.

14. The computer readable storage device of claim 8, wherein each of the model that is trained to detect a presence of a synthesized voice and the model that is trained to detect a presence of a user'"'"'s voice is a neural network based model.

15. A system comprising:

one or more computers; and

one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

receiving, by a mobile device, an audio signal;

determining, by the mobile device and using a model that is trained to detect a presence of a synthesized voice and a model that is trained to detect a presence of a user'"'"'s voice, that the audio signal likely includes both the synthesized voice and the user'"'"'s voice;

in response to determining, by the mobile device and using a model that is trained to detect a presence of a synthesized voice and a model that is trained to detect a presence of a user'"'"'s voice, that the audio signal likely includes both the synthesized voice and the user'"'"'s voice, suppressing, by the mobile device, operation of a speech synthesis module implemented by the mobile device;

after suppressing operation of the speech synthesis module, obtaining, by the mobile device, a transcription corresponding to the audio signal from an automated speech recognizer; and

providing, by the mobile device, the transcription for output.

16. The system of claim 15, wherein suppressing, by the mobile device, operation of a speech synthesis module implemented by the mobile device comprises initiating a reduction in an audio output level of the speech synthesis module.

17. The system of claim 16, wherein initiating a reduction in an audio output level of the speech synthesis module comprises interrupting output of the speech synthesis module.

18. The system of claim 15, further comprising:

obtaining a first vector corresponding to at least a portion of the audio signal;

comparing the first vector to a second vector corresponding to the model that is trained to detect a presence of a synthesized voice; and

determining that the audio signal comprises additional audio other than the synthesized voice based on a result of the comparison satisfying a threshold.

19. The system of claim 15, further comprising:

obtaining a first vector corresponding to at least a portion of the audio signal; and

determining that the audio signal comprises additional audio other than the synthesized voice based on the first vector satisfying a threshold.

20. The system of claim 15, wherein each of the model that is trained to detect a presence of a synthesized voice and the model that is trained to detect a presence of a user'"'"'s voice is one of:

an i-vector based model and a neural network based model.