Voice controlled audio recording system with adjustable beamforming

• US 9,984,675 B2
• Filed: 07/31/2013
• Issued: 05/29/2018
• Est. Priority Date: 05/24/2013
• Status: Active Grant

• Current Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Original Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Inventors: Schuster, Adrian M, Ivanov, Plamen A
• Primary Examiner(s): Shah, Paras D
• Assistant Examiner(s): Ogunbiyi, Oluwadamilola M
• Application Number:
  US13/956,335
• Publication Number:
  US 20140350926A1
• Time in Patent Office: 1,763 Days
• Field of Search: 704233
• CPC Class Codes: G10L 15/00 Speech recognition G10L17/00 takes precedence
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 21/0208 Noise filtering
• CPC Class Codes: H04R 1/406 microphones
• CPC Class Codes: H04R 2410/01 Noise reduction using microphones having different directional characteristics
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

A method of operation beamforms a plurality of microphone outputs to obtain a plurality of virtual microphone audio channels with at least one audio output channel and at least one audio control channel. The method performs voice recognition on the audio control channel to detect voice commands for controlling audio output channel attributes, and adjusts an audio channel attribute in response to detecting a voice command. Adjusting an attribute of the audio channel may be accomplished by, for example, controlling one or more parameters of an adjustable beamformer. The detected voice commands for controlling audio channel attributes may include voice commands for controlling audio sensitivity zooming, panning in a specified direction, focusing on a specified direction, blocking a specified direction, mixing a narrator'"'"'s voice, blocking a narrator'"'"'s voice, or reducing background noise. An apparatus that performs the method of operation is also disclosed.

Claims

1. A computer-implemented method, comprising:

receiving, by a beamformer of a device, and from one or more physical microphones, a plurality of audio signals;

generating, by the beamformer, and based on the plurality of audio signals, first and second beamforms that respectively correspond to first and second virtual microphones, the first virtual microphone being configured to receive data corresponding to audio control commands, and the second virtual microphone being configured to receive data that is recorded or transmitted by the device;

determining, by a voice command recognition module that is operatively coupled to the beamformer of the device, that audio data received through the first virtual microphone corresponds to a command directly specifying an adjustment to an attribute of a second virtual microphone; and

responsive to determining that the audio data received through the first virtual microphone corresponds to the command directly specifying the adjustment to the attribute of the second virtual microphone, adjusting the attribute of the second virtual microphone corresponding to the second beamform generated by the beamformer, wherein the attribute of the second virtual microphone comprises at least one non-directional audio attribute of the second virtual microphone.

2. The method of claim 1, wherein the attribute of the second virtual microphone comprises a signal conditioning algorithm applied to the data that is to be recorded or transmitted by the device.

3. The method of claim 2, wherein the signal conditioning algorithm comprises a noise suppression algorithm, a signal gain algorithm, a signal equalization algorithm, or an automatic gain control algorithm.

4. The method of claim 1, further comprising:

responsive to determining, by the voice command recognition module of the device, that the audio data received through the first virtual microphone corresponds to the command for adjusting the attribute of the second virtual microphone;

activating at least one adaptive filter; and

performing adaptive filtering using the at least one adaptive filter to attenuate the command in the second virtual microphone.

5. A system comprising:

one or more physical microphones;

one or more computers; and

one or more storage devices storing instructions that, when executed by the one or more computers, cause the one or more computers to;

receive, by a beamformer, and from the one or more physical microphones, a plurality of audio signals;

generate, by the beamformer, and based on the plurality of audio signals, first and second beamforms that respectively correspond to first and second virtual microphones, the first virtual microphone being configured to receive data corresponding to audio control commands, and the second virtual microphone being configured to receive data that is recorded or transmitted by the one or more computers;

determine, by a voice command recognition module that is operatively coupled to the beamformer, that audio data received through the first virtual microphone corresponds to a command directly specifying an adjustment to an attribute of a second virtual microphone; and

responsive to determining that the audio data received through the first virtual microphone corresponds to the command directly specifying the adjustment to the attribute of the second virtual microphone, adjust the attribute of the second virtual microphone corresponding to the second beamform generated by the beamformer, wherein the attribute of the second virtual microphone comprises at least one non-directional audio attribute of the second virtual microphone.

6. The system of claim 5, wherein the attribute of the second virtual microphone comprises a signal conditioning algorithm applied to the data that is to be recorded or transmitted.

7. The system of claim 6, wherein the signal conditioning algorithm comprises a noise suppression algorithm, a signal gain algorithm, a signal equalization algorithm, or an automatic gain control algorithm.

8. The system of claim 5, comprising:

responsive to determining, by the voice command recognition module, that the audio data received through the first virtual microphone corresponds to the command for adjusting the attribute of the second virtual microphone, the instructions further cause the one or more computers to;

activate at least one adaptive filter; and

perform adaptive filtering using the at least one adaptive filter to attenuate the command in the second virtual microphone.

9. A non-transitory computer-readable medium storing instructions executable by one or more computers that, upon execution, cause the one or more computers to perform operations comprising:

receiving, by a beamformer, and from one or more physical microphones, a plurality of audio signals;

generating, by the beamformer, and based on the plurality of audio signals, first and second beamforms that respectively correspond to first and second virtual microphones, the first virtual microphone being configured to receive data corresponding to audio control commands, and the second virtual microphone being configured to receive data that is recorded or transmitted by the one or more computers;

determining, by a voice command recognition module that is operatively coupled to the beamformer, that audio data received through the first virtual microphone corresponds to a command directly specifying an adjustment to an attribute of a second virtual microphone; and

responsive to determining that the audio data received through the first virtual microphone corresponds to the command directly specifying the adjustment to the attribute of the second virtual microphone, adjusting the attribute of the second virtual microphone corresponding to the second beamform generated by the beamformer, wherein the attribute of the second virtual microphone comprises at least one non-directional audio attribute of the second virtual microphone.

10. The non-transitory computer-readable storage medium of claim 9, wherein the attribute of the second virtual microphone comprises a signal conditioning algorithm applied to the data that is to be recorded or transmitted by the one or more computers.

11. The non-transitory computer-readable storage medium of claim 10, wherein the signal conditioning algorithm comprises a noise suppression algorithm, a signal gain algorithm, a signal equalization algorithm, or an automatic gain control algorithm.

12. The non-transitory computer-readable storage medium of claim 9, wherein the operations further comprise:

responsive to determining, by the voice command recognition module, that the audio data received through the first virtual microphone corresponds to the command for adjusting the attribute of the second virtual microphone;

activating at least one adaptive filter; and

performing adaptive filtering using the at least one adaptive filter to attenuate the command in the second virtual microphone.