Apparatus and Method for Beamforming to Obtain Voice and Noise Signals

• US 20140278394A1
• Filed: 07/31/2013
• Published: 09/18/2014
• Est. Priority Date: 03/12/2013
• Status: Active Grant

• Current Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Original Assignee: Motorola Mobility LLC (Lenovo Group Ltd.)
• Inventors: Davis, Giles T., Ivanov, Plamen A., Ramabadran, Tenkasi V., Singaraju, Snehitha, Bastyr, Kevin John, Oliveira, Rivanaldo S.
• Granted Patent: US 10,229,697 B2
• Time in Patent Office: Days
• US Class Current: 704/233
• CPC Class Codes: G10L 15/01 Assessment or evaluation of speech recognition systems
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence
• CPC Class Codes: G10L 2021/02165 Two microphones, one receiving mainly the noise signal and the other one mainly the speech signal
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 21/0208 Noise filtering
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

One method of operation includes beamforming a plurality of microphone outputs to obtain a plurality of virtual microphone audio channels. Each virtual microphone audio channel corresponds to a beamform. The virtual microphone audio channels include at least one voice channel and at least one noise channel. The method includes performing voice activity detection on the at least one voice channel and adjusting a corresponding voice beamform until voice activity detection indicates that voice is present on the at least one voice channel. Another method beamforms the plurality of microphone outputs to obtain a plurality of virtual microphone audio channels, where each virtual microphone audio channel corresponds to a beamform, and with at least one voice channel and at least one noise channel. The method performs voice recognition on the at least one voice channel and adjusts the corresponding voice beamform to improve a voice recognition confidence metric.

Claims

1. A method comprising:

beamforming a plurality of microphone outputs to obtain at least one virtual microphone channel;

performing voice recognition on the at least one virtual microphone channel; and

adjusting a corresponding beamform until voice recognition indicates one of the presence of voice on the at least one virtual microphone channel or that voice is not substantially present on the at least one virtual microphone channel.

2. The method of claim 1, wherein performing voice recognition further comprises:

performing voice recognition on the at least one virtual microphone channel using trained voice recognition that is trained to identify a specific speaker.

3. A method comprising:

beamforming a plurality of microphone outputs to obtain a plurality of virtual microphone audio channels, each virtual microphone audio channel corresponding to a beamform, comprising at least one voice channel and at least one noise channel;

performing voice activity detection on the at least one voice channel; and

adjusting a corresponding voice beamform until voice activity detection indicates that voice is present on the at least one voice channel.

4. The method of claim 3, further comprising:

performing voice activity detection on the at least one noise channel; and

adjusting a corresponding noise beamform until voice activity detection indicates that voice is not substantially present on the at least one noise channel.

5. The method of claim 4, further comprising:

performing energy estimation on the at least one noise channel; and

adjusting a corresponding noise beamform until energy estimation indicates that the at least one noise channel is receiving audio from a dominant audio energy source.

6. The method of claim 3, further comprising:

performing voice recognition on the at least one voice channel; and

adjusting the corresponding voice beamform to improve a voice recognition confidence metric of the voice recognition.

7. The method of claim 3, further comprising:

performing voice recognition on the at least one noise channel; and

adjusting the corresponding noise beamform to decrease a voice recognition confidence metric of the voice recognition.

8. The method of claim 7, wherein performing voice recognition on the at least one noise channel, comprises:

performing voice recognition on the at least one noise channel using trained voice recognition that is trained to identify a specific speaker.

9. The method of claim 3, further comprising:

configuring the plurality of microphone outputs initially base on a detected orientation of a corresponding group of microphones.

10. A method comprising:

beamforming a plurality of microphone outputs to obtain a plurality of virtual microphone audio channels, each virtual microphone audio channel corresponding to a beamform, comprising at least one voice channel and at least one noise channel;

performing voice recognition on the at least one voice channel; and

adjusting the corresponding voice beamform to improve a voice recognition confidence metric of the voice recognition.

11. The method of claim 10, wherein performing voice recognition on the at least one voice channel, comprises:

performing voice recognition on the at least one voice channel using trained voice recognition that is trained to identify a specific speaker.

12. The method of claim 10, further comprising:

performing voice activity detection on the at least one noise channel; and

adjusting a corresponding noise beamform until voice activity detection indicates that voice is not substantially present on the at least one noise channel.

13. The method of claim 12, further comprising:

performing energy estimation on the at least one noise channel; and

adjusting the corresponding noise beamform until energy estimation indicates that the at least one noise channel is receiving audio from a dominant audio energy source.

14. The method of claim 10, further comprising:

performing voice activity detection on the at least one noise channel; and

adjusting a corresponding noise beamform until voice activity detection indicates that voice is present on the at least one noise channel.

15. The method of claim 10, further comprising:

performing voice recognition on the at least one noise channel; and

adjusting the corresponding noise beamform to decrease a voice recognition confidence metric of the voice recognition.

16. The method of claim 14, further comprising:

performing voice recognition on the at least one noise channel in response to voice activity detection indicating that voice is present on the at least one noise channel.

17. The method of claim 16, wherein performing voice recognition comprises:

performing voice recognition on the at least one noise channel using trained voice recognition that is trained to identify a specific speaker.

18. The method of claim 17, further comprising:

adjusting the corresponding noise beamform to decrease a voice recognition confidence metric of the trained voice recognition.

19. An apparatus comprising:

a beamformer, operatively coupled to a plurality of microphone outputs, the beamformer operative to provide, as beamformer outputs, a plurality of virtual microphone audio channels each virtual microphone audio channel corresponding to a beamform, comprising at least one voice channel and at least one noise channel;

a beamformer controller, operatively coupled to the beamformer, the beamformer controller operative to;

monitor the at least one voice channel and the at least one noise channel to determine if voice is present on either of the at least one voice channel or the at least one noise channel; and

control the beamformer to adjust a beamform corresponding to the at least one voice channel until voice is present on the at least one voice channel.

20. The apparatus of claim 19, wherein the beamformer controller is further operative to:

control the beamformer to adjust a beamform corresponding to the at least one noise channel until voice is not substantially present on the at least one noise channel.

21. The apparatus of claim 19, further comprising:

a voice activity detector, operatively coupled to the beamformer to receive the at least one voice channel and to the beamformer controller; and

wherein the beamformer controller is operative to monitor the at least one voice channel to determine if voice is present by monitoring input received from the voice activity detector.

22. The apparatus of claim 19, further comprising:

a voice recognition engine, operatively coupled to the beamformer to receive the at least one voice channel, and to the beamformer controller, the voice recognition engine operative to perform voice recognition on the at least one voice channel to detect voice; and

wherein the beamformer controller is operative to monitor the at least one voice channel to determine if voice is present by monitoring input received from the voice recognition engine.