Audio processing using an intelligent microphone

• US 10,269,343 B2
• Filed: 08/27/2015
• Issued: 04/23/2019
• Est. Priority Date: 08/28/2014
• Status: Active Grant

• Current Assignee: Analog Devices, Inc.
• Original Assignee: Analog Devices, Inc.
• Inventors: Wingate, David
• Primary Examiner(s): Tran, Thang V
• Application Number:
  US15/505,736
• Publication Number:
  US 20170243577A1
• Time in Patent Office: 1,335 Days
• CPC Class Codes: G10L 15/01 Assessment or evaluation of speech recognition systems
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 21/0208 Noise filtering
• CPC Class Codes: G10L 21/0216 characterised by the method used for estimating noise
• CPC Class Codes: G10L 21/0272 Voice signal separating
• CPC Class Codes: G10L 25/84 for discriminating voice from noise
• CPC Class Codes: H04R 1/406 microphones
• CPC Class Codes: H04R 2410/00 Microphones
• CPC Class Codes: H04R 2410/01 Noise reduction using microphones having different directional characteristics
• CPC Class Codes: H04R 2499/11 Transducers incorporated or for use in hand-held devices, e.g. mobile phones, PDA's, camera's
• CPC Class Codes: H04R 2499/13 Acoustic transducers and sound field adaptation in vehicles
• CPC Class Codes: H04R 3/00 Circuits for transducers , loudspeakers or microphones
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

The present disclosure relates generally to improving audio processing using an intelligent microphone and, more particularly, to techniques for processing audio received at a microphone with integrated analog-to-digital conversion, digital signal processing, acoustic source separation, and for further processing by a speech recognition system. Embodiments of the present disclosure include intelligent microphone systems designed to collect and process high-quality audio input efficiently. Systems and method for audio processing using an intelligent microphone include an integrated package with one or more microphones, analog-to-digital converters (ADCs), digital signal processors (DSPs), source separation modules, memory, and automatic speech recognition. Systems and methods are also provided for audio processing using an intelligent microphone that includes a microphone array and uses a preprogrammed audio beamformer calibrated to the included microphone array.

Claims

1. An integrated circuit for providing advanced speech processing as a single package, comprising:

a microphone array for generating respective audio signals corresponding to distinct microphones in the microphone array;

circuitry for processing the respective audio signals, the circuitry comprising;

source separation module for separating sources present in the respective audio signals and generating one or more improved audio signals;

automatic speech recognition module for extracting speech information in the one or more improved audio signals and determining confidence level of extracted speech information; and

a feedback path for the confidence level to control one or more parameters affecting the source separation module.

2. The integrated circuit of claim 1, wherein:

the source separation module determines a level of noise in the respective audio signals; and

the circuitry further includes a feedforward path for the level of the noise to control the automatic speech recognition module.

3. The integrated circuit of claim 1, further comprising:

one or more sensors for sensing a source near the microphone array;

wherein the circuitry further includes a control path for an output of the one or more sensors to control the automatic speech recognition module.

4. The integrated circuit of claim 1, wherein:

the circuitry for processing the respective audio signals further comprises a voice activity detecting module for detecting voice activity in the respective audio signals; and

the circuitry further includes a control path for an output of the voice activity detecting module to control the automatic speech recognition module.

5. An integrated circuit for providing advanced speech processing as a single package, comprising:

a microphone array for generating respective audio signals corresponding to distinct microphones in the microphone array; and

circuitry for processing the respective audio signals, the circuitry comprising;

a noise reduction module for reducing noise in the respective audio signals and generating one or more improved audio signals;

an automatic speech recognition module for extracting speech information in the one or more improved audio signals and determining confidence level of extracted speech information; and

a feedback path for the confidence level to control one or more parameters affecting sensitivity of the noise reduction module.

6. The integrated circuit of claim 5, wherein:

the circuitry for processing the respective audio signals further comprises;

a source separation module for separating sources present in the respective audio signals and determining a level of noise in the one or more improved audio signals; and

a feedback path for the level of noise to control one or more parameters affecting sensitivity of the noise reduction module.

7. The integrated circuit of claim 5, wherein:

the noise reduction module determines a level of noise in the respective audio signals; and

the circuitry further includes a feedforward path for the level of the noise to control the automatic speech recognition module.

8. An integrated circuit for providing advanced speech processing as a single package, comprising:

a microphone array for generating respective audio signals corresponding to distinct microphones in the microphone array; and

circuitry for processing the respective audio signals comprises;

a beamforming module for adaptively steering a beam of the microphone array;

a source separation module for separating sources present in the respective audio signals and determining a level of noise in the respective audio signals; and

a feedback path for the level of noise to control one or more parameters affecting the beam being formed by the beamforming module.

9. An integrated circuit for providing advanced speech processing as a single package, comprising:

a microphone array for generating respective audio signals corresponding to distinct microphones in the microphone array; and

circuitry for processing the respective audio signals comprises;

a beamforming module for adaptively steering a beam of the microphone array;

a noise reduction module for determining a level of noise in the respective audio signals; and

a feedback path for the level of noise to control one or more parameters affecting the beam being formed by the beamforming module.

10. An integrated circuit for providing advanced speech processing as a single package, comprising:

a microphone array for generating respective audio signals corresponding to distinct microphones in the microphone array; and

one or more sensors for sensing a source near the microphone array;

circuitry for processing the respective audio signals, the circuitry comprising;

a beamforming module for adaptively steering a beam of the microphone array;

an automatic speech recognition module for extracting speech information in the respective audio signals and determining confidence level of extracted speech information;

a feedback path for the confidence level to control one or more parameters affecting the beam being formed by the beamforming module; and

a control path for an output of the one or more sensors to control one or more parameters affecting the beam being formed by the beamforming module.

11. The integrated circuit of claim 10, wherein the microphone array is a quad microphone comprising four microphone elements arranged in a geometric configuration.

12. An integrated circuit for providing advanced speech processing as a single package, comprising:

a microphone array for generating respective audio signals corresponding to distinct microphones in the microphone array; and

circuitry for processing the respective audio signals, the circuitry comprising;

a beamforming module for adaptively steering a beam of the microphone array;

an automatic speech recognition module for extracting speech information in the respective audio signals and determining confidence level of extracted speech information;

a feedback path for the confidence level to control one or more parameters affecting the beam being formed by the beamforming module;

a voice activity detecting module for detecting voice activity in the respective audio signals; and

a control path for an output of the voice activity detecting module to control one or more parameters affecting the beam being formed by the beamforming module.

13. An integrated circuit for providing advanced speech processing as a single package, comprising:

a microphone array for generating respective audio signals corresponding to distinct microphones in the microphone array;

circuitry for processing the respective audio signals, the circuitry comprising;

a beamforming module for adaptively steering a beam of the microphone array;

an automatic speech recognition module for extracting speech information in the respective audio signals and determining confidence level of extracted speech information;

a feedback path for the confidence level to control one or more parameters affecting the beam being formed by the beamforming module; and

a motion sensing module for sensing movement of an electronic device having the microphone array and the circuitry; and

wherein the beamforming module determines changes in position or orientation of the microphone array relative to an audio source of interest based on the respective audio signals and movement information from the motion sensing module adjusts beamforming functionality in response to the changes.

14. A method for audio processing within a single integrated package, the method comprising:

generating audio signals by microphones of a microphone array within the single integrated package; and

processing the audio signals by circuitry in the same integrated package as the microphone array, wherein;

the circuitry in the same integrated package includes two or more of the following modules;

source separation module, automatic speech recognition module, adaptive beamforming module, noise reduction module, and voice activity detection module; and

the processing includes;

determining a confidence level of extracted speech information by the automatic speech recognition module; and

controlling one or more parameters affecting a sensitivity of the source separation module based on the confidence level.

15. The method of claim 14, further comprising:

transmitting over a network interface the processed audio signals to a remote automatic speech recognition service over a network.

16. The method of claim 14, wherein the processing comprises:

controlling, based on the confidence level, one or more parameters affecting one or more of;

a sensitivity of the noise reduction module and a beam formed by the adaptive beamforming module.

17. A method for audio processing within a single integrated package, the method comprising:

generating audio signals by microphones of a microphone array within the single integrated package; and

processing the audio signals by circuitry in the same integrated package as the microphone array, wherein;

the circuitry in the same integrated package includes two or more of the following modules;

source separation module, automatic speech recognition module, adaptive beamforming module, noise reduction module, and voice activity detection module; and

the processing includes;

determining a level of noise by one or more of;

the source separation module and the noise reduction module; and

controlling one or more parameters affecting a beam formed by the adaptive beamforming module based on the level of noise.

18. A method for audio processing within a single integrated package, the method comprising:

generating audio signals by microphones of a microphone array within the single integrated package; and

processing the audio signals by circuitry in the same integrated package as the microphone array, wherein;

the circuitry in the same integrated package includes two or more of the following modules;

source separation module, automatic speech recognition module, adaptive beamforming module, noise reduction module, and voice activity detection module; and

the processing includes;

determining a level of noise by one or more of;

the source separation module and the noise reduction module; and

controlling the automatic speech recognition module based on the level of noise.

19. The method of claim 18, wherein controlling the automatic speech recognition module to stop extracting speech information if the level of noise is greater than a threshold.

20. The method of claim 18, wherein controlling the automatic speech recognition module comprises requesting the automatic speech recognition module to only output speech information extracted above a certain confidence level if the level of noise is greater than a threshold.