Multi-sourced noise suppression

• US 9,799,330 B2
• Filed: 08/27/2015
• Issued: 10/24/2017
• Est. Priority Date: 08/28/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Knowles Electronics Llc (Knowles Corporation)
• Inventors: Nemala, Sridhar Krishna, Woodruff, John, Verma, Tony, Caldwell, Frederic
• Primary Examiner(s): Riley, Marcus T
• Application Number:
  US14/838,133
• Publication Number:
  US 20160063997A1
• Time in Patent Office: 789 Days
• Field of Search: None
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2021/02082 the noise being echo, reverberation of the speech
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 21/0216 characterised by the method used for estimating noise
• CPC Class Codes: G10L 25/06 the extracted parameters being correlation coefficients
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

Systems and methods for multi-sourced noise suppression are provided. An example system may receive streams of audio data including a voice signal and noise, the voice signal including a spoken word. The streams of audio data are provided by distributed audio devices. The system can assign weights to the audio streams based at least partially on quality of the audio streams. The weights of audio streams can be determined based on signal-to-noise ratios (SNRs). The system may further process, based on the weights, the audio stream to generate cleaned speech. Each audio device comprises microphone(s) and can be associated with the Internet of Things (IoT), such that the audio devices are Internet of Things devices. The processing can include noise suppression and reduction and echo cancellation. The cleaned speech can be provided to a remote device for further processing which may include Automatic Speech Recognition (ASR).

Claims

1. A method for multi-sourced noise suppression, the method comprising:

assigning weights to audio streams, the audio streams being provided substantially synchronously by a plurality of audio devices, the weights depending on quality of the audio streams, wherein the assigning weights includes generating an acoustic activity map by locating, identifying and mapping target sounds and noise sources in at least one of a single room and multi-room environment, so as to create a multidimensional acoustic view of the environment;

based on the weights, performing noise suppression processing on the audio streams to generate a cleaned voice signal;

providing the cleaned voice signal from the noise suppression processing to at least one remote device for further processing; and

based on the acoustic activity map, selecting an optimal one of the plurality of audio devices to communicate with the user.

2. The method of claim 1, wherein each of the audio devices includes at least one microphone and the audio devices are connected in a dynamic network of connected devices, such that the audio devices are connected as part of an Internet of Things environment.

3. The method of claim 1, wherein the weights are proportional to at least one quality metric for the audio stream, the quality metric comprising at least one signal to noise ratio (SNR).

4. The method of claim 1, wherein the performing of the noise suppression processing is provided in combination with at least one of:

performing noise reduction; and

performing echo cancellation.

5. The method of claim 1, further including continually updating acoustic signatures between the audio devices based on one or more sound sources located in the vicinity of the audio devices.

6. The method of claim 1, wherein auditory scene analysis and scene classifiers are used for the identifying of target sounds and noise sources.

7. The method of claim 1, wherein the audio streams include time stamps, the method further comprising, based on the time stamps, synchronizing the audio devices to a common time source.

8. The method of claim 7, further comprising, based on the acoustic activity map, assigning weights to the audio streams based on the SNR quality metric.

9. The method of claim 8, further comprising, based on the acoustic activity map, assigning weights to the audio streams further based on the degree to which the associated audio device, that provides a respective one of the audio streams, measures noise.

10. The method of claim 1, wherein the communication with the user is via a loudspeaker of the optimal audio device.

11. The method of claim 7, wherein the performing of the noise suppression processing is provided in combination with at least one of:

performing noise reduction; and

performing echo cancellation.

12. The method of claim 1, wherein the audio streams comprise at least one voice command to perform at least one of activating the remote device and communicating with another user.

13. The method of claim 1, wherein the further processing comprises automatic speech recognition (ASR) processing of the cleaned voice signal.

14. The method of claim 13, wherein, based on the ASR processing, a context of a command to connect to another user is recognized and the cleaned voice signal is communicated to at least one of the audio devices located in proximity to the other user for establishing two way communication therewith.

18. The method of claim 1, wherein one or more of the plurality of audio devices is incorporated in an Internet of Things device.

15. A system for multi-sourced audio processing, the system comprising:

a processor; and

a memory communicatively coupled with the processor, the memory storing instructions, which, when executed by the processor, perform a method comprising;

assigning weights to audio streams, the audio streams being provided substantially synchronously by a plurality of audio devices, the weights depending on quality of the audio streams;

based on the weights, performing noise suppression processing on the audio streams to generate a cleaned voice signal, and

providing the cleaned voice signal from the noise suppression processing to a remote device for further processing,

wherein each of the audio devices includes at least one microphone and wherein the plurality of audio devices are physically separate from each other but connected in a dynamic network of connected devices, such that the audio devices are connected as part of an Internet of Things environment.

16. The system of claim 15, wherein the assigning weights includes generating an acoustic activity map by locating, identifying and mapping target sounds and noise sources in at least one of a single room and multi-room environment, so as to create a multidimensional acoustic view of the environment.

20. The system of claim 15, further comprising:

a controller for receiving the audio streams from the plurality of audio devices via the dynamic network and for performing the noise suppression processing on the received audio streams.

17. A non-transitory computer-readable storage medium having embodied thereon instructions, which, when executed by at least one processor, perform steps of a method, the method comprising:

assigning weights to audio streams, the audio streams being provided substantially synchronously by a plurality of audio devices, the weights depending on quality of the audio streams, wherein the assigning weights includes generating an acoustic activity map by locating, identifying and mapping target sounds and noise sources in at least one of a single room and multi-room environment, so as to create a multidimensional acoustic view of the environment;

based on the weights, performing noise suppression processing on the audio streams to generate a cleaned voice signal;

providing the cleaned voice signal from the noise suppression processing to at least one remote device for further processing; and

based on the acoustic activity map, selecting an optimal one of the plurality of audio devices to communicate with the user.

19. The non-transitory computer-readable medium of claim 17, wherein one or more of the plurality of audio devices is incorporated in an Internet of Things device.