Signal source separation

• US 9,460,732 B2
• Filed: 12/23/2013
• Issued: 10/04/2016
• Est. Priority Date: 02/13/2013
• Status: Active Grant

• Current Assignee: Analog Devices, Inc.
• Original Assignee: Analog Devices, Inc.
• Inventors: Wingate, David, Stein, Noah
• Primary Examiner(s): HUBER, PAUL W
• Application Number:
  US14/138,587
• Publication Number:
  US 20140226838A1
• Time in Patent Office: 1,016 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G10L 21/0272 Voice signal separating
• CPC Class Codes: H04R 1/406 microphones
• CPC Class Codes: H04R 2201/003 Mems transducers or their use
• CPC Class Codes: H04R 2430/21 Direction finding using differential microphone array [DMA]

Abstract

In one aspect, a microphone with closely spaced elements is used to acquire multiple signals from which a signal from a desired source is separated. The signal separation approach uses a combination of direction-of-arrival information or other information determined from variation such as phase, delay, and amplitude among the acquired signals, as well as structural information for the signal from the source of interest and/or for the interfering signals. Through this combination of information, the elements may be spaced more closely than may be effective for conventional beamforming approaches. In some examples, all the microphone elements are integrated into a single a micro-electrical-mechanical system (MEMS).

Claims

1. An audio signal separation system for signal separation according to source in an acoustic signal, the system comprising:

a micro-electrical-mechanical system (MEMS) microphone unit including

a plurality of acoustic ports, each port for sensing an acoustic environment at a spatial location relative to microphone unit, a minimum spacing between the spatial locations being less than 3 millimeters,

a plurality of microphone elements, each coupled to an acoustic port of the plurality of acoustic ports to acquire a signal based on an acoustic environment at the spatial location of said acoustic port, and

circuitry coupled to the microphone elements configured to provide one or more microphone signals together representing a representative acquired signal and a variation among the signals acquired by the microphone elements; and

an audio processor configured to process the one or more microphone signals from the microphone unit to output one or more separated signals comprising signals separated according to corresponding one or more sources of said signals from the representative acquired signal,

wherein the audio processor is configured to process the one or more microphone signals using direction of arrival information determined from the variation among the acquired signals and using signal structure of the one or more sources.

2. The audio signal separation system of claim 1, wherein the one or more microphone signals comprise a plurality of microphone signals, each microphone signal corresponding to a different microphone element of the plurality of microphone elements.

3. The audio signal separation system of claim 1, wherein the variation among the one or more acquired signals represents at least one of a relative phase variation and a relative delay variation among the acquired signals for each of a plurality of spectral components.

4. The audio signal separation system of claim 1, comprising a plurality of MEMS microphone units.

5. The audio signal separation system of claim 1, wherein at least some circuitry implemented the audio processor is integrated with the MEMS of the microphone unit.

6. The audio signal separation system of claim 1, wherein the microphone unit and the audio processor together form a kit, each implemented as an integrated device configured to communicate with one another in operation of the audio signal system.

7. The audio signal separation system of claim 1, wherein the signal structure of the one or more sources comprises voice signal structure.

8. The audio signal separation system of claim 1, wherein the audio processor is configured to process the signals by computing data representing characteristic variation among the acquired signals and selecting components of the representative acquired signal according to the characteristic variation.

9. The audio signal separation system of claim 8, wherein the selected components of the signal are characterized by time and frequency of said components.

10. The audio signal separation system of claim 8, wherein the audio processor is configured to compute a mask having values indexed by time and frequency, and wherein selecting the components includes combining the mask values with the representative acquired signal to form at least one of the signals output by the audio processor.

11. The audio signal separation system of claim 8, wherein data representing characteristic variation among the acquired signals comprises direction of arrival information.

12. The audio signal separation system of claim 1, wherein the audio processor comprises a module configured to identify components associated with at least one of the one or more sources using signal structure of said source.

13. The audio signal separation system of claim 12, wherein the module configured to identity the components is configured to combine direction of arrival estimates of multiple components of the signals from the microphones to select the components for forming the signal output from the audio processor.

14. The audio signal separation system of claim 13, wherein the module configured to identity the components is further configured to use confidence values associated with the direction of arrival estimates.

15. The audio signal separation system of claim 12, wherein the module configured to identity the components includes an input for accepting external information for use in identifying the desired components of the signals.

16. The audio signal separation system of claim 1, wherein the audio processor comprises a signal reconstruction module for processing one or more of the signals from the microphones according to identified components characterized by time and frequency to form the enhanced signal.

17. The audio signal separation system of claim 1, wherein the audio processor is configured to process the one or more microphone signals by using an iterative algorithm based on probabilistic inference approach and utilizing the direction of arrival information and the signal structure of the one or more sources.

18. The audio signal separation system of claim 1, wherein the audio processor is configured to process the one or more microphone signals by using an iterative algorithm configured to reach optimal spectral and temporal distributions of the one or more separated signals by iteratively updating estimated spectral and temporal distributions of the one or more separated signals to match the representative acquired signal.

19. The audio signal separation system of claim 18, wherein the audio processor is configured to perform the iterative updating until a predefined maximum number of iterations is reached or until the estimated spectral and temporal distributions of the one or more separated signals and the representative acquired signal reach a predefined degree of convergence.

20. The audio signal separation system of claim 17, wherein the probabilistic inference approach comprises a Belief Propagation approach.

21. The audio signal separation system of claim 1, wherein processing the one or more microphone signals using the direction of arrival (DOA) information and using the signal structure of the one or more sources comprises:

forming an approximation of the representative acquired signal, the approximation having a hidden multiple-source structure assuming that the representative acquired signal was generated by a number of distinct acoustic sources indexed by s=1, . . . , S and each acoustic source of the one or more sources is associated with a subset of prototype frequency distributions indexed by z=1, . . . , Z so that the approximation can be factorized into constituent parts;

performing a plurality of iterations of adjusting components of a model of the approximation to match the representative acquired signal; and

generating the one or more separated signals using the constituent parts of the approximation corresponding to the one or more sources.

22. The audio signal separation system of claim 21, wherein the approximation includes the DOA information determined from the variation among the acquired signals.

23. The audio signal separation system of claim 1, wherein processing the one or more microphone signals using the direction of arrival (DOA) information and using the signal structure of the one or more sources comprises:

computing time-dependent spectral characteristics from the one or more microphone signals, the spectral characteristics comprising a plurality of components, each component associated with a respective pair of frequency (f) and time (n) values;

computing DOA estimates from at least two of the one or more microphone signals, each computed component of the spectral characteristics having a corresponding one of the direction estimates (d);

combining the computed spectral characteristics and the computed DOA estimates to form a data structure representing a distribution P(f,n,d) indexed by frequency (f), time (n), and direction (d);

forming an approximation Q(f,n,d) of the distribution P(f,n,d), the approximation having a hidden multiple-source structure assuming that the representative acquired signal was generated by a number of distinct acoustic sources indexed by s=1, . . . , S and each acoustic source of the one or more sources is associated with a subset of prototype frequency distributions indexed by z=1, . . . , Z so that the approximation can be factorized into constituent parts;

performing a plurality of iterations of adjusting components of a model of the approximation Q(f,n,d) to match the distribution P(f,n,d); and

generating the one or more separated signals using the constituent parts of the approximation Q(f,n,d) corresponding to the one or more sources.

24. A non-transitory machine-readable storage medium storing instructions configured to, when executed, control a processor to perform signal separation according to source in an acoustic signal, the instructions comprising:

processing one or more microphone signals acquired by a plurality of microphone elements of a microphone unit and together representing a representative acquired signal and a variation among the signals acquired by the microphone elements to output one or more separated signals comprising signals separated according to corresponding one or more sources of said signals from the representative acquired signal,

wherein the one or more microphone signals are processed using direction of arrival information determined from the variation among the acquired signals and using signal structure of the one or more sources by;

forming an approximation of the representative acquired signal, the approximation having a hidden multiple-source structure assuming that the representative acquired signal was generated by a number of distinct acoustic sources indexed by s=1, . . . , S and each acoustic source of the one or more sources is associated with a subset of prototype frequency distributions indexed by z=1, . . . , Z so that the approximation can be factorized into constituent parts,

performing a plurality of iterations of adjusting components of a model of the approximation to match the representative acquired signal, and

generating the one or more separated signals using the constituent parts of the approximation corresponding to the one or more sources.

25. The non-transitory machine-readable storage medium of claim 24, wherein the approximation includes the direction of arrival information determined from the variation among the acquired signals.

26. The non-transitory machine-readable storage medium of claim 24, wherein processing the one or more microphone signals using the direction of arrival information and using the signal structure of the one or more sources further comprises:

computing time-dependent spectral characteristics from the one or more microphone signals, the spectral characteristics comprising a plurality of components, each component associated with a respective pair of frequency (f) and time (n) values,

computing DOA estimates from at least two of the one or more microphone signals, each computed component of the spectral characteristics having a corresponding one of the direction estimates (d), and

combining the computed spectral characteristics and the computed DOA estimates to form a data structure representing a distribution P(f,n,d) indexed by frequency (f), time (n), and direction (d),

wherein the approximation of the representative acquired signal comprises an approximation Q(f,n,d) of the distribution P(f,n,d).

27. The non-transitory machine-readable storage medium of claim 26, wherein the one or more separated signals are generated using a mask function M(f,n) computed for separating contributions of the one or more sources using the constituent parts of the approximation Q(f,n,d) corresponding to the one or more sources.

28. A method for performing signal separation according to source in an acoustic signal, the method comprising:

processing one or more microphone signals acquired by a plurality of microphone elements of a microphone unit and together representing a representative acquired signal and a variation among the signals acquired by the microphone elements to output one or more separated signals comprising signals separated according to corresponding one or more sources of said signals from the representative acquired signal,

wherein the one or more microphone signals are processed using direction of arrival information determined from the variation among the acquired signals and using signal structure of the one or more sources by;

forming an approximation of the representative acquired signal, the approximation having a hidden multiple-source structure assuming that the representative acquired signal was generated by a number of distinct acoustic sources indexed by s=1, . . . , S and each acoustic source of the one or more sources is associated with a subset of prototype frequency distributions indexed by z=1, . . . , Z so that the approximation can be factorized into constituent parts,

performing a plurality of iterations of adjusting components of a model of the approximation to match the representative acquired signal, and

generating the one or more separated signals using the constituent parts of the approximation corresponding to the one or more sources.

29. The method of claim 28, wherein the approximation includes the direction of arrival information determined from the variation among the acquired signals.

30. An audio signal separation system for signal separation according to source in an acoustic signal, the system comprising:

a micro-electrical-mechanical system (MEMS) microphone unit comprising a plurality of acoustic ports provided at different spatial locations and configured to acquire acoustic signals comprising contributions from a plurality of acoustic sources; and

a signal processing unit configured to process the acquired signals to separate contributions from a first acoustic source of the plurality of acoustic sources from contributions from other acoustic sources of the plurality of acoustic sources,

wherein processing comprises processing the acquired signals using direction of arrival information determined from a variation among the signals acquired via different acoustic ports and using signal structure of one or more acoustic sources of the plurality of acoustic sources by;

forming an approximation of the representative acquired signal, the approximation having a hidden multiple-source structure assuming that the representative acquired signal was generated by a number of distinct acoustic sources indexed by s=1, . . . , S and each acoustic source of the one or more sources is associated with a subset of prototype frequency distributions indexed by z=1, . . . , Z so that the approximation can be factorized into constituent parts,

performing a plurality of iterations of adjusting components of a model of the approximation to match the representative acquired signal, and

generating the one or more separated signals using the constituent parts of the approximation corresponding to the one or more sources.

31. The audio signal separation system of claim 30, wherein the variation among the signals acquired via different acoustic ports represents at least one of a relative phase variation and a relative delay variation among the acquired signals for each of a plurality of spectral components.

32. The audio signal separation system of claim 30, wherein the spatial locations of the acoustic ports are coplanar locations.

33. The audio signal separation system of claim 32, wherein the coplanar locations comprise a regular grid of locations.

34. The audio signal separation system of claim 30, wherein the MEMS microphone unit comprises a package having multiple surface faces, and wherein the acoustic ports are on multiple of the faces of the package.

35. The audio signal separation system of claim 30, wherein the approximation includes the DOA information determined from the variation among the acquired signals.

36. The audio signal separation system of claim 30, wherein processing the one or more microphone signals using the direction of arrival information and using the signal structure of the one or more sources further comprises:

computing time-dependent spectral characteristics from the one or more microphone signals, the spectral characteristics comprising a plurality of components, each component associated with a respective pair of frequency (f) and time (n) values,

computing DOA estimates from at least two of the one or more microphone signals, each computed component of the spectral characteristics having a corresponding one of the direction estimates (d), and

combining the computed spectral characteristics and the computed DOA estimates to form a data structure representing a distribution P(f,n,d) indexed by frequency (f), time (n), and direction (d),

wherein the approximation of the representative acquired signal comprises an approximation Q(f,n,d) of the distribution P(f,n,d).

37. A non-transitory machine-readable storage medium storing instructions configured to, when executed, control a processor to perform signal separation according to source in an acoustic signal, the instructions comprising:

processing signals representative of acoustic signals comprising contributions from a plurality of acoustic sources, the acoustic signals acquired by a plurality of acoustic ports of a micro-electrical-mechanical system (MEMS) microphone unit, to separate contributions from a first acoustic source of the plurality of acoustic sources from contributions from other acoustic sources of the plurality of acoustic sources,

wherein the signals are processed using direction of arrival information determined from a variation among the signals acquired via different acoustic ports and using signal structure of one or more acoustic sources of the plurality of acoustic sources by;

forming an approximation of the acquired signals, the approximation having a hidden multiple-source structure assuming that the representative acquired signals were generated by a number of distinct acoustic sources indexed by s=1, . . . , S and each acoustic source of the one or more sources is associated with a subset of prototype frequency distributions indexed by z=1, . . . , Z so that the approximation can be factorized into constituent parts,

performing a plurality of iterations of adjusting components of a model of the approximation to match the acquired signals, and

separating the contributions from the first acoustic source using the constituent parts of the approximation corresponding to the first acoustic source.

38. A method for performing signal separation according to source in an acoustic signal, the method comprising:

processing signals representative of acoustic signals comprising contributions from a plurality of acoustic sources, the acoustic signals acquired by a plurality of acoustic ports of a micro-electrical-mechanical system (MEMS) microphone unit, to separate contributions from a first acoustic source of the plurality of acoustic sources from contributions from other acoustic sources of the plurality of acoustic sources,

wherein the signals are processed using direction of arrival information determined from a variation among the signals acquired via different acoustic ports and using signal structure of one or more acoustic sources of the plurality of acoustic sources by;

forming an approximation of the acquired signals, the approximation having a hidden multiple-source structure assuming that the representative acquired signals were generated by a number of distinct acoustic sources indexed by s=1, . . . , S and each acoustic source of the one or more sources is associated with a subset of prototype frequency distributions indexed by z=1, . . . , Z so that the approximation can be factorized into constituent parts,

performing a plurality of iterations of adjusting components of a model of the approximation to match the acquired signals, and

separating the contributions from the first acoustic source using the constituent parts of the approximation corresponding to the first acoustic source.