Object sound extraction apparatus and object sound extraction method

• US 20080267423A1
• Filed: 04/07/2008
• Published: 10/30/2008
• Est. Priority Date: 04/26/2007
• Status: Abandoned Application

• Current Assignee: Klaus Schmidt
• Original Assignee: Klaus Schmidt
• Inventors: Hiekata, Takashi, Morita, Takashi, Ikeda, Yohei, Shimoda, Toshiaki
• Application Number:
  US12/078,839
• Publication Number:
  US 20080267423A1
• Time in Patent Office: Days
• US Class Current: 381/92
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

An object sound extraction apparatus includes sound source separation sections for separating and generating an object sound separation signal corresponding to an object sound and reference sound separation signals corresponding to the other reference sound based on each combination of a main acoustic signal and sub acoustic signals, an object sound separation signal synthesis section for synthesizing the object sound separation signals, and a spectrum subtraction processing section for extracting an acoustic signal corresponding to the object sound from the synthesis signal by performing a spectrum subtraction processing between the synthesis signal and the reference sound separation signals. Accordingly, in acoustic environments where the object sound and the noises are mixed in the acoustic signals obtained via the microphones, and the mixed conditions can vary, a high object sound extraction performance can be ensured by a small object sound extraction apparatus.

Claims

1. An object sound extraction apparatus comprising:

a main sound input section for mainly inputting an object sound generated by a predetermined object sound source and outputting a main acoustic signal;

sub voice input sections for mainly inputting one or more reference sounds generated by one or more sound sources other than the object sound source and outputting sub acoustic signals;

sound source separation sections for performing a sound source separation processing for separating and generating an object sound separation signal corresponding to the object sound and reference sound separation signals corresponding to the one or more reference sounds other than the object sound based on each combination of the main acoustic signal and the sub acoustic signals;

an object sound separation signal synthesis section for synthesizing the object sound separation signals and outputting a synthesis signal; and

a spectrum subtraction processing section for extracting an acoustic signal corresponding to the object sound from the synthesis signal by performing a spectrum subtraction processing between the synthesis signal and the reference sound separation signals, and outputting an extracted signal corresponding to the acoustic signal.

4. The object sound extraction apparatus according to claim 1, wherein the sound source separation processing is a sound source separation processing according to a blind source separation method based on an independent component analysis.

5. The object sound extraction apparatus according to claim 1, wherein the sound source separation processing is a sound source separation processing according a binary masking processing.

6. The object sound extraction apparatus according to claim 4, wherein the sound source separation processing according to the blind source separation method based on the independent component analysis limits the number of the sequential calculations by, to the acoustic signals time-sequentially outputted by the main voice input section or the sub voice input sections (or, a voice input section), sequentially performing a filter processing based on a predetermined separation matrix to generate separation signals, and performing a sequential calculation for calculating the separation matrix that is to be used for the filter processing that is subsequently performed using all block signals for each block signal divided in a predetermined period in the acoustic signals.

7. The object sound extraction apparatus according to claim 4, wherein the sound source separation processing according to the blind source separation method based on the independent component analysis, to the acoustic signals time-sequentially outputted by the main voice input section or the sub voice input sections, sequentially performs a filter processing based on a predetermined separation matrix to generate separation signals, and performs a sequential calculation for calculating the separation matrix that is to be used for the filter processing that is subsequently performed using signals for each of the signals of a part of time at a head side of block signals divided in a predetermined period in the time-sequentially inputted acoustic signals.

8. The object sound extraction apparatus according to claim 1, wherein the sub voice input sections are disposed at positions different from a position of the main voice input section respectively.

9. The object sound extraction apparatus according to claim 1, wherein the sub voice input sections have directivities in directions different from a directivity of the main voice input section respectively.

10. The object sound extraction apparatus according to claim 1, further comprising:

a main/sub acoustic signal specification section for specifying the main acoustic signal and the sub acoustic signals from three or more acoustic signals outputted by the main voice input section and the sub voice input sections (or three or more voice input sections); and

a signal switching section for switching transmission paths of the acoustic signals from the three or more voice input sections to the sound source separation sections according to a result specified by the main/sub acoustic signal specification section.

11. The object sound extraction apparatus according to claim 10, wherein the main/sub acoustic signal specification section specifies the main acoustic signal and the sub acoustic signals by comparing signal strengths of each of the three or more acoustic signals.

12. The object sound extraction apparatus according to claim 7, wherein the main/sub acoustic signal specification section specifies the main acoustic signal and the sub acoustic signals by comparing ratios of predetermined frequency components in each of the three or more acoustic signals.

2. An object sound extraction apparatus comprising:

a main sound input section for mainly inputting an object sound generated by a predetermined object sound source and outputting a main acoustic signal;

sub voice input sections for mainly inputting one or more reference sounds generated by one or more sound sources other than the object sound source and outputting sub acoustic signals;

sound source separation sections for performing a sound source separation processing for separating and generating an object sound separation signal corresponding to the object sound based on each combination of the main acoustic signal and the sub acoustic signals; and

a spectrum approximate signal extraction section for extracting an acoustic signal corresponding to the object sound from the object sound extraction signals and outputting an extracted signal corresponding to the acoustic signal by dividing the object sound separation signals into signal components of each of a plurality of frequency bands, and extracting signal components that satisfy a predetermined approximation condition between the object sound separation signals.

16. The object sound extraction apparatus according to claim 2, wherein the sound source separation processing is a sound source separation processing according to a blind source separation method based on an independent component analysis.

18. The object sound extraction apparatus according to claim 2, wherein the sound source separation processing is a sound source separation processing according a binary masking processing.

20. The object sound extraction apparatus according to claim 16, wherein the sound source separation processing according to the blind source separation method based on the independent component analysis limits the number of the sequential calculations by, to the acoustic signals time-sequentially outputted by the main voice input section or the sub voice input sections (or, a voice input section), sequentially performing a filter processing based on a predetermined separation matrix to generate separation signals, and performing a sequential calculation for calculating the separation matrix that is to be used for the filter processing that is subsequently performed using all block signals for each block signal divided in a predetermined period in the acoustic signals.

22. The object sound extraction apparatus according to claim 16, wherein the sound source separation processing according to the blind source separation method based on the independent component analysis, to the acoustic signals time-sequentially outputted by the main voice input section or the sub voice input sections, sequentially performs a filter processing based on a predetermined separation matrix to generate separation signals, and performs a sequential calculation for calculating the separation matrix that is to be used for the filter processing that is subsequently performed using signals for each of the signals of a part of time at a head side of block signals divided in a predetermined period in the time-sequentially inputted acoustic signals.

24. The object sound extraction apparatus according to claim 2, wherein the sub voice input sections are disposed at positions different from a position of the main voice input section respectively.

26. The object sound extraction apparatus according to claim 2, wherein the sub voice input sections have directivities in directions different from a directivity of the main voice input section respectively.

28. The object sound extraction apparatus according to claim 2, further comprising:

a main/sub acoustic signal specification section for specifying the main acoustic signal and the sub acoustic signals from three or more acoustic signals outputted by the main voice input section and the sub voice input sections (or three or more voice input sections); and

a signal switching section for switching transmission paths of the acoustic signals from the three or more voice input sections to the sound source separation sections according to a result specified by the main/sub acoustic signal specification section.

30. The object sound extraction apparatus according to claim 28, wherein the main/sub acoustic signal specification section specifies the main acoustic signal and the sub acoustic signals by comparing signal strengths of each of the three or more acoustic signals.

32. The object sound extraction apparatus according to claim 22, wherein the main/sub acoustic signal specification section specifies the main acoustic signal and the sub acoustic signals by comparing ratios of predetermined frequency components in each of the three or more acoustic signals.

3. An object sound extraction apparatus comprising:

a main sound input section for mainly inputting an object sound generated by a predetermined object sound source and outputting a main acoustic signal;

sub voice input sections for mainly inputting one or more reference sounds generated by one or more sound sources other than the object sound source and outputting sub acoustic signals;

sound source separation sections for performing a sound source separation processing for separating and generating a reference sound separation signal corresponding to the one or more reference sounds other than the object sound based on each combination of the main acoustic signal and the sub acoustic signals; and

a spectrum subtraction processing section for extracting an acoustic signal corresponding to the object sound from the main acoustic signal and outputting an extracted signal corresponding to the acoustic signal by performing a spectrum subtraction processing between the reference sound separation signals separated and generated by the main acoustic signal and the sound source separation sections.

17. The object sound extraction apparatus according to claim 3, wherein the sound source separation processing is a sound source separation processing according to a blind source separation method based on an independent component analysis.

19. The object sound extraction apparatus according to claim 3, wherein the sound source separation processing is a sound source separation processing according a binary masking processing.

21. The object sound extraction apparatus according to claim 17, wherein the sound source separation processing according to the blind source separation method based on the independent component analysis limits the number of the sequential calculations, by, to the acoustic signals time-sequentially outputted by the main voice input section or the sub voice input sections (or, a voice input section), sequentially performing a filter processing based on a predetermined separation matrix to generate separation signals, and performing a sequential calculation for calculating the separation matrix that is to be used for the filter processing that is subsequently performed using all block signals for each block signal divided in a predetermined period in the acoustic signals.

23. The object sound extraction apparatus according to claim 17, wherein the sound source separation processing according to the blind source separation method based on the independent component analysis, to the acoustic signals time-sequentially outputted by the main voice input section or the sub voice input sections, sequentially performs a filter processing based on a predetermined separation matrix to generate separation signals, and performs a sequential calculation for calculating the separation matrix that is to be used for the filter processing that is subsequently performed using signals for each of the signals of a part of time at a head side of block signals divided in a predetermined period in the time-sequentially inputted acoustic signals.

25. The object sound extraction apparatus according to claim 3, wherein the sub voice input sections are disposed at positions different from a position of the main voice input section respectively.

27. The object sound extraction apparatus according to claim 3, wherein the sub voice input sections have directivities in directions different from a directivity of the main voice input section respectively.

29. The object sound extraction apparatus according to claim 3, further comprising:

a main/sub acoustic signal specification section for specifying the main acoustic signal and the sub acoustic signals from three or more acoustic signals outputted by the main voice input section and the sub voice input sections (or three or more voice input sections); and

a signal switching section for switching transmission paths of the acoustic signals from the three or more voice input sections to the sound source separation sections according to a result specified by the main/sub acoustic signal specification section.

31. The object sound extraction apparatus according to claim 29, wherein the main/sub acoustic signal specification section specifies the main acoustic signal and the sub acoustic signals by comparing signal strengths of each of the three or more acoustic signals.

33. The object sound extraction apparatus according to claim 23, wherein the main/sub acoustic signal specification section specifies the main acoustic signal and the sub acoustic signals by comparing ratios of predetermined frequency components in each of the three or more acoustic signals.

13. An object sound extraction method comprising:

a main sound input processing for mainly inputting an object sound generated by a predetermined object sound source and outputting a main acoustic signal;

a sub voice input processing for mainly inputting one or more reference sounds generated by one or more sound sources other than the object sound source and outputting sub acoustic signals;

a sound source separation processing for performing a sound source separation processing for separating and generating an object sound separation signal corresponding to the object sound and reference sound separation signals corresponding to the one or more reference sounds other than the object sound based on each combination of the main acoustic signal and the sub acoustic signals;

an object sound separation signal synthesis processing for synthesizing the object sound separation signals and outputting a synthesis signal; and

a spectrum subtraction processing for extracting an acoustic signal corresponding to the object sound from the synthesis signal by performing a spectrum subtraction processing between the synthesis signal and the reference sound separation signals, and outputting an extracted signal corresponding to the acoustic signal.

14. An object sound extraction method comprising:

a main sound input processing for mainly inputting an object sound generated by a predetermined object sound source and outputting a main acoustic signal;

a sub voice input processing for mainly inputting one or more reference sounds generated by one or more sound sources other than the object sound source and outputting sub acoustic signals;

a sound source separation processing for performing a sound source separation processing for separating and generating an object sound separation signal corresponding to the object sound based on each combination of the main acoustic signal and the sub acoustic signals; and

a spectrum approximate signal extraction processing for extracting an acoustic signal corresponding to the object sound from the object sound extraction signals and outputting an extracted signal corresponding to the acoustic signal by dividing the object sound separation signals into signal components of each of a plurality of frequency bands, and extracting signal components that satisfy a predetermined approximation condition between the object sound separation signals.

15. An object sound extraction method comprising:

a main sound input processing for mainly inputting an object sound generated by a predetermined object sound source and outputting a main acoustic signal;

a sub voice input processing for mainly inputting one or more reference sounds generated by one or more sound sources other than the object sound source and outputting sub acoustic signals;

a sound source separation processing for performing a sound source separation processing for separating and generating a reference sound separation signal corresponding to the one or more reference sounds other than the object sound based on each combination of the main acoustic signal and the sub acoustic signals; and

a spectrum subtraction processing for extracting an acoustic signal corresponding to the object sound from the main acoustic signal and outputting an extracted signal corresponding to the acoustic signal by performing a spectrum subtraction processing between the reference sound separation signals separated and generated by the main acoustic signal and the sound source separation processing.