Speech collector in car cabin

• US 9,953,641 B2
• Filed: 10/20/2016
• Issued: 04/24/2018
• Est. Priority Date: 10/27/2015
• Status: Active Grant

• Current Assignee: Panasonic Intellectual Property Management Co., Ltd. (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Intellectual Property Management Co., Ltd. (Panasonic Holdings Corporation)
• Inventors: Furukawa, Hiroki, Kanamori, Takeo, Yuzuriha, Shinichi
• Primary Examiner(s): COLUCCI, MICHAEL C
• Application Number:
  US15/298,300
• Publication Number:
  US 20170116983A1
• Time in Patent Office: 551 Days
• Field of Search: 704 9, 704226, 348345, 381 711
• CPC Class Codes: B60R 16/0373 Voice control in general G10L
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2021/02087 the noise being separate speech, e.g. cocktail party
• CPC Class Codes: G10L 2021/02165 Two microphones, one receiving mainly the noise signal and the other one mainly the speech signal
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 2025/783 based on threshold decision
• CPC Class Codes: G10L 21/0216 characterised by the method used for estimating noise
• CPC Class Codes: G10L 21/0232 Processing in the frequency domain
• CPC Class Codes: G10L 25/87 Detection of discrete points within a voice signal

Abstract

An apparatus includes a first microphone disposed near a first seat in a car cabin, the first seat configured to seat a first occupant, a second microphone disposed near a second seat in the car cabin, the second seat configured to seat a second occupant, a processor, and a memory storing a computer program. The computer program, when executed, causes the processor to execute operations including: estimating a first mixed speech using a second signal acquired by the second microphone, the first mixed speech including a first speech that has been uttered by the second occupant and has entered the first microphone, and removing the first mixed speech from a first signal acquired by the first microphone.

Claims

1. An apparatus, comprising:

a first microphone disposed near a first seat in a car cabin, the first seat configured to seat a first occupant;

a second microphone disposed near a second seat in the car cabin, the second seat configured to seat a second occupant;

a processor; and

a memory storing a computer program, wherein

the computer program, when executed, causes the processor to execute operations including;

calculating a first envelope of a signal of the first microphone and a second envelope of a second signal of the second microphone;

detecting a first bottom level of the first envelope and a second bottom level of the second envelope;

setting a first threshold on the basis of the first bottom level and a second threshold on the basis of the second bottom level;

detecting a first speech section in which a level of the first signal is higher than the first threshold and a second speech section in which a level of the second signal is higher than the second threshold;

when one of the first and the second speech sections is detected, estimating a first or a second mixed speech using a second or a first signal acquired by the second or the first microphone, the first and the second mixed speech including a first or a second speech that has been uttered by the second or the first occupant and has entered the first or the second microphone respectively; and

removing the first or the second mixed speech from a first or the second signal acquired by the first or the second microphone.

2. The apparatus according to claim 1, further comprising:

a first switch disposed near the first seat; and

a second switch disposed near the second seat, wherein

the operations further include;

(1) when the first switch is pressed, estimating the first mixed speech using the second signal and removing the first mixed speech from the first signal; and

(2) when the second switch is pressed, estimating a second mixed speech using the first signal and removing the second mixed speech from the second signal.

3. The apparatus according to claim 1, wherein

the operations further include:

determining whether any of the first and second signals includes a speech section; and

(1) when it is determined that the first signal includes the speech section, estimating the first mixed speech using the second signal and removing the first mixed speech from the first signal; and

(2) when it is determined that the second signal includes the speech section, estimating a second mixed speech using the first signal and removing the second mixed speech from the second signal.

4. The apparatus according to claim 3, wherein

the first and second microphones are assigned priorities,

(3) when it is determined that both the first and second signals include the speech section and the priority of the first microphone which acquires the first signal is higher than the priority of the second microphone which acquires the second signal, the processor estimates the first mixed speech using the second signal and removes the first mixed speech from the first signal, and

(4) when it is determined that both the first and second signals include the speech section and the priority of the second microphone which acquires the second signal is higher than the priority of the first microphone which acquires the first signal, the processor estimates the second mixed speech using the first signal and removes the second mixed speech from the second signal.

5. The apparatus according to claim 1, wherein the first and second microphones are unidirectional microphones.

6. The apparatus according to claim 1, wherein the first and second microphones are array microphones each consisting of a plurality of microphones.

7. The apparatus according to claim 1, wherein the processor estimates the first mixed speech and the second mixed speech using an adaptive filter having a filter coefficient that is corrected so that an output signal is uncorrelated with or independent of an input signal.

8. The apparatus according to claim 1, wherein

the first microphone is disposed closer to the first seat than to the second seat in the car cabin, and

the second microphone is disposed closer to the second seat than to the first seat in the car cabin.

9. The apparatus according to claim 1, wherein, when the first speech section is detected in the first signal, the computer program causes the processor to estimate the first mixed speech using the second signal and to remove the first mixed speech from the first signal and

when the second speech section is detected in the second signal, the computer program cause the processor to estimate a second mixed speech using the first signal and to remove the second mixed speech from the second signal.

10. The apparatus according to claim 1, wherein

when the first speech section is detected in the first signal, the level of the second signal is less than the second threshold, and

when the second speech section is detected in the second signal, the level of the first signal is less than the first threshold.

11. A method, comprising:

acquiring a first signal from a first microphone disposed near a first seat in a car cabin, the first seat configured to seat a first occupant;

acquiring a second signal from a second microphone disposed near a second seat in the car cabin, the second seat configured to seat a second occupant;

calculating a first envelope of a signal of the first microphone and a second envelope of a second signal of the second microphone;

detecting a first bottom level of the first envelope and a second bottom level of the second envelope;

setting a first threshold on the basis of the first bottom level and a second threshold on the basis of the second bottom level;

detecting a first speech section in which a level of the first signal is higher than the first threshold and a second speech section in which a level of the second signal is higher than the second threshold;

when a first or a second speech section is detected, estimating a first or a second mixed speech using the second or the first signal, the first or the second mixed speech including a first or a second speech that has been uttered by the second or the first occupant and has entered the first or the second microphone respectively; and

removing the first or the second mixed speech from the first or the second signal.

12. The method according to claim 11, further comprising

estimating, when the first speech section is detected in the first signal, a first mixed speech using the second signal and removing the first mixed speech from the first signal, and

estimating, when the second speech section is detected in the second signal, a second mixed speech using the first signal and removing the second mixed speech from the second signal.

13. The method according to claim 11, estimating, when a first switch positioned near a first seat is pressed, the first mixed speech using the second signal and removing the first mixed speech from the first signal;

and

estimating, when a second switch positioned near the second seat is pressed, the second mixed speech using the first signal and removing the second mixed speech from the second signal.

14. The method according to claim 11, further comprising assigning priorities to the first and second microphones, and

when it is determined that both the first and second signals include the speech section and the priority of the first microphone, which acquires the first signal is higher than the priority of the second microphone, which acquires the second signal, estimating the first mixed speech using the second signal and removing the first mixed speech from the first signal, and

when it is determined that both the first and second signals include the speech section and the priority of the second microphone, which acquires the second signal, is higher than the priority of the first microphone, which acquires the first signal, estimating the second mixed speech using the first signal and removing the second mixed speech from the second signal.

15. The method according to claim 11, wherein

when the first speech section is detected in the first signal, the level of the second signal is less than the second threshold, and

when the second speech section is detected in the second signal, the level of the first signal is less than the first threshold.

16. An apparatus, comprising:

a first microphone disposed near a first seat in a car cabin, the first seat configured to seat a first occupant;

a second microphone disposed near a second seat in the car cabin, the second seat configured to seat a second occupant;

a processor; and

a memory storing a computer program, wherein

the computer program, when executed, causes the processor to execute operations including;

acquiring a first signal using the first microphone;

acquiring a second signal using the second microphone;

calculating envelopes of the first and second signals;

detecting bottom levels of the envelopes;

setting a threshold on the basis of the bottom levels; and

determining a section in which levels of the first and second signals are higher than the threshold, as the speech section;

determining whether any of the first and second signals includes the speech section; and

(1) when it is determined that the first signal includes the speech section, estimating a first mixed speech using the second signal, the first mixed speech including a first speech that has been uttered by the second occupant and has entered the first microphone, and removing the first mixed speech from the first signal; and

(2) when it is determined that the second signal includes the speech section, estimating a second mixed speech using the first signal, the second mixed speech including a second speech that has been uttered by the first occupant and has entered the second microphone, and removing the second mixed speech from the second signal.

17. The apparatus according to claim 16, wherein

the first and second microphones are assigned priorities,

when it is determined that both the first and second signals include the speech section and the priority of the first microphone which acquires the first signal is higher than the priority of the second microphone which acquires the second signal, the processor estimates the first mixed speech using the second signal and removes the first mixed speech from the first signal, and

when it is determined that both the first and second signals include the speech section and the priority of the second microphone which acquires the second signal is higher than the priority of the first microphone which acquires the first signal, the processor estimates the second mixed speech using the first signal and removes the second mixed speech from the second signal.

18. The apparatus according to claim 16, wherein, when the first speech section is detected in the first signal, the computer program causes the processor to estimate a first mixed speech using the second signal and removes the first mixed speech from the first signal, and

when the second speech section is detected in the second signal, the computer program causes the processor to estimate a second mixed speech using the first signal and removes the second mixed speech from the second signal.

19. The apparatus according to claim 16, further comprising:

a first switch positioned near the first seat; and

a second switch positioned near the second seat, wherein

the operations further include;

when the first switch is pressed, estimating the first mixed speech using the second signal and removing the first mixed speech from the first signal; and

when the second switch is pressed, estimating the second mixed speech using the first signal and removing the second mixed speech from the second signal.

20. The apparatus according to claim 16, wherein

when the first speech section is detected in the first signal, the level of the second signal is less than the second threshold, and

when the second speech section is detected in the second signal, the level of the first signal is less than the first threshold.