Audio processing apparatus that outputs, among sounds surrounding user, sound to be provided to user

• US 10,510,361 B2
• Filed: 03/03/2016
• Issued: 12/17/2019
• Est. Priority Date: 03/10/2015
• Status: Active Grant

• Current Assignee: Panasonic Intellectual Property Management Co., Ltd. (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Intellectual Property Management Co., Ltd. (Panasonic Holdings Corporation)
• Inventors: Nomura, Kazuya
• Primary Examiner(s): Ojo, Oyesola C
• Application Number:
  US15/059,539
• Publication Number:
  US 20160267925A1
• Time in Patent Office: 1,384 Days
• Field of Search: 381 947
• CPC Class Codes: G10L 21/0272 Voice signal separating
• CPC Class Codes: G10L 25/78 Detection of presence or absence of voice signals switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10
• CPC Class Codes: G10L 25/81 for discriminating voice from music
• CPC Class Codes: G10L 25/84 for discriminating voice from noise
• CPC Class Codes: H04R 1/1083 Reduction of ambient noise active noise reduction per se G10K11/175; protective devices for the ear, e.g. providing acoustic protection A61F11/06
• CPC Class Codes: H04R 25/407 Circuits for combining signals of a plurality of transducers

Abstract

An audio processing apparatus is provided that includes an acquirer that acquires a surrounding audio signal indicating a sound surrounding a user. The audio processing apparatus also includes an audio extractor that extracts, from the acquired surrounding audio signal, a providing audio signal indicating a sound to be provided to the user. The audio processing apparatus further includes an output that outputs a first audio signal, indicating a main sound, and the providing audio signal.

Claims

1. An audio processing apparatus, comprising:

an acquirer that acquires a surrounding audio signal indicating a sound surrounding a user;

an audio extractor that extracts, from the acquired surrounding audio signal, (1) a main sound that is output without delay, (2) a providing audio signal, including prioritized sounds other than the main signal, that is stored and selectably output with or without delay, and (3) sounds that are not stored and do not need to be output;

a selector that selects one of a plurality of output patterns of audio signals; and

an output that outputs a first audio signal indicating a main sound and the providing audio signal,

wherein the selector selects any one of (i) a first output pattern in which the providing audio signal is output along with the main sound without a delay, (ii) a second output pattern in which the providing audio signal is output with a delay after only the main sound is output, and (iii) a third output pattern in which only the main sound is output.

2. The audio processing apparatus according to claim 1, further comprising:

an audio separator that separates the acquired surrounding audio signal into the first audio signal and a second audio signal indicating a sound different from the main sound,

wherein the audio extractor extracts the providing audio signal from the separated second audio signal, and

wherein the output outputs the separated first audio and also outputs the extracted providing audio signal.

3. The audio processing apparatus according to claim 2,

wherein the main sound includes a sound uttered by a person participating in a conversation.

4. The audio processing apparatus according to claim 1, further comprising:

an audio signal storage that stores the first audio signal in advance,

wherein the output outputs the first audio signal read out from the audio signal storage and also outputs the extracted providing audio signal.

5. The audio processing apparatus according to claim 4,

wherein the main sound includes music data.

6. The audio processing apparatus according to claim 1, further comprising:

a sample sound storage that stores a sample audio signal related to the providing audio signal,

wherein the audio extractor compares a feature amount of the surrounding audio signal with a feature amount of the sample audio signal recorded in the sample sound storage, and extracts an audio signal having a feature amount similar to the feature amount of the sample audio signal as the providing audio signal.

7. The audio processing apparatus according to claim 1,

wherein the audio processing apparatus further includes an audio output that outputs (i) the providing audio signal along with the first audio signal without a delay in a case in which the first output pattern is selected, (ii) the providing audio signal with a delay after outputting only the first audio signal in a case in which the second output pattern is selected, or (iii) only the first audio signal in a case in which the third output pattern is selected.

8. The audio processing apparatus according to claim 7, further comprising:

a no-voice segment detector that detects a no-voice segment extending from a point at which an output of the first audio signal finishes to a point at which a subsequent first audio signal is input,

wherein, in a case in which the second output pattern is selected, the audio output determines whether the no-voice segment has been detected by the no-voice segment detector, and in a case in which it is determined that the no-voice segment has been detected, the audio output outputs the providing audio signal with the delay in the no-voice segment.

9. The audio processing apparatus according to claim 7, further comprising:

a speech rate detector that detects a rate of speech in the first audio signal,

wherein, in a case in which the second output pattern is selected, the audio output determines whether the detected rate of speech is lower than a predetermined rate, and in a case in which it is determined that the rate of speech is lower than the predetermined rate, the audio output outputs the providing audio signal with the delay.

10. The audio processing apparatus according to claim 7, further comprising:

a no-voice segment detector that detects a no-voice segment extending from a point at which an output of the first audio signal finishes to a point at which a subsequent first audio signal is input,

wherein, in a case in which the second output pattern is selected, the audio output determines whether the detected no-voice segment extends for or longer than a predetermined duration, and in a case in which it is determined that the no-voice segment extends for or longer than the predetermined duration, the audio output outputs the providing audio signal with the delay in the no-voice segment.

13. The audio processing apparatus according to claim 1,

wherein the output outputs the extracted providing audio signal on a predetermined priority basis.

11. An audio processing method, comprising:

acquiring a surrounding audio signal indicating a sound surrounding a user;

extracting, from the acquired surrounding audio signal, (1) a main sound that is output without delay, (2) a providing audio signal, including prioritized sounds other than the main signal, that is stored and selectably output with or without delay, and (3) sounds that are not stored and do not need to be output;

selecting one of a plurality of output patterns of audio signals; and

outputting a first audio signal indicating a main sound and the providing audio signal,

wherein the selecting selects any one of (i) a first output pattern in which the providing audio signal is output along with the main sound without a delay, (ii) a second output pattern in which the providing audio signal is output with a delay after only the main sound is output, and (iii) a third output pattern in which only the main sound is output.

12. A non-transitory computer-readable recording medium having a program to be used in an audio processing apparatus recorded thereon, the program causing a computer of the audio processing apparatus to perform a method comprising:

acquiring a surrounding audio signal indicating a sound surrounding a user;

extracting, from the acquired surrounding audio signal, (1) a main sound that is output without delay, (2) a providing audio signal, including prioritized sounds other than the main signal, that is stored and selectably output with or without delay, and (3) sounds that are not stored and do not need to be output;

selecting one of a plurality of output patterns of audio signals; and

outputting a first audio signal indicating a main sound and the providing audio signal,

wherein the selecting selects any one of (i) a first output pattern in which the providing audio signal is output along with the main sound without a delay, (ii) a second output pattern in which the providing audio signal is output with a delay after only the main sound first audio signal is output, and (iii) a third output pattern in which only the main sound is output.