Signal processing apparatus, signal processing method, program, and recording medium

• US 20060291816A1
• Filed: 06/27/2006
• Published: 12/28/2006
• Est. Priority Date: 06/28/2005
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Ozawa, Kazuhiko
• Granted Patent: US 8,547,416 B2
• Time in Patent Office: Days
• US Class Current: 386/248
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 5/772 the recording apparatus and the television camera being placed in the same enclosure
• CPC Class Codes: H04N 5/775 between a recording apparatus and a television receiver
• CPC Class Codes: H04N 5/781 on disks or drums
• CPC Class Codes: H04N 5/85 on discs or drums
• CPC Class Codes: H04N 9/8042 involving data reduction

Abstract

A signal processing apparatus processing a video signal and an audio signal in synchronization with the video signal includes generating means for generating information indicating a probability of a certain subject appearing in the image on the basis of the video signal that is input; determining means for determining whether the certain subject appears in the image on the basis of the information generated by the generating means; and directional characteristic varying means for, if the determining means determines that the certain subject appears in the image, varying a directional characteristic of the audio signal so as to increase the level of the audio signal collected from the direction of the subject and/or to decrease the levels of the audio signals collected from directions other than the direction of the subject.

Claims

1. A signal processing apparatus processing a video signal and an audio signal in synchronization with the video signal, the signal processing apparatus comprising:

generating means for generating information indicating a probability of a certain subject appearing in the image on the basis of the video signal that is input;

determining means for determining whether the certain subject appears in the image on the basis of the information generated by the generating means; and

directional characteristic varying means for, if the determining means determines that the certain subject appears in the image, varying a directional characteristic of the audio signal so as to increase the level of the audio signal collected from the direction of the subject and/or to decrease the levels of the audio signals collected from directions other than the direction of the subject.

2. The signal processing apparatus according to claim 1, further comprising frequency characteristic varying means for, if the determining means determines that the certain subject appears in the image, varying a frequency characteristic of the audio signal so as to increase the level of the frequency band including a sound emitted from the subject and/or to decrease the levels of frequency bands other than the frequency band including the sound emitted from the subject.

3. The signal processing apparatus according to claim 2,

wherein the generating means includes

extracting means for extracting an area including the pixels of a certain color from the image corresponding to the input video signal;

detecting means for detecting a characteristic value pattern of the pixels in the area extracted by the extracting means on the basis of a luminance or a color difference of the pixels in the area;

storing means for storing in advance the characteristic value pattern detected from the image including the certain subject; and

calculating means for comparing the characteristic value pattern detected by the detecting means with the characteristic value pattern stored in the storing means to calculate a correlation coefficient between the characteristic value patterns,

wherein the correlation coefficient calculated by the calculating means is generated as the information indicating the probability of the certain subject appearing in the image.

4. The signal processing apparatus according to claim 3, further comprising image cropping means for cutting out a certain area in the image,

wherein the extracting means extracts the area including the pixels of the certain color from the certain area cut out by the image cropping means.

5. The signal processing apparatus according to claim 4,

wherein the certain subject is a person, and

wherein, if the determining means determines that the person appears in the image, the directional characteristic varying means varies the directional characteristic of the audio signal so as to increase the level of the audio signal collected from a forward direction and/or to decrease the levels of the audio signals collected from directions other than the forward direction and the frequency characteristic varying means varies the frequency characteristic of the audio signal so as to increase the level of the frequency band including the voice of the person and/or to decrease the levels of frequency bands other than the frequency band including the voice of the person.

6. The signal processing apparatus according to claim 4, further comprising setting means for setting a control level used for controlling the degree of the variation of the directional characteristic of the audio signal by the directional characteristic varying means or the degree of the variation of the frequency characteristic of the audio signal by the frequency characteristic varying means.

7. The signal processing apparatus according to claim 6,

wherein the setting means normalizes information indicating the size of the area extracted by the extracting means and sets the normalized information as the control level.

8. The signal processing apparatus according to claim 6,

wherein the setting means normalizes the correlation coefficient calculated by the calculating means and sets the normalized correlation coefficient as the control level.

9. The signal processing apparatus according to claim 8,

wherein the video signal and the audio signal subjected to the process in the directional characteristic varying means or the frequency characteristic varying means are recorded on a recording medium by a recording unit.

10. The signal processing apparatus according to claim 8,

wherein the video signal and the audio signal are output by a reproducing apparatus that reproduces the data recorded on a recording medium.

11. A signal processing method for a signal processing apparatus processing a video signal and an audio signal in synchronization with the video signal, the method comprising the steps of:

generating information indicating a probability of a certain subject appearing in the image on the basis of the video signal that is input;

determining whether the certain subject appears in the image on the basis of the generated information; and

varying, if it is determined that the certain subject appears in the image, a directional characteristic of the audio signal so as to increase the level of the audio signal collected from the direction of the subject and/or to decrease the levels of the audio signals collected from directions other than the direction of the subject.

12. A program causing a signal processing apparatus that processes a video signal and an audio signal in synchronization with the video signal to perform signal processing, the program causing a computer to execute the steps of:

controlling generation of information indicating a probability of a certain subject appearing in the image on the basis of the video signal that is input;

controlling determination of whether the certain subject appears in the image on the basis of the generated information; and

controlling, if it is determined that the certain subject appears in the image, variation of a directional characteristic of the audio signal so as to increase the level of the audio signal collected from the direction of the subject and/or to decrease the levels of the audio signals collected from directions other than the direction of the subject.

13. A recording medium having the program according to claim 12 recorded therein.

14. A signal processing apparatus processing an audio signal associated with an image of a subject, captured by imaging means, the signal processing apparatus comprising:

image receiving means for receiving the image of the subject;

sound receiving means for receiving a sound;

model storing means for storing characteristic value models of a plurality of subjects;

characteristic value extracting means for extracting a characteristic value of the image of the subject;

detecting means for comparing the characteristic value of the image of the subject, extracted by the characteristic value extracting means, with the characteristic value models stored in the model storing means to detect an image of a certain subject; and

controlling means for, after the image of the certain subject is detected, enhancing a directional characteristic of the sound receiving means, which is associated with the detected image of the certain subject, and/or the level of a frequency band, which is associated with the image of the certain subject, or reducing the directional characteristic of the sound receiving means, which is not associated with the detected image of the certain subject, and/or the level of the frequency band, which is not associated with the image of the certain subject.

15. A signal processing apparatus processing a video signal and an audio signal in synchronization with the video signal, the signal processing apparatus comprising:

a generating unit generating information indicating a probability of a certain subject appearing in the image on the basis of the video signal that is input;

a determining unit determining whether the certain subject appears in the image on the basis of the information generated by the generating unit; and

a directional characteristic varying unit varying, if the determining unit determines that the certain subject appears in the image, a directional characteristic of the audio signal so as to increase the level of the audio signal collected from the direction of the subject and/or to decrease the levels of the audio signals collected from directions other than the direction of the subject.

16. A signal processing apparatus processing an audio signal associated with an image of a subject, captured by an imaging unit, the signal processing apparatus comprising:

an image receiving unit receiving the image of the subject;

a sound receiving unit receiving a sound;

a model storing unit storing characteristic value models of a plurality of subjects;

a characteristic value extracting unit extracting a characteristic value of the image of the subject;

a detecting unit comparing the characteristic value of the image of the subject, extracted by the characteristic value extracting unit, with the characteristic value models stored in the model storing unit to detect an image of a certain subject; and

a controlling unit enhancing, after the image of the certain subject is detected, a directional characteristic of the sound receiving unit, which is associated with the detected image of the certain subject, and/or the level of a frequency band, which is associated with the image of the certain subject, or reducing the directional characteristic of the sound receiving unit, which is not associated with the detected image of the certain subject, and/or the level of the frequency band, which is not associated with the image of the certain subject.