IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD AND PROGRAM

• US 20110273531A1
• Filed: 12/03/2010
• Published: 11/10/2011
• Est. Priority Date: 12/14/2009
• Status: Active Grant

• Current Assignee: Saturn Licensing LLC (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: ITO, Atsushi, YAMAZAKI, Toshio, KOBAYASHI, Seiji
• Granted Patent: US 8,866,884 B2
• Time in Patent Office: Days
• US Class Current: 348/43
• CPC Class Codes: H04N 13/261 with monoscopic-to-stereoscopic image conversion
• CPC Class Codes: H04N 13/275 from 3D object models, e.g. computer-generated stereoscopic image signals
• CPC Class Codes: H04N 2213/003 Aspects relating to the "2D+depth" image format

Abstract

An image processing apparatus includes an image input unit that inputs a two-dimensional image signal, a depth information output unit that inputs or generates depth information of image areas constituting the two-dimensional image signal, an image conversion unit that receives the image signal and the depth information from the image input unit and the depth information output unit, and generates and outputs a left eye image and a right eye image for realizing binocular stereoscopic vision, and an image output unit that outputs the left and right eye images. The image conversion unit extracts a spatial feature value of the input image signal, and performs an image conversion process including an emphasis process applying the feature value and the depth information with respect to the input image signal, thereby generating at least one of the left eye image and the right eye image.

Claims

1. An image processing apparatus comprising:

an image input unit that inputs a two-dimensional image signal;

a depth information output unit that inputs or generates depth information of image areas constituting the two-dimensional image signal;

an image conversion unit that receives the image signal and the depth information from the image input unit and the depth information output unit, and generates and outputs a left eye image and a right eye image for realizing binocular stereoscopic vision; and

an image output unit that outputs the left eye image and the right eye image, which are output from the image conversion unit,

wherein the image conversion unit extracts a spatial feature value of the input image signal, and performs an image conversion process including an emphasis process applying the feature value and the depth information with respect to the input image signal, thereby generating at least one of the left eye image and the right eye image.

2. The image processing apparatus according to claim 1, wherein the image conversion unit extracts a luminance differential signal of the input image signal, sets the luminance differential signal as the feature value, generates a corrected differential signal by performing gain control based on depth information in units of image areas with respect to the differential signal, generates any one of conversion signals, which are obtained by adding the corrected differential signal to the input image signal or subtracting the corrected differential signal from the input image signal, as the left eye image or the right eye image, and outputs a non-conversion signal, which corresponds to the input image signal not subject to a process, as an eye image different from the conversion signal.

3. The image processing apparatus according to claim 1, wherein the image conversion unit extracts a luminance differential signal of the input image signal, sets the luminance differential signal as the feature value, generates a corrected differential signal by performing gain control based on depth information in units of image areas with respect to the differential signal, generates signals, which are obtained by adding the corrected differential signal to the input image signal and subtracting the corrected differential signal from the input image signal, and generates a pair of the two signals as a pair of the left eye image and the right eye image.

4. The image processing apparatus according to claim 2 or 3, wherein the image conversion unit performs a correction process of the luminance differential signal based on large gain with respect to an image area where the depth information which is output from the depth information output unit is large, and a correction process of the luminance differential signal based on small gain with respect to an image area where the depth information is small, thereby generating the corrected differential signal.

5. The image processing apparatus according to any one of claims 2 and 3, wherein the image conversion unit generates signals, which are obtained by adding a signal obtained by non-linearly converting the corrected differential signal to the input image signal or subtracting the signal from the input image signal, and generates any one of the signals as the left eye image or the right eye image.

6. The image processing apparatus according to claim 1, wherein the image conversion unit comprises:

a phase control signal generation part that generates a differential signal for an input signal;

a non-linear conversion part that performs a non-linear conversion process with respect to the differential signal generated by the phase control signal generation part;

an image synthesizing part that adds output of the non-linear conversion part to the input signal or subtracting the output of the non-linear conversion part from the input signal; and

processing paths through which an image conversion process by image conversion modules including the phase control signal generation part, the non-linear conversion part and the image synthesizing part is repeated,

wherein the image conversion unit selects and synthesizes output of each image conversion module constituting the processing path according to the depth information which is output from the depth information output unit, thereby generating the left eye image or the right eye image.

7. The image processing apparatus according to claim 1, wherein the image conversion unit comprises:

a phase control signal generation part that generates a differential signal for an input signal;

a gain control part that performs gain control based on a gain coefficient, which is set according to the depth information which is output from the depth information output unit, with respect to the differential signal generated by the phase control signal generation part, thereby generating a corrected differential signal;

a non-linear conversion part that performs a non-linear conversion process with respect to output of the gain control part;

an image synthesizing part that adds output of the non-linear conversion part to the input signal or subtracting the output of the non-linear conversion part from the input signal;

processing paths through which an image conversion process by image conversion modules including the phase control signal generation part, the gain control part, the non-linear conversion part and the image synthesizing part is repeated;

a weight value calculation section that sets weight values of each image conversion module constituting the processing path according to the depth information which is output from the depth information output unit; and

an image synthesizing section that synthesizes output of each image conversion module constituting the processing path according to the weight values calculated by the weight value calculation part, thereby generating the left eye image or the right eye image.

8. The image processing apparatus according to claim 1, wherein the image conversion unit comprises:

a phase control signal generation part that generates a differential signal for an input signal;

a multiplication processing part that generates a corrected differential signal by multiplying the differential signal generated by the phase control signal generation part by a gain coefficient set according to the depth information which is output from the depth information output unit;

a non-linear conversion part that performs a non-linear conversion process with respect to output of the multiplication processing part;

an image synthesizing part that adds output of the non-linear conversion part to the input signal or subtracting the output of the non-linear conversion part from the input signal;

processing paths through which an image conversion process by image conversion modules including the phase control signal generation part, the multiplication processing part, the non-linear conversion part and the image synthesizing part is repeated;

a weight value calculation section that sets weight values of each image conversion module constituting the processing path according to the depth information which is output from the depth information output unit; and

an image synthesizing section that synthesizes output of each image conversion module constituting the processing path according to the weight values calculated by the weight value calculation part, thereby generating the left eye image or the right eye image.

9. The image processing apparatus according to any one of claims 1 to 8, wherein the image conversion unit generates the left eye image and the right eye image with respect to each frame constituting a moving image.

10. The image processing apparatus according to claim 9, further comprising an image output unit that alternately outputs the left eye image and the right eye image, which is generated by the image conversion unit, at a frame rate twice as fast as a frame rate of an input image frame.

11. The image processing apparatus according to any one of claims 1 to 8, wherein the image conversion unit alternately generates either the left eye image or the right eye image with respect to each frame constituting a moving image.

12. The image processing apparatus according to any one of claims 1 to 8, wherein the image conversion unit generates the left eye image and the right eye image with respect to each frame constituting a moving image, and generates a binocular parallax image alternately including line data constituting the generated left and right eye images.

13. The image processing apparatus according to any one of claims 1 to 12, wherein the image conversion unit generates the left eye image and the right eye image based on a setting in which an addition signal of the generated left and right eye images is equivalent to or almost equivalent to the input signal.

14. The image processing apparatus according to any one of claims 1 to 13, further comprising an image display unit that displays the images generated by the image conversion unit.

15. The image processing apparatus according to claim 14, wherein the image display unit performs a time division stereoscopic display process of alternately outputting the left eye image and the right eye image.

16. The image processing apparatus according to claim 15, wherein, when the image display unit performs a time division stereoscopic display process of alternately outputting the left eye image and the right eye image, the image display unit switches an output switching timing of the left eye image and the right eye image in synchronization with shutter switching of right and left parts of glasses worn by an image observer.

17. The image processing apparatus according to claim 14, wherein the image display unit has a configuration in which polarizing filters are bonded to the front surface thereof, and displays the binocular parallax image alternately including the line data constituting the left eye image and the right eye image generated by the image conversion unit, polarizing directions of the polarizing filters being set to change for each horizontal line.

18. An image processing method in an image processing apparatus, comprising the steps of:

inputting a two-dimensional image signal by an image input unit;

receiving or generating depth information of image areas constituting the two-dimensional image signal by a depth information output unit;

receiving the image signal, which is output from the image input unit, and the depth information, which is output from the depth information output unit, and generating and outputting a left eye image and a right eye image for realizing binocular stereoscopic vision by an image conversion unit; and

outputting the left eye image and the right eye image, which are output from the image conversion unit, by an image output unit

wherein, in the step of receiving the image signal and the depth information, a spatial feature value of the input image signal is extracted, and an image conversion process including an emphasis process applying the feature value and the depth information is performed with respect to the input image signal, resulting in generation of at least one of the left eye image and the right eye image.

19. A program that causes a computer to execute image processing of an image processing apparatus, the image processing comprising:

inputting a two-dimensional image signal by an image input unit;

receiving or generating depth information of image areas constituting the two-dimensional image signal by a depth information output unit;

receiving the image signal, which is output from the image input unit, and the depth information, which is output from the depth information output unit, and generating and outputting a left eye image and a right eye image for realizing binocular stereoscopic vision by an image conversion unit; and

outputting the left eye image and the right eye image, which are output from the image conversion unit, by an image output unit

wherein, in the step of receiving the image signal and the depth information, a spatial feature value of the input image signal is extracted, and an image conversion process including an emphasis process applying the feature value and the depth information is performed with respect to the input image signal, resulting in generation of at least one of the left eye image and the right eye image.