IMAGE PROCESSING DEVICE, METHOD AND RECORDING MEDIUM, STEREOSCOPIC IMAGE CAPTURE DEVICE, PORTABLE ELECTRONIC APPARATUS, PRINTER, AND STEREOSCOPIC IMAGE PLAYER DEVICE

• US 20140198188A1
• Filed: 03/18/2014
• Published: 07/17/2014
• Est. Priority Date: 09/21/2011
• Status: Active Grant

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Inventors: IZAWA, Katsutoshi
• Granted Patent: US 9,247,227 B2
• Time in Patent Office: Days
• US Class Current: 348/49
• CPC Class Codes: H04N 13/106 Processing image signals for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/207 using a single 2D image sensor
• CPC Class Codes: H04N 13/225 using parallax barriers
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

An image processing method includes: acquiring a first image and second image with a parallax, calculating a parallax indicating a shift amount of a corresponding pixel of the second image with respect to each pixel of the first image, deciding an image processing coefficient to perform edge emphasis or edge correction based on the parallax with respect to every image processing target pixel of the first image and the second image, and performing image processing on the first image and the second image using the image processing coefficient.

Claims

1. An image processing device comprising:

an image acquisition unit configured to acquire a first image and a second image with a parallax, which are taken as a stereoscopic image;

a parallax calculation unit configured to calculate a parallax indicating a shift amount of a corresponding pixel of the second image with respect to each pixel of the first image acquired by the image acquisition unit;

an image processing coefficient decision unit configured to decide an image processing coefficient to perform edge emphasis or edge correction with respect to every image processing target pixel of the first image and the second image acquired by the image acquisition unit, and decide the image processing coefficient for the target pixel based on the parallax calculated by the parallax calculation unit; and

an image processing unit configured to perform image processing using the image processing coefficient decided by the image processing coefficient decision unit for the first image and the second image acquired by the image acquisition unit.

2. The image processing device according to claim 1,

wherein the image processing coefficient decision unit decides the image processing coefficient such that a center of the image processing coefficient for the edge emphasis or the edge correction shifts from a center of a computation target pixel.

3. The image processing device according to claim 2,

wherein the image processing coefficient decision unit shifts the center of the image processing coefficient from the center of the computation target pixel according to a shift direction of the parallax.

4. The image processing device according to claim 1, wherein:

the image processing coefficient decision unit decides a filter coefficient as the image processing coefficient; and

the image processing unit performs filter processing by a filter using the filter coefficient.

5. The image processing device according to claim 4,

wherein the filter is a one-dimensional filter or a two-dimensional filter.

6. The image processing device according to claim 4,

wherein the image processing coefficient decision unit decides a filter coefficient in which at least one of a degree of the edge emphasis or the edge correction and a frequency band varies according to the parallax.

7. The image processing device according to claim 6,

wherein the image processing coefficient decision unit decides at least one of a filter coefficient that reduces the degree of the edge emphasis or the edge correction as the parallax increases and a filter coefficient that widens the frequency band as the parallax increases.

8. The image processing device according to claim 1,

wherein, when the parallax calculated by the parallax calculation unit is input for the target pixel, the image processing coefficient decision unit calculates a parallax shift amount according to a predetermined function for parallax correction or reads out a corresponding parallax shift amount from a predetermined look-up table for parallax correction, and

the image processing coefficient decision unit shifts the center of the image processing coefficient from the center of the computation target pixel according to the calculated or read parallax shift amount.

9. The image processing device according to claim 8, further comprising

a selection unit including a plurality of the predetermined functions or look-up tables for parallax correction, the selection unit configured to select any function or look-up table from the plurality of functions or the look-up tables by a user setting.

10. The image processing device according to claim 1,

wherein the parallax calculation unit detects a pixel on the second image corresponding to the target pixel of the first image by block matching between an image of a predetermined block size based on the target pixel of the first image and the second image, and

the parallax calculation unit calculates a parallax between the target pixel of the first image and the detected pixel on the second image.

11. The image processing device according to claim 10, wherein:

the parallax calculation unit holds images of the predetermined block size in the first image and the second image used at a calculation of the parallax;

the image processing unit has a computation target pixel range of a same size as the predetermined block size; and

the image processing unit performs image processing using the images of the predetermined block size of the first image and the second image held by the parallax calculation unit and the image processing coefficient of the computation target pixel range.

12. The image processing device according to claim 1, wherein:

the first image and the second image are color images of three primary colors; and

the image processing unit performs image processing using an identical image processing coefficient for each of color signals indicating the color images of three primary colors.

13. The image processing device according to claim 1, wherein:

the first image and the second image are color images of three primary colors; and

the image processing unit performs image processing using an identical image processing coefficient for a luminance signal and a color-difference signal that are generated from the color images of the three primary colors.

16. A stereoscopic image capture device comprising

the image processing device according to claim 1,

wherein the image acquisition unit includes:

a single imaging optical system; and

an imaging element in which object images passing through different regions of the imaging optical system are subjected to pupil division and formed, the imaging element configured to which acquire a first image and a second image with different parallaxes by performing photoelectric conversion on the object images passing through the different regions.

17. The stereoscopic image capture device according to claim 16,

wherein the imaging element includes a first group pixel and a second group pixel for photoelectric conversion, which are arranged in a matrix manner in an entire surface of an exposure region of the imaging element, the first group pixel being restricted in a light receiving direction of a light flux so as to receive light of only an object image passing through a first region of the imaging optical system, and the second group image being restricted in a light receiving direction of a light flux so as to receive light of only an object image passing through a second region of the imaging optical system, and

the imaging element can read out the first image and the second image from the first group pixel and the second group pixel.

18. A portable electronic apparatus comprising

the stereoscopic image capture device according to claim 16.

19. A printer comprising:

the image processing device according to claim 1; and

a print unit configured to create a photographic print for stereoscopic view based on the first image and the second image processed by the image processing device.

20. A stereoscopic image player device comprising:

the image processing device according to claim 1; and

an image display unit configured to display an image for stereoscopic view based on the first image and the second image processed by the image processing device.

14. An image processing method comprising:

an image acquisition step of acquiring a first image and a second image with a parallax, which are taken as a stereoscopic image;

a parallax calculation step of calculating a parallax indicating a shift amount of a corresponding pixel of the second image with respect to each pixel of the first image acquired in the image acquisition step;

an image processing coefficient decision step of deciding an image processing coefficient to perform edge emphasis or edge correction with respect to every image processing target pixel of the first image and the second image acquired in the image acquisition step, and deciding the image processing coefficient for the target pixel based on the parallax calculated in the parallax calculation step; and

an image processing step of performing image processing using the image processing coefficient decided in the image processing coefficient decision step for the first image and the second image acquired in the image acquisition step.

15. A computer-readable non-transitory recording medium in which, when an instruction stored in the recording medium is read out by a processor, the processor executes:

an image acquisition step of acquiring a first image and a second image with a parallax, which are taken as a stereoscopic image;

a parallax calculation step of calculating a parallax indicating a shift amount of a corresponding pixel of the second image with respect to each pixel of the first image acquired in the image acquisition step;

an image processing coefficient decision step of deciding an image processing coefficient to perform edge emphasis or edge correction with respect to every image processing target pixel of the first image and the second image acquired in the image acquisition step, and deciding the image processing coefficient for the target pixel based on the parallax calculated in the parallax calculation step; and

an image processing step of performing image processing using the image processing coefficient decided in the image processing coefficient decision step for the first image and the second image acquired in the image acquisition step.