IMAGE PROCESSING DEVICE, IMAGE PROCESSING METHOD, IMAGE PROCESSING PROGRAM, AND IMAGING DEVICE

• US 20110199458A1
• Filed: 02/09/2011
• Published: 08/18/2011
• Est. Priority Date: 02/16/2010
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Yamamoto, Kenji, Hayasaka, Kengo
• Granted Patent: US 9,131,222 B2
• Time in Patent Office: Days
• US Class Current: 348/43
• CPC Class Codes: G06T 2207/10052 Images from lightfield camera
• CPC Class Codes: G06T 7/557 from light fields, e.g. from plenoptic cameras
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: H04N 13/232 using fly-eye lenses, e.g. arrangements of circular lenses

Abstract

An image processing device includes a phase difference detection portion configured to detect phase difference between parallax images by performing correlation value calculation with a plurality of parallax images, and generate phase difference distribution in an image. The phase difference detection portion performs the phase difference detection individually along two or more directions different from each other, and generates the phase difference distribution by utilizing results of the phase difference detection regarding the two or more directions.

Claims

1. An image processing device comprising:

a phase difference detection portion configured to detect a phase difference between parallax images by performing a correlation value calculation with a plurality of parallax images, and generate phase difference distribution in an image, wherein

the phase difference detection portion performs the phase difference detection individually along two or more directions different from each other, and generates the phase difference distribution by utilizing results of the phase difference detection regarding the two or more directions.

2. The image processing device according to claim 1, wherein

the phase difference detection portion makes a determination as to reliability of phase difference for each of the results of the phase difference detection regarding the two or more directions on each unit area basis, and

the phase difference detection portion employs the result of the phase difference detection regarding a direction of highest reliability on each unit area basis and combines the results, based on results of the determination as to reliability, to thereby generate the phase difference distribution.

3. The image processing device according to claim 2, wherein

the phase difference detection portion makes the determination as to reliability based on a correlation characteristic line indicating a relationship between a pixel position and a correlation value in the correlation value calculation.

4. The image processing device according to claim 1, wherein

regarding at least one of parallax images of the two or more directions, the phase difference detection portion executes edge detection processing to detect an edge area along a parallax direction of the at least one of parallax images, and

the phase difference detection portion employs the result of the phase difference detection regarding a direction different from direction of a detected edge area on each unit area basis and combines the results, based on a result of the edge detection, to thereby generate the phase difference distribution.

5. The image processing device according to claim 4, wherein

the phase difference detection portion performs the phase difference detection along the same direction as direction of the edge detection processing for a unit area from which no edge area is detected in the edge detection processing, and

the phase difference detection portion performs the phase difference detection along one of directions different from the direction of the edge detection processing for a unit area from which an edge area is detected in the edge detection processing.

6. The image processing device according to claim 1, wherein

the phase difference detection portion performs the phase difference detection individually along a horizontal direction and a vertical direction in the plurality of parallax images, and

the phase difference detection portion generates the phase difference distribution by utilizing the result of the phase difference detection regarding the horizontal direction and the result of the phase difference detection regarding the vertical direction.

7. The image processing device according to claim 1, wherein

the plurality of parallax images are generated by an imaging optical system having an imaging lens, an imaging element to acquire imaging data based on a received light beam, and a microlens array that is disposed between the imaging lens and the imaging element and is configured by allocating one microlens to a plurality of pixels of the imaging element, based on imaging data acquired in a state in which traveling direction of a light beam is retained.

8. The image processing device according to claim 7, further comprising

a distance information calculation portion configured to calculate information on distance from the imaging lens to a refocusing plane based on the phase difference distribution generated by the phase difference detection portion.

9. The image processing device according to claim 8, further comprising

a refocusing factor setting portion configured to set a refocusing factor based on the information on distance calculated by the distance information calculation portion.

10. The image processing device according to claim 9, further comprising

a rearrangement processing block configured to execute rearrangement processing for the imaging data by using the refocusing factor set by the refocusing factor setting portion.

11. An image processing method comprising:

detecting a phase difference between parallax images by performing a correlation value calculation with a plurality of parallax images, and generating phase difference distribution in an image, wherein

the phase difference detection is performed individually along two or more directions different from each other, and the phase difference distribution is generated by utilizing results of the phase difference detection regarding the two or more directions.

12. An image processing computer program product stored on a computer readable medium that when executed by a processor performs a step of:

detecting a phase difference between parallax images by performing a correlation value calculation with a plurality of parallax images, and generating phase difference distribution in an image, wherein

the phase difference detection is performed individually along two or more directions different from each other, and the phase difference distribution is generated by utilizing results of the phase difference detection regarding the two or more directions.

13. An imaging device comprising:

an imaging optical system; and

an image processing device configured to execute image processing for imaging data acquired by the imaging optical system, wherein

the image processing device has a phase difference detection portion that detects a phase difference between parallax images by performing correlation value calculation with a plurality of parallax images obtained from the imaging data directly or indirectly, and generates phase difference distribution in an image, and

the phase difference detection portion performs the phase difference detection individually along two or more directions different from each other, and generates the phase difference distribution by utilizing results of the phase difference detection regarding the two or more directions.

14. The imaging device according to claim 13, wherein

the imaging optical system has an imaging lens, an imaging element to acquire the imaging data based on a received light beam, and a microlens array that is disposed between the imaging lens and the imaging element and is configured by allocating one microlens to a plurality of pixels of the imaging element, and

the plurality of parallax images are generated based on the imaging data acquired in a state in which traveling direction of a light beam is retained.

15. An image processing device comprising

a phase difference detection means for detecting a phase difference between parallax images by performing correlation value calculation with a plurality of parallax images, and generating phase difference distribution in an image, wherein

phase difference detection means performs the phase difference detection individually along two or more directions different from each other, and generates the phase difference distribution by utilizing results of the phase difference detection regarding the two or more directions.