Imaging device, imaging method and recording medium for adjusting imaging conditions of optical systems based on viewpoint images

• US 8,773,509 B2
• Filed: 07/16/2010
• Issued: 07/08/2014
• Est. Priority Date: 07/17/2009
• Status: Expired due to Fees

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Inventors: Pan, Yi
• Primary Examiner(s): Patel, Jay
• Assistant Examiner(s): GEROLEO, FRANCIS
• Application Number:
  US12/838,074
• Publication Number:
  US 20110012998A1
• Time in Patent Office: 1,453 Days
• Field of Search: 348/47
• US Class Current: 348/47
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 23/673 based on contrast or high frequency components of image signals, e.g. hill climbing method
• CPC Class Codes: H04N 23/71 Circuitry for evaluating the brightness variation
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means

Abstract

An imaging method includes: calculating an amount of parallax between a reference optical system and an adjustment target optical system; setting coordinates of an imaging condition evaluation region corresponding to the first viewpoint image outputted by the reference optical system; calculating coordinates of an imaging condition evaluation region corresponding to the second viewpoint image outputted by the adjustment target optical system, based on the set coordinates of the imaging condition evaluation region corresponding to the first viewpoint image, and on the calculated amount of parallax; and adjusting imaging conditions of the reference optical system and the adjustment target optical system, based on image data in the imaging condition evaluation region corresponding to the first viewpoint image, at the set coordinates, and on image data in the imaging condition evaluation region corresponding to the second viewpoint image, at the calculated coordinates, and outputting the viewpoint images in the adjusted imaging conditions.

Claims

1. An imaging device comprising:

an imaging unit configured to photoelectrically convert subject images formed via a first optical system and a second optical system respectively, by an imaging element, and output a first viewpoint image and a second viewpoint image;

a setting unit configured to set coordinates of the center of an automatic focusing (AF) evaluation value calculation region of the first viewpoint image, said AF evaluation value calculation region being used for automatic focusing for said first viewpoint image;

a parallax amount calculation unit configured to calculate an amount of parallax of corresponding subjects between the first viewpoint image and the second viewpoint image based on a parallax vector connecting

a point associated with a subject in the first viewpoint image, the point being detected in the AF evaluation value calculation region of the first viewpoint image whose coordinates of the center have been set by the setting unit and

a corresponding point associated with a corresponding subject in the second viewpoint image, wherein said corresponding point in the second viewpoint image corresponds to the point in the first viewpoint image;

a determination unit configured to determine coordinates of the center of an automatic focusing (AF) evaluation value calculation region of the second viewpoint image, based on

the coordinates of the center of the AF evaluation value calculation region of the first viewpoint image set by the setting unit and

the amount of parallax calculated by the parallax amount calculation unit, said AF evaluation value calculation region of the second viewpoint image being used for automatic focusing for said second viewpoint image; and

a control unit configured to

adjust an imaging condition of the first optical system based on image data in the AF evaluation value calculation region of the first viewpoint image, for focusing said first viewpoint image,

adjust an imaging condition of the second optical system based on image data in the AF evaluation value calculation region of the second viewpoint image, for focusing said second viewpoint image, and

control the imaging unit to output the first viewpoint image and the second viewpoint image under the respective adjusted imaging conditions for the first and second optical systems.

2. The imaging device according to claim 1, wherein

the imaging condition evaluation region includes a focus evaluation value calculation region.

3. The imaging device according to claim 2, further comprising:

a reference optical system-focus control unit, while changing a focus position of the reference optical system, configured to detect a focused position at which a contrast in the focus evaluation value calculation region located at the coordinates set by the setting unit becomes a local maximum, in the first viewpoint image outputted by the reference optical system, and to move the focus position of the reference optical system to the detected focused position; and

an adjustment target optical system-focus control unit, while changing a focus position of the adjustment target optical system, configured to detect a focused position at which a contrast in the focus evaluation value calculation region located at the coordinates calculated by the calculation unit becomes a local maximum, in the second viewpoint image outputted by the adjustment target optical system, and to move the focus position of the adjustment target optical system to the detected focused position.

4. The imaging device according to claim 1, wherein

the imaging condition evaluation region includes a subject luminance calculation region.

5. The imaging device according to claim 4, further comprising:

a reference optical system-exposure control unit configured to control an exposure of the reference optical system so that a subject luminance detected from the subject luminance calculation region located at the coordinates set by the setting unit becomes appropriate, in the first viewpoint image outputted by the reference optical system; and

an adjustment target optical system-exposure control unit configured to control an exposure of the adjustment target optical system so that a subject luminance detected from the subject luminance calculation region located at the coordinates calculated by the calculation unit becomes appropriate, in the second viewpoint image outputted by the adjustment target optical system.

6. The imaging device according to claim 4, further comprising:

a reference optical system-exposure control unit configured to control an exposure of the reference optical system so that an average weight of subject luminances detected from respective divided regions of the subject luminance calculation region including a plurality of the divided regions which are weighted with the coordinates set by the setting unit at the center, becomes appropriate, in the first viewpoint image outputted by the reference optical system; and

an adjustment target optical system-exposure control unit configured to control an exposure of the adjustment target optical system so that an average weight of subject luminances detected from respective divided regions of the subject luminance calculation region including a plurality of the divided regions which are weighted with the coordinates calculated by the calculation unit at the center, becomes appropriate, in the second viewpoint image outputted by the adjustment target optical system.

7. The imaging device according to claim 1, wherein

the imaging condition evaluation region includes a light adjustment-luminance calculation region.

8. The imaging device according to claim 1, wherein

the parallax amount calculation unit calculates the amount of parallax between the reference optical system and the adjustment target optical system, based on a parallax vector connecting a feature point detected from the imaging condition evaluation region whose coordinates have been set by the setting unit and which corresponds to the reference optical system, and a corresponding point corresponding to the feature point in the viewpoint image from the adjustment target optical system.

9. The imaging device according to claim 8, wherein the parallax amount calculation unit configured to calculate the parallax amount based on the average value of a plurality of parallax vectors for subjects with different distances, the mode value of the plurality of the parallax vectors, the longest parallax vector among the plurality of the parallax vectors, or the nearest parallax vector to the imaging device among the plurality of the parallax vectors.

10. The imaging device according to claim 9, wherein the center of the AF evaluation value calculation region of the first viewpoint image is the center of a subject detected as a specific type of a subject.

11. The imaging device according to claim 1, wherein

the parallax amount calculation unit calculates the amount of parallax depending on at least one of a cross-point adjustment amount, an automatic parallax adjustment amount and a manual parallax adjustment amount.

12. The imaging device according to claim 1, further comprising a stereoscopic image output unit configured to output a stereoscopic image based on the left and right viewpoint images outputted by the imaging unit.

13. An imaging method in which an imaging device comprising an imaging unit configured to photoelectrically convert subject images formed via a first optical system and a second optical system respectively, by an imaging element, and output a first viewpoint image and a second viewpoint image, executes the steps of:

setting coordinates of the center of an automatic focusing (AF) evaluation value calculation region of the first viewpoint image, said AF evaluation value calculation region being used for automatic focusing for said first viewpoint image;

calculating an amount of parallax of corresponding subjects between the first viewpoint image and the second viewpoint image based on a parallax vector connecting

a point associated with a subject in the first viewpoint image, the point being detected in the AF evaluation value calculation region of the first viewpoint image whose coordinates of the center have been set by the setting unit and

a corresponding point associated with a corresponding subject in the second viewpoint image, wherein said corresponding point in the second viewpoint image corresponds to the point in the first viewpoint image;

determining coordinates of the center of an automatic focusing (AF) evaluation value calculation region of the second viewpoint image, based on

the coordinates of the center of the AF evaluation value calculation region of the first viewpoint image and

the calculated amount of parallax, said AF evaluation value calculation region of the second viewpoint image being used for automatic focusing for said second viewpoint image; and

adjusting an imaging condition of the first optical system based on image data in the AF evaluation value calculation region of the first viewpoint image, for focusing said first viewpoint image, adjusting an imaging condition of the second optical system based on image data in the AF evaluation value calculation region of the second viewpoint image, for focusing said second viewpoint image, and controlling the imaging unit to output the first viewpoint image and the second viewpoint image under the respective adjusted imaging conditions for the first and second optical systems.

14. A non-transitory computer-readable recording medium including instructions stored thereon, such that when the instructions are read and executed by a processor, the processor is configured to perform the steps of:

setting coordinates of the center of an automatic focusing (AF) evaluation value calculation region of the first viewpoint image, said AF evaluation value calculation region being used for automatic focusing for said first viewpoint image;

calculating an amount of parallax of corresponding subjects between a first viewpoint image outputted by a first optical system and a second viewpoint image outputted by a second optical system, based on a parallax vector connecting

a point associated with a subject in the first viewpoint image, the point being detected in the AF evaluation value calculation region of the first viewpoint image whose coordinates of the center have been set by the setting unit and

a corresponding point associated with a corresponding subject in the second viewpoint image, wherein said corresponding point in the second viewpoint image corresponds to the point in the first viewpoint image;

determining coordinates of the center of an AF evaluation value calculation region of the second viewpoint image, based on

the coordinates of the center of the AF evaluation value calculation region of the first viewpoint image, and

the calculated amount of parallax, said AF evaluation value calculation region of the second viewpoint image being used for automatic focusing for said second viewpoint image; and

adjusting an imaging condition of the first optical system based on image data in the AF evaluation value calculation region of the first viewpoint image, for focusing said first viewpoint image, adjusting an imaging condition of the second optical system based on image data in the AF evaluation value calculation region of the second viewpoint image, for focusing said second viewpoint image, and outputting the first viewpoint image and the second viewpoint image under the respective adjusted imaging conditions for the first and second optical systems.