IMAGING APPARATUS, METHOD, PROGRAM, AND RECORDING MEDIUM USED IN THE PROGRAM

• US 20110298917A1
• Filed: 11/02/2010
• Published: 12/08/2011
• Est. Priority Date: 03/19/2010
• Status: Active Grant

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Inventors: Yanagita, Satoshi
• Granted Patent: US 8,310,538 B2
• Time in Patent Office: Days
• US Class Current: 348/135
• CPC Class Codes: G03B 5/02 Lateral adjustment of lens
• CPC Class Codes: G03B 5/08 Swing backs
• CPC Class Codes: H04N 13/139 Format conversion, e.g. of frame-rate or size
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6812 based on additional sensors, e.g. acceleration sensors
• CPC Class Codes: H04N 23/687 by shifting the lens or sensor position
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

According to the present invention, a cut-out size with a predetermined aspect ratio common to a plurality of images acquired from each of a plurality of imaging units is determined based on a minimum size among sizes of possible areas around an initial optical axis center, and an image for output is cut out from each of the plurality of images in the common cut-out size determined by a size determination unit based on the initial optical axis center before each vibration correction of the plurality of imaging units. Alternatively, according to the present invention, a cut-out size with a predetermined aspect ratio common to images from each of a plurality of imaging units is determined based on a minimum value of sizes of possible cut-out areas independent of vibration correction corresponding to the plurality of imaging units, and an image for output is cut out from each of the plurality of imaging units in the common cut-out size determined by the size determination unit based on the initial optical axis center before each vibration correction of the plurality of imaging units. As a result, output images, in which the optical axis center position is maintained before and after the camera shake correction, can be obtained, and stereoscopic vision is possible using the images for output.

Claims

1. An imaging apparatus comprising:

a plurality of imaging units that take subject images from different viewpoints;

a vibration detection unit that detects each vibration of the imaging units; and

a vibration correction unit that corrects the vibration of the subject image taken by each imaging unit based on the vibration of each imaging unit detected by the vibration detection unit, the imaging apparatus comprising;

a size determination unit that determines a cut-out size for cutting out an image for output from an image acquired from each of the plurality of imaging units, the size determination unit determining the cut-out size with a predetermined aspect ratio common to a plurality of images acquired from each of the plurality of imaging units based on a possible area in a minimum size among sizes of possible areas around an initial optical axis center included in areas common to prescribed imaging areas based on the initial optical axis center before each vibration correction by the plurality of imaging units and imaging areas after each vibration correction by the plurality of imaging units; and

a cut-out unit that cuts out the image for output from each of the plurality of images in the common cut-out size determined by the size determination unit based on the initial optical axis center before each vibration correction by the plurality of imaging units.

5. The imaging apparatus according to claim 1, comprising

an image complement unit that complements a complement target area, which is beyond a prescribed cut-out range, by an image of an effective pixel area of the imaging units equivalent to the complement target area if the image for output cut out by the cut-out unit includes the complement target area.

6. The imaging apparatus according to claim 1, comprising

a color complement unit that complements a complement target area, which is beyond a prescribed cut-out range of the imaging units, by a predetermined color if the image for output cut-out by the cut-out unit includes the complement target area.

7. The imaging apparatus according to claim 1, comprising

a panoramic image creation unit that creates a panoramic image by combining each image based on the initial optical axis center of each image for output cut out by the cut-out unit.

8. The imaging apparatus according to claim 1, comprising

a stereo matching calculation unit that performs stereo matching by setting an epipolar line based on the initial optical axis center of each image for output cut out by the cut-out unit and by calculating a correlation of each image for output along the epipolar line.

9. The imaging apparatus according to claim 1, comprising

a storage unit that associates and stores each image from each of the plurality of imaging units and the initial optical axis center position as well as the cut-out size of each image.

10. The imaging apparatus according to claim 1, comprising

a storage unit that associates and stores each image for output corresponding to each image acquired at the same imaging time from each of the plurality of imaging units.

11. The imaging apparatus according to claim 5, comprising

a storage unit that associates, with the image for output, coordinates of the complement target area of the image for output and identification information of the image for output including the minimum complement target area to store the coordinates and the identification information.

12. The imaging apparatus according to claim 1, comprising

a parallax adjustment unit that determines the cut-out positions of the images for output so that a parallax between the images for output becomes a predetermined parallax amount while the cut-out size of the images for output is maintained.

13. The imaging apparatus according to claim 1, comprising

an output unit that outputs a plane image or a stereoscopic image based on the images cut out by the cut-out unit.

14. The imaging apparatus according to claim 1, comprising

a designation unit that receives a designation of an enlargement position, wherein

the cut-out unit changes the position of cutting out the image for output in accordance with the enlargement position if the enlargement position received by the designation unit reaches a borderline of cutting out the image for output from the image.

15. The imaging apparatus according to claim 11, comprising

a plane image output unit that outputs the image with the minimum complement target area as a plane image based on the identification information stored in the storage unit.

16. The imaging apparatus according to claim 11, comprising:

a color complement unit that complements the complement target area stored in the storage unit by a predetermined color; and

an output unit that outputs a plane image or a stereoscopic image based on the image for which the color complement unit has complemented the color.

17. The imaging apparatus according to claim 9, comprising

a panoramic image creation unit that cuts out each image for output based on the initial optical axis center position and the cut-out size corresponding to each image stored in the storage unit and then combines each output image to create a panoramic image.

18. The imaging apparatus according to claim 9, comprising

a stereo matching calculation unit that cuts out each output image based on the initial optical axis center position and the cut-out size corresponding to each image stored in the storage unit and then sets the epipolar line to each image for output based on the initial optical axis center to calculate the correlation of each image for output along the epipolar line to perform the stereo matching.

2. An imaging apparatus comprising:

a plurality of imaging units that take subject images from different viewpoints;

a vibration detection unit that detects each vibration of the imaging units; and

a vibration correction unit that corrects the vibration of the subject image taken by each imaging unit based on the vibration of each imaging unit detected by the vibration detection unit, the imaging apparatus comprising;

a size determination unit that determines a cut-out size for cutting out an image for output from an image acquired from each of the plurality of imaging units, the size determination unit determining, for each of the plurality of imaging units, a possible cut-out area around an initial optical axis center included in an invariant imaging area independent of vibration correction by the vibration correction unit for each of the plurality of imaging units and then determining the cut-out size with a predetermined aspect ratio common to the images from each of the plurality of imaging units based on a minimum value of sizes of the possible cut-out areas corresponding to the plurality of imaging units; and

a cut-out unit that cuts out the image for output from each of the plurality of imaging units in the common cut-out size determined by the size determination unit based on the initial optical axis center before each vibration correction by the plurality of imaging units.

3. The imaging apparatus according to claim 2, wherein

the size determination unit determines, for each of the plurality of imaging units, the invariant imaging area based on a common area between two different imaging areas displaced to the maximum extent in a vertical direction and/or a horizontal direction by the vibration correction by the vibration correction unit for each of the plurality of imaging units.

4. The imaging apparatus according to claim 3, wherein

the size determination unit determines, for each of the plurality of imaging units, the common area between two different imaging areas displaced to the maximum extent in the vertical direction and/or the horizontal direction obtained by the vibration correction carried out at least twice by the vibration correction unit for each of the plurality of imaging units and sets the common area determined for each of the plurality of imaging units as the invariant imaging area corresponding to each imaging unit.

19. An imaging method executed by an imaging apparatus, the imaging apparatus comprising:

a plurality of imaging units that take subject images from different viewpoints;

a vibration detection unit that detects each vibration of the imaging units; and

a vibration correction unit that corrects the vibration of the subject image taken by each imaging unit based on the vibration of each imaging unit detected by the vibration detection unit, the imaging method comprising;

a step of determining a cut-out size with a predetermined aspect ratio common to a plurality of images acquired from each of the plurality of imaging units based on a possible area in a minimum size among sizes of possible areas around an initial optical axis center included in areas common to prescribed imaging areas based on the initial optical axis center before each vibration correction by the plurality of imaging units and imaging areas after each vibration correction by the plurality of imaging units to determine the cut-out size for cutting out an image for output from an image acquired from each of the plurality of imaging units; and

a step of cutting out the image for output from each of the plurality of images in the common cut-out size determined by the size determination unit based on the initial optical axis center before each vibration correction by the plurality of imaging units.

21. A program for causing the imaging apparatus to execute the imaging method according to claim 19.

22. A recording medium recording a computer-readable code of the program according to claim 21.

20. An imaging method executed by an imaging apparatus, the imaging apparatus comprising:

a plurality of imaging units that take subject images from different viewpoints;

a vibration detection unit that detects each vibration of the imaging units; and

a vibration correction unit that corrects the vibration of the subject image taken by each imaging unit based on the vibration of each imaging unit detected by the vibration detection unit, the imaging method comprising;

a step of determining, for each of the plurality of imaging units, a possible cut-out area around an initial optical axis center included in an invariant imaging area independent of vibration correction by the vibration correction unit for each of the plurality of imaging units and then determining a cut-out size with a predetermined aspect ratio common to the images from each of the plurality of imaging units based on a minimum value of sizes of the possible cut-out areas corresponding to the plurality of imaging units to determine the cut-out size for cutting out an image for output from an image acquired from each of the plurality of imaging units; and

a step of cutting out the image for output from each of the plurality of imaging units in the common cut-out size determined by the size determination unit based on the initial optical axis center before each vibration correction by the plurality of imaging units.