STEREO CAMERA

• US 20120293633A1
• Filed: 01/12/2011
• Published: 11/22/2012
• Est. Priority Date: 02/02/2010
• Status: Active Grant

• Current Assignee: Konica Minolta Holdings Incorporated (Konica Minolta Inc.)
• Original Assignee: Konica Minolta Holdings Incorporated (Konica Minolta Inc.)
• Inventors: Yamato, Hiroshi
• Granted Patent: US 9,109,891 B2
• Time in Patent Office: Days
• US Class Current: 348/47
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area
• CPC Class Codes: G03B 35/08 by simultaneous recording
• CPC Class Codes: G03B 35/10 having single camera with stereoscopic-base-defining system
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/25 using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 2213/005 Aspects relating to the "3D+depth" image format

Abstract

The present invention relates to a stereo camera which includes: a main camera which photographs a first image; a sub-camera which has camera parameters different from those of the main camera and photographs a second image; a distance information acquiring part which associates each pixel in the first and second images and acquires distance information including parallax information; and a right/left image generating part which generates a stereo view image based on one of the first and second images and the distance information.

Claims

1. A stereo camera, comprising:

a first imaging part which photographs a first image;

a second imaging part which has camera parameters different from those of said first imaging part and photographs a second image;

a distance information acquiring part which associates each pixel in said first and second images and acquires distance information including parallax information; and

a right/left image generating part which generates a stereo view image based on one of said first and second images and said distance information.

2. The stereo camera according to claim 1, wherein said distance information acquiring part decides a correspondence detection area where correspondence detection is performed, based on photographing information at the time of photographing said first and second images.

3. The stereo camera according to claim 2, wherein said distance information acquiring part regards one of said first and second images as a template and performs template matching with the other, to decide an area with the highest degree of agreement as said correspondence detection area.

4. The stereo camera according to claim 3, wherein, when said first and second image have different photographing magnifications, said distance information acquiring part performs transformation of resolution processing on an image to become said template, so as to create said template.

5. The stereo camera according to claim 2, wherein said distance information acquiring part extracts object candidate areas that become object candidates by object recognition with respect to said first and second images, and performs comparison among the obtained object candidate areas, to decide an area with the highest degree of agreement as said correspondence detection area.

6. The stereo camera according to claim 2, wherein said distance information acquiring part sets an optical axial center of the other image on one image with respect to said first and second images, aligns the optical axial center of said other image set on said one image with the optical axial center of said other image, converts said one image to an image size fitted with an imaging device having photographed the other image and superimposes said images, to decide said correspondence detection area.

7. The stereo camera according to claim 2, wherein said distance information acquiring part decides, with respect to said first and second images, said correspondence detection area at a ratio of vertical angles of view centered on the optical axial centers.

8. The stereo camera according to claim 2, wherein said distance information acquiring part decides one-dimensional area along epipolar lines as said correspondence detection area with respect to said first and second images.

9. The stereo camera according to claim 2, wherein said distance information acquiring part sets, with respect to respective said correspondence detection areas of said first and second images, a base position inside the area to be on the base side of correspondence detection to a position of a sub-pixel having a finer size than a pixel size, to perform correspondence detection.

10. The stereo camera according to claim 2, wherein said distance information acquiring part regards an image having said correspondence detection area with a larger number of pixels as a base image, out of respective said correspondence detection areas of said first and second images, to perform correspondence detection.

11. The stereo camera according to claim 2, wherein, when zoom photographing is performed at the time of photographing said first and second images, said distance information acquiring part changes a sampling interval of corresponding points in accordance with a zooming degree.

12. The stereo camera according to claim 11, wherein said distance information acquiring part decreases a sampling interval of said corresponding points with increase in zoom magnification.

13. The stereo camera according to claim 2, wherein said distance information acquiring part changes a sampling interval of corresponding points in accordance with the number of pixels of one with a smaller number of pixels out of said respective correspondence detection areas of said first and second images.

14. The stereo camera according to claim 2, wherein when a window for use in correspondence detection is applied to an object surface, said distance information acquiring part sets an aspect ratio of the window such that the aspect ratio on the object surface is isotropic, to perform correspondence detection.

15. The stereo camera according to claim 2, wherein said first imaging part has a higher resolution than said second imaging part, and said right/left image generating part regards said first image as a main image, displaces an image in said correspondence detection area inside said second image in accordance with said parallax information, and regards said first image and said new second image as said right/left images.

16. The stereo camera according to claim 15, wherein, when the number of pixels inside said correspondence detection area of said second image is smaller than the number of pixels inside said correspondence detection area of said first image, said right/left image generating part compensates pixel information from said first image.

17. The stereo camera according to claim 2, wherein said first imaging part has a higher resolution than said second imaging part, and when said second image has a higher photographing magnification than said first image, said right/left image generating part embeds said second image into said first image to generate a new second image, and regards said first image and said new second image as said right/left images.

18. The stereo camera according to claim 2, wherein said first imaging part has a higher resolution than said second imaging part, and when said first image has a higher photographing magnification than said second image, said right/left image generating part embeds said first image into said second image to generate a new first image, and regards said new first image and said second image as said right/left images.

19. The stereo camera according to claim 2, wherein said first imaging part has a zoom function, and when said first image is a zoomed image, said right/left image generating part generates an image with a changed base-line length in accordance with a zoom magnification, to generate a new first image.

20. The stereo camera according to claim 19, wherein

when generating said new first image, said right/left image generating part changes an image size so as to hold a photographic subject within an image even with a change in base-line length.

21. The stereo camera according to claim 1, wherein, a lens of said second imaging part is a fovea lens.

22. The stereo camera according to claim 1, wherein a lens of said second imaging part is an anamorphic lens.

23. The stereo camera according to claim 1, wherein said first imaging part has a higher resolution than said second imaging part, said stereo camera further comprises a sensor which senses arrangement of said stereo camera with said first and second imaging parts arrayed in a horizontally placed manner in parallel with a horizontal plane, and stops an operation of said distance information acquiring part in the case of sensing horizontal placement, and said right/left image generating part provides information of said first image to said second image to create a new second image, and regards said first image and said new second image as said right/left images.