Image combining method and apparatus

• US 5,646,679 A
• Filed: 06/28/1995
• Issued: 07/08/1997
• Est. Priority Date: 06/30/1994
• Status: Expired due to Term

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Iijima, Katsumi, Yano, Kotaro
• Primary Examiner(s): Chin, Tommy P.
• Assistant Examiner(s): DIEP, NHON THANH
• Application Number:
  US08/500,341
• Time in Patent Office: 741 Days
• Field of Search: 348/45, 348/47, 348/42, 348/36, 348/218, 348/586
• US Class Current: 348/47
• CPC Class Codes: G03B 2213/025 Sightline detection
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 3/4007 based on interpolation, e.g. bilinear interpolation image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0088 Synthesising a monoscopic image signal from stereoscopic images, e.g. synthesising a panoramic or high resolution monoscopic image
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

In image combining method of phototaking an object to be photographed by using a plurality of image taking systems with portions of fields of view thereof overlapping, and forming combined image information by combining pieces of image information obtained by the image taking systems, coordinates changing processing is performed by using the distance information of an object to be photographed, and image signals output from the respective image taking systems are compensated in accordance with the changed coordinates. One of the compensated image signals is selected with respect to a region where a far-distance portion of the object is concealed by a near-distance object, of the overlapping region of a combined image signal. With this operation, a plurality of image signals output from the respective image taking systems are combined into one image signal output in a state defined by an arbitrary object distance and the imaging magnification from an image taking system in which the viewpoint position and the optical axis direction are defined by the shift amount of the viewpoint position from each image taking system and the convergence angle of the optical axis.

Claims

1. An image combining method for image-taking an object by using a plurality of image taking systems, and forming combined image information by combining pieces of image information obtained by said image taking systems, comprising the steps of:

extracting corresponding point information between the pieces of image information;

calculating distance information by using optical information in each of said image taking systems and the corresponding point information;

sequentially generating two-dimensional coordinates of a display image based on the combined image information;

changing the two-dimensional coordinates to two-dimensional coordinates of each piece of image information by using the optical information and the distance information;

performing compensation processing for each piece of image information on the basis of the changed two-dimensional coordinates for each piece of image information;

determining an occlusion portion in which a far-distance portion of the object is concealed by a near-distance portion, of an overlapping region of images on the basis of the corresponding point information; and

selecting one of pieces of compensated image information with respect to the occlusion portion in forming combined image information by combining the pieces of compensated image information.

2. A method according to claim 1, wherein the step of selecting compensated image information with respect to the occlusion portion includes the step of selecting compensated image information exhibiting the least occlusion portion, of the pieces of compensated image information.

3. An image combining apparatus for image-taking an object by using a plurality of image taking systems, and forming combined image information by combining pieces of image information obtained by said image taking systems, comprising:

corresponding points extracting means for extracting corresponding point information between the pieces of image information;

distance information calculating means for calculating distance information by using optical information in each of said image taking systems and the corresponding point information;

address generating means for sequentially generating two-dimensional coordinates of a display image based on the combined image information;

coordinates changing means for changing the two-dimensional coordinates to two-dimensional coordinates of each piece of image information by using the optical information and the distance information;

image information compensating means for performing compensation processing for each piece of image information on the basis of the changed two-dimensional coordinates for each piece of image information;

determining means for determining an occlusion portion in which a far-distance portion of the object is concealed by a near-distance portion, of an overlapping region of phototaken images on the basis of the corresponding point information; and

selecting means for selecting one of pieces of compensated image information with respect to the occlusion portion in forming combined image information by combining the pieces of compensated image information.

4. An apparatus according to claim 3, wherein in selecting compensated image information with respect to the occlusion portion, compensated image information exhibiting the least occlusion portion is selected from the pieces of compensated image information.

5. An apparatus according to claim 3, further comprising image correlating means for dividing each compensated image information into small regions, and determining an occlusion portion in accordance with a similarity between corresponding small regions of the respective pieces of compensated image information.

6. An apparatus according to claim 5, wherein in selecting compensated image information with respect to the occlusion portion, compensated image information exhibiting the least occlusion portion is selected from the pieces of compensated image information.