Method and system for compensating for parallax in multiple camera systems

• US 7,277,118 B2
• Filed: 11/20/2001
• Issued: 10/02/2007
• Est. Priority Date: 08/09/1999
• Status: Expired due to Term

• Current Assignee: Fuji Xerox Company Limited (Xerox Holdings Corp.)
• Original Assignee: Fuji Xerox Company Limited (Xerox Holdings Corp.)
• Inventors: Foote, Jonathan T.
• Primary Examiner(s): Ometz; David
• Assistant Examiner(s): NGUYEN, LUONG TRUNG
• Application Number:
  US09/989,638
• Publication Number:
  US 20020122113A1
• Time in Patent Office: 2,142 Days
• Field of Search: 348/218.1, 348/36, 348/47, 348/48
• US Class Current: 348/36
• CPC Class Codes: G03B 37/04 with cameras or projectors providing touching or overlapping fields of view
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0088 Synthesising a monoscopic image signal from stereoscopic images, e.g. synthesising a panoramic or high resolution monoscopic image
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

A camera array captures plural component images which are combined into a single scene. In one embodiment, each camera of the array is a fixed digital camera. The images from each camera are warped to a common coordinate system and the disparity between overlapping images is reduced using disparity estimation techniques.

Claims

1. A method for combining at least a portion of a plurality of images, comprising the steps of:

obtaining a first image and a second image, wherein at least a portion of said first image and said second image include a common field of view;

adjusting at least a portion of said first image to reduce an image disparity between said common field of view of said first image and said second image, wherein said adjusting step further comprises;

determining a plurality of disparity differences between at least a portion of said first image and at least a portion of said second image, each of said plurality of disparity differences corresponding to a distance of adjustment of said first image with respect to said second image;

selecting a distance corresponding to a lowest disparity of difference as a first distance and

shifting said first image said first distance with respect to said second image; and

combining at least a portion of said first image and at least a portion of said second image subsequent to said step of adjusting.

2. The method of claim 1, further including the step of:

adjusting said second image to reduce said image disparity between said common field of view of said first image and said second image.

3. The method of claim 2, wherein said step of adjusting said second image includes shifting said second image a second distance.

4. The method of claim 3 wherein said second distance is a distance where the image disparity is reduced.

5. The method of claim 1 further including the step of:

warping said first image and said second image into a common coordinate system of a composite image subsequent to said step of obtaining.

6. The method of claim 1 further including the step of:

cross-fading said common field of view of said first image and said second image, subsequent to said step of adjusting.

7. An apparatus for producing a panoramic video, comprising:

a camera array including a plurality of cameras;

an image obtaining device, wherein said image obtaining device obtains a first image from a first camera in said camera array and a second image from a second camera in said camera array, wherein said first image and said second image include a common field of view;

an image adjustor, wherein said image adjustor adjusts at least a portion of said first image to reduce an image disparity between said common field of view of said first image and said second image and wherein the image adjustor is adapted to;

determine a plurality of disparity differences between at least a portion of the first image and the second image, each disparity difference corresponding to a distance of adjustment of the first image with respect to the second image;

select a distance corresponding to a lowest disparity difference in said plurality of disparity differences as a first distance; and

shift the first image said first distance with respect to said second image; and

,

an image combiner, wherein said image combiner combines at least a portion of said first image and at least a portion of said second image.