Self calibrating stereo camera

• US 8,797,387 B2
• Filed: 07/11/2011
• Issued: 08/05/2014
• Est. Priority Date: 04/27/2011
• Status: Active Grant

• Current Assignee: Aptina Imaging Corporation (ON Semiconductor Corporation)
• Original Assignee: Aptina Imaging Corporation (ON Semiconductor Corporation)
• Inventors: Kirsch, Graham, Huggett, Anthony R.
• Primary Examiner(s): Senfi, Behrooz
• Assistant Examiner(s): Jebari, Mohammed
• Application Number:
  US13/179,866
• Publication Number:
  US 20120274627A1
• Time in Patent Office: 1,121 Days
• Field of Search: 348/46, 348/47, 382/154, 700/259
• US Class Current: 348/46
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/85 Stereo camera calibration
• CPC Class Codes: H04N 13/246 Calibration of cameras

Abstract

A self calibrating stereo camera includes first and second spatial transform engines for directly receiving first and second images, respectively, of an object. The first and second spatial transform engines are coupled to a stereo display for displaying a fused object in stereo. A calibration module is coupled to the first and second spatial transform engines for aligning the first and second images, prior to display to a viewer. The first and second point extracting modules, respectively, receive the first and second images for extracting interest points from each image. A matching points module is coupled to the first and second point extracting modules for matching the interest points extracted by the first and second point extracting modules. The calibration module determines alignment error between the first and second images, in response to the interest point matches calculated by the matching points module.

Claims

1. A self calibrating stereo camera comprising:

first and second spatial transform engines for directly receiving first and second input images, respectively, of an object,

the first and second spatial transform engines coupled to a stereo display for outputting, respectively, first and second output images for display to a viewer,

a calibration module, coupled to the first and second spatial transform engines, for aligning the first and second output images prior to display,

first and second point extracting modules, respectively, receiving the first and second output images, for extracting interest points from each output image; and

a matching points module, coupled to the first and second point extracting modules, for matching the interest points extracted by the first and second point extracting modules,

wherein the calibration module determines alignment error between the first and second output images, in response to the interest point matches calculated by the matching points module and wherein the matching points module is configured to use Context-Adaptive Neighborhoods (CANs) of worms to match the interest points extracted from the first and second point extracting modules.

2. The self calibrating stereo camera of claim 1 wherein the calibration module determines a vertical error between at least one pair of matched points of interest in the first and second output images.

3. The self calibrating stereo camera of claim 2 wherein the calibration module calculates the vertical error using single value decomposition analysis.

4. The self calibrating stereo camera of claim 1 wherein

the calibration module is configured to correct settings in the first and second spatial transform engines, and

the first and second transform engines are configured to transform the first and second input images, respectively, based on the corrected settings.

5. The self calibrating stereo camera of claim 1 wherein

each of the first and second spatial transform engines are configured to spatially sample a respective input image and output a respective transformed image to the display, and

a transformed image from the first spatial transform engine is aligned to a transformed image from the second spatial transform engine.

6. A method of calibrating a stereo camera comprising the steps of:

calibrating vertical errors between pixels of a first image and corresponding pixels of a second image;

correcting the vertical errors between the pixels of the first and second images;

spatially transforming the first and second images, after correcting the vertical errors between the pixels of the first and second images, respectively;

outputting the spatially transformed first and second images to a display for viewing;

extracting interest points from the first and second images;

matching the extracted interest points between the first and second images;

outputting vertical misalignment distances between the extracted interest points, after matching the extracted interest points; and

correcting the vertical errors between the pixels of the first and second images in response to the outputted vertical misalignment distances, wherein matching the extracted interest points includes using belief propagation to match the interest points.

7. The method of claim 6 wherein calibrating vertical errors between pixels of a first image and corresponding pixels of a second image includes determining the vertical errors between pixels using single value decomposition analysis.

8. The method of claim 6 wherein spatially transforming the first and second images includes rotating, scaling, and/or shifting the first image with respect to the second image.

9. The method of claim 8 wherein rotating includes rotating the first image about three axes with respect to the second image.