Method for Improving Image Stitching Accuracy with Lens Distortion Correction and Device for Implementing the Same

• US 20070236595A1
• Filed: 04/09/2007
• Published: 10/11/2007
• Est. Priority Date: 04/10/2006
• Status: Active Grant

• Current Assignee: Sony Taiwan Ltd. (Sony Group Corp.)
• Original Assignee: Sony Taiwan Ltd. (Sony Group Corp.)
• Inventors: Mitsushita, Tatsumi, Pan, Patrick, Kuo, Benjamin, Lin, Christine
• Granted Patent: US 8,049,786 B2
• Time in Patent Office: Days
• US Class Current: 348/335
• CPC Class Codes: G06T 3/047 Fisheye or wide-angle transformations
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/81 for suppressing or minimising disturbance in the image signal generation
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

The invention provides a method for improving image stitching accuracy and a device for implementing the same, which apply to the multi-camera system for wide-angle image generation. Lens distortion causes mismatches of the features in the overlapping region of the images captured by the multi-camera system. As a result, the mismatches on the stitched wide-angle image are visible. The method and device for improving image stitching accuracy correct the lens distortion before stitching the images captured by the multi-camera system, so that the features in the overlapping region are matched and a seamless wide-angle image is generated by the stitching engine.

Claims

1. A device for improving image stitching accuracy, comprising:

a camera array consisting of a plurality of cameras for capturing images;

means for correcting lens distortion; and

an image stitching engine for transforming the captured images and stitching the transformed images into a seamless wide-angle image.

2. The device according to claim 1, further comprising a plurality of optional aperture controllers, wherein the aperture controllers are connected to the cameras respectively, and the image stitching engine sends an exposure control signal to the aperture controllers based on brightness of the captured images to control apertures of the cameras.

3. The device according to claim 2, wherein the means for correcting lens distortion comprises coordinate mapping and interpolation, the coordinate mapping generates corrected coordinates for the captured images based on lens parameters, and the lens parameters vary with the apertures of the cameras.

4. The device according to claim 3, wherein the means for correcting lens distortion utilizes polynomial algorithm, and the lens parameters are polynomial coefficients or a mapping table of the corrected coordinates.

5. The device according to claim 1, wherein the image stitching engine comprises means for transforming the captured images and means for blending and stitching the transformed images, and wherein the means for transforming the captured images comprises coordinate mapping and interpolation and cropping, the means for transforming the captured images generates transformed coordinates based on transformation parameters.

6. The device according to claim 5, wherein the means for correcting lens distortion is combined with the means for transforming the captured images.

7. The device according to claim 6, wherein the interpolation is performed only once.

8. A method for improving image stitching accuracy, comprising:

capturing images by a camera array;

correcting lens distortion; and

transforming the captured images and stitching the transformed images into a seamless wide-angle image.

9. The method according to claim 8, wherein the camera array comprises a plurality of cameras to which a plurality of aperture controllers are connected respectively, and the method further comprises sending an exposure control signal to the aperture controllers based on brightness of the captured images to control apertures of the cameras.

10. The method according to claim 9, wherein correcting the lens distortion comprises coordinate mapping and interpolation, the coordinate mapping generates corrected coordinates for the captured images based on lens parameters, and the lens parameters vary with the apertures of the cameras.

11. The method according to claim 10, wherein correcting the lens distortion utilizes polynomial algorithm, and the lens parameters are polynomial coefficients or a mapping table of the corrected coordinates.

12. The method according to claim 8, wherein transforming the captured images comprises coordinate mapping and interpolation and cropping, and transforming the captured images generates transformed coordinates based on transformation parameters.

13. The method according to claim 12, wherein correcting the lens distortion is combined with transforming the captured images.

14. The method according to claim 13, wherein the interpolation is performed only once.