System and method for camera calibration and images stitching

• US 20060268131A1
• Filed: 05/09/2006
• Published: 11/30/2006
• Est. Priority Date: 06/21/2002
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Cutler, Ross
• Granted Patent: US 7,936,374 B2
• Time in Patent Office: Days
• US Class Current: 348/239
• CPC Class Codes: G06T 2200/32 involving image mosaicing
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: H04N 1/3876 Recombination of partial images to recreate the original image
• CPC Class Codes: H04N 1/6027 Correction or control of colour gradation or colour contrast H04N1/6058 takes precedence
• CPC Class Codes: H04N 17/002 for television cameras
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 9/73 Colour balance circuits, e.g. white balance circuits or colour temperature control camera processing pipelines for colour balance H04N23/88

Abstract

A practical, real-time calibration of digital omnidirectional cameras in the areas of de-vignetting, brightness, contrast, and white balance control. Novel solutions for the color calibration of an omnidirectional camera rig, and an efficient method for devignetting images are presented. Additionally, a context-specific method of stitching images together into a panorama or a mosaic is provided.

Claims

1. A process for color calibrating a series of cameras capturing different portions of the same scene, comprising the following process actions:

selecting a reference camera from a series of cameras configured to capture different portions of a scene;

calibrating said reference camera to optimum color settings for capturing portions of said scene; and

propagating the optimum settings from the reference camera to all other cameras in the series for regions of camera overlap.

2. The process of claim 1 further comprising the process actions of:

continually recalibrating the reference camera; and

propagating any changes in the reference camera settings to the other cameras in the series.

3. The process of claim 1 wherein a different reference camera from the initial reference camera is calibrated.

4. The process of claim 1 wherein the process action of calibrating the reference camera to optimum color settings comprises setting the reference camera for optimum brightness.

5. The process of claim 1 wherein the process action of calibrating the reference camera to optimum color settings comprises setting the reference camera for optimum contrast.

6. The process of claim 1 wherein the process action of calibrating the reference camera to optimum color settings comprises setting the reference camera for optimum white balance.

7. The process of claim 4 wherein setting the reference camera for optimum brightness comprises the process action of:

adjusting the mean intensity of the scene until it reaches a value corresponding to a black reference value.

8. The process of claim 7 wherein the black reference value is determined by a black reference pixel region in the camera, and wherein the brightness is adjusted so that the mean black reference pixel region equals the black reference value.

10. The process of claim 4 wherein setting the reference camera for optimum brightness comprises the process actions of:

setting an exposure setting on the reference camera to zero; and

adjusting the mean intensity of the scene until it reaches a prescribed value corresponding to black.

11. The process of claim 5 wherein setting the reference camera for optimum contrast comprises the process actions of:

adjusting the contrast so as to make the mean contrast equal to a prescribed value.

12. The process of claim 5 wherein setting the reference camera for optimum contrast comprises the process action of:

adjusting the contrast so as to make the resulting image as bright as possible without saturation.

13. The process of claim 6 wherein setting the reference camera for optimum white balance comprises making red components, blue components and green components in the scene equal.

14. The process of claim 13 wherein setting the reference camera for optimum white balance comprises the process actions of:

inputting a white scene;

setting the amount of red in the scene equal to the amount of green in the scene by adjusting red channel gain; and

setting the amount of blue in the scene equal to the amount of green in the scene by adjusting blue channel gain.

15. The process of claim 6 wherein setting the reference camera for optimum white balance comprises:

adjusting the red channel gain of the camera so that the mean of the red component is equal to the mean of the green component; and

adjusting the blue channel gain of the camera so that the mean of the blue component is equal to the mean of the green component.

16. The process of claim 15 wherein adjusting the red and blue channel gains comprises the process actions of:

if the mean of the blue in the scene is less than the mean of the green in the scene, incrementing the blue gain;

if the mean of the blue in the scene is greater than the mean of the green in the scene, decrementing the blue gain;

repeating incrementing or decrementing the blue gain until the absolute value of the difference between the mean of the blue and the mean of the green is less than a prescribed threshold;

if the mean of the red in the scene is less than the mean of the green in the scene, incrementing the red gain;

if the mean of the red in the scene is greater than the mean of the green in the scene, decrementing the red gain; and

repeating incrementing or decrementing the red gain until the absolute value of the difference between the mean of the red and the mean of the green is less than a prescribed threshold.

9. (canceled)

17-27. -27. (canceled)