System and method for camera calibration and images stitching

• US 7,936,374 B2
• Filed: 05/09/2006
• Issued: 05/03/2011
• Est. Priority Date: 06/21/2002
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Cutler, Ross
• Primary Examiner(s): Ye; Lin
• Assistant Examiner(s): HSU, AMY R
• Application Number:
  US11/430,344
• Publication Number:
  US 20060268131A1
• Time in Patent Office: 1,820 Days
• Field of Search: 348/239, 348/211.22, 348/211.8, 348/211.3
• US Class Current: 348/211.11
• CPC Class Codes: G06T 2200/32 involving image mosaicing
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 5/40 using histogram techniques
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: H04N 1/3876 Recombination of partial images to recreate the original image
• CPC Class Codes: H04N 1/6027 Correction or control of colour gradation or colour contrast H04N1/6058 takes precedence
• CPC Class Codes: H04N 17/002 for television cameras
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/88 for colour balance, e.g. white-balance circuits or colour temperature control
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 9/73 Colour balance circuits, e.g. white balance circuits or colour temperature control camera processing pipelines for colour balance H04N23/88

Abstract

A practical, real-time calibration of digital omnidirectional cameras in the areas of de-vignetting, brightness, contrast, and white balance control. Novel solutions for the color calibration of an omnidirectional camera rig, and an efficient method for devignetting images are presented. Additionally, a context-specific method of stitching images together into a panorama or a mosaic is provided.

Claims

1. A method for color calibrating a series of cameras capturing different portions of a scene, the method comprising:

selecting a first of a plurality of cameras as a reference camera, wherein each of the plurality of cameras is configured to capture different portions of the scene;

calibrating color component mean values of the reference camera by adjusting color settings of the reference camera, wherein the calibrated color component mean values of the reference camera are based on a first overlapping portion of the scene as captured by the reference camera, and wherein a second camera of the plurality of cameras is configured to capture the first overlapping portion of the scene and a second overlapping portion of the scene;

adjusting color settings of the second camera, in response to the calibrating, to cause color component mean values of the second camera to substantially match the calibrated color component mean values of the reference camera, wherein the adjusting the color settings of the second camera is based on the first overlapping portion of the scene as captured by the second camera.

2. The method of claim 1 further comprising:

continually automatically recalibrating the reference camera; and

propagating, in response to the continually automatically recalibrating, any changes in the reference camera by the adjusting the color settings of the second camera and the adjusting the color settings of a third camera.

3. The method of claim 1 wherein a different camera of the plurality of cameras other than the first camera is selected as the reference camera.

4. The method of claim 1 wherein the calibrating further comprises calibrating a mean intensity value by adjusting brightness settings of the reference camera, wherein the calibrated mean intensity value of the reference camera is based on the first overlapping portion of the scene as captured by the reference camera.

5. The method of claim 4 wherein the calibrating the mean intensity value of the reference camera comprises adjusting the brightness settings to cause the mean intensity value to substantially match a black reference value.

6. The method of claim 5 wherein the black reference value is determined by a black reference pixel region in the reference camera.

7. The method of claim 4 wherein the calibrating the mean intensity value of the reference camera comprises:

setting an exposure setting of the reference camera to zero; and

adjusting the brightness settings to cause the mean intensity value to substantially match a black reference value.

8. The method of claim 1 wherein the calibrating further comprises calibrating a mean brightness value by adjusting contrast settings of the reference camera, wherein the calibrated mean brightness value of the reference camera is based on the first overlapping portion of the scene as captured by the reference camera.

9. The method of claim 8 wherein the calibrating the mean brightness value of the reference camera comprises adjusting the contrast settings to cause a mean brightness value of the first overlapping portion of the scene to substantially match a prescribed value.

10. The method of claim 8 wherein the calibrating the mean brightness value of the reference camera comprises adjusting the contrast settings to cause a substantially maximum brightness value without saturation.

11. The method of claim 1 wherein the calibrating further comprises calibrating a red mean value and a blue mean value and a green mean value by adjusting a red gain setting of the reference camera and a blue gain setting of the reference camera, wherein the calibrated red and blue and green mean values of the reference camera are based on the first overlapping portion of the scene as captured by the reference camera.

12. The method of claim 11 wherein the calibrating the red mean value and the blue mean value and the green mean value of the reference camera comprises adjusting the red gain value and a blue gain value to cause the mean red value and the mean blue value and the mean green value to be substantially equal.

13. The method of claim 12 wherein the calibrating the red mean value and the blue mean value and the green mean value of the reference camera comprises:

capturing, by the reference camera, a white scene;

adjusting the red gain value and a blue gain value to cause the mean red value and the mean blue value and the mean green value to be substantially equal based on the white scene as capture by the reference camera.

14. The method of claim 1 further comprising adjusting color settings of a third camera of the plurality of cameras, in response to the adjusting the color settings of the second camera, to cause color component mean values of the third camera to substantially match color component mean values of the second camera that are based the adjusted color settings of the second camera and that result from the second overlapping portion of the scene as captured by the second camera, wherein the third camera is configured to capture at least the second overlapping portion of the scene, and wherein the adjusting the color settings of the third camera is based on the second overlapping portion of the scene as captured by the third camera.

15. At least one computer-readable device that comprises computer-executable instructions that, when executed by a computer, cause the computer to perform a method for color calibrating a series of cameras capturing different portions of a scene, the method comprising:

selecting a first of a plurality of cameras as a reference camera, wherein each of the plurality of cameras is configured to capture different portions of the scene;

calibrating color component mean values of the reference camera by adjusting color settings of the reference camera, wherein the calibrated color component mean values of the reference camera are based on a first overlapping portion of the scene as captured by the reference camera, and wherein a second camera of the plurality of cameras is configured to capture the first overlapping portion of the scene and a second overlapping portion of the scene; and

adjusting color settings of the second camera, in response to the calibrating, to cause color component mean values of the second camera to substantially match the calibrated color component mean values of the reference camera, wherein the adjusting the color settings of the second camera is based on the first overlapping portion of the scene as captured by the second camera.

16. The computer-readable device of claim 15, the method further comprising adjusting color settings of a third camera of the plurality of cameras, in response to the adjusting the color settings of the second camera, to cause color component mean values of the third camera to substantially match color component mean values of the second camera that are based the adjusted color settings of the second camera and that result from the second overlapping portion of the scene as captured by the second camera, wherein the third camera is configured to capture at least the second overlapping portion of the scene, and wherein the adjusting the color settings of the third camera is based on the second overlapping portion of the scene as captured by the third camera.

17. The computer-readable device of claim 15, the method further comprising:

continually automatically recalibrating the reference camera; and

propagating, in response to the continually automatically recalibrating, any changes in the reference camera by the adjusting the color settings of the second camera and the adjusting the color settings of a third camera.

18. The computer-readable device of claim 15 wherein a different camera of the plurality of cameras other than the first camera is selected as the reference camera.

19. The computer-readable device of claim 15 wherein the calibrating further comprises calibrating a mean intensity value by adjusting brightness settings of the reference camera, wherein the calibrated mean intensity value of the reference camera is based on the first overlapping portion of the scene as captured by the reference camera.

20. The computer-readable device of claim 15 wherein the calibrating further comprises calibrating a mean brightness value by adjusting contrast settings of the reference camera, wherein the calibrated mean brightness value of the reference camera is based on the first overlapping portion of the scene as captured by the reference camera.

21. The computer-readable device of claim 15 wherein the calibrating further comprises calibrating a red mean value and a blue mean value and a green mean value by adjusting a red gain setting of the reference camera and a blue gain setting of the reference camera, wherein the calibrated red and blue and green mean values of the reference camera are based on the first overlapping portion of the scene as captured by the reference camera.