Temperature compensation in multi-camera photographic devices

• US 7,602,412 B2
• Filed: 12/30/2004
• Issued: 10/13/2009
• Est. Priority Date: 06/21/2002
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Cutler, Ross G.
• Primary Examiner(s): Diep; Nhon T
• Application Number:
  US11/027,187
• Publication Number:
  US 20050117034A1
• Time in Patent Office: 1,748 Days
• Field of Search: 348/36
• US Class Current: 348/36
• CPC Class Codes: G06T 2200/32 involving image mosaicing
• CPC Class Codes: G06T 5/92 based on global image properties
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: H04N 1/3876 Recombination of partial images to recreate the original image
• CPC Class Codes: H04N 1/6027 Correction or control of colour gradation or colour contrast H04N1/6058 takes precedence
• CPC Class Codes: H04N 17/002 for television cameras
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 9/73 Colour balance circuits, e.g. white balance circuits or colour temperature control camera processing pipelines for colour balance H04N23/88

Abstract

A temperature measuring device measures ambient temperature in which a multi-camera photographic device is operated. A remapping function that maps individual images into a panoramic space is varied according to the measured temperature. The cameras and remapping tables are calibrated at various temperatures. When the ambient temperature is measured, values from a nearest matching remapping table are used, or new values are interpolated/extrapolated from one or more nearest matching tables. The remapping function is thus calibrated to properly remap the individual images into the panoramic image.

Claims

1. A method, comprising:

determining an ambient temperature corresponding to a camera environment;

determining one or more remapping tables corresponding to the ambient temperature; and

using remapping table values contained in the one or more remapping tables in a remapping function that remaps multiple images to form a panoramic image,

wherein each of the multiple images is taken from a different one of multiple distinct cameras positioned to have a small non-zero distance between their virtual centers of projection so as to reduce parallax error at a predefined distance from the multiple distinct cameras.

2. The method as recited in claim 1, wherein the step of using remapping table values further comprises using exact values contained in a single remapping table for the remapping function if the single remapping table corresponds exactly to the ambient temperature.

3. The method as recited in claim 1, wherein the step of using remapping table values further comprises extrapolating remapping values for use in the remapping function from a remapping table that most closely corresponds to the ambient temperature.

4. The method as recited in claim 1, wherein the step of using remapping table values further comprises deriving remapping values for use in the remapping function from at least two remapping tables that most closely correspond to the ambient temperature.

5. The method as recited in claim 4, wherein at least one remapping table corresponds to a temperature lower than the ambient temperature and at least one other remapping table corresponds to a temperature higher than the ambient temperature.

6. The method as recited in claim 1, wherein the camera environment is a panoramic camera that comprises two or more component cameras.

7. A system, comprising:

multiple cameras each having a lens with a focal length and aligned to have a small non-zero distance between virtual centers of projection relative to each other so as to reduce parallax error at a predefined distance from the cameras;

a temperature measuring device configured to measure an ambient temperature;

memory for storing an individual image from each of the multiple cameras;

at least two remapping tables, each remapping table corresponding to a calibration temperature; and

a remapping module configured to remap pixels from the individual images to a panoramic image according to one or more remapping tables according to the measured ambient temperature.

8. The system as recited in claim 7, wherein the remapping module is further configured to use a remapping table that corresponds to the measured ambient temperature.

9. The system as recited in claim 7, wherein the remapping module is further configured to remap pixels using values extrapolated from a remapping table that corresponds to a calibration temperature that is closer the measured ambient temperature than calibration temperatures corresponding to other remapping tables.

10. The system as recited in claim 7, wherein the remapping module is further configured to remap pixels using values interpolated from two or more remapping tables.

11. The system as recited in claim 7, wherein the remapping module is further configured to remap pixels using values interpolated from a first remapping table that corresponds to a calibration temperature that is nearest to but not greater than the measured ambient temperature and a second remapping table that corresponds to a calibration temperature that is nearest to but not less than the measured ambient temperature.

12. One or more computer storage media containing executable instructions that, when executed, perform the following steps:

receiving an ambient temperature;

receiving multiple individual images, each individual image being taken by an individual camera, the individual cameras positioned to have a small non-zero distance between their virtual centers of projection so as to reduce parallax error at a predefined distance from the cameras;

determining one or more appropriate remapping tables according to the ambient temperature;

remapping the individual images onto panoramic space according to the remapping table so that a panoramic image can be created from the panoramic space.

13. The one or more computer storage media as recited in claim 12, wherein the determining step further comprises determining a remapping table that corresponds directly to the ambient temperature.

14. The one or more computer storage media as recited in claim 12, wherein the determining step further comprises:

determining a remapping table that most nearly corresponds to the ambient temperature; and

extrapolating appropriate remapping values from the remapping table.

15. The one or more computer storage media as recited in claim 12, wherein the determining step further comprises:

identifying a first remapping table corresponding to a first calibration temperature that is nearest to but not greater than calibration temperatures corresponding to other remapping tables;

identifying a second remapping table corresponding to a second calibration temperature that is nearest to but not less than calibration temperatures corresponding to other remapping tables; and

deriving appropriate remapping values by interpolating between corresponding values from the first remapping table and the second remapping table.

16. The one or more computer storage media as recited in claim 12, wherein the at least one remapping table further comprises at least two remapping tables, each remapping table corresponding to a different calibration temperature.

17. The one or more computer storage media as recited in claim 12, further comprising the multiple individual images.