METHOD, DEVICE, AND SYSTEM FOR COMPUTING A SPHERICAL PROJECTION IMAGE BASED ON TWO-DIMENSIONAL IMAGES

• US 20140340427A1
• Filed: 01/17/2013
• Published: 11/20/2014
• Est. Priority Date: 01/18/2012
• Status: Abandoned Application

• Current Assignee: Logos Technologies, LLP
• Original Assignee: Logos Technologies, LLP
• Inventors: Baker, Patrick Terry
• Application Number:
  US14/366,405
• Publication Number:
  US 20140340427A1
• Time in Patent Office: Days
• US Class Current: 345/641
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2215/08 Gnomonic or central projection
• CPC Class Codes: G06T 3/12 Panospheric to cylindrical image transformations
• CPC Class Codes: G06T 3/14 Transformations for image registration, e.g. adjusting or mapping for alignment of images
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: H04N 13/207 using a single 2D image sensor
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/363 using image projection screens volumetric display H04N13/388
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 9/3185 Geometric adjustment, e.g. keystone or convergence optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14

Abstract

An image projection method for generating a panoramic image, the method including the steps of accessing images that were captured by a camera located at a source location, and each of the images being captured from a different angle of view, the source location being variable as a function of time, calibrating the images collectively to create a camera model that encodes orientation, optical distortion, and variable defects of the camera; matching overlapping areas of the images to generate calibrated image data, accessing a three-dimensional map, first projecting pixel coordinates of the calibrated image data into a three-dimensional space using the three-dimensional map to generate three-dimensional pixel data, and second projecting the three-dimensional pixel data to an azimuth-elevation coordinate system that is referenced from a fixed virtual to generate the panoramic image.

Claims

1. An image projection method for generating a panoramic image, the method performed on a computer having a first and a second memory, comprising:

accessing a plurality of images from the first memory, each of the plurality of images being captured by a camera located at a source location, and each of the plurality of images being captured from a different angle of view, the source location being variable as a function of time;

calibrating the plurality of images collectively to create a camera model that encodes orientation, optical distortion, and variable defects of the camera;

matching overlapping areas of the plurality of images to generate calibrated image data having improved knowledge on the orientation and source location of the camera;

accessing a three-dimensional map from the second memory;

first projecting pixel coordinates of the calibrated image data into a three-dimensional space using the three-dimensional map to generate three-dimensional pixel data; and

second projecting the three-dimensional pixel data to an azimuth-elevation coordinate system that is referenced from a fixed virtual viewpoint to generate transformed image data and using the transformed image data to generate the panoramic image.

2. The image projection method of claim 1, further comprising:

estimating the fixed virtual viewpoint to be in proximity of the source location; and

periodically changing a position of the fixed virtual viewpoint.

3. The image projection method of claim 1, further comprising:

generating a displayable image by warping the transformed image data based on the azimuth-elevation coordinate system.

4. A non-transitory computer readable medium having computer instructions recorded thereon, the computer instructions configured to perform an image processing method when executed on a computer having a first and a second memory, the method comprising the steps of:

accessing a plurality of images from the first memory, each of the plurality of images being captured by a camera located at a source location, and each of the plurality of images being captured from a different angle of view, the source location being variable as a function of time;

calibrating the plurality of images collectively to create a camera model that encodes orientation, optical distortion, and variable defects of the camera;

matching overlapping areas of the plurality of images to generate calibrated image data having improved knowledge on the orientation and source location of the camera;

accessing a three-dimensional map from the second memory;

first projecting pixel coordinates of the calibrated image data into a three-dimensional space using the three-dimensional map to generate three-dimensional pixel data; and

second projecting the three-dimensional pixel data to an azimuth-elevation coordinate system that is referenced from a fixed virtual viewpoint to generate transformed image data and using the transformed image data to generate the panoramic image.

5. The non-transitory computer-readable medium according to claim 4, said method further comprising:

estimating the fixed virtual viewpoint to be in proximity of the source location; and

periodically changing a position of the fixed virtual viewpoint.

6. A computer system for generating panoramic images, comprising:

a first memory having a plurality of two-dimensional images stored thereon, each of the plurality of images captured from a scenery by a camera located a source location, and each of the plurality of images being captured from a different angle of view, the source location being variable as a function of time;

a second memory having a three-dimensional map from the scenery; and

a hardware processor configured to

calibrate the plurality of images collectively to create a camera model that encodes orientation, optical distortion, and variable defects of the camera;

match overlapping areas of the plurality of images to generate calibrated image data having improved knowledge on the orientation and source location of the camera;

first project pixel coordinates of the calibrated image data into a three-dimensional space using the three-dimensional map to generate three-dimensional pixel data; and

second project the three-dimensional pixel data to an azimuth-elevation coordinate system that is referenced from a fixed virtual viewpoint to generate transformed image data and using the transformed image data to generate the panoramic image.

7. The system according to claim 6, said hardware processor further configured to

estimate the fixed virtual viewpoint to be in proximity of the source location, and

periodically change a position of the fixed virtual viewpoint.