Mosaic oblique images and methods of making and using same

• US 10,489,953 B2
• Filed: 04/27/2018
• Issued: 11/26/2019
• Est. Priority Date: 08/30/2006
• Status: Active Grant

• Current Assignee: Pictometry International Corporation (Eagleview Technologies, Inc. /Wa/)
• Original Assignee: Pictometry International Corporation (Eagleview Technologies, Inc. /Wa/)
• Inventors: Schultz, Stephen, Giuffrida, Frank, Gray, Robert
• Primary Examiner(s): Rush, Eric
• Application Number:
  US15/965,222
• Publication Number:
  US 20180247442A1
• Time in Patent Office: 578 Days
• Field of Search: 382100, 382154, 382284, 382285, 382294, 345629, 345646, 348144, 348147
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/30176 Document
• CPC Class Codes: G06T 3/06 Topological mapping of higher dimensional structures onto lower dimensional surfaces
• CPC Class Codes: G06T 3/18 Image warping, e.g. rearranging pixels individually
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85

Abstract

A computer system running image processing software receives an identification of a desired scene of a geographical area for which an oblique-mosaic image is desired including one or more geometry parameters of a virtual camera; creates a mathematical model of the virtual camera having mathematical values that define the camera geometry parameters that configure the model to capture the geographical area, and looking down at an oblique angle; creates a ground elevation model of the ground and vertical structures within the oblique-mosaic pixel map, wherein source images were captured at an oblique angle and compass direction similar to the oblique angle and compass direction of the virtual camera; and reprojects, with the mathematical model, source oblique image pixels of the overlapping source images for pixels included in the oblique-mosaic pixel map using the ground elevation model to thereby create the oblique-mosaic image of the geographical area.

Claims

1. A method for creating an aerial oblique-mosaic image from a plurality of aerial source oblique images, comprising the steps of:

identifying a desired geographical area to be imaged and collected into the aerial oblique-mosaic image;

creating a mathematical model of a sensor of a virtual camera, where the virtual camera has an elevation greater than an elevation of the area to be imaged, the mathematical model having an oblique-mosaic pixel map for the sensor of the desired area;

determining surface locations for pixels included in the oblique-mosaic pixel map, wherein the surface locations for the pixels include first surface locations of the ground and second surface locations of objects above the ground;

for pixels included in the oblique-mosaic pixel map, selecting aerial source oblique images of the surface locations of the pixels captured at an oblique angle and compass direction similar to an oblique angle and compass direction of the virtual camera; and

reprojecting source oblique image pixels of the area to be imaged for pixels included in the oblique-mosaic pixel map to thereby create the aerial oblique-mosaic image of the desired geographical area, the aerial oblique-mosaic image having a natural appearance.

2. The method of claim 1, wherein the step of determining is further defined by projecting the pixels in the oblique-mosaic pixel map through a perspective of the virtual camera to determine a corresponding surface location for the pixels in the oblique-mosaic pixel map.

3. The method of claim 1, wherein the step of selecting is selecting from multiple aerial source oblique images representing a same surface location.

4. The method of claim 3, wherein the pixels of each aerial source oblique image that represent the same surface location are compared to determine which source pixel is most representative of the same surface location, the more representative pixel to be included in the aerial oblique-mosaic image.

5. The method of claim 1, wherein the step of reprojecting is further defined by reprojecting the source oblique image pixels to match a size and shape of a represented surface location as taken from the elevation, compass direction, and oblique angle of the virtual camera.

6. The method of claim 1, wherein the step of reprojecting is further defined by removing effects of elevation from the aerial source oblique images prior to reprojection and then adding the effects of elevation to the aerial oblique-mosaic image after reprojection.

7. The method of claim 1, further comprising the step of minimizing lean of vertical structures within the aerial oblique-mosaic image.

8. The method of claim 7, wherein the step of minimizing lean of vertical structures within the aerial oblique-mosaic image is defined further as the steps of creating an elevation model from the aerial source oblique images taking into account the vertical structures, and reprojecting at least one aerial source oblique image pixel of the area to be imaged.

9. The method of claim 7, wherein the step of minimizing lean of vertical structures is defined further as matching vertical structures in multiple ones of the aerial source oblique images and shifting pixels apparent location in at least one of the aerial source oblique images by a relative height above a ground model.

10. The method of claim 1, wherein metadata is stored with the aerial oblique-mosaic image.

11. The method of claim 1, wherein the aerial oblique-mosaic image is geo-referenced.

12. The method of claim 1, wherein the step of selecting is selecting from multiple ones of the aerial source oblique image representing a same surface location and having meta-data indicative of a location of a camera used to capture the aerial source oblique images and orientation of the camera.

13. The method of claim 1, wherein the step of selecting is selecting from multiple ones of the aerial source oblique image representing a same surface location and having meta-data indicative of a location of a camera used to capture the aerial source oblique images, orientation of the camera, size of a sensor of the camera, and focal length of a lens of the camera.

14. A system, comprising:

a computer system running image processing software that when executed by the computer system causes the computer system to;

identify a desired geographical area to be imaged and collected into an aerial oblique-mosaic image;

create a mathematical model of a sensor of a virtual camera, where the virtual camera has an elevation greater than an elevation of the area to be imaged, the mathematical model having an oblique-mosaic pixel map for the sensor of the desired area;

determine surface locations for pixels included in the oblique-mosaic pixel map, wherein the surface locations for the pixels include first surface locations of the ground and second surface locations of objects above the ground;

for pixels included in the oblique-mosaic pixel map, select aerial source oblique images of the surface locations of the pixels captured at an oblique angle and compass direction similar to an oblique angle and compass direction of the virtual camera; and

reproject source oblique image pixels of the area to be imaged for pixels included in the oblique-mosaic pixel map to thereby create the aerial oblique-mosaic image of the desired geographical area, the aerial oblique-mosaic image having a natural appearance.

15. The system of claim 14, wherein the step of determining is further defined by projecting the pixels in the oblique-mosaic pixel map through a perspective of the virtual camera to determine a corresponding surface location for the pixels in the oblique-mosaic pixel map.

16. The system of claim 14, wherein the step of selecting is selecting from multiple ones of the aerial source oblique images representing a same surface location.

17. The system of claim 16, wherein the pixels of each aerial source oblique image that represent the same surface location are compared to determine which source pixel is most representative of the same surface location, the more representative pixel to be included in the aerial oblique-mosaic image.

18. The system of claim 14, wherein the step of reprojecting is further defined by reprojecting the source oblique image pixels to match a size and shape of a represented surface location as taken from the elevation, compass direction, and oblique angle of the virtual camera.

19. The system of claim 14, wherein the step of reprojecting is further defined by removing effects of elevation from the aerial source oblique images prior to reprojection and then adding the effects of elevation to the aerial oblique-mosaic image after reprojection.

20. The system of claim 14, further comprising the step of minimizing lean of vertical structures within the aerial oblique-mosaic image.