Orthorectifying stitched oblique imagery to a nadir view, and applications thereof

• US 20110109719A1
• Filed: 11/09/2009
• Published: 05/12/2011
• Est. Priority Date: 11/09/2009
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Zelinka, Stephen D., Wilson, Randall, Kirmse, Andrew, Rhodes, Charles C.
• Granted Patent: US 8,159,524 B2
• Time in Patent Office: Days
• US Class Current: 348/42
• CPC Class Codes: G01C 11/02 Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
• CPC Class Codes: G06T 3/08 Projecting images onto non-planar surfaces, e.g. geodetic screens

Abstract

Embodiments of this invention relates to orthorectifying oblique imagery to a nadir view. In an embodiment, a method orthorectifies oblique photographic imagery. An oblique photographic image is projected onto a three-dimensional model of terrain. Points are sampled from the projected photographic image at the intersection of the three-dimensional model of terrain and parallel rays extended from a virtual viewport having a nadir perspective. The sampled points are assembled into an orthorectified image. Finally, the orthorectified image or the sampled points are adjusted approximately according to a difference in tilt angle between a camera that captured the oblique photographic image and the viewport having the nadir perspective to generate a foreshortened orthorectified image. Each location in the foreshortened orthorectified image corresponds linearly to a corresponding location in a two-dimensional map.

Claims

1. A method for orthorectifying oblique photographic imagery, comprising:

(a) projecting an oblique photographic image onto three-dimensional model of terrain;

(b) sampling points of the projected photographic image from a viewport having a nadir perspective, the points being at the intersection of the three-dimensional model of terrain and parallel rays extended from the viewport;

(c) assembling the sampled points into an orthorectified image; and

(d) adjusting the orthorectified image or the sampled points approximately according to a difference in tilt angle between a camera that captured the oblique photographic image and the viewport having the nadir perspective to generate a foreshortened orthorectified image,

wherein each location in the foreshortened orthorectified image corresponds linearly to a corresponding location in a two-dimensional map.

2. The method of claim 1, wherein the adjusting (d) comprises shrinking the orthorectified image in the vertical direction.

3. The method of claim 2, wherein the shrinking comprises shrinking the orthorectified image in the vertical direction by a factor between 1.154 to 2.0 inclusive.

4. The method of claim 1, wherein the adjusting (d) comprises adjusting the sampling points such that points are sampled more frequently in the horizontal direction than in the vertical direction.

5. The method of claim 4, wherein the distance between points sampled in the horizontal direction times a factor is approximately equal to the distance between points sampled in the vertical direction, wherein the factor is between 1.154 to 2.0 inclusive.

6. The method of claim 1, further comprising:

(e) prior to the projecting (a), smoothing the three-dimensional model of terrain,

wherein the projecting (a) comprises projecting the oblique photographic image onto the smoothed three-dimensional model of terrain.

7. The method of claim 6, wherein the smoothing (e) comprises applying a box filter algorithm to the three-dimensional model of terrain.

8. The method of claim 1, wherein the oblique photographic image is an aerial image of the Earth, and

wherein the three-dimensional model of terrain is a three-dimensional model of the Earth.

9. The method of claim 1, further comprising:

(e) repeating (a)-(c) for each of a plurality of oblique photographic images to determine a plurality of orthorectified images; and

(f) determining a mosaic of orthorectified images based on the plurality of orthorectified images using a graph cut algorithm.

10. The method of claim 1, further comprising:

(e) overlaying map data onto at least a portion of the foreshortened orthorectified image, wherein each element of the map data is positioned at a location on the foreshortened orthorectified image that corresponds linearly to a corresponding location of the element of the map data.

11. The method of claim 10, wherein the map data comprises road data.

12. A system for orthorectifying oblique photographic imagery, comprising:

a projector module configured to project an oblique photographic image onto three-dimensional model of terrain; and

a sampler module configured to;

sample points from the projected photographic image from a viewport having a nadir perspective, the points being located at the intersection of the three-dimensional model of terrain and parallel rays extended from the viewport, and

assemble the sampled points into an orthorectified image; and

a foreshortening module configured to adjust the orthorectified image or the sampled points approximately according to a difference in tilt angle between a camera that captured the oblique photographic image and the viewport having the nadir perspective to generate a foreshortened orthorectified image,

wherein each location in the foreshortened orthorectified image corresponds linearly to a corresponding location in a two-dimensional map.

13. The system of claim 12, wherein the foreshortening module is configured to shrink the orthorectified image in the vertical direction.

14. The system of claim 13, wherein the foreshortening module is configured to shrink the orthorectified image in the vertical direction by a factor between 1.154 to 2.0 inclusive.

15. The system of claim 12, wherein the foreshortening module is configured to adjust the sampling points such that points are sampled more frequently in the horizontal direction than in the vertical direction.

16. The system of claim 15, wherein the number of points in the vertical direction times a factor is approximately equal to the number of points sampled in the horizontal direction, wherein the factor is between 1.154 to 2.0 inclusive.

17. The system of claim 12, further comprising:

a smoothing module configured to smooth the three-dimensional model of terrain,

wherein the projector module is configured to project an oblique photographic image onto the smoothed three-dimensional model of terrain.

18. The system of claim 17, wherein the smoothing module is configured to apply a box filter algorithm to the three-dimensional model of terrain.

19. The system of claim 12, wherein the oblique photographic image is an aerial image of the Earth.

20. The system of claim 12, further comprising:

an orthorectification module configured to instruct the projector module and sampler module to repeatedly project and sample for a plurality of oblique photographic images to determine a plurality of foreshortened orthorectified images; and

a mosaic module configured to determine a mosaic of orthorectified images based on the plurality of foreshortened orthorectified images using a graph cut algorithm.

21. The system of claim 20, further comprising:

a tile locator module that identifies an orthorectified image tile from the mosaic of orthorectified images corresponding to a location; and

a map data locator module that identifies map data corresponding linearly to the location.

22. The system of claim 21, wherein the map data comprises road data.

23. A method for orthorectifying oblique photographic imagery, comprising:

(a) determining a point on a three-dimensional model of terrain for each point in a nadir viewport, each point being at the intersection of the three-dimensional model of terrain and parallel rays extended from the nadir viewport;

(b) projecting each point determined in (a) to a position of a camera model that took an oblique photographic image to sample a point on the oblique photographic image;

(c) assembling the sampled points into an orthorectified image; and

(d) adjusting the orthorectified image or the sampled points approximately according to a difference in tilt angle between a camera that captured the oblique photographic image and the viewport having the nadir perspective to generate a foreshortened orthorectified image,

wherein each location in the foreshortened orthorectified image corresponds linearly to a corresponding location in a two-dimensional map.