Interface for applying a photogrammetry algorithm to panoramic photographic images

• US 8,817,067 B1
• Filed: 07/29/2011
• Issued: 08/26/2014
• Est. Priority Date: 07/29/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Fan, Zhe, Shattuck, Scott, Limber, Mark, Brown, Brian Gammon, Simpson, Matthew Robert, Reinhardt, Tilman, Lowrie, Matt
• Primary Examiner(s): Kelley, Christopher S
• Assistant Examiner(s): Tarko, Asmamaw G
• Application Number:
  US13/193,992
• Time in Patent Office: 1,124 Days
• Field of Search: 348/64, 348/36
• US Class Current: 348/36
• CPC Class Codes: H04N 23/58 Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

Embodiments relate to selecting textures for a panoramic photographic image in image-based three-dimensional modeling. In a first embodiment, a computer-implemented method includes a method for inputting a panoramic photographic image that uses a photogrammetry algorithm to adjust a plurality of camera parameters for the panoramic photographic image. In the method, a street-level panoramic photographic image is selected by a user from a street-level panoramic image display site. An area of the street-level panoramic image is also identified by the user. A photographic image at the area of the street-level panoramic image is captured. Camera parameters representing a camera that took the photographic image is determined. A position on the photographic image is mapped to a position on a three-dimensional model. The camera parameters are adjusted so that the three-dimensional model is rendered onto the photographic image from a perspective specified by the adjusted camera parameters.

Claims

1. A computer-implemented method comprising:

(a) receiving, by one or more computing devices, a first input, specified by a user, that identifies a street-level panoramic image from a street-level panoramic image display site, the street-level panoramic image being geocoded to a geographic location;

(b) receiving, by the one or more computing devices, a second input, specified by the user, that identifies an area of the street-level panoramic image;

(c) capturing, by the one or more computing devices, a photographic image at the area of the street-level panoramic image;

(d) determining, by the one or more computing devices, a plurality of camera parameters representing a camera that took the photographic image, the plurality of camera parameters including;

(i) position data corresponding to the geographic location of the street-level panoramic image and (ii) orientation data corresponding to an orientation of the area within the street-level panoramic image;

(e) receiving, by the one or more computing devices, a third input mapping a position on the photographic image to a feature of a three-dimensional model; and

(f) adjusting, by the one or more computing devices, the plurality of camera parameters such that, when the three-dimensional model is rendered onto the photographic image from a perspective specified by the adjusted plurality of camera parameters, the feature of the three-dimensional model appears at the position on the photographic image, wherein a photogrammetry algorithm is used to adjust the plurality of camera parameters according to the third input mapping the position on the photographic image to the feature on the three-dimensional model such that a geometry of the three-dimensional model can be determined.

2. The method of claim 1, further comprising:

(g) texture mapping the photographic image onto the three-dimensional model from a perspective specified by the adjusted plurality of camera parameters.

3. The method of claim 2, wherein a textured geometry includes photographic images that were not selected from the street-level panoramic image display site.

4. The method of claim 1, wherein the capturing (c) comprises capturing the panoramic photographic image from an image displayed in a viewport of the street-level panoramic image display site.

5. The method of claim 1, wherein the receiving (a) comprises maneuvering an icon to a position, specified by the user, in a mapping frame, wherein the position of the icon in the mapping frame corresponds to the image displayed in the viewport of the street-level panoramic image display site.

6. A computing system comprising:

a position icon module implemented by one or more processors that receives a first input, specified by a user, that identifies a street-level panoramic image from a street-level panoramic image display site, the street-level panoramic image being geocoded to a geographic location;

a panorama viewer module implemented by one or more processors that receives a second input, specified by the user, that identifies an area of the street-level panoramic image;

a street-level panoramic image display site module implemented by one or more processors that captures a photographic image at the area of the street-level panoramic image and determines a plurality of camera parameters representing a camera that took the photographic image, the plurality of camera parameters including;

(i) position data corresponding to the geographic location of the street-level panoramic image and (ii) orientation data corresponding to an orientation of the area within the street-level panoramic image;

a user constraint module implemented by one or more processors that receives a third input mapping a position on the photographic image to a feature of a three-dimensional model; and

a photogrammetry module implemented by one or more processors that adjusts the plurality of camera parameters such that, when the three-dimensional model is rendered onto the photographic image from a perspective specified by the adjusted plurality of camera parameters, the feature of the three-dimensional model appears at the position on the photographic image, wherein a photogrammetry algorithm is used to adjust the plurality of camera parameters according to the third input mapping the position on the photographic image to the feature on the three-dimensional model such that a geometry of the three-dimensional model can be determined.

7. The system of claim 6, further comprising:

a texture map module that texture maps the photographic image onto the three dimensional model from a perspective specified by the adjusted plurality of camera parameters.

8. The system of claim 7, wherein a textured geometry includes photographic images that were not selected from the street-level panoramic image display site.

9. The system of claim 6, wherein the street-level panoramic image display site module captures the panoramic photographic image from an image displayed in a viewport of the street-level panoramic image display site.

10. The system of claim 6, wherein the position icon module maneuvers an icon to a position, specified by the user, in a mapping frame, wherein the position of the icon in the mapping frame corresponds to the image displayed in the viewport of the street-level panoramic image display site.

11. A non-transitory computer storage medium encoded with a computer program, the program comprising instructions that when executed by one or more processors cause the one or more processors to perform operations comprising:

(a) receiving a first input, specified by a user, that identifies a street-level panoramic image from a street-level panoramic image display site, the street-level panoramic image being geocoded to a geographic location;

(b) receiving a second input, specified by the user, that identifies an area of the street-level panoramic image;

(c) capturing a photographic image at the area of the street-level panoramic image;

(d) determining a plurality of camera parameters representing a camera that took the photographic image, the plurality of camera parameters including;

(i) position data corresponding to the geographic location of the street-level panoramic image and (ii) orientation data corresponding to an orientation of the area within the street-level panoramic image;

(e) receiving a third input mapping a position on the photographic image to a feature of a three-dimensional model; and

(f) adjusting the plurality of camera parameters such that, when the three-dimensional model is rendered onto the photographic image from a perspective specified by the adjusted plurality of camera parameters, the feature of the three-dimensional model appears at the position on the photographic image, wherein a photogrammetry algorithm is used to adjust the plurality of camera parameters according to the third input mapping the position on the photographic image to the feature on the three-dimensional model such that a geometry of the three-dimensional model can be determined.

12. The computer storage medium of claim 11, the operations further comprising:

(g) texture mapping the photographic image onto the three-dimensional model from a perspective specified by the adjusted plurality of camera parameters.

13. The computer storage medium of claim 12, wherein a textured geometry includes photographic images that were not selected from the street-level panoramic image display site.

14. The computer storage medium of claim 11, wherein the capturing (c) comprises capturing the panoramic photographic image from an image displayed in a viewport of the street-level panoramic image display site.

15. The computer storage medium of claim 11, wherein the receiving (a) comprises maneuvering an icon to a position, specified by the user, in a mapping frame, wherein the position of the icon in the mapping frame corresponds to the image displayed in the viewport of the street-level panoramic image display site.