Registering photos in a geographic information system, and applications thereof

• US 9,037,599 B1
• Filed: 05/29/2008
• Issued: 05/19/2015
• Est. Priority Date: 05/29/2007
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Bailly, Francois, McClendon, Brian, Rohlf, John, Szybalski, Andrew Timothy
• Primary Examiner(s): Btit, Jacob F
• Assistant Examiner(s): Holland, Sheryl
• Application Number:
  US12/129,526
• Time in Patent Office: 2,546 Days
• Field of Search: 707/769, 707/999.919
• US Class Current: 707/769
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/748 Hypervideo linking data to content, e.g. by linking an URL to a video object in the context of video distribution systems H04N21/858
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting

Abstract

Embodiments of the present invention relate to displaying and navigating within photo placemarks in a geographic information system. In an embodiment, a method registers a photographic image in a geographic information system. A location of the photographic image within geographic information of the geographic information system is determined based on the geographic information and a metadata of the photographic image. The location of the photographic image is stored.

Claims

1. A computer-implemented method for registering a photographic image in a three-dimensional (3D) geographic environment of a geographic information system, comprising:

(a) determining, by one or more computing devices, a virtual location of the photographic image within the 3D geographic environment based on the 3D geographic environment and metadata of the photographic image, wherein the virtual location within the 3D geographic environment corresponds to a geolocation where the photographic image was captured;

(b) determining, by the one or more computing devices, an orientation direction of the photographic image within the 3D geographic environment by matching one or more physical features captured in the photographic image with one or more physical features represented in the 3D geographic environment, wherein the virtual location and the orientation direction are used to align a scene captured in the photographic image with a 3D scene in the 3D geographic environment;

(c) constructing, by the one or more computing devices, an image pyramid of the photographic image, the image pyramid containing multiple image versions having different resolutions, wherein the constructing comprises determining a resolution and constructing the image pyramid based on the resolution, the determining a resolution comprising calculating a number of levels in the image pyramid based on a tile size, the calculating comprising calculating the number of levels to be the higher of log₂(W/T) and log₂(H/T), wherein W is a width of the photographic image, wherein H is a height of the photographic image, and wherein T is the tile size;

(d) storing, by the one or more computing devices, the image pyramid, the virtual location and the orientation direction of the photographic image;

(e) receiving, by the one or more computing devices, a user input during navigation into the 3D scene in the 3D geographic environment selecting the aligned photographic image within the 3D geographic environment;

(f) in response to receipt of the user input, displaying, by the one or more computing devices, a given image version of the photographic image having a first resolution as stored in the image pyramid; and

(g) in response to an input to zoom into the photographic image, displaying, by the one or more computing devices, a different image version of the photographic image stored in the image pyramid, the different image version having a second greater resolution than the first resolution of the given image version initially displayed.

2. The computer-implemented method of claim 1, further comprising:

(h) determining, by the one or more computing devices, a distance value for a feature captured in the photographic image, wherein the distance describes a distance between the feature and the photographic image'"'"'s capture device with the photographic image was captured, and wherein determining the virtual location is further based on the distance value.

3. The computer-implemented method of claim 2, wherein the determining (h) includes determining the distance value based on an EXIF header of the photographic image.

4. The computer-implemented method of claim 3, wherein the distance value is based on a focal length included in the EXIF header.

5. The computer-implemented method of claim 1, wherein the determining (a) comprises determining the location of the photographic image according to an EXIF header of the photographic image.

6. The computer-implemented method of claim 1, wherein determining (b) includes aligning a feature captured in the photographic image with the feature represented in the 3D scene, wherein the feature captured in the photographic image and the feature represented in the 3D scene are selected by a user.

7. The computer-implemented method of claim 1, wherein determining the virtual location of the photographic image further comprises:

(i) identifying at least two physical features captured in the photographic image;

(ii) identifying at least two a matching physical features represented in the 3D scene; and

(iii) determining the virtual location of the photographic image such that the physical features captured in the photographic image align with the matching physical features represented in the geographic information.

8. A computer system for registering a photographic image in a three-dimensional (3D) geographic environment of a geographic information system comprising:

a photo registrar, implemented on a processor, configured to;

(i) determine a virtual location of the photographic image within the 3D geographic environment based on the 3D geographic environment and metadata of the photographic image, wherein the virtual location within the 3D geographic environment corresponds to a geolocation where the photographic image was captured;

(ii) determine an orientation direction of the photographic image within the 3D geographic environment by matching one or more physical features captured in the photographic image with one or more physical features represented in the 3D geographic environment, wherein the virtual location and the orientation direction are used to align a scene captured in the photographic image with a 3D scene in the 3D geographic environment;

(iii) construct an image pyramid of the photographic image, the image pyramid containing multiple image versions having different resolutions, wherein the constructing comprises determining a resolution and constructing the image pyramid based on the resolution, the determining a resolution comprising calculating a number of levels in the image pyramid based on a tile size, the calculating comprising calculating the number of levels to be the higher of log₂(W/T) and log₂(H/T), wherein W is a width of the photographic image, wherein H is a height of the photographic image, and wherein T is the tile size;

(iv) store the image pyramid, the virtual location and the orientation direction of the photographic image;

(v) receive a user input during navigation into the 3D scene in the 3D geographic environment selecting the aligned photographic image within the 3D geographic environment;

(vi) in response to receipt of the user input, display a given image version of the photographic image having a first resolution as stored in the image pyramid; and

(vii) in response to an input to zoom into the photographic image, display a different image version of the photographic image stored in the image pyramid, the different image version having a second greater resolution than the first resolution of the given image version initially displayed.

9. The computer system of claim 8, wherein the photo registrar is further configured to determine a distance value for a feature captured in the photographic image, wherein the distance describes a distance between the feature and the photographic image'"'"'s capture device with the photographic image was captured, and wherein determining the virtual location is further based on the distance value.

10. The computer system of claim 9, wherein the photo registrar is further configured to determine the distance value based on an EXIF header of the photographic image.

11. The computer system of claim 10, wherein the distance value is based on a focal length included in the EXIF header.

12. The computer system of claim 8, wherein the photo registrar is further configured to determining the virtual location of the photographic image according to an EXIF header of the photographic image.

13. The computer system of claim 8, wherein the photo registrar is further configured to determine the orientation direction by aligning a feature captured in the photographic image with the feature represented in the 3D scene, wherein the feature captured in the photographic image and the feature represented in the 3D scene are selected by a user.

14. The computer system of claim 8, wherein the photo registrar is further configured to determine the virtual location of the photographic image by:

(i) identifying at least two physical features captured in the photographic image;

(ii) identifying at least two a matching physical features represented in the 3D scene; and

(iii) determining the virtual location of the photographic image such that the physical features captured in the photographic image align with the matching physical features represented in the geographic information.

15. A non-transitory computer-readable storage medium having instructions encoded thereon that, when executed by one or more computing devices, causes the one or more computing devices to perform operations comprising:

(a) determining a virtual location of a photographic image within a 3D geographic environment based on the 3D geographic environment and metadata of the photographic image, wherein the virtual location within the 3D geographic environment corresponds to a geolocation where the photographic image was captured;

(b) determining an orientation direction of the photographic image within the 3D geographic environment by matching one or more physical features captured in the photographic image with one or more physical features represented in the 3D geographic environment, wherein the virtual location and the orientation direction are used to align a scene captured in the photographic image with a 3D scene in the 3D geographic environment;

(c) constructing an image pyramid of the photographic image, the image pyramid containing multiple image versions having different resolutions, wherein the constructing comprises determining a resolution and constructing the image pyramid based on the resolution the determining a resolution comprising calculating a number of levels in the image pyramid based on a tile size, the calculating comprising calculating the number of levels to be the higher of log₂(W/T) and log₂(H/T), wherein W is a width of the photographic image, wherein H is a height of the photographic image, and wherein T is the tile size;

(d) storing the image pyramid, the virtual location, and the orientation direction of the photographic image;

(e) receiving a user input during navigation into the 3D scene in the 3D geographic environment selecting the aligned photographic image within the 3D geographic environment;

(f) in response to receipt of the user input, displaying a given image version of the photographic image having a first resolution as stored in the image pyramid; and

(g) in response to an input to zoom into the photographic image, displaying a different image version of the photographic image stored in the image pyramid, the different image version having a second greater resolution than the first resolution of the given image version initially displayed.

16. The non-transitory computer-readable storage medium of claim 15, further comprising:

(h) determining a distance value for a feature captured in the photographic image, wherein the distance describes a distance between the feature and the photographic image'"'"'s capture device with the photographic image was captured, and wherein determining the virtual location is further based on the distance value.

17. The non-transitory computer-readable storage medium of claim 16, wherein the determining (h) includes determining the distance value based on an EXIF header of the photographic image.

18. The non-transitory computer-implemented method of claim 17, wherein the distance value is based on a focal length included in the EXIF header.

19. The non-transitory computer-readable storage medium of claim 15, wherein the determining (a) comprises determining the location of the photographic image according to an EXIF header of the photographic image.

20. The non-transitory computer-readable storage medium of claim 15, wherein determining (b) includes aligning a feature captured in the photographic image with the feature represented in the 3D scene, wherein the feature captured in the photographic image and the feature represented in the 3D scene are selected by a user.

21. The non-transitory computer-readable storage medium of claim 15, wherein determining the virtual location of the photographic image further comprises:

(i) identifying at least two physical features captured in the photographic image;

(ii) identifying at least two a matching physical features represented in the 3D scene; and

(iii) determining the virtual location of the photographic image such that the physical features captured in the photographic image align with the matching physical features represented in the geographic information.