Geographically locating and posing images in a large-scale image repository and processing framework

• US 9,208,171 B1
• Filed: 09/05/2013
• Issued: 12/08/2015
• Est. Priority Date: 09/05/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Cotting, Daniel, Kehl, Roland, Bluntschli, Boris, Sikora, Krzysztof, Smietanka, Wojciech Stanislaw, Stefcek, Martin
• Primary Examiner(s): Jalil, Neveen Abel
• Assistant Examiner(s): Levitian, Karina
• Application Number:
  US14/018,989
• Time in Patent Office: 824 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/587 using geographical or spatial information, e.g. location

Abstract

Aspects of the disclosure relate generally to systems and methods for geographically locating images. For example, images from different sources may be associated with different types of location information or simply none at all. In order to reduce inconsistency among images, location information may be gathered for an image using bitmap processing, metadata processing, and information retrieved from where the image was found. This location information can be filtered to remove less reliable or conflicting information. Images may be clustered based on their appearance on an interactive online resource that corresponds to a user-defined event, based on image similarity, and by their appearance in a user photo album. The location information of the images of a cluster is then copied to all of the other images of that cluster.

Claims

1. A method comprising:

receiving, by one or more computing devices, a first image associated with an identifier;

using, by the one or more computing devices, the identifier, locating an interactive online resource that corresponds to a user-defined event, the interactive online resource allowing different users to upload images related to the user defined event;

identifying a second image that is from a uniform resource locator (URL) different from a first URL of the interactive online resource and that is identical to the first image;

for the first image, generating, by the one or more computing devices, a first set of location data based on at least information found at the second URL;

generating, by the one or more computing devices, a cluster of images that are all associated with the identifier, the cluster including at least the first image and one other image, wherein the one other image is associated with a second set of location data;

identifying, by the one or more computing devices, a source of each location data in the first set of location data and the second set of location data;

determining, by the one or more computing devices, a reliability level of each source wherein the reliability level of each source comprises assigning a value to each source;

ranking, by the one or more computing devices, each of the location data according to the reliability level;

filtering, by the one or more computing devices, the first set of location data and the second set of location data based on the rankings to create a filtered first set of location data and a filtered second set of location data;

combining, by the one or more computing devices, the filtered first set of location data and the filtered second set of location data into a third set of location data; and

assigning, by the one or more computing devices, the third set of location data to each image of the cluster of images, such that the first image receives at least the filtered second set of location data associated with the one other image and the one other image receives at least the filtered first set of location data associated with the first image.

2. The method of claim 1, further comprising filtering the first set of location data based on sources for the location data of the first set of location data.

3. The method of claim 1, further comprising filtering the first set of location data to remove conflicting location data.

4. The method of claim 1, further comprising:

ranking the third set of location data based on reliability of sources of the third set of location data; and

filtering the third set of location data based on the rankings.

5. The method of claim 1, further comprising filtering the third set of location data to remove conflicting location data.

6. The method of claim 1, further comprising:

identifying conflicting location data of the third set of location data; and

merging the conflicting location data into new non-conflicting location data.

7. The method of claim 1 wherein determining_the reliability level of each source comprises assigning_the value to each source such that user-provided coordinates are ranked higher than sources comprising location data found at a URL.

8. The method of claim 1, wherein the clustered images are a first cluster of images and the method further comprises:

generating a second cluster of images based on whether images meet a threshold level of similarity such that the second cluster of images includes the first image, but not the one other image; and

assigning the third set of location data to each image of the second cluster of images.

9. The method of claim 8, wherein the second cluster of images includes a third image associated with a fourth set of location data and the method further comprises assigning the fourth set of location data to the images of the second cluster such that the first and third images are associated with both the third set of location data and the fourth set of location data and the one other image is associated with the third set of location data.

10. The method of claim 1, wherein the clustered images are a first cluster of images and the method further comprises:

generating a second cluster of images based on whether images are included in a same user defined album of images such that the second cluster of images includes the first image, but not the one other image; and

assigning the third set of location data to each image of the second cluster of images.

11. The method of claim 1, wherein the interactive online resource is a web page and the identifier is a uniform resource locator (URL).

12. A system comprising:

memory storing a first image associated with an identifier and one other image;

one or more computing devices configured to;

use the identifier, locating an interactive online resource that corresponds to a user-defined event, the interactive online resource allowing different users to upload images related to the user defined event;

identifying a second image that is from a uniform resource locator (URL) different from a first URL of the interactive online resource and that is identical to the first image;

for the first image, generate a first set of location data based on at least information found at the second URL;

generate a cluster of images that are all associated with the identifier, the cluster including at least the first image and the one other image, wherein the one other image is associated with a second set of location data;

identify a source of each location data in the first set of location data and the second set of location data;

determine a reliability level of each source wherein the reliability level of each source comprises assigning a value to each source;

rank each of the location data according to the reliability level;

filter the first set of location data and the second set of location data based on the rankings to create a filtered first set of location data and a filtered second set of location data;

combine the filtered first set of location data and the filtered second set of location data into a third set of location data; and

assign the third set of location data to each image of the cluster of images, such that the first image receives at least the filtered second set of location data associated with the one other image and the one other image receives at least the filtered first set of location data associated with the first image.

13. The system of claim 12, wherein the one or more computing devices are further configured to filter the first set of location data based on sources for the location data of the first set of location data.

14. The system of claim 12, wherein the one or more computing devices are further configured to filtering the first set of location data to remove conflicting location data.

15. The system of claim 12, wherein the one or more computing devices are further configured to:

rank the third set of location data based on reliability of sources of the third set of location data; and

filter the third set of location data based on the rankings.

16. The system of claim 12, wherein the processor is further configured to filtering the third set of location data to remove conflicting location data.

17. The system of claim 12, wherein one or more computing devices are further configured to:

identify conflicting location data of the third set of location data; and

merge the conflicting location data into new non-conflicting location data.

18. The system of claim 12 wherein determining_the reliability level of each source comprises assigning_the value to each source such that user-provided coordinates are ranked higher than sources comprising location data found at a URL.

19. The system of claim 12, wherein the clustered images are a first cluster of images and the one or more computing devices are further configured to:

generate a second cluster of images based on whether images meet a threshold level of similarity such that the second cluster of images includes the first image, but not the one other image; and

assign the third set of location data to each image of the second cluster of images.

20. The system of claim 12, wherein the clustered images are a first cluster of images, and the one or more computing devices are further configured to:

generate a second cluster of images based on whether images are included in a same user defined album of images such that the second cluster of images includes the first image, but not the one other image; and

assign the third set of location data to each image of the second cluster of images.