Automatic discovery of popular landmarks

• US 9,014,511 B2
• Filed: 09/14/2012
• Issued: 04/21/2015
• Est. Priority Date: 05/12/2008
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Brucher, Fernando A., Buddemeier, Ulrich, Adam, Hartwig, Neven, Hartmut
• Primary Examiner(s): Wu, Jingge
• Application Number:
  US13/619,652
• Publication Number:
  US 20130138685A1
• Time in Patent Office: 949 Days
• Field of Search: None
• US Class Current: 382/305
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/587 using geographical or spatial information, e.g. location
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 7/08 Sorting, i.e. grouping record carriers in numerical or other ordered sequence according to the classification of at least some of the information they carry by merging two or more sets of carriers in ordered sequence G06F7/16
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85

Abstract

In one embodiment the present invention is a method for populating and updating a database of images of landmarks including geo-clustering geo-tagged images according to geographic proximity to generate one or more geo-clusters, and visual-clustering the one or more geo-clusters according to image similarity to generate one or more visual clusters. In another embodiment, the present invention is a system for identifying landmarks from digital images, including the following components: a database of geo-tagged images; a landmark database; a geo-clustering module; and a visual clustering module. In other embodiments the present invention may be a method of enhancing user queries to retrieve images of landmarks, or a method of automatically tagging a new digital image with text labels.

Claims

1. A method of enhancing user queries to retrieve images of landmarks, comprising:

receiving by a computer a user query;

identifying by the computer one or more trigger words in the user query;

selecting by the computer one or more corresponding tags from a landmark database corresponding to the one or more trigger words, generating a supplemented user query;

supplementing by the computer the user query with the one or more corresponding tags and a popularity criteria, to generate a supplemented user query;

retrieving by the computer landmark images based on the supplemented user query;

generating by the computer a user interface including the one or more retrieved landmark images;

causing one or more summary lists of visual clusters for the retrieved landmark images to be displayed on the user interface, wherein each summary list corresponds to one of the retrieved landmark images; and

causing popularity information based on the popularity criteria to be displayed on the one or more summary lists on the user interface.

2. The method of claim 1, further comprising:

ordering the retrieved images according to the popularity criteria.

3. The method of claim 2, wherein the popularity criteria includes the popularity of landmarks based on a number of unique user identifiers associated with images having each landmark.

4. The method of claim 3, wherein the specific popularity criteria includes the number of images with unique user identifiers within a predetermined radius of a cluster of images.

5. The method of claim 1, wherein each summary list includes information on whether the visual clusters have been modified.

6. The method of claim 1, wherein the popularity criteria includes the number of images with unique user identifiers within a predetermined radius of a cluster of images.

7. The method of claim 1, wherein each summary list includes descriptive information about the visual clusters.

8. The method of claim 1, wherein each summary list includes a link that can be used to retrieve further details about the visual clusters.

9. A computer program product comprising a computer usable medium having control logic stored therein for causing a computer to enhance user queries, said control logic comprising:

a first computer readable program code that enables the computer to receive a user query and identify one or more trigger words in the user query;

a second computer readable program code that enables the computer to select one or more corresponding tags from a landmark database corresponding to the one or more trigger words;

a third computer readable program code that enables the computer to supplement the user query with the one or more corresponding tags to generate a supplemented user query;

a fourth computer readable program code that enables the computer to retrieve landmarks based on the supplemented user query; and

a fifth computer readable program code that enables the computer to generate a user interface including the one or more retrieved landmarks and display one or more summary lists of visual clusters for the retrieved landmarks on the user interface, wherein each summary list corresponds to one of the retrieved landmarks, and to display popularity information on the one or more summary lists.

10. The method of claim 9 further comprising:

a sixth computer readable program code that enables the computer to order the retrieved landmarks according to the specific popularity criteria.

11. The method of claim 9, wherein the popularity criteria includes a popularity of landmarks based on a number of unique user identifiers associated with images having each landmark.

12. The computer program product of claim 9, wherein each summary list includes descriptive information about the visual clusters.

13. The computer program product of claim 9, wherein each summary list includes a link that can be used to retrieve further details about the visual clusters.

14. A method comprising:

receiving by a computer a user query;

identifying by the computer one or more trigger words in the user query;

selecting by the computer one or more corresponding tags from a landmark database corresponding to the one or more trigger words;

supplementing by the computer the user query with the one or more corresponding tags, generating a supplemented user query;

retrieving by the computer one or more landmarks based on the supplemented user query;

generating by the computer a user interface including the one or more retrieved landmarks; and

causing one or more summary lists of visual clusters for the retrieved landmarks to be displayed on the user interface, wherein each summary list corresponds to one of the retrieved landmarks; and

causing popularity information to be displayed on the one or more summary lists on the user interface.

15. The method of claim 14, wherein each summary list includes descriptive information about the visual clusters.

16. The method of claim 14, wherein each summary list includes a link that can be used to retrieve further details about the visual clusters.

17. The method of claim 14, wherein each summary list includes a location to receive user input specific to the retrieved landmarks.

18. The method of claim 14, wherein each summary list includes information on whether the visual clusters have been modified.

19. The method of claim 14, wherein the retrieved landmarks are ordered according to the popularity criteria.

20. The method of claim 14, wherein the popularity criteria includes the popularity of landmarks based on a number of unique user identifiers associated with images having each landmark.