Rank-based image piling

• US 8,660,358 B1
• Filed: 02/16/2012
• Issued: 02/25/2014
• Est. Priority Date: 02/18/2011
• Status: Expired due to Fees

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Jones, Jonah, Bergboer, Niek, Schaffalitzky, Frederik
• Primary Examiner(s): Rahmjoo, Mike
• Application Number:
  US13/397,973
• Time in Patent Office: 740 Days
• Field of Search: 382/190, 382/154, 382/199, 382/240, 382/176, 382/325, 715/246, 345/629, 358/1.18
• US Class Current: 382/190
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selectively providing images. In one aspect, a method includes receiving image data that specify feature values for a plurality of images. The image data include, for each image, location data that specify a geographic location for the image. A group of images in which each image has location data specifying a geographic location that is within a threshold distance of a reference location are selected. Pairs of matching images are selected from the group of images. A reference image for the geographic location is selected from the pairs of matching images. Data that cause presentation in a map space of a photo collection image that includes a visual representation of the reference image are provided. The photo collection image is presented in the map space and at a map position for the geographic location.

Claims

1. A method performed by data processing apparatus, the method comprising:

receiving image data that specify feature values for a plurality of images and location data that specify a geographic location for each of the images;

selecting different groups of images based on a zoom level at which a map portion is presentable in the map space;

selecting, by one or more data processing apparatus, a group of images in which each image has a geographic location that is within a threshold distance of a reference location, wherein the threshold distance of the reference location is proportional to a size of a geographic region that is represented by the map portion;

selecting pairs of matching images from the group of images, each selection being based on a pair of matching images having an image similarity score that meets a similarity threshold, the similarity score for each pair being based, at least in part, on the feature values of each image in the pair of images;

selecting, from the pairs of matching images, a reference image for the geographic location, the reference image being selected based, at least in part, on a number of the matching pairs that include the reference image;

determining, by one or more data processing apparatus, an image effect to use when presenting the reference image, the image effect determined based on the number of matching pairs; and

providing, by one or more data processing apparatus, data that cause presentation in a map space of a photo collection image that includes a visual representation of the reference image, the photo collection image being presented with the determined image effect, and the photo collection image being presented in the map space and at a map position for the geographic location, wherein a number of photos visually represented by the photo collection image is proportional to the number of the matching pairs that include the reference image.

2. The method of claim 1, wherein selecting a reference image for the geographic location comprises selecting, from the pairs of images, the reference image based at least in part on user feedback data specifying popularity scores for images in the pairs.

3. The method of claim 2, wherein selecting a reference image for the geographic location comprises:

ranking the images in the pairs based on the popularity scores, the rank of each image being proportional to the popularity score for the image; and

selecting the reference image based on the ranking.

4. The method of claim 1, wherein selecting a reference image comprises selecting a reference image for each of a plurality of different geographic locations, the different geographic locations for which a reference image is selected being based, at least in part, on a zoom level at which the map portion will be presented.

5. The method of claim 4, further comprising selecting the geographic locations to prevent overlap of two or more reference images in the map space.

6. The method of claim 5, wherein selecting the geographic locations comprises:

for each geographic location that is specified by the location data;

determining a number of pairs of matching images for the geographic location;

ranking the geographic locations based on the number of pairs of matching images; and

selecting one or more of the geographic locations based on the ranking.

7. The method of claim 1, further comprising receiving from a user device, a request for a map space, wherein providing the data comprise providing the data to the user device.

8. A non-transitory computer storage medium encoded with a computer program, the program comprising instructions that when executed by data processing apparatus cause the data processing apparatus to perform operations comprising:

receiving image data that specify feature values for a plurality of images and location data that specify a geographic location for each of the images;

selecting different groups of images based on a zoom level at which a map portion is presentable in the map space;

selecting a group of images in which each image has a geographic location that is within a threshold distance of a reference location, wherein the threshold distance of the reference location is proportional to a size of a geographic region that is represented by the map portion;

selecting pairs of matching images from the group of images, each selection being based on a pair of matching images having an image similarity score that meets a similarity threshold, the similarity score for each pair being based, at least in part, on the feature values of each image in the pair of images;

selecting, from the pairs of matching images, a reference image for the geographic location, the reference image being selected based, at least in part, on a number of the matching pairs that include the reference image;

determining an image effect to use when presenting the reference image, the image effect determined based on the number of matching pairs; and

providing data that cause presentation in a map space of the reference image and a photo collection image, the photo collection image being presented with the determined image effect, and the reference image and photo collection image being presented in the map space and at a map position for the geographic location, wherein a number of photos visually represented by the photo collection image is proportional to the number of the matching pairs that include the reference image.

9. The computer storage medium of claim 8, wherein the program further comprises instructions that cause the data processing apparatus to perform operations including:

ranking the images in the pairs based on popularity scores for the images, the rank of each image being proportional to the popularity score for the image; and

selecting the reference image based on the ranking.

10. A system comprising:

a data store storing image data that specify feature values for a plurality of images, the data store further storing, for each image, location data that specify a geographic location for the image; and

one or more computers operable to interact with the data store and to perform operations including;

receiving image data from the data store;

selecting different groups of images based on a zoom level at which a map portion is presentable in the map space;

selecting a group of images in which each image has a geographic location that is within a threshold distance of a reference location, wherein the threshold distance of the reference location is proportional to a size of a geographic region that is represented by the map portion;

selecting pairs of matching images from the group of images, each selection being based on a pair of matching images having an image similarity score that meets a similarity threshold, the similarity score for each pair being based, at least in part, on the feature values of each image in the pair of images;

selecting, from the pairs of matching images, a reference image for the geographic location, the reference image being selected based, at least in part, on a number of the matching pairs that include the reference image;

determining an image effect to use when presenting the reference image, the image effect determined based on the number of matching pairs and being an image effect in which a number of photos visually represented by the photo collection image is proportional to the number of matching pairs that include the reference image; and

providing data that cause presentation in a map space of the reference image and a photo collection image, the photo collection image being presented with the determined image effect, and the reference image and photo collection image being presented in the map space and at a map position for the geographic location, wherein a number of photos visually represented by the photo collection image is proportional to the number of the matching pairs that include the reference image.

11. The system of claim 10, wherein the one or more computers are further operable to perform operations including selecting, from the pairs of images, the reference image based at least in part on user feedback data specifying popularity scores for images in the pairs.

12. The system of claim 11, wherein the one or more computers are further operable to perform operations including:

ranking the images in the pairs based on the popularity scores, the rank of each image being proportional to the popularity score for the image; and

selecting the reference image based on the ranking.

13. The system of claim 10, wherein the one or more computers are further operable to perform operations including selecting the geographic locations to prevent overlap of two or more reference images in the map space.

14. The system of claim 13, wherein the one or more computers are further operable to perform operations including:

for each geographic location that is specified by the location data;

determining a number of pairs of matching images for the geographic location;

ranking the geographic locations based on the number of pairs of matching images; and

selecting one or more of the geographic locations based on the ranking.

15. The system of claim 10, further comprising a user device, wherein the one or more computers are further operable to interact with the user device and to perform operations including:

receiving a request for a map space from the user device; and

providing the data that cause presentation of the reference image and the photo collection image to the user device.