Selecting photographs for a destination or point of interest

• US 10,510,129 B2
• Filed: 10/12/2017
• Issued: 12/17/2019
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: James, Barnaby John, Kolluri, Bala Venkata Sai Ravi Krishna
• Primary Examiner(s): Fujita, Katrina R
• Application Number:
  US15/782,351
• Publication Number:
  US 20180033099A1
• Time in Patent Office: 796 Days
• CPC Class Codes: G01C 21/3679 Retrieval, searching and output of POI information, e.g. hotels, restaurants, shops, filling stations, parking facilities G01C21/3611 takes precedence
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/95 Retrieval from the web
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06Q 30/0625 Directed, with specific intent or strategy
• CPC Class Codes: G06Q 30/0627 using item specifications
• CPC Class Codes: G06Q 50/14 Travel agencies
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 7/11 Region-based segmentation

Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for computerized travel services. One of the methods includes identifying photographs using an index of photographs, the photographs being identified from the index as photographs geographically related to a point of interest or destination and having a creation timestamp corresponding to a time of the year; determining for each of the photographs, a relevancy score based at least in part on: selection success data of the photograph for image queries referring to the point of interest or destination, and references to the point of interest or destination in documents associated with the photograph; and selecting a selected photograph from the photographs based at least in part on a respective visual quality score and the respective relevancy scores, the visual quality score representing a degree of visual quality of the respective photographs.

Claims

1. A method to provide photographs in response to search queries, comprising:

retrieving, using one or more computing devices, geographic data comprising a logical hierarchy of terms indicating relationships between one or more geographic destinations or points of interest including at least a point of interest indicated by data in a search query;

identifying, using the one or more computing devices, a plurality of photographs geographically related to the point of interest;

determining for each of the plurality of photographs, using the one or more computing devices, a relevancy score based at least in part on references to the point of interest in documents associated with the photograph, wherein determining references to the point of interest that are associated with the photograph comprises;

identifying electronic documents associated with the photograph based on one or more criteria; and

determining which of the documents associated with the photograph reference the point of interest, wherein a document is determined to reference the point of interest when the document comprises terms that are relevant to the point of interest identified from the geographic data comprising the logical hierarchy of terms;

selecting, using the one or more computing devices, a particular photograph from the plurality of photographs based at least in part on a respective relevancy score for each photograph; and

providing, using the one or more computing devices and to a user computing device, the particular photograph in response to the search query for presentation of the particular photograph via the user computing device.

2. The method of claim 1, wherein the plurality of photographs is identified according to temporal metadata and geographic metadata.

3. The method of claim 1, wherein each of the plurality of photographs is associated with a respective geographic location that is within a threshold distance of the point of interest.

4. The method of claim 1, wherein the relevancy score is based further in part on selection success data comprising click-through data or explicit feedback data.

5. The method of claim 1, wherein the plurality of photographs are identified based at least in part on photograph clustering using non-geographic and geographic metadata.

6. The method of claim 1, wherein each of the image queries referring to the point of interest includes a term or terms associated with the point of interest.

7. The method of claim 1, wherein the documents comprise web pages, pages in a social network, or pages in a photo sharing service.

8. The method of claim 1, wherein the selection success data of the photograph is for image queries that refer to an activity or the point of interest and the references in documents associated with the photograph are to the point of interest or the activity.

9. The method of claim 1, wherein the particular photograph from the plurality of photographs is selected further based at least in part on a respective visual quality score for each photograph, wherein the visual quality score is based at least in part on at least one of an exposure rating, a color or saturation rating, a clarity rating, a photographer rating, a camera rating, or a user feedback rating.

10. The method of claim 1, wherein the respective relevancy scores are based at least in part on computer image recognition of an object associated with the point of interest.

11. The method of claim 1, further comprising receiving, using the one or more computing devices and from the user computing device, the search query comprising the data indicative of the point of interest.

12. The method of claim 1, wherein the plurality of photographs are identified from an index of photographs.

13. A system to provide photographs in response to search queries, comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

retrieve geographic data comprising a logical hierarchy of terms indicating relationships between one or more geographic destinations or points of interest including at least a point of interest indicated by data in a search query;

identify a plurality of photographs geographically related to the point of interest;

determine, for each of the plurality of photographs, a relevancy score based at least in part on references to the point of interest in documents associated with the photograph, wherein determining references to the point of interest that are associated with the photograph comprises;

identifying electronic documents associated with the photograph based on one or more criteria; and

determining which of the documents associated with the photograph reference the point of interest, wherein a document is determined to reference the point of interest when the document comprises terms that are relevant to the point of interest identified from the geographic data comprising the logical hierarchy of terms;

selecting a particular photograph from the plurality of photographs based at least in part on a respective relevancy score for each photograph; and

providing, to a user computing device, the particular photograph in response to the search query for presentation of the particular photograph via the user computing device.

14. The system of claim 13, wherein the plurality of photographs is identified according to temporal metadata and geographic metadata.

15. The system of claim 13, wherein each of the plurality of photographs is associated with a respective geographic location that is within a threshold distance of the point of interest.

16. The system of claim 13, wherein the one or more computers are further operable to execute instructions stored in the one or more storage devices to cause the one or more computers to receive, from the user computing device, the search query comprising the data indicative of the point of interest.

17. The system of claim 13, wherein the plurality of photographs are identified from an index of photographs.

18. A computer program product, comprising:

a non-transitory computer readable storage medium encoded with computer-executable instructions that, when executed by one or more computers, cause the one or more computers to provide photographs in response to search queries, the computer-executable instructions comprising;

computer executable instructions to retrieve geographic data comprising a logical hierarchy of terms indicating relationships between one or more geographic destinations or points of interest including at least a point of interest indicated by data in a search query;

computer executable instructions to identify a plurality of photographs geographically related to the point of interest;

computer executable instructions to determine, for each of the plurality of photographs, a relevancy score based at least in part on references to the point of interest in documents associated with the photograph, wherein determining references to the point of interest that are associated with the photograph comprises;

identifying electronic documents associated with the photograph based on one or more criteria; and

determining which of the documents associated with the photograph reference the point of interest, wherein a document is determined to reference the point of interest when the document comprises terms that are relevant to the point of interest identified from the geographic data comprising the logical hierarchy of terms;

computer executable instructions to select a particular photograph from the plurality of photographs based at least in part on the respective relevancy score for each photograph; and

computer-executable instructions to provide to a user computing device, the particular photograph in response to the search query for presentation of the particular photograph via the user computing device.

19. The computer program product of claim 18, further comprising computer-executable instructions to receive, from the user computing device, the search query comprising the data indicative of the point of interest.

20. The computer program product of claim 18, wherein the plurality of photographs are identified from an index of photographs.