Geocoding of Images Based on Search Queries

• US 20150178322A1
• Filed: 12/20/2013
• Published: 06/25/2015
• Est. Priority Date: 12/20/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Cotting, Daniel, Smietanka, Wojciech Stanislaw, Bluntschli, Boris, Dumazet, Nicolas
• Granted Patent: US 9,552,375 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information

Abstract

Systems and methods for determining a geocode for an image based on user-provided search queries and corresponding user selections are provided. One example method includes determining a selection value for each of a plurality of search strings associated with an image based at least in part on user selection data. The method includes generating a textual document for the image based at least in part on the selection values. The textual document includes one or more of the plurality of search strings. The method includes identifying a plurality of geographic entities by analyzing the textual document using a textual processor. The method includes selecting one of the plurality of geographic entities as a primary geographic entity and associating, by the one or more computing devices, a geocode associated with the primary geographic entity with the image.

Claims

1. A method for geocoding images, the method comprising:

determining, by one or more computing devices, a selection value for each of a plurality of search strings associated with an image, wherein the selection value for each search string is based at least in part on the number of instances in which the image was selected when the image was presented as a search result responsive to such search string;

generating, by the one or more computing devices, a textual document for the image based at least in part on the selection values, wherein the textual document includes one or more of the plurality of search strings;

identifying, by the one or more computing devices, a plurality of geographic entities by analyzing the textual document using a textual processor, wherein a plurality of geocodes are respectively associated with the plurality of geographic entities;

selecting, by the one or more computing devices, one of the plurality of geographic entities as a primary geographic entity; and

associating, by the one or more computing devices, the geocode associated with the primary geographic entity with the image.

2. The method of claim 1, further comprising:

receiving, by the one or more computing devices, a search query;

identifying, by the one or more computing devices, the image as responsive to the search query based at least in part on the geocode associated with the image; and

providing, by the one or more computing device, the image as a search result for the search query.

3. The method of claim 1, further comprising, prior to determining the selection value for each of the plurality of search strings:

receiving, by the one or more computing devices, an image search query from a user, wherein the image search query includes one of the plurality of search strings;

identifying, by the one or more computing devices, a plurality of response images in response to the image search query, wherein the image is included in the plurality of response images;

receiving, by the one or more computing devices, data indicating that the user has selected the image from the plurality of response images; and

in response to receiving the data indicating that the user has selected the image from the plurality of response images, incrementing, by the one or more computing devices, the selection value associated with search string included in the image search query.

4. The method of claim 1, wherein the selection value for each search string associated with the image comprises the number of instances in which the image was selected when the image is presented as a search result responsive to such search string.

5. The method of claim 1, wherein the textual document for the image comprises a predetermined number of the plurality of search strings having the selection values of greatest magnitude, and wherein the predetermined number of the plurality of search strings are ordered according to their corresponding selection values.

6. The method of claim 1, wherein:

the textual processor provides a relevance score for each of the plurality of geographic entities; and

selecting, by the one or more computing devices, one of the plurality of plurality of geographic entities comprises selecting, by the one or more computing devices, the geographic entity having the relevance score of greatest magnitude.

7. The method of claim 1, further comprising ordering, by the one or more computing devices, the plurality of geographic entities into a sequence based on confidence;

and wherein selecting, by the one or more computing devices, one of the plurality of plurality of geographic entities comprises selecting, by the one or more computing devices, the first sequential geographic entity.

8. The method of claim 7, further comprising, after selecting, by the one or more computing devices, the first sequential geographic entity:

determining, by the one or more computing devices for each sequential geographic entity, whether such geographic entity encompasses the most recently selected geographic entity; and

when it is determined that a geographic entity encompasses the most recently selected geographic entity, selecting, by the one or more computing devices, such geographic entity as an auxiliary geographic entity for the image.

9. The method of claim 1, further comprising selecting, by the one or more computing devices, one or more auxiliary geographic entities from the plurality of geographic entities, wherein the one or more auxiliary geographic entities respectively encompass the primary geographic entity.

10. The method of claim 1, further comprising determining, by the one or more computing devices, a precision value for the image based at least in part on a geographic area covered by primary geographic entity.

11. The method of claim 1, further comprising determining, by the one or more computing devices, a quality value for the primary geographic entity by performing operations, the operations comprising:

identifying, by the one or more computing devices for each of the plurality of geographic entities, a plurality of other images for which the primary geographic entity has been selected;

obtaining, by the one or more computing devices, adjunct location data for each of the plurality of other images, wherein the adjunct location data specifies a location;

determining, by the one or more computing devices for each of the plurality of other images, the quality value for the primary geographic entity based at least in part on the percentage of the locations specified by the adjunct location data that are encompassed by the primary geographic entity.

12. A computing system comprising one or more processors and one or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising:

determining, for each of a plurality of images, a number of user selections associated with each of a plurality of search strings;

generating, for each of the plurality of images, a textual document that lists one or more of the plurality of search strings, wherein the one or more listed search strings are ordered according to their respective numbers of user selections;

identifying, for each of the plurality of images, a plurality of entities based at least in part on the textual document associated with such image, wherein the plurality of entities for each image are ordered into a sequence according to a confidence metric, and wherein one or more of the plurality of entities identified based on each textual document comprise geographic entities respectively having associated geocodes; and

selecting, for each of the plurality of images, the geocode of the first sequential geographic entity as the primary geocode for such image.

13. The computing system of claim 12, wherein the number of user selections for each of the plurality of search strings for each of the plurality of images comprises the number of instances in which such image was returned as a search result in response to such search string and then selected.

14. The computing system of claim 12, wherein identifying, for each of the plurality of images, a plurality of entities based at least in part on the textual document associated with such image comprises analyzing, for each of the plurality of images, the textual document associated with such image using a textual processor.

15. The computing system of claim 14, wherein the confidence metric for the plurality of entities identified for each document comprises a plurality of relevance values assigned to the plurality of entities by the textual processor.

16. The computing system of claim 12, wherein the operations further comprise:

determining, for each of the plurality of images, a precision value based at least in part on a geographic area associated with the first sequential geographic entity for such image; and

determining, for each geographic entity, a quality value based at least in part on a percentage of associated image locations that are encompassed by such geographic entity.

17. One or more non-transitory computer-readable media storing instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations, the operations comprising:

generating a text string for an image, wherein the text string comprises a plurality search strings for which the image was returned as a search result and selected the greatest number of times;

processing the text string for the image to identify a plurality of geographic entities, wherein a plurality of confidence scores are respectively associated with the plurality of geographic entities; and

associating with the image a geocode associated with the geographic entity having the largest confidence score.

18. The one or more non-transitory computer-readable media of claim 17, wherein:

associating the geocode associated with the geographic entity having the largest confidence score with the image comprises selecting the geocode associated with the geographic entity having the largest confidence score as the primary geocode for the image; and

the operations further comprise iteratively considering the plurality of geographic entities in order of confidence scores to select one or more auxiliary geocodes.

19. The one or more non-transitory computer-readable media of claim 18, wherein iteratively considering the plurality of geographic entities in order of confidence scores to select one or more auxiliary geocodes comprises performing a plurality of iterations, each iteration comprising:

obtaining the geographic entity having the next largest confidence score;

determining whether the geographic entity associated with the most recently selected geocode is encompassed within the obtained geographic entity;

when it is determined that the geographic entity associated with the most recently selected geocode is encompassed within the obtained geographic entity, selecting the geocode associated with the obtained geographic entity as an auxiliary geocode.

20. The one or more non-transitory computer-readable media of claim 17, wherein:

the text string comprises the plurality of search strings in an order of decreasing number of user selections; and

the confidence score associated with each of the plurality of geographic entities is based at least in part on the order of the search strings.