SELECTING REPRESENTATIVE IMAGES FOR ESTABLISHMENTS

• US 20140003650A1
• Filed: 09/06/2013
• Published: 01/02/2014
• Est. Priority Date: 06/18/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Urbach, Shlomo, Yadid, Tal, Netzer, Yuval, Frome, Andrea, Ben-Haim, Noam
• Granted Patent: US 8,811,656 B2
• Time in Patent Office: Days
• US Class Current: 382/100
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/268 Lexical context

Abstract

Establishments are identified in geo-tagged images. According to one aspect, text regions are located in a geo-tagged image and text strings in the text regions are recognized using Optical Character Recognition (OCR) techniques. Text phrases are extracted from information associated with establishments known to be near the geographic location specified in the geo-tag of the image. The text strings recognized in the image are compared with the phrases for the establishments for approximate matches, and an establishment is selected as the establishment in the image based on the approximate matches. According to another aspect, text strings recognized in a collection of geo-tagged images are compared with phrases for establishments in the geographic area identified by the geo-tags to generate scores for image-establishment pairs. Establishments in each of the large collection of images as well as representative images showing each establishment are identified using the scores.

Claims

1. A computer-implemented method of identifying a representative geo-tagged image for an establishment, comprising:

recognizing text strings in a plurality of geo-tagged images;

identifying an establishment near geographic locations specified by geo-tags of the plurality of geo-tagged images;

extracting a plurality of phrases from information associated with the establishment;

comparing text strings recognized in the plurality of geo-tagged images with the extracted phrases to derive one or more matches between the text strings and the phrases;

scoring a plurality of image-establishment pairs responsive to the comparisons, an image-establishment pair pairing a geo-tagged image having a recognized text string that matches a phrase extracted from information associated with the establishment; and

selecting a representative geo-tagged image for the establishment from among the plurality of image-establishment pairs responsive to the scores.

2. The method of claim 1, wherein the phrase comprises an n-gram of the information associated with the establishment, and wherein a match comprises an occurrence of the n-gram in a text string.

3. The method of claim 1, wherein the one or more matches comprise an approximate match within a predetermined threshold matching rate.

4. The method of claim 1, wherein scoring the plurality of image-establishment pairs comprises:

assigning a score to the image-establishment pair based on one or more of the following factors;

an information field from which the phrase is extracted,

a confidence measuring an accuracy of the recognized text string,

a prevalence of the phrase in a language associated with the phrase,

a prevalence of the phrase in a geographic area specified by a geo-tag of the geo-tagged image, and

a distance between a geographic location specified by the geo-tag of the geo-tagged image and a location of the establishment,

wherein the selecting step comprises selecting the representative geo-tagged image based on the score.

5. The method of claim 1, wherein a plurality of text strings are recognized in the geo-tagged image, and the score assigned to the image-establishment pair is based on comparisons of the plurality of phrases with the plurality of recognized text strings.

6. The method of claim 1, wherein a plurality of establishments are identified near geographic locations specified by geo-tags of the plurality of geo-tagged images, a plurality of phrases are extracted from information associated with the establishments, the text strings are compared with the plurality of extracted phrases to derive a plurality of matches between the text strings and the phrases and produce the plurality of image-establishment pairs, and representative geo-tagged images are selected for the plurality of establishments from among the plurality of image-establishment pairs responsive to the scores.

7. A computer system for identifying a representative geo-tagged image for an establishment, comprising:

a non-transitory computer-readable storage medium comprising executable computer program code for;

recognizing text strings in a plurality of geo-tagged images,

identifying an establishment near geographic locations specified by geo-tags of the plurality of geo-tagged images,

extracting a plurality of phrases from information associated with the establishment,

comparing text strings recognized in the plurality of geo-tagged images with the extracted phrases to derive one or more matches between the text strings and the phrases,

scoring a plurality of image-establishment pairs responsive to the comparisons, an image-establishment pair pairing a geo-tagged image having a recognized text string that matches a phrase extracted from information associated with the establishment, and

selecting a representative geo-tagged image for the establishment from among the plurality of image-establishment pairs responsive to the scores; and

a processor for executing the executable computer program code.

8. The computer system of claim 7, wherein the phrase comprises an n-gram of the information associated with the establishment, and wherein a match comprises an occurrence of the n-gram in a text string.

9. The computer system of claim 7, wherein the one or more matches comprise an approximate match within a predetermined threshold matching rate.

10. The computer system of claim 7, wherein scoring the plurality of image-establishment pairs comprises:

assigning a score to the image-establishment pair based on one or more of the following factors;

an information field from which the phrase is extracted,

a confidence measuring an accuracy of the recognized text string,

a prevalence of the phrase in a language associated with the phrase,

a prevalence of the phrase in a geographic area specified by a geo-tag of the geo-tagged image,

a distance between a geographic location specified by the geo-tag of the geo-tagged image and a location of the establishment,

wherein the selecting comprises selecting the representative geo-tagged image based on the score.

11. The computer system of claim 7, wherein a plurality of text strings are recognized in the geo-tagged image, and the score assigned to the image-establishment pair is based on comparisons of the plurality of phrases with the plurality of recognized text strings.

12. The computer system of claim 7, wherein a plurality of establishments are identified near geographic locations specified by geo-tags of the plurality of geo-tagged images, a plurality of phrases are extracted from information associated with the establishments, the text strings are compared with the plurality of extracted phrases to derive a plurality of matches between the text strings and the phrases to produce the plurality of image-establishment pairs, and representative geo-tagged images are selected for the plurality of establishments from among the plurality of image-establishment pairs responsive to the scores.

13. A non-transitory computer-readable storage medium storing executable computer program instructions for identifying a representative geo-tagged image for an establishment, the computer program instructions comprising instructions for:

recognizing text strings in a plurality of geo-tagged images;

identifying an establishment near geographic locations specified by geo-tags of the plurality of geo-tagged images;

extracting a plurality of phrases from information associated with the establishment;

comparing text strings recognized in the plurality of geo-tagged images with the extracted phrases to derive one or more matches between the text strings and the phrases;

scoring a plurality of image-establishment pairs responsive to the comparisons, an image-establishment pair pairing a geo-tagged image having a recognized text string that matches a phrase extracted from information associated with the establishment; and

selecting a representative geo-tagged image for the establishment from among the plurality of image-establishment pairs responsive to the scores.

14. The storage medium of claim 13, wherein the phrase comprises an n-gram of the information associated with the establishment, and wherein a match comprises an occurrence of the n-gram in a text string.

15. The storage medium of claim 13, wherein the one or more matches comprise an approximate match within a predetermined threshold matching rate.

16. The storage medium of claim 13, wherein scoring the plurality of image-establishment pairs comprises:

assigning a score to the image-establishment pair based on one or more of the following factors;

an information field from which the phrase is extracted,

a confidence measuring an accuracy of the recognized text string,

a prevalence of the phrase in a language associated with the phrase,

a prevalence of the phrase in a geographic area specified by a geo-tag of the geo-tagged image,

a distance between a geographic location specified by the geo-tag of the geo-tagged image and a location of the establishment,

wherein the selecting comprises selecting the representative geo-tagged image based on the score.

17. The storage medium of claim 13, wherein a plurality of text strings are recognized in the geo-tagged image, and the score assigned to the image-establishment pair is based on comparisons of the plurality of phrases with the plurality of recognized text strings.

18. The storage medium of claim 13, wherein a plurality of establishments are identified near geographic locations specified by geo-tags of the plurality of geo-tagged images, a plurality of phrases are extracted from information associated with the establishments, the text strings are compared with the plurality of extracted phrases to derive a plurality of matches between the text strings and the phrases to produce the plurality of image-establishment pairs, and representative geo-tagged images are selected for the plurality of establishments from among the plurality of image-establishment pairs responsive to the scores.