Selecting representative images for establishments

• US 8,811,656 B2
• Filed: 09/06/2013
• Issued: 08/19/2014
• Est. Priority Date: 06/18/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Urbach, Shlomo, Yadid, Tal, Netzer, Yuval, Frome, Andrea, Ben-Haim, Noam
• Primary Examiner(s): Carter, Aaron W
• Application Number:
  US14/019,670
• Publication Number:
  US 20140003650A1
• Time in Patent Office: 347 Days
• Field of Search: 382/100, 382/182, 382/218
• US Class Current: 382/100
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/268 Lexical context

Abstract

Establishments are identified in geo-tagged images. According to one aspect, text regions are located in a geo-tagged image and text strings in the text regions are recognized using Optical Character Recognition (OCR) techniques. Text phrases are extracted from information associated with establishments known to be near the geographic location specified in the geo-tag of the image. The text strings recognized in the image are compared with the phrases for the establishments for approximate matches, and an establishment is selected as the establishment in the image based on the approximate matches. According to another aspect, text strings recognized in a collection of geo-tagged images are compared with phrases for establishments in the geographic area identified by the geo-tags to generate scores for image-establishment pairs. Establishments in each of the large collection of images as well as representative images showing each establishment are identified using the scores.

Claims

1. A computer-implemented method, comprising:

recognizing, with one or more computing devices, one or more text strings in each of a plurality of geo-tagged images, each of the plurality of geo-tagged images indicating a geographical location associated with its corresponding geo-tagged image;

identifying, with the one or more computing devices, one or more establishments associated with the geographical locations of the plurality of geo-tagged images;

for at least one specific establishment of the one or more establishments;

(i) extracting, with the one or more computing devices, one or more phrases from information associated with the specific establishment;

(ii) comparing, with the one or more computing devices, the one or more text strings recognized in the plurality of geo-tagged images with the extracted one or more phrases to derive one or more matches; and

(iii) selecting, with the one or more computing devices, a representative geo-tagged image for the specific establishment from among the plurality of geo-tagged images based at least in part on one or more matches of a specific geo-tagged image with the specific establishment.

2. The method of claim 1, wherein selecting a representative geo-tagged image comprises scoring each of one or more image-establishment pairs to obtain a corresponding score, wherein selecting the representative geo-tagged image for the specific establishment is further based on the corresponding scores.

3. The method of claim 2, wherein:

each of the one or more matches comprises an approximate match between a specific text string of the one or more text strings and a specific phrase of the extracted one or more phrases, and

scoring each of the one or more image-establishment pairs comprises assigning a score to a specific image-establishment pair based on one or more of the following factors;

(i) an information field from which the specific phrase is extracted,

(ii) a confidence measuring an accuracy of the recognized specific text string,

(iii) a prevalence of the specific phrase in a language of the specific phrase,

(iv) a prevalence of the specific phrase in the geographical area corresponding to the specific geo-tagged image, and

(v) a distance between the geographical area corresponding to the specific geo-tagged image and a location of the specific establishment.

4. The method of claim 1, wherein each of the one or more matches comprises an approximate match between a specific text string of the one or more text strings and a specific phrase of the extracted one or more phrases.

5. The method of claim 1, wherein the information associated with the specific establishment comprises at least one of (i) a name of the specific establishment, (ii) a category of the specific establishment, (iii) a phone number of the specific establishment, and (iv) an address of the specific establishment.

6. The method of claim 1, wherein comparing the one or more text strings recognized in the plurality of geo-tagged images with the extracted one or more phrases includes filtering the extracted one or more phrases.

7. The method of claim 1, wherein the identifying one or more establishments associated with the geographical locations of the plurality of geo-tagged images includes identifying establishments within a radius of the geographical location of each of the plurality of geo-tagged images.

8. The method of claim 7, wherein the radius of a specific geo-tagged image is based on an accuracy of the geographical location of the specific geo-tagged image.

9. The method of claim 1, further comprising recognizing one or more logos in at least one of the plurality of geo-tagged images, wherein selecting the representative geo-tagged image for the specific establishment from among the plurality of geo-tagged images is further based on comparing the one or more logos with a logo corresponding to the specific establishment.

10. A computer system, comprising:

a non-transitory computer-readable storage medium comprising executable computer program code for;

recognizing, with one or more computing devices, one or more text strings in each of a plurality of geo-tagged images, each of the plurality of geo-tagged images indicating a geographical location associated with its corresponding geo-tagged image;

identifying, with the one or more computing devices, one or more establishments associated with the geographical locations of the plurality of geo-tagged images;

for at least one specific establishment of the one or more establishments;

(i) extracting, with the one or more computing devices, one or more phrases from information associated with the specific establishment;

(ii) comparing, with the one or more computing devices, the one or more text strings recognized in the plurality of geo-tagged images with the extracted one or more phrases to derive one or more matches; and

(iii) selecting, with the one or more computing devices, a representative geo-tagged image for the specific establishment from among the plurality of geo-tagged images based at least in part on one or more matches of a specific geo-tagged image with the specific establishment; and

at least one processor for executing the executable computer program code.

11. The computer system of claim 10, selecting a representative geo-tagged image comprises scoring each of one or more image-establishment pairs to obtain a corresponding score, wherein selecting the representative geo-tagged image for the specific establishment is further based on the corresponding scores.

12. The computer system of claim 11, wherein:

each of the one or more matches comprises an approximate match between a specific text string of the one or more text strings and a specific phrase of the extracted one or more phrases, and

scoring each of the one or more image-establishment pairs comprises assigning a score to a specific image-establishment pair based on one or more of the following factors;

(i) an information field from which the specific phrase is extracted,

(ii) a confidence measuring an accuracy of the recognized specific text string,

(iii) a prevalence of the specific phrase in a language of the specific phrase,

(iv) a prevalence of the specific phrase in the geographical area corresponding to the specific geo-tagged image, and

(v) a distance between the geographical area corresponding to the specific geo-tagged image and a location of the specific establishment.

13. The computer system of claim 10, wherein each of the one or more matches comprises an approximate match between a specific text string of the one or more text strings and a specific phrase of the extracted one or more phrases.

14. The computer system of claim 10, wherein the information associated with the specific establishment comprises at least one of (i) a name of the specific establishment, (ii) a category of the specific establishment, (iii) a phone number of the specific establishment, and (iv) an address of the specific establishment.

15. The computer system of claim 10, wherein comparing the one or more text strings recognized in the plurality of geo-tagged images with the extracted one or more phrases includes filtering the extracted one or more phrases.

16. The computer system of claim 10, wherein the identifying one or more establishments near the geographical locations of the plurality of geo-tagged images includes identifying establishments within a radius of the geographical location of each of the plurality of geo-tagged images.

17. The computer system of claim 16, wherein the radius of a specific geo-tagged image is based on an accuracy of the geographical location of the specific geo-tagged image.

18. The computer system of claim 10, wherein the non-transitory computer-readable storage medium further comprises executable computer program code for recognizing one or more logos in at least one of the plurality of geo-tagged images, wherein selecting the representative geo-tagged image for the specific establishment from among the plurality of geo-tagged images is further based on comparing the one or more logos with a logo corresponding to the specific establishment.

19. A non-transitory computer-readable storage medium storing executable computer program instructions for:

recognizing, with one or more computing devices, one or more text strings in each of a plurality of geo-tagged images, each of the plurality of geo-tagged images indicating a geographical location associated with its corresponding geo-tagged image;

identifying, with the one or more computing devices, one or more establishments associated with the geographical locations of the plurality of geo-tagged images;

for at least one specific establishment of the one or more establishments;

(i) extracting, with the one or more computing devices, one or more phrases from information associated with the specific establishment;

(ii) comparing, with the one or more computing devices, the one or more text strings recognized in the plurality of geo-tagged images with the extracted one or more phrases to derive one or more matches; and

(iii) selecting, with the one or more computing devices, a representative geo-tagged image for the specific establishment from among the plurality of geo-tagged images based at least in part on one or more matches of a specific geo-tagged image with the specific establishment.

20. The storage medium of claim 19, wherein:

each of the one or more matches comprises an approximate match between a specific text string of the one or more text strings and a specific phrase of the extracted one or more phrases, and

scoring each of the one or more image-establishment pairs comprises assigning a score to a specific image-establishment pair based on one or more of the following factors;

(i) an information field from which the specific phrase is extracted,

(ii) a confidence measuring an accuracy of the recognized specific text string,

(iii) a prevalence of the specific phrase in a language of the specific phrase,

(iv) a prevalence of the specific phrase in the geographical area corresponding to the specific geo-tagged image, and

(v) a distance between the geographical area corresponding to the specific geo-tagged image and a location of the specific establishment.