USING EXTRACTED IMAGE TEXT

• US 20130039570A1
• Filed: 09/15/2012
• Published: 02/14/2013
• Est. Priority Date: 06/29/2006
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Vincent, Luc, Ulges, Adrian
• Granted Patent: US 8,744,173 B2
• Time in Patent Office: Days
• US Class Current: 382/159
• CPC Class Codes: G01S 19/51 Relative positioning
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names
• CPC Class Codes: G06V 30/10 Character recognition

Abstract

Methods, systems, and apparatus including computer program products for using extracted image text are provided. In one implementation, a computer-implemented method is provided. The method includes receiving an input of one or more image search terms and identifying keywords from the received one or more image search terms. The method also includes searching a collection of keywords including keywords extracted from image text, retrieving an image associated with extracted image text corresponding to one or more of the image search terms, and presenting the image.

Claims

1. A computer-implemented method comprising:

dividing an image into a plurality of sub-regions;

identifying two or more adjacent sub-regions of the image that contain text, wherein the identified adjacent sub-regions share overlapping image pixels;

combining the identified adjacent sub-regions into a candidate text region; and

performing optical character recognition on the candidate text region.

2. The method of claim 1 wherein identifying the adjacent sub-regions that contain text comprises:

extracting one or more features from each sub-region; and

providing the extracted features to a trained classifier to determine whether each sub-region contains text.

3. The method of claim 2, wherein the classifier has been trained on images of city street scenes, and wherein each image of a city street scene includes identified regions of text and regions of non-text.

4. The method of claim 2, further comprising:

scaling the image to multiple sizes prior to extracting the one or more features; and

determining that the sub-region contains text at two or more sizes.

5. The method of claim 2, wherein extracting features from each sub-region includes detecting corner features within the sub-region.

6. The method of claim 2, wherein extracting features from each sub-region includes computing projection profiles in each sub-region.

7. The method of claim 1, further comprising:

determining a minimum size for text in candidate text regions;

determining that a different second candidate text region contains text that is smaller than the minimum size; and

bypassing optical character recognition for the second candidate text region.

8. The method of claim 1, further comprising:

receiving ranging data associated with a different second image;

generating a planar map of the second image using the ranging data;

determining that a second candidate text region of the second image is not located on a single plane; and

bypassing optical character recognition for the second candidate text region.

9. The method of claim 8, wherein the ranging data comprises a distance from a camera position when the second image was taken to an object shown in the second image.

10. The method of claim 8, wherein generating a planar map comprises decomposing the image into planar and non-planar regions, and wherein determining that the second candidate text region is not located on a single plane is based on determining that second candidate text region corresponds to a non-planar region.

11. The method of claim 1, further comprising:

correcting perspective distortion in the image.

12. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

dividing an image into a plurality of sub-regions;

identifying two or more adjacent sub-regions of the image that contain text, wherein the identified adjacent sub-regions share overlapping image pixels;

combining the identified adjacent sub-regions into a candidate text region; and

performing optical character recognition on the candidate text region.

13. The system of claim 12 wherein identifying the adjacent sub-regions that contain text comprises:

extracting one or more features from each sub-region; and

providing the extracted features to a trained classifier to determine whether each sub-region contains text.

14. The system of claim 13, wherein the classifier has been trained on images of city street scenes, and wherein each image of a city street scene includes identified regions of text and regions of non-text.

15. The system of claim 13, wherein the operations further comprise:

scaling the image to multiple sizes prior to extracting the one or more features; and

determining that the sub-region contains text at two or more sizes.

16. The system of claim 13, wherein extracting features from each sub-region includes detecting corner features within the sub-region.

17. The system of claim 13, wherein extracting features from each sub-region includes computing projection profiles in each sub-region.

18. The system of claim 12, wherein the operations further comprise:

determining a minimum size for text in candidate text regions;

determining that a different second candidate text region contains text that is smaller than the minimum size; and

bypassing optical character recognition for the second candidate text region.

19. The system of claim 12, wherein the operations further comprise:

receiving ranging data associated with a different second image;

generating a planar map of the second image using the ranging data;

determining that a second candidate text region of the second image is not located on a single plane; and

bypassing optical character recognition for the second candidate text region.

20. The system of claim 19, wherein generating a planar map comprises decomposing the image into planar and non-planar regions, and wherein determining that the second candidate text region is not located on a single plane is based on determining that second candidate text region corresponds to a non-planar region.