Database assisted OCR for street scenes and other images

• US 20070140595A1
• Filed: 12/16/2005
• Published: 06/21/2007
• Est. Priority Date: 12/16/2005
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Vincent, Luc, Taylor, Bret
• Granted Patent: US 7,917,286 B2
• Time in Patent Office: Days
• US Class Current: 382/310
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/262 using context analysis, e.g. lexical, syntactic or semantic context

Abstract

Optical character recognition (OCR) for images such as a street scene image is generally a difficult problem because of the variety of fonts, styles, colors, sizes, orientations, occlusions and partial occlusions that can be observed in the textual content of such scenes. However, a database query can provide useful information that can assist the OCR process. For instance, a query to a digital mapping database can provide information such as one or more businesses in a vicinity, the street name, and a range of possible addresses. In accordance with an embodiment of the present invention, this mapping information is used as prior information or constraints for an OCR engine that is interpreting the corresponding street scene image, resulting in much greater accuracy of the digital map data provided to the user.

Claims

1. A method for assisting optical character recognition (OCR) of a street scene image using a mapping system database, the method comprising:

determining a target GPS location for a street scene image using known GPS data associated with that street scene image;

estimating a street address of the target GPS location;

identifying a target address range based on the street address of the target GPS location;

querying a mapping system database to identify a business name having a street address in the target address range;

performing OCR of the street scene image to determine if key words associated with the identified business name are present; and

in response to determining that at least one key word associated with the identified business name is present, determining an actual GPS location for the street address of that business name, based on the known GPS data.

2. The method of claim 1 further comprising:

updating the mapping system database to include the actual GPS location.

3. The method of claim 1 further comprising:

repeating the method for a number of additional target GPS locations.

4. The method of claim 1 wherein performing OCR of the street scene image to determine if key words associated with the identified business name are present further includes performing image analysis of the street scene image to determine if expected features associated with the identified business name are present.

5. The method of claim 1 wherein the street scene image includes one street address.

6. The method of claim 1 wherein the street scene image is a panoramic image that includes a plurality of street addresses.

7. The method of claim 1 wherein the known GPS data includes known GPS locations for at least two locations captured in the street scene image.

8. The method of claim 1 wherein the mapping system database includes a business listings directory.

9. A method for assisting optical character recognition (OCR) of a street scene image using a database, the method comprising:

querying a database to identify a feature expected in a street scene image of one or more street addresses, the street scene image associated with known GPS data;

performing OCR of the street scene image to determine if the expected feature is present in the street scene image; and

in response to determining that the expected feature is present, determining an actual GPS location for a street address associated with that expected feature.

10. The method of claim 9 further comprising:

updating the database to include the actual GPS location.

11. The method of claim 9 wherein querying the database identifies a number of textual and non-textual features.

12. The method of claim 11 further comprising:

performing image analysis of the street scene image to determine if non-textual expected features are present.

13. The method of claim 9 wherein the street scene image includes one street address.

14. The method of claim 9 wherein the street scene image is a panoramic image that includes a plurality of street addresses.

15. The method of claim 9 wherein the known GPS data includes known GPS locations for at least two locations captured in the street scene image.

16. A method for assisting optical character recognition (OCR) of an image using a database, the method comprising:

querying a database to identify at least one keyword corresponding to text expected to be in an image; and

performing OCR of the image to determine if the keyword is present in the image.

17. The method of claim 16 wherein the keyword is further associated with a street address or a key event.

18. The method of claim 16 wherein querying the database identifies a number of textual and non-textual expected features.

19. The method of claim 18 further comprising:

performing image analysis of the image to determine if non-textual expected features are present.

20. The method of claim 16 wherein the image is one of a photograph or video frame.

21. The method of claim 16 wherein the image is associated with known GPS location data, and in response to determining that the keyword is present, the method further comprises determining an actual GPS location associated with that keyword.