LABEL-EMBEDDING FOR TEXT RECOGNITION

• US 20140219563A1
• Filed: 02/01/2013
• Published: 08/07/2014
• Est. Priority Date: 02/01/2013
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Perronnin, Florent C., Rodriguez-Serrano, Jose Antonio
• Granted Patent: US 9,008,429 B2
• Time in Patent Office: Days
• US Class Current: 382/182
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06V 20/625 License plates
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names
• CPC Class Codes: G06V 30/19173 Classification techniques
• CPC Class Codes: G06V 30/224 of printed characters having additional code marks or containing code marks

Abstract

A system and method for comparing a text image and a character string are provided. The method includes embedding a character string into a vectorial space by extracting a set of features from the character string and generating a character string representation based on the extracted features, such as a spatial pyramid bag of characters (SPBOC) representation. A text image is embedded into a vectorial space by extracting a set of features from the text image and generating a text image representation based on the text image extracted features. A compatibility between the text image representation and the character string representation is computed, which includes computing a function of the text image representation and character string representation.

Claims

1. A method for comparing a text image and a character string comprising:

embedding a character string into a vectorial space, comprising extracting a set of features from the character string and generating a character string representation based on the extracted character string features;

embedding a text image into a vectorial space, comprising extracting a set of features from the text image and generating a text image representation based on the extracted text image features; and

computing a compatibility between the text image representation and character string representation comprising computing a function of the text image representation and character string representation,

wherein at least one of the embedding and the computing of the compatibility is performed with a processor.

2. The method of claim 1, wherein the function of the text image representation and character string representation is in a bilinear form.

3. The method of claim 2 wherein the function is of the form:

F(x,y;

W)=θ

(x)^TWφ

(y) 

 

(5)

where F represents the compatibility between an image and a character string, given a matrix of weights W,

θ

(x) represents one of the text image representation and the character string representation and φ

(y) represents the other of the text image representation and the character string representation, and T represents the transpose operator,

or where an approximation W≈

U^TV is used, with Uε

<sup>R×

D</sup>, Vε

<sup>R×

E</sup> where R<

D, is a function of the form;

F(x,y;

W)=(Uθ

(x))^T(Vφ

(y)). 

 

(6)

4. The method of claim 1, wherein the computing of the compatibility comprises determining whether the computed function meets a predetermined threshold.

5. The method of claim 1, wherein the character string comprises a sequence of characters, each character in the sequence being drawn from a finite set of characters.

6. The method of claim 1, wherein the character string is in a standard format selected from ASCII, UNICODE, and UTF-8.

7. The method of claim 1, wherein the character string comprises a license plate number.

8. The method of claim 1, wherein the text image comprises a captured image of a license plate.

9. The method of claim 1, wherein the character string embedding comprises extraction of a spatial pyramid bag-of-characters.

10. The method of claim 9, wherein the extraction of a spatial pyramid bag-of-characters comprises partitioning the character string into a plurality of regions and extracting a representation of each of the regions, the character string representation being derived from the region representations.

11. The method of claim 10, wherein the partitioning of the character string into a plurality of regions and extracting a representation of each of the regions is repeated at least once, wherein in each of the repeats, the partitioning includes partitioning one of the previously-generated regions.

12. The method of claim 10, wherein the partition generates a spatial pyramid of regions which includes at least three levels, each of at least two of the levels including a plurality of regions.

13. The method of claim 12, wherein the spatial pyramid includes at least seven regions.

14. The method of claim 1, wherein parameters of the compatibility function have been learned through optimization of a ranking objective function.

15. The method of claim 1, further comprising learning parameters of the compatibility function using a training set of text images that are each labeled with a respective character string corresponding to text present in the image.

16. The method of claim 1, wherein the text image embedding comprises extraction and aggregation of local descriptors using Fisher vectors.

17. The method of claim 1, further comprising outputting information based on the computed compatibility between the text image representation and the character string representation.

18. The method of claim 1, wherein the character string comprises a vocabulary of character strings, and the method comprises outputting a character string from the vocabulary of character strings having a character string representation with a highest computed compatibility with the text image representation.

19. The method of claim 1, wherein the text image comprises a collection of text images and the method comprises outputting a text image from the collection of text images having a text image representation with a highest computed compatibility with the text image representation.

20. A computer program product comprising a non-transitory recording medium storing instructions, which when executed on a computer, causes the computer to perform the method of claim 1.

21. A system comprising memory which stores instructions for performing the method of claim 1 and a processor in communication with the memory for executing the instructions.

22. A system for comparing a text image and a character string comprising:

a text string representation generator for generating a character string representation based on features extracted from a character string, the character string consisting of a sequence of characters, the text string representation comprising a spatial pyramid bag of characters representation;

a text image representation generator for generating a text image representation based on features extracted from a text image;

a comparator for computing a compatibility between the text image representation and the character string representation;

an output component for outputting information based on the computed compatibility between at least one character string representation and at least one text image representation; and

a processor which implements the text string representation generator, text image representation generator;

comparator, and output component.

23. A method for comparing a text image and a character string comprising:

for at least one character string comprising a sequence of characters, extracting a set of features from the character string, comprising partitioning the character string to form a spatial pyramid of regions and for each region, generating a representation of the region comprising counting occurrences of each of a finite set of characters in the region;

generating a character string representation based on the region representations;

extracting a set of features from the text image and generating a text image representation based thereon; and

computing a compatibility between the text image representation and the character string representation comprising embedding at least one of the character string representation and the text image representation with a matrix of learned parameters, the compatibility being a function of the at least one embedded representation,

wherein at least one of the embedding and the computing of the compatibility is performed with a processor.