SYSTEM AND METHOD FOR OCR OUTPUT VERIFICATION

• US 20140355835A1
• Filed: 05/28/2013
• Published: 12/04/2014
• Est. Priority Date: 05/28/2013
• Status: Active Grant

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Rodriguez-Serrano, Jose Antonio, Perronnin, Florent C.
• Granted Patent: US 9,384,423 B2
• Time in Patent Office: Days
• US Class Current: 382/105
• CPC Class Codes: G06F 18/213 Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
• CPC Class Codes: G06V 10/7715 Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
• CPC Class Codes: G06V 20/625 License plates
• CPC Class Codes: G06V 20/63 Scene text, e.g. street names
• CPC Class Codes: G06V 30/19127 Extracting features by transforming the feature space, e.g. multidimensional scaling; Mappings, e.g. subspace methods
• CPC Class Codes: G06V 30/2272 with lexical matching

Abstract

A system and method for computing confidence in an output of a text recognition system includes performing character recognition on an input text image with a text recognition system to generate a candidate string of characters. A first representation is generated, based on the candidate string of characters, and a second representation is generated based on the input text image. A confidence in the candidate string of characters is computed based on a computed similarity between the first and second representations in a common embedding space.

Claims

1. A method for computing confidence comprising:

with a text recognition system, performing character recognition on an input text image to generate a candidate string of characters;

generating a first representation based on the candidate string of characters;

generating a second representation based on the input text image;

computing a confidence in the candidate string of characters based on a computed similarity between the first and second representations in a common embedding space, at least one of the first and second representations being projected into the common embedding space; and

wherein at least one of the performing character recognition, generating the first representation, generating the second representation, and computing the confidence is performed with a computer processor.

2. The method of claim 1, wherein the computed similarity is computed as the dot product of the projected first and second representations.

3. The method of claim 1, wherein the first and second representations comprise multidimensional feature vectors.

4. The method of claim 1, wherein the generating of the first representation comprises embedding the character string into a vectorial space, the embedding comprising extracting a set of features from the character string and generating a character string representation based on the extracted character string features.

5. The method of claim 4, where the embedding comprises generating a spatial pyramid bag of characters, the generating comprising partitioning the character string into a plurality of regions and extracting a representation of each of the regions, the character string representation being derived from the region representations.

6. The method of claim 1, wherein generating of the first representation comprises synthesizing a synthetic image of the character string and computing a representation based on the synthetic image.

7. The method of claim 6, where the synthesizing of the synthetic image comprises at least one of:

representing characters of the character string in a font identified from the input image in the performed character recognition; and

.

representing non-textual information extracted from the image.

8. The method of claim 1 wherein the generating of the second representation comprises generating a statistical representation of features extracted from patches of the image using a generative model.

9. The method of claim 1, further comprising learning a projection for projecting the at least one of the first and second representations into the common space, the projection being learned with a structured output prediction algorithm.

10. The method of claim 1, further comprising outputting information based on the computed confidence.

11. The method of claim 10, wherein the outputting of information comprises outputting the candidate string of characters when the computed confidence meets a predetermined confidence threshold.

12. The method of claim 1, wherein the performing character recognition on the input image comprises, with the text recognition system, generating a plurality of candidate strings of characters for the same input text image, the method further comprising ranking the plurality of candidate strings based on the respective computed confidences.

13. The method of claim 12, wherein the generating of the plurality of candidate strings of characters comprises identifying candidate strings based on associated confidence scores output by the text recognition system.

14. The method of claim 1, further comprising comparing the computed confidence to a threshold and when the confidence does not meet the threshold, initiating a further review of the input image.

15. The method of claim 14, wherein the further review comprises at least one of:

performing character recognition on the input text image with a second text recognition system,

initiating a manual review of the text image; and

computing a similarity between the input text image and a set of stored text images, each annotated with a corresponding character string.

16. The method of claim 1, further comprising computing the similarity between the first and second representations using a function of the form:

c(t,I)=x(I)^T·

W·

e(t) 

 

(1)

where x(I) represents one of the text image representation and the character string representation, e(t) represents the other of the text image representation and the character string representation, T represents the transpose operator, and W represents a matrix of weights.

17. The method of claim 1, wherein the character string comprises a sequence of characters, each character in the sequence being drawn from a finite set of characters.

18. The method of claim 1, wherein the character string comprises a license plate number and the text image comprises a captured image of a license plate.

19. A computer program product comprising a non-transitory recording medium storing instructions, which when executed on a computer, causes the computer to perform the method of claim 1.

20. A system comprising memory which stores instructions for performing the method of claim 1 and a processor in communication with the memory for executing the instructions.

21. A system for computing confidence comprising:

a text image representation component which generates a text image representation based on features extracted from a text image;

a text string representation component which receives a character string from an associated text recognition system as a candidate string for the text image and generates a character string representation based on features extracted from the character string;

a comparison component which computes a confidence in the character string based on a similarity between the text image representation and the character string representation in a learned embedding space;

an information output component which outputs information based on the computed confidence; and

a processor which implements the text image representation component, text string representation component, comparison component, and information output component.

22. A license plate recognition method comprising:

performing optical character recognition on an image of a license plate to generate a character string as a candidate license plate number;

computing a confidence in the candidate license plate number based on a function which embeds a representation of the license plate image and a representation of the candidate license plate number in an embedding space with a matrix of weights learned on an annotated set of license plate images; and

where the confidence meets a threshold, outputting the character string as a recognized license plate number for the image of the license plate, otherwise initiating another process for license plate number recognition;

wherein at least one of the performing, computing and outputting is performed with a processor.