Recognizing text in image data

• US 10,095,925 B1
• Filed: 12/18/2017
• Issued: 10/09/2018
• Est. Priority Date: 12/18/2017
• Status: Active Grant

• Current Assignee: Capital One Services LLC (Capital One Financial Corporation)
• Original Assignee: Capital One Services LLC (Capital One Financial Corporation)
• Inventors: Tripuraneni, Subhashini, Barco, Jr., Joseph R.
• Primary Examiner(s): Allison, Andrae S
• Application Number:
  US15/845,623
• Time in Patent Office: 295 Days
• Field of Search: None
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/12 Detection or correction of errors, e.g. by rescanning the pattern
• CPC Class Codes: G06V 30/1473 Recognising objects as potential recognition candidates based on visual cues, e.g. shapes
• CPC Class Codes: G06V 30/224 of printed characters having additional code marks or containing code marks
• CPC Class Codes: G06V 30/413 Classification of content, e.g. text, photographs or tables
• CPC Class Codes: G06V 30/414 Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text
• CPC Class Codes: G06V 30/416 Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors

Abstract

A device may receive image data representing a document, the document including: text, and edges. Based on the edges, the device may identify, a segment of interest within the image data and crop the segment of interest to obtain a portion of the image data. In addition, the device may perform optical character recognition on the portion of the image data, the optical character recognition producing recognized text. The device may obtain, based on the recognized text, validation data that includes verification text, and determine whether the recognized text is verified based on the verification text. Based on a result of the determination, the device may perform an action.

Claims

1. A device, comprising:

one or more memory devices; and

one or more processors, communicatively connected to the one or more memory devices, to;

receive image data representing a document,

the document including;

text, and

a plurality of edges;

identify, based on the plurality of edges, a segment of interest within the image data;

crop the segment of interest to obtain a portion of the image data;

perform optical character recognition on the portion of the image data,

the optical character recognition producing recognized text;

obtain, based on the recognized text, validation data,

the validation data including verification text;

determine whether the recognized text is verified based on the verification text,

the recognized text including a recognized account identifier,

the recognized account identifier being compared to a corresponding account from a database,

information from the corresponding account being compared to other information recognized in the image data to determine whether the recognized account identifier is accurate; and

perform an action based on a result of the determination.

2. The device of claim 1, where the one or more processors are further to:

receive input indicating a document type associated with the image data, and

where the one or more processors, when identifying the segment of interest within the image data, are to;

identify the segment of interest based on the document type.

3. The device of claim 1, where the one or more processors are further to:

identify, based on one or more features included in the image data, a document type associated with the image data, and

where the one or more processors, when identifying the segment of interest within the image data, are to;

identify the segment of interest based on the document type.

4. The device of claim 1, where the one or more processors are further to:

receive input indicating text of interest, and

where the one or more processors, when identifying the segment of interest within the image data, are to;

identify the segment of interest based on the text of interest.

5. The device of claim 1, where the one or more processors are further to:

determine a document type associated with the image data; and

obtain, based on the document type, data identifying a plurality of segments associated with the document type,

where the one or more processors, when identifying the segment of interest within the image data, are to identify the segment of interest from one of the plurality of segments.

6. The device of claim 5, where the one or more processors are further to:

receive input indicating text of interest, and

where the one or more processors, when identifying the segment of interest from one of the plurality of segments, are to;

identify the segment of interest based on the text of interest and data indicating that the segment of interest includes the text of interest.

7. The device of claim 1, where the one or more processors are further to:

classify the segment of interest based on at least one feature of the segment of interest, and where

the one or more processors, when performing optical character recognition on the portion of the image data, are to;

perform the optical character recognition based on a classification associated with the segment of interest.

8. The device of claim 1, where the one or more processors are further to:

preprocess the image data by performing at least one of;

a contrast adjustment operation;

a skew correction operation;

or

a noise reduction operation.

9. A non-transitory computer-readable medium storing instructions, the instructions comprising:

one or more instructions that, when executed by one or more processors, cause the one or more processors to;

receive image data representing a document,

the document including;

text, and

a plurality of edges;

identify, based on the plurality of edges, a plurality of segments within the image data;

identify, from the plurality of segments, a segment of interest based on a location of the segment of interest within the image data;

crop the segment of interest to obtain a portion of the image data;

perform optical character recognition on the portion of the image data,

the optical character recognition producing recognized text;

obtain, based on the recognized text, validation data,

the validation data including verification text;

determine whether the recognized text is verified based on the verification text,

the recognized text including a recognized account identifier,

the recognized account identifier being compared to a corresponding account from a database,

information from the corresponding account being compared to other information recognized in the image data to determine whether the recognized account identifier is accurate; and

perform an action based on the recognized text.

10. The non-transitory computer-readable medium of claim 9, where the one or more instructions, that cause the one or more processors to perform the action based on the recognized text, cause the one or more processors to:

identify, based on the recognized text, at least one data structure to be updated; and

update the at least one data structure using the recognized text.

11. The non-transitory computer-readable medium of claim 9, where the one or more instructions, that cause the one or more processors to perform the action based on the recognized text, cause the one or more processors to:

provide the recognized text as output; and

receive, based on the output, user input indicating the validation data.

12. The non-transitory computer-readable medium of claim 9, where the one or more instructions, that cause the one or more processors to perform the action based on the recognized text, cause the one or more processors to:

cause an optical character recognition model that was used to perform optical character recognition on the segment of interest to be retrained based on the recognized text.

13. The non-transitory computer-readable medium of claim 9, where the one or more instructions, when executed by the one or more processors, further cause the one or more processors to:

determine a document type associated with the image data; and

obtain, based on the document type, data identifying a classification for each of the plurality of segments.

14. The non-transitory computer-readable medium of claim 13, where the one or more instructions, that cause the one or more processors to identify, from the plurality of segments, the segment of interest based on the location of the segment of interest within the image data, further cause the one or more processors to:

identify the segment of interest based on the classification for the segment of interest.

15. The non-transitory computer-readable medium of claim 9, where the validation data is obtained from at least one of:

an address database,

an application, or

a webpage.

16. A method, comprising:

receiving, by a device, image data representing a document,

the document including;

text, and

a plurality of edges;

receiving, by the device, data identifying text of interest associated with the image data;

identifying, by the device and based on the plurality of edges, a plurality of segments within the image data;

identifying, by the device and from the plurality of segments, a segment of interest,

the segment of interest including the text of interest;

cropping, by the device, the segment of interest to obtain a portion of the image data;

performing, by the device, optical character recognition on the portion of the image data,

the optical character recognition producing recognized text,

the recognized text including the text of interest;

obtaining, by the device, based on the recognized text, validation data,

the validation data including verification text;

determining, by the device, whether the recognized text is verified based on the verification text,

the recognized text including a recognized account identifier,

the recognized account identifier being compared to a corresponding account from a database,

information from the corresponding account being compared to other information recognized in the image data to determine whether the recognized account identifier is accurate; and

performing, by the device, an action based on the recognized text.

17. The method of claim 16, where:

the segment of interest is defined by at least a portion of the plurality of edges; and

cropping the segment of interest to obtain a portion of the image data comprises;

cropping the portion of the image data in a manner designed to exclude edges that define the segment of interest.

18. The method of claim 16, further comprising:

identifying a document type associated with the image data; and

classifying the segment of interest based on the document type.

19. The method of claim 18, where classifying the segment of interest based on the document type comprises:

determining that a first location of the segment of interest matches a second location of a document segment identified in data associated with the document type; and

classifying, based on determining that the first location matches the second location, the segment of interest as being in a first class that matches a second class associated with the document segment.

20. The method of claim 18, where classifying the segment of interest based on the document type comprises:

determining that a first portion of the recognized text matches a second portion of document text identified in data associated with the document type; and

classifying, based on determining that the first portion matches the second portion, the segment of interest as being in a first class that matches a second class associated with the document text.