Systems and methods for mobile image capture and content processing

• US 9,886,628 B2
• Filed: 07/17/2017
• Issued: 02/06/2018
• Est. Priority Date: 01/18/2008
• Status: Active Grant

• Current Assignee: Mitek Systems Incorporated
• Original Assignee: Mitek Systems Incorporated
• Inventors: Nepomniachtchi, Grigori, Strange, Mike
• Primary Examiner(s): Motsinger, Sean
• Application Number:
  US15/652,126
• Publication Number:
  US 20170316263A1
• Time in Patent Office: 204 Days
• CPC Class Codes: G06Q 20/3276 using a pictured code, e.g. barcode or QR-code, being read by the M-device
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/1475 Inclination or skew detection or correction of characters or of image to be recognised
• CPC Class Codes: G06V 30/162 Quantising the image signal
• CPC Class Codes: G06V 30/166 Normalisation of pattern dimensions
• CPC Class Codes: G07C 2209/41 with means for the generation of identity documents
• CPC Class Codes: G07C 9/00 Individual registration on entry or exit
• CPC Class Codes: H04N 1/00244 with a server, e.g. an internet server fax-servers or the like for store and forward H04N1/324
• CPC Class Codes: H04N 1/00307 with a mobile telephone apparatus constructional features of portable telephone sets H04M1/0202; mobile radio systems H04W84/00
• CPC Class Codes: H04N 1/00708 Size or dimensions
• CPC Class Codes: H04N 1/00718 Skew
• CPC Class Codes: H04N 1/00734 Optical detectors
• CPC Class Codes: H04N 2101/00 Still video cameras
• CPC Class Codes: H04N 2201/001 Sharing resources, e.g. processing power or memory, with a connected apparatus or enhancing the capability of the still picture apparatus
• CPC Class Codes: H04N 2201/0084 Digital still camera

Abstract

Systems and methods are provided for processing and extracting content from an image captured using a mobile device. In one embodiment, an image is captured by a mobile device and corrected to improve the quality of the image. The corrected image is then further processed by adjusting the image, identifying the format and layout of the DL, binarizing the image and extracting the content using optical character recognition (OCR). Multiple methods of image adjusting may be implemented to accurately assess features of the DL, and a secondary layout identification process may be performed to ensure that the content being extracted is properly classified.

Claims

1. A computer-implemented method of processing a document image, comprising:

receiving a document image captured by a mobile device;

performing a shape detection test on the document image to produce a first set of coordinates representing a set of edges of the document image;

performing a feature detection test on the document image to produce a second set of coordinates associated with a feature set, wherein the feature detection test comprise a feature matching operation between the feature set of the document image and a document image template;

combining the first set of coordinates and the second set of coordinates to generate a set of adjustment parameters associated with the document image;

cropping the document image based at least on the adjustment parameters;

binarizing the cropped image to produce a binarized cropped image; and

outputting the binarized cropped image.

2. The computer-implemented method of claim 1, wherein prior to performing the feature detection test on the document image, the method further comprises pre-filtering the document image using a low-pass noise suppression filter.

3. The computer-implemented method of claim 1, wherein the adjustment parameters includes both coordinates associated with the feature set of the document image and coordinates representing the set of edges of the document image.

4. The computer-implemented method of claim 1, wherein the feature detection test detects a set of feature points in the document image by detecting a set of local maxima in the document image.

5. The computer-implemented method of claim 4, wherein the feature detection test constructs the feature set by computing local gradients and color distributions in the areas of the set of feature points.

6. The computer-implemented method of claim 1, wherein the shape detection test includes a rectangular shape detection test.

7. The computer-implemented method of claim 1, wherein prior to combining the first set of coordinates and the second set of coordinates to generate the adjustment parameters, a rounded corner detection test on the document image to produce a third set of coordinates associated with the feature set of the document image.

8. The computer-implemented method of claim 7, wherein generating the set of adjustment parameters further includes combining the first set of coordinates and the second set of coordinates with the third set of coordinates.

9. The computer-implemented method of claim 1, wherein performing the feature detection test on the document image includes:

detecting a set of feature points in the document image;

constructing a set of feature descriptors based on the set of feature points;

wherein the feature matching operation between the set of feature descriptors and a set of predetermined features of the document template.

10. The computer-implemented method of claim 9, wherein detecting the set of feature points in the document image includes detecting a set of local maxima in the document image.

11. The computer-implemented method of claim 9, wherein constructing the set of feature descriptors based on the set of feature points includes computing local gradients and color distributions in the areas of the set of feature points.

12. The computer-implemented method of claim 1, wherein the outputs of the feature matching operation include a transformation matrix containing a set of matched feature points.

13. A system of processing a document image captured by a mobile device, comprising:

a processor;

a memory coupled to the processor;

a receiving unit which receives a document image captured by a mobile device;

a shape detection unit which performs a shape detection test on the document image to produce a first set of coordinates representing a set of edges of the document image;

a feature detection unit which performs a feature detection test on the document image to produce a second set of coordinates associated with the document image, wherein the feature detection test comprise a feature matching operation between the feature set of the document image and a document image template;

an adjustment parameter generation unit which combines the first set of coordinates and the second set of coordinates to generate a set of adjustment parameters;

an adjusting unit which crops the document image based at least on the adjustment parameters to remove a portion of the document image to obtain a cropped image;

a binarization unit which binarizes the cropped image to produce a binarized cropped image; and

an output unit which outputs the binarized cropped image.

14. The system of claim 13, further comprising a pre-filtering unit which pre-filters the document image using a low-pass filter for noise suppression prior to performing the feature detection test on the document image.

15. The system of claim 13, wherein the adjustment parameter generation unit is further configured to select either the first set of coordinates or the second set of coordinates which is determined to be more accurately associated with borders of the document image.

16. The system of claim 13, wherein the adjustment parameter generation unit is further configured to combine the first set of coordinates and the second set of coordinates with known information of a size and an aspect ratio of the document image.

17. The system of claim 13, wherein a rounded corner detection test on the document image is to produce a third set of coordinates associated with the feature set of the document image, and wherein the adjustment parameter generation unit is further configured to combine the first set of coordinates and the second set of coordinates with the third set of coordinates.

18. The system of claim 13, wherein the feature detection unit is configured to perform the feature detection test on the document image by:

detecting a set of feature points in the document image;

constructing a set of feature descriptors based on the set of feature points;

wherein the feature matching operation between the set of feature descriptors and a set of predetermined features of the document template.

19. The system of claim 18, wherein the feature detection unit is configured to detect the set of feature points in the document image by detecting a set of local maxima in the document image.

20. The system of claim 18, wherein the feature detection unit is configured to construct the set of feature descriptors by computing local gradients and color distributions in the areas of the set of feature points.