Methods for mobile image capture and processing of documents

• US 7,953,268 B2
• Filed: 12/30/2008
• Issued: 05/31/2011
• Est. Priority Date: 01/18/2008
• Status: Active Grant

• Current Assignee: Mitek Systems Incorporated
• Original Assignee: Mitek Systems Incorporated
• Inventors: Nepomniachtchi, Grigori
• Primary Examiner(s): Mehta; Bhavesh M
• Assistant Examiner(s): MOTSINGER, SEAN T
• Application Number:
  US12/346,071
• Publication Number:
  US 20090185736A1
• Time in Patent Office: 882 Days
• Field of Search: 382137-140, 705 39- 45, 235/379
• US Class Current: 382/137
• CPC Class Codes: G06Q 20/042 characterized in that the payment protocol involves at least one cheque
• CPC Class Codes: G06Q 20/10 specially adapted for electronic funds transfer [EFT] systems; specially adapted for home banking systems
• CPC Class Codes: G06Q 20/108 Remote banking, e.g. home banking
• CPC Class Codes: G06V 10/20 Image preprocessing
• CPC Class Codes: H04N 1/00204 with a digital computer or a digital computer system, e.g. an internet server programmed control between transmitter and receiver or between image input and image output device H04N1/32561
• CPC Class Codes: H04N 1/00244 with a server, e.g. an internet server fax-servers or the like for store and forward H04N1/324
• CPC Class Codes: H04N 1/00307 with a mobile telephone apparatus constructional features of portable telephone sets H04M1/0202; mobile radio systems H04W84/00
• CPC Class Codes: H04N 1/00816 Determining the reading area, e.g. eliminating reading of margins
• CPC Class Codes: H04N 1/387 Composing, repositioning or otherwise geometrically modifying originals
• CPC Class Codes: H04N 1/403 Discrimination between the two tones in the picture signal of a two-tone original
• CPC Class Codes: H04N 2101/00 Still video cameras
• CPC Class Codes: H04N 2201/001 Sharing resources, e.g. processing power or memory, with a connected apparatus or enhancing the capability of the still picture apparatus

Abstract

The present invention relates to automated document processing and more particularly, to methods and systems for document image capture and processing using mobile devices. In accordance with various embodiments, methods and systems for document image capture on a mobile communication device are provided such that the image is optimized and enhanced for data extraction from the document as depicted. These methods and systems may comprise capturing an image of a document using a mobile communication device; transmitting the image to a server; and processing the image to create a bi-tonal image of the document for data extraction. Additionally, these methods and systems may comprise capturing a first image of a document using the mobile communication device; automatically detecting the document within the image; geometrically correcting the image; binarizing the image; correcting the orientation of the image; correcting the size of the image; and outputting the resulting image of the document.

Claims

1. A method for mobile image capture and processing for documents, comprising:

capturing a first image of a document using the mobile communication device;

automatically detecting the document within the first image using a processor of the communication device, wherein detecting the document within the first image further comprises;

converting the first image to a color icon image, wherein the first image is in color;

reducing color within the color icon image, resulting in a gray-scale icon image; and

determining a plurality of corners belonging to the document within the gray-scale icon image;

geometrically correcting the first image using the processor of the communication device, resulting in a second image;

transmitting the second image to a server;

binarizing the second image using a processor of the server, resulting in a third image;

correcting the orientation of the third image using the processor of the server, resulting in a fourth image;

outputting the fourth image of the document using the processor of the server.

2. The method of claim 1, wherein automatically detecting the document comprises determining a plurality of corners belonging to the document within the first image.

3. The method of claim 1, wherein geometrically correcting the first image comprises:

reducing color within the first image, wherein the first image is in color and the color reduction of the first image results in a first gray-scale image;

building a projective transformation model, wherein the projective transformation model maps the document within the first gray-scale image to a gray-scale document image;

applying the projective transformation model to the first image, resulting in the second image as a gray-scale document image.

4. The method of claim 3, wherein geometrically correcting the first image further comprises:

correcting the orientation of the document within a gray-scale icon image if the document within the first image is in landscape orientation; and

building the projective transformation model from the gray-scale icon image.

5. The method of claim 1, wherein the third image comprises a bi-tonal image.

6. The method of claim 1, wherein correcting the orientation of the third image comprises correcting the orientation of the document within the third image if the document is in upside-down orientation.

7. The method of claim 6, wherein correcting the orientation of the third image further comprises determining the orientation of the document within the third image using a relevant object of known position on the document.

9. The method of claim 1, wherein outputting the fourth image of the document comprises outputting a bi-tonal image of the document.

10. The method of claim 1, wherein the mobile communication device is a camera phone.

8. A computer-implemented method for mobile image capture and processing for documents, wherein one or more processors are programmed perform the steps comprising:

receiving a color image of a document from a mobile device, the color image being captured using an image capture device of the mobile device;

detecting the financial document in the color image;

geometrically correcting the color image and generating a gray-scale image;

binarizing the gray-scale image to produce a bi-tonal image

correcting the orientation of the bi-tonal image;

correcting the size of the bi-tonal image using the processor of the server, resulting in a size-corrected bi-tonal image, wherein correcting the size of the bi-tonal image comprises;

reading a relevant object of a known position on the document within the bi-tonal image;

computing an average width of the relevant object;

computing a scaling factor based on the average width of the relevant object;

using the scaling factor to determine whether the bi-tonal image needs a size correction;

applying a size correction to the color image, resulting in a resized image;

geometrically correcting the resized image, resulting in a corrected image; and

outputting the corrected image of the document.

11. A computer program product comprising a non-transitory computer readable medium having computer program code embodied therein for enabling a computing device to perform the operations of:

receiving a color image of a document from a mobile device, the color image being captured using an image capture device of the mobile device;

detecting the financial document in the color image;

geometrically correcting the color image and generating a gray-scale image;

binarizing the gray-scale image to produce a bi-tonal image

correcting the orientation of the bi-tonal image;

correcting the size of the bi-tonal image using the processor of the server, resulting in a size-corrected bi-tonal image, wherein correcting the size of the bi-tonal image comprises;

reading a relevant object of a known position on the document within the bi-tonal image;

computing an average width of relevant object of a known position;

computing a scaling factor based on the average width of the relevant object of a known position;

using the scaling factor to determine whether the fourth image needs a size correction;

applying a size correction to the color image, resulting in a resized image;

geometrically correcting the resized image, resulting in the geometrically corrected resized image;

binarizing the geometrically corrected resized image resulting in the size-corrected bi-tonal image; and

outputting the size-corrected bi-tonal image.

12. A computer-implemented method for image capture and processing of financial documents, wherein one or more processors are programmed to perform steps comprising:

receiving a color image of a financial document from a mobile device, the color image being captured using an image capture device of the mobile device;

detecting the financial document in the color image;

geometrically correct the color image and generating a gray-scale image, geometrically correcting the color image and generating a geometrically corrected gray-scale image by;

reducing the color depth of the color image, wherein reducing the color depth of the color image further comprises

eliminating over-sharpening in the color image,

eliminating small dark objects in the color image, and

suppressing high local contrast objects in the color image;

generating a gray-scale image corresponding to the color image; and

applying perspective transformation to the interior of a quadrangle comprising found document corner positions included in the gray-scale image,

binarizing the gray-scale image to produce a bi-tonal image,

correcting the orientation of the bi-tonal image, and

correcting the size of the bi-tonal image.

13. A computer-implemented method for image capture and processing of financial documents, wherein one or more processors are programmed to perform steps comprising:

receiving a color image of a financial document from a mobile device, the color image being captured using an image capture device of the mobile device;

detecting the financial document in the color image;

geometrically correcting the color image and generating a gray-scale image;

binarizing the gray-scale image to produce a bi-tonal image, wherein binarizing the geometrically corrected image to produce a bi-tonal image further comprises;

choosing a pixel on the gray-scale image;

computing an average value and standard deviation for the chosen pixel;

whether the standard deviation is too small and if it is too small then converting the chosen pixel to white and if it is not too small then converting the chosen pixel to black or white based on the intensity; and

choosing another pixel an repeating the computing and determining steps until there are no more pixels to chose;

correcting the orientation of the bi-tonal image; and

correcting the size of the bi-tonal image.

14. A computer-implemented method for image capture and processing of financial documents, wherein one or more processors are programmed to perform steps comprising:

receiving a color image of a financial document from a mobile device, the color image being captured using an image capture device of the mobile device;

detecting the financial document in the color image, wherein detecting the financial document in the color image further comprises;

converting the color image into a smaller color image;

performing color depth reduction on the smaller color image to generate a small gray-scale image; and

determining document corner positions for the financial document in the small gray-scale image;

geometrically correcting the color image and generate a gray-scale image,

binarizing the gray-scale image to produce a bi-tonal image,

correcting the orientation of the bi-tonal image, and

correcting the size of the bi-tonal image.

15. The method of claim 14, wherein converting the color image into a smaller color image further comprises:

eliminating over-sharpening in the color image;

eliminating small dark objects in the color image; and

suppressing high local contrast objects in the color image.

16. The method of claim 14, wherein performing color depth reduction on the smaller color image further comprises:

dividing the smaller color image into a grid;

computing an average color for a central part of the smaller image;

computing an average color for the remainder of the smaller color image;

finding a linear transformation by mapping the central part color on white and mapping the color for the rest of the smaller image on black; and

transforming RGB values for the image into gray-shades to produce a smaller gray image.

17. The method of claim 16, further comprising:

finding voting points in the smaller gray image for each side of the financial document;

finding side candidates for the financial document based on the voting points;

choosing best side candidates from the side candidates; and

determining document corner positions for the financial document based on the best side candidates.

18. The method of claim 17, geometrically correcting the color image and generate a geometrically corrected gray-scale image further comprises:

correcting the landscape orientation of the smaller gray-scale image based on the determined document corner positions;

building a projective transformation model of the smaller gray-scale image; and

applying perspective transformation to an interior of a quadrangle included in the smaller gray-scale image.