Systems and methods for mobile image capture and processing

• US 9,165,188 B2
• Filed: 01/11/2013
• Issued: 10/20/2015
• Est. Priority Date: 01/12/2012
• Status: Active Grant

• Current Assignee: Tungsten Automation Corp.
• Original Assignee: Kofax Incorporated
• Inventors: Thrasher, Christopher W., Macciola, Anthony, Shustorovich, Alexander
• Primary Examiner(s): MARIAM, DANIEL G
• Application Number:
  US13/740,138
• Publication Number:
  US 20130182959A1
• Time in Patent Office: 1,012 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 17/40 Data acquisition and logging for input to computer G06F3/00
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06Q 40/08 Insurance
• CPC Class Codes: G06Q 50/18 Legal services
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: G06T 11/20 Drawing from basic elements, e.g. lines or circles
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/20164 Salient point detection; Corner detection
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 2207/30176 Document
• CPC Class Codes: G06T 3/02 Affine transformations for image registration G06T3/147; for image mosaicing G06T3/4038
• CPC Class Codes: G06T 3/10 Selection of transformation methods according to the characteristics of the input images
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/00 Image analysis
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/143 : involving probabilistic approaches, e.g. Markov random field [MRF] modelling
• CPC Class Codes: G06T 7/194 : involving foreground-background segmentation
• CPC Class Codes: G06V 10/247 : by affine transforms, e.g. correction due to perspective effects; Quadrilaterals, e.g. trapezoids
• CPC Class Codes: G06V 10/25 : Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 30/413 : Classification of content, e.g. text, photographs or tables
• CPC Class Codes: G06V 30/414 : Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text
• CPC Class Codes: H04N 1/00116 : of multimedia information
• CPC Class Codes: H04N 1/00251 : with an apparatus for taking photographic images, e.g. a camera
• CPC Class Codes: H04N 1/00403 : Voice input means, e.g. voice commands
• CPC Class Codes: H04N 1/387 : Composing, repositioning or otherwise geometrically modifying originals
• CPC Class Codes: H04N 1/3872 : Repositioning or masking
• CPC Class Codes: H04N 1/3878 : Skew detection or correction
• CPC Class Codes: H04N 1/40 : Picture signal circuits H04N1/387 takes precedence
• CPC Class Codes: H04N 2201/0081 : Image reader H04N2201/0091 - H04N2201/0094 take precedence
• CPC Class Codes: H04N 2201/0096 : Portable devices

Abstract

In various embodiments, methods, systems, and computer program products for processing digital images captured by a mobile device are disclosed. Myriad features enable and/or facilitate processing of such digital images using a mobile device that would otherwise be technically impossible or impractical, and furthermore address unique challenges presented by images captured using a camera rather than a traditional flat-bed scanner, paper-feed scanner or multifunction peripheral.

Claims

1. A method for estimating a resolution of a digital image comprising a digital representation of a document, the method comprising, using at least one processor:

detecting a plurality of connected components of a plurality of non-background elements in the digital image;

determining a plurality of likely characters based on the plurality of connected components;

determining one or more average character dimensions based on the plurality of likely characters; and

estimating the resolution of the digital image based on the one or more average character dimensions, and wherein the processor is part of a mobile device, the mobile device having an integrated camera.

2. The method as recited in claim 1, further comprising:

estimating one or more dimensions of the digital representation of the document based on the estimated resolution of the digital image;

comparing the one or more estimated dimensions of the digital representation of the document to one or more known dimensions of a plurality of known document types;

matching the digital representation of the document to one or more of the plurality of known document types based on the comparison;

determining whether the match satisfies one or more quality control criteria; and

adjusting the estimated resolution of the digital representation of the document based on the known dimensions of the known document type upon determining the match satisfies the one or more quality control criteria.

3. The method as recited in claim 1, further comprising, for one or more of the connected components, computing one or more of:

a number of on-off transitions within the connected component;

a black pixel density within the connected component;

an aspect ratio of the connected component; and

calculating a likelihood that one or more of the connected components represents a text character based on one or more of the computed black pixel density, the computed number of on-off transitions, and the computed aspect ratio.

4. The method as recited in claim 1, further comprising binarizing the digital image to generate a bitonal image.

5. The method as recited in claim 1, wherein the average character dimensions comprise an average character width and an average character height.

6. The method as recited in claim 1, further comprising:

determining a character height of at least two of the plurality of characters;

calculating an average character height based on each character height of the at least two characters;

determining a character width of at least two of the plurality of characters;

calculating an average character width based on each character width of the at least two characters;

performing at least one comparison selected from a group consisting of;

comparing the average character height to a reference average character height; and

comparing the average character width to a reference average character width; and

estimating the resolution of the digital image based on the at least one comparison,

wherein each of the reference average character height and the reference average character width correspond to one or more reference characters, each reference character being characterized by a known character width and a known character height.

7. The method as recited in claim 6, wherein each reference character corresponds to a digital representation of a character obtained from scanning one or more documents at a resolution of at least about 300 DPI, and

wherein each reference character corresponds to one or more common fonts.

8. A method as recited in claim 1, further comprising:

estimating one or more dimensions of the digital representation of the document based on the estimated resolution of the digital representation of the document;

computing an average character width from the average character dimensions;

computing an average character height from the average character dimensions;

comparing the average character width to the average character height;

estimating an orientation of the digital representation of the document based on the comparison; and

matching the digital representation of the document to a known document type based on the estimated dimensions and the estimated orientation.

9. The method as recited in claim 8, wherein the one or more known document types comprise one or more of:

a Letter size document;

a Legal size document;

an A3 document;

an A4 (European Letter size) document;

an A5 document;

a ledger/tabloid document;

a driver license;

a business card;

a personal check;

a business check;

a 2.25-inch wide receipt; and

a 3.125-inch wide receipt.

10. The method as recited in claim 1, wherein determining the plurality of likely characters based on the plurality of connected components further comprises determining a number of light-to-dark transitions in at least one direction within a region of interest in the digital image.

11. The method as recited in claim 10, wherein determining the plurality of likely characters based on the plurality of connected components further comprises determining an identity of one or more characters depicted within the region of interest based on the number of light-to-dark transitions in the at least one direction.

12. The method as recited in claim 10, wherein the at least one direction includes a vertical direction and a horizontal direction.

13. The method as recited in claim 1, wherein the plurality of likely characters are characterized by a font selected from Arial, Times New Roman, Helvetica, Courier, Courier New, and Tahoma.

14. The method as recited in claim 1, wherein estimating the resolution of the digital image further comprises comparing the one or more average character dimensions to known dimensions of a plurality of reference characters, and

wherein the plurality of likely characters are characterized by a font selected from Arial, Times New Roman, Helvetica, Courier, Courier New, and Tahoma.

15. The method as recited in claim 1, wherein estimating the resolution of the digital image further comprises comparing the one or more average character dimensions to known dimensions of a plurality of reference characters, and

wherein each of the reference characters is characterized by a font selected from Arial, Times New Roman, Helvetica, Courier, Courier New, and Tahoma.

16. A system, comprising:

a processor configured to execute logic;

logic for detecting a plurality of connected components of a plurality of non-background elements in a digital image;

logic for determining a plurality of likely characters based on the plurality of connected components;

logic for determining one or more average character dimensions based on the plurality of likely characters; and

logic for estimating a resolution of the digital image based on the one or more average character dimensions, and wherein the processor is part of a mobile device, the mobile device having an integrated camera.

17. The system as recited in claim 16, further comprising logic configured to:

for one or more of the connected components, compute one or more of;

a number of on-off transitions within the connected component;

a black pixel density within the connected component;

an aspect ratio of the connected component; and

calculate a likelihood that one or more of the connected components represents a text character based on one or more of the computed black pixel density, the computed number of on-off transitions, and the computed aspect ratio.

18. A computer program product comprising a non-transitory computer readable storage medium having computer readable program code being configured to cause a processor of a mobile device, upon execution thereof, to:

stored thereon, the computer readable program code;

detect a plurality of connected components of a plurality of non-background elements in a digital image;

determine a plurality of likely characters based on the plurality of connected components;

determine one or more average character dimensions based on a plurality of likely text characters; and

estimate a resolution of the digital image based on the one or more average character dimensions.

19. The computer program product as recited in claim 18, the computer readable program code being further configured to cause the processor to:

for one or more of the connected components, compute one or more of;

a number of on-off transitions within the connected component;

a black pixel density within the connected component;

an aspect ratio of the connected component; and

calculate a likelihood that one or more of the connected components represents a text character based on one or more of the computed black pixel density, the computed number of on-off transitions, and the computed aspect ratio.