SYSTEMS AND METHODS FOR MOBILE IMAGE CAPTURE AND PROCESSING

• US 20130182959A1
• Filed: 01/11/2013
• Published: 07/18/2013
• Est. Priority Date: 01/12/2012
• Status: Active Grant

• Current Assignee: Kofax Incorporated
• Original Assignee: Kofax Incorporated
• Inventors: Thrasher, Christopher W., Macciola, Anthony, Shustorovich, Alexander
• Granted Patent: US 9,165,188 B2
• Time in Patent Office: Days
• US Class Current: 382/195
• CPC Class Codes: G06F 17/40 Data acquisition and logging for input to computer G06F3/00
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06Q 40/08 Insurance
• CPC Class Codes: G06Q 50/18 Legal services
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: G06T 11/20 Drawing from basic elements, e.g. lines or circles
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/20164 Salient point detection; Corner detection
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 2207/30176 Document
• CPC Class Codes: G06T 3/02 Affine transformations for image registration G06T3/147; for image mosaicing G06T3/4038
• CPC Class Codes: G06T 3/10 Selection of transformation methods according to the characteristics of the input images
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/00 Image analysis
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/143 : involving probabilistic approaches, e.g. Markov random field [MRF] modelling
• CPC Class Codes: G06T 7/194 : involving foreground-background segmentation
• CPC Class Codes: G06V 10/247 : by affine transforms, e.g. correction due to perspective effects; Quadrilaterals, e.g. trapezoids
• CPC Class Codes: G06V 10/25 : Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 30/413 : Classification of content, e.g. text, photographs or tables
• CPC Class Codes: G06V 30/414 : Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text
• CPC Class Codes: H04N 1/00116 : of multimedia information
• CPC Class Codes: H04N 1/00251 : with an apparatus for taking photographic images, e.g. a camera
• CPC Class Codes: H04N 1/00403 : Voice input means, e.g. voice commands
• CPC Class Codes: H04N 1/387 : Composing, repositioning or otherwise geometrically modifying originals
• CPC Class Codes: H04N 1/3872 : Repositioning or masking
• CPC Class Codes: H04N 1/3878 : Skew detection or correction
• CPC Class Codes: H04N 1/40 : Picture signal circuits H04N1/387 takes precedence
• CPC Class Codes: H04N 2201/0081 : Image reader H04N2201/0091 - H04N2201/0094 take precedence
• CPC Class Codes: H04N 2201/0096 : Portable devices

Abstract

In various embodiments, methods, systems, and computer program products for processing digital images captured by a mobile device are disclosed. Myriad features enable and/or facilitate processing of such digital images using a mobile device that would otherwise be technically impossible or impractical, and furthermore address unique challenges presented by images captured using a camera rather than a traditional flat-bed scanner, paper-feed scanner or multifunction peripheral.

Claims

1. A method for estimating a resolution of a digital image comprising a digital representation of a document, the method comprising:

detecting a plurality of connected components of a plurality of non-background elements in the digital image;

determining a plurality of likely characters based on the plurality of connected components;

determining one or more average character dimensions based on the plurality of likely text characters; and

estimating the resolution of the digital image based on the one or more average character dimensions.

2. The method as recited in claim 1, further comprising:

estimating one or more dimensions of the digital representation of the document based on the estimated resolution of the digital image;

comparing the one or more estimated dimensions of the digital representation of the document to one or more known dimensions of a plurality of known document types;

matching the digital representation of the document to one or more of the plurality of known document types based on the comparison;

determining whether the match satisfies one or more quality control criteria; and

adjusting the estimated resolution of the digital representation of the document based on the known dimensions of the known document type upon determining the match satisfies the one or more quality control criteria.

3. The method as recited in claim 1, further comprising, for one or more connected components, computing one or more of:

a number of on-off transitions within the connected component;

a black pixel density within the connected component;

an aspect ratio of the connected component; and

calculating a likelihood that one or more of the connected components represents a text character based on one or more of the black pixel density, the number of on-off transitions, and the aspect ratio.

4. The method as recited in claim 1, further comprising binarizing the digital image to generate a bitonal image.

5. The method as recited in claim 1, wherein the average character dimensions comprise an average character width and an average character height.

6. The method as recited in claim 1, further comprising:

determining a character height of at least two of the plurality of text characters;

calculating an average character height based on each character height of the at least two text characters;

determining a character width of at least two of the plurality of text characters;

calculating an average character width based on each character width of the at least two text characters;

performing at least one comparison selected from a group consisting of;

comparing the average character height to a reference average character height; and

comparing the average character width to a reference average character width; and

estimating the resolution of the digital image based on the at least one comparison,

wherein each of the reference average character height and the reference average character width correspond to one or more reference characters, each reference character being characterized by a known character width and a known character height.

7. The method as recited in claim 6, wherein each reference character corresponds to a digital representation of a character obtained from scanning one or more documents at a reference resolution, and

wherein each reference character corresponds to one or more common fonts.

8. A method as recited in claim 1, further comprising:

estimating one or more dimensions of the digital representation of the document based on the estimated resolution of the digital representation of the document;

computing an average character width from the average character dimensions;

computing an average character height from the average character dimensions;

comparing the average character width to the average character height;

estimating an orientation of the digital representation of the document based on the comparison; and

matching the digital representation of the document to a known document type based on the estimated dimensions and the estimated orientation.

9. The method as recited in claim 8, wherein the one or more known document types comprise one or more of:

a Letter size document;

a Legal size document;

an A3 document;

an A4 (European Letter size) document;

an A5 document;

a ledger/tabloid document;

a driver license;

a business card;

a personal check;

a business check;

a 2.25-inch wide receipt; and

a 3.125-inch wide receipt.

10. The method as recited in claim 1, wherein the processor is part of a mobile device, the mobile device having an integrated camera.

11. A system, comprising:

a processor configured to execute logic;

logic for detecting a plurality of connected components of a plurality of non-background elements in the digital image;

logic for determining a plurality of likely characters based on the plurality of connected components;

logic for determining one or more average character dimensions based on the plurality of likely text characters; and

logic for estimating the resolution of the digital image based on the one or more average character dimensions.

12. A computer program product comprising a computer readable storage medium having computer readable program code stored thereon, the computer readable program code comprising:

computer readable program code configured to detect a plurality of connected components of a plurality of non-background elements in the digital image;

computer readable program code configured to determine a plurality of likely characters based on the plurality of connected components;

computer readable program code configured to determine one or more average character dimensions based on the plurality of likely text characters; and

computer readable program code configured to estimate the resolution of the digital image based on the one or more average character dimensions.