SYSTEMS AND METHODS FOR MOBILE IMAGE CAPTURE AND PROCESSING

• US 20130182292A1
• Filed: 01/11/2013
• Published: 07/18/2013
• Est. Priority Date: 01/12/2012
• Status: Active Grant

• Current Assignee: Kofax Incorporated
• Original Assignee: Kofax Incorporated
• Inventors: Thrasher, Christopher W., Macciola, Anthony, Shustorovich, Alexander
• Granted Patent: US 8,879,120 B2
• Time in Patent Office: Days
• US Class Current: 358/448
• CPC Class Codes: G06F 17/40 Data acquisition and logging for input to computer G06F3/00
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06Q 40/08 Insurance
• CPC Class Codes: G06Q 50/18 Legal services
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: G06T 11/20 Drawing from basic elements, e.g. lines or circles
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/20164 Salient point detection; Corner detection
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 2207/30176 Document
• CPC Class Codes: G06T 3/02 Affine transformations for image registration G06T3/147; for image mosaicing G06T3/4038
• CPC Class Codes: G06T 3/10 Selection of transformation methods according to the characteristics of the input images
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/00 Image analysis
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/143 : involving probabilistic approaches, e.g. Markov random field [MRF] modelling
• CPC Class Codes: G06T 7/194 : involving foreground-background segmentation
• CPC Class Codes: G06V 10/247 : by affine transforms, e.g. correction due to perspective effects; Quadrilaterals, e.g. trapezoids
• CPC Class Codes: G06V 10/25 : Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 30/413 : Classification of content, e.g. text, photographs or tables
• CPC Class Codes: G06V 30/414 : Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text
• CPC Class Codes: H04N 1/00116 : of multimedia information
• CPC Class Codes: H04N 1/00251 : with an apparatus for taking photographic images, e.g. a camera
• CPC Class Codes: H04N 1/00403 : Voice input means, e.g. voice commands
• CPC Class Codes: H04N 1/387 : Composing, repositioning or otherwise geometrically modifying originals
• CPC Class Codes: H04N 1/3872 : Repositioning or masking
• CPC Class Codes: H04N 1/3878 : Skew detection or correction
• CPC Class Codes: H04N 1/40 : Picture signal circuits H04N1/387 takes precedence
• CPC Class Codes: H04N 2201/0081 : Image reader H04N2201/0091 - H04N2201/0094 take precedence
• CPC Class Codes: H04N 2201/0096 : Portable devices

Abstract

In various embodiments, methods, systems, and computer program products for processing digital images captured by a mobile device are disclosed. Myriad features enable and/or facilitate processing of such digital images using a mobile device that would otherwise be technically impossible or impractical, and furthermore address unique challenges presented by images captured using a camera rather than a traditional flat-bed scanner, paper-feed scanner or multifunction peripheral.

Claims

1. A method, comprising:

dividing, using a processor, a tetragon comprising a digital representation of a document in a digital image into a plurality of sections, each section comprising a plurality of pixels;

for each section;

determining whether the section contains one or more sharp pixel-to-pixel transitions in a first direction;

counting a total number of first direction sharp pixel-to-pixel transitions for the section (S_S1);

determining whether the section contains one or more blurred pixel-to-pixel transitions in the first direction;

counting a total number of first-direction blurred pixel-to-pixel transitions for the section (S_B1);

determining whether the section contains one or more sharp pixel-to-pixel transitions in a second direction;

counting a total number of second-direction sharp pixel-to-pixel transitions for the section (S_S2);

determining whether the section contains one or more blurred pixel-to-pixel transitions in the second direction;

counting a total number of second-direction blurred pixel-to-pixel transitions for the section (S_B2);

determining the section is blank upon determining;

S_S1 is less than a predetermined sharp transition threshold,

S_B1 is less than a predetermined blurred transition threshold,

S_S2 is less than a predetermined sharp transition threshold, and

S_B2 is less than a predetermined blurred transition threshold; and

determining, for all non-blank sections, a first direction blur ratio r₁=S_S1/S_B1;

determining, for all non-blank sections, a second direction blur ratio r₂=S_S2/S_B2;

determining that a non-blank section is blurred in the first direction upon

determining that r₁ is less than a predefined section blur ratio threshold; and

determining that a non-blank section is blurred in the second direction upon determining that r₂ is less than the predefined section blur ratio threshold; and

determining that a non-blank section is bluffed upon determining one or more of;

the section is blurred in the first direction, and

the section is blurred in the section direction; and

determining a total number of blurred sections;

calculating an image blur ratio R comprising;

the total number blurred sections to a total number of sections; and

determining the digital image is blurred upon determining the image blur ratio is greater than a predetermined image blur threshold.

2. The method as recited in claim 1, further comprising:

determining, for each section a distribution of brightness values of the plurality of pixels;

determining a characteristic variability v of the distribution of brightness values;

calculating a noticeable brightness transition threshold η

based on v;

calculating a large brightness transition threshold μ

based on η

;

analyzing, for each pixel within the plurality of pixels, a directional pattern of brightness change in a window surrounding the pixel; and

identifying one or more of;

the sharp pixel-to-pixel transition and the blurred pixel-to-pixel transitions based on the analysis.

3. The method as recited in claim 2, further comprising:

defining a plurality of center pixels;

sequentially analyzing each of the plurality of center pixels within one or more small windows of pixels surrounding each center pixel;

identifying the sharp pixel-to-pixel transition upon determining;

the large brightness transition exists within an immediate vicinity of one of the center pixels;

a first small brightness variation exists before the large brightness transition; and

a second small brightness variation exists after the large brightness transition;

detecting the sharp pixel-to-pixel transition upon determining;

the large transition exists within one or more of the small windows,

a monotonic change in brightness exists in the large transition; and

detecting the bluffed pixel-to-pixel transition upon determining;

a noticeable transition occurs within a small window; and

the monotonic change in brightness exists in the noticeable transition.

4. The method as recited in claim 3, further comprising, for each section:

counting a total number of sharp transitions in each of one or more chosen directions;

counting a total number of blurred transitions in each chosen direction;

determining that a section is blank upon determining;

the total number of sharp transitions is less than a predefined sharp transition threshold and

the total number of blurred transitions is less than a predefined blurred transition threshold;

determining the non-blank section is blurred upon determining a section blurriness ratio comprising the total number of sharp transitions to the total number of blurred transitions is less than a section blur ratio threshold in at least one of the chosen directions; and

determining that the section is sharp upon determining the section is neither blank nor blurred.

5. The method as recited in claim 4, further comprising:

determining a total number of blank sections within the plurality of sections (N_blank);

determining a total number of blurred sections within the plurality of sections (N_blur);

determining a total number of sharp sections within the plurality of sections (N_sharp);

determining a blurriness ratio (R_B)=N_blur/(N_blur+N_sharp); and

determining that the digital image is sharp if the R_B is less than a blurriness threshold.

6. The method as recited in claim 1, wherein the processor is part of a mobile device, the mobile device having an integrated camera.

7. A system, comprising:

a processor configured to execute logic;

logic for dividing, using a processor, a tetragon comprising a digital representation of a document in a digital image into a plurality of sections, each section comprising a plurality of pixels;

logic for determining whether the section contains one or more sharp pixel-to-pixel transitions in a first direction;

logic for counting a total number of first direction sharp pixel-to-pixel transitions for the section (S_S1);

logic for determining whether the section contains one or more blurred pixel-to-pixel transitions in the first direction;

logic for counting a total number of first-direction blurred pixel-to-pixel transitions for the section (S_B1);

logic for determining whether the section contains one or more sharp pixel-to-pixel transitions in a second direction;

logic for counting a total number of second direction sharp pixel-to-pixel transitions for the section (S_S2);

logic for determining whether the section contains one or more blurred pixel-to-pixel transitions in the second direction;

logic for counting a total number of second-direction blurred pixel-to-pixel transitions for the section (S_B2);

logic for determining the section is blank upon determining;

S_S1 is less than a predetermined sharp transition threshold,

S_B1 is less than a predetermined blurred transition threshold,

S_S2 is less than a predetermined sharp transition threshold, and

S_B2 is less than a predetermined blurred transition threshold; and

logic for determining, for all non-blank sections, a first direction blur ratio r₁=S_S1/S_B1;

logic for determining, for all non-blank sections, a second direction blur ratio r₂=S_S2/S_B2;

logic for determining that a non-blank section is blurred in the first direction upon determining that r₁ is less than a predefined section blur ratio threshold; and

logic for determining that a non-blank section is blurred in the second direction upon determining that r₂ is less than the predefined section blur ratio threshold; and

logic for determining that a non-blank section is blurred upon determining one or more of;

the section is blurred in the first direction; and

the section is blurred in the section direction; and

logic for determining a total number of blurred sections;

logic for calculating an image blur ratio R comprising;

the total number blurred sections to a total number of sections; and

logic for determining the digital image is blurred upon determining the image blur ratio is greater than a predetermined image blur threshold.

8. A computer program product comprising a computer readable storage medium having computer readable program code stored thereon, the computer readable program code comprising:

computer readable program code configured to divide, use a processor, a tetragon comprise a digital representation of a document in a digital image into a plurality of sections, each section comprise a plurality of pixels;

computer readable program code configured to determine whether the section contains one or more sharp pixel-to-pixel transitions in a first direction;

computer readable program code configured to count a total number of first direction sharp pixel-to-pixel transitions for the section (S_S1);

computer readable program code configured to determine whether the section contains one or more blurred pixel-to-pixel transitions in the first direction;

computer readable program code configured to count a total number of first-direction blurred pixel-to-pixel transitions for the section (S_B1);

computer readable program code configured to determine whether the section contains one or more sharp pixel-to-pixel transitions in a second direction;

computer readable program code configured to count a total number of second direction sharp pixel-to-pixel transitions for the section (S_S2);

computer readable program code configured to determine whether the section contains one or more blurred pixel-to-pixel transitions in the second direction;

computer readable program code configured to count a total number of second-direction blurred pixel-to-pixel transitions for the section (S_B1);

computer readable program code configured to determine the section is blank upon determining;

S_s1 is less than a predetermined sharp transition threshold,

S_B1 is less than a predetermined blurred transition threshold,

S_S2 is less than a predetermined sharp transition threshold, and

S_B2 is less than a predetermined blurred transition threshold; and

computer readable program code configured to determine, for all non-blank sections, a first direction blur ratio r₁=S_S1/S_B1;

computer readable program code configured to determine, for all non-blank sections, a second direction blur ratio r₂=S_S2/S_B2;

computer readable program code configured to determine that a non-blank section is blurred in the first direction upon determine that r₁ is less than a predefined section blur ratio threshold; and

computer readable program code configured to determine that a non-blank section is blurred in the second direction upon determine that r₂ is less than the predefined section blur ratio threshold; and

computer readable program code configured to determine that a non-blank section is blurred upon determine one or more of;

the section is blurred in the first direction; and

the section is blurred in the section direction; and

computer readable program code configured to determine a total number of blurred sections;

computer readable program code configured to calculate an image blur ratio R comprising;

the total number blurred sections to a total number of sections; and

computer readable program code configured to determine the digital image is blurred upon determine the image blur ratio is greater than a predetermined image blur threshold.