Systems and methods for mobile image capture and processing

• US 9,117,117 B2
• Filed: 12/12/2014
• Issued: 08/25/2015
• Est. Priority Date: 01/12/2012
• Status: Active Grant

• Current Assignee: Kofax Incorporated
• Original Assignee: Kofax Incorporated
• Inventors: Macciola, Anthony, Shustorovich, Alexander, Thrasher, Christopher W.
• Primary Examiner(s): AZARIAN, SEYED H
• Application Number:
  US14/569,375
• Publication Number:
  US 20150098628A1
• Time in Patent Office: 256 Days
• Field of Search: 382/100, 382/103, 382/106, 382113-116, 382135-140, 382/155, 382/168, 382173-194, 382/199, 382209-220, 382/232, 382/254, 382/266, 382/274, 382/276, 382286-294, 382/305, 382/312, 382/321, 382/167, 382/112, 705/38, 235/379, 709/203, 348/222.1
• US Class Current: 1/1
• CPC Class Codes: G06F 17/40 Data acquisition and logging for input to computer G06F3/00
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06Q 40/08 Insurance
• CPC Class Codes: G06Q 50/18 Legal services
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: G06T 11/20 Drawing from basic elements, e.g. lines or circles
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/20164 Salient point detection; Corner detection
• CPC Class Codes: G06T 2207/30004 Biomedical image processing
• CPC Class Codes: G06T 2207/30176 Document
• CPC Class Codes: G06T 3/02 Affine transformations for image registration G06T3/147; for image mosaicing G06T3/4038
• CPC Class Codes: G06T 3/10 Selection of transformation methods according to the characteristics of the input images
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 7/00 Image analysis
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/143 : involving probabilistic approaches, e.g. Markov random field [MRF] modelling
• CPC Class Codes: G06T 7/194 : involving foreground-background segmentation
• CPC Class Codes: G06V 10/247 : by affine transforms, e.g. correction due to perspective effects; Quadrilaterals, e.g. trapezoids
• CPC Class Codes: G06V 10/25 : Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 30/413 : Classification of content, e.g. text, photographs or tables
• CPC Class Codes: G06V 30/414 : Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text
• CPC Class Codes: H04N 1/00116 : of multimedia information
• CPC Class Codes: H04N 1/00251 : with an apparatus for taking photographic images, e.g. a camera
• CPC Class Codes: H04N 1/00403 : Voice input means, e.g. voice commands
• CPC Class Codes: H04N 1/387 : Composing, repositioning or otherwise geometrically modifying originals
• CPC Class Codes: H04N 1/3872 : Repositioning or masking
• CPC Class Codes: H04N 1/3878 : Skew detection or correction
• CPC Class Codes: H04N 1/40 : Picture signal circuits H04N1/387 takes precedence
• CPC Class Codes: H04N 2201/0081 : Image reader H04N2201/0091 - H04N2201/0094 take precedence
• CPC Class Codes: H04N 2201/0096 : Portable devices

Abstract

In various embodiments, methods, systems, and computer program products for capturing and processing digital images captured by a mobile device are disclosed. The claimed algorithms are specifically configured to perform and facilitate loan application processing by capturing an image of a document using a mobile device, and analyzing the image (optionally in conjunction with additional data that may also be captured, determined, or otherwise provided to the loan application process) to determine loan-relevant information. Select loan-relevant information may be extracted, compiled, and/or analyzed to facilitate processing of the loan application. Feedback may be provided to facilitate facile application processing, e.g. by ensuring all requisite information is submitted with the loan application. Image capture and document detection are preferably performed using the mobile device, while all other functions may be performed using the mobile device, a remote server, or some combination thereof.

Claims

1. A method, comprising:

capturing image data using a mobile device, the image data depicting a digital representation of a document; and

using at least one processor;

defining, based on the image data, a plurality of candidate edge points corresponding to the document;

defining four sides of a tetragon based on at least some of the plurality of candidate edge points;

determining a document type corresponding to the document is one of;

“

loan document;

”

“

data relating to loan application;

”

“

data not relating to loan application;

” and

“

unknown;

” and

displaying at least one of;

loan-relevant information from the captured image; and

a loan status corresponding to the document,

wherein the displaying is performed using a display of the mobile device.

2. The method as recited in claim 1, further comprising associating the determined document type with the captured image data.

3. The method as recited in claim 1, wherein the document type “

data relating to loan application”

corresponds to one or more of a termite report, a proof of residence, an income verification, a property comparison, an appraisal, a proof of insurance, a form of personal identification, a signature, an audio clip comprising a statement by a user, an image of property subject to a loan, and video data depicting property subject to a loan.

4. The method as recited in claim 3, further comprising:

capturing one or more of the audio clip and the video data using the mobile device;

extracting loan-relevant information from on one or more of the document, the audio clip and the video data; and

submitting loan application data comprising at least one of the captured image, the determined document type, extracted loan-relevant information, and one or more signatures relating to the document.

5. The method as recited in claim 1, further comprising receiving one or more signatures relating to the loan document, and associating the one or more signatures with the captured image data.

6. The method as recited in claim 5, wherein receiving the one or more signatures further comprises capturing the one or more signatures using the mobile device.

7. The method as recited in claim 6, wherein capturing the one or more signatures comprises one or more of:

a user inputting a digital signature via a display of the mobile device; and

capturing an image of a signature.

8. The method as recited in claim 6, further comprising validating one or more of a user identification and a user authorization.

9. The method as recited in claim 1, further comprising authenticating an identity of a user performing the capturing using the mobile device prior to performing the capturing,

wherein the authenticating further comprises determining the user identity corresponds to either an authorized loan officer or a loan applicant.

10. The method as recited in claim 1, further comprising extracting loan-relevant information from the captured image.

11. The method as recited in claim 10, wherein the displaying comprises displaying the extracted loan-relevant information using the display of the mobile device.

12. The method as recited in claim 11, further comprising prompting a user for feedback regarding the displayed loan-relevant information.

13. The method as recited in claim 10, further comprising transmitting data to a remote server, the data comprising at least one of the captured image, the determined document type, the extracted loan-relevant information, and one or more signatures relating to the document.

14. The method as recited in claim 13, further comprising, prior to the transmitting, analyzing the data to detect an absence of one or more required components selected from the captured image, the determined document type, the extracted loan-relevant information, and the one or more signatures relating to the loan document.

15. The method as recited in claim 14, further comprising, in response to detecting the absence of one or more of the required components, providing an indication of the absence to a user via the mobile device.

16. The method as recited in claim 14, further comprising:

in response to detecting the absence of one or more of the required components, prompting a user of the mobile device to provide the one or more required components detected to be absent; and

transmitting the data including the one or more required components to the remote server in response to receiving the one or more required components.

17. The method as recited in claim 16, wherein the prompting, the transmitting, and the detecting are each performed using the mobile device.

18. The method as recited in claim 1, further comprising transmitting data to a remote server, the data comprising at least one of the captured image, the determined document type, loan-relevant information, and one or more signatures relating to the document.

19. The method as recited in claim 1, wherein each side of the tetragon corresponds to a different side of the document, and

wherein the tetragon bounds the digital representation of the document.

20. computer program product comprising a non-transitory computer readable medium having embodied therewith computer readable program code, the computer readable program code comprising:

computer readable program code configured to capture image data using a mobile device, the image data depicting a digital representation of a document;

computer readable program code configured to define, based on the image data, a plurality of candidate edge points corresponding to the document;

computer readable program code configured to define four sides of a tetragon based on at least some of the plurality of candidate edge points; and

computer readable program code configured to determine a document type corresponding to the document is one of;

“

loan document;

”

“

data relating to loan application;

”

“

data not relating to loan application;

” and

“

unknown”

; and

computer readable program code configured to display at least one of;

loan-relevant information from the captured image; and

a loan status corresponding to the document, wherein the displaying is performed using a display of the mobile device.