Mobile image capture, processing, and electronic form generation

• US 9,275,281 B2
• Filed: 07/20/2015
• Issued: 03/01/2016
• Est. Priority Date: 01/12/2012
• Status: Active Grant

• Current Assignee: Kofax Incorporated
• Original Assignee: Kofax Incorporated
• Inventors: Macciola, Anthony
• Primary Examiner(s): AZARIAN, SEYED H
• Application Number:
  US14/804,276
• Publication Number:
  US 20150324639A1
• Time in Patent Office: 225 Days
• Field of Search: 382/100, 382/103, 382/106, 382113-116, 382135-140, 382/155, 382/168, 382173-194, 382/199, 382209-220, 382/232, 382/112, 382/254, 382/266, 382/274, 382/276, 382286-294, 382/167, 382/305, 382/312, 382/321, 348/222.1, 709/203, 235/379, 705/38
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10008 from scanner, fax or copier
• CPC Class Codes: G06T 2207/20008 Globally adaptive
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/30176 Document
• CPC Class Codes: G06T 3/02 Affine transformations for image registration G06T3/147; for image mosaicing G06T3/4038
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/143 involving probabilistic approaches, e.g. Markov random field [MRF] modelling
• CPC Class Codes: G06T 7/181 involving edge growing; involving edge linking
• CPC Class Codes: G06V 10/242 by image rotation, e.g. by 90 degrees
• CPC Class Codes: G06V 10/247 by affine transforms, e.g. correction due to perspective effects; Quadrilaterals, e.g. trapezoids
• CPC Class Codes: G06V 30/414 Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text
• CPC Class Codes: G06V 30/416 Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors
• CPC Class Codes: H04N 1/00127 Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture
• CPC Class Codes: H04N 1/00204 with a digital computer or a digital computer system, e.g. an internet server programmed control between transmitter and receiver or between image input and image output device H04N1/32561
• CPC Class Codes: H04N 1/00307 with a mobile telephone apparatus constructional features of portable telephone sets H04M1/0202; mobile radio systems H04W84/00
• CPC Class Codes: H04N 1/387 Composing, repositioning or otherwise geometrically modifying originals
• CPC Class Codes: H04N 1/3872 Repositioning or masking
• CPC Class Codes: H04N 1/3878 Skew detection or correction
• CPC Class Codes: H04N 1/40 Picture signal circuits H04N1/387 takes precedence

Abstract

In various embodiments, methods, systems, and computer program products for capturing and processing digital images captured by a mobile device are disclosed. In one embodiment, a method includes capturing image data using a mobile device, the image data depicting a digital representation of a document; defining, based on the image data, a plurality of candidate edge points corresponding to the document; defining four sides of a tetragon based on at least some of the plurality of candidate edge points; determining a plurality of fields within the tetragon; for each field, determining at least a field location and a field data type; associating each determined field location with each field data type to generate a plurality of metadata labels; and associating the plurality of metadata labels with an image of an electronic form.

Claims

1. A computer implemented method, comprising:

capturing image data using a mobile device, the image data depicting a digital representation of a document;

defining, using a possessor and based on the image data, a plurality of candidate edge points corresponding to the document;

defining four sides of a tetragon based on at least some of the plurality of candidate edge points;

determining a plurality of fields within the tetragon;

for each field, determining at least a field location and a field data type;

associating each determined field location with each field data type to generate a plurality of metadata labels;

associating the plurality of metadata labels with an image of an electronic form

extracting data from the tetragon; and

populating the electronic form with the extracted data.

2. The method as recited in claim 1, wherein the electronic form is the digital representation of the document within the tetragon.

3. The method as recited in claim 1, further comprising generating the electronic form based on the digital representation of the document within the tetragon.

4. The method as recited in claim 3, wherein generating the electronic form comprises:

extracting the tetragon;

rectangularizing the tetragon;

rendering the rectangularized tetragon as a background layer of a digital image;

overlaying a plurality of fields on the background layer; and

associating one of the metadata labels with each of the plurality of fields based on a location of the field.

5. The method as recited in claim 1, further comprising generating the electronic form based on an image of a standard form.

6. The method as recited in claim 1, further comprising determining a type of the document, wherein determining the plurality of fields within the tetragon is based at least in part on the determined type of document.

7. The method as recited in claim 1, further comprising validating the extracted data.

8. The method as recited in claim 1, wherein the electronic form comprises a fillable printable document format (PDF) document.

9. A system, comprising:

a processor configured to execute logic, the logic being configured to cause the processor to;

receive image data captured using a mobile device, the image data depicting a digital representation of a document;

define, based on the image data, a plurality of candidate edge points corresponding to the document;

define four sides of a tetragon based on at least some of the plurality of candidate edge points;

determine a plurality of fields within the tetragon;

for each field, determine at least a field location and a field data type;

associate each determined field location with each field data type to generate a plurality of metadata labels;

associate the plurality of metadata labels with an image of an electronic form

extract data from the tetragon; and

populate the electronic form with the extracted data.

10. The system as recited in claim 9, wherein the electronic form is the digital representation of the document within the tetragon.

11. The system as recited in claim 9, further comprising logic configured to cause the processor to generate the electronic form based on the digital representation of the document within the tetragon.

12. The system as recited in claim 11, wherein the logic configured to cause the processor to generate the electronic form comprises logic configured to cause the processor to:

extracting the tetragon;

rectangularizing the tetragon;

rendering the rectangularized tetragon as a background layer of a digital image;

overlaying a plurality of fields on the background layer; and

associating one of the metadata labels with each of the plurality of fields based on a location of the field.

13. The system as recited in claim 9, further comprising logic configured to cause the processor to generate the electronic form based on an image of a standard form.

14. The system as recited in claim 9, further comprising logic configured to cause the processor to determine a type of the document, wherein determining the plurality of fields within the tetragon is based at least in part on the determined type of document.

15. The system as recited in claim 9, further comprising logic configured to cause the processor to validate the extracted data.

16. The system as recited in claim 9, wherein the electronic form comprises a fillable printable document format (PDF) document.

17. The system as recited in claim 16, further comprising transmitting the PDF document via electronic mail.

18. A computer program product comprising a non-transitory computer readable medium having embodied therewith computer readable program code, the computer readable program code comprising:

computer readable program code configured to capture image data using a mobile device, the image data depicting a digital representation of a document;

computer readable program code configured to define, based on the image data, a plurality of candidate edge points corresponding to the document;

computer readable program code configured to define four sides of a tetragon based on at least some of the plurality of candidate edge points; and

computer readable program code configured to determine a plurality of fields within the tetragon;

computer readable program code configured to for each field, determine at least a field location and a field data type;

computer readable program code configured to associate each determined field location with each field data type to generate a plurality of metadata labels;

computer readable program code configured to associate the plurality of metadata labels with an image of an electronic form

extract data from the tetragon; and

populate the electronic form with the extracted data.