SYSTEMS AND METHODS FOR CLASSIFYING OBJECTS IN DIGITAL IMAGES CAPTURED USING MOBILE DEVICES

• US 20140270536A1
• Filed: 03/13/2014
• Published: 09/18/2014
• Est. Priority Date: 03/13/2013
• Status: Active Grant

• Current Assignee: Kofax Incorporated
• Original Assignee: Kofax Incorporated
• Inventors: Amtrup, Jan W., Macciola, Anthony, Thompson, Stephen Michael, Ma, Jiyong
• Granted Patent: US 9,311,531 B2
• Time in Patent Office: Days
• US Class Current: 382/195
• CPC Class Codes: G06F 2218/08 Feature extraction
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06T 2207/20104 Interactive definition of region of interest [ROI]
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06V 30/40 Document-oriented image-based pattern recognition
• CPC Class Codes: G06V 30/418 Document matching, e.g. of document images

Abstract

A method includes receiving or capturing a digital image using a mobile device, and using a processor of the mobile device to: determine whether an object depicted in the digital image belongs to a particular object class among a plurality of object classes; determine one or more object features of the object based at least in part on the particular object class at least partially in response to determining the object belongs to the particular object class; build or select an extraction model based at least in part on the one or more determined object features; and extract data from the digital image using the extraction model. The extraction model excludes, and/or the extraction process does not utilize, optical character recognition (OCR) techniques. Related systems and computer program products are also disclosed.

Claims

1. A method, comprising:

receiving or capturing a digital image using a mobile device;

using a processor of the mobile device to;

determine whether an object depicted in the digital image belongs to a particular object class among a plurality of object classes;

determine one or more object features of the object based at least in part on the particular object class at least partially in response to determining the object belongs to the particular object class;

build or select an extraction model based at least in part on the one or more determined object features; and

extract data from the digital image using the extraction model.

2. The method as recited in claim 1, wherein the extraction model further comprises performing optical character recognition (OCR), wherein the OCR is performed on a selected portion of the digital image excluding one or more portions of the received or captured image.

3. The method as recited in claim 1, wherein the one or more object features comprise one or more regions of interest of the object.

4. The method as recited in claim 3, wherein the one or more of the regions of interest comprise one or more text characters.

5. The method as recited in claim 1, wherein the extraction model is built, wherein building the extraction model comprises:

mapping one or more of a feature vector, a list of feature vectors and a feature matrix to one or more of the object features; and

associating at least one metadata label with each mapped object feature.

6. The method as recited in claim 1, further comprising:

training the extraction model based on one or more additional object features of at least one additional object belonging to the object class.

7. The method as recited in claim 6, wherein the extraction model is trained using the processor of the mobile device;

and further comprising storing and/or exporting the trained extraction model.

8. The method as recited in claim 7, wherein the at least one additional object comprises at least four additional objects.

9. The method as recited in claim 1, further comprising:

performing OCR on one or more regions of the digital image corresponding to one or more of the object features and/or other object features.

10. The method as recited in claim 1, further comprising:

detecting one or more lines of text in the object.

11. The method as recited in claim 10, the detecting comprising:

projecting the digital image onto a single dimension;

determining a distribution of light and dark areas along the projection;

determining a plurality of dark pixel densities, each dark pixel density corresponding to a position along the projection;

determining whether each dark pixel density is greater than a probable text line threshold; and

designating each position as a text line upon determining the corresponding dark pixel density is greater than the probable text line threshold.

12. The method as recited in claim 11, further comprising:

projecting each color channel of the digital image onto a single channel along the single dimension.

13. A method, comprising:

receiving or capturing a digital image using a mobile device;

using a processor of the mobile device;

determining whether an object depicted in the digital image belongs to a particular object class among a plurality of object classes;

displaying the digital image on a display of the mobile device upon determining the object does not belong to any of the plurality of object classes;

receiving user input via the display of the mobile device, the user input identifying one or more regions of interest in the object;

building and/or selecting an extraction model based at least in part on the user input; and

extracting data from the digital image based at least in part on the extraction model.

14. The method as recited in claim 13, wherein the extracting further comprises performing optical character recognition (OCR), wherein the OCR is performed on a selected portion of the digital image excluding one or more portions of the received or captured image.

15. The method as recited in claim 13, wherein one or more of the regions of interest comprises one or more text characters.

16. The method as recited in claim 13, further comprising generating a new object class based at least in part on the user input.

17. The method as recited in claim 16, wherein the extraction model is configured to extract data from a plurality of objects belonging to the new object class.

18. The method as recited in claim 16, wherein the user input comprises designating a plurality of regions of interest in a plurality of objects.

19. The method as recited in claim 16, wherein building the extraction model comprises:

mapping one or more of a feature vector, a list of feature vectors and a feature matrix to one or more of the object features; and

associating at least one metadata label with each mapped object feature.

20. The method as recited in claim 16, further comprising:

training the extraction model based on one or more additional object features of at least one additional object belonging to the object class.

21. The method as recited in claim 20, wherein the at least one additional object comprises at least four additional objects.

22. The method as recited in claim 16, further comprising:

performing OCR on one or more regions of the digital image corresponding to one or more of the object features and/or other object features.

23. The method as recited in claim 16, further comprising:

detecting one or more lines of text in the object.

24. The method as recited in claim 16, the detecting comprising:

projecting the digital image onto a single dimension;

determining a distribution of light and dark areas along the projection;

determining a plurality of dark pixel densities, each dark pixel density corresponding to a position along the projection;

determining whether each dark pixel density is greater than a probable text line threshold; and

designating each position as a text line upon determining the corresponding dark pixel density is greater than the probable text line threshold.

25. The method as recited in claim 24, further comprising:

projecting each color channel of the digital image onto a single channel along the single dimension.

26. A computer program product comprising:

a computer readable storage medium having program code embodied therewith, the program code readable/executable by a processor to;

receive a digital image captured by a mobile device;

determine whether an object depicted in the digital image belongs to a particular object class among a plurality of object classes;

determine one or more object features of the object based at least in part on the particular object class upon determining the object belongs to the particular object class;

build or select an extraction model based at least in part on the one or more object features; and

extract data from the digital image based at least in part on the extraction model.