Systems and methods for detecting and classifying objects in video captured using mobile devices

• US 9,253,349 B2
• Filed: 08/29/2014
• Issued: 02/02/2016
• Est. Priority Date: 05/03/2013
• Status: Active Grant

• Current Assignee: Kofax Incorporated
• Original Assignee: Kofax Incorporated
• Inventors: Amtrup, Jan W., Ma, Jiyong, Macciola, Anthony
• Primary Examiner(s): Safaipour, Houshang
• Application Number:
  US14/473,950
• Publication Number:
  US 20140368890A1
• Time in Patent Office: 522 Days
• Field of Search: 358/1.15, 358/1.13, 358/473, 358/474
• US Class Current: 1/1
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06V 2201/09 Recognition of logos
• CPC Class Codes: G06V 30/224 of printed characters having additional code marks or containing code marks
• CPC Class Codes: H04N 1/00251 with an apparatus for taking photographic images, e.g. a camera
• CPC Class Codes: H04N 1/00307 with a mobile telephone apparatus constructional features of portable telephone sets H04M1/0202; mobile radio systems H04W84/00
• CPC Class Codes: H04N 1/00801 according to characteristics of the original
• CPC Class Codes: H04N 1/19594 using a television camera or a still video camera
• CPC Class Codes: H04N 2201/0032 where the still picture apparatus acts as the slave
• CPC Class Codes: H04N 2201/0075 by a user operated remote control device, e.g. receiving instructions from a user via a computer terminal or mobile telephone handset
• CPC Class Codes: H04N 2201/0084 Digital still camera
• CPC Class Codes: H04N 2201/0458 Additional arrangements for improving or optimising scanning resolution or quality
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 5/265 Mixing

Abstract

A method includes capturing plural frames of video data using a mobile device. The frames are analyzed to determine whether any depict an object exhibiting one or more defining characteristics, and if so, whether those frame(s) depicting the object also satisfy one or more predetermined quality control criteria. If one or more of the frames depict the object and also satisfy the one or more predetermined quality control criteria, the method further includes automatically capturing an image of the object. Exemplary defining characteristics are specified for various types of object, particularly objects comprising documents. Related systems and computer program products are also disclosed. The presently disclosed techniques and systems represent translational developments across the fields of image processing and business process management. Improved analytical techniques enable processing of image captured using cameras rather than traditional scanner technology, and facilitate distribution, tracking and analysis of documents and information throughout business processes.

Claims

1. A method, comprising:

capturing video data using a mobile device, the video data comprising a plurality of frames;

determining whether one or more of the frames depict an object exhibiting one or more defining characteristics;

determining whether one or more of the frame(s) determined to depict the object also satisfy one or more predetermined quality control criteria; and

in response to determining one or more of the frames depict the object and also satisfy the one or more predetermined quality control criteria, automatically capturing an image of the object.

2. The method as recited in claim 1, further comprising initiating a business process in response to determining the one or more of the frames depict the object and also satisfy the one or more predetermined quality control criteria, wherein the business process relies at least in part on the automatically captured image.

3. The method as recited in claim 1, wherein the image is characterized by a resolution higher than a resolution of any of the plurality of frames.

4. The method as recited in claim 1, wherein the defining characteristics comprise one or more of:

an object shape;

an object contour;

one or more object dimensions; and

an object color profile.

5. The method as recited in claim 1, wherein determining whether one or more of the frames depict the object comprises performing optical character recognition (OCR).

6. The method as recited in claim 1, wherein the object is a document.

7. The method as recited in claim 6, wherein the defining characteristics comprise one or more of:

textual information depicted on the document;

a photograph depicted on the document;

a barcode depicted on the document;

a logo depicted on the document;

an emblem depicted on the document;

a seal depicted on the document; and

an aspect ratio of the document.

8. The method as recited in claim 6, wherein the document corresponds to a loan application.

9. The method as recited in claim 8, wherein the defining characteristics comprise the document depicting one or more of:

a customer name;

a guarantor name;

a customer address;

a date;

an account number;

a loan number;

a digital signature;

a handwritten signature; and

a photograph.

10. The method as recited in claim 6, wherein the document comprises one or more of:

a receipt;

an expense report;

a credit card; and

a check.

11. The method as recited in claim 10, wherein the defining characteristics comprise the document depicting one or more of:

a credit card number;

a name;

an expense amount; and

an order number.

12. The method as recited in claim 6, wherein the document comprises a medical record.

13. The method as recited in claim 6, wherein the document corresponds to an insurance claim.

14. The method as recited in claim 13, wherein the defining characteristics comprise the document depicting one or more of:

a photograph of an automobile;

a photograph of an accident report;

a photograph of a proof of insurance document;

a photograph of a driver'"'"'s license;

a photograph of a quote; and

a signature.

15. The method as recited in claim 6, wherein the document corresponds to a delivery.

16. The method as recited in claim 15, wherein the defining characteristics comprise the document depicting one or more of:

a signature;

a proof of delivery;

a package number;

a photograph of a package; and

a declaration of package contents.

17. The method as recited in claim 6, wherein the document corresponds to an educational program application.

18. The method as recited in claim 17, wherein the defining characteristics comprise the document depicting one or more of:

an applicant name;

a photograph of an applicant; and

a student identification document.

19. A system, comprising a processor and logic executable by the processor, the logic being configured to cause the processor to:

capture video data using a mobile device, the video data comprising a plurality of frames;

determine whether one or more of the frames depict an object exhibiting one or more defining characteristics;

determine whether one or more of the frame(s) determined to depict the object also satisfy one or more predetermined quality control criteria; and

in response to determining one or more of the frames depict the object and also satisfy the one or more predetermined quality control criteria, automatically capture an image of the object.

20. A computer program product, comprising a non-transitory computer readable storage medium having embodied therewith compute readable program code configured to cause a processor to:

capture video data using a mobile device, the video data comprising a plurality of frames;

determine whether one or more of the frames depict an object exhibiting one or more defining characteristics;

determine whether one or more of the frame(s) determined to depict the object also satisfy one or more predetermined quality control criteria; and

in response to determining one or more of the frames depict the object and also satisfy the one or more predetermined quality control criteria, automatically capture an image of the object.