Automatically capturing and cropping image of check from video sequence for banking or other computing application

• US 9,652,690 B2
• Filed: 02/27/2015
• Issued: 05/16/2017
• Est. Priority Date: 02/27/2015
• Status: Active Grant

• Current Assignee: Lexmark International Incorporated (Ninestar Corporation)
• Original Assignee: Lexmark International Incorporated (Ninestar Corporation)
• Inventors: Simpson, Stacy Lee, Eid, Ahmed Hamad Mohamed, Cholewo, Tomasz Jan, Daniel, Stuart Willard
• Primary Examiner(s): Vu, Kim
• Assistant Examiner(s): PATEL, PINALBEN V
• Application Number:
  US14/633,955
• Publication Number:
  US 20160253573A1
• Time in Patent Office: 809 Days
• CPC Class Codes: G06Q 20/042 characterized in that the payment protocol involves at least one cheque
• CPC Class Codes: G06Q 20/0425 the cheque being electronic only
• CPC Class Codes: G06Q 20/108 Remote banking, e.g. home banking
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20132 Image cropping
• CPC Class Codes: G06T 2207/30176 Document
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/19013 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching specially adapted for image segmentation G06T7/10; specially adapted for the analysis of motion G06T7/20; specially adapted for image alignment G06T7/30; specially adapted for the calculation of depth from stereo images G06T7/50; specially adapted for position determination G06T7/70
• CPC Class Codes: G06V 30/416 Extracting the logical structure, e.g. chapters, sections or page numbers; Identifying elements of the document, e.g. authors

Abstract

A mobile device with camera automatically captures an image of a check from a video sequence. A computing application assesses quality metrics of a frame of the video and, if acceptable, initiates capture of the check in that frame without user selection. Metrics include an aspect ratio of the check, image quality of the routing transit symbols that delineate a routing transit number on a MICR line of the check, distances between the routing transit symbols and to an edge of the check, recognition of digits of the routing transit number, checksum of the routing transit numbers, and image sharpness. Other embodiments note cropping of the check from the background of the image, properly orienting the check for viewing, and providing color coded visual feedback to users about the quality of the image frame about the check, to name a few.

Claims

1. A method for automatically capturing an image of a check from a video sequence of images of the check without user selection, the video sequence having a plurality of frames, comprising:

determining an aspect ratio of the check;

finding in a single frame of the plurality of frames of the video sequence both routing transit symbols that delineate a routing transit number on a magnetic ink character recognition (MICR) line of the check;

matching to a template each of the routing transit symbols;

determining a first distance from one of the routing transit symbols to the other of the routing transit symbols and a second distance from said one of the routing transit symbols to an edge of the check;

calculating a ratio of the first distance and the second distance;

comparing each digit of the routing transit number to templates of digits to determine values of the routing transit number;

capturing the image of the check from the single frame; and

determining if the aspect ratio, a worst match of the matching to the template, the ratio of the first and second distances, and the comparing said each digit of the routing transit number fall within a predetermined range of acceptable values before said capturing the image of the check from the single frame.

2. The method of claim 1, further including cropping the check from the single frame, including eliminating background pixels.

3. The method of claim 1, further including assessing stability of the video sequence.

4. The method of claim 1, further including receiving for processing another frame of the video sequence.

5. The method of claim 1, further including determining an orientation of the check in the single frame.

6. The method of claim 5, wherein the finding said both routing transit symbols occurs relative to only a bottom or top portion of the check depending on said orientation.

7. The method of claim 5, further including rotating the check for a user to view from left-to-right if the orientation of the check is upside-down as users read from left-to-right.

8. The method of claim 1, further including calculating a checksum of the routing transit number.

9. The method of claim 1, further including calculating a sharpness of the image of the check in the single frame.

10. The method of claim 1, further including calculating a possible bounding border of the check in the single frame.

11. The method of claim 10, further including displaying to a user the possible bounding border with color-coding according to a calculated quality of the possible bounding border.

12. The method of claim 1, wherein the comparing said each digit of the routing transit number to the templates of digits to determine values of the routing transit number further includes comparing said each digit of the routing transit number to templates of digits ranging from values 0 to 9 and selecting a best matching image template therefore.

13. The method of claim 1, further including calculating an image width to an image height of the check in the single frame to said determine the aspect ratio, wherein an acceptable range of aspect ratios ranges from about 1.7 to about 3.1.

14. The method of claim 1, wherein the determining said first and second distances further includes determining an (x, y) position coordinate of a bounding box about said both routing transit symbols.

15. The method of claim 14, further including determining said first and second distances from said (x, y) position coordinate of the bounding box about said both routing transit symbols.

16. The method of claim 8, wherein the calculating the checksum further includes executing a Modulo 10 of weighted sums of the determined values of the routing transit number.