Image analysis and processing pipeline with real-time feedback and autocapture capabilities, and visualization and configuration system

• US 10,691,966 B2
• Filed: 03/29/2019
• Issued: 06/23/2020
• Est. Priority Date: 08/20/2018
• Status: Active Grant

• Current Assignee: Capital One Services LLC (Capital One Financial Corporation)
• Original Assignee: Capital One Services LLC (Capital One Financial Corporation)
• Inventors: Pribble, Jason, Jarvis, Daniel Alan, Capurso, Nicholas, Cerney, Eric
• Primary Examiner(s): Rahmjoo, Manuchehr
• Application Number:
  US16/369,807
• Publication Number:
  US 20200057901A1
• Time in Patent Office: 452 Days
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/20104 Interactive definition of region of interest [ROI]
• CPC Class Codes: G06T 2207/20132 Image cropping
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06V 10/141 Control of illumination
• CPC Class Codes: G06V 10/242 by image rotation, e.g. by 90 degrees
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/30 Noise filtering
• CPC Class Codes: G06V 10/443 by matching or filtering
• CPC Class Codes: G06V 10/993 Evaluation of the quality of the acquired pattern
• CPC Class Codes: G06V 30/414 Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text

Abstract

A device may capture a plurality of preview frames of a document, and for each preview frame of the plurality of preview frames, process the preview frame to identify an object in the preview frame. Processing the preview frame may include converting the preview frame into a grayscale image, generating a blurred image based on the grayscale image, detecting a plurality of edges in the blurred image, defining at least one bounding rectangle based on the plurality of edges, and determining an outline of the object based on the at least one bounding rectangle. The device may determine whether a value of an image parameter, associated with the one or more preview frames, satisfies a threshold, and provide feedback to a user of the device, or automatically capture an image of the document, based on determining whether the value of the image parameter satisfies the threshold.

Claims

1. A method, comprising:

processing, by a device, a preview frame to identify an object,

the preview frame being obtained from a camera of the device and provided for display;

determining, by the device and based on identifying the object, whether a value of an image parameter, associated with the preview frame, satisfies a first threshold,

the image parameter relating to a distance between the device and the object;

applying, by the device, a filter to the preview frame for glare detection;

determining, by the device and based on applying the filter, whether a value relating to glare satisfies a second threshold; and

selectively;

providing, by the device, first feedback based on determining that the value of the image parameter does not satisfy the first threshold,

the first feedback including an instruction to perform an action with respect to at least one of the device or the object;

providing, by the device, second feedback based on determining that the value relating to glare does not satisfy the second threshold;

or

automatically capturing, by the device, an image of the object based on determining that the value of the image parameter satisfies the first threshold and the value relating to glare satisfies the second threshold.

2. The method of claim 1, where the preview frame is one of a plurality of preview frames obtained in real-time or near real-time.

3. The method of claim 1, where processing the preview frame comprises:

converting the preview frame into a grayscale image.

4. The method of claim 3, where processing the preview frame further comprises at least one of:

blurring the grayscale image,

applying another filter to the grayscale image,

detecting shapes in the preview frame, or

defining boundaries in the preview frame.

5. The method of claim 1, further comprising:

determining whether a value relating to contrast satisfies a third threshold; and

providing third feedback based on determining that the value relating to contrast does not satisfy the third threshold.

6. The method of claim 1, further comprising:

determining an outline of the object; and

causing a boundary box, corresponding to the outline, to be displayed.

7. The method of claim 1, further comprising:

determining an outline of the object;

causing a boundary box, corresponding to the outline, to be displayed; and

periodically updating, based on movements of the device, at least one of;

a position of the boundary box,

an orientation of the boundary box, or

a size of the boundary box.

8. A device, comprising:

one or more memories; and

one or more processors, communicatively coupled to the one or more memories, configured to;

process a preview frame to identify an object,

the preview frame being obtained from a camera of the device and provided for display;

determine, based on identifying the object, whether a value of an image parameter, associated with the preview frame, satisfies a first threshold,

the image parameter relating to a distance between the device and the object;

apply a filter to the preview frame for glare detection;

determine, based on applying the filter, whether a value relating to glare satisfies a second threshold; and

selectively;

provide first feedback based on determining that the value of the image parameter does not satisfy the first threshold,

the first feedback including an instruction to perform an action with respect to at least one of the device or the object;

provide second feedback based on determining that the value relating to glare does not satisfy the second threshold;

or

automatically capture an image of the object based on determining that the value of the image parameter satisfies the first threshold and the value relating to glare satisfies the second threshold.

9. The device of claim 8, where the value of the image parameter is a ratio based on a width of an outline associated with the object, and a width of a display.

10. The device of claim 8, where the one or more processors, when providing the first feedback, are to:

provide the first feedback in real-time or near-real time.

11. The device of claim 8, where the one or more processors, when automatically capturing the image of the object based on determining that the value of the image parameter satisfies the first threshold and the value relating to glare satisfies the second threshold, are to:

automatically capture the image of the object based on determining that multiple values, associated with image parameters for multiple preview frames, satisfy corresponding first thresholds and based on determining that the value relating to glare satisfies the second threshold.

12. The device of claim 8, where the value of the image parameter is a first value of the image parameter;

where the preview frame is a first preview frame; and

where the one or more processors are further to;

perform an analysis of the image based on capturing the image;

determine, based on performing the analysis, whether a second value of the image parameter, associated with the image, satisfies the first threshold; and

process a second preview frame based on determining that the second value of the image parameter does not satisfy the first threshold.

13. The device of claim 8, where the value of the image parameter is a first value of the image parameter;

and

where the one or more processors are further to;

perform an analysis of the image based on capturing the image;

determine, based on performing the analysis, whether a second value of the image parameter, associated with the image, satisfies the first threshold; and

crop, based on determining that the second value of the image parameter satisfies the first threshold, the object from the image.

14. The device of claim 8, where the value of the image parameter is a first value of the image parameter;

and

where the one or more processors are further to;

perform an analysis of the image based on capturing the image;

determine, based on performing the analysis, whether a second value of the image parameter, associated with the image, satisfies the first threshold; and

provide, based on determining that the second value of the image parameter satisfies the first threshold, the image to a server.

15. A non-transitory computer-readable medium storing instructions, the instructions comprising:

one or more instructions that, when executed by one or more processors, cause the one or more processors to;

process a preview frame to identify an object,

the preview frame being obtained from a camera of a device and provided for display;

determine, based on identifying the object, whether a value of an image parameter, associated with the preview frame, satisfies a first threshold,

the image parameter relating to a distance between the device and the object;

apply a filter to the preview frame for glare detection;

determine, based on applying the filter, whether a value relating to glare satisfies a second threshold; and

selectively;

provide first feedback based on determining that the value of the image parameter does not satisfy the first threshold,

the first feedback including an instruction to perform an action with respect to at least one of the device or the object;

provide second feedback based on determining that the value relating to glare does not satisfy the second threshold;

or

automatically capture an image of the object based on determining that the value of the image parameter satisfies the first threshold and the value relating to glare satisfies the second threshold.

16. The non-transitory computer-readable medium of claim 15, where the one or more instructions, when executed by the one or more processors, further cause the one or more processors to:

provide, for display, at least one option for adjusting a configuration parameter associated with the image.

17. The non-transitory computer-readable medium of claim 15, where the one or more instructions, that cause the one or more processors to provide the first feedback, cause the one or more processors to:

provide the first feedback in real-time or near-real time.

18. The non-transitory computer-readable medium of claim 15, where the one or more instructions, that cause the one or more processors to automatically capture the image of the object based on determining that the value of the image parameter satisfies the first threshold and the value relating to glare satisfies the second threshold, cause the one or more processors to:

automatically capture the image of the object based on determining that multiple values, associated with image parameters for multiple preview frames, satisfy corresponding first thresholds and based on the value relating to glare satisfies the second threshold.

19. The non-transitory computer-readable medium of claim 15, where the value of the image parameter is a first value of the image parameter;

where the preview frame is a first preview frame; and

where the one or more instructions, when executed by the one or more processors, further cause the one or more processors to;

perform an analysis of the image based on capturing the image;

determine, based on performing the analysis, whether a second value of the image parameter, associated with the image, satisfies the first threshold; and

process a second preview frame based on determining that the second value of the image parameter does not satisfy the first threshold.

20. The non-transitory computer-readable medium of claim 15, where the value of the image parameter is a first value of the image parameter;

and

where the one or more instructions, when executed by the one or more processors, further cause the one or more processors to;

perform an analysis of the image based on capturing the image;

determine, based on performing the analysis, whether a second value of the image parameter, associated with the image, satisfies the first threshold; and

crop, based on determining that the second value of the image parameter satisfies the first threshold, the object from the image.