System and method for assessing usability of captured images

• US 9,870,609 B2
• Filed: 06/03/2016
• Issued: 01/16/2018
• Est. Priority Date: 06/03/2016
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Inventors: Kompalli, Pramod Sankar, Sharma, Arjun, Howe, Richard L.
• Primary Examiner(s): Chawan, Sheela C
• Application Number:
  US15/172,526
• Publication Number:
  US 20170352143A1
• Time in Patent Office: 592 Days
• Field of Search: 382157, 382162, 382190, 382255
• CPC Class Codes: B64U 10/13 Flying platforms
• CPC Class Codes: B64U 10/14 with four distinct rotor axes, e.g. quadcopters
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 18/24133 Distances to prototypes
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 10/993 Evaluation of the quality of the acquired pattern
• CPC Class Codes: G06V 30/19173 Classification techniques

Abstract

A system estimates quality of a digital image by accessing a corpus of digital images of one or more subjects, such as a facet of a property. The system will receive, for at least a subset of the corpus, an indicator that one or more patches of each image in the subset is out of focus. The system will train a classifier by obtaining a feature representation of each pixel in each image, along with a focus value that represents an extent to which each pixel in the image is in focus or out of focus. The system will use the classifier to analyze pixels of a new digital image and assess whether each analyzed pixel in the new digital image is in focus or out of focus. The system may use the image to assess whether an incident occurred, such as storm-related damage to the property.

Claims

1. A system for estimating image quality of a digital image, the system comprising:

a processing device; and

a computer-readable medium containing programming instructions that are configured to cause the processing device to;

access a corpus of digital images of one or more subjects taken by one or more image capturing devices,

receive, for at least a subset of the corpus of digital images, an indicator that one or more patches of each image in the subset is out of focus,

train a classifier by obtaining and storing in a data store;

an n-dimensional feature representation of each pixel in each image; and

a focus value that represents an extent to which each pixel in the image is in focus or out of focus, and

use the classifier to analyze pixels of a new digital image and output an assessment of whether each analyzed pixel in the new digital image is in focus or out of focus.

2. The system of claim 1, further comprising additional programming instructions that are configured to cause the processing device to:

analyze the assessments prepared for each of a plurality of new digital images;

for each new digital image that has been assessed, determine whether the assessment for the new digital image satisfies a usability criterion; and

if an assessment for a new digital image satisfies a usability criterion, use that new digital image to determine whether that new digital image depicts damage that corresponds to a particular incident, otherwise not do so.

3. The system of claim 1, wherein the usability criterion comprises:

a requirement that all pixels of the new digital image have at least a threshold focus value;

a requirement that an average focus value for all pixels in the new digital image be equal to or above a threshold focus value;

a requirement that no more than a threshold number of pixels in the new digital image have a focus value that is less than a minimum acceptable focus value;

or

a requirement that no more than a first threshold number of pixels in the new digital image have a focus value that is less than a first minimum acceptable focus value, and that no more than a second threshold number of pixels in the new digital image have a focus value that is less than a second minimum acceptable focus value.

4. The system of claim 1, wherein the instructions to obtain the n-dimensional feature representation of each pixel in each image comprise instructions to use a Convolutional Neural Network to:

apply n filters to a neighborhood around each pixel;

determine whether each filtered neighborhood matches a kernel;

assign a value at or near 1 to the pixels of any filtered neighborhood that matches the kernel;

assign a value at or near zero to the pixels of any filtered neighborhood that does not match the kernel;

use the assigned values to determine the focus value.

5. The system of claim 4, wherein the programming instructions that are configured to cause the processing device to use the assigned values to determine the focus value comprise instructions to do so using a two-layer neural network with sigmoid outputs.

6. The system of claim 1, wherein the processing device is part of a mobile electronic device having a display and an imaging device, and the programming instructions are also configured to cause the processing device to:

cause the display to output a captured image while the imaging device is capturing additional images of one or more subjects; and

with each captured image, cause the mobile electronic device to output the assessment of whether each analyzed pixel in captured image is in focus or out of focus.

7. The system of claim 1, further comprising:

an imaging device; and

additional programming instructions configured to cause the processing device to;

determine whether the assessment for the new digital image satisfies a usability criterion; and

if the assessment does not satisfy the usability criterion, automatically cause the imaging device to capture an additional new digital image.

8. The system of claim 1, further comprising:

an aerial drone that includes an imaging device and a transceiver; and

additional programming instructions that are configured to cause the processing device to;

determine whether the assessment for the new digital image satisfies a usability criterion; and

if the assessment does not satisfy the usability criterion, cause the imaging device of the aerial drone to capture an additional new digital image.

9. The system of claim 1, further comprising additional programming instructions that are configured to cause the processing device to:

if the assessment does not satisfy the usability criterion, prompt a user of the mobile electronic device to enter a navigational command for the aerial drone; and

in response to receiving the navigational command, use a transmitter of the mobile electronic device to send the navigational command to the aerial drone.

10. A system for estimating image quality of a digital image, the system comprising:

a processing device;

an imaging device; and

a computer-readable medium containing programming instructions that are configured to cause the processing device to;

access a corpus of digital images of one or more subjects taken by one or more image capturing devices,

receive, for at least a subset of the corpus of digital images, an indicator that one or more patches of each image in the subset is out of focus,

train a classifier by obtaining and storing in a data store;

an n-dimensional feature representation of each pixel in each image; and

a focus value that represents an extent to which each pixel in the image is in focus or out of focus, and

use the classifier to analyze pixels of a new digital image and generate an assessment of whether each analyzed pixel in the new digital image is in focus or out of f determine whether the assessment for the new digital image satisfies a usability criterion; and

if the assessment satisfies the usability criterion, use the new digital image to determine whether that new digital image depicts damage that corresponds to a particular incident, otherwise automatically cause the imaging device to capture an additional new digital image.

11. The system of claim 10, wherein the usability criterion comprises:

a requirement that all pixels of the new digital image have at least a threshold focus value;

a requirement that an average focus value for all pixels in the new digital image be equal to or above a threshold focus value;

a requirement that no more than a threshold number of pixels in the new digital image have a focus value that is less than a minimum acceptable focus value;

or

a requirement that no more than a first threshold number of pixels in the new digital image have a focus value that is less than a first minimum acceptable focus value, and that no more than a second threshold number of pixels in the new digital image have a focus value that is less than a second minimum acceptable focus value.

12. The system of claim 10, further comprising:

an aerial drone that includes the imaging device and a transceiver; and

additional programming instructions that are configured to cause the processing device to;

if the assessment does not satisfy the usability criterion, prompt a user of the mobile electronic device to enter a navigational command for the aerial drone, and

in response to receiving the navigational command, use a transmitter of the mobile electronic device to send the navigational command to the aerial drone.

13. A method of estimating image quality of a digital image, the method comprising:

by a processing device, executing programming instructions that cause the processing device to;

access a corpus of digital images of one or more subjects taken by one or more image capturing devices,

receive, for at least a subset of the corpus of digital images, an indicator that one or more patches of each image in the subset is out of focus,

train a classifier by obtaining and storing in a data store;

an n-dimensional feature representation of each pixel in each image; and

a focus value that represents an extent to which each pixel in the image is in focus or out of focus, and

use the classifier to analyze pixels of a new digital image and output an assessment of whether each analyzed pixel in the new digital image is in focus or out of focus.

14. The method of claim 13, further comprising, by the processing device:

analyzing the assessments prepared for each of a plurality of new digital images;

for each new digital image that has been assessed, determining whether the assessment for the new digital image satisfies a usability criterion; and

if an assessment for a new digital image satisfies a usability criterion, using that new digital image to determine whether that new digital image depicts damage that corresponds to a particular incident, otherwise not doing so.

15. The method of claim 13, wherein the usability criterion comprises:

a requirement that all pixels of the new digital image have at least a threshold focus value;

a requirement that an average focus value for all pixels in the new digital image be equal to or above a threshold focus value;

a requirement that no more than a threshold number of pixels in the new digital image have a focus value that is less than a minimum acceptable focus value;

or

a requirement that no more than a first threshold number of pixels in the new digital image have a focus value that is less than a first minimum acceptable focus value, and that no more than a second threshold number of pixels in the new digital image have a focus value that is less than a second minimum acceptable focus value.

16. The method of claim 13, wherein obtaining the n-dimensional feature representation of each pixel in each image comprises using a Convolutional Neural Network to:

apply n filters to a neighborhood around each pixel;

determine whether each filtered neighborhood matches a kernel;

assign a value at or near 1 to the pixels of any filtered neighborhood that matches the kernel;

assign a value at or near zero to the pixels of any filtered neighborhood that does not match the kernel;

use the assigned values to determine the focus value.

17. The method of claim 13, further comprising, by the processing device:

causing a display of a mobile electronic device to output a captured image while the mobile electronic device is capturing additional images of one or more subjects; and

with each captured image, causing the mobile electronic device to output the assessment of whether each analyzed pixel in captured image is in focus or out of focus.

18. The method of claim 13, further comprising, by the processing device:

determining whether the assessment for the new digital image satisfies a usability criterion; and

if the assessment does not satisfy the usability criterion, automatically causing an imaging device to capture an additional new digital image.

19. The method of claim 13, wherein the imaging device is a component of an aerial done.

20. The method of claim 19, further comprising:

if the assessment does not satisfy the usability criterion, prompting a user of a mobile electronic device to enter a navigational command for the aerial drone; and

in response to receiving the navigational command, using a transmitter of the mobile electronic device to send the navigational command to the aerial drone.