Facial image quality assessment

• US 9,047,538 B2
• Filed: 05/10/2013
• Issued: 06/02/2015
• Est. Priority Date: 06/15/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Nechyba, Michael Christian, Sipe, Michael Andrew
• Primary Examiner(s): Giles, Nicholas
• Assistant Examiner(s): Tissire, Abdelaaziz
• Application Number:
  US13/891,843
• Publication Number:
  US 20130336527A1
• Time in Patent Office: 753 Days
• Field of Search: 348/222.1
• US Class Current: 1/1
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06V 10/507 Summing image-intensity values; Histogram projection analysis
• CPC Class Codes: G06V 10/993 Evaluation of the quality of the acquired pattern
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: H04M 2250/00 Details of telephonic subscriber devices
• CPC Class Codes: H04N 23/40 Circuit details for pick-up tubes
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/634 Warning indications
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/673 based on contrast or high frequency components of image signals, e.g. hill climbing method
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/71 Circuitry for evaluating the brightness variation

Abstract

An example method includes capturing, by a camera of a mobile computing device, an image, determining whether the image includes a representation of at least a portion of a face, and, when the image includes the representation of at least the portion of the face, analyzing characteristics of the image. The characteristics include at least one of a tonal distribution of the image that is associated with a darkness-based mapping of a plurality of pixels of the image, and a plurality of spatial frequencies of the image that are associated with a visual transition between adjacent pixels of the image. The method further includes classifying, by the mobile computing device, a quality of the image based at least in part on the analyzed characteristics of the image.

Claims

1. A method comprising:

capturing, by a camera coupled to a computing device, an image;

determining whether the image includes a representation of at least a portion of a face;

responsive to determining that the image includes the representation of at least the portion of the face;

comparing one or more spatial frequencies of the image to corresponding spatial frequencies of one or more previously classified images of a plurality of previously classified images, wherein each previously classified image of the plurality of previously classified images is classified as one of a good quality and a poor quality based on the spatial frequencies the respective previously classified image, and wherein each spatial frequency of the one or more spatial frequencies is associated with a visual transition between two or more adjacent pixels of the image; and

classifying the image as a good quality image or a poor quality image based on the comparison.

2. The method of claim 1 further comprising:

determining, based on the comparison of the one or more spatial frequencies of the image to the corresponding spatial frequencies of the one or more previously classified images of the plurality of previously classified images, whether the image includes less than or equal to a threshold amount of motion blur;

responsive to determining that the image satisfies the threshold amount of motion blur, classifying the image as a good quality image; and

responsive to determining that the image does not satisfy the threshold amount of motion blur, classifying the image as a poor quality image.

3. The method of claim 1,

wherein the plurality of previously classified images includes a first portion of previously classified images classified as good quality images and a second portion of previously classified images classified as poor quality images,

wherein comparing the one or more spatial frequencies of the image to the corresponding spatial frequencies of the one or more previously classified images of the plurality of previously classified images comprises:

computing a first difference between the one or more spatial frequencies of the image and the corresponding spatial frequencies of each of the first portion of previously classified images; and

computing a second difference between the one or more spatial frequencies of the image and the corresponding spatial frequencies of each of the second portion of previously classified images, and

wherein classifying the image as a good quality image or a poor quality image based on the comparison comprises classifying the image as a good quality image or a poor quality image based on a comparison of the first computed difference to the second computed difference.

4. The method of claim 1, further comprising:

identifying one or more target regions within the image;

associating each spatial frequency of the plurality of spatial frequencies with a target region of the identified one or more target regions; and

computing a frequency content associated with each target region from the one or more target regions based at least on one or more spatial frequencies associated with each target region.

5. The method of claim 4, further comprising:

determining at least one energy value associated with the frequency content associated with each target region, wherein the at least one energy value is associated with a high frequency band;

comparing the at least one energy value to a threshold energy value, wherein the threshold energy value is associated with a high frequency band of a prototype vector;

if the at least one energy value is less than or equal to the threshold energy value, classifying the image as a high motion blur image; and

if the at least one energy value is greater than the threshold energy value, classifying the image as a low motion blur image.

6. The method of claim 4, further comprising:

generating a frequency representation of the image at least in part by applying a wavelet transform to the frequency content associated with each target region, wherein applying the wavelet transform comprises expressing two or more spatial frequencies of the frequency content as two or more corresponding wavelets; and

computing a transition between the two or more corresponding wavelets.

7. The method of claim 4,

wherein the one or more target regions comprise a representation of an eye of the face, and

wherein computing the frequency content associated with each target region comprises computing a frequency content based on one or more spatial frequencies associated with a representation of an iris and a representation of a sclera of the eye, the method further comprising:

determining at least one energy value associated with the frequency content associated with the representation of the eye, wherein the at least one energy value is associated with a very high frequency band;

comparing the at least one energy value to a threshold energy value, wherein the threshold energy value is associated with the very high frequency band of a prototype vector;

if the at least one energy value is less than or equal to the threshold energy value, classifying the image as a high motion blur image; and

if the at least one energy value is greater than the threshold energy value, classifying the image as a low motion blur image.

8. The method of claim 7, wherein the very high frequency band is associated with frequency values that are greater than 0.8.

9. The method of claim 4,

wherein the one or more target regions comprise a representation of a mouth region of the face, and

wherein computing the frequency content associated with each target region comprises computing a frequency content based on one or more spatial frequencies associated with the mouth region, the method further comprising:

determining at least one energy value associated with the frequency content associated with the representation of the mouth region, wherein the at least one energy value is associated with a high frequency band;

comparing the at least one energy value to a threshold energy value, wherein the threshold energy value is associated with the high frequency band of a prototype vector;

if the at least one energy value does not satisfy the threshold energy value, classifying the image as a high motion blur image; and

if the at least one energy value satisfies the threshold energy value, classifying the image as a low motion blur image.

10. The method of claim 9, wherein the high frequency band is associated with frequency values that are greater than 0.7.

11. The method of claim 1, further comprising:

responsive to classifying the image as a poor quality image based on the comparison, outputting a notification that indicates motion blur as a predicted cause of the poor quality image.

12. A device comprising:

at least one programmable processor; and

at least one camera operable to capture an image;

wherein the at least one processor is configured to;

determine whether the image includes a representation of at least a portion of a face; and

responsive to determining that the image includes the representation of at least the portion of the face;

compare one or more spatial frequencies of the image to corresponding spatial frequencies of each previously classified image of a plurality of previously classified images, wherein each previously classified image of the plurality of previously classified images is classified as one of a good quality and a poor quality based on the spatial frequencies the respective previously classified image, and wherein each spatial frequency of the one or more spatial frequencies is associated with a visual transition between two or more adjacent pixels of the image; and

classify the image as a good quality image or a poor quality image based on the comparison.

13. The device of claim 12, further comprising at least one output device configured to, responsive to determining that the quality of the image is classified as poor quality, output a notification that the image is a poor quality image.

14. The device of claim 12, wherein the captured image is a first captured image, and wherein the at least one programmable processor is further configured to:

responsive to determining that the quality of the image is classified as poor quality, send an instruction to the camera to capture a second image.

15. The device of claim 14, further comprising an output device, wherein the at least one programmable processor is further configured to:

send an instruction to the output device to output a prompt to a user, wherein the prompt indicates the capture of the second image.

16. The device of claim 12, wherein the at least one programmable processor is further configured to:

determine, based on comparing one or more spatial frequencies of the image to corresponding spatial frequencies of each previously classified image of a plurality of previously classified images, whether the image includes less than or equal to a threshold amount of motion blur;

responsive to determining that the image satisfies the threshold amount of motion blur, classify the image as a good quality image; and

responsive to determining that the image does not satisfy the threshold amount of motion blur, classify the image as a poor quality image.

17. The device of claim 12,

wherein the plurality of previously classified images includes a first portion of previously classified images classified as good quality images and a second portion of previously classified images classified as poor quality images,

wherein, to compare the one or more spatial frequencies of the image to the corresponding spatial frequencies of each previously classified image of the plurality of previously classified images, the at least one programmable processor is configured to:

compute a first difference between the one or more spatial frequencies of the image and the corresponding spatial frequencies of the first portion of previously classified images; and

compute a second difference between the one or more spatial frequencies of the image and the corresponding spatial frequencies of the second portion of previously classified images, and

wherein, to classify the image as a good quality image or a poor quality image based on the comparison, the at least one programmable processor is configured to classify the image as a good quality image or a poor quality image based on a comparison of the first computed difference to the second computed difference.

18. The device of claim 12, wherein the at least one programmable processor is further configured to:

identify one or more target regions within the image;

associate each spatial frequency of the plurality of spatial frequencies with a target region of the identified one or more target regions; and

compute a frequency content associated with each target region from the one or more target regions based at least on one or more spatial frequencies associated with each target region.

19. A computer-readable storage device comprising instructions that, when executed, cause at least one processor of a computing device to:

capture, using a camera operatively coupled to the computing device, an image;

determine whether the image includes a representation of at least a portion of a face;

responsive to determining that the image includes the representation of at least the portion of the face;

compare one or more spatial frequencies of the image to corresponding spatial frequencies of each previously classified image of a plurality of previously classified images, wherein each previously classified image of the plurality of previously classified images is classified as one of a good quality and a poor quality based on the spatial frequencies the respective previously classified image, and wherein each spatial frequency of the one or more spatial frequencies is associated with a visual transition between two or more adjacent pixels of the image; and

classify the image as a good quality image or a poor quality image based on the comparison.

20. The computer-readable storage device of claim 19, further comprising instructions that, when executed, cause the at least one processor of the computing device to:

determine, based on comparing one or more spatial frequencies of the image to corresponding spatial frequencies of each previously classified image of a plurality of previously classified images, whether the image includes less than or equal to a threshold amount of motion blur;

responsive to determining that the image satisfies the threshold amount of motion blur, classify the image as a good quality image; and

responsive to determining that the image does not satisfy the threshold amount of motion blur, classify the image as a poor quality image.

21. The computer-readable storage device of claim 19,

wherein the plurality of previously classified images includes a first portion of previously classified images classified as good quality images and a second portion of previously classified images classified as poor quality images,

wherein the instructions that cause the processor of the computing device to compare the one or more spatial frequencies of the image to the corresponding spatial frequencies of each previously classified image of the plurality of previously classified images comprise instructions that, when executed, cause the processor of the computing device to:

compute a first difference between the one or more spatial frequencies of the image and the corresponding spatial frequencies of the first portion of previously classified images; and

compute a second difference between the one or more spatial frequencies of the image and the corresponding spatial frequencies of the second portion of previously classified images, and

wherein the instructions that cause the processor of the computing device to classify the image as a good quality image or a poor quality image based on the comparison comprise instructions that, when executed, cause the processor of the computing device to classify the image as a good quality image or a poor quality image based on a comparison of the first computed difference to the second computed difference.

22. The computer-readable storage device of claim 19, further comprising instructions that, when executed, cause the processor of the computing device to:

identify one or more target regions within the image;

associate each spatial frequency of the plurality of spatial frequencies with a target region of the identified one or more target regions; and

compute a frequency content associated with each target region from the one or more target regions based at least on one or more spatial frequencies associated with each target region.