Facial image quality assessment

• US 8,441,548 B1
• Filed: 06/15/2012
• Issued: 05/14/2013
• Est. Priority Date: 06/15/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Nechyba, Michael Christian, Sipe, Michael Andrew
• Primary Examiner(s): Velez, Roberto
• Assistant Examiner(s): Tissire, Abdelaaziz
• Application Number:
  US13/524,745
• Time in Patent Office: 333 Days
• Field of Search: 348/254, 348/333.04, 348E17002-E17004, 382/172
• US Class Current: 348/222.1
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 21/32 using biometric data, e.g. fingerprints, iris scans or voiceprints
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06V 10/507 Summing image-intensity values; Histogram projection analysis
• CPC Class Codes: G06V 10/993 Evaluation of the quality of the acquired pattern
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: H04M 2250/00 Details of telephonic subscriber devices
• CPC Class Codes: H04N 23/40 Circuit details for pick-up tubes
• CPC Class Codes: H04N 23/61 based on recognised objects
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/634 Warning indications
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/673 based on contrast or high frequency components of image signals, e.g. hill climbing method
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/71 Circuitry for evaluating the brightness variation

Abstract

An example method includes capturing, by a camera of a mobile computing device, an image, determining whether the image includes a representation of at least a portion of a face, and, when the image includes the representation of at least the portion of the face, analyzing characteristics of the image. The characteristics include at least one of a tonal distribution of the image that is associated with a darkness-based mapping of a plurality of pixels of the image, and a plurality of spatial frequencies of the image that are associated with a visual transition between adjacent pixels of the image. The method further includes classifying, by the mobile computing device, a quality of the image based at least in part on the analyzed characteristics of the image.

Claims

1. A method comprising:

capturing, by a camera of a mobile computing device, an image;

determining whether the image includes a representation of at least a portion of a face; and

when the image includes the representation of at least the portion of the face;

computing a difference between a tonal distribution of the image and a tonal distribution of each previously classified image from a group of previously classified images, wherein the tonal distribution is associated with a darkness-based mapping of a plurality of pixels of the image, wherein at least one previously classified image from the group of previously classified images is classified as a good quality image, and wherein at least one previously classified image from the group of previously classified images is classified as a poor quality image;

determining a smallest computed difference between the tonal distribution of the image and the tonal distribution of a previously classified image from the group of previously classified images based at least in part on the computed differences; and

classifying, by the mobile computing device, a quality of the image by at least classifying the image as being of a same quality as the previously classified image associated with the smallest computed difference.

2. The method of claim 1, wherein computing the difference between the tonal distribution of the image and the tonal distribution of each previously classified image from the group of previously classified images further comprises:

computing a statistic associated with the tonal distribution of the image;

computing at least one corresponding statistic associated with the tonal distribution of each previously classified image from the group of previously classified images; and

comparing the computed statistic associated with the tonal distribution of the image to each of the at least one computed corresponding statistic associated with the tonal distribution of each previously classified image from the group of previously classified images.

3. The method of claim 2, wherein:

the computed statistic associated with the tonal distribution of the image includes at least one of

     1) a peak count associated with the tonal distribution,

     2) an entropy associated with the tonal distribution, and

     3) a variance associated with the tonal distribution; and

each of the at least one computed corresponding statistic associated with the tonal distribution of each previously classified image from the group of previously classified images includes at least one of

     1) a peak count associated with the tonal distribution,

     2) an entropy associated with the tonal distribution, and

     3) a variance associated with the tonal distribution.

4. The method of claim 1, wherein computing the difference comprises at least one of 1) comparing a light pixel density of the tonal distribution of the image to a light pixel density of the tonal distribution of each previously classified image from the group of previously classified images and 2) comparing a dark pixel density of the tonal distribution of the image to a dark pixel density of the tonal distribution of each previously classified image from the group of previously classified images.

5. The method of claim 1, further comprising:

generating a first group of prototype vectors based at least in part on a first plurality of images previously classified as good quality images, wherein each prototype vector of the first group of prototype vectors corresponds to a mean value of a first set of tonal ratios associated with the first plurality of previously classified images;

generating a second group of prototype vectors based at least in part on a second plurality of images previously classified as poor quality images, wherein each of the prototype vectors of the second group of prototype vectors corresponds to a mean value of a second set of tonal ratios associated with the second plurality of previously classified images; and

generating one or more vectors of the image based at least in part on one or more tonal ratios associated with the image.

6. The method of claim 5, further comprising:

computing a first difference between the one or more vectors of the image and the first group of prototype vectors;

computing a second difference between the one or more vectors of the image and the second group of prototype vectors;

determining whether the first difference is less than the second difference;

classifying the image as a good quality image when the first difference is less than the second difference; and

classifying the image as a poor quality image when the second difference is less than the first difference.

7. The method of claim 1, further comprising:

when the image is classified as a good quality image, determining that the image is eligible for authentication; and

determining whether the user is an authenticated user of the mobile computing device based at least in part on the image based at least in part on classification of the image as a good quality image.

8. The method of claim 1 further comprising:

determining whether the image includes less than or equal to a threshold amount of motion blur;

when the image includes less than or equal to the threshold amount of motion blur, classifying the image as a good quality image; and

when the image includes more than the threshold amount of motion blur, classifying the image as a poor quality image.

9. The method of claim 1, wherein classifying the image comprises:

comparing one or more spatial frequencies of the image to corresponding spatial frequencies of each previously classified image of a plurality of previously classified images, wherein each spatial frequency of the one or more spatial frequencies of the image is associated with a visual transition between two or more adjacent pixels of the image, wherein each previously classified image of the plurality of previously classified images is classified as one of a good quality and a poor quality based on the corresponding spatial frequencies the respective previously classified image; and

classifying the image as a good quality image or a poor quality image based on the comparison.

10. The method of claim 9,

wherein the plurality of previously classified images includes a first portion of previously classified images classified as good quality images and a second portion of previously classified images classified as poor quality images,

wherein comparing the one or more spatial frequencies of the image to the corresponding spatial frequencies of each previously classified image of the plurality of previously classified images comprises:

computing a first difference between the one or more spatial frequencies of the image and the corresponding spatial frequencies of the first portion of previously classified images; and

computing a second difference between the one or more spatial frequencies of the image and the corresponding spatial frequencies of the second portion of previously classified images, and

wherein classifying the image as a good quality image or a poor quality image based on the comparison comprises;

classifying the image as a good quality image when the first computed difference is less than the second computed difference; and

classifying the image as a poor quality image when the second computed difference is less than or equal to the first computed difference.

11. The method of claim 9, further comprising:

identifying one or more target regions within the image;

associating each spatial frequency of the plurality of spatial frequencies with a target region of the identified one or more target regions; and

computing a frequency content associated with each target region from the one or more target regions based at least on one or more spatial frequencies associated with each target region.

12. The method of claim 11, further comprising:

determining at least one energy value associated with the frequency content associated with each target region, wherein the at least one energy value is associated with a high frequency band;

comparing the at least one energy value to a threshold energy value, wherein the threshold energy value is associated with a high frequency band of a prototype vector;

if the at least one energy value is less than or equal to the threshold energy value, classifying the image as a high motion blur image; and

if the at least one energy value is greater than the threshold energy value, classifying the image as a low motion blur image.

13. The method of claim 11, further comprising:

generating a frequency representation of the image at least in part by applying a wavelet transform to the frequency content associated with each target region, wherein applying the wavelet transform comprises expressing two or more spatial frequencies of the frequency content as two or more corresponding wavelets; and

computing a transition between the two or more corresponding wavelets.

14. The method of claim 1 further comprising:

when the image does not include the representation of at least the portion of the face, classifying the image as a poor quality image.

15. The method of claim 1, wherein the notification includes an indication of a predicted cause of the poor quality image.

16. The method of claim 15, wherein the predicted cause includes at least one of 1) improper lighting conditions, and 2) motion blur.

17. The method of claim 15, wherein the notification includes an instruction to activate or deactivate a flashbulb of the mobile computing device when the predicted cause includes improper lighting conditions.

18. The method of claim 17, further comprising:

when the predicted cause includes improper lighting conditions, determining that a cause of the improper lighting conditions is at least one of

     1) overexposure, and

     2) backlighting, in the image.

19. The method of claim 17, wherein the notification further includes one or more instructions on how to correct the predicted cause of the poor quality image.

20. The method of claim 1, further comprising:

when the quality of the image is classified as poor quality, outputting a notification that the image is a poor quality image.

21. The method of claim 1, further comprising:

when the quality of the image is classified as poor quality, adjusting at least one of

     1) a flash brightness associated with the camera,

     2) a flash activation associated with the camera, and

     3) a shutter speed associated with the camera.

22. A computer-readable storage device comprising instructions for causing at least one processor of a mobile computing device to perform operations comprising:

capturing, by a camera of the mobile computing device, an image;

determining whether the image includes a representation of at least a portion of a face; and

when the image includes the representation of at least the portion of the face;

computing a difference between the tonal distribution of the image and a tonal distribution of each previously classified image from a group of previously classified images, wherein at least one previously classified image from the group of previously classified images is classified as a good quality image, and wherein at least one previously classified image from the group of previously classified images is classified as a poor quality image;

determining a smallest computed difference between the tonal distribution of the image and the tonal distribution of a previously classified image from the group of previously classified images based at least in part on the computed differences;

classifying a quality of the image by at least classifying the image as being of a same quality as the previously classified image associated with the smallest computed difference; and

when the quality of the image is classified as poor quality, outputting a notification that the image is a poor quality image.

23. A mobile computing device comprising:

at least one processor; and

at least one camera operable to capture an image,

wherein the at least one processor is configured to;

determine that the image includes a representation of at least a portion of a face;

compute a difference between a tonal distribution of the image and a tonal distribution of each previously classified image from a group of previously classified images, wherein the tonal distribution is associated with a darkness-based mapping of a plurality of pixels of the image, wherein at least one previously classified image from the group of previously classified images is classified as a good quality image, and wherein at least one previously classified image from the group of previously classified images is classified as a poor quality image;

determine a smallest computed difference between the tonal distribution of the image and the tonal distribution of a previously classified image from the group of previously classified images based at least in part on the computed differences; and

classify a quality of the image by at least classifying the image as being of a same quality as the previously classified image associated with the smallest computed difference.

24. The mobile computing device of claim 23, further comprising at least one output device configured to, when the quality of the image is classified as poor quality, output a notification that the image is a poor quality image.

25. The mobile computing device of claim 23, further comprising a flashbulb coupled to the at least one camera, wherein the at least one processor is further configured to:

when the quality of the image is classified as poor quality, adjust at least one of

     1) a flash brightness associated with the flashbulb, and

     2) a flash activation associated with the flashbulb.

26. The mobile computing device of claim 25, wherein the at least one processor is further configured to:

use a previously classified quality of a prior facial image to adjust at least one of

     1) the flash brightness associated with the flashbulb, and

     2) the flash activation associated with the flashbulb.

27. The mobile computing device of claim 25, wherein the captured image is a first captured image, and wherein the at least one processor is further configured to:

when the quality of the image is classified as poor quality, then send an instruction to the camera to capture a second image.

28. The mobile computing device of claim 27, further comprising an output device, wherein the at least one processor is further configured to:

send an instruction to the output device to output a prompt to a user, wherein the prompt indicates the capture of the second image.