Creation of representative content based on facial analysis

• US 9,639,742 B2
• Filed: 04/28/2014
• Issued: 05/02/2017
• Est. Priority Date: 04/28/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Lee, David, Chan, Chunkit Jacky, Ricard, Doug, Scott, Stacia, Light, Allison
• Primary Examiner(s): Kassa, Yosef
• Application Number:
  US14/264,012
• Publication Number:
  US 20150310261A1
• Time in Patent Office: 1,100 Days
• Field of Search: 382115, 382117, 382118, 382128, 382203, 382209, 382282, 340 581, 340 582, 340 583
• CPC Class Codes: C07C 1/00 Preparation of hydrocarbons from one or more compounds, none of them being a hydrocarbon
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction

Abstract

Technologies for analyzing various features detected in a face detected in an image. Such features may include at least the eyes, eyebrows, node, and mouth of a face. Such analyzing may include scoring aspects of these features. Such scores may be weighted. The analysis may be used to determine expressions and/or poses of the face, as well as indicate if the face is smiling, frowning, or neutral. An overall quality score for the face may also be determined and provided based on the aforementioned analysis as well as whether or not the face is near an edge of the image or cut-off in the image. Finally, a face signature may be determined that uniquely identifies the face, at least within the scope of the analyzed features. Such face signatures may be used to detect a similar face in other images.

Claims

1. A method performed on a computing device, the method comprising:

receiving, by the computing device, information including an image, a face identifier that identifies a face detected in the image, and a set of facial feature descriptors that indicate features of the detected face;

analyzing, by the computing device based on the set of facial feature descriptors, features of the detected face, the features including a mouth of the detected face and at least one eye of the detected face; and

determining, by the computing device based on the received information and the analyzing, an expression of the detected face.

2. The method of claim 1 where the analyzing the at least one eye comprises determining an open-closed state of the eye based on first points that indicate a width of the eye and second points that indicate a height of the eye.

3. The method of claim 1 where the analyzing the mouth determining an open-closed state of the mouth based on first points that indicate a width of the mouth and second points that indicate a height of the mouth.

4. The method of claim 1 further comprising determining an expression of the detected face based on the received information and on the analyzing, where the determined expression is selected from a set comprising smiling, frowning, neutral, and other.

5. The method of claim 4 where the determining comprises comparing points that indicate horizontal edges of the mouth to at least one point that indicates a location of a lip of the mouth.

6. The method of claim 1 further comprising determining a face signature that uniquely identifies the detected face within a scope of the analyzed features.

7. The method of claim 6 where the face signature is sufficient for identifying faces in other images that belong to a same person as does the detected face.

8. A system comprising a computing device and at least one program module that are together configured for performing actions comprising:

receiving, by the computing device, information including an image, a face identifier that identifies a face detected in the image, and a set of facial feature descriptors that indicate features of the detected face;

analyzing, by the computing device based on the set of facial feature descriptors, features of the detected face, the features including a mouth of the detected face and at least one eye of the detected face; and

determining, by the computing device based on the received information and the analyzing, an expression of the detected face.

9. The system of claim 8 where the analyzing the at least one eye comprises determining an open-closed state of the eye based on first points that indicate a width of the eye and second points that indicate a height of the eye.

10. The system of claim 8 where the analyzing the mouth determining an open-closed state of the mouth based on first points that indicate a width of the mouth and second points that indicate a height of the mouth.

11. The system of claim 8, the actions further comprising determining an expression of the detected face based on the received information and on the analyzing, where the determined expression is selected from a set comprising smiling, frowning, neutral, and other.

12. The system of claim 11 where the determining comprises comparing points that indicate horizontal edges of the mouth to at least one point that indicates a location of a lip of the mouth.

13. The system of claim 8, the actions further comprising determining a face signature that uniquely identifies the detected face within a scope of the analyzed features.

14. The system of claim 13 where the face signature is sufficient for identifying faces in other images that belong to a same person as does the detected face.

15. At least one computer-readable media storing computer-executable instructions that, when executed by a computing device, cause the computing device to perform actions comprising:

receiving, by the computing device, information including an image, a face identifier that identifies a face detected in the image, and a set of facial feature descriptors that indicate features of the face;

analyzing, by the computing device based on the set of facial feature descriptors, features of the detected face, the features including a mouth of the detected face and at least one eye of the detected face; and

determining, by the computing device based on the received information and the analyzing, an expression of the detected face.

16. The at least one computer-readable media of claim 15 where the analyzing the at least one eye comprises determining an open-closed state of the eye based on first points that indicate a width of the eye and second points that indicate a height of the eye.

17. The at least one computer-readable media of claim 15 where the analyzing the mouth determining an open-closed state of the mouth based on first points that indicate a width of the mouth and second points that indicate a height of the mouth.

18. The at least one computer-readable media of claim 15, the actions further comprising determining an expression of the detected face based on the received information and on the analyzing, where the determined expression is selected from a set comprising smiling, frowning, neutral, and other.

19. The at least one computer-readable media of claim 18 where the determining comprises comparing points that indicate horizontal edges of the mouth to at least one point that indicates a location of a lip of the mouth.

20. The at least one computer-readable media of claim 15, the actions further comprising determining a face signature that uniquely identifies the detected face within a scope of the analyzed features, where the face signature is sufficient for identifying faces in other images that belong to a same person as does the detected face.