DEEP MULTI-TASK LEARNING FRAMEWORK FOR FACE DETECTION, LANDMARK LOCALIZATION, POSE ESTIMATION, AND GENDER RECOGNITION

• US 20180211099A1
• Filed: 07/20/2016
• Published: 07/26/2018
• Est. Priority Date: 07/20/2015
• Status: Active Grant

• Current Assignee: University of Maryland, College Park
• Original Assignee: University of Maryland, College Park
• Inventors: RANJAN, Rajeev, PATEL, Vishal M., CHELLAPPA, Ramalingam, CASTILLO, Carlos D.
• Granted Patent: US 10,860,837 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/993 Evaluation of the quality of the acquired pattern
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships

Abstract

Various image processing may benefit from the application deep convolutional neural networks. For example, a deep multi-task learning framework may assist face detection, for example when combined with landmark localization, pose estimation, and gender recognition. An apparatus can include a first module of at least three modules configured to generate class independent region proposals to provide a region. The apparatus can also include a second module of the at least three modules configured to classify the region as face or non-face using a multi-task analysis. The apparatus can further include a third module configured to perform post-processing on the classified region.

Claims

1. An apparatus, comprising:

a first module of at least three modules, wherein the first module configured to generate class independent region proposals to provide a region;

a second module of the at least three modules is configured to classify the region as face or non-face using a multi-task analysis; and

a third module of the at least three modules is configured to perform post-processing on the classified region.

2. The apparatus of claim 1, wherein the second module comprises a five convolutional layers with three fully connected layers.

3. The apparatus of claim 2, wherein the second module further comprises a network configured to fuse the three fully connected layers.

4. The apparatus of claim 3, wherein the second module further comprises separate networks for face detection, landmark detection, visibility determination, pose estimation, and gender determination.

5. The apparatus of claim 1, wherein the third module comprises at least one of an iterative region proposal or landmark-based non-maximum suppression.

15. The apparatus of claim 1, wherein the third module comprises at least one of an iterative region proposal or landmark-based non-maximum suppression.

6. An apparatus, comprising:

at least one processor; and

at least one memory including computer program instructions,

wherein the at least one memory and the computer program instructions are configured to

select a set of data for facial analysis; and

apply the set of data to a network comprising at least three modules,

wherein a first module of the at least three modules is configured to generate class independent region proposals to provide a region,

wherein a second module of the at least three modules is configured to classify the region as face or non-face using a multi-task analysis,

wherein a third module of the at least three modules is configured to perform post-processing on the classified region.

7. The apparatus of claim 6, wherein the second module comprises a five convolutional layers with three fully connected layers.

8. The apparatus of claim 7, wherein the second module further comprises a network configured to fuse the three fully connected layers.

9. The apparatus of claim 8, wherein the second module further comprises separate networks for face detection, landmark detection, visibility determination, pose estimation, and gender determination.

10. The apparatus of claim 6, wherein the third module comprises at least one of an iterative region proposal or landmark-based non-maximum suppression.

11. A method, comprising:

selecting a set of data for facial analysis; and

applying the set of data to a network comprising at least three modules,

wherein a first module of the at least three modules is configured to generate class independent region proposals to provide a region,

wherein a second module of the at least three modules is configured to classify the region as face or non-face using a multi-task analysis,

wherein a third module of the at least three modules is configured to perform post-processing on the classified region.

12. The method of claim 11, wherein the second module comprises a five convolutional layers with three fully connected layers.

13. The method of claim 12, wherein the second module further comprises a network configured to fuse the three fully connected layers.

14. The method of claim 13, wherein the second module further comprises separate networks for face detection, landmark detection, visibility determination, pose estimation, and gender determination.

16. An apparatus, comprising:

means for selecting a set of data for facial analysis; and

means for applying the set of data to a network comprising at least three modules,

wherein a first module of the at least three modules is configured to generate class independent region proposals to provide a region,

wherein a second module of the at least three modules is configured to classify the region as face or non-face using a multi-task analysis,

wherein a third module of the at least three modules is configured to perform post-processing on the classified region.

17. The apparatus of claim 16, wherein the second module comprises a five convolutional layers with three fully connected layers.

18. The apparatus of claim 17, wherein the second module further comprises a network configured to fuse the three fully connected layers.

19. The apparatus of claim 18, wherein the second module further comprises separate networks for face detection, landmark detection, visibility determination, pose estimation, and gender determination.

20. The apparatus of claim 16, wherein the third module comprises at least one of an iterative region proposal or landmark-based non-maximum suppression.