Systems and methods for determining personal characteristics

• US 8,582,807 B2
• Filed: 05/31/2010
• Issued: 11/12/2013
• Est. Priority Date: 03/15/2010
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Laboratories America Inc (NEC Corporation)
• Inventors: Yang, Ming, Zhu, Shenghuo, Lv, Fengjun, Yu, Kai
• Primary Examiner(s): Akhavannik, Hadi
• Application Number:
  US12/790,979
• Publication Number:
  US 20110222724A1
• Time in Patent Office: 1,261 Days
• Field of Search: 382/103
• US Class Current: 382/103
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/178 estimating age from face image; using age information for improving recognition

Abstract

Systems and methods are disclosed for determining personal characteristics from images by generating a baseline gender model and an age estimation model using one or more convolutional neural networks (CNNs); capturing correspondences of faces by face tracking, and applying incremental learning to the CNNs and enforcing correspondence constraint such that CNN outputs are consistent and stable for one person.

Claims

1. A computer implemented method to classify camera images, comprising:

a. generating a baseline gender model and an age estimation model using one or more convolutional neural networks (CNNs);

b. capturing correspondences of faces by face tracking;

c. applying a stochastic update rule for each face image determined as

2. The method of claim 1, comprising collecting correspondences of faces by face tracking.

3. The method of claim 1, comprising providing incremental learning in the neural network by enforcing a correspondence constraint.

4. The method of claim 1, comprising implementing incremental training with an online stochastic gradient descent process.

5. The method of claim 1, comprising applying a correspondence given by visual tracking to update a pre-trained model.

6. The method of claim 1, comprising:

a. performing face detection and tracking;

b. aligning the detected faces;

c. normalizing the faces to a plurality of patches; and

d. sending the normalized faces to the CNNs to estimate gender and age.

7. The method of claim 1, comprising applying CNNs to face detection and face alignment.

8. The method of claim 1, comprising performing multi-hypothesis visual tracking to obtain correspondence of faces in successive video frames.

9. The method of claim 1, comprising updating the baseline models using data collected online to avoid model drift.

10. The method of claim 1, comprising applying weakly supervised incremental training to face correspondences.

11. A system to classify camera images, comprising

a. means for generating a baseline gender model and an age estimation model using one or more convolutional neural networks (CNNs);

b. means for capturing correspondences of faces by face tracking;

c. means for applying a stochastic update rule for each face image determined as

12. The system of claim 11, comprising means for collecting correspondences of faces by face tracking.

13. The system of claim 11, comprising means for providing incremental learning in the neural network by enforcing a correspondence constraint.

14. The system of claim 11, comprising means for implementing incremental training with an online stochastic gradient descent process.

15. The system of claim 11, comprising means for applying a correspondence given by visual tracking to update a pre-trained model.

16. The system of claim 11, comprising:

a. means for performing face detection and tracking;

b. means for aligning the detected faces;

c. means for normalizing the faces to a plurality of patches; and

d. means for sending the normalized faces to the CNNs to estimate gender and age.

17. The system of claim 11, comprising means for applying CNNs to face detection and face alignment.

18. The system of claim 11, comprising means for performing multi-hypothesis visual tracking to obtain correspondence of faces in successive video frames.

19. The system of claim 11, comprising means for updating the baseline models using data collected online to avoid model drift.

20. The system of claim 11, comprising means for applying weakly supervised incremental training to face correspondences.