Method and apparatus for recognizing object, and method and apparatus for training recognizer

• US 9,928,410 B2
• Filed: 11/11/2015
• Issued: 03/27/2018
• Est. Priority Date: 11/24/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Yoo, Byungin, Kim, Namjoon, Lee, Changkyo, Choi, Chang Kyu, Han, Jaejoon
• Primary Examiner(s): Do, Anh H
• Application Number:
  US14/938,365
• Publication Number:
  US 20160148080A1
• Time in Patent Office: 867 Days
• Field of Search: 382157, 382118, 382159, 382211, 382229, 382310, 1 1, 359 29, 359559, 707754, 707783, 70799902, 707E1705, 715209, 715210, 715230, 715234
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/082 modifying the architecture, e.g. adding, deleting or silencing nodes or connections
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/174 Facial expression recognition

Abstract

A recognition method includes receiving an input image; and recognizing a plurality of elements associated with the input image using a single recognizer pre-trained to recognize a plurality of elements simultaneously.

Claims

1. A recognition method comprising:

receiving an input image including a face region; and

recognizing a plurality of elements of the input image using a single recognizer including a convolutional neural network (CNN) pre-trained to recognize the plurality of elements simultaneously,

the plurality of elements including elements associated with the face region.

2. The recognition method of claim 1, wherein the plurality of elements comprises:

an identity (ID) that identifies the input image; and

at least one attribute associated with the input image.

3. The recognition method of claim 2, wherein the ID identifies at least an object included in the input image.

4. The recognition method of claim 2, wherein the at least one attribute comprises at least one of:

a gender corresponding to the face region;

an age corresponding to the face region;

an ethnic group corresponding to the face region;

an attractiveness corresponding to the face region;

a facial expression corresponding to the face region;

or

an emotion corresponding to the face region.

5. The recognition method of claim 4, wherein the at least one attribute includes at least two different attributes from among the gender, the age, the ethnic group, the attractiveness, and the facial expression.

6. The recognition method of claim 1, wherein the recognizer includes a neural network, and

the recognizing includes calculating feature values corresponding to the plurality of elements based on pre-learned weights between nodes included in the neural network.

7. The recognition method of claim 1, wherein the recognizing includes generating a plurality of feature images based on the input image.

8. The recognition method of claim 7, wherein the plurality of feature images comprises at least one of:

a color channel image from which illumination noise is removed;

an oriented-gradient magnitude channel image;

a skin probability channel image;

or

a local binary pattern channel image.

9. The recognition method of claim 7, wherein the recognizing comprises:

filtering the plurality of feature images; and

outputting feature values corresponding to the plurality of elements based on an output of the filtering.

10. The recognition method of claim 9, wherein the recognizing further comprises:

recognizing the plurality of elements based on the feature values.

11. The recognition method of claim 1, wherein the recognizing comprises:

acquiring a plurality of part images corresponding to parts of a face included in the input image; and

generating a plurality of feature images corresponding to each of the plurality of part images.

12. The recognition method of claim 11, wherein the recognizing comprises:

outputting first feature values corresponding to the plurality of elements based on outputs of a plurality of part recognition modules,

wherein each of the plurality of part recognition modules is configured to

filter feature images of a corresponding part image; and

output feature values corresponding to elements associated with the corresponding part image based on an output of the filtering.

13. The recognition method of claim 12, wherein the recognizing further comprises:

recognizing the plurality of elements based on the output first feature values.

14. The recognition method of claim 1, further comprising:

comparing the plurality of elements to a plurality of elements associated with a reference image; and

determining whether the input image matches the reference image based on a result of the comparing.

15. The recognition method of claim 14, wherein the comparing comprises:

generating a feature vector based on the plurality of elements; and

comparing the feature vector to a reference vector of the reference image.

16. A method of training a recognizer, the method comprising:

receiving a training image; and

training a recognizer configured to recognize a plurality of elements of an input image that includes a face region, based on the training image and a plurality of elements labeled in the training image, such that the recognizer includes a convolutional neural network (CNN) pre-trained to recognize the plurality of elements of the input image simultaneously,

the plurality of elements of the input image including elements associated with the face region.

17. The method of claim 16, wherein the plurality of elements comprise:

an identity (ID) that identifies the training image; and

at least one attribute associated with the training image.

18. The method of claim 17, wherein the ID includes information that identifies at least an object included in the training image.

19. The method of claim 17, wherein the at least one attribute comprises at least one of:

a gender corresponding to the face region;

an age corresponding to the face region;

an ethnic group corresponding to the face region;

an attractiveness corresponding to the face region;

a facial expression corresponding to the face region;

or

an emotion corresponding to the face region.

20. The method of claim 19, wherein the at least one attribute includes at least two different attributes from among the gender, the age, the ethnic group, the attractiveness, and the facial expression.

21. The method of claim 16, wherein the training comprises calculating losses corresponding to the plurality of elements.

22. The method of claim 21, wherein the recognizer comprises a neural network, and

the training includes training the recognizer to learn weights between nodes included in the neural network based on the losses.

23. The method of claim 16, wherein the recognizer comprises a neural network, and

the training includes activating nodes included in the neural network based on a stochastic piecewise linear (PWL) model.

24. The method of claim 16, wherein the training comprises generating a plurality of feature images based on the training image.

25. The method of claim 24, wherein the plurality of feature images comprises at least one of:

a color channel image from which illumination noise is removed;

an oriented-gradient magnitude channel image;

a skin probability channel image;

or

a local binary pattern channel image.

26. The method of claim 24, wherein the training comprises:

training the recognizer to filter the plurality of feature images; and

training the recognizer to output feature values corresponding to the plurality of elements based on an output of the filtering of the plurality of feature images.

27. The method of claim 26, wherein the training further comprises:

training the recognizer to recognize the plurality of elements based on the output feature values.

28. The method of claim 16, wherein the training comprises:

acquiring a plurality of part images corresponding to parts of a face included in the training image.

29. The method of claim 28, wherein different elements are labeled in the plurality of part images.

30. The method of claim 28, wherein the training further comprises:

generating a plurality of feature images corresponding to each of the plurality of part images.

31. The method of claim 30, wherein the training comprises:

training the recognizer to output first feature values corresponding to the plurality of elements based on outputs of a plurality of part recognition modules corresponding to the plurality of part images; and

for each of the plurality of part recognition modules,

training the part recognition module to filter feature images of a corresponding part image, and

training the part recognition module to output feature values corresponding to elements associated with the corresponding part image based on an output of the filtering.

32. The method of claim 31, wherein the training further comprises:

training the recognizer to recognize the plurality of elements based on the output first feature values.

33. A non-transitory computer-readable medium comprising program code that, when executed by a processor, performs functions according to the method of claim 16.

34. A recognition apparatus comprising:

a memory storing instructions; and

one or more processors configured to execute the instructions such that the one or more processors are configured to,

receive an input image including a face region; and

recognize a plurality of elements of the input image using a single recognizer including a convolutional neural network (CNN) pre-trained to recognize the plurality of elements simultaneously,

the plurality of elements including elements associated with the face region.

35. An apparatus for training a recognizer, the apparatus comprising:

a memory storing instructions; and

one or more processors configured to execute the instructions such that the one or more processors are configured to,

receive a training image; and

train a recognizer configured to recognize a plurality of elements of an input image that includes a face region, based on the training image and a plurality of elements labeled in the training image, such that the recognizer includes a convolutional neural network (CNN) pre-trained to recognize the plurality of elements of the input image simultaneously,

the plurality of elements of the input image including elements associated with the face region.