IMAGE RECOGNITION METHOD AND APPARATUS, IMAGE VERIFICATION METHOD AND APPARATUS, LEARNING METHOD AND APPARATUS TO RECOGNIZE IMAGE, AND LEARNING METHOD AND APPARATUS TO VERIFY IMAGE

• US 20160171346A1
• Filed: 12/03/2015
• Published: 06/16/2016
• Est. Priority Date: 12/15/2014
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: HAN, Seungju, SUH, Sungjoo, HAN, Jaejoon, CHOI, Chang Kyu
• Granted Patent: US 10,346,726 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 18/2413 based on distances to training or reference patterns
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/084 Backpropagation, e.g. using gradient descent
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 40/169 Holistic features and representations, i.e. based on the facial image taken as a whole
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

A method of recognizing a feature of an image may include receiving an input image including an object; extracting first feature information using a first layer of a neural network, the first feature information indicating a first feature corresponding to the input image among a plurality of first features; extracting second feature information using a second layer of the neural network, the second feature information indicating a second feature among a plurality of second features, the indicated second feature corresponding to the first feature information; and recognizing an element corresponding to the object based on the first feature information and the second feature information.

Claims

1. A method of recognizing a feature of an image, the method comprising:

receiving an input image including an object;

extracting first feature information using a first layer of a neural network, the first feature information indicating a first feature corresponding to the input image among a plurality of first features;

extracting second feature information using a second layer of the neural network, the second feature information indicating a second feature among a plurality of second features, the indicated second feature corresponding to the first feature information; and

recognizing an element corresponding to the object based on the first feature information and the second feature information.

2. The method of claim 1, wherein the recognizing comprises:

generating a feature vector based on the first feature information and the second feature information; and

performing a comparison operation based on the generated feature vector and a reference vector corresponding to each of a plurality of elements; and

recognizing the element corresponding to the object based on a result of the comparison operation.

3. The method of claim 2, wherein the generating comprises:

generating the feature vector based on values corresponding to all nodes in the first layer and values corresponding to all nodes in the second layer.

4. The method of claim 1

wherein the recognizing comprises calculating a feature value corresponding to the element based on weights between nodes included in the neural network.

5. The method of claim 1, wherein the extracting of the first feature information includes performing a filtering of information corresponding to the input image using a filter with a first size, and

wherein the extracting of the second feature information includes performing a filtering of the first feature information using a filter with a second size.

6. The method of claim 5, wherein the extracting of the first feature information further includes extracting representative values from the information corresponding to the input image.

7. The method of claim 5, wherein the extracting of the second feature information further includes extracting representative values from the first feature information.

8. The method of claim 1, further comprising extracting m-th feature information using an m-th layer of the neural network, the m-th feature information indicating an m-th feature among a plurality of m-th features, the indicated m-th feature corresponding to a combination of the first feature information through the (m−

1)-th feature information,

wherein m is an integer equal to or greater than 3, and

wherein the recognizing of the element includes recognizing the element based on the first feature information through the m-th feature information.

9. An apparatus for recognizing a feature of an image;

the apparatus comprising;

a memory storing computer-readable instructions; and

one or more processors configured to execute the computer-readable instructions such that the one or more processors are configured to,

receive an input image including an object;

extract first feature information using a first layer, the first feature information indicating a first feature among a plurality of first feature information, the indicated first feature corresponding to the input image;

extract second feature information using a second layer, the second feature information indicating a second feature among a plurality of second features, the indicated second feature corresponding to the first feature information; and

recognize an element corresponding to the object based on the first feature information and the second feature information.

10. The apparatus of claim 9, wherein the one or more processors are configured to,

generate a feature vector based on the first feature information and the second feature information,

perform a comparison operation based on the generated feature vector and a reference vector corresponding to each of a plurality of elements, and

recognize the element corresponding to the object based on a result of the comparison operation.

11. The apparatus of claim 9, wherein the one or more processors are further configured to,

extract third feature information using a third layer of the neural network, the third feature information indicating a third feature among a plurality of third features, the indicated third feature corresponding to the second feature information, and

recognize the element corresponding to the object based on the first feature information, the second feature information, and the third feature information.

12. A method of learning a feature to recognize an image, the method comprising:

receiving a training element and a training image associated with a training object; and

learning a parameter of a recognizer such that the recognizer recognizes the training element from the training image, the recognizer being configured to recognize a plurality of elements from first feature information extracted from an input image using a first layer of a neural network and second feature information extracted using a second layer of the neural network.

13. The method of claim 12, wherein the learning of the parameter of the recognizer comprises:

learning a parameter of the first layer so that first features are extracted from the training image; and

learning a parameter of the second layer so that second features are extracted from at least one of the first features.

14. The method of claim 13, wherein the learning of the parameter of the recognizer further comprises:

learning the parameter of the recognizer such that the recognizer recognizes the training element based on the first features and the second features.

15. The method of claim 13, wherein the learning of the parameter of the recognizer comprises:

calculating a loss corresponding to the training element.

16. The method of claim 15, wherein the recognizer includes the neural network, and

wherein the learning of the parameter of the recognizer further comprises:

learning weights between nodes included in the neural network, based on the loss.

17. An apparatus for learning a feature to recognize an image, the apparatus comprising:

a memory storing computer-readable instructions; and

one or more processors configured to execute the computer-readable instructions such that the one or more processors are configured to,

receive a training element and a training image associated with a training object;

implement a recognizer; and

learn a parameter of the recognizer such that the recognizer recognizes the training element from the training image, the recognizer being configured to recognize a plurality of elements from first feature information extracted from an input image using a first layer of a neural network and second feature information extracted using a second layer of the neural network.

18. A method of verifying a feature of an image, the method comprising:

receiving a first image including a first object;

extracting first feature information using a first layer of a neural network, the first feature information indicating a first feature among a plurality of first features, the indicated first feature corresponding to the first image;

extracting second feature information using a second layer of the neural network, the second feature information indicating a second feature among a plurality of second features, the indicated second feature corresponding to the first feature information; and

determining whether the object of the first image is similar to a second object of a second image, based on the first feature information and the second feature information of the first image and based on first feature information and second feature information associated with the second image.

19. The method of claim 18, wherein the determining comprises:

generating a feature vector associated with the first image based on the first feature information and the second feature information of the first image;

generating a feature vector associated with the second image based on the first feature information and the second feature information associated with the second image; and

performing a comparison operation based on the feature vector associated with the first image and the feature vector associated with the second image, and determining whether the first object is similar to the second based on a result of the comparison operation.

20. The method of claim 19, wherein the generating of the feature vector associated with the first image includes generating the feature vector associated with the first image based on feature values calculated based on the first feature information and the second feature information of the first image, and

wherein the generating of the feature vector associated with the second image includes generating the feature vector associated with the second image based on feature values calculated based on the first feature information and the second feature information associated with the second image.

21. The method of claim 18, wherein the extracting of the first feature information includes extracting first feature information associated with the second image, using the first layer, and

wherein the extracting of the second feature information includes extracting second feature information associated with the second image, using the second layer.

22. The method of claim 18, wherein the extracting of the first feature information includes extracting first feature information associated with the second image, using a layer of the neural network that is separated from the first layer, and

wherein the extracting of the second feature information includes extracting second feature information associated with the second image, using a layer of the neural network that is separated from the second layer.

23. An apparatus for verifying a feature of an image, the apparatus comprising:

a memory storing computer-readable instructions; and

one or more processors configured to execute the computer-readable instructions such that the one or more processors are configured to,

receive a first image including a first object;

extract first feature information using a first layer of a neural network, the first feature information indicating a first feature among a plurality of first features, the first feature corresponding to the first image;

extract second feature information using a second layer of a neural network, the second feature information indicating a second feature among a plurality of second features, the second feature corresponding to the first feature information among; and

determine whether the first object is similar to a second object of a second image, based on the first feature information and the second feature information of the first image and based on first feature information and second feature information associated with the second image.

24. The apparatus of claim 23, wherein one or more processors are further configured to,

generate a feature vector associated with the first image based on the first feature information and the second feature information of the first image, and to generate a feature vector associated with the second image based on the first feature information and the second feature information associated with the second image,

compare the feature vector associated with the first image to the feature vector associated with the second image, and

determine whether the first object is similar to the second object.

25. The apparatus of claim 23, wherein the one or more processors are further configured to,

extract third feature information using a third layer of the neural network, the third feature information indicating a third feature corresponding to the second feature information among third features,

determine whether the first object is similar to the second object, based on the first feature information, the second feature information and the third feature information of the first image and based on the first feature information, the second feature information and third feature information associated with the second image.

26. A method of learning a feature to verify an image, the method comprising:

receiving a pair of training images, and training information corresponding to the pair of training images; and

learning a parameter of a verifier so that a result of comparing, by the verifier, the training images corresponds to the training information, the verifier being configured to determine whether two input images are similar to each other, based on first feature information extracted from the two input images using a first layer of a neural network and based on second feature information extracted using a second layer of the neural network.

27. The method of claim 26, wherein the training information comprises:

information indicating whether or not the training images include identical objects.

28. The method of claim 26, wherein the learning of the parameter of the verifier comprises:

learning a parameter of the first layer such that first features are extracted from each of the training images; and

learning a parameter of the second layer such that second features are extracted from the first features.

29. The method of claim 28, wherein the learning of the parameter of the verifier further comprises:

learning the parameter of the verifier such that the verifier outputs the training information based on the first features and the second features corresponding to each of the training images.

30. The method of claim 26, wherein the learning comprises:

calculating a loss corresponding to the training information.

31. The method of claim 30, wherein the verifier includes the neural network, and

wherein the learning of the parameter of the verifier further comprises:

learning weights between nodes included in the neural network, based on the loss.

32. An apparatus for learning a feature to verify an image, the apparatus comprising:

a memory storing computer-readable instructions; and

one or more processors configured to execute the computer-readable instructions such that the one or more processors are configured to,

receive a pair of training images, and training information corresponding to the pair of training images, and

learn a parameter of a verifier so that a result of comparing, by the verifier, the training images corresponds to the training information, the verifier being configured to determine whether two input images are similar to each other, based on first feature information extracted from the two input images using a first layer of a neural network and based on second feature information extracted using a second layer of the neural network.