DYNAMIC EMOTION RECOGNITION IN UNCONSTRAINED SCENARIOS

• US 20190325203A1
• Filed: 01/20/2017
• Published: 10/24/2019
• Est. Priority Date: 01/20/2017
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Yao, Anbang, Cai, Dongqi, Hu, Ping, Wang, Shandong, Chen, Yurong
• Granted Patent: US 11,151,361 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/2413 based on distances to training or reference patterns
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/048 Activation functions
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 40/174 Facial expression recognition

Abstract

An apparatus for dynamic emotion recognition in unconstrained scenarios is described herein. The apparatus comprises a controller to pre-process image data and a phase-convolution mechanism to build lower levels of a CNN such that the filters form pairs in phase. The apparatus also comprises a phase-residual mechanism configured to build middle layers of the CNN via plurality of residual functions and an inception-residual mechanism to build top layers of the CNN by introducing multi-scale feature extraction. Further, the apparatus comprises a fully connected mechanism to classify extracted features.

Claims

1-25. -25. (canceled)

26. An apparatus for dynamic emotion recognition in unconstrained scenarios, comprising:

a controller to pre-process image data;

a phase-convolution mechanism to build lower layers of a CNN, wherein the layers comprise a plurality of filters and filters of the lower layers form pairs in phase;

a phase-residual mechanism to build middle layers of the CNN via a plurality of residual functions;

an inception-residual mechanism to build top layers of the CNN via multi-scale feature extraction; and

a fully connected mechanism to perform feature abstraction and final emotion recognition.

27. The apparatus of claim 26, wherein the phase-convolution mechanism convolves the pre-processed image data and applies a Concatenated Rectified Linear Unit (CReLU) to the pre-processed image data.

28. The apparatus of claim 26, wherein the phase-residual mechanism is to reduce redundancy in the plurality of filters of the CNN.

29. The apparatus of claim 26, wherein the phase-residual mechanism is to increase the depth of the CNN by stacking specifically designed composite layers.

30. The apparatus of claim 26, wherein the phase-residual mechanism applies a linear projection, identity shortcut connection, and a Concatenated Rectified Linear Unit (CReLU) to data from the phase-convolution mechanism.

31. The apparatus of claim 26, wherein the inception-residual mechanism generates a multi-scale feature map.

32. The apparatus of claim 26, wherein the fully connected mechanism comprises a first fully connected layer that is to perform the feature abstraction and a second fully connected layer that is to classify a plurality of final features.

33. The apparatus of claim 26, wherein the CNN classifies a plurality of features in real-time.

34. A method for dynamic emotion recognition in unconstrained scenarios, comprising:

pre-processing image data;

building a plurality of lower levels of a CNN using the image data wherein filters of the lower levels form pairs in phase via a Concatenated Rectified Linear Unit (CReLU);

building a plurality of middle layers of the CNN via a plurality of residual functions;

building a plurality of top layers of the CNN via multi-scale feature extraction that is to extract a plurality of features; and

classifying the extracted plurality of features.

35. The method of claim 34, wherein building the lower levels of the CNN comprises:

generating an identical copy of a plurality of linear responses after convolution;

negating results of the convolution;

applying the Concatenated Rectified Linear Unit (CReLU); and

max-pooling to determine a number of feature maps.

36. The method of claim 34, wherein the Concatenated Rectified Linear Unit (CReLU) copies a plurality of linear responses after convolution, negates the linear responses, concatenates both parts of the activation, and applies a rectified linear unit (ReLU).

37. The method of claim 34, wherein the Concatenated Rectified Linear Unit (CReLU) maintains positive and negative phase information for a plurality of features by mathematically characterizing the convolutional layers in terms of a reconstruction property.

38. The method of claim 34, wherein building the plurality of middle layers of the CNN comprises reducing redundancy in filters of the CNN via residual functions with reference to the inputs to the middle layers.

39. The method of claim 34, wherein the building the plurality of middle layers of the CNN is to increase a depth of the CNN by stacking composite layers specifically designed to fit a residual mapping.

40. The method of claim 34, wherein the building the plurality of middle layers of the CNN results in a same number of filters for a same output feature map size, and in response to the feature map size being halved, the number of filters is doubled.

41. The method of claim 34, wherein building the top layers of the CNN comprises an inception-residual block that generates a multi-scale feature map comprising a plurality of features at different scales.

42. A system for dynamic emotion recognition in unconstrained scenarios, comprising, comprising

a three-channel input comprising a gray-scale face image, a corresponding basic Local Binary Patterns (LBP), and mean LBP feature maps;

a memory that is to store instructions and the three channel input; and

a processor communicatively coupled to the memory, wherein when the processor is to execute the instructions, the processor is to;

building a plurality of lower levels of a CNN using the three channel input, wherein filters of the lower levels form pairs in phase;

building a plurality of middle layers of the CNN via a plurality of residual functions;

building a plurality of top layers of the CNN via multi-scale feature extraction that is to extract a plurality of features; and

classifying the extracted plurality of features.

43. The system of claim 42, wherein building the lower levels of the CNN comprises:

generating an identical copy of a plurality of linear responses after convolution;

negating results of the convolution;

applying a Concatenated Rectified Linear Unit (CReLU); and

max-pooling to determine a number of feature maps.

44. The system of claim 42, wherein a fully connected mechanism comprises a first fully connected layer that is to perform a final feature abstraction and a second fully connected layer that is to classify the final features.

45. The system of claim 42, wherein the multi-scale feature extraction increases a width of the CNN.

46. A tangible, non-transitory, computer-readable medium comprising instructions that, when executed by a processor, direct the processor to:

pre-process image data;

build a plurality of lower levels of a CNN using the image data wherein filters of the lower levels form pairs in phase;

build a plurality of middle layers of the CNN via a plurality of residual functions;

build a plurality of top layers of the CNN via multi-scale feature extraction that is to extract a plurality of features; and

classify the extracted plurality of features.

47. The computer readable medium of claim 46, wherein building the lower levels of the CNN comprises:

generating an identical copy of a plurality of linear responses after convolution;

negating results of the convolution;

applying a Concatenated Rectified Linear Unit (CReLU); and

max-pooling to determine a number of feature maps.

48. The computer readable medium of claim 46, wherein building the plurality of middle layers of the CNN comprises reducing redundancy in filters of the CNN via residual functions with reference to the inputs to the middle layers.

49. The computer readable medium of claim 46, wherein the building the plurality of middle layers of the CNN is to increase a depth of the CNN by stacking composite layers specifically designed to fit a residual mapping.

50. The computer readable medium of claim 46, wherein the building the plurality of middle layers of the CNN results in a same number of filters for a same output feature map size, and in response to the feature map size being halved, the number of filters is doubled.