Method and apparatus for recognising expression using expression-gesture dictionary

• US 10,068,131 B2
• Filed: 01/29/2014
• Issued: 09/04/2018
• Est. Priority Date: 01/28/2014
• Status: Active Grant

• Current Assignee: Industry-Academic Cooperation Foundation, Yeungnam University
• Original Assignee: Industry-Academic Cooperation Foundation, Yeungnam University
• Inventors: Lee, Chan Su, Jang, Ja Soon
• Primary Examiner(s): Saini, Amandeep
• Application Number:
  US15/114,445
• Publication Number:
  US 20160342828A1
• Time in Patent Office: 1,679 Days
• Field of Search: 382103
• CPC Class Codes: G06F 18/2413 based on distances to training or reference patterns
• CPC Class Codes: G06F 18/28 Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/176 Dynamic expression

Abstract

An apparatus for recognizing expression using an expression-gesture dictionary, includes a learning image acquisitor to obtain data from a learning expression, perform a normalization based on the data, track a change of a dense motion from a reference frame, and generate expression learning data, an expression-gesture dictionary and expression-gesture dictionary learner to represent and store a numerical value for expression recognition for each expression using a local support map in an image coordinate space for a motion flow with respect to a set of changes of the dense motion, an expression classifier learner to learn an expression classification for each expression based on a weight of data on the expression-gesture dictionary, a recognition image acquisitor to obtain data from a recognition target, and generate recognition data, and an expression recognizer to analyze an expression weight on the recognition data, and recognize an expression by the expression classifier learner.

Claims

1. An apparatus for recognizing an expression using an expression-gesture dictionary, the apparatus comprising:

a learning image acquisitor configured to obtain position data of a face and eyes from a learning expression, perform a first normalization based on the obtained position data, track a change of a dense motion of the learning expression from a reference frame of a neutral expression, and generate expression learning data;

an expression-gesture dictionary and expression-gesture dictionary learner configured to represent and store a numerical value for expression recognition for each expression according to a dictionary learning method satisfying a given limiting condition using a local support map in an image coordinate space for a motion flow with respect to a set of changes of the dense motion of the learning expression after initializing the expression-gesture dictionary;

an expression classifier learner configured to learn an expression classification for each expression based on a weight of data on the expression-gesture dictionary;

a recognition image acquisitor configured to obtain position data of a face and eyes from a recognition target, perform a second normalization based on the obtained position data, track a change of a dense motion of the recognition target from the reference frame of the neutral expression, and generate recognition data; and

an expression recognizer configured to analyze an expression weight on data to be recognized, determine a closest classification by the expression classifier learner, and recognize an expression,

wherein the first normalization removes a peripheral region regardless of an expression in a facial region by giving an offset as a predetermined ratio based on positions of two eyes after aligning a center of the two eyes to be a reference point based on positions of the two eyes which are detected, and sets a position coordinate of a feature portion.

2. The apparatus for recognizing the expression using the expression-gesture dictionary of claim 1, wherein the expression-gesture dictionary learner obtains the numerical value for expression recognition by a method of obtaining an optimal value for an object function of Equation 1 according to a dictionary learning method,

3. The apparatus for recognizing the expression using the expression-gesture dictionary of claim 1, wherein the expression-gesture dictionary learner obtains an optimal weight α

_i by Equation 2 when a numerical value D of the expression-gesture dictionary is given,

4. The apparatus for recognizing the expression using the expression-gesture dictionary of claim 1, wherein, when a collection of given weight values is referred to as A, the expression-gesture dictionary learner obtains a numerical value D of the expression-gesture dictionary by Equation 3,

5. A method for recognizing an expression using an expression-gesture dictionary, comprising:

obtaining a learning image including obtaining position data of a face and eyes from a learning expression, performing a first normalization based on the obtained position data, tracking a change of a dense motion from a reference frame of a neutral expression, and generating expression learning data;

learning an expression-gesture dictionary including representing and storing a numerical value for expression recognition for each expression according to a dictionary learning method satisfying a given limiting condition using a local support map in an image coordinate space for a motion flow with respect to a set of changes of the dense motion for the learning expression after initializing the expression-gesture dictionary;

learning an expression classifier including learning an expression classification for each expression based on a weight of data on the expression-gesture dictionary;

obtaining a recognition image including obtaining position data of a face and eyes from a recognition target, performing a second normalization based on the obtained position data, tracking a change of a dense motion from the reference frame of the neutral expression, and generating recognition data; and

recognizing an expression including analyzing an expression weight on data to be recognized, determining a closest classification by the expression classifier learner, and recognizing an expression,

wherein the first normalization removes a peripheral region regardless of an expression in a facial region by giving an offset as a predetermined ratio based on positions of two eyes after aligning a center of the two eyes to be a reference point based on positions of the two eyes which are detected, and sets a position coordinate of a feature portion.

6. The method for recognizing the expression using the expression-gesture dictionary of claim 5, wherein the learning of the expression-gesture dictionary further comprises:

estimating a dictionary weight after initializing the expression-gesture dictionary,

wherein the estimating of the dictionary weight is performed based on a regeneration error test, and

the regeneration error test obtains a combined weight of dictionary data obtained through dictionary learning and dictionary data on given original data, generates estimation data corresponding to the original data based on a component of each dictionary data and a value of the obtained combined weight, and obtains a difference between the estimation data and the original data.

7. The method for recognizing the expression using the expression-gesture dictionary of claim 5, wherein the learning of the expression-gesture dictionary obtains the numerical value for expression recognition by a method of obtaining an optimal value for an object function of Equation 1 according to a dictionary learning method,