Apparatus and method for generating shape model of object and apparatus and method for automatically searching for feature points of object employing the same

• US 20060210168A1
• Filed: 03/02/2006
• Published: 09/21/2006
• Est. Priority Date: 03/02/2005
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kee, Seokcheol, Kim, Whoiyul, Hwang, Wonjun
• Granted Patent: US 7,711,156 B2
• Time in Patent Office: Days
• US Class Current: 382/203
• CPC Class Codes: G06V 10/7557 based on appearance, e.g. active appearance models [AAM]
• CPC Class Codes: G06V 40/169 Holistic features and representations, i.e. based on the facial image taken as a whole

Abstract

An apparatus and method for generating a shape model of an object and an apparatus and method for automatically searching for feature points of an object employing the same. The apparatus for automatically searching for feature points of an object includes: a shape model generation unit which generates a shape model including a mean model of an object, and statistical feature data of each feature point forming a mean shape, by training a plurality of training sets in which a plurality of feature points are labeled for the object; an initial position resetting unit which resets an initial position of each feature point of the mean shape according to a pose of an input image; and a feature point extraction unit which adapts the mean shape whose initial position is reset, to the input image, by using the statistical feature data of each feature point of the mean shape, and extracts feature points of the input image.

Claims

1. An apparatus for searching for feature points of an object, comprising:

a shape model generation unit which generates a shape model including a mean model of an object, and statistical feature data of each feature point forming a mean shape, by training a plurality of training sets in which a plurality of feature points are labeled for the object;

an initial position resetting unit which resets an initial position of each feature point of the mean shape according to a pose of an input image; and

a feature point extraction unit which adapts to the input image the mean shape whose initial position is reset, using the statistical feature data of each feature point of the mean shape, and extracts feature points of the input image.

2. The apparatus of claim 1, wherein the shape model generation unit comprises:

a training database which stores the plurality of training sets in which the plurality of feature points are labeled for the object;

a mean shape generation unit which generates the mean shape of the plurality of training sets; and

a statistical feature data generation unit which generates the statistical feature data for an image structure in a vicinity of a feature point, using a 2-dimensional main template of each feature point forming the mean shape.

3. The apparatus of claim 2, wherein the statistical feature data generation unit normalizes all pixels belonging to the 2-dimensional main template of each feature point, using a gray level mean of an area in which a characteristic of the object is represented.

4. The apparatus of claim 1, wherein, for a plurality of groups formed with a plurality of feature points obtained from the input image, the initial position resetting unit performs a genetic algorithm, using an objective function generated from the statistical feature data of each of the feature points.

5. The apparatus of claim 1, wherein the feature point extraction unit further comprisesa shape model adaptation unit which extracts feature points of the input image, by performing a shape model adaptation process in units of 2-dimensional sub-templates in the 2-dimensional main template of each feature point of the mean shape whose initial position is reset on the input image.

6. The apparatus of claim 5, wherein the shape model adaptation unit performs normalization of all pixels belonging to the 2-dimensional main template of each feature point, using a gray level mean of an area in which a characteristic of the input image is represented.

7. A method of searching for feature points of an object, comprising:

generating a shape model including a mean model of an object, and statistical feature data of each feature point forming a mean shape, by training a plurality of training sets in which a plurality of feature points are labeled for the object;

resetting an initial position of each feature point of the mean shape according to a pose of an input image; and

adapting to the input image the mean shape whose initial position is reset, using the statistical feature data of each feature point of the mean shape, and extracting feature points of the input image.

8. The method of claim 7, wherein the generating of the shape model comprises:

providing a training database which stores the plurality of training sets in which the plurality of feature points are labeled for the object;

generating the mean shape of the plurality of training sets; and

generating the statistical feature data for an image structure in a vicinity of a feature point, using a 2-dimensional main template of each feature point forming the mean shape.

9. The method of claim 8, wherein, in the generating the statistical feature data, by using the following equation, all pixels belonging to the 2-dimensional main template of each point are normalized:

$\begin{array}{cc}{m}_i=\frac{1}{N}<br><br>\underset{j=1}{\overset{N}{\sum <br><br>}}<br><br>t_{\mathrm{ij}}<br><br><br><br>{\mathrm{\sigma <br><br>}}_i^2=\frac{1}{N}<br><br>\underset{j=1}{\overset{N}{\sum <br><br>}}<br><br>{\left(t_{\mathrm{ij}}-m_i\right)}^2<br><br><br><br>t_{\mathrm{ij}}-><br><br>\frac{t_{\mathrm{ij}}-m_i}{\mathrm{\sigma <br><br>}}\times <br><br>128+128+n-m_i,\mathrm{and}& \text{\hspace{1em}<br><br>}\end{array}$

wherein m_i denotes a gray level mean of pixels (t_ij) belonging to the main template obtained from an i-th feature point, σ

_i² denotes a variance between each pixel (t_ij) and a mean (m_i), N denotes a number of samples belonging to the main template, and n denotes a gray level mean of an area in which a characteristic of the input image is represented.

10. The method of claim 7, wherein, in the resetting the initial position, for a plurality of groups formed with a plurality of feature points obtained from the input image, a genetic algorithm is performed by using an objective function generated from the statistical feature data of each of the feature points.

11. The method of claim 10, wherein the objective function is defined by the following equation:

$f<br><br>\left(x\right)=\underset{i=0}{\overset{n-1}{\sum <br><br>}}<br><br>{\left(g<br><br>\left(x_i,y_i\right)-{\stackrel{\_}{m}}_i\right)}^T<br><br>S_i^{-1}<br><br>\left(g<br><br>\left(x_i,y_i\right)-{\stackrel{\_}{m}}_i\right),\mathrm{and}$

wherein g(x_i, y_i) denotes a value obtained by setting a main template for an input image while taking (x_i, y_i) as a center of the template, and then performing a normalization process for a gray level difference value between each pixel included in the main template and an adjacent pixel, {overscore (m)}_i denotes a mean of the main template in relation to an i-th feature point forming the mean shape ({overscore (x)}), and S_i<sup>−

1</sup> denotes a inverse matrix of a covariance matrix of the main template of the i-th feature point forming the mean shape ({overscore (x)}).

12. The method of claim 7, wherein the extracting the feature points comprises extracting feature points of the input image, by performing a shape model adaptation process using a 2-dimensional main template of each feature point of the mean shape whose initial position is reset in the input image.

13. The method of claim 12, wherein, in the process for adapting a shape model, by using the following equation, all pixels belonging to the 2-dimensional main template of each point are normalized:

$\begin{array}{cc}{m}_i=\frac{1}{N}<br><br>\underset{j=1}{\overset{N}{\sum <br><br>}}<br><br>t_{\mathrm{ij}}<br><br><br><br>{\mathrm{\sigma <br><br>}}_i^2=\frac{1}{N}<br><br>\underset{j=1}{\overset{N}{\sum <br><br>}}<br><br>{\left(t_{\mathrm{ij}}-m_i\right)}^2<br><br><br><br>t_{\mathrm{ij}}-><br><br>\frac{t_{\mathrm{ij}}-m_i}{\mathrm{\sigma <br><br>}}\times <br><br>128+128+n-m_i,\mathrm{and}& \text{\hspace{1em}<br><br>}\end{array}$

wherein m_i denotes a gray level mean of pixels (t_ij) belonging to the main template obtained from an i-th feature point, σ

_i² denotes a variance between each pixel (t_ij) and a mean (m_i), N denotes a number of samples belonging to the main template, and n denotes a gray level mean of an area in which a characteristic of the object is represented.

14. A computer readable recording medium having embodied thereon a computer program for executing a method of searching for feature points of an object, the method comprising:

generating a shape model including a mean model of an object, and statistical feature data of each feature point forming a mean shape, by training a plurality of training sets in which a plurality of feature points are labeled for the object;

resetting an initial position of each feature point of the mean shape according to the pose of an input image; and

adapting to the input image the mean shape whose initial position is reset, using the statistical feature data of each feature point of the mean shape, and extracting feature points of the input image.

15. An apparatus for generating a shape model of an object, comprising:

a training database which stores a plurality of training sets in which a plurality of feature points are labeled for an object;

a mean shape generation unit which generates a mean shape of the plurality of training sets; and

a statistical feature data generation unit which generates statistical feature data for an image structure in a vicinity of a feature point, using a 2-dimensional main template of each feature point forming the mean shape.

16. The apparatus of claim 15, wherein the statistical feature data generation unit normalizes all pixels belonging to the 2-dimensional main template of each feature point, using a gray level mean of an area in which a characteristic of the object is represented.

17. A method of generating a shape model of an object, comprising:

providing a training database which stores a plurality of training sets in which a plurality of feature points are labeled for an object;

generating a mean shape of the plurality of training sets; and

generating statistical feature data for an image structure in a vicinity of a feature point, using a 2-dimensional main template of each feature point of the mean shape.

18. The method of claim 17, wherein, in the generating the statistical feature data, by using the following equation, all pixels belonging to the 2-dimensional main template of each point are normalized:

$m_i<br><br>\frac{1}{N}<br><br>\underset{j=1}{\overset{N}{\sum <br><br>}}<br><br>t_{\mathrm{ij}}$ ${\mathrm{\sigma <br><br>}}_i^2=\frac{1}{N}<br><br>\underset{j=1}{\overset{N}{\sum <br><br>}}<br><br>{\left(t_{\mathrm{ij}}-m_i\right)}^2,\mathrm{and}$ $t_{\mathrm{ij}}\to <br><br>\frac{t_{\mathrm{ij}}-m_i}{\mathrm{\sigma <br><br>}}\times <br><br>128+128+n-m_i$

wherein m_i denotes a gray level mean of pixels (t_ij) belonging to the main template obtained from an i-th feature point, σ

_i² denotes a variance between each pixel (t_ij) and a mean (m_i), N denotes a number of samples belonging to the main template, and n denotes a gray level mean of an area in which a characteristic of the input image is represented.

19. A computer readable recording medium having embodied thereon a computer program for executing a method of generating a shape model of an object, wherein the method comprises:

providing a training database which stores a plurality of training sets in which a plurality of feature points are labeled for an object;

generating a mean shape of the plurality of training sets; and

generating statistical feature data for an image structure in a vicinity of a feature point, using a 2-dimensional main template of each feature point of the mean shape.