Method and apparatus for positioning feature point

• US 10,096,132 B2
• Filed: 01/26/2017
• Issued: 10/09/2018
• Est. Priority Date: 01/27/2016
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Zhou, Mingcai, Heo, Jingu, Hong, Tao, Li, Weiming, Wang, Zairan, Liu, Zhihua, Nam, Dong Kyung, Wang, Haitao
• Primary Examiner(s): Seth, Manav
• Application Number:
  US15/416,271
• Publication Number:
  US 20170213359A1
• Time in Patent Office: 621 Days
• CPC Class Codes: G06F 18/213 Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/2415 based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 7/40 Analysis of texture depth or shape recovery from texture G06T7/529
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 10/17 using hand-held instruments
• CPC Class Codes: G06V 10/42 Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/7715 Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods
• CPC Class Codes: G06V 10/774 Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 40/165 using facial parts and geometric relationships
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation

Abstract

A feature point positioning apparatus includes a memory storing computer-executable instructions; and one or more processors configured to execute the computer-executable instructions such that the one or more processors are configured to, iteratively update a first form coefficient based on, a nonlinear feature extracted from an image, and a regression factor matrix obtained through training, and detect a position of the feature point of the image based on, the updated first form coefficient, and a statistical form model obtained through training.

Claims

1. A feature point positioning apparatus comprising:

a memory storing computer-executable instructions; and

one or more processors configured to execute the computer-executable instructions such that the one or more processors are configured to,

iteratively update a first form coefficient based on,

a nonlinear feature extracted from an image, and

a regression factor matrix obtained through training, and

detect a position of the feature point of the image based on,

the updated first form coefficient, and

a statistical form model obtained through training.

2. The feature point positioning apparatus of claim 1, wherein the one or more processors are configured to execute the computer-executable instructions such that the one or more processors are configured to obtain a similarity transformation matrix by arranging an intermediate form and an average form obtained through an update, and perform a similarity transformation on the intermediate form and the image based on the similarity transformation matrix.

3. The feature point positioning apparatus of claim 2, wherein the one or more processors are configured to execute the computer-executable instructions such that the one or more processors are configured to,

arrange the intermediate form and the average form obtained through the update using a high resolution pyramid frame,

perform positioning on a feature point of a first resolution image based on a threshold, and

obtain the similarity transformation matrix by performing positioning on a feature point of a second resolution image and mapping, to the second resolution image, a result of the performing of the positioning on the feature point of the first resolution image, and a second resolution is higher than a first resolution.

4. The feature point positioning apparatus of claim 3, wherein the one or more processors are configured to execute the computer-executable instructions such that the one or more processors are configured to perform inverse transformation, using a coordinate system of the image, on a feature point obtained based on the similarity transformation matrix obtained through an iterative update.

5. The feature point positioning apparatus of claim 2, wherein,

the one or more processors are configured to execute the computer-executable instructions such that the one or more processors are configured to obtain a similarity transformation matrix A_k used for a k-th iterative update of the first form coefficient using a target function arg min_Ak∥

S₀−

A_k·

s(P_kⁱ)∥

²,

S₀ denotes an average form, and

s(P_kⁱ) is an i-th sample form after the iterative update is performed k times.

6. The feature point positioning apparatus of claim 1, wherein the one or more processors are configured to execute the computer-executable instructions such that the one or more processors are configured to determine the regression factor matrix based on the nonlinear feature and an extent to which a second form coefficient of a training sample is updated.

7. The feature point positioning apparatus of claim 6, wherein the one or more processors are configured to execute the computer-executable instructions such that the one or more processors are configured to learn an average texture feature from the training sample and determine a value of a difference between the average texture feature and a nonlinear texture feature extracted from the training sample as a nonlinear feature of the training sample.

8. The feature point positioning apparatus of claim 6, wherein the one or more processors are configured to execute the computer-executable instructions such that the one or more processors are configured such that a nonlinear texture feature is one type of the nonlinear texture feature or a combination of at least two types of the nonlinear texture feature, and the one or more processors use, in different iterations, the combination of at least two types of the nonlinear texture feature or identical nonlinear texture features and different nonlinear texture features.

9. The feature point positioning apparatus of claim 6, wherein the one or more processors are configured to execute the computer-executable instructions such that the one or more processors are configured to determine the regression factor matrix by minimizing an error between the extent to which the second form coefficient is updated and an extent to which a third form coefficient is updated, calculated based on the nonlinear feature.

10. The feature point positioning apparatus of claim 9 wherein,

the one or more processors are configured to execute the computer-executable instructions such that the one or more processors are configured to obtain a regression factor matrix R_k used for a k-th iterative update of the third form coefficient using a target function arg min_RkΣ

_i=1∥

Δ

p_kⁱ−

R_k*Ø

_kⁱ∥

²,

N denotes a total number of training samples,

Δ

p_kⁱ=p_*ⁱ−

p_kⁱ, denotes the extent to which the second form coefficient of an i-th sample is updated when the iterative update is performed k times,

Ø

_kⁱ denotes a nonlinear feature of the i-th sample when the iterative update is performed k times, and

R_k*Ø

_kⁱ denotes the extent to which the third form coefficient is updated, obtained and calculated based on the nonlinear feature of the i-th sample, when the iterative update is performed k times.

11. A method of positioning a feature point, the method comprising:

extracting a nonlinear feature of an image;

iteratively updating a first form coefficient based on the nonlinear feature and a regression factor matrix obtained through training; and

detecting a position of a feature point of the image based on the updated first form coefficient and a statistical form model obtained through training.

12. The method of claim 11, wherein the updating comprises:

obtaining a similarity transformation matrix by arranging an intermediate form and an average form obtained through an update; and

performing a similarity transformation on the intermediate form and the image based on the similarity transformation matrix.

13. The method of claim 12 wherein,

the obtaining comprises:

arranging the intermediate form and the average form obtained through the update using a high resolution pyramid frame;

performing positioning on a feature point of a first resolution image based on a threshold; and

performing positioning on a feature point of a second resolution image by mapping, to a second resolution image, a result of the performing positioning on a feature point of a first resolution image, and

a resolution of the second resolution image is higher than a resolution of the first resolution image.

14. The method of claim 13, further comprising:

performing inverse transformation, using a coordinate system of the image, on a feature point obtained based on the similarity transformation matrix obtained through an iterative update.

15. The method of claim 12 wherein,

a similarity transformation matrix A_k used for a k-th iterative update of the first form coefficient is obtained using a target function arg min_Ak∥

S₀−

A_k·

s(P_kⁱ)∥

²,

S₀ denotes an average form, and

s(P_kⁱ) is an i-th sample form after the iterative update is performed k times.

16. The method of claim 11, further comprising:

determining a nonlinear feature of a training sample; and

obtaining the regression factor matrix based on the nonlinear feature of the training sample and an extent to which a second form coefficient of the training sample is updated.

17. The method of claim 16, wherein the nonlinear feature of the training sample includes a nonlinear texture feature, and the determining of the nonlinear feature of the training sample includes learning an average texture feature from the training sample, and determining a value of a difference between the average texture feature and a nonlinear texture feature extracted from the training sample as a nonlinear feature of the training sample.

18. The method of claim 16, wherein a nonlinear texture feature is a combination of at least one type of the nonlinear texture feature, and the combination of at least one type of the nonlinear texture feature is used when the iterative update is performed.

19. The method of claim 16, wherein the obtaining comprises:

determining the regression factor matrix by minimizing an error between the extent to which the second form coefficient is updated and an extent to which a third form coefficient is updated, calculated based on the nonlinear feature of the training sample.

20. The method of claim 19, wherein,

a regression factor matrix R_k used for a k-th iterative update of the third form coefficient is obtained using a target function arg min_RkΣ

_i=1^N∥

Δ

p_kⁱ−

R_k*Ø

_kⁱ∥

²,

N denotes a total number of training samples,

Δ

p_kⁱ=p_*ⁱ−

p_kⁱ denotes the extent to which the second form coefficient of an i-th sample is updated when the iterative update is performed k times,

Ø

_kⁱ denotes a nonlinear feature of the i-th sample when the iterative update is performed k times, and

R_k*Ø

_kⁱ denotes the extent to which the third form coefficient is updated, obtained and calculated based on the nonlinear feature of the i-th sample, when the iterative update is performed k times.