APPARATUS AND METHOD FOR TRACKING OBJECT USING FEATURE DESCRIPTOR, AND APPARATUS AND METHOD FOR REMOVING GARBAGE FEATURE

• US 20130322763A1
• Filed: 05/31/2013
• Published: 12/05/2013
• Est. Priority Date: 05/31/2012
• Status: Active Grant

• Current Assignee: Samsung SDS (China) Co., Ltd. (Samsung Group)
• Original Assignee: Samsung SDS (China) Co., Ltd. (Samsung Group)
• Inventors: HEU, Jun Hee, YI, Jeong Seon, JEONG, Yong Wook
• Granted Patent: US 9,552,642 B2
• Time in Patent Office: Days
• US Class Current: 382/195
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration

Abstract

An apparatus and method for tracking an object using a feature descriptor and an apparatus and method for removing a garbage feature are disclosed. A feature descriptor generation unit generates a plurality of features descriptors indicating information of a plurality of features extracted from an input image from which an object of interest is desired to be detected. A matching unit matches the feature descriptors with feature descriptors of a target object stored in advance, and determines the feature descriptors of the object of interest corresponding to the target object. A feature point removal unit removes feature descriptors that do not meet a geometric comparison condition from among the feature descriptors of the object of interest, and establishes final feature descriptors of the object of interest.

Claims

1. An apparatus configured to track an object, the apparatus comprising:

a feature descriptor generator configured to generate feature descriptors indicating information of a plurality of features extracted from an input image from which an object of interest is to be detected and defining extracted feature descriptors of the object of interest;

a matcher configured to match the extracted feature descriptors of the object of interest with target object feature descriptors of a target object stored in advance, and to determine one or more of the extracted feature descriptors of the object of interest corresponding to the target object; and

a feature point remover configured to remove extracted feature descriptors of the object of interest that do not meet a geometric comparison condition from among the extracted feature descriptors of the object of interest so as to define one or more final feature descriptors of the object of interest.

2. The apparatus according to claim 1, wherein the feature descriptor generator comprises:

an image pyramid generator configured to generate an image pyramid including differential images having a resolution which gradually changes from the input image;

a corner point detector configured to detect a corner feature from each differential image constituting the image pyramid, and determine a corner feature detected in common from the respective differential images of the image pyramid as a scale invariant feature; and

a descriptor generator configured to generate a feature descriptor for the scale invariant feature.

3. The apparatus according to claim 2, wherein:

the image pyramid generator is configured to generate the differential images by applying a Gaussian filter to the input image by a previously set number of times.

4. The apparatus according to claim 2, wherein:

when a previously set number of continuous pixels, among pixels located at a previously set distance from a center point that is a pixel selected from one of the differential images, have a pixel value which is greater or smaller than a previously set threshold value in comparison to the center point, the corner point detector determines the center point is the corner feature.

5. The apparatus according to claim 2, wherein:

the descriptor generator is configured to divide a pixel area having a previously set size around the scale invariant feature into a plurality of subareas, and generate the feature descriptor in a vector form corresponding to the scale invariant feature based on a histogram of a gradient value generated for each subarea.

6. The apparatus according to claim 5, wherein:

the descriptor generator is configured to generate the feature descriptor in a 36-dimensional vector form by dividing the pixel area into 9 subareas, and calculate a histogram for four gradient directions to correspond to the respective subareas.

7. The apparatus according to claim 1, wherein:

the feature point remover is configured to establish the final feature descriptors of the object of interest using a geometric comparison condition in which a ratio of perspective transformation determinants calculated for the extracted feature descriptors of the object of interest is equal to a ratio of perspective transformation determinants calculated for the target object feature descriptors of the target object.

8. The apparatus according to claim 7, wherein:

the feature point remover is configured to establish the final feature descriptors of the object of interest by repeatedly performing, on all the extracted feature descriptors of the object of interest, a process of removing a previously set number of extracted feature descriptors that do not meet the geometric comparison condition from five extracted feature descriptors randomly selected from among the extracted feature descriptors of the object of interest, adding a new feature descriptor, and applying the geometric comparison condition.

9. The apparatus according to claim 1, further comprising:

a storage configured to store the final feature descriptors of the object of interest in a database.

10. An apparatus for removing a garbage feature, the apparatus comprising:

an input device configured to receive feature descriptors of an object of interest extracted from an input image and defining extracted feature descriptors of the object of interest through matching with target object feature descriptors of a target object stored in advance;

a feature processor configured to apply a geometric comparison condition to the extracted feature descriptors of the object of interest; and

a feature establisher configured to remove extracted feature descriptors of the object of interest that do not meet the geometric comparison condition from among the extracted feature descriptors of the object of interest so as to establish final feature descriptors of the object of interest.

11. The apparatus according to claim 10, wherein:

the feature processor is configured to use a geometric comparison condition in which a ratio of perspective transformation determinants calculated for the extracted feature descriptors of the object of interest is equal to a ratio of perspective transformation determinants calculated for the target object feature descriptors of the target object corresponding to the extracted feature descriptors of the object of interest.

12. The apparatus according to claim 10, wherein:

the feature processor is configured to repeatedly perform, on all the extracted feature descriptors of the object of interest, a process of applying the geometric comparison condition to five feature descriptors randomly selected from among the extracted feature descriptors of the object of interest and target object feature descriptors of the corresponding target object, adding a new feature descriptor to other extracted feature descriptors of the object of interest after the extracted feature descriptors of the object of interest that do not meet the geometric comparison condition are removed by the feature establisher, and applying the geometric comparison condition.

13. A method of tracking an object, the method comprising:

(a) generating feature descriptors indicating information of a plurality of features extracted from an input image from which an object of interest is to be detected and defining extracted feature descriptors;

(b) matching the extracted feature descriptors of the object of interest with target object feature descriptors of a target object stored in advance, and determining one or more of the extracted feature descriptors of the object of interest corresponding to the target object; and

(c) removing extracted feature descriptors of the object of interest that do not meet a geometric comparison condition from among the extracted feature descriptors of the object of interest so as to establish final feature descriptors of the object of interest.

14. The method according to claim 13, wherein operation (a) includes:

(a1) generating an image pyramid including differential images having a resolution which gradually changes from the input image;

(a2) detecting a corner feature from each differential image constituting the image pyramid, and determining the corner feature detected in common from the respective differential images of the image pyramid as a scale invariant feature; and

(a3) generating a feature descriptor for the scale invariant feature.

15. The method according to claim 14, wherein:

operation (a1) includes generating the differential images by applying a Gaussian filter to the input image by a previously set number of times.

16. The method according to claim 14, wherein:

when a previously set number of continuous pixels, among pixels located at a previously set distance from a center point that is a pixel selected from the differential image, have a pixel value which is greater or smaller than a previously set threshold value in comparison to the center point, operation (a2) includes determining the center point as the corner feature.

17. The method according to claim 14, wherein:

operation (a3) includes dividing a pixel area having a previously set size around the scale invariant feature into a plurality of subareas, and generating the feature descriptor in a vector form corresponding to the scale invariant feature based on a histogram of a gradient value generated for each subarea.

18. The method according to claim 17, wherein:

operation (a3) includes generating the feature descriptor in a 36-dimensional vector form by dividing the pixel area into 9 subareas, and calculating a histogram for four gradient directions to correspond to the respective subareas.

19. The method according to claim 13, wherein:

operation (c) includes establishing the final feature descriptors of the object of interest using a geometric comparison condition in which a ratio of perspective transformation determinants calculated for the extracted feature descriptors of the object of interest is equal to a ratio of perspective transformation determinants calculated for the target object feature descriptors of the target object.

20. The method according to claim 19, wherein:

operation (c) includes establishing the final feature descriptors of the object of interest by repeatedly performing, on all the extracted feature descriptors of the object of interest, a process of removing a previously set number of extracted feature descriptors of the object of interest that do not meet the geometric comparison condition from five extracted feature descriptors of the object of interest randomly selected from among the extracted feature descriptors of the object of interest, adding a new feature descriptor, and applying the geometric comparison condition.

21. The method according to claim 13, further comprising:

(d) storing the final feature descriptors of the object of interest in a database.

22. A method of removing a garbage feature, the method comprising:

(a) receiving feature descriptors of an object of interest extracted from an input image and defining extracted feature descriptors through matching with target object feature descriptors of a target object stored in advance;

(b) applying a geometric comparison condition to the extracted feature descriptors of the object of interest; and

(c) removing extracted feature descriptors of the object of interest that do not meet the geometric comparison condition from among the extracted feature descriptors of the object of interest so as to establish final feature descriptors of the object of interest.

23. The method according to claim 22, wherein:

operation (b) includes using a geometric comparison condition in which a ratio of perspective transformation determinants calculated for the extracted feature descriptors of the object of interest is equal to a ratio of perspective transformation determinants calculated for the target object feature descriptors of the target object corresponding to the extracted feature descriptors of the object of interest.

24. The method according to claim 22, wherein:

operation (b) includes repeatedly performing, on all the extracted feature descriptors of the object of interest, a process of applying the geometric comparison condition to five extracted feature descriptors of the object of interest randomly selected from among the extracted feature descriptors of the object of interest and target object feature descriptors of the corresponding target object, adding a new feature descriptor to other extracted feature descriptors of the object of interest after the extracted feature descriptors of the object of interest that do not meet the geometric comparison condition are removed by the removing of the extracted feature descriptors of the object of interest, and applying the geometric comparison condition.

25. A non-transitory computer-readable medium having recorded thereon a program which causes a computer to execute a method of tracking an object, the method comprising:

(a) generating feature descriptors indicating information of a plurality of features extracted from an input image from which an object of interest is to be detected and defining extracted feature descriptors;

(b) matching the extracted feature descriptors of the object of interest with target object feature descriptors of a target object stored in advance, and determining one or more of the extracted feature descriptors of the object of interest corresponding to the target object; and

(c) removing extracted feature descriptors of the object of interest that do not meet a geometric comparison condition from among the extracted feature descriptors of the object of interest so as to establish final feature descriptors of the object of interest.

26. A non-transitory computer-readable medium having recorded thereon a program which causes a computer to execute a method of removing a garbage feature, the method comprising:

(a) receiving feature descriptors of an object of interest extracted from an input image and defining extracted feature descriptors through matching with target object feature descriptors of a target object stored in advance;

(b) applying a geometric comparison condition to the extracted feature descriptors of the object of interest; and

(c) removing extracted feature descriptors of the object of interest that do not meet the geometric comparison condition from among the extracted feature descriptors of the object of interest so as to establish final feature descriptors of the object of interest.

27. An apparatus to be implemented in a mobile device, the apparatus comprising:

a garbage feature remover configured to remove feature descriptors that do not satisfy a geometric comparison condition from among feature descriptors of an object of interest, and to establish final feature descriptors of the object of interest based on the removed feature descriptors; and

a storage configured to store the final feature descriptors and to further store a target object,

wherein the geometric comparison condition comprises a condition based on matching a first ratio of first perspective transformation determinants calculated for the feature descriptors of the object of interest with a second ratio of second perspective transformation determinants calculated for target object feature descriptors of the stored target object.

28. The apparatus according to claim 27, wherein, to determine whether the feature descriptors of the object of interest do not satisfy the geometric comparison condition, the garbage feature remover uses five points located on a single plane on which both the object of the interest and the stored target object are represented.

29. The apparatus according to claim 28, wherein the garbage feature remover uses the geometric comparison condition to generate an invariant perspective transformation ratio of determinants based on the five points.