Apparatus and method for tracking object using feature descriptor, and apparatus and method for removing garbage feature

• US 9,552,642 B2
• Filed: 05/31/2013
• Issued: 01/24/2017
• Est. Priority Date: 05/31/2012
• Status: Active Grant

• Current Assignee: Samsung SDS (China) Co., Ltd. (Samsung Group)
• Original Assignee: Samsung SDS (China) Co., Ltd. (Samsung Group)
• Inventors: Yi, Jeong Seon, Jeong, Yong Wook, Heu, Jun Hee
• Primary Examiner(s): Park, Edward
• Application Number:
  US13/906,783
• Publication Number:
  US 20130322763A1
• Time in Patent Office: 1,334 Days
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration

Abstract

An apparatus and method for tracking an object using a feature descriptor and an apparatus and method for removing a garbage feature are disclosed. A feature descriptor generation unit generates a plurality of features descriptors indicating information of a plurality of features extracted from an input image from which an object of interest is desired to be detected. A matching unit matches the feature descriptors with feature descriptors of a target object stored in advance, and determines the feature descriptors of the object of interest corresponding to the target object. A feature point removal unit removes feature descriptors that do not meet a geometric comparison condition from among the feature descriptors of the object of interest, and establishes final feature descriptors of the object of interest.

Claims

1. An apparatus configured to track an object, the apparatus comprising:

a processor comprising;

a feature descriptor generator configured to generate feature descriptors indicating information of a plurality of features extracted from an input image from which an object of interest is to be detected and defining extracted feature descriptors of the object of interest;

a matcher configured to match the extracted feature descriptors of the object of interest with target object feature descriptors of a target object stored in advance, and to determine one or more of the extracted feature descriptors of the object of interest corresponding to the target object; and

a feature point remover configured to remove extracted feature descriptors of the object of interest that do not meet a geometric comparison condition from among the extracted feature descriptors of the object of interest so as to define one or more final feature descriptors of the object of interest,

wherein the feature descriptor generator comprises;

an image pyramid generator configured to generate an image pyramid including differential images having a resolution which progressively changes from a resolution of the input image to a predetermined resolution different from the resolution of the input image;

a corner point detector configured to detect a corner feature from each differential image constituting the image pyramid, and determine a corner feature detected in common from the respective differential images of the image pyramid as a scale invariant feature; and

a descriptor generator configured to generate a feature descriptor for the scale invariant feature.

2. The apparatus according to claim 1, wherein:

the image pyramid generator is configured to generate the differential images by applying a Gaussian filter to the input image by a previously set number of times.

3. The apparatus according to claim 1, wherein:

when a previously set number of continuous pixels, among pixels located at a previously set distance from a center point that is a pixel selected from one of the differential images, have a pixel value which is greater or smaller than a previously set threshold value in comparison to the center point, the corner point detector determines the center point is the corner feature.

4. The apparatus according to claim 1, wherein:

the descriptor generator is configured to divide a pixel area having a previously set size around the scale invariant feature into a plurality of subareas, and generate the feature descriptor in a vector form corresponding to the scale invariant feature based on a histogram of a gradient value generated for each subarea.

5. The apparatus according to claim 4, wherein:

the descriptor generator is configured to generate the feature descriptor in a 36-dimensional vector form by dividing the pixel area into 9 subareas, and calculate a histogram for four gradient directions to correspond to the respective subareas.

6. The apparatus according to claim 1, further comprising:

a storage configured to store the final feature descriptors of the object of interest in a database.

7. An apparatus configured to track an object, the apparatus comprising:

a processor comprising;

a feature descriptor generator configured to generate feature descriptors indicating information of a plurality of features extracted from an input image from which an object of interest is to be detected and defining extracted feature descriptors of the object of interest;

a matcher configured to match the extracted feature descriptors of the object of interest with target object feature descriptors of a target object stored in advance, and to determine one or more of the extracted feature descriptors of the object of interest corresponding to the target object; and

a feature point remover configured to remove extracted feature descriptors of the object of interest that do not meet a geometric comparison condition from among the extracted feature descriptors of the object of interest so as to define one or more final feature descriptors of the object of interest, wherein;

the feature point remover is configured to establish the final feature descriptors of the object of interest using a geometric comparison condition in which a ratio of perspective transformation determinants calculated for the extracted feature descriptors of the object of interest is equal to a ratio of perspective transformation determinants calculated for the target object feature descriptors of the target object.

8. The apparatus according to claim 7, wherein:

the feature point remover is configured to establish the final feature descriptors of the object of interest by repeatedly performing, on all the extracted feature descriptors of the object of interest, a process of removing a previously set number of extracted feature descriptors that do not meet the geometric comparison condition from five extracted feature descriptors randomly selected from among the extracted feature descriptors of the object of interest, adding a new feature descriptor, and applying the geometric comparison condition.

9. An apparatus for removing a garbage feature, the apparatus comprising:

a processor comprising;

an input device configured to receive feature descriptors of an object of interest extracted from an input image and defining extracted feature descriptors of the object of interest through matching with target object feature descriptors of a target object stored in advance;

a feature processor configured to apply a geometric comparison condition to the extracted feature descriptors of the object of interest; and

a feature establisher configured to remove extracted feature descriptors of the object of interest that do not meet the geometric comparison condition from among the extracted feature descriptors of the object of interest so as to establish final feature descriptors of the object of interest,

wherein the feature descriptors are generated by;

generating an image pyramid including differential images having a resolution which progressively changes from a resolution of the input image to a predetermined resolution different from the resolution of the input image;

detecting a corner feature from each differential image constituting the image pyramid, and determining a corner feature detected in common from the respective differential images of the image pyramid as a scale invariant feature; and

generating a feature descriptor for the scale invariant feature.

10. The apparatus according to claim 9, wherein:

the feature processor is configured to repeatedly perform, on all the extracted feature descriptors of the object of interest, a process of applying the geometric comparison condition to five feature descriptors randomly selected from among the extracted feature descriptors of the object of interest and target object feature descriptors of the corresponding target object, adding a new feature descriptor to other extracted feature descriptors of the object of interest after the extracted feature descriptors of the object of interest that do not meet the geometric comparison condition are removed by the feature establisher, and applying the geometric comparison condition.

11. An apparatus for removing a garbage feature, the apparatus comprising:

a processor comprising;

an input device configured to receive feature descriptors of an object of interest extracted from an input image and defining extracted feature descriptors of the object of interest through matching with target object feature descriptors of a target object stored in advance;

a feature processor configured to apply a geometric comparison condition to the extracted feature descriptors of the object of interest; and

a feature establisher configured to remove extracted feature descriptors of the object of interest that do not meet the geometric comparison condition from among the extracted feature descriptors of the object of interest so as to establish final feature descriptors of the object of interest, wherein;

the feature processor is configured to use a geometric comparison condition in which a ratio of perspective transformation determinants calculated for the extracted feature descriptors of the object of interest is equal to a ratio of perspective transformation determinants calculated for the target object feature descriptors of the target object corresponding to the extracted feature descriptors of the object of interest.

12. A method of tracking an object, the method comprising:

(a) generating feature descriptors indicating information of a plurality of features extracted from an input image from which an object of interest is to be detected and defining extracted feature descriptors;

(b) matching the extracted feature descriptors of the object of interest with target object feature descriptors of a target object stored in advance, and determining one or more of the extracted feature descriptors of the object of interest corresponding to the target object; and

(c) removing extracted feature descriptors of the object of interest that do not meet a geometric comparison condition from among the extracted feature descriptors of the object of interest so as to establish final feature descriptors of the object of interest, wherein operation (a) includes;

(a1) generating an image pyramid including differential images having a resolution which progressively changes from a resolution of the input image to a predetermined resolution different from the resolution of the input image, and generating the feature descriptors based on the image pyramid;

(a2) detecting a corner feature from each differential image constituting the image pyramid, and determining a corner feature detected in common from the respective differential images of the image pyramid as a scale invariant feature; and

(a3) generating a feature descriptor for the scale invariant feature.

13. The method according to claim 12, wherein:

operation (a1) includes generating the differential images by applying a Gaussian filter to the input image by a previously set number of times.

14. The method according to claim 12, wherein:

when a previously set number of continuous pixels, among pixels located at a previously set distance from a center point that is a pixel selected from the differential image, have a pixel value which is greater or smaller than a previously set threshold value in comparison to the center point, operation (a2) includes determining the center point as the corner feature.

15. The method according to claim 12, wherein:

operation (a3) includes dividing a pixel area having a previously set size around the scale invariant feature into a plurality of subareas, and generating the feature descriptor in a vector form corresponding to the scale invariant feature based on a histogram of a gradient value generated for each subarea.

16. The method according to claim 15, wherein:

operation (a3) includes generating the feature descriptor in a 36-dimensional vector form by dividing the pixel area into 9 subareas, and calculating a histogram for four gradient directions to correspond to the respective subareas.

17. The method according to claim 12, further comprising:

(d) storing the final feature descriptors of the object of interest in a database.

18. A method of tracking an object, the method comprising:

(a) generating feature descriptors indicating information of a plurality of features extracted from an input image from which an object of interest is to be detected and defining extracted feature descriptors;

(b) matching the extracted feature descriptors of the object of interest with target object feature descriptors of a target object stored in advance, and determining one or more of the extracted feature descriptors of the object of interest corresponding to the target object; and

(c) removing extracted feature descriptors of the object of interest that do not meet a geometric comparison condition from among the extracted feature descriptors of the object of interest so as to establish final feature descriptors of the object of interest, wherein;

operation (c) includes establishing the final feature descriptors of the object of interest using a geometric comparison condition in which a ratio of perspective transformation determinants calculated for the extracted feature descriptors of the object of interest is equal to a ratio of perspective transformation determinants calculated for the target object feature descriptors of the target object.

19. The method according to claim 18, wherein:

operation (c) includes establishing the final feature descriptors of the object of interest by repeatedly performing, on all the extracted feature descriptors of the object of interest, a process of removing a previously set number of extracted feature descriptors of the object of interest that do not meet the geometric comparison condition from five extracted feature descriptors of the object of interest randomly selected from among the extracted feature descriptors of the object of interest, adding a new feature descriptor, and applying the geometric comparison condition.

20. A method of removing a garbage feature, the method comprising:

(a) receiving feature descriptors of an object of interest extracted from an input image and defining extracted feature descriptors through matching with target object feature descriptors of a target object stored in advance;

(b) applying a geometric comparison condition to the extracted feature descriptors of the object of interest; and

(c) removing extracted feature descriptors of the object of interest that do not meet the geometric comparison condition from among the extracted feature descriptors of the object of interest so as to establish final feature descriptors of the object of interest,

wherein the feature descriptors are generated by;

generating an image pyramid including differential images having a resolution which progressively changes from a resolution of the input image to a predetermined resolution different from the resolution of the input image;

detecting a corner feature from each differential image constituting the image pyramid, and determining a corner feature detected in common from the respective differential images of the image pyramid as a scale invariant feature; and

generating a feature descriptor for the scale invariant feature.

21. A method of removing a garbage feature, the method comprising:

(a) receiving feature descriptors of an object of interest extracted from an input image and defining extracted feature descriptors through matching with target object feature descriptors of a target object stored in advance;

(b) applying a geometric comparison condition to the extracted feature descriptors of the object of interest; and

(c) removing extracted feature descriptors of the object of interest that do not meet the geometric comparison condition from among the extracted feature descriptors of the object of interest so as to establish final feature descriptors of the object of interest, wherein;

operation (b) includes using a geometric comparison condition in which a ratio of perspective transformation determinants calculated for the extracted feature descriptors of the object of interest is equal to a ratio of perspective transformation determinants calculated for the target object feature descriptors of the target object corresponding to the extracted feature descriptors of the object of interest.

22. A method of removing a garbage feature, the method comprising:

(a) receiving feature descriptors of an object of interest extracted from an input image and defining extracted feature descriptors through matching with target object feature descriptors of a target object stored in advance;

(b) applying a geometric comparison condition to the extracted feature descriptors of the object of interest; and

(c) removing extracted feature descriptors of the object of interest that do not meet the geometric comparison condition from among the extracted feature descriptors of the object of interest so as to establish final feature descriptors of the object of interest, wherein;

operation (b) includes repeatedly performing, on all the extracted feature descriptors of the object of interest, a process of applying the geometric comparison condition to five extracted feature descriptors of the object of interest randomly selected from among the extracted feature descriptors of the object of interest and target object feature descriptors of the corresponding target object, adding a new feature descriptor to other extracted feature descriptors of the object of interest after the extracted feature descriptors of the object of interest that do not meet the geometric comparison condition are removed by the removing of the extracted feature descriptors of the object of interest, and applying the geometric comparison condition.

23. A non-transitory computer-readable medium having recorded thereon a program which causes a computer to execute a method of tracking an object, the method comprising:

(a) generating feature descriptors indicating information of a plurality of features extracted from an input image from which an object of interest is to be detected and defining extracted feature descriptors;

(b) matching the extracted feature descriptors of the object of interest with target object feature descriptors of a target object stored in advance, and determining one or more of the extracted feature descriptors of the object of interest corresponding to the target object; and

(c) removing extracted feature descriptors of the object of interest that do not meet a geometric comparison condition from among the extracted feature descriptors of the object of interest so as to establish final feature descriptors of the object of interest, wherein operation (a) includes;

(a1) generating an image pyramid including differential images having a resolution which progressively changes from a resolution of the input image to a predetermined resolution different from the resolution of the input image, and generating the feature descriptors based on the image pyramid;

(a2) detecting a corner feature from each differential image constituting the image pyramid, and determining the corner feature detected in common from the respective differential images of the image pyramid as a scale invariant feature; and

(a3) generating a feature descriptor for the scale invariant feature.

24. A non-transitory computer-readable medium having recorded thereon a program which causes a computer to execute a method of removing a garbage feature, the method comprising:

(a) receiving feature descriptors of an object of interest extracted from an input image and defining extracted feature descriptors through matching with target object feature descriptors of a target object stored in advance;

(b) applying a geometric comparison condition to the extracted feature descriptors of the object of interest; and

(c) removing extracted feature descriptors of the object of interest that do not meet the geometric comparison condition from among the extracted feature descriptors of the object of interest so as to establish final feature descriptors of the object of interest,

wherein the feature descriptors are generated by;

generating an image pyramid including differential images having a resolution which progressively changes from a resolution of the input image to a predetermined resolution different from the resolution of the input image;

detecting a corner feature from each differential image constituting the image pyramid, and determining a corner feature detected in common from the respective differential images of the image pyramid as a scale invariant feature; and

generating a feature descriptor for the scale invariant feature.