Object Recognition Device, Object Recognition Method, Object Recognition Program, Feature Registration Device, Feature Registration Method, and Feature Registration Program

• US 20070269107A1
• Filed: 03/29/2007
• Published: 11/22/2007
• Est. Priority Date: 03/31/2006
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Yoshigahara, Takayuki, Wu, Weiguo, Nakamura, Akira, IWAI, Yoshiaki
• Granted Patent: US 8,009,879 B2
• Time in Patent Office: Days
• US Class Current: 382/168
• CPC Class Codes: G06V 10/421 by analysing segments intersecting the pattern
• CPC Class Codes: G06V 10/757 Matching configurations of points or features

Abstract

Disclosed herein is an object recognition device for recognition of an object that exists in an image, the device including: an edge image creator configured to create an edge image from a target image; a local feature extractor configured to extract a feature of each edge point on an edge image, and determine a base point and a support point from edge points in a local area defined for the edge image; a feature matching unit configured to implement, with reference to a memory, feature matching of a base point and a support point to thereby acquire a matching pair; and a matching pair determiner configured to determine final matching pairs by employing matching pairs having a highest matching cost among matching pairs obtained by the feature matching unit, wherein an object in a target image is identified based on matching pairs determined by the matching pair determiner.

Claims

1. An object recognition device for recognition of an object that exists in an image, the device comprising:

an edge image creator configured to create an edge image from a target image including an object as a target of object recognition;

a local feature extractor configured to extract a feature of each edge point on an edge image created by the edge image creator, and determine a base point and a support point from edge points in a local area defined for the edge image, the base point being a point to which attention should be paid in the local area, the support point being a point supporting the base point;

a feature matching unit configured to implement, with reference to a memory, feature matching of a base point and a support point that each have a feature extracted by the local feature extractor to thereby acquire a matching pair, the memory storing therein a base point, a feature of the base point, a support point that supports the base point, a feature of the support point, and a geometric positional relationship between the base point and the support point, the base point being a point to which attention should be paid in a vicinity of a local area defined for an edge image created in advance depending on the object, the base point being determined from edge points in the vicinity of the local area; and

a matching pair determiner configured to determine final matching pairs by employing matching pairs having a highest matching cost among matching pairs obtained by the feature matching unit, wherein

an object in a target image is identified based on matching pairs determined by the matching pair determiner.

2. The object recognition device according to claim 1, wherein

the local feature extractor divides a circle having a fixed radius along a circumferential direction and a radius direction, and translates into a histogram the number of edge points that exist in each of bins arising from the division to thereby extract a local feature of each of all edge points.

3. The object recognition device according to claim 1, wherein

the local feature extractor extracts a reference axis direction and implements rotation normalization for realization of rotation invariance of an extracted feature.

4. The object recognition device according to claim 3, wherein

the local feature extractor creates a histogram by voting an angle of a direction from a point of interest to a peripheral edge point in a small area centered at the point of interest, and extracts a direction having an angle corresponding to a maximum value in the histogram as the reference axis direction.

5. The object recognition device according to claim 4, wherein

the local feature extractor obtains an angle of the reference axis direction for each point and shifts the histogram by the obtained angle in a circumferential direction, to thereby implement rotation normalization of the feature.

6. The object recognition device according to claim 1, wherein

the local feature extractor determines a reference circle having any optional radius and determines the base point and a plurality of the support points from the edge points in a vicinity of the reference circle.

7. The object recognition device according to claim 1, wherein

the local feature extractor determines a reference circle having a proper radius for each of a plurality of multiresolution images that are created by an multiresolution image creator and have different resolutions, and determines the base point and a plurality of the support points from the edge points in a vicinity of the reference circle.

8. The object recognition device according to claim 1, wherein

the feature matching unit corrects a position of a support point of a model in accordance with a reference axis of a point on a target, and calculates a feature cost between a base point of the model and the point on the target after aligning the base point of the model with the position of the point on the target, and the feature matching unit searches for a point offering a lowest feature cost in a search region for a support point.

9. The object recognition device according to claim 1, further comprising

a mismatching pair remover configured to remove a mismatching pair from matching pairs obtained by the feature matching unit.

10. An object recognition method for recognition of an object that exists in an image, the method comprising the steps of:

(a) creating an edge image from a target image including an object as a target of object recognition;

(b) extracting a feature of each edge point on an edge image created in the step (a), and determining a base point and a support point from edge points in a local area defined for the edge image, the base point being a point to which attention should be paid in the local area, the support point being a point supporting the base point;

(c) implementing, with reference to a memory, feature matching of a base point and a support point that each have a feature extracted in the step (b) to thereby acquire a matching pair, the memory storing therein a base point, a feature of the base point, a support point that supports the base point, a feature of the support point, and a geometric positional relationship between the base point and the support point, the base point being a point to which attention should be paid in a vicinity of a local area defined for an edge image created in advance depending on the object, the base point being determined from edge points in the vicinity of the local area; and

(d) determining final matching pairs by employing matching pairs having a highest matching cost among matching pairs obtained in the step (c), wherein

an object in a target image is identified based on matching pairs determined in the step (d).

11. An object recognition program executed by a computer that is used as an object recognition device for recognition of an object that exists in an image, the program comprising the steps of:

(a) creating an edge image from a target image including an object as a target of object recognition;

(b) extracting a feature of each edge point on an edge image created in the step (a), and determining a base point and a support point from edge points in a local area defined for the edge image, the base point being a point to which attention should be paid in the local area, the support point being a point supporting the base point;

(c) implementing, with reference to a memory, feature matching of a base point and a support point that each have a feature extracted in the step (b) to thereby acquire a matching pair, the memory storing therein a base point, a feature of the base point, a support point that supports the base point, a feature of the support point, and a geometric positional relationship between the base point and the support point, the base point being a point to which attention should be paid in a vicinity of a local area defined for an edge image created in advance depending on the object, the base point being determined from edge points in the vicinity of the local area; and

(d) determining final matching pairs by employing matching pairs having a highest matching cost among matching pairs obtained in the step (c), wherein

an object in a target image is identified based on matching pairs determined in the step (d).

12. A feature registration device that registers a feature used in object recognition processing for recognition of an object that exists in an image, the device comprising:

an edge image creator configured to create an edge image of a model of an object in advance; and

a model feature extractor configured to extract a feature of each edge point on an edge image created by the edge image creator, and determine a base point and a support point from edge points in a local area defined for the edge image, the model feature extractor storing in a memory the base point, a feature of the base point, the support point, a feature of the support point, and a geometric positional relationship between the base point and the support point, the base point being a point to which attention should be paid in the local area, the support point being a point supporting the base point.

13. The feature registration device according to claim 12, wherein

the model feature extractor determines a reference circle having any optional radius and determines a base point and a plurality of support points from the edge points in a vicinity of the reference circle, and the model feature extractor calculates a relative positional relationship between each support point and the base point.

14. The feature registration device according to claim 13, wherein

the model feature extractor calculates a relative distance and a relative angle of each support point with respect to the base point as the relative positional relationship.

15. The feature registration device according to claim 13, wherein

the model feature extractor determines a center pixel and a radius for determination of a reference circle by evaluating a degree of distribution of edge points in a vicinity of the whole of the reference circle.

16. The feature registration device according to claim 13, wherein

the model feature extractor evaluates the degree of distribution of the edge points based on an entropy value calculated by using a reference circle radius, a distance from a reference circle center to each edge point, and an angle formed by a fixed axis and a segment between the reference circle center and each edge point.

17. The feature registration device according to claim 13, wherein

the model feature extractor divides the reference circle into a plurality of regions along a circumferential direction and defines a sampling area in each region to thereby define a point closest to the reference circle in each sampling area as a sampling point, and the model feature extractor refers to a history of positions of already-determined base points to thereby select from a sampling point group a point other than a point near a pixel corresponding to the already-defined base point so that the selected base point is defined as a newly-determined base point and other sampling points are defined as support points.

18. The feature registration device according to claim 13, wherein

the model feature extractor determines the reference circle by using a feature of each edge point on an edge image created by the edge image creator.

19. The feature registration device according to claim 18, wherein

the model feature extractor extracts an area that offers a large distance between local features of edge points on the area to thereby determine the reference circle.

20. The feature registration device according to claim 19, wherein

the model feature extractor divides a predetermined reference circle depending on a radius of a local feature, and extracts a local feature for each region arising from the division, and the model feature extractor calculates a distance between features to create a map of a distance between features, and determines the reference circle by using the map.

21. The feature registration device according to claim 20, wherein

the model feature extractor uses an entropy value of a distance between features as an evaluation criterion.

22. The feature registration device according to claim 12, wherein

the model feature extractor determines a reference circle having a proper radius for each of a plurality of multiresolution images that are created by an multiresolution image creator and have different resolutions, and determines the base point and a plurality of the support points from the edge points in a vicinity of the reference circle.

23. A feature registration method for registration of a feature used in object recognition processing for recognition of an object that exists in an image, the method comprising the steps of:

(a) creating an edge image of a model of an object in advance; and

(b) extracting a feature of each edge point on an edge image created in the step (a), determining a base point and a support point from edge points in a local area defined for the edge image, and storing in a memory the base point, a feature of the base point, the support point, a feature of the support point, and a geometric positional relationship between the base point and the support point, the base point being a point to which attention should be paid in the local area, the support point being a point supporting the base point.

24. A feature registration program executed by a computer as a device that registers a feature used in object recognition processing for recognition of an object that exists in an image, the program comprising the steps of:

(a) creating an edge image of a model of an object in advance; and

(b) extracting a feature of each edge point on an edge image created in the step (a), determining a base point and a support point from edge points in a local area defined for the edge image, and storing in a memory the base point, a feature of the base point, the support point, a feature of the support point, and a geometric positional relationship between the base point and the support point, the base point being a point to which attention should be paid in the local area, the support point being a point supporting the base point.