IMAGE PROCESSING DEVICE AND METHOD FOR DETECTING IMAGE OF OBJECT TO BE DETECTED FROM INPUT DATA

• US 20170132451A1
• Filed: 11/04/2016
• Published: 05/11/2017
• Est. Priority Date: 11/06/2015
• Status: Active Grant

• Current Assignee: Fanuc Corporation
• Original Assignee: Fanuc Corporation
• Inventors: WARASHINA, Fumikazu, NAMIKI, Yuta
• Granted Patent: US 11,227,144 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/28 Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
• CPC Class Codes: G06F 18/41 Interactive pattern learning with a human teacher
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06T 7/77 using statistical methods
• CPC Class Codes: G06V 10/757 Matching configurations of points or features
• CPC Class Codes: G06V 10/7788 the supervisor being a human, e.g. interactive learning with a human teacher
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects

Abstract

An image processing device that detects an image of an object from input data captured by a vision sensor, on the basis of a model pattern including a set of a plurality of first feature points representing the shape of the object, includes an object detection unit that detects images of the object by matching between a plurality of second feature points extracted from the input data and a plurality of first feature points forming the model pattern, a corresponding point selection unit that selects, for each image of the object, second feature points corresponding to the first feature points and stores the selected second feature points as corresponding points, and a model pattern correction unit that calculates a statistic of a predetermined physical quantity of the plurality of corresponding points associated with each first feature point and corrects the first feature point on the basis of the statistic.

Claims

1. An image processing device that detects an image of an object from input data captured by a vision sensor, on the basis of a model pattern comprising a set of a plurality of first feature points representing a shape of the object, the device comprising:

an object detection unit that detects, for each of at least one input data obtained by capturing the object, at least one image of the object by matching between a plurality of second feature points extracted from the input data and a plurality of first feature points forming the model pattern;

a corresponding point selection unit that selects, for each of the at least one image of the object detected from the at least one input data, second feature points corresponding to the plurality of first feature points forming the model pattern from the plurality of second feature points forming the image and stores as corresponding points, the selected second feature points in association with the first feature points; and

a model pattern correction unit that calculates, for each of the plurality of first feature points forming the model pattern, a statistic of a predetermined physical quantity of the plurality of corresponding points associated with the first feature point and corrects the first feature point on the basis of the calculated statistic of the predetermined physical quantity.

2. The image processing device according to claim 1, wherein the corresponding point selection unit

obtains, for each of the plurality of first feature points forming the model pattern, a determination index for determining correspondence/non-correspondence in selection of the corresponding points on the basis of the statistic of the predetermined physical quantity of the plurality of corresponding points associated with the first feature point and stores the determination index in association with the first feature point, and

determines, for each of the plurality of first feature points forming the model pattern, whether the first feature point and the second feature point match each other on the basis of the predetermined physical quantity of the second feature point extracted from the one input data, and the determination index associated with the first feature point.

3. The image processing device according to claim 1, wherein

the corresponding point selection unit

stores, for each of the at least one detected image of the object, as a feature point candidate, at least one second feature point which corresponds to none of the first feature points forming the model pattern of the plurality of second feature points forming the image, and

selects a second feature point corresponding to the at least one stored feature point candidate and stores as a corresponding point, the selected second feature point in association with the feature point candidate, and

the model pattern correction unit further calculates, for each of the at least one stored feature point candidate, a statistic of a predetermined physical quantity of a plurality of corresponding points associated with the feature point candidate and adds the feature point candidate to the model pattern as a first feature point on the basis of the statistic of the predetermined physical quantity.

4. An image processing device that detects an image of an object from input data using a model pattern comprising a set of a plurality of first feature points representing a shape of the object, the device comprising:

an object detection unit that detects, for each of at least one input data obtained by capturing the object, at least one image of the object by matching between a plurality of second feature points extracted from the input data and a plurality of first feature points forming the model pattern; and

a corresponding point selection unit that selects, for each of the at least one image of the object detected from the at least one input data, second feature points corresponding to the plurality of first feature points forming the model pattern from the plurality of second feature points forming the image and stores as corresponding points, the selected second feature points in association with the first feature points, calculates a predetermined statistic of a predetermined physical quantity of the corresponding points for each of the plurality of first feature points forming the model pattern, and obtains a determination index for determining correspondence/non-correspondence in the matching on the basis of the calculated predetermined statistic and stores the determination index in association with the first feature point,

wherein the object detection unit further determines, for each of the plurality of first feature points forming the model pattern, whether the first feature point and the second feature point match each other on the basis of the predetermined physical quantity of the second feature point extracted from the one input data, and the determination index associated with the first feature point.

5. The image processing device according to claim 2, wherein the stored determination index comprises the calculated predetermined statistic.

6. The image processing device according to claim 4, wherein the stored determination index comprises the calculated predetermined statistic.

7. The image processing device according to claim 1, wherein the vision sensor comprises a camera and the input data comprises one of a grayscale image and a color image.

8. The image processing device according to claim 4, wherein the vision sensor comprises a camera and the input data comprises one of a grayscale image and a color image.

9. The image processing device according to claim 1, wherein the vision sensor comprises a three-dimensional sensor and the input data comprises one of a range image and a set of three-dimensional points serving as a point cloud.

10. The image processing device according to claim 4, wherein the vision sensor comprises a three-dimensional sensor and the input data comprises one of a range image and a set of three-dimensional points serving as a point cloud.

11. The image processing device according to claim 1, wherein the corresponding point selection unit excludes an inappropriate image of the object from the at least one image of the object and selects a corresponding point from only an appropriate image of the object.

12. The image processing device according to claim 1, wherein the model pattern correction unit excludes an inappropriate corresponding point from the plurality of corresponding points associated with the first feature point and calculates a statistic of the predetermined physical quantity using only an appropriate corresponding point.

13. An image processing method for detecting an image of an object from input data captured by a vision sensor, on the basis of a model pattern comprising a set of a plurality of first feature points representing a shape of the object, the method comprising:

a first step of detecting, for each of at least one input data obtained by capturing the object, at least one image of the object by matching between a plurality of second feature points extracted from the input data and a plurality of first feature points forming the model pattern;

a second step of selecting, for each of the at least one image of the object detected from the at least one input data, second feature points corresponding to the plurality of first feature points forming the model pattern from the plurality of second feature points forming the image and storing as corresponding points, the selected second feature points in association with the first feature points; and

a third step of calculating, for each of the plurality of first feature points forming the model pattern, a statistic of a predetermined physical quantity of the plurality of corresponding points associated with the first feature point and correcting the first feature point on the basis of the calculated statistic of the predetermined physical quantity.

14. The image processing method according to claim 13, further comprising, between the second step and the third step:

a step of excluding, for each of the plurality of first feature points forming the model pattern, an inappropriate corresponding point from the plurality of corresponding points associated with the first feature point.

15. The image processing method according to claim 13, further comprising, between the first step and the second step:

a step of excluding an inappropriate image of the object from the at least one detected image of the object.