Image processing method, image processing device, and robot system

• US 10,207,409 B2
• Filed: 10/14/2015
• Issued: 02/19/2019
• Est. Priority Date: 10/16/2014
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Odagiri, Jun
• Primary Examiner(s): Kassa, Yosef
• Application Number:
  US14/883,033
• Publication Number:
  US 20160110840A1
• Time in Patent Office: 1,224 Days
• Field of Search: 382153, 382209, 382266, 382279, 382282, 382291, 382307, 700245, 700248, 700261
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 2219/39391 Visual servoing, track end effector with camera image feedback
• CPC Class Codes: G05B 2219/39393 Camera detects projected image, compare with reference image, position end effector
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 10/46 Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features colour feature extraction G06V10/56
• CPC Class Codes: G06V 10/752 Contour matching
• CPC Class Codes: Y10S 901/02 Arm motion controller
• CPC Class Codes: Y10S 901/28 Joint
• CPC Class Codes: Y10S 901/47 Optical

Abstract

An image processing method can suppress detection accuracy of a detection target object from being lowered even if the detection target object has a different surface condition because of the influence of various kinds of noise. The image processing method includes the following operations of generating a captured model edge image by executing edge extraction processing on a captured model image acquired by capturing a detection target object, executing pattern matching of the captured model edge image and a model edge image, calculating similarity at respective edge points in the model edge image in the pattern matching of the captured model edge image and the model edge image, selecting an edge point to be eliminated based on the similarity from among the respective edge points in the model edge image, and generating an edge image acquired by eliminating the selected edge point as a final model edge image.

Claims

1. An image processing method comprising:

generating a plurality of captured model edge images by executing edge extraction processing on a plurality of captured model images acquired by capturing a target object;

executing pattern matching of the captured model edge images and a model edge image related to the detection target object;

calculating similarity at respective edge points in the model edge image;

calculating a variance and an average of the similarity at the edge points in the model edge image in relation to the captured model edge images respectively;

eliminating an edge point from the model edge image based on at least one of the average and the variance not to use the edge point for the pattern matching.

2. The image processing method according to claim 1,

wherein, in the eliminating, an edge point for which the variance is not less than a predetermined value or for which the average is not greater than a predetermined value is eliminated from the model edge image.

3. The image processing method according to claim 1,

wherein, in the eliminating, among all edge points, some edge points that are included in a predetermined percentage in terms of the variance in a descending order or are included in a predetermined percentage in terms of the average in an ascending order are eliminated from the model edge image.

4. A method for generating an edge image that includes a plurality of edge points, comprising:

acquiring a plurality of images by capturing a target object;

extracting edges from the plurality of images;

calculating a variance of similarity between corresponding edge points of the edges extracted from the plurality of images; and

generating an edge image while leaving edge points for which the variance satisfies a predetermined condition and while eliminating edge points for which the variance does not satisfy the predetermined condition.

5. The method according to claim 4,

wherein edge points that are included in a predetermined percentage in terms of the variance in an ascending order are eliminated.

6. The method according to claim 4, wherein after the edge image is generated, patter matching is executed using the generated edge image.

7. A non-transitory computer readable storage medium storing a program for causing a computer to implement the method comprising:

acquiring a plurality of images by capturing a target object;

extracting edges from the plurality of images;

calculating a variance of similarity between corresponding edge points of the edges extracted from the plurality of images; and

generating an edge image while leaving edge points for which the variance satisfies a predetermined condition and while eliminating edge points for which the variance does not satisfy the predetermined condition.

8. The non-transitory computer readable storage medium according to claim 7, wherein after the edge image is generated, patter matching is executed using the generated edge image.

9. A system, comprising:

an image processing apparatus configured to execute

acquiring a plurality of images by capturing a target object,

extracting edges from the plurality of images,

calculating a variance of similarity between corresponding edge points of the edges extracted from the plurality of images, and

generating an edge image for pattern matching while leaving edge points for which the variance satisfies a predetermined condition and while eliminating edge points for which the variance does not satisfy the predetermined condition; and

a storage configured to store the generated edge image for the pattern matching.

10. The system according to claim 9, further comprising:

a camera,

wherein the plurality of images is acquired by means of the camera.

11. The system according to claim 10, wherein the image processing apparatus executes the pattern matching using the image acquired by means of the camera and using the edge image for the pattern matching.

12. The system according to claim 11, further comprising:

a robot body configured to be able to perform work on the target object; and

a controller configured to control the robot body,

wherein the controller controls the robot body on a basis of a result of the pattern matching.

13. The system according to claim 9, wherein after the edge image is generated, patter matching is executed using the generated edge image.

14. A method for controlling a system, comprising:

acquiring a plurality of images by capturing a target object;

extracting edges from the plurality of images;

calculating a variance of similarity between corresponding edge points of the edges extracted from the plurality of images;

generating an edge image for pattern matching while leaving edge points for which the variance satisfies a predetermined condition and while eliminating edge points for which the variance does not satisfy the predetermined condition; and

storing the generated edge image for the pattern matching.

15. The method according to claim 14, wherein edge points that are included in a predetermined percentage in terms of the variance in an ascending order are eliminated.

16. The method according to claim 14, further comprising:

detecting a position or an orientation of the target object, or both the position and the orientation, using the edge image for the pattern matching.

17. The method according to claim 14, wherein after the edge image is generated, patter matching is executed using the generated edge image.