Image processing device, system, image processing method, and image processing program

• US 9,607,244 B2
• Filed: 02/19/2015
• Issued: 03/28/2017
• Est. Priority Date: 03/13/2014
• Status: Active Grant

• Current Assignee: Omron Corporation
• Original Assignee: Omron Corporation
• Inventors: Minato, Yoshihisa, Kato, Yutaka
• Primary Examiner(s): Vu, Kim
• Assistant Examiner(s): Bloom, Nathan
• Application Number:
  US14/625,894
• Publication Number:
  US 20150262415A1
• Time in Patent Office: 768 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: B07C 5/3422 using video scanning devices, e.g. TV-cameras
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/30164 Workpiece; Machine component
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 10/752 Contour matching
• CPC Class Codes: G06V 10/757 Matching configurations of points or features
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: G06V 2201/06 Recognition of objects for industrial automation

Abstract

Disclosed is an image processing device that includes: a storage unit that holds model information including information indicating a plurality of feature points of a detection object; a correspondence-relationship determination unit that extracts a plurality of feature points included in an input image as a two-dimensional image or a three-dimensional image and that determines a correspondence relationship between an extracted plurality of feature points and a plurality of feature points of the model information; and a position estimating unit that estimates, based on the correspondence relationship, one or a plurality of second positions of the input image corresponding to one or a plurality of first positions set relatively to the detection object. The one or plurality of first positions is included in a point, a line, or a region.

Claims

1. An image processing device comprising:

a processor; and

a memory including instructions that, when executed by the processor, cause the processor to perform operations including;

storing model information comprising information indicating a plurality of feature points of a detection object;

extracting a plurality of feature points comprised in an input image and configured to determine a correspondence relationship between an extracted plurality of feature points and a plurality of feature points of the model information;

estimating, based on the correspondence relationship, one or a plurality of second positions of the input image corresponding to one or a plurality of first positions set relatively to the detection object, the one or plurality of first positions being comprised in a point, a line, or a region; and

relating, as one to one, each feature point of the model information and each feature point extracted from the input image, and is also configured to determine, from mutually-related features, an interpolation function indicating a correspondence relationship between a set of a plurality of feature points of the model information and a set of a plurality of feature points extracted from the input image.

2. The image processing device according to claim 1, the operations further including extracting a plurality of feature points from a model image comprising the detection object as a photographic subject.

3. The image processing device according to claim 1, the operations further including determining a function that expresses a correspondence relationship by using a whole or a part of a set of feature points related as one to one.

4. The image processing device according to claim 3, the operations further including calculating coordinate values of the second position from coordinate values expressing a pre-set first position and a function expressing the correspondence relationship.

5. The image processing device according to claim 1, further comprising a display configured to display the estimated one or plurality of second positions in superposition with the input image.

6. The image processing device according to claim 1,

the operations further including estimating each of a plurality of second positions corresponding to a plurality of the first positions relatively set to the detection object, and

generating a command from the plurality of second positions to an external device.

7. A system comprising:

an image processor; and

a robot configured to operate following an output from the processor, wherein the image processor comprises;

a processor; and

a memory including instructions that, when executed by the processor, cause the processor to perform operations including;

storing model information comprising information indicating a plurality of feature points of a detection object;

extracting a plurality of feature points comprised in an input image and configured to determine a correspondence relationship between an extracted plurality of feature points and a plurality of feature points of the model information;

estimating, based on the correspondence relationship, one or a plurality of second positions of the input image corresponding to one or a plurality of first positions set relatively to the detection object;

generating a command to the robot based on the estimated second position, the one or plurality of first positions being comprised in a point, a line, or a region; and

relating, as one to one, each feature point of the model information and each feature point extracted from the input image, and is also configured to determine, from mutually-related features, an interpolation function indicating a correspondence relationship between a set of a plurality of feature points of the model information and a set of a plurality of feature points extracted from the input image.

8. The system according to claim 7,

the operations further including estimating each of a plurality of second positions corresponding to a plurality of the first positions relatively set to the detection object, and

generating the command from the plurality of second positions.

9. An image processing method comprising:

obtaining model information comprising information indicating a plurality of feature points of a detection object;

extracting a plurality of feature points comprised in an input image and determining a correspondence relationship between an extracted plurality of feature points and a plurality of feature points of the model information;

estimating, based on the correspondence relationship, one or a plurality of second positions of the input image corresponding to one or a plurality of first positions set relatively to the detection object, the one or plurality of first positions being comprised in a point, a line, or a region;

relating, as one to one, each feature point of the model information and each feature point extracted from the input image, and is also configured to determine, from mutually-related features, an interpolation function indicating a correspondence relationship between a set of a plurality of feature points of the model information and a set of a plurality of feature points extracted from the input image.

10. A non-transitory computer readable storage medium that stores a computer program for image processing, the computer program, when executed by a processor, causing the processor to perform operations comprising:

obtaining model information comprising information indicating a plurality of feature points of a detection object;

extracting a plurality of feature points comprised in an input image and determining a correspondence relationship between an extracted plurality of feature points and a plurality of feature points of the model information;

estimating, based on the correspondence relationship, one or a plurality of second positions of the input image corresponding to one or a plurality of first positions set relatively to the detection object, the one or plurality of first positions being comprised in a point, a line, or a region; and

relating, as one to one, each feature point of the model information and each feature point extracted from the input image, and is also configured to determine, from mutually-related features, an interpolation function indicating a correspondence relationship between a set of a plurality of feature points of the model information and a set of a plurality of feature points extracted from the input image.