Information processing apparatus, method of controlling information processing apparatus, and storage medium

• US 10,252,417 B2
• Filed: 02/27/2017
• Issued: 04/09/2019
• Est. Priority Date: 03/02/2016
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Watanabe, Daisuke
• Primary Examiner(s): Desire, Gregory M
• Application Number:
  US15/443,429
• Publication Number:
  US 20170256046A1
• Time in Patent Office: 771 Days
• CPC Class Codes: B25J 9/1612 characterised by the hand, wrist, grip control
• CPC Class Codes: B25J 9/1687 Assembly, peg and hole, palletising, straight line, weaving pattern movement
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 2207/30164 Workpiece; Machine component
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 10/42 Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
• CPC Class Codes: G06V 10/467 Encoded features or binary features, e.g. local binary patterns [LBP]
• CPC Class Codes: G06V 10/761 Proximity, similarity or dissimilarity measures

Abstract

An information processing apparatus, comprises an obtainment unit configured to obtain a relative position and orientation between a first object and a second object; an specifying unit configured to specify an occlusion region where the second object occludes the first object, from an image that includes the first object and the second object, based on the relative position and orientation, a first shape model that represents a shape of the first object, and a second shape model that represents a shape of the second object; and a generation unit configured to generate information for obtaining a position and orientation of the first object based on the occlusion region.

Claims

1. An information processing apparatus, comprising:

one or more processors; and

a memory coupled to the one or more processors, the memory having stored thereon instructions which, when executed by the one or more processors, cause the information processing apparatus to;

obtain a relative position and orientation between a first object and a second object;

specify an occlusion region where the second object occludes the first object, from an image that includes the first object and the second object, based on the relative position and orientation, a first shape model that represents a shape of the first object, and a second shape model that represents a shape of the second object;

select, from the first shape model, model features other than model features that correspond to the specified occlusion region in the image;

calculate the position and orientation of the first object by associating the selected model features with image features in the image; and

determine an assembly state based on the position and orientation of the first object.

2. The information processing apparatus according to claim 1, wherein the instructions, when executed by the one or more processors, further cause the information processing apparatus to obtain each of the first shape model and the second shape model.

3. The information processing apparatus according to claim 1, wherein in selecting the model features, the instructions, when executed by the one or more processors, cause the information processing apparatus to:

calculate, with respect to the occlusion region, an occlusion degree that represents a degree at which the second object occludes the first object,

set a level of contribution when obtaining the position and orientation of the first object, based on the occlusion degree, and

select the model features using the level of contribution.

4. The information processing apparatus according to claim 1, wherein in specifying the occlusion region, the instructions, when executed by the one or more processors, cause the information processing apparatus to specify the occlusion region based on a first relative position and orientation between the first object and the second object that is stored in advance, and an approximate position and orientation of the first object.

5. The information processing apparatus according to claim 4, wherein in specifying the occlusion region, the instructions, when executed by the one or more processors, cause the information processing apparatus to specify the occlusion region based on a result of rendering model information of the first object based on the approximate position and orientation of the first object, and a result of rendering the second object based on an approximate position and orientation of the second object that is obtained based on the first relative position and orientation.

6. The information processing apparatus according to claim 1, wherein in specifying the occlusion region, the instructions, when executed by the one or more processors, cause the information processing apparatus to specify the occlusion region by generating an image indicating a region in the image occupied by the second object, based on an approximate position and orientation of the second object and the second shape model of the second object.

7. The information processing apparatus according to claim 1, wherein the instructions, when executed by the one or more processors, further cause the information processing apparatus to obtain a position and orientation of the second object based on the occlusion region and model information of the second object.

8. The information processing apparatus according to claim 7,

wherein the instructions, when executed by the one or more processors, further cause the information processing apparatus to estimate a first relative position and orientation between the first object and the second object, based on the position and orientation of the first object and the position and orientation of the second object, and

wherein in determining the assembly state, the instructions, when executed by the one or more processors, cause the information processing apparatus to determine the assembly state based further on the estimated first relative position and orientation.

9. The information processing apparatus according to claim 8, wherein in determining the assembly state, the instructions, when executed by the one or more processors, cause the information processing apparatus to determine the assembly state based further on a difference between the first relative position and orientation and a second relative position and orientation between the first object and the second object that is stored in advance.

10. An information processing apparatus comprising:

one or more processors; and

a memory coupled to the one or more processors, the memory having stored thereon instructions which, when executed by the one or more processors, cause the information processing apparatus to;

obtain a relative position and orientation between a first object and a second object;

specify an occlusion region where the second object occludes the first object, from an image that includes the first object and the second object, based on the relative position and orientation, a first shape model that represents a shape of the first object, and a second shape model that represents a shape of the second object;

select image features other than image features included in the specified occlusion region in the image;

calculate the position and orientation of the first object by associating the selected image features with model features of the first shape model; and

determine an assembly state based on the position and orientation of the first object.

11. The information processing apparatus according to claim 10, wherein the image features in the image is a geometric feature of the image.

12. A method of controlling an information processing apparatus, comprising:

obtaining a relative position and orientation between a first object and a second object;

specifying an occlusion region where the second object occludes the first object, from an image that includes the first object and the second object, based on the relative position and orientation, a first shape model that represents a shape of the first object, and a second shape model that represents a shape of the second object;

selecting, from the first shape model, model features other than model features that correspond to the specified occlusion region in the image;

calculating the position and orientation of the first object by associating the selected model features with image features in the image; and

determining an assembly state based on the position and orientation of the first object.

13. A non-transitory computer-readable storage medium storing a computer program for causing a computer to execute a method comprising:

obtaining a relative position and orientation between a first object and a second object;

specifying an occlusion region where the second object occludes the first object, from an image that includes the first object and the second object, based on the relative position and orientation, a first shape model that represents a shape of the first object, and a second shape model that represents a shape of the second object;

selecting, from the first shape model, model features other than model features that correspond to the specified occlusion region in the image;

calculating the position and orientation of the first object by associating the selected model features with image features in the image; and

determining an assembly state based on the position and orientation of the first object.

14. A method of controlling an information processing apparatus, comprising:

obtaining a relative position and orientation between a first object and a second object;

specifying an occlusion region where the second object occludes the first object, from an image that includes the first object and the second object, based on the relative position and orientation, a first shape model that represents a shape of the first object, and a second shape model that represents a shape of the second object;

selecting image features other than image features included in the specified occlusion region in the image;

calculating the position and orientation of the first object by associating the selected image features with model features of the first shape model; and

determining an assembly state based on the position and orientation of the first object.

15. A non-transitory computer-readable storage medium storing a computer program for causing a computer to execute a method comprising:

obtaining a relative position and orientation between a first object and a second object;

specifying an occlusion region where the second object occludes the first object, from an image that includes the first object and the second object, based on the relative position and orientation, a first shape model that represents a shape of the first object, and a second shape model that represents a shape of the second object;

selecting image features other than image features included in the specified occlusion region in the image;

calculating the position and orientation of the first object by associating the selected image features with model features of the first shape model; and

determining an assembly state based on the position and orientation of the first object.