Information processing apparatus, information processing method, and program

• US 10,325,336 B2
• Filed: 11/09/2015
• Issued: 06/18/2019
• Est. Priority Date: 11/28/2014
• Status: Active Grant

• Current Assignee: Canon Ayutthaya Limited (Canon Inc.)
• Original Assignee: Canon Ayutthaya Limited (Canon Inc.)
• Inventors: Kotake, Daisuke, Fujiki, Masakazu, Tateno, Keisuke
• Primary Examiner(s): Bayat, Ali
• Application Number:
  US15/529,438
• Publication Number:
  US 20170358048A1
• Time in Patent Office: 1,317 Days
• CPC Class Codes: G06T 1/0007 Image acquisition
• CPC Class Codes: G06T 2207/30164 Workpiece; Machine component
• CPC Class Codes: G06T 2219/2004 Aligning objects, relative positioning of parts
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06T 7/75 involving models

Abstract

An information processing apparatus for determining a placement of an image capturing apparatus capable of accurately measuring the position and orientation of an object through model fitting includes a holding unit that holds a 3D model representing a 3D shape of the object, a generating unit that generates candidates of the position and orientation of the image capturing apparatus relative to the object, an evaluating unit that evaluates the candidates of the position and orientation of the image capturing apparatus relative to the object based on a result of matching between data that virtually generates the object observed when the image capturing apparatus is placed in the position and orientation generated by the generating unit and the 3D model, and a determining unit that determines the position and orientation of the image capturing apparatus relative to the object based on the result of evaluation performed by the evaluating unit.

Claims

1. An information processing apparatus for determining a position and orientation of an image capturing apparatus that captures an image of an object, comprising:

one or more processors; and

one or more memory devices, the one or more memory devices storing a 3D shape model representing a 3D shape of the object, the one or more memory devices storing executable instructions, which when executed by the one or more processors, cause the information processing apparatus to perform operations comprising;

generating candidates of the position and orientation of the image capturing apparatus relative to the object;

evaluating the candidates of the position and orientation of the image capturing apparatus relative to the object on the basis of a result of matching between virtually generated data of the object with the image capturing apparatus placed in the position and orientation generated by the generating and the 3D shape model;

deriving the position and orientation of the object from the result of matching,

wherein the evaluating evaluates the candidates of the position and orientation of the image capturing apparatus relative to the object by evaluating the derived position and orientation,

wherein the deriving derives a plurality of positions and orientations of the object by generating, in the vicinity of the position and orientation generated by the generating, a plurality of positions and orientations that differ from the position and orientation and matching a feature in the virtually generated data of the object observed from the originally generated position and orientation with a feature of the 3D shape model placed at each position and orientation generated in the vicinity of the position and orientation; and

determining the position and orientation of the image capturing apparatus relative to the object on the basis of the result of evaluation performed by the evaluating.

2. The information processing apparatus according to claim 1, wherein the deriving derives the position and orientation of the object by matching a feature in the virtually generated data of the object with the image capturing apparatus placed in the position and orientation generated by the generating with a feature in the 3D shape model.

3. The information processing apparatus according to claim 1, wherein the evaluating includes calculating a probability of the derived position and orientation being obtained and evaluating the derived position and orientation on the basis of the probability.

4. The information processing apparatus according to claim 3, wherein the information processing apparatus calculates the number of the position and orientation values having the probability that is lower than a threshold value for each of the plurality of position and orientation values and determines the position and orientation of the image capturing apparatus relative to the object on the basis of the number.

5. The information processing apparatus according to claim 1, wherein the evaluating evaluates the candidates of the position and orientation of the image capturing apparatus relative to the object on the basis of an error function obtained from the result of matching.

6. The information processing apparatus according to claim 1, wherein the generating generates a plurality of candidates of the position and orientation, the evaluating evaluates a position and orientation derived from each of the position and orientation, and the determining determines the position and orientation of the image capturing apparatus relative to the object by selecting one of the position and orientation from among the candidates of the position and orientation on the basis of the result of evaluation performed by the evaluating.

7. The information processing apparatus according to claim 1, wherein the generating generates a plurality of candidates of the position and orientation, the evaluating evaluates the position and orientation derived from each of the candidates of the position and orientation, and the determining determines the position and orientation of the image capturing apparatus relative to the object by generating a new position and orientation on the basis of the candidates of the position and orientation and the result of evaluation performed by the evaluating.

8. The information processing apparatus according to claim 1, wherein the image capturing apparatus is a camera that captures a grayscale image.

9. The information processing apparatus according to claim 1, wherein the image capturing apparatus is a sensor that acquires one of a range image or a 3D point cloud.

10. The information processing apparatus according to claim 1, the operations further comprising:

presenting the determined position and orientation to a user.

11. An information processing method for determining a position and orientation of an image capturing apparatus that captures an image of an object, comprising:

generating candidates of the position and orientation of the image capturing apparatus relative to the object;

evaluating the candidates of the position and orientation of the image capturing apparatus relative to the object on the basis of a result of matching between virtually generated data of the object with the image capturing apparatus placed in the generated position and orientation and a 3D shape model representing a 3D shape of the object;

deriving the position and orientation of the object from the result of matching,

wherein the evaluating evaluates the candidates of the position and orientation of the image capturing apparatus relative to the object by evaluating the derived position and orientation,

wherein the deriving derives a plurality of positions and orientations of the object by generating, in the vicinity of the position and orientation generated by the generating, a plurality of positions and orientations that differ from the position and orientation and matching a feature in the virtually generated data of the object observed from the originally generated position and orientation with a feature of the 3D shape model placed at each position and orientation generated in the vicinity of the position and orientation; and

determining the position and orientation of the image capturing apparatus relative to the object on the basis of the obtained result of evaluation.

12. A non-transitory computer-readable storage medium storing therein a program for determining a position and orientation of an image capturing apparatus that captures an image of an object, the program including executable instructions, which when executed by one or more processors of an information processing apparatus, cause the information processing apparatus to perform operations comprising:

generating candidates of the position and orientation of the image capturing apparatus relative to the object;

evaluating the candidates of the position and orientation of the image capturing apparatus relative to the object on the basis of a result of matching between virtually generated data of the object with the image capturing apparatus placed in the position and orientation generated by the generating and a 3D shape model representing a 3D shape of the object;

deriving the position and orientation of the object from the result of matching,

wherein the evaluating evaluates the candidates of the position and orientation of the image capturing apparatus relative to the object by evaluating the derived position and orientation,

wherein the deriving derives a plurality of positions and orientations of the object by generating, in the vicinity of the position and orientation generated by the generating, a plurality of positions and orientations that differ from the position and orientation and matching a feature in the virtually generated data of the object observed from the originally generated position and orientation with a feature of the 3D shape model placed at each position and orientation generated in the vicinity of the position and orientation; and

determining the position and orientation of the image capturing apparatus relative to the object on the basis of the result of evaluation performed by the evaluating.

13. An information processing apparatus for determining a position and orientation of an image capturing apparatus that captures an image of an object, comprising:

one or more processors; and

one or more memory devices, the one or more memory devices storing a 3D shape model representing a 3D shape of the object, the one or more memory devices storing executable instructions, which when executed by the one or more processors, cause the information processing apparatus to perform operations comprising;

generating a plurality of candidates of the position and orientation of the image capturing apparatus relative to the object;

evaluating each of the candidates of the position and orientation of the image capturing apparatus relative to the object by deriving a plurality of positions and orientations of the image capturing apparatus relative to the object in the vicinity of the position and orientation generated by the generating and calculating a value of evaluation function based on the derived plurality of positions and orientations; and

determining the position and orientation of the image capturing apparatus relative to the object on the basis of the result of evaluation performed by the evaluating.