Position and orientation measuring apparatus, information processing apparatus and information processing method

• US 9,704,248 B2
• Filed: 02/05/2014
• Issued: 07/11/2017
• Est. Priority Date: 02/07/2013
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Hirota, Yuichiro, Tateno, Keisuke
• Primary Examiner(s): Shah, Utpal
• Application Number:
  US14/173,741
• Publication Number:
  US 20140219502A1
• Time in Patent Office: 1,252 Days
• CPC Class Codes: G01B 11/002 for measuring two or more coordinates
• CPC Class Codes: G01B 11/2513 with several lines being projected in more than one direction, e.g. grids, patterns
• CPC Class Codes: G01S 5/163 Determination of attitude using inertial means G01C9/00; control of attitude G05D1/49
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects

Abstract

There is provided a position and orientation measurement apparatus, information processing apparatus, and an information processing method, capable of performing robust measurement of a position and orientation. In order to achieve the apparatuses and method, at least one coarse position and orientation of a target object is acquired from an image including the target object, at least one candidate position and orientation is newly generated as an initial value used for deriving a position and orientation of the target object based on the acquired coarse position and orientation, and the position and orientation of the target object in the image is derived by using model information of the target object and by performing at least once of fitting processing of the candidate position and orientation generated as the initial value with the target object in the image.

Claims

1. A position and orientation measurement apparatus comprising:

at least one memory storing computer executable instructions; and

at least one processor configured to execute the computer executable instructions to implement;

an acquisition unit configured to acquire, from an image including a target object, at least one coarse position and orientation of the target object, wherein the coarse position and orientation is acquired based on a predetermined resolution;

a generation unit configured to generate, based on the acquired coarse position and orientation and the predetermined resolution, at least one candidate position and orientation which is different from the coarse position and orientation; and

a deriving unit configured to derive a position and orientation of the target object in the image based on the generated position and orientation and model information of the target object.

2. The position and orientation measurement apparatus according to claim 1, wherein the acquisition unit acquires the at least one coarse position and orientation of the target object in the image by performing pattern matching with respect to the image.

3. The position and orientation measurement apparatus according to claim 1, wherein the acquisition unit performs pattern matching by searching the image by using the model information of the target object.

4. The position and orientation measurement apparatus according to claim 1, wherein the generation unit generates at least one candidate position and orientation by changing a parameter representing at least one of the position and the orientation in a range of the predetermined resolution with respect to the acquired coarse position and orientation.

5. The position and orientation measurement apparatus according to claim 1, wherein the predetermined resolution is a measurement accuracy of when the coarse position and orientation is measured.

6. The position and orientation measurement apparatus according to claim 1, wherein the generation unit generates at least one candidate position and orientation based on the acquired coarse position and orientation, further texture information of the target object, and the image.

7. The position and orientation measurement apparatus according to claim 1, wherein the generation unit generates at least one candidate position and orientation based on further shape information of the target object.

8. The position and orientation measurement apparatus according to claim 1, wherein the deriving unit derives the position and orientation of the target object in the image by extracting an edge from the model information of the target object the in the generated candidate position and orientation and comparing the extracted edge and an edge extracted from the image.

9. An information processing apparatus comprising:

the position and orientation measurement apparatus according to claim 1;

a holding unit configured to hold the target object; and

a control unit configured to control the holding unit based on the position and orientation of the target object measured by the position and orientation measurement apparatus, wherein

the at least one memory is configured to implement the holding unit, and

the at least one processor is configured to execute the computer executable instructions to implement the control unit.

12. The apparatus according to claim 1, further comprising an association unit configured to associate each of model features of model information of the target object, set based on corresponding one of a plurality of positions and orientations including the newly generated candidate position and orientation with an image feature of the target object of the image, wherein the at least one processor is configured to execute the computer executable instructions to implement the association unit.

13. The apparatus to claim 1, further comprising:

an updating unit configured to (i) obtain, for each of a plurality of candidate positions and orientations, a result of associating an image feature with a model feature of model information of the target object based on each of a plurality of candidate positions and orientations including the generated at least one candidate position and orientation and (ii) update each of the plurality of candidate positions and orientations based on a difference between the image feature and the model feature associated with each other for each of a plurality of candidate positions and orientations; and

a selection unit configured to select at least one position and orientation among the plurality of the candidate position and orientation based on a result of processing by the updating unit,

wherein the deriving unit derives the position and orientation of the target object in the image based on the selected at least one candidate position and orientation.

10. A position and orientation measurement method comprising:

acquiring, from an image including a target object, at least one coarse position and orientation of the target object, wherein the coarse position and orientation is acquired based on a predetermined resolution;

generating, based on the acquired coarse position and orientation and the predetermined resolution, at least one candidate position and orientation; and

deriving a position and orientation of the target object in the image based on the generated position and orientation and model information of the target object.

11. A non-transitory computer-readable storage medium storing a computer-executable program capable of causing a computer to execute a position and orientation measurement method, the position and orientation measurement method comprising:

acquiring, from an image including a target object, at least one coarse position and orientation of the target object, wherein the coarse position and orientation is acquired based on a predetermined resolution;

generating, based on the acquired coarse position and orientation and the predetermined resolution, at least one candidate position and orientation; and

deriving a position and orientation of the target object in the image based on the generated position and orientation and model information of the target object.

14. A position and orientation measurement method comprising:

acquiring, from an image including a target object, a plurality of coarse positions and orientations of the target object;

generating, based on a distribution of the acquired plurality of coarse positions and orientations of the target object, at least one candidate position and orientation;

performing an updating step that (i) obtains, for each of a plurality of candidate positions and orientations, a result of associating an image feature with a model feature of model information of the target object based on each of a plurality of candidate positions and orientations including the generated at least one candidate position and orientation and (ii) updates each of the plurality of candidate positions and orientations based on a difference between the image feature and the model feature associated with each other for each of a plurality of candidate positions and orientations;

selecting at least one position and orientation among the plurality of candidate positions and orientations based on a result of processing by the updating step; and

deriving the position and orientation of the target object in the image based on the selected at least one position and orientation.

15. A non-transitory computer-readable storage medium storing a computer-executable program capable of causing a computer to execute a position and orientation measurement method, the position and orientation measurement method comprising:

acquiring, from an image including a target object, a plurality of coarse positions and orientations of the target object;

generating, based on a distribution of the acquired plurality of coarse positions and orientations of the target object, at least one candidate position and orientation;

performing an updating step that (i) obtains, for each of a plurality of candidate positions and orientations, a result of associating an image feature with a model feature of model information of the target object based on each of a plurality of candidate positions and orientations including the generated at least one candidate position and orientation and (ii) updates each of the plurality of candidate positions and orientations based on a difference between the image feature and the model feature associated with each other for each of a plurality of candidate positions and orientations;

selecting at least one position and orientation among the plurality of candidate positions and orientations based on a result of processing by the updating step; and

deriving the position and orientation of the target object in the image based on the selected at least one position and orientation.

16. A position and orientation measurement apparatus comprising:

at least one memory storing computer executable instructions; and

at least one processor configured to execute the computer executable instructions to implement;

an acquisition unit configured to acquire, from an image including a target object, a plurality of coarse positions and orientations of the target object;

a generation unit configured to generate, based on a distribution of the acquired plurality of coarse positions and orientations of the target object, at least one candidate position and orientation;

an updating unit configured to (i) obtain, for each of a plurality of candidate positions and orientations, a result of associating an image feature with a model feature of model information of the target object based on each of a plurality of candidate positions and orientations including the generated at least one candidate position and orientation and (ii) update each of the plurality of candidate positions and orientations based on a difference between the image feature and the model feature associated with each other for each of a plurality of candidate positions and orientations;

a selection unit configure to select at least one position and orientation among the plurality of candidate positions and orientations based on a result of processing by the updating unit; and

a deriving unit configured to derive the position and orientation of the target object in the image based on the selected at least one position and orientation.

17. The position and orientation measurement apparatus according to claim 16, wherein the generation unit approximates a distribution of the acquired plurality of coarse positions and orientations with a predetermined distribution function, and generates at least one candidate position and orientation according to the approximated distribution.

18. The position and orientation measurement apparatus according to claim 16, wherein the generation unit divides a distribution of the acquired coarse position and orientation into a plurality of areas and generates at least one candidate position and orientation based on the divided area.

19. The position and orientation measurement apparatus according to claim 16, wherein the at least one processor executes the computer executable instructions to implement an evaluation unit, and the evaluation unit evaluates an amount of a misalignment between the model feature of model information, set by the generated candidate position and orientation, and the image feature of the image is made smaller.