Position and orientation measurement method and apparatus

• US 8,019,114 B2
• Filed: 09/12/2007
• Issued: 09/13/2011
• Est. Priority Date: 09/14/2006
• Status: Expired due to Fees

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Satoh, Kiyohide, Uchiyama, Shinji, Endo, Takaaki
• Primary Examiner(s): Akhavannik; Hadi
• Application Number:
  US11/854,121
• Publication Number:
  US 20080069405A1
• Time in Patent Office: 1,462 Days
• Field of Search: 382/100
• US Class Current: 382/100
• CPC Class Codes: G01B 11/002 for measuring two or more coordinates
• CPC Class Codes: G01B 11/2504 Calibration devices
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

A position and orientation measurement apparatus has a subjective-view-index detection unit which detects indices on a physical space from an image captured by a subjective-view camera, an additional subjective-view-index detection unit which detects indices on the physical space from an image captured by an additional subjective-view camera having a spatial resolution different from the subjective-view camera, and a position and orientation estimation unit which calculates the position and orientation of the subjective-view camera using information associated with the image coordinates of the detected indices.

Claims

1. A position and orientation measurement method comprising:

a first image input step of inputting a first image captured by a first image capture device which captures an image of a physical space;

a second image input step of inputting a second image captured by a second image capture device which has a fixed position and orientation relative to the first image capture device and has a spatial resolution different from the first image capture device;

a first detection step of detecting, from the first image, a feature amount associated with image coordinates of an index on the physical space;

a second detection step of detecting, from the second image, a feature amount associated with image coordinates of an index on the physical space; and

a position and orientation calculation step of calculating a position and orientation of the first image capture device based on the feature amounts associated with the image coordinates of the index detected in the first detection step and the second detection step,

wherein the position and orientation calculation step calculates the position and orientation by multiplying the feature amount associated with the image coordinates of the index detected in the detection step associated with the image captured by the image capture device with the higher spatial resolution of the first detection step and the second detection step by a weight value larger than the feature amount associated with the image coordinates of the index detected in the detection step associated with the image captured by the image capture device with the lower spatial resolution.

2. A position and orientation measurement method according to claim 1, wherein the position and orientation calculation step comprising:

a lower-resolution position and orientation calculation step of calculating a position and orientation of the first image capture device based on the feature amount associated with the image coordinates of the index detected in the detection step associated with the image captured by the image capture device with the lower spatial resolution of the first detection step and the second detection step; and

a correction step of correcting at least one of the position and orientation of the first image capture device calculated in the position and orientation calculation step based on the feature amount associated with the image coordinates of the index detected in the detection step associated with the image captured by the image capture device with the higher spatial resolution of the first detection step and the second detection step.

3. A position and orientation measurement method according to claim 1, wherein the position and orientation calculation step comprising:

a higher-resolution position and orientation calculation step of calculating a position and orientation of the first image capture device based on the feature amount associated with the image coordinates of the index detected in the detection step associated with the image captured by the image capture device with the higher spatial resolution of the first detection step and the second detection step; and

a correction step of correcting the position of the first image capture device calculated in the position and orientation calculation step based on the feature amount associated with the image coordinates of the index detected in the detection step associated with the image captured by the image capture device with the lower spatial resolution of the first detection step and the second detection step.

4. The method according to claim 1, wherein the first image capture device and the second image capture device have the same image resolution and different angles of view.

5. The method according to claim 1, further comprising a sensor measurement value input step of inputting a measurement value from a sensor which measures the position or the orientation of the first image capture device, or time derivatives thereof,

wherein the position and orientation calculation step calculates the position and orientation of the first image capture device based on the feature amounts associated with the image coordinates of the index detected in the first detection step and the second detection step, and the measurement value input in the sensor measurement value input step.

6. The method according to claim 1, further comprising:

a third image input step of inputting a third image captured by a third image capture device which captures an image of the first image capture device and has a fixed position and orientation; and

a third detection step of detecting a feature amount associated with image coordinates of a second index which has a fixed position and orientation relative to the first image capture device,

wherein the position and orientation calculation step calculates the position and orientation of the first image capture device based on the feature amounts associated with the image coordinates of the index detected in the first detection step and the second detection step, and the feature amount associated with the image coordinates of the second index detected in the third detection step.

7. The method according to claim 1, wherein the first image is an image to be presented to an observer, and the first image capture device has a fixed position or orientation, or a fixed angle of view relative to display unit which presenting the first image to the observer.

8. The method according to claim 1, wherein a direction of an optical axis of the first image capture device is different from a direction of an optical axis of the second image capture device.

9. The method according to claim 1, wherein the second image capture device includes a plurality of image capture devices,

the second image input step inputs the second image from each of the plurality of image capture devices, and

the second detection step detects feature amounts associated with the image coordinates of the index on the physical space from a plurality of the second images input in the second image input step.

10. The method according to claim 9, wherein the plurality of image capture devices include an image capture device having a higher spatial resolution than the first image capture device, and an image capture device having a lower spatial resolution than the first image capture device.

11. The method according to claim 9, wherein a direction of an optical axis of at least one of the plurality of image capture devices is different from a direction of an optical axis of at least one of the remaining image capture devices.

12. The method according to claim 1, wherein the first image capture device and the second image capture device capture images observed in different wavelength ranges.

14. A non-transitory computer-readable recording medium storing a program for making a computer execute a position and orientation calculation method according to claim 1.

13. A position and orientation measurement apparatus comprising:

first image input unit which inputs a first image captured by a first image capture device which captures an image of a physical space;

second image input unit which inputs a second image captured by a second image capture device which has a fixed position and orientation relative to the first image capture device and has a spatial resolution different from the first image capture device;

first detection unit which detects, from the first image, a feature amount associated with image coordinates of an index on the physical space;

second detection unit which detects, from the second image, a feature amount associated with image coordinates of an index on the physical space; and

position and orientation calculation unit which calculates a position and orientation of the first image capture device based on the feature amounts associated with the image coordinates of the index detected by the first detection unit and the second detection unit,

wherein the position and orientation calculation unit calculates the position and orientation by multiplying the feature amount associated with the image coordinates of the index detected by the detection unit associated with the image captured by the image capture device with the higher spatial resolution of the first detection unit and the second detection unit by a weight value larger than the feature amount associated with the image coordinates of the index detected in the detection unit associated with the image captured by the image capture device with the lower spatial resolution.