INFORMATION PROCESSING APPARATUS AND INFORMATION PROCESSING METHOD

• US 20080228422A1
• Filed: 03/11/2008
• Published: 09/18/2008
• Est. Priority Date: 03/13/2007
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Satoh, Kiyohide
• Granted Patent: US 7,640,129 B2
• Time in Patent Office: Days
• US Class Current: 702/92
• CPC Class Codes: G01C 11/00 Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry

Abstract

A calibration information calculation unit (540) calculates a plurality of candidates of first coordinate transformation information using a plurality of candidates of second coordinate transformation information, a sensor measured value, and the position and orientation of a video camera (100) on the world coordinate system. The calibration information calculation unit (540) then calculates a piece of first coordinate transformation information by combining the plurality of calculated candidates. Then, the calibration information calculation unit (540) makes iterative calculations for correcting calibration information using a candidate of the second coordinate transformation information and the first coordinate transformation information as initial values of the calibration information.

Claims

1. An information processing apparatus comprising:

an image acquisition unit adapted to acquire a physical space image obtained by sensing a physical space on which a plurality of indices whose world coordinates are known are allocated using an image sensing device;

a coordinate acquisition unit adapted to acquire image coordinates of the indices in the physical space image with reference to the image sensing device;

a position and orientation acquisition unit adapted to acquire a position and orientation of a survey point on the image sensing device in a coordinate system with reference to a sensor measured by the sensor;

a candidate acquisition unit adapted to acquire a plurality of candidates of second transformation information required to transform between a coordinate system with reference to the image sensing device and a coordinate system with reference to the survey point;

an estimation unit adapted to calculate an estimated value of a position and orientation of the image sensing device on the world coordinate system using the acquired image coordinates of the indices and the positions of the indices on the world coordinate system;

a calculation unit adapted to calculate first transformation information used to transform between the coordinate system with reference to the sensor and the world coordinate system using a selected one of the second transformation information selected from among the plurality of candidates of the second transformation information, the measured value of the position and orientation, and the estimated value of the position and orientation; and

a correction unit adapted to make iterative calculations for correcting calibration information used to calibrate the position and orientation in the coordinate system with reference to the sensor to the world coordinate system using, as initial values of the calibration information, the selected second transformation information and the calculated first transformation information.

2. The apparatus according to claim 1, wherein the candidates of the second transformation information are stored in advance.

3. The apparatus according to claim 1, wherein said candidate acquisition unit generates the candidates of the second transformation information using random numbers.

4. The apparatus according to claim 1, further comprising a unit adapted to check if the correction of the calibration information by said correction unit has succeeded.

5. The apparatus according to claim 1, said correction unit comprising a unit adapted to select calibration information with a smallest one of residual errors calculated by processing for correcting the calibration information.

6. The apparatus according to claim 1, further comprising:

a unit adapted to calibrate the position and orientation with reference to the sensor using the calibration information corrected by said correction unit; and

a unit adapted to output the calibrated position and orientation.

7. The apparatus according to claim 1, further comprising:

a generation unit adapted to calibrate the position and orientation with reference to the sensor using the corrected calibration information, and to generate a virtual space image seen from a viewpoint having the calibrated position and orientation; and

an output unit adapted to generate a composite image by compositing the generated virtual space image and the physical space image, and to output the composite image to a display device.

8. The apparatus according to claim 1, wherein said calculation means calculates the candidates of the first transformation information using the measured positions and orientations at different time points and the position and orientation of the image sensing device and calculates the first transformation information based on the candidates of the first transformation information.

9. An information processing apparatus for calculating calibration information of a sensor used to measure a position and orientation of an image sensing device, said apparatus comprising:

an input unit adapted to input measured values of the sensor at different positions and orientations of the image sensing device;

an acquisition unit adapted to acquire actually measured values of coordinates of a plurality of feature points included in images sensed by the image sensing device at the different positions and orientations; and

a calculation unit adapted to calculate the calibration information based on the plurality of measured values input by said input unit, the plurality of actually measured values acquired by said acquisition unit, and pre-stored world coordinates of the plurality of feature points,

wherein the calibration information includes first transformation information used to transform a position and orientation on a sensor coordinate system based on the sensor into a position and orientation on a world coordinate system, and second transformation information used to transform a position and orientation as measured values of a survey point by the sensor into a position and orientation on a coordinate system based on the image sensing device, and

said calculation unit comprises;

a first calculation unit adapted to calculate a position and orientation of the image sensing device based on the actually measured values of the coordinates of the feature points acquired by said acquisition unit and the world coordinates of these feature points at least at one position and orientation of the different positions and orientations;

a first setting unit adapted to set approximate values of the second transformation information;

a second calculation unit adapted to calculate approximate values of the first transformation information corresponding to the approximate values of the second transformation information set by said first setting unit based on the position and orientation of the image sensing device calculated by said first calculation unit and the measured values of the sensor at these position and orientation; and

a correction unit adapted to correct the calibration information based on the measured values of the sensor at the different positions and orientations input by said input unit and the actually measured values of the coordinates of the feature points at the different positions and orientations acquired by said acquisition unit using, as initial values of the calibration information, the approximate values of the second transformation information set by said first setting unit, and the approximate values of the first transformation information calculated by said second calculation unit.

10. An information processing method comprising:

an image acquisition step of acquiring a physical space image obtained by sensing a physical space on which a plurality of indices whose world coordinates are known are allocated using an image sensing device;

a coordinate acquisition step of acquiring image coordinates of the indices in the physical space image with reference to the image sensing device;

a position and orientation acquisition step of acquiring a position and orientation of a survey point on the image sensing device in a coordinate system with reference to a sensor measured by the sensor;

a candidate acquisition step of acquiring a plurality of candidates of second transformation information required to transform between a coordinate system with reference to the image sensing device and a coordinate system with reference to the survey point;

an estimation step of calculating an estimated value of a position and orientation of the image sensing device on the world coordinate system using the acquired image coordinates of the indices and the positions of the indices on the world coordinate system;

a calculation step of calculating first transformation information used to transform between the coordinate system with reference to the sensor and the world coordinate system using a selected one of the second transformation information from among the plurality of candidates of the second transformation information, the measured value of the position and orientation, and the estimated value of the position and orientation; and

a correction step of making iterative calculations for correcting calibration information used to calibrate the position and orientation in the coordinate system with reference to the sensor to the world coordinate system using, as initial values of the calibration information, the selected second transformation information and the calculated first transformation information.

12. A computer-readable storage medium storing a computer program for making a computer execute an information processing method according to claim 10.

11. An information processing method for calculating calibration information of a sensor used to measure a position and orientation of an image sensing device, the method comprising:

an input step of inputting measured values of the sensor at different positions and orientations of the image sensing device;

an acquisition step of acquiring actually measured values of coordinates of a plurality of feature points included in images sensed by the image sensing device at the different positions and orientations; and

a calculation step of calculating the calibration information based on the plurality of measured values input in the input step, the plurality of actually measured values acquired in the acquisition step, and pre-stored world coordinates of the plurality of feature points,

wherein the calibration information includes first transformation information used to transform a position and orientation on a sensor coordinate system based on the sensor into a position and orientation on a world coordinate system, and second transformation information used to transform a position and orientation as measured values of a survey point by the sensor into a position and orientation on a coordinate system based on the image sensing device, and

the calculation step comprises;

a first calculation step of calculating a position and orientation of the image sensing device based on the actually measured values of the coordinates of the feature points acquired in the acquisition step and the world coordinates of these feature points at least at one position and orientation of the different positions and orientations;

a first setting step of setting approximate values of the second transformation information;

a second calculation step of calculating approximate values of the first transformation information corresponding to the approximate values of the second transformation information set in the first setting step based on the position and orientation of the image sensing device calculated in the first calculation step and the measured values of the sensor at these position and orientation; and

a correction step of correcting the calibration information based on the measured values of the sensor at the different positions and orientations input in the input step and the actually measured values of the coordinates of the feature points at the different positions and orientations acquired in the acquisition step using, as initial values of the calibration information, the approximate values of the second transformation information set in the first setting step, and the approximate values of the first transformation information calculated in the second calculation step.