Sensor calibration apparatus, sensor calibration method, program, storage medium, information processing method, and information processing apparatus

• US 7,130,754 B2
• Filed: 07/16/2004
• Issued: 10/31/2006
• Est. Priority Date: 03/19/2002
• Status: Expired due to Term

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Kobayashi, Toshihiro, Satoh, Kiyohide, Uchiyama, Shinji
• Primary Examiner(s): Nghiem, Michael
• Assistant Examiner(s): Walling, Meagan S
• Application Number:
  US10/892,104
• Publication Number:
  US 20040249594A1
• Time in Patent Office: 837 Days
• Field of Search: 702/94, 702/95, 702/150, 702/151, 702/152, 702/153, 382/154, 382/285, 382/282, 382/284, 345/419
• US Class Current: 702/95
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

A calibration information calculation unit 340 calculates the first coordinate positions of feature points included in images obtained by an image sensing apparatus at timings from an instruction unit 350 using position data, on world coordinate system, of a plurality of feature points held by a world coordinate holding unit 310 and the measured values of a position/posture sensor 130 input to a data management unit 330 at the timings. The unit 340 receives the second coordinate positions, which are acquired by an image coordinate acquisition unit 320, of the feature points included in the images obtained by the image sensing device at the timings. The unit 340 calculates calibration information using the first and second coordinate positions.

Claims

1. An information processing method comprising the steps of:

holding values of coordinate positions of a plurality of markers on a predetermined coordinate system;

acquiring a captured image obtained by sensing an image of a real space on which the markers are arranged using an image sensing unit;

adjusting parameters used upon detecting markers, based on a user'"'"'s instruction;

detecting the markers included in the captured image;

identifying the markers that have been detected on the basis of the values; and

combining information corresponding to an identification result in said identifying step, on the captured image, and displaying a resulting combined image.

2. A method according to claim 1, wherein combined information resulting from the combining indicates positions of the markers that have been detected in the captured image.

3. A method according to claim 1, wherein combined information resulting from the combining indicates values of coordinate positions of markers corresponding to the markers identified on the captured image.

4. A method according claim 1, wherein combined information resulting from the combining indicates that the markers that have been detected cannot be identified on the captured image, when such markers cannot be identified.

5. A method according to claim 1, wherein the parameters include threshold values of colors and size information of each color region.

6. A method according to claim 1, further comprising the step of identifying parameters of markers that have been detected.

7. A program stored on a computer-readable storage medium, said program including code having instructions which cause a computer to execute an information processing method according to claim 1.

8. A computer-readable storage medium which stores a program, the program having code including instructions which cause a computer to execute an information processing method according to claim 1.

9. An information processing apparatus comprising:

a holding unit adapted to hold values of coordinate positions of a plurality of markers on a predetermined coordinate system;

an acquiring unit adapted to acquire a captured image obtained by sensing an image of a real space on which the markers are arranged using an image sensing unit;

an adjusting unit adapted to adjust parameters used upon detecting markers based on a user'"'"'s instruction;

a detecting unit adapted to detect the markers included in the captured image;

an identifying unit adapted to identify the markers detected by said detecting unit on the basis of the values held by said holding unit; and

a combining unit adapted to combine information corresponding to an identification result obtained by said identifying unit on the captured image, and displaying a resulting combined image.

10. An information processing method for calculating calibration information used to transform a position and orientation of an image capturing device measured by a sensor into that of a world coordinate system, including steps of

acquiring a plurality of images captured in a physical space where a plurality of markers are arranged, by the capturing device, wherein world coordinates of the each of marker is known;

acquiring a plurality of measured values, from the sensor, corresponding to each of the captured images;

detecting image coordinates of markers included in the captured image;

selecting the captured image to be used to calculate the calibration information from the acquired captured images according to a user'"'"'s instruction;

calculating the calibration information using the measured value of the selected captured image, the detected image coordinates of each of the markers included in the selected captured image, and the world coordinates of each of the markers;

combining a captured image with a sign indicating image coordinates of each of markers included in the captured image, identification information of each of the markers included in the captured image, and a sign indicating image coordinates calculated from world coordinates of each of the markers included in the captured image by using the calculated calibration information; and

displaying the combined image.

11. A method according to claim 10, wherein the image coordinate of the marker is automatically detected by analyzing the captured image, and further comprising, deleting the detected image coordinate of the marker, and adding an image coordinate of a marker included in the captured image, according to user'"'"'s instruction based on the displayed image.

12. The method according to claim 10, wherein the displayed image corresponds to the selected captured image from the plurality of captured images according to user'"'"'s instruction.

13. The method according to claim 10, further comprising a step of displaying errors of the markers based on the calibration information and displaying an average value of the errors.

14. The method according to claim 10, further comprising a step of adjusting detection condition to detect the image coordinate, according to user'"'"'s instruction.

15. A computer-readable storage medium which stores a program that when executed in a computer performs an information processing method, the method comprising the steps of:

holding values of coordinate positions of a plurality of markers on a predetermined coordinate system;

acquiring a captured image obtained by sensing an image of a real space on which the markers are arranged using an image sensing unit;

adjusting parameters used upon detecting markers, based on a user'"'"'s instruction;

detecting the markers included in the captured image;

identifying the markers that have been detected on the basis of the values; and

combining information corresponding to an identification result in said identifying step, on the captured image, and displaying a resulting combined image.

16. An information processing apparatus comprising:

a holding unit for holding values of coordinate positions of a plurality of markers on a predetermined coordinate system;

an acquiring unit for acquiring a captured image obtained by sensing an image of a real space on which the markers are arranged using an image sensing unit;

an adjusting unit for adjusting parameters used upon detecting markers, based on a user'"'"'s instruction;

a detection unit for detecting the markers included in the captured image;

an identifying unit for identifying the markers that have been detected on the basis of the values; and

a combining unit for combining information corresponding to an identification result obtained in said identifying unit, on the captured image, and for displaying a resulting combined image.