Calibration method and apparatus

• US 20060221098A1
• Filed: 03/28/2006
• Published: 10/05/2006
• Est. Priority Date: 04/01/2005
• Status: Active Grant

• Current Assignee: Canon Ayutthaya Limited (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Katano, Yasuo, Matsui, Taichi
• Granted Patent: US 7,542,051 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G06T 2207/20092 Interactive image processing based on input by user
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

An instruction is detected which indicates that a physical object having a fixed plane where the physical object can be moved, and a virtual object rendered so as to be positioned on the plane where the physical object can be moved, are observed as being in a predetermined positional relationship. Then, a difference between a position and orientation value of the virtual object and a position and orientation value of the physical object is acquired at the time when the instruction is detected, so that the difference is used as a correction value to correct the positional difference between the physical object and the virtual object.

Claims

1. A calibration method for acquiring a correction value to correct a difference in position and orientation between a physical object and a virtual object superimposed on a captured image of the physical object, the calibration method comprising:

a measuring step of measuring position and orientation of the physical object;

a virtual object generation step of generating an image of the virtual object existing on a plane on which the physical object can be moved;

a displaying step of displaying a composite image formed by superimposing the image of the virtual object on the captured image of the physical object;

a detecting step of detecting an instruction to acquire a correction value to correct a difference between position and orientation of the physical object and position and orientation of the virtual object in the composite image; and

an acquiring step of acquiring, in response to the instruction, the correction value expressed by parallel movement of the physical object on the plane and rotation of the physical object about an axis set in the direction perpendicular to the plane.

2. The calibration method according to claim 1, wherein the detecting step detects that a predetermined relationship is satisfied, on the basis of the instruction inputted by an observer of the composite image.

3. A calibration method for acquiring a correction value to correct a difference in position and orientation between a physical object and a virtual object superimposed on a captured image of the physical object, the calibration method comprising:

a first measuring step of measuring position and orientation of the physical object which can be moved;

a capturing step of capturing an image of the physical object by a camera provided on a head mounted type display device worn by an observer;

a second measuring step of measuring position and orientation of the camera;

a virtual object generation step of rendering a virtual object on the basis of the position and orientation measured in the second measuring step, in a manner that the virtual object is to be observed by the observer as existing on a predetermined plane;

a displaying step of compositing the rendered virtual object and the image captured by the camera and of displaying the composite image on the head mounted type display device;

a detecting step of detecting an instruction by the observer, the instruction indicating that the physical object is observed by the observer with a condition that a predetermined positional relationship between the physical object and the virtual object is satisfied; and

an acquiring step of acquiring, as the correction value, a difference of position and orientation between the physical object and the virtual object at the time when the instruction is detected,

wherein in the virtual object generation step, the virtual object is rendered in a moved and/or rotated manner on the plane on the basis of the position and orientation measured in the first measuring step.

4. The calibration method according to claim 3, wherein the predetermined plane is a horizontal plane and the direction perpendicular to the plane is a vertical direction.

5. The calibration method according to claim 3, wherein the virtual object generation step renders the virtual object corresponding to the position and orientation of the physical object.

6. The calibration method according to claim 3, wherein the virtual object generation step renders the virtual object by using position and orientation which are corrected by using the acquired correction value.

7. A calibration apparatus for acquiring a correction value to correct a difference in position and orientation between a physical object and a virtual object superimposed on a captured image of the physical object, the calibration apparatus comprising:

a measuring unit adapted to measure position and orientation of the physical object;

a virtual object generation unit adapted to generate an image of the virtual object existing on a plane on which the physical object can be moved;

a displaying unit adapted to display a composite image formed by superimposing the image of the virtual object on the captured image of the physical object;

a detecting unit adapted to detect an instruction to acquire a correction value to correct a difference between position and orientation of the physical object and position and orientation of the virtual object in the composite image; and

an acquiring unit adapted to acquire, in response to the instruction, the correction value expressed by parallel movement of the physical object on the plane and rotation of the physical object about an axis set in the direction perpendicular to the plane.

8. The calibration apparatus according to claim 7, wherein the detecting unit detects that a predetermined relationship is satisfied, on the basis of the instruction inputted by an observer of the composite image.

9. A calibration apparatus for acquiring a correction value to correct a difference in position and orientation between a physical object and a virtual object superimposed on a captured image of the physical object, the calibration apparatus comprising:

a first measuring unit adapted to measure position and orientation of the physical object which can be moved;

a capturing unit adapted to capture an image of the physical object by a camera provided on a head mounted type display device worn by an observer;

a second measuring unit adapted to measure position and orientation of the camera;

a virtual object generation unit adapted to render a virtual object on the basis of the position and orientation measured in the second measuring unit, in a manner that the virtual object is to be observed by the observer as existing on a predetermined plane;

a displaying unit adapted to composite the rendered virtual object and the image captured by the camera and to display the composite image on the head mounted type display device;

a detecting unit adapted to detect an instruction by the observer, the instruction indicating that the physical object is observed by the observer with a condition that a predetermined positional relationship between the physical object and the virtual object is satisfied; and

an acquiring unit adapted to acquire, as the correction value, a difference of position and orientation between the physical object and the virtual object at the time when the instruction is detected,

wherein in the virtual object generation unit, the virtual object is rendered in a moved and/or rotated manner on the plane on the basis of the position and orientation measured in the first measuring unit.

10. The calibration apparatus according to claim 9, wherein the predetermined plane is a horizontal plane and the direction perpendicular to the plane is a vertical direction.

11. The calibration apparatus according to claim 9, wherein the virtual object generation unit renders the virtual object corresponding to the position and orientation of the physical object.

12. The calibration apparatus according to claim 9, wherein the virtual object generation unit renders the virtual object by using position and orientation which are corrected by using the acquired correction value.