Image compositing method and apparatus

• US 20060050087A1
• Filed: 09/02/2005
• Published: 03/09/2006
• Est. Priority Date: 09/06/2004
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Tanimura, Kaname, Ohshima, Toshikazu
• Granted Patent: US 7,928,977 B2
• Time in Patent Office: Days
• US Class Current: 345/629
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

A three-dimensional computer graphics model which represents a measurable range of a position and orientation sensor is prepared in advance. A composite image is generated by superimposing an image of a virtual object which represents the measurable range of the position and orientation sensor on an actually-sensed image in real space where the position and orientation sensor is placed. The composite image, when displayed, makes it possible to visually recognize the measurable range of the position and orientation sensor.

Claims

1. An image compositing method for superimposing a computer graphics image on an actually-sensed image, comprising:

a setting step of setting a three-dimensional computer graphics model which represents a measurable range of a position and orientation sensor;

a sensor position information acquisition step of acquiring position information about the position and orientation sensor;

a CG image generating step of generating a computer graphics image which represents the measurable range using the position information about the position and orientation sensor and the three-dimensional computer graphics model; and

an image compositing step of compositing the actually-sensed image with the computer graphics image.

2. The image compositing method according to claim 1, wherein the sensor position information acquisition step acquires the position information about the sensor using an image of a marker attached to the position and orientation sensor and prestored position information about the marker, where the image of the marker is contained in the actually-sensed image.

3. The image compositing method according to claim 1, wherein:

the sensor position information acquisition step detects the marker in the actually-sensed image, and prepares transformation conditions according to positional relationship between a marker coordinate system in which the marker is defined and a camera coordinate system in which the actually-sensed image is defined, based on the detected marker; and

the CG image generating step generates the computer graphics image from the three-dimensional computer graphics model based on the transformation conditions.

9. A program for making a computer perform the image compositing method according to claim 1.

10. A computer-readable recording medium for storing the program according to claim 9.

11. The image compositing method according to claim 1, comprising:

a setting step of setting a three-dimensional computer graphics model which represents a measurable range of an objective camera instead of the position and orientation sensor; and

an objective-camera position and orientation information acquisition step of acquiring position information about the objective camera, wherein

the CG image generating step generates the computer graphics image from the position information about the objective camera and the three-dimensional computer graphics model.

4. An image compositing method for superimposing a computer graphics image on an actually-sensed image using position and orientation information measured by a position and orientation sensor with a predetermined measurable range, comprising:

a setting step of setting a three-dimensional computer graphics model which represents the predetermined measurable range;

a camera position and orientation information acquisition step of acquiring position and orientation information about a camera which shoots the actually-sensed image, using the position and orientation sensor;

a CG image generating step of generating a computer graphics image which represents the predetermined measurable range using the camera position and orientation information and the three-dimensional computer graphics model; and

an image compositing step of compositing the actually-sensed image with the computer graphics image.

5. An image compositing method for compositing an actually-sensed image shot by a camera installed on head-mounted display worn by an observer with an image of a virtual object generated from a previously prepared three-dimensional computer graphics model of the virtual object and position and orientation information about the camera and displaying the composite image on the head-mounted display, comprising:

a setting step of setting a three-dimensional computer graphics model which represents a measurable range of a position and orientation sensor placed in a space in which an observer exists; and

an image generating step of generating an image of a virtual object, representing the measurable range.

6. An image compositing apparatus which superimposes a computer graphics image on an actually-sensed image, comprising:

setting unit adapted to set a three-dimensional computer graphics model which represents a measurable range of a position and orientation sensor;

sensor position information acquisition unit adapted to acquire position information about the position and orientation sensor;

CG image generating unit adapted to generate a computer graphics image which represents the measurable range using the position information about the position and orientation sensor and the three-dimensional computer graphics model; and

image compositing unit adapted to composite the actually-sensed image with the computer graphics image.

7. An image compositing apparatus which superimposes a computer graphics image on an actually-sensed image using position and orientation information measured by a position and orientation sensor with a predetermined measurable range, comprising:

setting unit adapted to set a three-dimensional computer graphics model which represents the predetermined measurable range;

camera position and orientation information acquisition adapted to acquire position and orientation information about a camera which shoots the actually-sensed image, using the position and orientation sensor;

CG image generating unit adapted to generate a computer graphics image which represents the predetermined measurable range using the camera position and orientation information and the three-dimensional computer graphics model; and

image compositing unit adapted to composite the actually-sensed image with the computer graphics image.

8. An image compositing apparatus which composites an actually-sensed image shot by a camera installed on head-mounted display worn by an observer with an image of a virtual object generated from a previously prepared three-dimensional computer graphics model of the virtual object and position and orientation information about the camera and displaying the composite image on the head-mounted display, comprising:

setting unit adapted to set a three-dimensional computer graphics model which represents a measurable range of a position and orientation sensor placed in a space in which an observer exists; and

image generating unit adapted to generate an image of a virtual object, representing the measurable range.