Image compositing method and apparatus for superimposing a computer graphics image on an actually-sensed image

• US 7,928,977 B2
• Filed: 09/02/2005
• Issued: 04/19/2011
• Est. Priority Date: 09/06/2004
• Status: Expired due to Fees

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Ohshima, Toshikazu, Tanimura, Kaname
• Primary Examiner(s): BROOME, SAID A
• Application Number:
  US11/217,469
• Publication Number:
  US 20060050087A1
• Time in Patent Office: 2,055 Days
• Field of Search: None
• US Class Current: 345/419
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

A three-dimensional computer graphics model which represents a measurable range of a position and orientation sensor is prepared in advance. A composite image is generated by superimposing an image of a virtual object which represents the measurable range of the position and orientation sensor on an actually-sensed image in real space where the position and orientation sensor is placed. The composite image, when displayed, makes it possible to visually recognize the measurable range of the position and orientation sensor.

Claims

1. An image compositing method for presenting a composite image to a user wearing a head mounted display, comprising:

an acquisition step of acquiring, by using a position and orientation sensor, a position and orientation of the head mounted display;

a compositing step of generating a first composite image by compositing an actually-sensed image and a first computer graphics image based on the acquired position and orientation;

a superimposing step of superimposing a second computer graphics image, which represents a measurable range of the position and orientation sensor, on the first composite image only when the user, within the measurable range, approaches a boundary of the measurable range of the position and orientation sensor, wherein said superimposing step generates a second composite image by superimposing the second computer graphics image on the first composite image;

a presenting step of presenting the first or second composite image to the user via the head mounted display;

a setting step of setting three-dimensional computer graphics models which represent at least two measurable ranges; and

a CG image generating step of generating a third computer graphics image which represents an overlap region of at least two measurable ranges, the color of the third computer graphics image being different from the color of the second computer graphics image;

wherein said superimposing step further generates a third composite image by superimposing the third computer graphics image on the actually-sensed image when the user is outside of the measurable ranges; and

wherein said presenting step further presents the third composite image when the user is outside of the measurable range.

2. The image compositing method according to claim 1, wherein the acquisition step acquires the position and orientation of the head mounted display using an image of a marker attached to the position and orientation sensor and prestored position information about the marker, where the image of the marker is contained in the actually-sensed image.

3. A non-transitory computer-readable recording medium for storing a program for making a computer perform the image compositing method according to claim 1.

4. An image compositing apparatus for presenting a composite image to a user wearing a head mounted display, comprising:

an acquisition unit for acquiring, by using a position and orientation sensor, a position and orientation of the head mounted display;

a compositing unit for generating a first composite image by compositing an actually-sensed image and a first computer graphics image based on the acquired position and orientation;

a superimposing unit for superimposing a second computer graphics image, which represents a measurable range of the position and orientation sensor, on the first composite image only when the user, within the measurable range, approaches a boundary of the measurable range of the position and orientation sensor, wherein said superimposing unit generates a second composite image by superimposing the second computer graphics image on the first composite image;

a presenting unit for presenting the first or the second composite image to the user via the head mounted display;

a setting unit of setting three-dimensional computer graphics models which represent at least two measurable ranges; and

a CG image generating unit of generating a third computer graphics image which represents an overlap region of at least two measurable ranges, the color of the third computer graphics image being different from the color of the second computer graphics image;

wherein said superimposing unit further generates a third composite image by superimposing the third computer graphics image on the actually-sensed image when the user is outside of the measurable ranges; and

wherein said presenting unit further presents the third composite image when the user is outside of the measurable range.