Video combining apparatus and method

• US 7,487,468 B2
• Filed: 09/29/2003
• Issued: 02/03/2009
• Est. Priority Date: 09/30/2002
• Status: Expired due to Fees

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Tanaka, Rika, Tomite, Kaname, Ohshima, Toshikazu
• Primary Examiner(s): Huynh; Ba
• Assistant Examiner(s): Augustine; Nicholas
• Application Number:
  US10/671,611
• Publication Number:
  US 20040070611A1
• Time in Patent Office: 1,954 Days
• Field of Search: 715/863, 715/744, 715756-757, 715/773, 345 7- 9, 345156-157, 382/103
• US Class Current: 715/863
• CPC Class Codes: H04N 21/4122 additional display device, e.g. video projector digital output for controlling a plurality of local displays G06F3/1423
• CPC Class Codes: H04N 21/4143 embedded in a Personal Computer [PC]
• CPC Class Codes: H04N 21/42222 Additional components integrated in the remote control device, e.g. timer, speaker, sensors for detecting position, direction or movement of the remote control, microphone or battery charging device
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/42653 for processing graphics
• CPC Class Codes: H04N 21/431 Generation of visual interfaces for content selection or interaction; Content or additional data rendering
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 5/44504 Circuit details of the additional information generator, e.g. details of the character or graphics signal generator, overlay mixing circuits

Abstract

A video combining apparatus to superimpose a virtual image such a CG image on a video image of the real world or on a see-through type display device. An area in which the virtual image is not to be displayed can be easily designated by a user. If the user holds a frame with markers in his/her sight, the frame is image-sensed in the video image of the real world. The area designated by the user is detected by detecting the position of the marker in the video image, and the virtual image is not superimposed in this area.

Claims

1. A video combining method for superimposing a computer graphics image generated by a computer on a real world observed by a user, said method comprising the steps of:

a first extracting step of extracting a region having a particular color from the real world observed by the user;

a second extracting step of extracting a closed region within the region extracted by the first extracting step;

an inputting step of inputting position and orientation information of a view point of the user;

a generating step of generating a computer graphics image based on the position and orientation information;

a determining step of determining an image elimination region over which the computer graphics image is not to be superimposed, based on (a) the closed region and (b) the position and orientation information; and

a superimposing step of superimposing the computer graphics image on the real world observed by the user except for the image elimination region.

2. The video combining method according to claim 1, wherein the first extracting step extracts as the region having the particular color a region corresponding to at least one hand of the user.

5. A computer-readable storage medium storing a program for causing a computer to effect the method of claim 1.

3. A video combining apparatus for superimposing a computer graphics image generated by a computer on a real world observed by a user, said apparatus comprising:

first extracting means for extracting a region having a particular color from the real world observed by the user;

second extracting means for extracting a closed region within the region extracted by the first extracting means;

inputting means for inputting position and orientation information of a view point of the user;

generating means for generating a computer graphics image based on the position and orientation information;

determining means for determining an image elimination region over which the computer graphics image is not to be superimposed, based on (a) the closed region and (b) the position and orientation information; and

superimposing means for superimposing the computer graphics image on the real world observed by the user except for the image elimination region.

4. The video combining apparatus according to claim 3, wherein the first extracting means extracts as the region having the particular color a region corresponding to at least one hand of the user.

6. A video combining method for superimposing a computer graphics image generated by a computer on a video image of a real world observed by a user, said method comprising the steps of:

a first extracting step of extracting a region having a particular color from the video image of the real world observed by the user;

a second extracting step of extracting a closed region within the region extracted by the first extracting step;

an inputting step of inputting position and orientation information of a view point of the user;

a generating step of generating a computer graphics image based on the position and orientation information;

a determining step of determining an image elimination region over which the computer graphics image is not to be superimposed, based on (a) the closed region and (b) the position and orientation information; and

a superimposing step of superimposing the computer graphics image on the video image of the real world observed by the user except for the image elimination region.

7. The video combining method according to claim 6, wherein the first extracting step extracts as the region having the particular color a region corresponding to at least one hand of the user.

10. A computer-readable storage medium storing a program for causing a computer to effect the method of claim 6.

8. A video combining apparatus for superimposing a computer graphics image generated by a computer on a video image of a real world observed by a user, said apparatus comprising:

first extracting means for extracting a region having a particular color from the video image of the real world observed by the user;

second extracting means for extracting a closed region within the region extracted by the first extracting means;

inputting means for inputting position and orientation information of a view point of the user;

generating means for generating a computer graphics image based on the position and orientation information;

determining means for determining an image elimination region over which the computer graphics image is not to be superimposed, based on (a) the closed region and (b) the position and orientation information; and

superimposing means for superimposing the computer graphics image on the video image of the real world observed by the user except for the image elimination region.

9. The video combining apparatus according to claim 8, wherein the first extracting means extracts as the region having the particular color a region corresponding to at least one hand of the user.