Image processing method and image processing apparatus

• US 7,808,450 B2
• Filed: 04/07/2006
• Issued: 10/05/2010
• Est. Priority Date: 04/20/2005
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Ohshima, Toshikazu, Wanda, Koichiro
• Primary Examiner(s): Awad; Amr
• Assistant Examiner(s): BOYD, JONATHAN A
• Application Number:
  US11/400,000
• Publication Number:
  US 20060273984A1
• Time in Patent Office: 1,642 Days
• Field of Search: 345 7- 9, 345/633
• US Class Current: 345/8
• CPC Class Codes: A63F 13/10 Control of the course of the game, e.g. start, progess, end
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/45 Controlling the progress of the video game
• CPC Class Codes: A63F 13/52 involving aspects of the displayed game scene
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/6692 using special effects, generally involving post-processing, e.g. blooming
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: A63F 2300/8023 the game being played by multiple players at a common site, e.g. in an arena, theatre, shopping mall using a large public display
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

When a plurality of mixed reality (MR) experiencing persons experience the same mixed reality, there is a possibility that the experiencing persons come into contact with one another. An image processing apparatus capable of reporting to the MR experiencing persons that there is a possibility of contact. Therefore, a real space image that is image captured from the position and orientation of a user'"'"'s viewpoint is drawn. The position and orientation of the user'"'"'s viewpoint at this time is detected by a sensor unit. It is determined whether or not the viewpoint position is smaller than or equal to an attention distance at which there is a possibility of contact. If the viewpoint position is smaller than or equal to the attention distance, an attention display for this fact is performed.

Claims

1. An image processing method comprising:

taking a real space image observed from the position and orientation of a user'"'"'s viewpoint;

detecting the position and orientation of the user'"'"'s viewpoint;

generating a combined image by combining computer graphics based on the position and orientation of the user'"'"'s viewpoint with the real space image;

providing a mixed reality by displaying the combined image to the user on a head-mounted display;

setting a first attention area based on the physique of the user and a second attention area based on the physique of at least one other user experiencing the mixed reality;

measuring a distance between the first attention area and the second attention area;

determining whether the distance between the first attention area and the second attention area becomes smaller than or equal to a first predetermined value;

for each of the at least one other user, reporting to the user that a respective one of at least one other user approaches the user when it is determined that the distance between the first attention area and the second attention area becomes smaller than or equal to the first predetermined value;

determining whether the distance between the first attention area and the second attention area becomes smaller than or equal to a second predetermined value which is smaller than the first predetermined value after it is determined that the distance between the first attention area and the second attention area becomes smaller than or equal to the first predetermined value; and

not combining the computer graphics with the real space image when it is determined that the distance between the first attention area and the second attention area becomes smaller than or equal to the second predetermined value.

2. The image processing method according to claim 1, wherein the type of report that a respective one other user approaches the user differs depending on the distance between the first attention area and the second attention area.

3. The image processing method according to claim 1, wherein the first predetermined value and the second predetermined value used for the determination of whether the distance between the first attention area and the second attention area-becomes smaller than or equal to the first predetermined value and the second predetermined value differs depending on the orientation, the moving direction, and the speed of the user.

4. The image processing method according to claim 1, wherein reporting that a respective one other user approaches the user is made to the respective one other user.

5. The image processing method according to claim 1, wherein the report to the user is made in such a manner that computer graphics of the combined image to be displayed to the user is made to be semi-transparent depending on the distance so that the drawing of the real space is not hidden.

6. The image processing method according to claim 1, wherein the report to the user is made in such a manner that, based on the distance and the speed of the user, computer graphics of the combined image to be displayed to the user is made to be semi-transparent so that the drawing of the real space is not hidden.

7. A non-transitory recording medium having recorded thereon a program for enabling a computer to perform an image processing method comprising:

taking a real space image observed from the position and orientation of a user'"'"'s viewpoint;

detecting the position and orientation of the user'"'"'s viewpoint;

generating a combined image by combining computer graphics based on the position and orientation of the user'"'"'s viewpoint with the real space image;

providing a mixed reality by displaying the combined image to the user on a head-mounted display;

setting a first attention area based on the physique of the user and a second attention area based on the physique of at least one other user experiencing the mixed reality;

measuring a distance between the first attention area and the second attention area;

determining whether the distance between the first attention area and the second attention area becomes smaller than or equal to a first predetermined value;

for each of at least one other user, reporting to the user that a respective one of at least one other user approaches the user when it is determined that the distance between the first attention area and the second attention area becomes smaller than or equal to the first predetermined value

determining whether the distance between the first attention area and the second attention area becomes smaller than or equal to a second predetermined value which is smaller than the first predetermined value after it is determined that the distance between the first attention area and the second attention area becomes smaller than or equal to the first predetermined value; and

not combining the computer graphics with the real space image when it is determined that the distance between the first attention area and the second attention area becomes smaller than or equal to the second predetermined value.

8. An image processing apparatus comprising:

an imaging unit configured to take a real space image observed from the position and orientation of a user'"'"'s viewpoint;

a detection unit configured to detect the position and orientation of the user'"'"'s viewpoint;

a combining unit configured to generate a combined image by combining computer graphics based on the position and orientation of the viewpoint with the real space image;

a display unit configured to provide a mixed reality by displaying the combined image to the user on a head-mounted display;

a setting unit configured to set a first attention area based on the physique of the user and a second attention area based on the physique of the at least one other user experiencing the mixed reality;

a measurement unit configured to measure distances between the first attention area and the second attention area;

a determination unit configured to determine whether the distance between the first attention area and the second attention area becomes smaller than or equal to a first predetermined value; and

a reporting unit configured to report to the user, for each of at least one other user, that a respective one of at least one other user approaches the user when it is determined by the determination unit that the distance between the first attention area and the second attention area becomes smaller than or equal to the first predetermined value,

wherein the determination unit is configured to determine whether the distance between the first attention area and the second attention area becomes smaller than or equal to a second predetermined value which is smaller than the first predetermined value after it is determined that the distance between the first attention area and the second attention area becomes smaller than or equal to the first predetermined value, and

wherein the combining unit is configured to not combine the computer graphics with the real space image when it is determined by the determining unit that the distance between the first attention area and the second attention area becomes smaller than or equal to the second predetermined value.