Mixed reality presentation apparatus and control method thereof

• US 20020057280A1
• Filed: 03/26/2001
• Published: 05/16/2002
• Est. Priority Date: 11/24/2000
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Kitayama, Akihiro, Satoh, Kiyohide, Anabuki, Mahoro
• Granted Patent: US 6,633,304 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G02B 2027/0112 comprising device for genereting colour display
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 2027/0198 System for aligning or maintaining alignment of an image in a predetermined direction
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G09G 5/06 using colour palettes, e.g. look-up tables

Abstract

A mixed reality presentation apparatus and control method thereof which can allow for suitable display of a virtual object depending on tone, brightness, and lighting conditions of a physical space on which the virtual object is superimposed and displayed is disclosed. The mixed reality presentation apparatus according to the present invention includes: a display setting adjustor 106 which adjusts the display colors and position of virtual information and the lighting settings based on the tone of the physical space which the observer sees and it determines the display colors such that a large tone difference from the physical space image is achieved for virtual text information and virtual wire frame information as well as the lighting settings used for drawing a three-dimensional virtual object with a tone such that it is merged with an object existing in the physical space.

Claims

1. A mixed reality presentation apparatus for presenting a mixed reality to an observer by superimposing a virtual space image composed of a virtual object image on a physical space or an image thereof visually identified by said observer and presenting the virtual space image to said observer, comprising:

tone detection means for detecting a tone of a predetermined region in said physical space or the image thereof;

tone determination means for determining the tone of said virtual object image based on detection results from said tone detection means and the type of said virtual object image to be superimposed and displayed or the purpose of the display thereof;

virtual space image generation means for generating said virtual space image composed of said virtual object image based on the determination results from said tone determination means; and

presentation means for presenting said virtual space image generated by said virtual space image generation means to the observer by superimposing said virtual space image on said physical space or the image thereof.

2. The mixed reality presentation apparatus according to claim 1, wherein said tone determination means determines said tone based on the tone of a region of said physical space or the image thereof corresponding to the display region of said virtual object image.

3. The mixed reality presentation apparatus according to claim 1, wherein said tone determination means determines said tone based on the tone of said physical space or the whole image thereof.

4. The mixed reality presentation apparatus according to claim 1, wherein said tone determination means determines as said tone an lighting color used for drawing said virtual object image based on the tone of said physical space or the whole image thereof.

5. The mixed reality presentation apparatus according to claim 1, wherein said tone determination means varies the size of a difference between the tone detected by said tone detection means and the tone to be determined, depending on the type of said virtual object image or the purpose of the display thereof.

6. The mixed reality presentation apparatus according to claim 1, further comprising display position changing means for changing the display position of said virtual object image when the difference between the predetermined tone for said virtual object image and the tone determined by said tone determination means is more than a predetermined value and for supplying said predetermined tone as the determination results from said tone determination means to said virtual space image generation means if the difference between the tone determined by said tone determination means and said predetermined tone is equal to or less than said predetermined value at the changed display position.

7. The mixed reality presentation apparatus according to claim 6, wherein said display position changing means supplies said predetermined tone as the determination results from said tone determination means to said virtual space image generation means and instructs said virtual space image generation means to fill in the background of said virtual object image with the tone determined by said tone determination means, when the difference between said tones is more than said predetermined value after a predetermined number of display position changes.

8. A control method of a mixed reality presentation apparatus for presenting a mixed reality to an observer by superimposing a virtual space image composed of a virtual object image on the physical space or an image thereof visually identified by said observer and presenting the virtual space image to said observer, comprising:

a tone detection step of detecting the tone of a predetermined region in said physical space or the image thereof;

a tone determination step of determining the tone of said virtual object image based on the detection results from said tone detection step and the type of the virtual object image to be superimposed and displayed or the purpose of the display thereof;

a virtual space image generation step of generating said virtual space image composed of said virtual object image based on the determination results from said tone determination step; and

a presentation step of presenting said virtual space image generated by said virtual space image generation step to the observer by superimposing said virtual space image on said physical space or the image thereof.

9. The control method of a mixed reality presentation apparatus according to claim 8, wherein said tone determination step determines said tone based on the tone of a region of said physical space or the image thereof corresponding to the display region of said virtual object image.

10. The control method of a mixed reality presentation apparatus according to claim 8, wherein said tone determination step determines said tone based on the tone of said physical space or the whole image thereof.

11. The control method of a mixed reality presentation apparatus according to claim 8, wherein said tone determination step determines as said tone an lighting color used for drawing said virtual object image based on the tone of said physical space or the whole image thereof.

12. The control method of a mixed reality presentation apparatus according to claim 8, wherein said tone determination step varies the size of a difference between the tone detected by said tone detection step and the tone to be determined, depending on the type of said virtual object image or the purpose of the display thereof.

13. The control method of a mixed reality presentation apparatus according to claim 8, further comprising a display position changing step of changing the display position of said virtual object image when the difference between the predetermined tone for said virtual object image and the tone determined by said tone determination step is more than a predetermined value and of supplying said predetermined tone as the determination results from said tone determination step to said virtual space image generation step if the difference between the tone determined by said tone determination step and said predetermined tone is equal to or less than said predetermined value at the changed display position.

14. The control method of a mixed reality presentation apparatus according to claim 13, wherein said display position changing step supplies said predetermined tone as the determination results from said tone determination step to said virtual space image generation step and instructs said virtual space image generation step to fill in the background of said virtual object image with the tone determined by said tone determination step, when the difference between said tones is more than said predetermined value after a predetermined number of display position changes.

15. A computer-readable storage medium storing a control program of a mixed reality presentation apparatus for presenting a mixed reality to an observer by superimposing a virtual space image composed of a virtual object image on the physical space or an image thereof visually identified by the observer and presenting the virtual space image to the observer, wherein said program comprises:

a tone detection process program for detecting the tone of a predetermined region in said physical space or the image thereof;

a tone determination process program for determining the tone of said virtual object image based on the detection results from said tone detection process program and the type of said virtual object image to be superimposed and displayed or the purpose of the display thereof;

a virtual space image generation process program for generating said virtual space image composed of said virtual object image based on the determination results from said tone determination process program; and

a presentation process program for presenting said virtual space image generated by said virtual space image generation process program to the observer by superimposing said virtual space image on said physical space or the image thereof.