Mixed reality realizing system

• US 6,795,041 B2
• Filed: 07/02/2002
• Issued: 09/21/2004
• Est. Priority Date: 03/31/2000
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Hitachi Zosen Corporation
• Inventors: Ogawa, Kenji, Saeki, Toshiro, Tamura, Haruyo
• Primary Examiner(s): Nguyen, Jimmy H.
• Application Number:
  US10/189,127
• Publication Number:
  US 20040004583A1
• Time in Patent Office: 812 Days
• Field of Search: 345/7, 345/8, 345/9, 348/222, 348/115, 359/13, 349/11
• US Class Current: 345/7
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted

Abstract

According to a mixed reality realizing system of the present invention, when a viewer wears a head mount display (HMD) including a video display and a CCD camera, and views an exhibit using an optical see-through type system, identification signals and position signals transmitted from infrared LEDs located around the exhibit are received to determine a position of the exhibit, and to superimpose contents such as explanations of the exhibit, which have been stored in advance in a portable computer, on a predetermined position on the video display where the exhibit is displayed.

Claims

1. A mixed reality realizing system, comprising identification signal transmitters located around a target object to transmit identification signals of the target object, a plurality of position indicators for indicating a position of the target object, and a portable type terminal device comprising a head mount display, a controller and a computer,

the head mount display of said portable type terminal device comprising:

a video screen;

an identification signal receiver for receiving the identification signals of the target object transmitted from the identification signal transmitters; and

a camera for shooting the plurality of position indicators,

the computer comprising;

a storage for storing, in advance, data including coordinate positions of the plurality of position indicators on an image shot by the camera from a reference position previously determined based on the target object and contents relating to the target object;

a control operation unit for receiving the signals from the identification signal receiver and retrieving the contents relating to the target object from the storage;

a coordinate position recognizing unit for recognizing, in real time, coordinate positions of the plurality of position indicators on the image shot by the camera;

a position-information operation unit for calculating a visible position and a real position of the target object based upon the plurality of coordinate positions obtained from the position indicators; and

a content superimposing unit for superimposing the contents relating to the target object on the target object on the video display of the head mount display based on the information on the plurality of positions obtained from the position-information operation unit, the contents having been registered in advance in the storage,

wherein the position-information operation unit calculates a difference between each coordinate position of the position indicator on the image registered in advance in the storage of the computer, and a coordinate position recognized in real time by the coordinate position recognizing unit, thereby to superimpose in real time the contents relating to the target object on the target object based on the calculation result.

2. The mixed reality realizing system as defined in claim 1, wherein an optical see-through type system is used as the head mount display, the optical see-through type system including a half mirror for permeating a reflected light from the target object toward a visible position of the target object, and outputting no background image shot by the camera to the video display.

3. The mixed reality realizing system as defined in claim 1, wherein a video see-through type system is used as the head mount display, the video see-through type system including a both-sided mirror located on a reflected light course from the target object toward the visible position of the target object to allow the reflected light from the target object to refract in other directions than the one toward the visible position, and outputting a background image shot by the camera to the video display.