Extended virtual table: an optical extension for table-like projection systems

• US 6,803,928 B2
• Filed: 01/17/2002
• Issued: 10/12/2004
• Est. Priority Date: 06/06/2000
• Status: Expired due to Term

• Current Assignee: Fraunhofer-Gesselshaft Zur Foerderung Der Angewandten Forschung E.V.
• Original Assignee: Fraunhofer Gesellschaft Zur Foerderung Der Angewandten Forsching E.V.
• Inventors: Stork, Andre, Bimber, Oliver, Encarnacao, L. Miguel
• Primary Examiner(s): Chow, Dennis-Doon
• Assistant Examiner(s): Kumar, Srilakshmi K.
• Application Number:
  US10/031,465
• Publication Number:
  US 20030085866A1
• Time in Patent Office: 999 Days
• Field of Search: 345/433, 345/419, 345/757, 345/7, 345/8, 345/9, 345/755, 340/980, 342/463, 359/480
• US Class Current: 715/757
• CPC Class Codes: G05B 19/186 Generation of screw- or gearlike surfaces
• CPC Class Codes: G05B 2219/32014 Augmented reality assists operator in maintenance, repair, programming, assembly, use of head mounted display with 2-D 3-D display and voice feedback, voice and gesture command
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041

Abstract

Apparatus that uses a large transflective mirror to extend a virtual reality system such as a virtual table that employs a projection plane to produce the virtual reality. The transflexive mirror is positioned relative to the projection plane such that the plane of the mirror intersects the projection plane and the angle of the mirror relative to the projection plane is such that the user of the system who looks at the mirror sees the projection plane reflected in the mirror. The virtual reality system is responsive to the position of the mirror and the direction in which a user is looking and produces separate virtual realities on the projection plane: one when the user is looking at the mirror and another when the user is looking at the projection plane. The virtual reality that the user sees when looking at the mirror may or may not be coherent with the virtual reality that the user sees when looking at the projection plane. The two virtual realities may share a global coordinate system that is divided into two parts by the plane of the mirror, and what the user sees when looking into the mirror may be what the user would see looking through a window into the part of the global coordinate system behind the mirror. Because the mirror is transflective, real objects whose locations are known to the virtual reality system can he placed behind the mirror and the virtual reality reflected in the mirror may be used to augment the real objects. One use of such augmentation is virtual trial assembly of a virtual mockup with a physical mockup.

Claims

1. A virtual environment system comprising:

apparatus for producing a virtual environment on a projection plane;

a planar mirror, the planar mirror being of substantial size relative to the projection plane and being positioned relative to the projection plane such that the plane of the mirror intersects the projection plane and the angle of the mirror relative to the projection plane is such that a user of the system who looks at the mirror sees the projection plane reflected therein; and

a tracker that tracks the position and orientation of the eyes of a user of the virtual environment system,

the apparatus for producing the virtual environment receiving a current position of the mirror and producing a first virtual environment on the projection plane when the tracker indicates that the user is looking at the mirror and a second virtual environment on the projection plane when the tracker indicates that the user is looking at the projection plane.

2. The virtual environment system set forth in claim 1 wherein:

the user may move a virtual object produced in the apparatus between the first and second virtual environments.

3. The virtual environment system set forth in claim 1 wherein:

the first virtual environment is coherent with the second virtual environment.

4. The virtual environment system set forth in claim 3 wherein:

the mirror divides a space having a single global coordinate system, the single global coordinate system being employed by the apparatus to produce the virtual environment.

5. The virtual environment system set forth in claim 4 wherein:

the first virtual environment is a view of the portion of the virtual environment which is behind the mirror in the global coordinate system as it would be seen from the direction and point of view of the user if the mirror were transparent and the user were looking through the mirror into the global coordinate system.

6. The virtual environment system set forth in claim 1 wherein:

the first virtual environment is not coherent with the second virtual environment.

7. The virtual environment system set forth in claim 6 wherein:

the mirror functions as a magic lens.

8. The virtual environment system set forth in claim 1 wherein:

the mirror divides a space having a single global coordinate system, the single global coordinate system being employed by the apparatus for producing the virtual environment.

9. The virtual environment system set forth in claim 8 wherein:

the mirror is transflective when there is a light source on the side of the mirror that does not reflect the projection plane.

10. The virtual environment system set forth in claim 9 further comprising:

a real object, the real object being visible through the mirror by a user of the virtual environment system when the mirror is transflective and having a location in the global coordinate system,

the apparatus for producing the virtual environment responding to the location of the real object by producing a virtual object in the first virtual environment that is reflected in the mirror such that the virtual object augments the real object.

11. The virtual environment system set forth in claim 10 wherein:

the virtual object is a representation of an object that is intended to fit together with the real object.

12. The virtual environment system set forth in claim 11 wherein:

the real object is a physical mockup and the virtual object is a virtual mockup.

13. The virtual environment system set forth in claim 6 wherein:

the apparatus for producing a virtual environment is attached to a video camera aimed at another such apparatus by a network; and

the first virtual environment includes a view of the other apparatus for producing a virtual environment and the other apparatus'"'"' users, the view being produced by the video camera.

14. The virtual environment system set fort in claim 13 wherein:

the other apparatus is another virtual environment system and is attached via the network to another video camera aimed at the apparatus for producing a virtual environment and the first virtual environment in the other virtual environment system includes a view of the virtual environment system and the virtual environment system'"'"'s users.

15. The virtual environment system set forth in claim 14 wherein:

the apparatus for producing a virtual environment in the virtual environment system and the apparatus for producing a virtual environment in the other virtual environment system further include a microphone and a loudspeaker, the loudspeaker in each of the systems receiving input via the network from the microphone in the other system.