Virtual reality immersion system

• US 20040104935A1
• Filed: 07/28/2003
• Published: 06/03/2004
• Est. Priority Date: 01/26/2001
• Status: Abandoned Application

• Current Assignee: Zaxel Systems, Inc.
• Original Assignee: Zaxel Systems, Inc.
• Inventors: Williamson, Todd, Suzuki, Norihisa
• Application Number:
  US10/628,951
• Publication Number:
  US 20040104935A1
• Time in Patent Office: Days
• US Class Current: 345/757
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 7/564 from contours
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G06V 10/10 Image acquisition document image scanning and transmission H04N1/00; control of digital cameras H04N23/60

Abstract

A virtual reality immersion system provides a head mounted display that contains a video camera and a video display. A plurality of target markers are distributed within a virtual environment room where each target is distinct from all other targets in the virtual environment room and distinct from rotated versions of itself. An automatic calibration program selects pair of targets from an image from the video camera. The selected pairs of targets are identified and the position of each target is calculated relative to the camera and relative to each other. The positions of each target pair are added to a list of relative target transforms. Video signals are processed to calculate the position of targets detected in each frame image using the relative target transforms and the effects of viewing angles are detected and a higher weight is given to targets that are detected at more reliable angles. The invention dynamically streams 3D content to the user through the video display. Once the target positions have been calculated, the invention determines the user position within the environment and when the user changes his viewpoint, the information from the calculated user position is used to change the position and angle of the 3D content and the 3D content is repositioned and streamed to the video display.

Claims

1. A process for a immersing a user in a three dimensional virtual reality environment room, comprising the steps of:

providing a head mounted display worn by a user;

wherein said head mounted display comprises a video camera and video display;

providing a plurality of target markers;

distributing said plurality of target markers within said room;

wherein each of said plurality of target markers are distinct from all other target markers in said plurality of target markers and distinct from rotated versions of itself;

receiving a video signal of a portion of said room from said video camera;

identifying at least one target marker in said video signal;

calculating user position within said room using relative positioning of identified target marker(s);

streaming three dimensional video content to the user through said video display; and

dynamically repositioning the user'"'"'s perspective viewpoint within said three dimensional video content by using said calculated user position to adjust position and viewing angle within said three dimensional video content.

2. The process of claim 1, further comprising the step of:

providing target marker calibration means for automatically calibrating relative positions of said plurality of target markers within said room.

3. The process of claim 2, wherein said target marker calibration means detects said plurality of target markers in said room using said video signal, and wherein said target marker calibration means identifies pairs of target markers within said room.

4. The process of claim 3, wherein said target marker calibration means determines positioning of each target in a target pair relative to said video camera, wherein said target marker calibration means calculates positioning of each target in a target pair relative to each other, and wherein said target marker calibration means stores the relative positioning of the target pair in a list of relative target transforms.

5. The process of claim 4, wherein said calculating user position step determines the relative positioning of identified target markers using said relative target transforms.

6. The process of claim 5, wherein said calculating user position step detects the effects of viewing angles and gives higher weight to target markers that are detected at more reliable angles.

7. The process of claim 1, wherein said receiving step receives said video signal via a wireless link.

8. The process of claim 1, wherein said streaming step streams said three dimensional video content to the user through said video display via a wireless link.

9. The process of claim 1, wherein said streaming step overlays said three dimensional video content onto said video signal and sends a resulting combined signal to said video display.

10. The process of claim 1, wherein said plurality of target markers are a sufficient number such that at least one target marker is always visible in said video signal.

11. An apparatus for a immersing a user in a three dimensional virtual reality environment room, comprising:

a head mounted display worn by a user;

wherein said head mounted display comprises a video camera and video display;

a plurality of target markers;

means for distributing said plurality of target markers within said room;

wherein each of said plurality of target markers are distinct from all other target markers in said plurality of target markers and distinct from rotated versions of itself;

a module for receiving a video signal of a portion of said room from said video camera;

a module for identifying at least one target marker in said video signal;

a module for calculating user position within said room using relative positioning of identified target marker(s);

a module for streaming three dimensional video content to the user through said video display; and

a module for dynamically repositioning the user'"'"'s perspective viewpoint within said three dimensional video content by using said calculated user position to adjust position and viewing angle within said three dimensional video content.

12. The apparatus of claim 11, further comprising:

target marker calibration means for automatically calibrating relative positions of said plurality of target markers within said room.

13. The apparatus of claim 12, wherein said target marker calibration means detects said plurality of target markers in said room using said video signal, and wherein said target marker calibration means identifies pairs of target markers within said room.

14. The apparatus of claim 13, wherein said target marker calibration means determines positioning of each target in a target pair relative to said video camera, wherein said target marker calibration means calculates positioning of each target in a target pair relative to each other, and wherein said target marker calibration means stores the relative positioning of the target pair in a list of relative target transforms.

15. The apparatus of claim 14, wherein said calculating user position module determines the relative positioning of identified target markers using said relative target transforms.

16. The apparatus of claim 15, wherein said calculating user position module detects the effects of viewing angles and gives higher weight to target markers that are detected at more reliable angles.

17. The apparatus of claim 11, wherein said receiving module receives said video signal via a wireless link.

18. The apparatus of claim 11, wherein said streaming module streams said three dimensional video content to the user through said video display via a wireless link.

19. The apparatus of claim 11, wherein said streaming module overlays said three dimensional video content onto said video signal and sends a resulting combined signal to said video display.

20. The apparatus of claim 11, wherein said plurality of target markers are a sufficient number such that at least one target marker is always visible in said video signal.

21. A process for tracking a video camera in a three dimensional virtual reality environment room, comprising the steps of:

providing a video camera movable within said room;

providing a plurality of target markers;

distributing said plurality of target markers within said room;

wherein each of said plurality of target markers are distinct from all other target markers in said plurality of target markers and distinct from rotated versions of itself;

receiving a video signal of a portion of said room from said video camera;

identifying at least one target marker in said video signal; and

calculating video camera position within said room using relative positioning of identified target marker(s).

22. The process of claim 21, further comprising the step of:

providing target marker calibration means for automatically calibrating relative positions of said plurality of target markers within said room.

23. The process of claim 22, wherein said target marker calibration means detects said plurality of target markers in said room using said video signal, and wherein said target marker calibration means identifies pairs of target markers within said room.

24. The process of claim 23, wherein said target marker calibration means determines positioning of each target in a target pair relative to said video camera, wherein said target marker calibration means calculates positioning of each target in a target pair relative to each other, and wherein said target marker calibration means stores the relative positioning of the target pair in a list of relative target transforms.

25. The process of claim 24, wherein said calculating video camera position step determines the relative positioning of identified target markers using said relative target transforms.

26. The process of claim 25, wherein said calculating video camera position step detects the effects of viewing angles and gives higher weight to target markers that are detected at more reliable angles.

27. The process of claim 21, wherein said receiving step receives said video signal via a wireless link.

28. The process of claim 21, wherein said plurality of target markers are a sufficient number such that at least one target marker is always visible in said video signal.

29. An apparatus for tracking a video camera in a three dimensional virtual reality environment room, comprising:

a video camera movable within said room;

a plurality of target markers;

means for distributing said plurality of target markers within said room;

wherein each of said plurality of target markers are distinct from all other target markers in said plurality of target markers and distinct from rotated versions of itself;

a module for receiving a video signal of a portion of said room from said video camera; and

a module for identifying at least one target marker in said video signal;

a module for calculating video camera position within said room using relative positioning of identified target marker(s).

30. The apparatus of claim 29, further comprising:

target marker calibration means for automatically calibrating relative positions of said plurality of target markers within said room.

31. The apparatus of claim 30, wherein said target marker calibration means detects said plurality of target markers in said room using said video signal, and wherein said target marker calibration means identifies pairs of target markers within said room.

32. The apparatus of claim 31, wherein said target marker calibration means determines positioning of each target in a target pair relative to said video camera, wherein said target marker calibration means calculates positioning of each target in a target pair relative to each other, and wherein said target marker calibration means stores the relative positioning of the target pair in a list of relative target transforms.

33. The apparatus of claim 32, wherein said calculating video camera position module determines the relative positioning of identified target markers using said relative target transforms.

34. The apparatus of claim 33, wherein said calculating video camera position module detects the effects of viewing angles and gives higher weight to target markers that are detected at more reliable angles.

35. The apparatus of claim 29, wherein said receiving module receives said video signal via a wireless link.

36. The apparatus of claim 29, wherein said plurality of target markers are a sufficient number such that at least one target marker is always visible in said video signal.