System and method for creating a navigable, three-dimensional virtual reality environment having ultra-wide field of view
First Claim
1. A system for creating a navigable, panoramic three-dimensional virtual reality environment with stereoscopic perception of three-dimensional depth to a user, comprising:
- a first pair of optical elements facing in a same first direction with their respective optical axes disposed parallel to one another, each optical element having a field of view of at least 80 degrees;
a second pair of optical elements facing in a same second direction different from the first direction with their respective optical axes disposed parallel to one another, each optical element having a field of view of at least 80 degrees;
a position detector configured to determine a direction of view of the user;
an image renderer disposed in communication with the optical elements to receive from each of the first and second pairs of optical elements, respectively, first and second video image streams of image frames, the image renderer disposed in communication with the position detector to receive the direction of view and configured to create an output stream of merged stereo image frames corresponding to the direction of view of the user, the merged frames comprising image frames from the first video streams which are temporally matched to respective image frames from the second video streams; and
a head mounted display disposed in communication with the image renderer to receive the output stream and configured to display the output stream to the user, whereby a navigable, panoramic three-dimensional virtual reality environment having stereoscopic perception of three-dimensional depth is created.
2 Assignments
0 Petitions
Accused Products
Abstract
The present invention relates to a system and method for capturing video of a real-world scene over a field of view that may exceed the field of view of a user, manipulating the captured video, and then stereoscopically displaying the manipulated image to the user in a head mounted display to create a virtual environment having length, width, and depth in the image. By capturing and manipulating video for a field of view that exceeds the field of view of the user, the system and method can quickly respond to movement by the user to update the display allowing the user to look and pan around, i.e., navigate, inside the three-dimensional virtual environment.
13 Citations
25 Claims
-
1. A system for creating a navigable, panoramic three-dimensional virtual reality environment with stereoscopic perception of three-dimensional depth to a user, comprising:
-
a first pair of optical elements facing in a same first direction with their respective optical axes disposed parallel to one another, each optical element having a field of view of at least 80 degrees; a second pair of optical elements facing in a same second direction different from the first direction with their respective optical axes disposed parallel to one another, each optical element having a field of view of at least 80 degrees; a position detector configured to determine a direction of view of the user; an image renderer disposed in communication with the optical elements to receive from each of the first and second pairs of optical elements, respectively, first and second video image streams of image frames, the image renderer disposed in communication with the position detector to receive the direction of view and configured to create an output stream of merged stereo image frames corresponding to the direction of view of the user, the merged frames comprising image frames from the first video streams which are temporally matched to respective image frames from the second video streams; and a head mounted display disposed in communication with the image renderer to receive the output stream and configured to display the output stream to the user, whereby a navigable, panoramic three-dimensional virtual reality environment having stereoscopic perception of three-dimensional depth is created. - View Dependent Claims (2, 3, 4, 5, 6, 7, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23)
-
-
8. A method for rendering video image streams to create a navigable, panoramic three-dimensional virtual reality environment having stereoscopic perception of three-dimensional depth comprising the steps of:
-
providing a first pair of optical elements facing in a same first direction with their respective optical axes disposed parallel to one another, each optical element having a field of view of at least 80 degrees; providing a second pair of optical elements facing in a same second direction different from the first direction with their respective optical axes disposed parallel to one another, each optical element having a field of view of at least 80 degrees; receiving video image streams of a scene comprising first and second video image streams taken from each of the first and second pairs of optical elements, respectively, the first video image streams having a plurality of first image frames and the second video image streams having a plurality of second image frames; merging respective pairs of the first and second image frames in stereo format to create a video output stream of merged stereoscopic image pairs; receiving position data indicating a direction of sight of a user; calculating a first and second region of interest of the merged stereoscopic image pairs based on the position data; and displaying the first and second regions of interest on a head mounted stereoscopic display, whereby a navigable, panoramic three-dimensional virtual reality environment having stereoscopic perception of three-dimensional depth is created. - View Dependent Claims (9, 10, 11, 12, 24, 25)
-
Specification