Tactile Interaction in Virtual Environments
1 Assignment
0 Petitions
Accused Products
Abstract
Tactile virtual reality (VR) and/or mixed reality (MR) experiences are described. Techniques described herein include receiving data from a sensor and accessing a position and an orientation of a real object that is physically present in a real scene. Furthermore, techniques described herein include identifying the real object based at least in part on the position and the orientation of the real object and causing a graphical element corresponding to the real object to be rendered on a display of a VR and/or MR display device. The graphical element can be determined based at least in part on a VR and/or MR application. The techniques described herein include determining an interaction with the real object and causing a functionality associated with the graphical element to be performed in the VR or MR environment rendered via the VR and/or MR display device, respectively.
-
Citations
21 Claims
-
1. (canceled)
-
2. A method comprising:
-
accessing, by one or more processors in a head-mounted display (HMD), video data captured by a camera in the HMD; identifying, by the one or more processors, based at least in part on the video data, a real object that is physically present in a real scene, the real object being held by a user wearing the HMD; mapping, by the one or more processors, the real object to a virtual object having a same three-dimensional geometry as the real object, the virtual object being different from the real object; generating, by the one or more processors, a virtual reality scene based on the video data, the virtual object replacing the real object in the virtual reality scene; presenting the virtual reality scene in a display of the HMD; detecting, by the one or more processors, a motion of the real object being held by the user; determining, by the one or more processors, a functionality of the virtual object based on the motion; changing, by the one or more processors, a view of the virtual object based on the determined functionality; and presenting, after changing the view of the virtual object, the virtual reality scene in the display of the HMD. - View Dependent Claims (3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17)
-
-
18. A head-mounted display (HMD) comprising:
-
a display; a camera for capturing video data in front of the HMD; a memory comprising instructions; and one or more computer processors, wherein the instructions, when executed by the one or more computer processors, cause the one or more computer processors to perform operations comprising; accessing video data captured by the camera; identifying, based at least in part on the video data, a real object that is physically present in a real scene, the real object being held by a user wearing the HMD; mapping the real object to a virtual object having a same three-dimensional geometry as the real object, the virtual object being different from the real object; generating a virtual reality scene based on the video data, the virtual object replacing the real object in the virtual reality scene; presenting the virtual reality scene in the display of the HMD; detecting a motion of the real object being held by the user; determining a functionality of the virtual object based on the motion; changing a view of the virtual object based on the determined functionality; and presenting, after changing the view of the virtual object, the virtual reality scene in the display of the HMD. - View Dependent Claims (19, 20)
-
-
21. A non-transitory machine-readable storage medium including instructions that, when executed by a processor, cause the processor to perform operations comprising:
-
accessing video data captured by a camera in a head-mounted display (HMD); identifying, based at least in part on the video data, a real object that is physically present in a real scene, the real object being held by a user wearing the HMD; mapping the real object to a virtual object having a same three-dimensional geometry as the real object, the virtual object being different from the real object; generating a virtual reality scene based on the video data, the virtual object replacing the real object in the virtual reality scene; presenting the virtual reality scene in a display of the HMD; detecting a motion of the real object being held by the user; determining a functionality of the virtual object based on the motion; changing a view of the virtual object based on the determined functionality; and presenting, after changing the view of the virtual object, the virtual reality scene in the display of the HMD.
-
Specification