Tactile interaction in virtual environments
First Claim
1. A system comprising:
- a sensor;
one or more processors;
memory; and
one or more modules stored in the memory and executable by the one or more processors to perform operations comprising;
receiving first data from the sensor;
identifying, based at least in part on the first data, a first real object and a second real object that are physically present in a real scene;
determining a position of the first real object and a position of the second real object in the real scene;
accessing a first virtual element and a second virtual element representing at least one of a geometry or surface details of the first real object and the second real object;
causing a first virtual object and a second virtual object, corresponding to the first virtual element and the second virtual element respectively, to be rendered in a virtual reality scene based on a virtual reality application;
receiving second data from the sensor;
detecting, based on the second data, an interaction where the second real object is placed within a threshold distance of the first real object in the real scene; and
causing, in response to the interaction, presentation of a third virtual object to be rendered on the virtual reality scene to replace the first virtual object and the second virtual object.
1 Assignment
0 Petitions
Accused Products
Abstract
Tactile virtual reality (VR) and/or mixed reality (MR) experiences are described. Techniques described herein include receiving data from a sensor and accessing a position and an orientation of a real object that is physically present in a real scene. Furthermore, techniques described herein include identifying the real object based at least in part on the position and the orientation of the real object and causing a graphical element corresponding to the real object to be rendered on a display of a VR and/or MR display device. The graphical element can be determined based at least in part on a VR and/or MR application. The techniques described herein include determining an interaction with the real object and causing a functionality associated with the graphical element to be performed in the VR or MR environment rendered via the VR and/or MR display device, respectively.
-
Citations
18 Claims
-
1. A system comprising:
-
a sensor; one or more processors; memory; and one or more modules stored in the memory and executable by the one or more processors to perform operations comprising; receiving first data from the sensor; identifying, based at least in part on the first data, a first real object and a second real object that are physically present in a real scene; determining a position of the first real object and a position of the second real object in the real scene; accessing a first virtual element and a second virtual element representing at least one of a geometry or surface details of the first real object and the second real object; causing a first virtual object and a second virtual object, corresponding to the first virtual element and the second virtual element respectively, to be rendered in a virtual reality scene based on a virtual reality application; receiving second data from the sensor; detecting, based on the second data, an interaction where the second real object is placed within a threshold distance of the first real object in the real scene; and causing, in response to the interaction, presentation of a third virtual object to be rendered on the virtual reality scene to replace the first virtual object and the second virtual object. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13)
-
-
14. A computer-implemented method, the method comprising:
-
receiving first data from a sensor; accessing, based at least in part on the first data received from the sensor, a position and an orientation of a first real object that is physically present in a real scene; identifying the first real object based at least in part on the position and the orientation of the first real object; causing a first graphical element corresponding to the first real object to be rendered on a display of a virtual reality display device, the first graphical element being determined based at least in part on a first virtual reality application; receiving second data from the sensor; determining, based at least in part on the second data, a first interaction with the first real object; causing a first functionality associated with the first graphical element to be performed; determining that the first virtual reality application is terminated; determining that a second virtual reality application is activated; and causing a second graphical element corresponding to the first real object to be rendered on the display of the virtual reality display device, the second graphical element being determined based at least in part on a second virtual reality application and being different from the first graphical element. - View Dependent Claims (15, 16, 17)
-
-
18. One or more computer storage media having computer-executable instructions that, when executed by one or more processors, configure the one or more processors to perform operations comprising:
-
receiving first data from a sensor; identifying a real object that is physically located in a real environment based at least in part on the first data; determining, based at least in part on the first data, a geometry of the real object; assigning a unique identifier to the real object; generating a virtual element that maps to the geometry of the real object; assigning the unique identifier to the virtual element; performing semantic recognition on the real object to determine an identity of the real object; based at least in part on results of the semantic recognition, associating one or more graphical elements with the unique identifier; based at least in part on identifying the real object, determining the unique identifier corresponding to the real object; and accessing the virtual element and the one or more graphical elements; rendering a virtual object that corresponds to the real object in a virtual reality environment, the virtual object being associated with; the virtual element that represents the real object; a graphical element corresponding to the virtual element, the graphical element changing based at least in part on different virtual reality applications; and one or more functionalities, the one or more functionalities changing based at least in part on at least one of the different virtual reality applications or different interactions with the real object; receiving second data from the sensor; determining, based at least in part on the second data, an interaction with the real object; and causing, based at least in part on the interaction, a performance of the one or more functionalities in at least one of a mixed reality environment or a virtual reality environment.
-
Specification