METHODS AND SYSTEMS FOR DETERMINING POSITIONAL DATA FOR THREE-DIMENSIONAL INTERACTIONS INSIDE VIRTUAL REALITY ENVIRONMENTS

• US 20180039341A1
• Filed: 08/02/2017
• Published: 02/08/2018
• Est. Priority Date: 08/03/2016
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: DU BOIS, Paul Lamont, HACKETT, Patrick Ryan
• Granted Patent: US 10,345,925 B2
• Time in Patent Office: Days
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 2300/8082 Virtual reality
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: H04W 4/80 Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication

Abstract

In one general aspect, systems and methods are described for determining positional data for three-dimensional interactions inside virtual reality environments. The methods may include receiving inertial measurement unit (IMU) data from at least one mobile device of a plurality of mobile devices operating in a virtual reality environment, obtaining absolute position and orientation data from at least one tracked controller, comparing the IMU data from the at least one mobile device with the absolute position and orientation data of the at least one tracked controller, detecting a correlation between the IMU data from the at least one mobile device and the absolute position and orientation data of the at least one tracked controller. Based on the detected correlation, the method may include determining absolute position and orientation for the at least one mobile device based on a determined relative orientation for the at least one tracked controller.

Claims

1. A computer-implemented method comprising:

receiving inertial measurement unit (IMU) data from at least one mobile device of a plurality of mobile devices operating in a virtual reality environment;

obtaining absolute position and orientation data from at least one tracked controller of a plurality of controllers operating in the virtual reality environment;

in response to receiving a request to determine absolute position and orientation data for the at least one mobile device,

comparing the IMU data from the at least one mobile device with the absolute position and orientation data of the at least one tracked controller;

detecting a correlation between the IMU data from the at least one mobile device and the absolute position and orientation data of the at least one tracked controller;

based on the detected correlation, storing data indicating that the at least one mobile device from the plurality of mobile devices is connected to the at least one tracked controller from the plurality of controllers;

generating a relative orientation for the at least one mobile device based on the stored data; and

determining absolute position and orientation for the at least one mobile device based on the relative orientation for the at least one tracked controller; and

communicatively coupling the at least one mobile device and the at least one tracked controller, the coupling being based on the determined absolute position and orientation for the at least one mobile device.

2. The method of claim 1, wherein communicatively coupling the at least one mobile device and the at least one tracked controller includes wirelessly communicating sensed data from the at least one tracked controller to the at least one mobile device.

3. The method of claim 2, wherein the communicative coupling provides enhanced sensed data to be communicated from the at least one tracked controller to the at least one mobile device.

4. The method of claim 1, wherein the correlation between the IMU data from the at least one mobile device and the absolute position and orientation data of the at least one tracked controller is detected by determining a plurality of common directional movements being carried out by both the at least one mobile device and the at least one tracked controller in the virtual reality environment.

5. The method of claim 1, wherein the at least one mobile device includes a virtual reality viewer that provides at least three degrees of freedom of data and the at least one tracked controller provides six degrees of freedom of data.

6. The method of claim 1, wherein the detected correlation signifies that the mobile device is rigidly attached to the at least one tracked controller.

7. The method of claim 1, further comprising:

determining a lack of correlation between the IMU data from the at least one mobile device and the absolute position and orientation of the at least one tracked controller; and

responding to the request with a communication indicating the lack of correlation.

8. The method of claim 1, further comprising:

determining that the at least one mobile device is associated with the at least one tracked controller;

providing virtual reality content to the at least one mobile device according to the determined absolute position and orientation for the at least one mobile device; and

in response to determining that one or more additional mobile devices are accessing the virtual reality environment within a predefined proximity to the at least one tracked controller, providing the virtual reality content to the one or more additional mobile devices, the virtual reality content provided to the one or more additional mobile devices being adjusted for a determined relative orientation generated for the one or more additional mobile devices.

9. A non-transitory, machine-readable medium having instructions stored thereon, the instructions, when executed by a processor, cause a computing device to:

receive inertial measurement unit (IMU) data from at least one mobile device of a plurality of mobile devices operating in a virtual reality environment;

obtain absolute position and orientation data from at least one tracked controller of a plurality of controllers operating in the virtual reality environment;

in response to receiving a request to determine absolute position and orientation data for the at least one mobile device,

compare the IMU data from the at least one mobile device with the absolute position and orientation data of the at least one tracked controller;

detect a correlation between the IMU data from the at least one mobile device and the absolute position and orientation data of the at least one tracked controller;

based on the detected correlation, store data indicating that the at least one mobile device from the plurality of mobile devices is connected to the at least one tracked controller from the plurality of controllers;

generate a relative orientation for the at least one mobile device based on the stored data; and

determine absolute position and orientation for the at least one mobile device based on the relative orientation for the at least one tracked controller; and

communicatively couple the at least one mobile device and the at least one tracked controller, the coupling being based on the determined absolute position and orientation for the at least one mobile device.

10. The machine-readable medium of claim 9, wherein communicatively coupling the at least one mobile device and the at least one tracked controller includes wirelessly communicating sensed data from the at least one tracked controller to the at least one mobile device.

11. The machine-readable medium of claim 10, wherein the communicative coupling provides enhanced sensed data to be communicated from the at least one tracked controller to the at least one mobile device.

12. The machine-readable medium of claim 9, wherein the correlation between the IMU data from the at least one mobile device and the absolute position and orientation data of the at least one tracked controller is detected by determining a plurality of common directional movements being carried out by both the at least one mobile device and the at least one tracked controller in the virtual reality environment.

13. The machine-readable medium of claim 9, wherein the at least one mobile device includes a virtual reality viewer that provides at least three degrees of freedom of data and the at least one tracked controller provides six degrees of freedom of data.

14. The machine-readable medium of claim 9 wherein the detected correlation signifies that the mobile device is rigidly attached to the at least one tracked controller.

15. The machine-readable medium of claim 9, further comprising:

determining that the at least one mobile device is associated with the at least one tracked controller;

providing virtual reality content to the at least one mobile device according to the determined absolute position and orientation for the at least one mobile device; and

in response to determining that one or more additional mobile devices are accessing the virtual reality environment within a predefined proximity to the at least one tracked controller, providing the virtual reality content to the one or more additional mobile devices, the virtual reality content provided to the one or more additional mobile devices being adjusted for a determined relative orientation generated for the one or more additional mobile devices.

16. A system comprising:

memory; and

at least one processor coupled to the memory, the at least one processor being configured to;

receive inertial measurement unit (IMU) data from at least one mobile device of a plurality of mobile devices operating in a virtual reality environment;

obtain absolute position and orientation data from at least one tracked controller of a plurality of controllers operating in the virtual reality environment;

in response to receiving a request to determine absolute position and orientation data for the at least one mobile device,

compare the IMU data from the at least one mobile device with the absolute position and orientation data of the at least one tracked controller;

detect a correlation between the IMU data from the at least one mobile device and the absolute position and orientation data of the at least one tracked controller;

based on the detected correlation, store data indicating that the at least one mobile device from the plurality of mobile devices is connected to the at least one tracked controller from the plurality of controllers;

generate a relative orientation for the at least one mobile device based on the stored data; and

determine absolute position and orientation for the at least one mobile device based on the relative orientation for the at least one tracked controller; and

communicatively couple the at least one mobile device and the at least one tracked controller, the coupling being based on the determined absolute position and orientation for the at least one mobile device.

17. The system of claim 16, wherein communicatively coupling the at least one mobile device and the at least one tracked controller includes wirelessly communicating sensed data from the at least one tracked controller to the at least one mobile device.

18. The system of claim 16, wherein the correlation between the IMU data from the at least one mobile device and the absolute position and orientation data of the at least one tracked controller is detected by determining a plurality of common directional movements being carried out by both the at least one mobile device and the at least one tracked controller in the virtual reality environment.

19. The system of claim 16, wherein the at least one mobile device includes a virtual reality viewer that provides at least three degrees of freedom of data and the at least one tracked controller provides six degrees of freedom of data.

20. The system of claim 16 wherein the detected correlation signifies that the mobile device is rigidly attached to the at least one tracked controller.