Multi-user virtual reality processing

• US 10,269,089 B2
• Filed: 05/22/2018
• Issued: 04/23/2019
• Est. Priority Date: 12/16/2015
• Status: Active Grant

• Current Assignee: Worldviz LLC
• Original Assignee: Worldviz LLC
• Inventors: Lashkari, Farshid, Beall, Andrew C.
• Primary Examiner(s): Xiao, Ke
• Assistant Examiner(s): Tran, Kim Thanh T
• Application Number:
  US15/986,054
• Publication Number:
  US 20180268513A1
• Time in Patent Office: 336 Days
• Field of Search: 345633, 345504
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

A multi-user virtual reality and/or augmented reality cluster system enabling the projection and tracking of users in a simulated environment is described. The disclosed cluster system enables both timely user action display in the first person and a timely view of a user'"'"'s action by other participants of the simulation. The cluster system integrates motion capture cameras, distributed computing, and tracking devices to provide a simulation which seems lifelike to all participants and observers.

Claims

1. A cluster system, comprising:

a master computing device;

a plurality of motion tracking cameras, wherein the plurality of motion tracking cameras are placeable in a physical space coincident with a first user, the plurality of motion tracking cameras configured to communicate position data, corresponding to detected infrared light, to the master computing device;

a marker, wearable or holdable by the first user, configured to emit a repetitive pattern of infrared light pulses;

non-transitory media storing instructions readable by the cluster system, that when executed by the cluster system, cause the cluster system to;

access configuration information indicating at least one type of operation is non-privileged;

receive, at the master computing device, physical position information corresponding to a position of the first user in the physical space, and

provide, by the master computing device, a first instruction enabling an image corresponding to the first user to be rendered at a first virtual position in a display device associated with a second user, wherein the first virtual position is based at least in part on the physical position information corresponding to the position of the first user in the physical space and on a viewpoint of the second user;

receive, at the master computing device, position data corresponding to the marker; and

at least partly in response to the receipt of the position data corresponding to the marker and on the accessed configuration information, determine that a non-privileged operation is to be performed by the master computing device and not by another computing device associated with the cluster system, wherein the non-privileged operation comprises enabling;

synchronized movement to be rendered, in a head mounted display associated with the first user, of a virtual representation of the marker from a viewpoint of the first user corresponding to received position changes associated with the marker, and

synchronized movement to be rendered, in a head mounted display associated with the second user, of the virtual representation of the marker from a viewpoint of the second user corresponding to received position changes associated with the marker.

2. The cluster system of claim 1, wherein the first user is geographically remote from the second user.

3. The cluster system of claim 1, wherein the marker is configured to be attached to a user limb.

4. The cluster system of claim 1, wherein the marker is attached to or included as a component of a handheld device.

5. The cluster system of claim 1, wherein the cluster system is configured to derive an identity of the marker from a detected repetitive pattern of infrared light pulses.

6. The cluster system of claim 1, wherein the cluster system is configured to enable, via a first user interface, an operator to indicate what types of operations are privileged and what types of operations are non-privileged.

7. The cluster system of claim 1, wherein the cluster system is configured to automatically determine what types of operations are privileged and what types of operations are non-privileged based on detected performance attributes of the cluster system.

8. The cluster system of claim 1, wherein a privileged operation is processed in a local computing context.

9. A cluster system, comprising:

a master computing device;

a first computing device;

a plurality of motion tracking cameras, wherein the plurality of motion tracking cameras positionable in a physical space coincident with a first user, the plurality of motion tracking cameras configured to communicate position data derived from detected infrared light to the master computing device;

a marker, wearable or holdable by the first user, configured to emit a repetitive pattern of infrared light pulses;

non-transitory media storing instructions readable by the cluster system, that when executed by the cluster system, cause the cluster system to;

access configuration information indicating operations that are privileged and operations that are non-privileged;

receive, at the master computing device, position information corresponding to a position of the first user in the physical space, and

provide, by the master computing device, a first instruction enabling an image corresponding to the first user to be rendered at a first virtual position in a display device associated with a second user, wherein the first virtual position is based at least in part on the information corresponding to the position of the first user in the physical space and on a viewpoint of the second user;

receive, at the first client computing device, position data corresponding to the marker; and

at least partly in response to the receipt of the position data corresponding to the marker and on the accessed configuration information, determine that a privileged operation is to be performed by the first computing device and not by another computing device associated with the cluster system, wherein the privileged operation comprises enabling;

synchronized movement to be rendered, in a head mounted display associated with the first user, of a virtual representation of the marker from a viewpoint of the first user corresponding to received position changes associated with the marker; and

synchronized movement to be rendered, in a head mounted display associated with the second user, of the virtual representation of the marker from a viewpoint of the second user corresponding to received position changes associated with the marker.

10. The cluster system of claim 9, wherein the first user is geographically remote from the second user.

11. The cluster system of claim 9, wherein the first computing device is associated with and local to the first user.

12. The cluster system of claim 9, wherein the marker is configured to be attached to a user limb.

13. The cluster system of claim 9, wherein the marker is attached to or included as a component of a handheld device.

14. The cluster system of claim 9, wherein the cluster system is configured to derive an identity of the marker from a detected repetitive pattern of infrared light pulses.

15. The cluster system of claim 9, wherein the cluster system is configured to enable, via a first user interface, an operator to indicate which operations are privileged and which operations are non-privileged.

16. The cluster system of claim 9, wherein the cluster system determines which operations are privileged and which operations are non-privileged.

17. The cluster system of claim 9, wherein a privileged operation is processed in a local computing context.

18. A cluster system, comprising:

a master computing device;

a plurality of motion tracking cameras, wherein the plurality of motion tracking cameras locatable in a physical space coincident with a first user, the plurality of motion tracking cameras configured to detect infrared light and to communicate position data derived from detected infrared light to the master computing device;

non-transitory media storing instructions readable by the cluster system, that when executed by the cluster system, cause the cluster system to;

access configuration information comprising information indicating an operation is non-privileged;

receive, at the master computing device, from one or more of the plurality of motion tracking cameras, position data corresponding to a marker associated with the first user; and

at least partly in response to the receipt of the position data corresponding to the marker and on the accessed configuration information, determine that a non-privileged operation is to be performed by the master computing device wherein the non-privileged operation comprises enabling;

synchronized movement to be rendered in a head mounted display associated with the first user of a virtual representation of the marker from a viewpoint of the first user corresponding to received position changes associated with the marker, and

synchronized movement to be rendered in a head mounted display associated with the second user of the virtual representation of the marker from a viewpoint of the second user corresponding to received position changes associated with the marker.

19. The cluster system of claim 18, the cluster system further configured to receive at the master computing device, a position of the first user in the physical space, and

provide a first instruction, by the master computing device, enabling an image corresponding to the first user to be rendered at a first virtual position in a display device associated with a second user, wherein the first virtual position is based at least in part on the position of the first user in the physical space and on a viewpoint of the second user.