Massive simultaneous remote digital presence world

• US 10,671,152 B2
• Filed: 08/07/2018
• Issued: 06/02/2020
• Est. Priority Date: 05/06/2011
• Status: Active Grant

• Current Assignee: Magic Leap, Inc.
• Original Assignee: Magic Leap, Inc.
• Inventors: Abovitz, Rony
• Primary Examiner(s): Klicos, Nicholas
• Application Number:
  US16/057,518
• Publication Number:
  US 20180348856A1
• Time in Patent Office: 665 Days
• CPC Class Codes: A63F 2300/10 characterized by input arrangements for converting player-generated signals into game device control signals
• CPC Class Codes: A63F 2300/8082 Virtual reality
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 2340/125 wherein one of the images is motion video
• CPC Class Codes: G09G 2340/14 Solving problems related to the presentation of information to be displayed
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user

Abstract

Various methods and apparatus are described herein for enabling one or more users to interface with virtual or augmented reality environments. An example system includes a computing network having computer servers interconnected through high bandwidth interfaces to gateways for processing data and/or for enabling communication of data between the servers and one or more local user interface devices. The servers include memory, processing circuitry, and software for designing and/or controlling virtual worlds, as well as for storing and processing user data and data provided by other components of the system. One or more virtual worlds may be presented to a user through a user device for the user to experience and interact. A large number of users may each use a device to simultaneously interface with one or more digital worlds by using the device to observe and interact with each other and with objects produced within the digital worlds.

Claims

1. A system for presenting virtual content, the system comprising:

a wearable user device configured to display virtual content,

a display operatively coupled to the wearable user device and configured to present the virtual content to a user,

memory,

processing circuitry,

software stored in the memory and executable by the processing circuitry to render the virtual content for presentation by the display,

a gateway operatively coupled to, and distinct from, the wearable user device and configured to monitor and regulate data representing the rendered virtual content to be displayed by the software; and

a user sensing system connected to the wearable user device comprising a camera configured to detect an angular measurement of at least one eye pupil of the user,

wherein the gateway monitors and regulates the display of the data representing the rendered virtual content to allow an optimum data processing of the wearable user device based on the detected head pose or angular measurement, wherein the system is configured such that at least a portion of the virtual world changes in response to a change in the virtual world data,

wherein, in conjunction with the virtual world changes in response to a change in the virtual world data, at least a portion of the virtual world data is changed in response to a static physical object external to the user and sensed by the wearable user device,

wherein the static physical object external to the user comprises a mapped object in a physical environment in vicinity of the first user,

wherein the optimum data processing comprises prioritizing a plurality of renderings, such that processing a rendering of a dynamic virtual object is prioritized over a rendering of a static virtual object and processing a rendering of data in a field of view having less than sixty degrees forward of the angular measurement of the user'"'"'s pupil is prioritized over a rendering of data outside the field of view, where the plurality of renderings are performed by the gateway device and transmitted to the wearable user device, and

wherein the change in virtual world data represents rendering at least one of the dynamic virtual object and the static virtual object with the static physical object external to the user according to a predetermined relationship.

2. The system of claim 1, wherein the optimum data processing is determined by a type of the virtual content data.

3. The system of claim 2, wherein the type is a dynamic virtual object and the gateway transmits the virtual content over a high priority data stream.

4. The system of claim 2, wherein the type is a static object and the gateway stores the virtual content in a local memory buffer.

5. The system of claim 1, wherein the optimum data processing is determined by a location of the virtual content within a user'"'"'s field of view.

6. The system of claim 5, wherein the location is a central portion of the user'"'"'s field of view and the gateway transmits virtual world data over a high priority data stream.

7. The system of claim 5, wherein the location is a periphery of the user'"'"'s field of view and the gateway transmits virtual world data to a local memory buffer.

8. The system according to claim 1, wherein the change in virtual world data is presented to a second user device for presentation to a second user according to the predetermined relationship.

9. The system according to claim 1, wherein the virtual world is operable to be rendered by at least one of the computer servers or the wearable user device.

10. The system according to claim 1, wherein the virtual world is presented in at least one of a two-dimensional format or three-dimensional format.

11. The system according to claim 1, wherein the wearable user device is operable to provide an interface for enabling interaction between a user and the virtual world in at least one of an augmented reality mode, a virtual reality mode, or a combination of augmented and virtual reality mode.

12. The system according to claim 1, wherein the virtual world data is transmitted over a data network.

13. The system according to claim 1, wherein the computer network is operable to receive at least a portion of the virtual world data from the wearable user device.

14. The system according to claim 1, wherein at least a portion of the virtual world data transmitted to the wearable user device comprises instructions for generating at least a portion of the virtual world.

15. The system of claim 1, wherein the wearable device is configured to be a head-worn device.

16. The system of claim 1, further comprising an environment-sensing system coupled to the wearable user device configured to obtain data from the physical environment around a user.

17. The system of claim 16, wherein the environment-sensing system detects key features of the static physical object external to the user.

18. The system of claim 16, wherein the computer network receives data from the environment-sensing system to trigger a transmission of the virtual world data to the wearable user device.