Systems and methods for a plurality of users to interact with each other in augmented or virtual reality systems

• US 10,234,939 B2
• Filed: 05/02/2015
• Issued: 03/19/2019
• Est. Priority Date: 03/11/2013
• Status: Active Grant

• Current Assignee: Magic Leap, Inc.
• Original Assignee: Magic Leap, Inc.
• Inventors: Miller, Samuel A., Abovitz, Rony
• Primary Examiner(s): Patel, Nitin
• Assistant Examiner(s): Stone, Robert M
• Application Number:
  US14/702,710
• Publication Number:
  US 20150234463A1
• Time in Patent Office: 1,417 Days
• CPC Class Codes: G02B 2027/0127 comprising devices increasing the depth of field
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2219/024 Multi-user, collaborative environment
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G09G 2370/02 Networking aspects
• CPC Class Codes: G09G 2370/20 Details of the management of multiple sources of image data
• CPC Class Codes: G09G 5/006 Details of the interface to the display terminal specific for a display terminal using a CRT G09G1/167; using a flat panel G09G3/2096; circuits for interfacing with colour displays G09G5/04
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/131 : Protocols for games, networked simulations or virtual reality

Abstract

One embodiment is directed to a system for enabling two or more users to interact within a virtual world comprising virtual world data, comprising a computer network comprising one or more computing devices, the one or more computing devices comprising memory, processing circuitry, and software stored at least in part in the memory and executable by the processing circuitry to process at least a portion of the virtual world data; wherein at least a first portion of the virtual world data originates from a first user virtual world local to a first user, and wherein the computer network is operable to transmit the first portion to a user device for presentation to a second user, such that the second user may experience the first portion from the location of the second user, such that aspects of the first user virtual world are effectively passed to the second user.

Claims

1. A system for interaction between a first user and a second user within a virtual world comprising virtual world data, the system comprising:

a first head-mounted user display device associated with the first user at a first geographical location;

a second head-mounted user display device associated with the second user at a second geographical location different from the first geographical location;

a processor configured to sense an inanimate real world object at the first geographic location of the first head-mounted user display device, wherein the inanimate real world object at the first geographic location is selected for display by the first user; and

wherein the virtual world data includes at least a virtual representation of the inanimate real world object selected for display by the first user, wherein the inanimate real world object selected for display by the first user sensed at the first geographic location by the first head-mounted user display device appears to be physically present in an augmented reality view of the virtual world displayed to the second user using the second head-mounted user display device, the virtual representation of the inanimate real world object generated as a result of the first head-mounted user display device sensing the inanimate real world object selected for display by the first user to generate a virtual representation of the inanimate real world object as the inanimate real world object appears in the first geographic location,

wherein first head-mounted user display device is operable to display a blended reality view of the virtual world to the first user at the first geographical location based on the virtual world data, and

wherein the second head-mounted user display device is operable to display a virtual reality view of the virtual world to the second user at the second geographical location based on the virtual world data, such that the second user can see the virtual representation of the inanimate real world object that is selected for display by the first user through the second head-mounted user display device at the second geographical location.

2. The system of claim 1, wherein the virtual world changes in response to a change in the virtual world data.

3. The system of claim 1, wherein the virtual world data changes in response to a physical object sensed by the first head-mounted user display device.

4. The system of claim 3, wherein the change in the virtual world data comprises adding virtual object data corresponding to a virtual object in response to the first head-mounted user display device sensing the physical object.

5. The system of claim 4, wherein the change in the virtual world data is presented to the second head-mounted user display device for presentation of the virtual object to the second user.

6. The system of claim 1, wherein at least one of the first head-mounted user display device and the second head-mounted user display device is operable to render the virtual world.

7. The system of claim 1, wherein the virtual world is presented in a two-dimensional format.

8. The system of claim 1, wherein the virtual world is presented in a three-dimensional format.

9. The system of claim 1, wherein the first head-mounted user display device is operable to provide an interface for interaction between the first user and the virtual world in a blended reality mode.

10. The system of claim 9, wherein the second head-mounted user display device is operable to provide an interface for interaction between the second user and the virtual world in a combination of augmented and virtual reality mode.

11. The system of claim 1, wherein the second head-mounted user display device is operable to provide an interface for interaction between the second user and the virtual world in a virtual reality mode.

12. The system of claim 1, further comprising a data network operatively coupling the first head-mounted user display device and the second head-mounted user display device, wherein the virtual world data is transmitted over the data network.

13. The system of claim 1, wherein the virtual world data comprises instructions for generating the virtual world.

14. The system of claim 1, further comprising a gateway, wherein the virtual world data is transmitted to the gateway for at least one of processing or distribution.