SYSTEMS AND METHODS FOR A SHARED MIXED REALITY EXPERIENCE

• US 20160104452A1
• Filed: 05/23/2014
• Published: 04/14/2016
• Est. Priority Date: 05/24/2013
• Status: Active Grant

• Current Assignee: AWE Company Limited, Ryerson University
• Original Assignee: AWE Company Limited, Reyerson University
• Inventors: Guan, Ling, Kyan, Matthew, Krishna, Srinivas, Bond, Jason Raymond Thomas, Dong, Nan, Khan, Naimul Mefraz, Nan, Xiaoming, He, Yifeng, Biggs, Edward
• Granted Patent: US 9,940,897 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/35 Details of game servers
• CPC Class Codes: A63F 13/65 automatically by game devices or servers from real world data, e.g. measurement in live racing competition
• CPC Class Codes: A63F 13/847 Cooperative playing, e.g. requiring coordinated actions from several players to achieve a common goal
• CPC Class Codes: A63F 2300/6661 for changing the position of the virtual camera
• CPC Class Codes: G01S 5/0284 Relative positioning
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/1423 controlling a plurality of local displays, e.g. CRT and flat panel display
• CPC Class Codes: G06F 3/165 Management of the audio stream, e.g. setting of volume, audio stream path
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G09G 5/006 Details of the interface to the display terminal specific for a display terminal using a CRT G09G1/167; using a flat panel G09G3/2096; circuits for interfacing with colour displays G09G5/04

Abstract

A method for sharing a mixed reality experience (mixed reality content, mixed reality event) between one or more computing devices is disclosed. The method includes: determining a spatial location and a spatial orientation (spatial data) of the one or more computing devices each having a camera; mapping the (spatial) location and/or the spatial orientation (spatial data) of each of the one or more computing devices into a mixed reality manager; and presenting an event that is shared among the one or more computing devices, and, the presenting of the event is experienced simultaneously and varies among each of the one or more computing devices depending on the location or the orientation or both.

Claims

1. A method for sharing a mixed reality between one or more computing devices, comprising:

determining a location and an orientation of a camera input of the one or more computing devices;

mapping the location and the orientation of each of the one or more computing devices into a mixed reality manager; and

presenting an event that is shared among the one or more computing devices, and, the presenting of the event is experienced simultaneously and varies among each of the one or more computing devices depending on the location or the orientation or both.

2. The method of claim 1, wherein:

the step of presenting further comprises;

registering the camera input with the event for aligning a real world view from the camera input with a virtual world view of the event.

3. The method of claim 1, wherein:

the step of presenting further comprises;

determining occlusion between one or more real objects in a real world view from a camera providing the camera input and one or more virtual objects in a virtual world view of the event.

4. The method of claim 1, wherein:

the step of presenting further comprises;

matching lighting between a real world view from the camera input and a virtual world view from the event.

5. The method of claim 1, wherein:

the event is an interactive performance.

6. The method of claim 1, wherein:

the one or more computing devices comprise one or more input devices for interacting with the event.

7. The method of claim 1, further comprising:

performing an audio content associated with the virtual world content.

8. The method of claim 1, further comprising:

performing an audio content.

9. The method of claim 1, further comprising:

determining spatial data of a device having a real world camera; and

aligning a virtual world camera in a virtual world with the real world camera using the spatial data of the device so that a point of view of the virtual world camera in the virtual world is aligned with a point of view of the real world camera.

10. A system for sharing a mixed reality between one or more computing devices, each of the one or more computing devices providing an output, a camera input, a location, and an orientation of the camera input, the system comprising:

a mixed reality manager for coordinating an event among the one or more computing devices based on the location and a point of view for each of the one or more computing devices; and

a mixed reality renderer for rendering the event simultaneously on the output of the one or more computing devices, and the rendering of the event varies among each of the one or more computing devices depending on the location or the point of view or both.

11. The system of claim 10, wherein:

the mixed reality renderer is further configured to;

register the camera input with the event for aligning a real world view from the camera input with a virtual world view of the event.

12. The system of claim 10, wherein:

the mixed reality renderer is further configured to;

determine occlusion between one or more real world objects in a real world view and one or more virtual world objects in a virtual world view.

13. The system of claim 10, wherein:

the mixed reality renderer is further configured to;

match lighting between a real world view from the camera input and a virtual world view from the event.

14. The system of claim 10, wherein:

the event is an interactive performance.

15. The system of claim 10, wherein:

the one or more computing devices comprise one or more input devices for interacting with the event.

16. The system of claim 10, wherein:

a device is configured to perform an audio content associated with the virtual world content.

17. The system of claim 10, wherein:

a device is configured to perform an audio content.

18. The system of claim 10, further comprising:

a tracking assembly configured to determine spatial data of a device having a real world camera; and

an experience system configured to aligning a virtual world camera in a virtual world with the real world camera using the spatial data of the device so that a point of view of the virtual world camera in the virtual world is aligned with a point of view of the real world camera.

19. A system for sharing a mixed reality between mixed reality devices, each of the mixed reality devices having a real world camera, the system comprising:

a mixed reality manager being configured to coordinate display of a mixed-reality event having a virtual world content and a real world content among the mixed reality devices based on a point of view for each of the mixed reality devices; and

a mixed reality renderer being configured to render the mixed-reality event among the mixed reality devices, in which rendition of the mixed-reality event is experienced simultaneously via the mixed reality devices, and the point of view of the mixed-reality event that is rendered on a selected mixed reality device varies from the point of view of the mixed-reality event that is rendered on another selected mixed reality device depending on spatial data of the selected mixed reality device.

20. A method for sharing a mixed reality event between mixed reality devices, comprising:

determining a spatial data of the mixed reality devices;

mapping the spatial data of each of the mixed reality devices into a mixed reality manager; and

presenting a mixed-reality event having virtual world content and real word content that is shared among the mixed reality devices, in which presentation of the mixed-reality event is experienced simultaneously via the mixed reality devices, and a point of view of the mixed reality event that is experienced on a selected mixed reality device varies from the point of view of the mixed reality event that is experienced by another selected mixed reality device depending on the spatial data of the selected mixed reality device.

21. The method of claim 20, wherein:

the spatial data of the selected mixed reality device includes any one of;

(A) spatial location, (B) spatial orientation, and (C) the spatial location and the spatial orientation.