Collaborative multi-user virtual reality

• US 10,430,147 B2
• Filed: 04/17/2017
• Issued: 10/01/2019
• Est. Priority Date: 04/17/2017
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Vembar, Deepak S., Kuwahara, Atsuo, Sakthivel, Chandrasekaran, Venkataraman, Radhakrishnan, Insko, Brent E., Kalra, Anupreet S., Labbe, Hugues, Koker, Altug, Apodaca, Michael, Xiao, Kai, Boles, Jeffery S., Lake, Adam T., Cimini, David M., Vembu, Balaji, Ould-Ahmed-Vall, Elmoustapha, Kwiatkowski, Jacek, Laws, Philip R., Shah, Ankur N., Appu, Abhishek R., Ray, Joydeep, Fu, Wenyin, Kaburlasos, Nikos, Surti, Prasoonkumar, Borole, Bhushan M.
• Primary Examiner(s): Gray, Ryan M
• Application Number:
  US15/489,096
• Publication Number:
  US 20180300098A1
• Time in Patent Office: 897 Days
• Field of Search: None
• CPC Class Codes: G06F 3/1431 using a single graphics controller
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G09G 2340/02 Handling of images in compressed format, e.g. JPEG, MPEG
• CPC Class Codes: G09G 2350/00 Solving problems of bandwidth in display systems
• CPC Class Codes: G09G 2352/00 Parallel handling of streams of display data
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 2360/08 Power processing, i.e. workload management for processors involved in display operations, such as CPUs or GPUs
• CPC Class Codes: G09G 2360/121 using a cache memory
• CPC Class Codes: G09G 2360/125 using unified memory architecture [UMA]
• CPC Class Codes: G09G 2370/02 Networking aspects
• CPC Class Codes: G09G 3/003 to produce spatial visual effects
• CPC Class Codes: G09G 5/363 Graphics controllers
• CPC Class Codes: H04L 65/4025 where none of the additional parallel sessions is real time or time sensitive, e.g. downloading a file in a parallel FTP session, initiating an email or combinational services
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality

Abstract

An embodiment of a graphics apparatus may include a processor, memory communicatively coupled to the processor, and a collaboration engine communicatively coupled to the processor to identify a shared graphics component between two or more users in an environment, and share the shared graphics components with the two or more users in the environment. Embodiments of the collaboration engine may include one or more of a centralized sharer, a depth sharer, a shared preprocessor, a multi-port graphics subsystem, and a decode sharer. Other embodiments are disclosed and claimed.

Claims

1. A graphics apparatus, comprising:

a processor;

memory communicatively coupled to the processor; and

a collaboration engine communicatively coupled to the processor to;

determine one or more of a viewport difference between first and second viewports or an orientation difference between first and second orientations, wherein the first viewport and the first orientation are associated with a first user of two or more users, and the second orientation and the second viewport are associated with a second user of the two or more users;

if the one or more of the viewport difference or the orientation difference is under a threshold, identify that a displayable object is to be part of a shared graphics component, wherein the displayable object is to be an augmented reality object or a virtual reality object, wherein the displayable object is to be rendered for the first user and the second user at a same orientation and a same position;

if the one or more of the viewport difference or the orientation difference is over the threshold, identify that the displayable object is not to be part of the shared graphics component so that the displayable object is to be rendered for the first user and the second user at one or more of different orientations or different positions;

generate the shared graphics component; and

share the shared graphics component with the first and second users.

2. The apparatus of claim 1, wherein the collaboration engine is further to:

identify that the one or more of the viewport difference or the orientation difference is over the threshold;

generate a first separate graphics component that includes the displayable object at a first display orientation and a first display position;

generate a second separate graphics component that includes the displayable object at a second display orientation and a second display position, wherein the first display orientation is different from the second display orientation and the first display position is different from the second display position;

broadcast the shared graphics component to all of the two or more users; and

distribute the first and second separate graphics components separately to the first and second users respectively.

3. The apparatus of claim 2, wherein the collaboration engine is further to:

split a workload for the shared graphics component based on a target virtual reality apparatus.

4. The apparatus of claim 1, wherein the collaboration engine further includes a depth sharer to:

collect depth information from an independent source independent of the first user; and

share the depth information with the first user.

5. The apparatus of claim 4, wherein the depth sharer is further to:

collect visibility information from the independent source which is outside a field of view of the first user; and

share the visibility information with the first user.

6. The apparatus of claim 1, wherein the collaboration engine further includes a shared preprocessor to:

pre-compute information related to the shared graphics component; and

share the pre-computed information with the two or more users.

7. The apparatus of claim 6, wherein the pre-computed information includes one or more of geometry information, primitive information, and physics information.

8. The apparatus of claim 1, wherein the collaboration engine further includes a multi-port graphics subsystem to:

support a different user on each port of the multi-port graphics subsystem.

9. The apparatus of claim 8, wherein the multi-port graphics subsystem is further to:

support a left eye display and a right eye display of a first virtual reality apparatus when only the first virtual reality apparatus is connected to the multi-port graphics subsystem;

support the first virtual reality apparatus and a second virtual reality apparatus when both the first virtual reality apparatus and the second virtual reality apparatus are connected to the multi-port graphics subsystem; and

share graphics primitives between the first virtual reality apparatus and the second virtual reality apparatus when both the first virtual reality apparatus and the second virtual reality apparatus are connected to the multi-port graphics subsystem.

10. The apparatus of claim 1, wherein the collaboration engine further includes a decode sharer to:

identify an overlapped region of a shared scene between a first view region of the first user and a second view region of the second user;

decode the overlapped region; and

share the decoded overlapped region with both the first user and the second user.

11. The apparatus of claim 10, wherein the decode sharer is further to:

prioritize the decode based on the overlapped region.

12. A method of graphics collaborating, comprising:

determining one or more of a viewport difference between first and second viewports or an orientation difference between first and second orientations, wherein the first viewport and the first orientation are associated with a first user of two or more users, and the second orientation and the second viewport are associated with a second user of the two or more users;

if the one or more of the viewport difference or the orientation difference is under a threshold, identifying that a displayable object is part of a shared graphics component, wherein the displayable object is an augmented reality object or a virtual reality object, wherein the displayable object is rendered for the first user and the second user at a same orientation and a same position;

if the one or more of the viewport difference or the orientation difference is over the threshold, identifying that the displayable object is not part of the shared graphics component so that the displayable object is to be rendered for the first user and the second user at one or more of different orientations or different positions;

generating the shared graphics component and

sharing the shared graphics component with the first and second users.

13. The method of claim 12, further comprising:

identifying that the one or more of the viewport difference or the orientation difference is over the threshold;

generating a first separate graphics component that includes the displayable object at a first display orientation and a first display position;

generating a second separate graphics component that includes the displayable object at a second display orientation and a second display position, wherein the first display orientation is different from the second display orientation and the first display position is different from the second display position;

broadcasting the shared graphics component to all of the two or more users; and

distributing the first and second separate graphics components separately to the first and second users respectively.

14. The method of claim 13, further comprising:

splitting a workload for the shared graphics component based on a target virtual reality apparatus.

15. The method of claim 12, further comprising:

collecting depth information from an independent source independent of the first user; and

sharing the depth information with the first user.

16. The method of claim 15, further comprising:

collecting visibility information from the independent source which is outside a field of view of the first user; and

sharing the visibility information with the first user.

17. The method of claim 12, further comprising:

pre-computing information related to the shared graphics component; and

sharing the pre-computed information with the two or more users.

18. The method of claim 17, wherein the pre-computed information includes one or more of geometry information, primitive information, and physics information.

19. The method of claim 12, further comprising:

supporting a different user on each port of a multi-port graphics subsystem.

20. The method of claim 19, further comprising:

supporting a left eye display and a right eye display of a first virtual reality apparatus when only the first virtual reality apparatus is connected to the multi-port graphics subsystem;

supporting the first virtual reality apparatus and a second virtual reality apparatus when both the first virtual reality apparatus and the second virtual reality apparatus are connected to the multi-port graphics subsystem; and

sharing graphics primitives between the first virtual reality apparatus and the second virtual reality apparatus when both the first virtual reality apparatus and the second virtual reality apparatus are connected to the multi-port graphics subsystem.

21. The method of claim 12 further comprising:

identifying an overlapped region of a shared scene between a first view region of the first user and a second view region of the second user;

decoding the overlapped region; and

sharing the decoded overlapped region with both the first user and the second user.

22. The method of claim 21, further comprising:

prioritizing the decoding based on the overlapped region.

23. At least one non-transitory computer readable medium, comprising a set of instructions, which when executed by a computing device cause the computing device to:

determine one or more of a viewport difference between first and second viewports or an orientation difference between first and second orientations, wherein the first viewport and the first orientation are associated with a first user of two or more users, and the second orientation and the second viewport are associated with a second user of the two or more users;

if the one or more of the viewport difference or the orientation difference is under a threshold, identify that a displayable object is to be part of a shared graphics component, wherein the displayable object is to be an augmented reality object or a virtual reality object, wherein the displayable object is to be rendered for the first user and the second user at a same orientation and a same position;

if the one or more of the viewport difference or the orientation difference is over the threshold, identify that the displayable object is not to be part of the shared graphics component so that the displayable object is to be rendered for the first user and the second user at one or more of different orientations or different positions;

generate the shared graphics component and

share the shared graphics component with the first and second users.

24. The at least one non-transitory computer readable medium of claim 23, comprising a further set of instructions, which when executed by a computing device cause the computing device to:

identify that the one or more of the viewport difference or the orientation difference is over the threshold;

generate a first separate graphics component that is to include the displayable object at a first display orientation and a first display position;

generate a second separate graphics component that is to include the displayable object at a second display orientation and a second display position, wherein the first display orientation is different from the second display orientation and the first display position is different from the second display position;

broadcast the shared graphics component to all of the two or more users; and

distribute the first and second separate graphics components separately to the first and second users respectively.

25. The at least one non-transitory computer readable medium of claim 24, comprising a further set of instructions, which when executed by a computing device cause the computing device to:

split a workload for the shared graphics component based on a target virtual reality apparatus.