System and Method for Combining Multiple Game or Application Views Into a Single Media Stream

• US 20170304724A1
• Filed: 07/10/2017
• Published: 10/26/2017
• Est. Priority Date: 03/12/2013
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment America LLC (Sony Group Corp.)
• Inventors: Cotter, Timothy
• Granted Patent: US 11,273,373 B2
• Time in Patent Office: Days
• CPC Class Codes: A63F 13/355 Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
• CPC Class Codes: A63F 13/795 for finding other players; for building a team; for providing a buddy list
• CPC Class Codes: A63F 13/86 Watching games played by other players
• CPC Class Codes: A63F 13/87 Communicating with other players during game play, e.g. by e-mail or chat
• CPC Class Codes: H04N 21/2365 Multiplexing of several video streams

Abstract

A system and method are described for combining multiple views of a video game or application into a single video stream (or set of video streams). For example, a system according to one embodiment comprises: a plurality of application/video game instances executed for a particular multi-player video game or application on a hosting service, the application/video game instances being executed in response to control signals received from a plurality of clients; compositing logic executed at the hosting service to combine video streams generated by each of the application/video game instances into a single video stream for a first one of the clients; and compression logic to compress the single video stream with low latency such that the user of the first one of the clients has the perception that the video game or application is being executed locally.

Claims

1. A system including a hosting service with a plurality of servers for executing application/video game instances for remote clients, comprising:

a plurality of application/video game instances for a particular multi-player video game or application on the hosting service, each of the application/video game instances being executed in response to control signals received from each of a plurality of remote clients, each remote client receiving a video stream to be presented on a screen associated with the remote client;

compositing logic executed at the hosting service to combine video streams generated by each of the application/video game instances into a single video stream for a first one of the remote clients, the first one of the remote clients being provided with options to view the video stream for the application/video game instance being played or watched via the first one of the remote clients or viewing the combined video streams being generated for the particular multi-player video game or application being executed by the servers of the hosting service; and

compression logic to compress the single video stream with low latency such that a user of the first one of the remote clients has a perception that the video game or application is being executed locally.

2. The system as in claim 1 wherein the compositing logic generates a separate view stream for each remote client, each video stream having a primary view generated from a perspective of a user of the remote client and a plurality of secondary views generated from a perspectives of the other users participating in the multi-player video game or application.

3. The system as in claim 2 wherein at least some of the remote clients receive video streams having a single view and other of the remote clients receive view streams having multiple-views generated by the compositing logic.

4. The system as in claim 1 wherein the compression logic compresses the video stream in response to feedback received indicating a current state of a communication channel.

5. The system as in claim 2 wherein the views combined by the compositing logic are consistent for the view streams generated for each of the remote clients.

6. The system as in claim 2 wherein each of the remote client devices has different view processing capabilities.

7. The system as in claim 6 wherein at least some of the remote client devices are touch-screen tablet devices.

8. The system as in claim 6 wherein at least some of the remote client devices are smartphone devices.

9. A method executed by a hosting service with a plurality of servers for executing application/video game instances for remote clients, comprising:

executing, by a server, a plurality of application/video game instances for a particular multi-player video game or application on the hosting service, the application/video game instances being executed in response to control signals received from a plurality of remote clients, each remote client receiving a video stream to be presented on a screen associated with the remote client;

combining video streams generated by each of the application/video game instances into a single video stream for a first one of the remote clients, the first one of the remote clients being provided with an option to view the video stream for the application/video game instance being played or watched via the first one of the remote clients or viewing the combined video streams being generated for the particular multi-player video game or application being executed by the servers of the hosting service; and

compressing the single video stream with low latency such that a user of the first one of the remote clients has a perception that the video game or application is being executed locally.

10. The method as in claim 9 further comprising generating a separate view stream for each remote client, each video stream having a primary view generated from a perspective of a user of the remote client and a plurality of secondary views generated from a perspectives of the other users participating in the multi-player video game or application.

11. The method as in claim 10 wherein at least some of the remote clients receive video streams having a single view and other of the remote clients receive view streams having multiple-views generated by the compositing logic.

12. The method as in claim 9 wherein the compression logic compresses the video stream in response to feedback received indicating a current state of a communication channel.

13. The method as in claim 10 wherein the views combined by the compositing logic are consistent for the view streams generated for each of the remote clients.

14. The method as in claim 10 wherein each of the remote client devices has different view processing capabilities.

15. The method as in claim 14 wherein at least one of the remote client devices is touch-screen tablet devices.

16. The method as in claim 14 wherein at least one of the remote client devices is a smartphone device.