System and method for streaming game video

• US 10,369,465 B2
• Filed: 06/05/2017
• Issued: 08/06/2019
• Est. Priority Date: 12/10/2002
• Status: Expired due to Term

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment America LLC (Sony Group Corp.)
• Inventors: Perlman, Stephen G., van der Laan, Roger, Cotter, Timothy, Furman, Scott, McCool, Robert, Buckley, Ian
• Primary Examiner(s): Duffy, David
• Assistant Examiner(s): Doshi, Ankit B
• Application Number:
  US15/614,586
• Publication Number:
  US 20170266550A1
• Time in Patent Office: 792 Days
• CPC Class Codes: A63F 13/12 involving interaction between a plurality of game devices, e.g. transmisison or distribution systems protocols for networked virtual reality, networked games H04L29/06034
• CPC Class Codes: A63F 13/30 Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
• CPC Class Codes: A63F 13/33 using wide area network [WAN] connections
• CPC Class Codes: A63F 13/35 Details of game servers
• CPC Class Codes: A63F 13/355 Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
• CPC Class Codes: A63F 13/358 Adapting the game course according to the network or server load, e.g. for reducing latency due to different connection speeds between clients
• CPC Class Codes: A63F 13/49 Saving the game status; Pausing or ending the game
• CPC Class Codes: A63F 13/493 Resuming a game, e.g. after pausing, malfunction or power failure
• CPC Class Codes: A63F 13/50 Controlling the output signals based on the game progress
• CPC Class Codes: A63F 2300/402 Communication between platforms, i.e. physical link to protocol
• CPC Class Codes: A63F 2300/407 Data transfer via internet
• CPC Class Codes: A63F 2300/534 for network load management, e.g. bandwidth optimization, latency reduction
• CPC Class Codes: A63F 2300/552 for downloading to client devices, e.g. using OS version, hardware or software profile of the client device
• CPC Class Codes: A63F 2300/554 by saving game or status data
• CPC Class Codes: A63F 2300/556 Player lists, e.g. online players, buddy list, black list
• CPC Class Codes: A63F 2300/572 Communication between players during game play of non game information, e.g. e-mail, chat, file transfer, streaming of audio and streaming of video
• CPC Class Codes: A63F 2300/577 for watching a game played by other players
• CPC Class Codes: A63F 2300/636 involving process of starting or resuming a game
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 65/611 : for multicast or broadcast systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06
• CPC Class Codes: H04L 65/752 : adapting media to network capabilities
• CPC Class Codes: H04L 65/765 : intermediate
• CPC Class Codes: H04L 67/131 : Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04N 19/107 : between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/132 : Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/146 : Data rate or code amount at the encoder output
• CPC Class Codes: H04N 19/169 : characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
• CPC Class Codes: H04N 19/172 : the region being a picture, frame or field
• CPC Class Codes: H04N 19/188 : the unit being a video data packet, e.g. a network abstraction layer [NAL] unit
• CPC Class Codes: H04N 19/436 : using parallelised computational arrangements
• CPC Class Codes: H04N 19/61 : in combination with predictive coding
• CPC Class Codes: H04N 21/233 : Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/2343 : involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/2381 : Adapting the multiplex stream to a specific network, e.g. an Internet Protocol [IP] network transmission of MPEG streams over ATM H04L12/5601
• CPC Class Codes: H04N 21/4781 : Games
• CPC Class Codes: H04N 21/6125 : involving transmission via Internet transmission by internet of broadcast information H04H60/82
• CPC Class Codes: H04N 21/6377 : directed to server one-way streaming services wherein the source is controlled by the destination H04L65/613
• CPC Class Codes: H04N 21/6405 : Multicasting data broadcast and multicast in packet switching networks H04L12/18
• CPC Class Codes: H04N 21/658 : Transmission by the client directed to the server
• CPC Class Codes: H04N 21/6587 : Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 7/106 : for domestic distribution

Abstract

A system and method are described for managing the state of an online video game. A method includes initiating a new online video game in response to user input from a client device, the online video game being in a first state on a first server when initiated; executing the online video game on the server, thereby causing the online video game to enter a second state; pausing or terminating the online video game; determining differences between the first state and the second state and generating difference data containing the differences; transmitting the difference data over a network to a second server; and recreating the second state from the difference data and the first state in response to user input indicating that the user wishes to resume the online video game and in response to the second server being selected as the server on which to execute the video game.

Claims

1. A server system, comprising:

a processor configured to execute a game application for simultaneous play by multiple users in accordance with signals received from multiple client computing devices respectively corresponding to the multiple users to generate multiple video streams of graphical output of the executed game application, the multiple video streams of graphical output of the executed game application respectively corresponding to play of the game application by the multiple users,

wherein the processor is configured to generate multiple user interfaces for respective transmission to the multiple client computing devices, each of the multiple user interfaces including separate video windows, the separate video windows in any one of the multiple user interfaces respectively displaying the multiple video streams of graphical output of the executed game application, each of the multiple user interfaces configured for transmission as a single user interface video stream to a respective one of the multiple client computing devices in lieu of separate transmission of the multiple video streams of graphical output of the executed game application to the respective one of the multiple client computing devices,

wherein at least one of the separate video windows in each of the multiple user interfaces is an interaction-selectable video window that is selectable to enable user interaction within the game application executing to generate the video stream of graphical output displayed within the interaction-selectable video window;

a network connection configured to connect the server system to a network over which data is transmittable to the multiple client computing devices; and

an encoder configured to compress each user interface video stream into one or more encoded formats for transmission through the network connection to corresponding ones of the multiple client computing devices.

2. The server system as recited in claim 1, wherein at least one of the separate video windows in a given user interface video stream includes display of a video capture of a user of a client computing device to which the given user interface video stream is transmitted.

3. The server system as recited in claim 1, wherein some of the multiple video streams of graphical output of the executed game application correspond to live play of the game application.

4. The server system as recited in claim 1, wherein some of the multiple video streams of graphical output of the executed game application correspond to a time-delayed presentation of live play of the game application.

5. The server system as recited in claim 1, wherein some of the multiple video streams of graphical output of the executed game application correspond to a view of a user playing the game application.

6. The server system as recited in claim 1, wherein some of the multiple video streams of graphical output of the executed game application correspond to a view that is different from a view of a user playing the game application.

7. The server system as recited in claim 1, wherein the processor is configured to generate a selection interface to enable selection of the multiple users corresponding to the multiple video streams of graphical output of the executed game application, the selection interface configured for transmission as a selection interface video stream to one or more of the multiple client computing devices, the encoder configured to compress the selection interface video stream into one or more encoded formats for transmission through the network connection to the one or more of the multiple client computing devices.

8. The server system as recited in claim 1, wherein the processor is configured to generate a selection interface to enable selection of a ranking to filter the multiple users corresponding to the multiple video streams of graphical output of the executed game application, the selection interface configured for transmission as a selection interface video stream to one or more of the multiple client computing devices, the encoder configured to compress the selection interface video stream into one or more encoded formats for transmission through the network connection to the one or more of the multiple client computing devices.

9. The server system as recited in claim 1, wherein the processor is configured to generate a selection interface to enable selection of a level of play to filter the multiple users corresponding to the multiple video streams of graphical output of the executed game application, the selection interface configured for transmission as a selection interface video stream to one or more of the multiple client computing devices, the encoder configured to compress the selection interface video stream into one or more encoded formats for transmission through the network connection to the one or more of the multiple client computing devices.

10. The server system as recited in claim 1, wherein the processor is configured to generate a selection interface to enable selection of a relationship among users to filter the multiple users corresponding to the multiple video streams of graphical output of the executed game application, the selection interface configured for transmission as a selection interface video stream to one or more of the multiple client computing devices, the encoder configured to compress the selection interface video stream into one or more encoded formats for transmission through the network connection to the one or more of the multiple client computing devices.

11. The server system as recited in claim 1, wherein the processor is configured to display the multiple video streams of graphical output of the executed game application within the separate video windows in accordance with display settings received from the multiple users.

12. The server system as recited in claim 1, wherein the processor is configured to scale the multiple video streams of graphical output of the executed game application to fit within the separate video windows of each of the multiple user interfaces.

13. The server system as recited in claim 1, wherein at least one of the separate video windows in each of the multiple user interfaces is an audio-selectable video window that is selectable to enable transmission of audio corresponding to the video stream of graphical output displayed within the audio-selectable video window.

14. The server system as recited in claim 1, wherein the processor is configured to modify a given one of the multiple user interfaces to show a zoomed-up view of a given video window upon selection of the given video window at one of the multiple client computing devices corresponding to the given one of the multiple user interfaces.

15. The server system as recited in claim 1, wherein upon selection of a given video window in a given user interface at a given one of the multiple client computing devices, the processor is configured to substitute a scaled-up version of the given video window in place of the given user interface transmitted in the single user interface video stream to the given one of the multiple client computing devices.

16. A method, comprising:

executing a game application for simultaneous play by multiple users in accordance with signals received from multiple client computing devices respectively corresponding to the multiple users to generate multiple video streams of graphical output of the executed game application, the multiple video streams of graphical output of the executed game application respectively corresponding to play of the game application by the multiple users;

generating multiple user interfaces for respective transmission to the multiple client computing devices, each of the multiple user interfaces including separate video windows, the separate video windows in any one of the multiple user interfaces respectively displaying the multiple video streams of graphical output of the executed game application, each of the multiple user interfaces configured for transmission as a single user interface video stream to a respective one of the multiple client computing devices in lieu of separate transmission of the multiple video streams of graphical output of the executed game application to the respective one of the multiple client computing devices,

wherein at least one of the separate video windows in each of the multiple user interfaces is an interaction-selectable video window that is selectable to enable user interaction within the game application executing to generate the video stream of graphical output displayed within the interaction-selectable video window;

compressing each user interface video stream into one or more encoded formats for transmission through a network connection to corresponding ones of the multiple client computing devices; and

transmitting each user interface video stream through the network connection to corresponding ones of the multiple client computing devices.

17. The method as recited in claim 16, wherein at least one of the separate video windows in a given user interface video stream includes display of a video capture of a user of a client computing device to which the given user interface video stream is transmitted.

18. The method as recited in claim 16, further comprising:

generating a selection interface to enable selection of the multiple users corresponding to the multiple video streams of graphical output of the executed game application;

configuring the selection interface for transmission as a selection interface video stream to one or more of the multiple client computing devices;

compressing the selection interface video stream into one or more encoded formats for transmission through the network connection to the one or more of the multiple client computing devices; and

transmitting each selection interface video stream through the network connection to the one or more of the multiple client computing devices.

19. The method as recited in claim 16, further comprising:

generating a selection interface to enable selection of a ranking to filter the multiple users corresponding to the multiple video streams of graphical output of the executed game application;

configuring the selection interface for transmission as a selection interface video stream to one or more of the multiple client computing devices;

compressing the selection interface video stream into one or more encoded formats for transmission through the network connection to the one or more of the multiple client computing devices; and

transmitting each selection interface video stream through the network connection to the one or more of the multiple client computing devices.

20. The method as recited in claim 16, further comprising:

generating a selection interface to enable selection of a level of play to filter the multiple users corresponding to the multiple video streams of graphical output of the executed game application;

configuring the selection interface for transmission as a selection interface video stream to one or more of the multiple client computing devices;

compressing the selection interface video stream into one or more encoded formats for transmission through the network connection to the one or more of the multiple client computing devices; and

transmitting each selection interface video stream through the network connection to the one or more of the multiple client computing devices.

21. The method as recited in claim 16, further comprising:

generating a selection interface to enable selection of a relationship among users to filter the multiple users corresponding to the multiple video streams of graphical output of the executed game application;

configuring the selection interface for transmission as a selection interface video stream to one or more of the multiple client computing devices;

compressing the selection interface video stream into one or more encoded formats for transmission through the network connection to the one or more of the multiple client computing devices; and

transmitting each selection interface video stream through the network connection to the one or more of the multiple client computing devices.