System and method for accelerated machine switching

• US 9,138,644 B2
• Filed: 08/07/2009
• Issued: 09/22/2015
• Est. Priority Date: 12/10/2002
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment America LLC (Sony Group Corp.)
• Inventors: Perlman, Stephen G., van der Laan, Roger, Cotter, Timothy, Furman, Scott, McCool, Robert, Buckley, Ian
• Primary Examiner(s): D'Agostino, Paul A.
• Assistant Examiner(s): DOSHI, ANKIT B
• Application Number:
  US12/538,077
• Publication Number:
  US 20100167809A1
• Time in Patent Office: 2,237 Days
• Field of Search: 463/42, 463/43
• US Class Current: 1/1
• CPC Class Codes: A63F 13/12 involving interaction between a plurality of game devices, e.g. transmisison or distribution systems protocols for networked virtual reality, networked games H04L29/06034
• CPC Class Codes: A63F 13/30 Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
• CPC Class Codes: A63F 13/33 using wide area network [WAN] connections
• CPC Class Codes: A63F 13/35 Details of game servers
• CPC Class Codes: A63F 13/355 Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
• CPC Class Codes: A63F 13/358 Adapting the game course according to the network or server load, e.g. for reducing latency due to different connection speeds between clients
• CPC Class Codes: A63F 13/49 Saving the game status; Pausing or ending the game
• CPC Class Codes: A63F 13/493 Resuming a game, e.g. after pausing, malfunction or power failure
• CPC Class Codes: A63F 13/50 Controlling the output signals based on the game progress
• CPC Class Codes: A63F 2300/402 Communication between platforms, i.e. physical link to protocol
• CPC Class Codes: A63F 2300/407 Data transfer via internet
• CPC Class Codes: A63F 2300/534 for network load management, e.g. bandwidth optimization, latency reduction
• CPC Class Codes: A63F 2300/552 for downloading to client devices, e.g. using OS version, hardware or software profile of the client device
• CPC Class Codes: A63F 2300/554 by saving game or status data
• CPC Class Codes: A63F 2300/556 Player lists, e.g. online players, buddy list, black list
• CPC Class Codes: A63F 2300/572 Communication between players during game play of non game information, e.g. e-mail, chat, file transfer, streaming of audio and streaming of video
• CPC Class Codes: A63F 2300/577 for watching a game played by other players
• CPC Class Codes: A63F 2300/636 involving process of starting or resuming a game
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 65/611 : for multicast or broadcast systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06
• CPC Class Codes: H04L 65/752 : adapting media to network capabilities
• CPC Class Codes: H04L 65/765 : intermediate
• CPC Class Codes: H04L 67/131 : Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04N 19/107 : between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/132 : Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/146 : Data rate or code amount at the encoder output
• CPC Class Codes: H04N 19/169 : characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
• CPC Class Codes: H04N 19/172 : the region being a picture, frame or field
• CPC Class Codes: H04N 19/188 : the unit being a video data packet, e.g. a network abstraction layer [NAL] unit
• CPC Class Codes: H04N 19/436 : using parallelised computational arrangements
• CPC Class Codes: H04N 19/61 : in combination with predictive coding
• CPC Class Codes: H04N 21/233 : Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/2343 : involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/2381 : Adapting the multiplex stream to a specific network, e.g. an Internet Protocol [IP] network transmission of MPEG streams over ATM H04L12/5601
• CPC Class Codes: H04N 21/4781 : Games
• CPC Class Codes: H04N 21/6125 : involving transmission via Internet transmission by internet of broadcast information H04H60/82
• CPC Class Codes: H04N 21/6377 : directed to server one-way streaming services wherein the source is controlled by the destination H04L65/613
• CPC Class Codes: H04N 21/6405 : Multicasting data broadcast and multicast in packet switching networks H04L12/18
• CPC Class Codes: H04N 21/658 : Transmission by the client directed to the server
• CPC Class Codes: H04N 21/6587 : Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 7/106 : for domestic distribution

Abstract

A computer-implemented system and method are described for managing the state of an online video game. For example, a method according to one embodiment comprises: initiating a new online video game in response to user input from a client device, the online video game being in a first state on a first server when initiated; executing the online video game on the server, thereby causing the online video game to enter into a second state; pausing or terminating the online video game; determining differences between the first state and the second state and generating difference data containing the differences; transmitting the difference data over a network to a second server, the second server; and recreating the second state from the difference data and the first state in response to user input indicating that the user wishes to resume the online video game and in response to the second server being selected as the server on which to execute the video game.

Claims

1. A computer-implemented method for managing the state of an online video game or application comprising:

initiating an online video game or application in response to user input from a client device, the online video game or application being in a first state on a first server when initiated;

executing the online video game or application on the first server, thereby causing the online video game or application to enter into a second state, the first server executing the video game or application in response to control signals transmitted from the client, generating video output resulting from the execution of the video game or application, compressing the video output to generate compressed video, and streaming the compressed video over a network including public network components to the client;

pausing or terminating the online video game or application responsive to a pause or terminate signal received from the client;

determining differences between the first state and the second state and generating difference data containing the differences;

transmitting the difference data over a network to a second server; and

recreating the second state from the difference data and the first state in response to user input indicating that the user wishes to resume the online video game or application and in response to the second server being selected as the server on which to execute the video game;

executing the online video game or application on the second server, the second server executing the video game or application in response to control signals transmitted from the client, generating video output resulting from the execution of the video game or application, compressing the video output to generate compressed video, and streaming the compressed video over the network including public network components to the client,

wherein the operations of receiving control signals transmitted from the client, executing the video game or application on either the first or the second server, compressing the video output and streaming the compressed video are performed with a latency such that the user has the perception that the selected video game or application is responding instantly to the control signals received from the client.

2. The method as in claim 1 further comprising:

updating the second state to include platform-dependent data associated with the second server on which the online video game or application will be executed.

3. The method as in claim 2 wherein the platform-dependent data comprises a network address of the second server.

4. The method as in claim 1 further comprising:

dynamically adjusting a data rate at which the compressed video output is streamed to the client.

5. The method as in claim 4 wherein the data rate is adjusted in response to a detected capacity of a communication channel between the first or second servers and the client.

6. The method as in claim 4 wherein the data rate is adjusted by modifying a frame rate of the video output.

7. The method as in claim 4 wherein the data rate is adjusted by modifying quality of the compressed video.

8. The method as in claim 4 wherein the data rate is adjusted by modifying image resolution of the compressed video.

9. The method as in claim 8 wherein when the image resolution is modified, a decoder on the client scales up the image to maintain the same image size on a display screen.

10. The method as in claim 1 wherein when a communication channel between the first or second servers and the client unexpectedly drops out, the first or second server pauses the video game or application.

11. The method as in claim 10 wherein if the communication channel drops out and a first user of the client is playing a multiplayer game, the other users are notified that the user has dropped out.

12. The method as in claim 11 wherein the video game is paused for the other users when the first user drops out.

13. The method as in claim 5 further comprising:

determining the capacity of the communication channel by sending an increasingly higher data rate test stream over the Internet to the client until packet loss and/or higher latency indicates the capacity has been exceeded.

14. The method as in claim 13 further comprising:

determining that the channel capacity has decreased based on a detected increase in packet loss and/or latency increasing; and

responsively reducing the data rate between the first or second server and the client until packet loss and/or latency have reached predetermined acceptable values.

15. The method as in claim 13 further comprising:

determining that the channel capacity may have increased based on continued measurements of packet loss and/or latency; and

responsively increasing the data rate between the first or second server and the client until packet loss and/or latency reaches a predetermined unacceptable level.

16. The method as in claim 1 wherein data packets of the compressed video stream are received by the client in a different order than in which they are displayed.

17. The method as in claim 1 further comprising:

employing forward error correction (FEC) techniques to protect specified portions of the compressed video stream and/or the control signals received from the client.

18. The method as in claim 17 where the FEC techniques are only employed to the extent required based on detected channel capacity of a communication channel between the client and the first or second servers.

19. The method as in claim 1 further comprising:

reducing latency by mitigating channel bandwidth peaks of a communication channel between the client and the first or second servers.

20. The method as in claim 1 wherein the wherein the operations of receiving control signals transmitted from the client, executing the video game or application on either the first or the second server, compressing the video output and streaming the compressed video output is performed at a finite latency of less than 80 ms.

21. The method as in claim 1 wherein the wherein the operations of receiving control signals transmitted from the client, executing the video game or application on either the first or the second server, compressing the video output and streaming the compressed video output is performed at a finite latency of less than 150 ms.

22. A system for managing the state of an online video game or application comprising:

a first server initiating an online video game or application in response to user input from a client device, the online video game or application being in a first state on the first server when initiated;

the first server executing the online video game or application on the first server, thereby causing the online video game or application to enter into a second state, the first server executing the video game or application in response to control signals transmitted from the client, generating video output resulting from the execution of the video game or application, compressing the video output to generate compressed video, and streaming the compressed video over a network including public network components to the client;

the first server pausing or terminating the online video game or application responsive to a pause or terminate signal received from the client;

the first server determining differences between the first state and the second state and generating difference data containing the differences;

a second server receiving the difference data from the first server over a network; and

the second server recreating the second state from the difference data and the first state in response to user input indicating that the user wishes to resume the online video game or application and in response to the second server being selected as the server on which to execute the video game;

the second server executing the online video game or application on the second server, the second server executing the video game or application in response to control signals transmitted from the client, generating video output resulting from the execution of the video game or application, compressing the video output to generate compressed video, and streaming the compressed video over the network including public network components to the client,

wherein the operations of receiving control signals transmitted from the client, executing the video game or application on either the first or the second server, compressing the video output and streaming the compressed video are performed with a latency such that the user has the perception that the selected video game or application is responding instantly to the control signals received from the client.

23. The system as in claim 22 wherein the second state is updated to include platform-dependent data associated with the second server on which the online video game or application will be executed.

24. The system as in claim 23 wherein the platform-dependent data comprises a network address of the second server.

25. The system as in claim 22 wherein the first or second servers dynamically adjust a data rate at which the compressed video output is streamed to the client.

26. The system as in claim 25 wherein the data rate is adjusted in response to a detected capacity of a communication channel between the first or second servers and the client.

27. The system as in claim 25 wherein the data rate is adjusted by modifying a frame rate of the video output.

28. The system as in claim 25 wherein the data rate is adjusted by modifying quality of the compressed video.

29. The system as in claim 25 wherein the data rate is adjusted by modifying image resolution of the compressed video.

30. The system as in claim 29 wherein when the image resolution is modified, a decoder on the client scales up the image to maintain the same image size on a display screen.

31. The system as in claim 22 wherein when a communication channel between the first or second servers and the client unexpectedly drops out, the first or second server pauses the video game or application.

32. The system as in claim 31 wherein if the communication channel drops out and a first user of the client is playing a multiplayer game, the other users are notified that the user has dropped out.

33. The system as in claim 32 wherein the video game is paused for the other users when the first user drops out.

34. The system as in claim 26 wherein the first or second servers determine the capacity of the communication channel by sending an increasingly higher data rate test stream over the Internet to the client until packet loss and/or higher latency indicates the capacity has been exceeded.

35. The system as in claim 34 wherein the first or second servers perform the additional operations of:

determining that the channel capacity has decreased based on a detected increase in packet loss and/or latency increasing; and

responsively reducing the data rate between the first or second server and the client until packet loss and/or latency have reached predetermined acceptable values.

36. The system as in claim 34 wherein the first or second servers perform the additional operations of:

determining that the channel capacity may have increased based on continued measurements of packet loss and/or latency; and

responsively increasing the data rate between the first or second server and the client until packet loss and/or latency reaches a predetermined unacceptable level.

37. The system as in claim 22 wherein data packets of the compressed video stream are received by the client in a different order than in which they are displayed.

38. The system as in claim 22 wherein the first or second servers perform the additional operations of:

employing forward error correction (FEC) techniques to protect specified portions of the compressed video stream and/or the control signals received from the client.

39. The system as in claim 38 where the FEC techniques are only employed to the extent required based on detected channel capacity of a communication channel between the client and the first or second servers.

40. The system as in claim 22 wherein the first or second servers perform the additional operations of:

reducing latency by mitigating channel bandwidth peaks of a communication channel between the client and the first or second servers.

41. The system as in claim 22 wherein the wherein the operations of receiving control signals transmitted from the client, executing the video game or application on either the first or the second server, compressing the video output and streaming the compressed video output is performed at a finite latency of less than 80 ms.

42. The system as in claim 22 wherein the wherein the operations of receiving control signals transmitted from the client, executing the video game or application on either the first or the second server, compressing the video output and streaming the compressed video output is performed at a finite latency of less than 150 ms.