System and method for storing program code and data within an application hosting center

• US 9,956,490 B2
• Filed: 01/23/2009
• Issued: 05/01/2018
• Est. Priority Date: 12/10/2002
• Status: Expired due to Term

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment America LLC (Sony Group Corp.)
• Inventors: Perlman, Stephen G., van der Laan, Roger
• Primary Examiner(s): STRANGE, AARON N
• Application Number:
  US12/359,171
• Publication Number:
  US 20090215531A1
• Time in Patent Office: 3,385 Days
• Field of Search: 709231
• CPC Class Codes: A63F 13/12 involving interaction between a plurality of game devices, e.g. transmisison or distribution systems protocols for networked virtual reality, networked games H04L29/06034
• CPC Class Codes: A63F 13/30 Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
• CPC Class Codes: A63F 13/335 using Internet
• CPC Class Codes: A63F 13/352 involving special game server arrangements, e.g. regional servers connected to a national server or a plurality of servers managing partitions of the game world
• CPC Class Codes: A63F 13/533 for prompting the player, e.g. by displaying a game menu
• CPC Class Codes: A63F 13/87 Communicating with other players during game play, e.g. by e-mail or chat
• CPC Class Codes: A63F 2300/402 Communication between platforms, i.e. physical link to protocol
• CPC Class Codes: A63F 2300/513 server hierarchy, e.g. local, regional, national or dedicated for different tasks, e.g. authenticating, billing
• CPC Class Codes: A63F 2300/538 for performing operations on behalf of the game client, e.g. rendering
• CPC Class Codes: A63F 2300/552 for downloading to client devices, e.g. using OS version, hardware or software profile of the client device
• CPC Class Codes: H04L 65/611 for multicast or broadcast systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06
• CPC Class Codes: H04L 65/65 Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/756 adapting media to device capabilities
• CPC Class Codes: H04L 65/765 intermediate
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04N 21/21 Server components or server architectures
• CPC Class Codes: H04N 21/2143 located in a single building, e.g. hotel, hospital or museum arrangements specially adapted for plural spots in a confined site in broadcast systems H04H20/63; adaptations for transmission by electric cable for domestic distribution in television systems H04N7/106
• CPC Class Codes: H04N 21/23 Processing of content or additional data; Elementary server operations; Server middleware
• CPC Class Codes: H04N 21/4781 Games
• CPC Class Codes: H04N 21/6125 : involving transmission via Internet transmission by internet of broadcast information H04H60/82
• CPC Class Codes: H04N 7/106 : for domestic distribution
• CPC Class Codes: H04W 84/12 : WLAN [Wireless Local Area Networks]
• CPC Class Codes: H04W 88/14 : Backbone network devices

Abstract

A system and method are described for storing program code and data within an application hosting center. For example, one embodiment of a computer-implemented method comprises: subdividing program code and/or data used to execute an online application into a first type and a second type; storing program code and data of the first type in a first type of memory, the first type of memory providing relatively low latency memory access; storing program code and data of the second type in a second type of memory, the second type of memory providing relatively higher latency memory access compared to the first type of memory; retrieving program code and data from the first memory and the second memory in response to a client request to execute an online application; and transmitting a streaming interactive video stream representing images generated by the application to the client.

Claims

1. A computer-implemented method comprising:

receiving, by a server center, control signals from a client over the Internet, wherein the server center includes a plurality of servers, wherein the plurality of servers include a first server and a second server;

executing, by the first server of the server center, a first interactive online video game in response to the control signals received from the client;

executing, by the second server of the server center, a second interactive online video game;

generating, by the server center, an interactive video stream including at least two frames resulting from the execution of the first and second interactive online video games, wherein the at least two frames include a first frame and a second frame, wherein the first frame is generated by the first server and the second frame is generated by the second server;

receiving, by the server center from the client via the Internet, a video stream of a player interfacing with the client;

identifying, by the server center, first server executing the first interactive video game;

providing, by the server center, the video stream of the player to the first server determined to execute the first interactive video game for integration of the video stream of the player with the first frame of the interactive video stream;

integrating, by the server center, a portion of the video stream of the player with the first frame of the interactive video stream to generate a first integrated video frame;

compressing, by a shared hardware compressor of the server center, the first integrated video frame to generate a first compressed frame;

saving, by the shared hardware compressor, a state of compression of the first compressed frame;

compressing, by the shared hardware compressor, the second video frame after compressing the first integrated video frame, to generate a second compressed frame;

saving, by the shared hardware compressor, a state of compression of the second compressed frame, wherein the state of compression used for the second compressed frame is different from the state of compression used for the first compressed frame; and

transmitting, by the server center, an interactive compressed video stream having the first and second compressed frames to another client over the Internet for display of an image of the first interactive online video game, an image of the second interactive online video game, and an image of the player by the other client, wherein the image of the player is configured to be displayed in a display area along with the image resulting from the execution of the first interactive online video game.

2. The method as in claim 1, wherein executing the first interactive online video game further comprises retrieving program code and data from a memory in response to a request from the client to execute the first interactive online video game, wherein the memory is a Flash memory.

3. The method as in claim 1, wherein the server center comprises a video game engine for executing the first interactive online video game in response to input signals received from the client and wherein an output of the video game engine comprises a portion of the interactive video stream having a low latency.

4. The method as in claim 1, wherein executing the first interactive online video game further comprises retrieving program code and data from a memory in response to a request from the client to execute the first interactive online video game, wherein the program code and data include an operating system on which the first interactive online video game is executed.

5. The method as in claim 1, wherein compressing the first integrated video frame further comprises:

determining characteristics of a network communication channel between the server center and the other client and processing capabilities of the other client; and

uniquely compressing the first integrated video frame based on the characteristics of the network communication channel between the server center and the other client and the processing capabilities of the other client.

6. The method as in claim 5, wherein compressing the first integrated video frame further comprises performing interframe coding and intraframe coding.

7. The method as in claim 6, wherein the interframe coding comprises one or more I-frames and wherein the intraframe coding comprises one or more B-frames or one or more P-frames.

8. The method as in claim 5, wherein uniquely compressing the interactive first integrated video frame based on the characteristics of the network communication channel between the server center and the other client and the processing capabilities of the other client comprises uniquely compressing the first integrated video frame based on a maximum allowable latency for the interactive compressed video stream to be received and decoded at the other client.

9. The method as in claim 5, wherein uniquely compressing the first integrated video frame comprises modifying a resolution of the first integrated video frame.

10. The method as in claim 5, further comprising:

detecting that the characteristics of the network communication channel between the server center and other client has dropped below a specified allowable quality level; and

pausing the execution of the video game on the server center in response to detecting that the characteristics of the network communication channel has dropped below the specified allowable quality level.

11. The method as in claim 5, wherein determining the characteristics of the network communication channel between the server center and the other client comprises detecting a level of packet loss, or latency, or a combination thereof associated with the network communication channel.

12. The method as in claim 11, further comprising adjusting a peak data rate of the compressed interactive video stream based on the detected packet loss, or the latency, or the combination thereof.

13. The method as in claim 12, wherein adjusting the peak data rate comprises adjusting a level of compression of the interactive video stream.

14. The method as in claim 12, further comprising:

determining that channel capacity of the network communication channel has increased based on continued measurements of packet loss or latency; and

increasing the peak data rate until the packet loss or the latency reaches an unacceptable level.

15. The method as in claim 1, wherein compressing the first integrated video frame further comprises:

mitigating channel bandwidth peaks by distributing large compressed video bandwidth peaks over a sequence of successive video frames.

16. The method as in claim 1, further comprising:

employing forward error correction (FEC) techniques to protect specified portions of the interactive compressed video stream.

17. The method as in claim 16, further comprising:

determining that a specified portion of the interactive compressed video stream results in a visual artifact; and

upon determining that the specified portion of the interactive compressed video stream results in the artifact, generating FEC data to protect the specified portion of the compressed interactive video stream.

18. The method of claim 1, wherein compression of the interactive video stream is adjusted according to a latency of a round trip between the server center and the other client, wherein the latency of the round trip is determined by sending a plurality of test frames to the other client, wherein the latency of the round trip is determined by receiving the plurality of test frames from the other client.

19. The method as in claim 1, wherein the at least two frames includes at least three frames, wherein the at least three frames include the first frame, the second frame, and a third frame, wherein the third frame is generated by the first server, wherein said providing, by the server center, the video stream of the player to the first server is performed for integration of the video stream of the player with the first and third frames, the method further comprising:

integrating, by the server center, another portion of the video stream of the player with the third frame of the interactive video stream to generate a third integrated video frame;

compressing, by the shared hardware compressor, the third integrated video frame after compressing the second integrated video frame to generate a third compressed frame, wherein the third integrated video frame is compressed by applying the state of compression for compressing the first compressed frame, wherein the interactive compressed video stream that is transmitted has the third compressed frame.

20. A system comprising:

a plurality of servers configured to receive control signals from a client over the Internet, wherein the plurality of servers include a first server and a second server;

a memory coupled to first server;

wherein the first server is configured to execute a first interactive online video game in response to the control signals received from the client, wherein to execute the first interactive online video game,

wherein the second server is configured to execute a second online interactive video game,

wherein the servers are configured to generate an interactive video stream including at least two frames resulting from the execution of the first and second interactive online video games, wherein the at least two frames include a first frame and a second frame, wherein the first frame is generated by the first server and the second frame is generated by the second server,

wherein the first server is configured to receive a video stream of a player interfacing with the client;

an inbound routing network coupled to the servers, wherein the inbound routing network is configured to identify the first server executing the first interactive video game,

wherein the inbound routing network is configured to provide the video stream of the player to the first server determined to execute the first interactive video game for integration of a portion of the video stream of the player with the first frame of the interactive video stream,

wherein the first server is configured to integrate the portion of the video stream with the first frame of the interactive video stream,

a compression device coupled to the server, wherein the compression device is configured to compress the first frame of the interactive video stream to generate a first compressed frame,

wherein the compression device is configured to save a state of compression of the first compressed frame,

wherein the compression device is configured to compress the second frame to generate a second compressed frame after compressing the first frame,

wherein the compression device is configured to save a state of the second compressed frame, wherein the state of compression of the second compressed frame is different from the state of compression of the first compressed frame; and

a routing network coupled to the compression device, wherein the routing network is configured to send an interactive compressed video stream having the first and second compressed frames to another client over the Internet for display of an image of the first interactive online video game, an image of the second interactive online video game, and an image of the player by the other client, wherein the image of the player is configured to be displayed in a display area along with the image resulting from the execution of the first interactive online video game.

21. The system as in claim 20, wherein the server center comprises a video game engine for executing the first interactive online video game in response to input signals received from the client and wherein an output of the video game engine comprises a portion of the interactive video stream having a low latency.

22. The system as in claim 20, wherein the first server is configured to retrieve program code and data from the memory in response to a request from the client to execute the first interactive online video game, wherein the program code and data include an operating system on which the first interactive online video game is executed.

23. The system as in claim 20, wherein to compress the interactive video stream, the compression device is configured to:

determine characteristics of a network communication channel between the first server and the other client and processing capabilities of the other client; and

uniquely compress the first integrated video frame based on the characteristics of the network communication channel between the first server and the other client and the processing capabilities of the other client.

24. The system as in claim 23, wherein to uniquely compress the first integrated video frame based on the characteristics of the network communication channel between the first server and the other client and the processing capabilities of the other client, the compression device is configured to uniquely compress the first integrated video frame based on a maximum allowable latency for the first integrated video frame to be received and decoded at the other client.

25. The system as in claim 23, wherein the compression device is configured to detect that the characteristics of the network communication channel between the first server and the other client has dropped below a specified allowable quality level, wherein the first server is configured to pause the execution of the video game in response to the detection that the characteristics of the network communication channel have dropped below the specified allowable quality level.

26. The system as in claim 23, wherein to determine characteristics of the network communication channel between the first server and the other client, the compression device is configured to detect a level of packet loss, or latency, or a combination thereof associated with the network communication channel.

27. The system as in claim 26, wherein the compression device is configured to adjust a peak data rate of the compressed interactive video stream based on the packet loss, or the latency, or the combination thereof, wherein to adjust the peak data rate, the compression device is configured to adjust a level of compression of the interactive video stream.

28. The system as in claim 23, wherein to uniquely compress the interactive video stream, the compression device is configured to modify a resolution of the interactive video stream.

29. The system as in claim 20, wherein to compress the interactive video stream, the compression device is configured to perform interframe coding and intraframe coding.

30. The system as in claim 29, wherein the interframe coding comprises one or more I-frames and wherein the intraframe coding comprises one or more B-frames or one or more P-frames.

31. The system as in claim 20, wherein to compress the interactive video stream, the compression device is configured to:

mitigate channel bandwidth peaks by distributing large compressed video bandwidth peaks over a sequence of successive video frames.

32. The system as in claim 20, wherein the compression device is configured to:

determine characteristics of a network communication channel between the first server and the other client and processing capabilities of the other client;

determine that a channel capacity of the network communication channel has increased based on continued measurements of packet loss or latency; and

increase the peak data rate until the packet loss or the latency reaches an unacceptable level.

33. The system as in claim 20, wherein the compression device is configured to adjust compression of the interactive video stream according to a latency of a round trip between the first server and the other client, wherein the compression device is configured to determine the latency of the round trip by sending a plurality of test frames to the other client, wherein the compression device is configured to determine the latency of the round trip by receiving the plurality of test frames from the other client.

34. A non-transitory computer readable medium containing program instructions, wherein execution of the program instructions by one or more processors of a computer system causes the one or more processors to carry out operations comprising:

receiving, by a server center, control signals from a client over the Internet, wherein the server center includes a plurality of servers, wherein the plurality of servers include a first server and a second server;

executing, by the first server of the server center, a first interactive online video game in response to the control signals received from the client;

executing, by the second server of the server center, a second interactive online video game;

generating, by the server center, an interactive video stream including at least two frames resulting from the execution of the first and second interactive online video games, wherein the at least two frames include a first frame and a second frame, wherein the first frame is generated by the first server and the second frame is generated by the second server;

receiving, by the server center from the client via the Internet, a video stream of a player interfacing with the client;

identifying, by the server center, the first server executing the first interactive video game;

providing, by the server center, the video stream of the player to the first server determined to execute the first interactive video game for integration of the video stream of the player with the first frame of the interactive video stream;

integrating, by the server center, a portion of the video stream of the player with the first frame of the interactive video stream to generate a first integrated video frame;

compressing, by a shared hardware compressor of the server center, the first integrated video frame to generate a first compressed frame;

saving, by the shared hardware compressor, a state of compression of the first compressed frame;

compressing, by the shared hardware compressor, the second video frame after compressing the first integrated video frame, to generate a second compressed frame;

saving, by the shared hardware compressor, a state of compression of the second compressed frame, wherein the state of compression used for the second compressed frame is different from the state of compression used for the first compressed frame; and

transmitting, by the server center, an interactive compressed video stream having the first and second compressed frames to another client over the Internet for display of an image of the first interactive online video game, an image of the second interactive online video game, and an image of the player by the other client, wherein the image of the player is configured to be displayed in a display area along with the image resulting from the execution of the first interactive online video game.

35. The non-transitory computer readable medium as in claim 34, wherein executing the first interactive online video game further comprises retrieving program code and data from a memory in response to a request from the client to execute the first interactive online video game, wherein the memory is a Flash memory.

36. The non-transitory computer readable medium as in claim 34, wherein the server center comprises a video game engine executing the first interactive online video game in response to input signals received from the client and wherein an output of the video game engine comprises a portion of the interactive video stream having a low latency.

37. The non-transitory computer readable medium as in claim 34, wherein executing the first interactive online video game further comprises retrieving program code and data from a memory in response to a request from the client to execute the first interactive online video game, wherein the program code and data include an operating system on which the first interactive online video game is executed.

38. The non-transitory computer readable medium as in claim 34, wherein compressing the interactive video stream further comprises:

determining characteristics of a network communication channel between the server center and the other client and processing capabilities of the other client; and

uniquely compressing the interactive video stream based on the characteristics of the network communication channel between the server center and the other client and the processing capabilities of the other client.

39. The non-transitory computer readable medium as in claim 38, further comprising the operations of:

detecting that the characteristics of the network communication channel between the server center and other client has dropped below a specified allowable quality level; and

pausing the execution of the video game on the server center in response to detecting that the characteristics of the network communication channel has dropped below the specified allowable quality level.

40. The non-transitory computer readable medium as in claim 34, wherein compressing the interactive video stream comprises performing interframe coding and intraframe coding.

41. The non-transitory computer readable medium as in claim 40, wherein the interframe coding comprises one or more I-frames and wherein the intraframe coding comprises one or more B-frames or one or more P-frames.