System and method for intelligently allocating client requests to server centers

• US 8,606,942 B2
• Filed: 01/23/2009
• Issued: 12/10/2013
• Est. Priority Date: 12/10/2002
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: OL2 Incorporated (Sony Group Corp.)
• Inventors: Perlman, Stephen G., van der Laan, Roger
• Primary Examiner(s): BURGESS, BARBARA N
• Application Number:
  US12/359,154
• Publication Number:
  US 20090215540A1
• Time in Patent Office: 1,782 Days
• Field of Search: 709/203, 709/217, 709/223, 709/224, 709/238, 709/239, 709/240, 709/241, 709/227, 709/228, 709/229
• US Class Current: 709/229
• CPC Class Codes: A63F 13/12 involving interaction between a plurality of game devices, e.g. transmisison or distribution systems protocols for networked virtual reality, networked games H04L29/06034
• CPC Class Codes: A63F 13/30 Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
• CPC Class Codes: A63F 13/335 using Internet
• CPC Class Codes: A63F 13/352 involving special game server arrangements, e.g. regional servers connected to a national server or a plurality of servers managing partitions of the game world
• CPC Class Codes: A63F 13/533 for prompting the player, e.g. by displaying a game menu
• CPC Class Codes: A63F 13/87 Communicating with other players during game play, e.g. by e-mail or chat
• CPC Class Codes: A63F 2300/402 Communication between platforms, i.e. physical link to protocol
• CPC Class Codes: A63F 2300/513 server hierarchy, e.g. local, regional, national or dedicated for different tasks, e.g. authenticating, billing
• CPC Class Codes: A63F 2300/538 for performing operations on behalf of the game client, e.g. rendering
• CPC Class Codes: A63F 2300/552 for downloading to client devices, e.g. using OS version, hardware or software profile of the client device
• CPC Class Codes: H04L 65/611 for multicast or broadcast systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06
• CPC Class Codes: H04L 65/65 Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/756 adapting media to device capabilities
• CPC Class Codes: H04L 65/765 intermediate
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04N 21/21 Server components or server architectures
• CPC Class Codes: H04N 21/2143 located in a single building, e.g. hotel, hospital or museum arrangements specially adapted for plural spots in a confined site in broadcast systems H04H20/63; adaptations for transmission by electric cable for domestic distribution in television systems H04N7/106
• CPC Class Codes: H04N 21/23 Processing of content or additional data; Elementary server operations; Server middleware
• CPC Class Codes: H04N 21/4781 Games
• CPC Class Codes: H04N 21/6125 : involving transmission via Internet transmission by internet of broadcast information H04H60/82
• CPC Class Codes: H04N 7/106 : for domestic distribution
• CPC Class Codes: H04W 84/12 : WLAN [Wireless Local Area Networks]
• CPC Class Codes: H04W 88/14 : Backbone network devices

Abstract

A system and method are described for intelligently allocating client requests to server centers provide real-time streaming interactive video. For example, one embodiment of a computer-implemented method comprises: strategically positioning a plurality of application server centers at different geographical locations; receiving a request from a client to execute an online application; determining the latency requirements based on the type of application requested by the client; and forwarding the client request to a particular application server center within the plurality based at least on the latency requirements of the requested application.

Claims

1. A computer-implemented method comprising:

receiving a request from a client of a user to execute an online video game or application over one or more network interfaces;

determining latency requirements of the video game or application requested by the client;

forwarding the client request to a particular hosting service center based on the latency requirements of the requested video game or application;

executing the video game or application on a server at a particular hosting service center, the execution of the video game or application producing an interactive compressed video stream delivered over one or more networks to the client such that the user has the perception that the selected video game or application is responding instantly; and

mitigating bandwidth peaks of the interactive compressed video stream by distributing large compressed video bandwidth peaks over a sequence of successive frames of the interactive compressed video stream;

wherein distributing large compressed video bandwidth peaks over a sequence of successive frames of the interactive compressed video stream further comprises transmitting compressed frames that have more data than can be transmitted at a peak data rate in one frame time over one or more additional frame times.

2. The method as in claim 1 wherein the application is an interactive video game.

3. The method as in claim 1 further comprising:

receiving packet streams from the client of the user containing control signals, the video game or application being operable to generate an uncompressed video stream responsive to the control signals; and

compressing the uncompressed video stream with low latency to generate the interactive compressed video stream.

4. The method as in claim 1 wherein during the additional frame times a client continues to display a previously decompressed frame.

5. The method as in claim 4 further comprising ignoring frames received during the additional frame times.

6. The method as in claim 5 wherein the video frame rate temporarily decreases during the additional frame times.

7. The method as in claim 5 wherein the additional frame time is 1 frame time.

8. The method as in claim 5 wherein the additional frame time is more than 1 frame time.

9. The method as in claim 4 further comprising:

transmitting an audio stream from the server executing the video game or application to the client, wherein during the additional frame times the audio stream continues to be compressed and decompressed on the client device unabated.

10. The method as in claim 1 further comprising dynamically adjusting a data rate of the interactive compressed video stream.

11. The method as in claim 10 wherein the data rate is adjusted in response to a detected maximum data rate of a communication channel between the server and the client.

12. The method as in claim 10 wherein the data rate is adjusted by modifying the frame rate.

13. The method as in claim 10 wherein the data rate is adjusted by modifying the quality of the interactive compressed video.

14. The method as in claim 10 wherein the data rate is adjusted by modifying an image resolution of the interactive compressed video.

15. The method as in claim 14 wherein when the image resolution is modified, the client scales up the image to maintain the same image size on a display screen.

16. The method as in claim 1 further comprising:

pausing the video game or application when a network connection between the server and the client becomes unresponsive.

17. The method as in claim 1 wherein if the loss of a portion of the interactive compressed video stream would result in a noticeable visual artifact, then generating forward error correction (FEC) data to protect the portion of the interactive compressed video stream.

18. The method as in claim 17 wherein FEC is only used if a disruption in the interactive compressed video stream would be more than 250 ms.

19. The method as in claim 1 further comprising initially determining a peak data rate by sending an increasingly higher data rate test stream to the client until packet loss and/or higher latency indicates channel capacity has been exceeded.

20. The method as in claim 19 further comprising:

determining that the channel capacity has decreased based on a detected increase in packet loss and/or latency increasing; and

responsively reducing the peak data rate until packet loss and/or latency have reached acceptable values.

21. The method as in claim 19 further comprising:

determining that the channel capacity may have increased based on continued measurements of packet loss and/or latency; and

responsively increasing the peak data rate until packet loss and/or latency reaches an unacceptable level.

22. The method as in claim 1 further comprising transmitting data packets to the client in a different order than in which they are displayed.

23. A system comprising at least one computer memory for storing program code and at least one central processing unit processing the program code and thereby performing the operations of:

receiving a request from a client of a user to execute an online video game or application over one or more network interfaces;

determining latency requirements of the video game or application requested by the client;

forwarding the client request to a particular hosting service center based on the latency requirements of the requested video game or application;

executing the video game or application on a server at a particular hosting service center, the execution of the video game or application producing an interactive compressed video stream delivered over one or more networks to the client such that the user has the perception that the selected video game or application is responding instantly; and

mitigating bandwidth peaks of the interactive compressed video stream by distributing large compressed video bandwidth peaks over a sequence of successive frames of the interactive compressed video stream;

wherein distributing large compressed video bandwidth peaks over a sequence of successive frames of the interactive compressed video stream further comprises transmitting compressed frames that have more data than can be transmitted at a peak data rate in one frame time over one or more additional frame times.

24. The system as in claim 23 wherein the application is an interactive video game.

25. The system as in claim 23 comprising additional program code executed by the processor to perform the additional operations of:

receiving packet streams from the client of the user containing control signals, the video game or application being operable to generate an uncompressed video stream responsive to the control signals; and

compressing the uncompressed video stream with low latency to generate the interactive compressed video stream.

26. The system as in claim 23 wherein during the additional frame times a client continues to display a previously decompressed frame.

27. The system as in claim 26 further comprising ignoring frames received during the additional frame times.

28. The system as in claim 27 wherein the video frame rate temporarily decreases during the additional frame times.

29. The system as in claim 27 wherein the additional frame time is 1 frame time.

30. The system as in claim 27 wherein the additional frame time is more than 1 frame time.

31. The system as in claim 26 comprising additional program code executed by the processor to perform the additional operations of:

transmitting an audio stream from the server executing the video game or application to the client, wherein during the additional frame times the audio stream continues to be compressed and decompressed on the client device unabated.

32. The system as in claim 23 further comprising dynamically adjusting a data rate of the interactive compressed video stream.

33. The system as in claim 32 wherein the data rate is adjusted in response to a detected maximum data rate of a communication channel between the server and the client.

34. The system as in claim 32 wherein the data rate is adjusted by modifying the frame rate.

35. The system as in claim 32 wherein the data rate is adjusted by modifying the quality of the interactive compressed video.

36. The system as in claim 33 wherein the data rate is adjusted by modifying an image resolution of the interactive compressed video.

37. The system as in claim 36 wherein when the image resolution is modified, the client scales up the image to maintain the same image size on a display screen.

38. The system as in claim 23 further comprising:

pausing the video game or application when a network connection between the server and the client becomes unresponsive.

39. The system as in claim 23 wherein if the loss of a portion of the interactive compressed video stream would result in a noticeable visual artifact, then generating forward error correction (FEC) data to protect the portion of the interactive compressed video stream.

40. The system as in claim 39 wherein FEC is only used if a disruption in the interactive compressed video stream would be more than 250 ms.

41. The system as in claim 23 further comprising initially determining a peak data rate by sending an increasingly higher data rate test stream to the client until packet loss and/or higher latency indicates channel capacity has been exceeded.

42. The system as in claim 41 comprising additional program code executed by the processor to perform the additional operations of:

determining that the channel capacity has decreased based on a detected increase in packet loss and/or latency increasing; and

responsively reducing the peak data rate until packet loss and/or latency have reached acceptable values.

43. The system as in claim 42 comprising additional program code executed by the processor to perform the additional operations of:

determining that the channel capacity may have increased based on continued measurements of packet loss and/or latency; and

responsively increasing the peak data rate until packet loss and/or latency reaches an unacceptable level.

44. The system as in claim 23 further comprising transmitting data packets to the client in a different order than in which they are displayed.