System and method for improving the graphics performance of hosted applications

• US 9,375,635 B2
• Filed: 03/07/2013
• Issued: 06/28/2016
• Est. Priority Date: 03/23/2009
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Interactive Entertainment America LLC (Sony Group Corp.)
• Inventors: Cotter, Timothy
• Primary Examiner(s): D'Agostino, Paul A.
• Assistant Examiner(s): DOSHI, ANKIT B
• Application Number:
  US13/789,621
• Publication Number:
  US 20130237316A1
• Time in Patent Office: 1,209 Days
• Field of Search: 463/42
• US Class Current: 1/1
• CPC Class Codes: A63F 13/335 using Internet
• CPC Class Codes: A63F 13/352 involving special game server arrangements, e.g. regional servers connected to a national server or a plurality of servers managing partitions of the game world
• CPC Class Codes: A63F 13/355 Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
• CPC Class Codes: A63F 2300/402 Communication between platforms, i.e. physical link to protocol
• CPC Class Codes: A63F 2300/407 Data transfer via internet
• CPC Class Codes: A63F 2300/534 for network load management, e.g. bandwidth optimization, latency reduction
• CPC Class Codes: A63F 2300/552 for downloading to client devices, e.g. using OS version, hardware or software profile of the client device
• CPC Class Codes: A63F 2300/554 by saving game or status data
• CPC Class Codes: A63F 2300/556 Player lists, e.g. online players, buddy list, black list
• CPC Class Codes: A63F 2300/5593 involving scheduling aspects
• CPC Class Codes: A63F 2300/572 Communication between players during game play of non game information, e.g. e-mail, chat, file transfer, streaming of audio and streaming of video
• CPC Class Codes: A63F 2300/577 for watching a game played by other players
• CPC Class Codes: A63F 2300/636 involving process of starting or resuming a game
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 65/611 for multicast or broadcast systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06
• CPC Class Codes: H04L 65/765 intermediate
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04N 21/233 Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/23406 involving management of server-side video buffer
• CPC Class Codes: H04N 21/2343 : involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/235 : Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/2381 : Adapting the multiplex stream to a specific network, e.g. an Internet Protocol [IP] network transmission of MPEG streams over ATM H04L12/5601
• CPC Class Codes: H04N 21/4781 : Games
• CPC Class Codes: H04N 21/6125 : involving transmission via Internet transmission by internet of broadcast information H04H60/82
• CPC Class Codes: H04N 21/6405 : Multicasting data broadcast and multicast in packet switching networks H04L12/18
• CPC Class Codes: H04N 21/6587 : Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 21/8458 : involving uncompressed content

Abstract

A system and method for efficiently performing graphics operations on a video game/application hosting service. One embodiment of a system comprises: an application/game server comprising a central processing unit and a graphics processing unit generating a series of video frames; a buffer management logic to manage the series of video frames; a shared buffer managed by the buffer management logic to store the video frames generated; wherein the buffer management logic continually monitors a signal indicating when a video display or video compression unit is ready to receive a next video frame and, responsive to detecting that the display or video compression unit is about to be ready, the buffer management logic to transfer the most recently completed from the shared buffer to a back buffer; and responsive to detecting the signal, the buffer management logic transferring the video frame from the back buffer to a front buffer.

Claims

1. A system comprising:

an application/game server comprising a central processing unit to process application/video game program code and a graphics processing unit (GPU) to process graphics commands and generate a series of video frames for the application/video game;

buffer management logic to manage storage and transmission of the series of video frames;

a shared buffer managed by the buffer management logic to store the video frames as quickly as the video frames can be generated by the application/game server responsive to execution of the video game or application;

wherein the buffer management logic continually monitors a signal indicating when a video display or video compression unit is ready to receive a next video frame and, responsive to the buffer management detecting that the display or video compression unit is about to be ready to receive the next video frame, the buffer management logic to transfer the most recently completed from the shared buffer to a back buffer; and

wherein in response to detecting the signal indicating that the display or video compression unit is ready to receive the next video frame, the buffer management logic transferring the video frame from the back buffer to a front buffer from which it is read out, line by line, and provided to the display or video compression unit for compressing the resulting video stream.

2. The system as in claim 1 wherein the signal indicating that the display or video encoder is ready to receive the next video frame comprises a VSYNC signal.

3. The system as in claim 1 wherein the video compression unit is to compress the video frames at low latency to generate a low latency compressed video stream.

4. The system as in claim 3 wherein the low latency compressed video stream is transmitted over a network to a client device.

5. The system as in claim 4 wherein the client device decompresses the low latency video stream and renders the video generated by the video game or application on a display.

6. The system as in claim 5 wherein the client device transmits control signals over the network in response to user input, the control signals received by the video game or application, wherein the video game or application generates the video frames in response to the control signals.

7. The system as in claim 6 wherein the operations of generating the control signals, executing the video game or application, compressing the video frames to generate the video stream, transmitting the video stream to the client are performed with a latency such that the user of the client has the perception that the video game or application is responding instantly to the control signals.

8. A method comprising:

executing program code for a video game or application including processing graphics commands and responsively generating a series of video frames for the application/video game;

managing storage and transmission of the series of video frames including storing the video frames as quickly as the video frames can be generated by the application/game server responsive to execution of the video game or application;

continually monitoring a signal indicating when a video display or video compression unit is ready to receive a next video frame and, responsive to detecting that the display or video compression unit is about to be ready to receive the next video frame, transferring the most recently completed from the shared buffer to a back buffer; and

wherein in response to detecting the signal indicating that the display or video compression unit is ready to receive the next video frame, transferring the video frame from the back buffer to a front buffer from which it is read out, line by line, and provided to the display or video compression unit for compressing the resulting video stream.

9. The method as in claim 8 wherein the signal indicating that the display or video encoder is ready to receive the next video frame comprises a VSYNC signal.

10. The method as in claim 8 wherein the video compression unit is to compress the video frames at low latency to generate a low latency compressed video stream.

11. The method as in claim 10 wherein the low latency compressed video stream is transmitted over a network to a client device.

12. The method as in claim 11 wherein the client device decompresses the low latency video stream and renders the video generated by the video game or application on a display.

13. The method as in claim 12 wherein the client device transmits control signals over the network in response to user input, the control signals received by the video game or application, wherein the video game or application generates the video frames in response to the control signals.

14. The method as in claim 13 wherein the operations of generating the control signals, executing the video game or application, compressing the video frames to generate the video stream, transmitting the video stream to the client are performed with a latency such that the user of the client has the perception that the video game or application is responding instantly to the control signals.