SYSTEM AMD METHOD FOR IMPROVING THE GRAPHICS PERFORMANCE OF HOSTED APPLICATIONS

• US 20120115601A1
• Filed: 09/14/2011
• Published: 05/10/2012
• Est. Priority Date: 12/10/2002
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment America LLC (Sony Group Corp.)
• Inventors: Dietrich, Douglas Sim Jr., Benitez, Nico, Cotter, Timothy
• Granted Patent: US 8,851,999 B2
• Time in Patent Office: Days
• US Class Current: 463/31
• CPC Class Codes: A63F 13/355 Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
• CPC Class Codes: A63F 13/358 Adapting the game course according to the network or server load, e.g. for reducing latency due to different connection speeds between clients
• CPC Class Codes: A63F 2300/402 Communication between platforms, i.e. physical link to protocol
• CPC Class Codes: A63F 2300/407 Data transfer via internet
• CPC Class Codes: A63F 2300/534 for network load management, e.g. bandwidth optimization, latency reduction
• CPC Class Codes: A63F 2300/552 for downloading to client devices, e.g. using OS version, hardware or software profile of the client device
• CPC Class Codes: A63F 2300/554 by saving game or status data
• CPC Class Codes: A63F 2300/556 Player lists, e.g. online players, buddy list, black list
• CPC Class Codes: A63F 2300/572 Communication between players during game play of non game information, e.g. e-mail, chat, file transfer, streaming of audio and streaming of video
• CPC Class Codes: A63F 2300/577 for watching a game played by other players
• CPC Class Codes: A63F 2300/636 involving process of starting or resuming a game
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 65/61 for supporting one-way streaming services, e.g. Internet radio
• CPC Class Codes: H04L 65/611 for multicast or broadcast systems for broadcast or conference H04L12/18; arrangements for broadcast or distribution combined with broadcast H04H20/00; arrangements for broadcast applications with a direct linkage to broadcast information or to broadcast space-time H04H60/00; selective distribution of broadcast services, e.g. multimedia broadcast multicast service [MBMS], H04W4/06
• CPC Class Codes: H04L 65/765 intermediate
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/146 Data rate or code amount at the encoder output
• CPC Class Codes: H04N 19/164 : Feedback from the receiver or from the transmission channel
• CPC Class Codes: H04N 19/172 : the region being a picture, frame or field
• CPC Class Codes: H04N 19/188 : the unit being a video data packet, e.g. a network abstraction layer [NAL] unit
• CPC Class Codes: H04N 19/436 : using parallelised computational arrangements
• CPC Class Codes: H04N 19/61 : in combination with predictive coding
• CPC Class Codes: H04N 21/233 : Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/23406 : involving management of server-side video buffer
• CPC Class Codes: H04N 21/2343 : involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/2381 : Adapting the multiplex stream to a specific network, e.g. an Internet Protocol [IP] network transmission of MPEG streams over ATM H04L12/5601
• CPC Class Codes: H04N 21/4781 : Games
• CPC Class Codes: H04N 21/6125 : involving transmission via Internet transmission by internet of broadcast information H04H60/82
• CPC Class Codes: H04N 21/6377 : directed to server one-way streaming services wherein the source is controlled by the destination H04L65/613
• CPC Class Codes: H04N 21/6405 : Multicasting data broadcast and multicast in packet switching networks H04L12/18
• CPC Class Codes: H04N 21/658 : Transmission by the client directed to the server
• CPC Class Codes: H04N 21/6587 : Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 21/8458 : involving uncompressed content

Abstract

A system for hosting video games comprising: a game server comprising a central processing unit to process video game program code and a graphics processing unit (GPU) to process graphics commands; one or more back buffers to store video frames in response to the execution of the graphics commands; a front buffer to receive a video frame for rendering on a display and outputting the video frame for display one scan line at a time at a designated scan out frequency, a subset of scan lines stored in the front buffer being associated with a vertical blanking interval (VBI); a frame processing module to increase the number of scan lines associated with the VBI to increase the likelihood that a new frame will be completed in a back buffer and ready for transfer to the front buffer at a time during the VBI.

Claims

1. A server-based system for hosting applications/video games for a plurality of clients comprising:

an application/game server comprising a central processing unit to process application/video game program code and a graphics processing unit (GPU) to process graphics commands and generate a series of video frames for the application/video game;

one or more back buffers to store video frames as the video frames are being created in response to the execution of the graphics commands;

a front buffer to receive a video frame for rendering on a display after the video frame has been completed in one of the one or more back buffers, the front buffer outputting the video frame for display one scan line at a time at a designated scan out frequency, a subset of scan lines stored in the front buffer being associated with a vertical blanking interval (VBI);

a frame processing module to perform real-time profiling of the GPU by performing the operations of;

determining a current mode in which the GPU is operating;

tracking how long it takes the GPU to execute certain types of operations while in the current mode of operation; and

updating a benchmark database indexed by the different hash values associated with the different GPU modes of operation and storing values representing the amount of time needed for each specified type of operation.

2. The system as in claim 1 wherein the certain types of operations comprises rendering of triangles.

3. The system as in claim 1 wherein the certain types of operations comprises rendering of polygons.

4. The system as in claim 1 wherein the frame processing module inserts timestamp fences around work being inserted into the GPU command queue, determines the amount of time consumed performing the work based on the differences between the timestamps, and continually updates the hash table to include an average per-operation time.

5. The system as in claim 1 wherein the different modes of operation include depth-only rendering;

lighting;

alpha blending; and

image processing.

6. The system as in claim 1 wherein the frame processing module subsequently queries the benchmark database to determine the amount of GPU work remaining in the command queue, and/or the number of busy work operations needed to reach a specified scan out region.

7. The system as in claim 6 wherein the specified scan out region comprises the vertical blanking interval (VBI) or a region outside of a designated center region of the front buffer.

8. The system as in claim 6 wherein the frame processing module inserts a number of busy work operations in the command queue needed to reach the specified scan out region.

9. A method for hosting applications/video games for a plurality of clients comprising:

an application/game server comprising a central processing unit to process application/video game program code and a graphics processing unit (GPU) to process graphics commands and generate a series of video frames for the application/video game;

one or more back buffers to store video frames as the video frames are being created in response to the execution of the graphics commands;

a front buffer to receive a video frame for rendering on a display after the video frame has been completed in one of the one or more back buffers, the front buffer outputting the video frame for display one scan line at a time at a designated scan out frequency, a subset of scan lines stored in the front buffer being associated with a vertical blanking interval (VBI);

a frame processing module to perform real-time profiling of the GPU by performing the operations of;

determining a current mode in which the GPU is operating;

tracking how long it takes the GPU to execute certain types of operations while in the current mode of operation; and

updating a benchmark database indexed by the different hash values associated with the different GPU modes of operation and storing values representing the amount of time needed for each specified type of operation

10. The method as in claim 9 wherein the certain types of operations comprises rendering of triangles.

11. The method as in claim 9 wherein the certain types of operations comprises rendering of polygons.

12. The method as in claim 9 further comprising:

inserting timestamp fences around work being inserted into the GPU command queue, determines the amount of time consumed performing the work based on the differences between the timestamps, and continually updates the hash table to include an average per-operation time.

13. The method as in claim 9 wherein the different modes of operation include depth-only rendering;

lighting;

alpha blending; and

image processing.

14. The method as in claim 9 further comprising:

subsequently quering the benchmark database to determine the amount of GPU work remaining in the command queue, and/or the number of busy work operations needed to reach a specified scan out region.

15. The method as in claim 14 wherein the specified scan out region comprises the vertical blanking interval (VBI) or a region outside of a designated center region of the front buffer.

16. The method as in claim 14 further comprising:

inserting a number of busy work operations in the command queue needed to reach the specified scan out region.