Computing system having multi-mode parallel graphics rendering subsystem (MMPGRS) employing real-time automatic scene profiling and mode control

• US 8,134,563 B2
• Filed: 10/30/2007
• Issued: 03/13/2012
• Est. Priority Date: 11/19/2003
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Lucid Information Technology, Ltd.
• Inventors: Bakalash, Reuven, Leviathan, Yaniv
• Primary Examiner(s): Wang, Andrew
• Assistant Examiner(s): Merouan, Abderrahim
• Application Number:
  US11/980,089
• Publication Number:
  US 20080129748A1
• Time in Patent Office: 1,596 Days
• Field of Search: 345/501, 345/502, 345/504, 345/505, 345/506, 345/419, 345/420, 345/421, 345/424, 345/441, 345/522, 345/582, 713/153, 713/189, 713/193, 713/194, 710/305, 710/306
• US Class Current: 345/505
• CPC Class Codes: G06F 9/5066 Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs mappping at compile time, see G06F8/451
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 2210/52 Parallel processing

Abstract

A parallel graphics rendering system is embodied within a host computing system and includes a plurality of graphic processing pipelines (GPPLs) and graphics processing modules. The parallel graphics rendering system supports one or more modes of parallel operation selected from the group consisting of object division, image division, and time division. a plurality of graphic processing pipelines The GPPLs support a parallel graphics rendering process that employs one or more of the object division, image division and/or time division modes of parallel operation in order to execute graphic commands and process graphics data, and render pixel-composited images containing graphics for display on a display device during the run-time of the graphics-based application. An automatic mode control module automatically controls the mode of parallel operation of the parallel graphics rendering system during the run-time of the graphics-based application. The graphics processing modules of the parallel graphics rendering system can be embodied within diverse system architectures.

Claims

1. A computing system comprising:

CPU memory for storing one or more graphics-based applications and a graphics library for generating graphics commands and data during the execution of said one or more graphics-based applications;

a CPU for executing said at least one of graphics-based applications during run-time;

a display device for displaying images of a 3D scene containing graphics during the execution of said graphics-based application; and

a multi-mode parallel graphics rendering subsystem (MMPGRS) supporting multiple modes of parallel operation and including;

(i) three parallelization stages including a decompose module, a distribute module and a recompose module, and

(ii) a plurality of graphic processing pipelines (GPPLs) for supporting a graphics rendering process that employs two or more of modes of parallel graphics rendering operation during a single session of said graphics-based application in order to execute graphic commands and process graphics data and generate images of said 3D scene for display on said display device;

wherein said GPPLs include a primary GPPL and one or more secondary GPPLs, and wherein each GPPL includes a frame buffer; and

a profiling and control mechanism (PCM) having a profiling and control cycle for automatically and dynamically profiling, on a frame by frame basis, said graphics-based application being executed by said CPU, and controlling the modes of parallel graphics rendering operation of said MMPGRS during the run-time of said graphics based application;

wherein for each said mode of parallel graphics rendering operation supported by said MMPGRS, said MMPGRS has one state of corresponding operation;

wherein said PCM performs profiling and control functions using multiple data stores, including

(i) a historical repository for continuously storing up acquired data having historical depth, for constructing behavioral profile of currently running graphics-based application, and

(ii) a behavioral profile database (DB) for storing an application profile library of prior-known graphics-based applications, and enriched by newly created profiles for each said prior-known graphics-based application using data accessed from said historical depository;

wherein said decompose module divides up the stream of graphic commands and data according to the mode of parallel graphics rendering operation determined by said PCM;

wherein said distribute module physically distributes the streams of graphics commands and data to said plurality of GPPLs;

wherein said GPPLs execute said graphics commands using said graphics data and generate partial pixel data sets associated with frames of pixel images to be composited in said MMPGRS;

wherein said recompose module merges together the partial pixel data sets produced from said GPPLs, according to said mode of parallel operation, and producing a final pixel data set within the frame buffer of the primary GPPL, for display on said display device;

wherein said decompose module, said distribute module and said recompose module each have multiple sub-states of operation, and cooperate to early out all functions required by the different modes of parallel graphics rendering operation supported on said MMPGRS;

wherein said PCM controls the sub-states of said decompose, distribute and recompose modules, and interstate transitions thereof; and

wherein each of said decompose, distribute and recompose modules is induced into a sub-state by setting parameters, and

wherein the state and thus the mode of parallel graphics rendering operation of said MMPGRS is established by the combination of sub-states.

2. The computing system of claim 1, wherein said plurality of GPPLs comprises a plurality of pairs of GPUs and video memory pipelines;

and wherein only one of said GPUs is designated as said primary GPPL and is responsible for driving said display device with a final pixel image maintained within a frame buffer (FB) in said primary GPPL, and all other GPUs function as said secondary GPPLs, supporting the pixel image recompositing process.

3. The computing system of claim 1,

wherein the sub-states of said decompose module are selected from the group consisting of object decomposition, image decomposition, alternate decomposition, and single GPPL for the object division, image division, time division and single GPPL (non parallel) modes of operation, respectively;

wherein the sub-states of said distribute module are selected from the group consisting of divide and broadcast sub-states for object division and image division modes of operation, and single GPPL sub-state for the time division and single GPPL mode of operation; and

wherein the sub-states of said recompose module are selected from the group consisting of (i) test based sub-state which carries out re-composition based on predefined tests performed on pixels of partial frame buffers, (ii) screen based sub-state which combines together parts of the final frame buffers, and (iii) a none mode which makes no merges, just moves one of the pipeline frame buffers to the display device, as required in time division parallelism or in single GPPL (non parallel) mode.

4. The computing system of claim 1, wherein said display device comprises an LCD panel.

5. The computing system of claim 2, wherein each said GPU further comprises a geometry processing subsystem and a pixel processing subsystem.

6. The computing system of claim 1, wherein when said graphics-based application starts, said PCM tries identifying whether said graphics-based application is previously known to said MMPGRS and its profile recorded in said behavioral database;

and wherein in the case of a previously known graphics-based application, the optimal starting state of said MMPGRS is recommended by said behavioral profile database; and

during the course of profiling said graphics-based application, said behavioral database assists said PCM.

7. The computing system of claim 1, wherein said profiling and control cycle comprises a periodical trial and error based control cycle.

8. The computing system of claim 1, wherein said profiling and control cycle comprises an event driven trial and error control cycle.

9. The computing system of claim 1, wherein said multiple modes of parallel graphics rendering operation are selected from the group consisting of an object division mode of parallel graphics rendering operation, an image division mode of parallel graphics rendering operation, and a time division mode of parallel graphics rendering operation.

10. The computing system of claim 9, wherein when determining when a state transition should occur within said MMPGRS, said PCM considers one or more of the following parameters selected from the group consisting of:

(1) high texture volume, where a high value of this parameter will trigger a transition to the image division and/or time division mode of operation;

(2) high screen resolution, where a high value of this parameter will trigger a transition to the image division, and/or time division mode of operation;

(3) high pixel layer depth, where a high value of this parameter will trigger a transition to the image division mode of operation;

(4) high polygon volume, where a high value of this parameter will trigger a transition to the object division mode of operation;

(5) FPS drop, where this parameter will trigger a transition to the trial and error cycle;

(6) same frame buffer, where this parameter will trigger use in successive frames, as a preventive condition from time division mode of operation; and

(7) high video memory footprint, where a high value of this parameter will trigger a transition to the object division mode of operation.

11. The computing system of claim 1, wherein said PCM tests whether the graphics-based application is listed in said behavioral profile database;

if the graphics-based application is listed in said behavioral database, then the profile of said graphics-based application is taken from said behavioral profile database, a preferred state is set, N successive frames are rendered, performance data collected, and added to said historical repository and analyzed for next optimal state; and

upon conclusion of said graphics-based application, said behavioral database is updated by the collected data from said historical repository.

12. The computing system of claim 3, wherein said PCM further comprises:

an application profiling and analysis module;

a parallel policy management module; and

a distributed graphics function control module;

wherein said application profiling and analysis module monitors and analyzes profiling data of the running graphics-based application, and performs analysis based on the data selected from the group consisting of;

(i) performance data collected from several sources selected from the group consisting of vendor'"'"'s driver, GPUs, chipset, and a graphic hub device, for estimating the performance and locating bottlenecks;

(ii) data stored in said historical repository; and

(iii) data stored in said behavioral profile database.

13. The computing system of claim 12, wherein said performance data includes data elements selected from the group consisting of:

(i) texture count;

(ii) screen resolution;

(iii) polygon volume;

(iv) at each GPU utilization of (a) geometry engine, (b) pixel engine, and (c) video memory;

(v) utilization of CPU;

(vi) total pixels rendered;

(vii) total geometric data rendered;

(viii) workload of each GPU; and

(ix) volumes of transferred data.

14. The computing system of claim 12, wherein said application profiling and analysis module processes said performance data for real time analysis and the performance of one or more tasks selected from the group consisting of:

(1) Recognition of application;

(2) Processing of trial and error results;

(3) Utilization of application profile from said behavioral profile database;

(4) Data Aggregation in said historical repository;

(5) Analysis of input performance data;

(6) Analysis based on integration of (a) frame-based atomic performance data, (b) aggregated data in said historical repository, and (c) data in said behavioral profile database;

(7) Detection of rendering algorithms used by application;

(8) Detection of use of frame buffer in next successive frame as a preventive condition for time division mode;

(9) Recognition of preventive conditions for other parallel modes;

(10) Evaluation of pixel layer depth at the pixel subsystem of each GPU;

(11) Frame/second count;

(12) Detection of critical events such as frame/second drop;

(13) Detection of bottlenecks in said GPPLs;

(14) Measure and balance of load among GPUs;

(15) Update behavioral profile database from historical depository; and

(16) Selection of optimal parallel mode of operation.

15. The computing system of claim 1, wherein said PCM, said decompose module, said distribute module and said recompose module are implemented as a software package within said CPU memory, and wherein said GGPLs include at least two GPUs supported on external graphics cards.

16. The computing system of claim 1, wherein said PCM and said decompose module are implemented as a software package within said CPU memory, while said distribute and recompose modules are implemented in a hardware-based graphics hub device, and wherein said GGPLs include at least two GPUs supported on external graphics cards connected to said CPU via said graphics hub device and a bridge circuit;

and wherein said display device is connected to said graphics hub device.

17. The computing system of claim 1, wherein said PCM and said decompose module are implemented as a software package within said CPU memory, while said distribute and recompose modules are implemented in a hardware-based graphics hub device supporting said GGPLs.

18. The computing system of claim 17, wherein said hardware-based graphics hub device is implemented as a single SOC-based graphics chip on a single graphics card to which said graphics hub device is connected.

19. The computing system of claim 1, wherein said PCM and said decompose module are implemented as a software package within said CPU memory, while said distribute and recompose modules are implemented in a hardware-based graphics hub device within an integrated graphics device (IGD) supporting a GPU, and wherein said GGPLs include at least one GPU on an external graphics card connected to said CPU via said IGD and a bridge circuit.

20. The computing system of claim 1, where said PCM and said decompose module are implemented as a software package within said CPU memory, while said distribute and recompose modules are implemented in a hardware-based graphics hub device within an integrated graphics device (IGD), and wherein said GGPLs include multiple graphics cards, each supporting at least one GPU, and being connected to said IGD, while said display device is connected to said IGD.