Multi-mode parallel graphics rendering system employing real-time automatic scene profiling and mode control

• US 8,085,273 B2
• Filed: 01/18/2007
• Issued: 12/27/2011
• Est. Priority Date: 11/19/2003
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Lucid Information Technology, Ltd.
• Inventors: Leviathan, Yaniv, Bakalash, Reuven
• Primary Examiner(s): Wang, Andrew
• Assistant Examiner(s): Merouan, Abderrahim
• Application Number:
  US11/655,735
• Publication Number:
  US 20080117217A1
• Time in Patent Office: 1,804 Days
• Field of Search: 345501-506, 382/304, 718102-108, 712 32- 34
• US Class Current: 345/505
• CPC Class Codes: G06F 9/5066 Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs mappping at compile time, see G06F8/451
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 2210/52 Parallel processing

Abstract

A multi-mode parallel 3-D graphics system having multiple graphics processing pipelines with multiple GPUs supporting a parallel graphics rendering process having time, frame and object division modes of operation, wherein each GPU comprises video memory, a geometry processing subsystem and a pixel processing subsystem, and wherein 3D scene profiling is performed in real-time, and the parallelization state/modes of the system are dynamically controlled to meet graphics application requirements. The multiple modes of parallel graphics rendering use real-time graphics application profiling, and dynamic control over time-division, frame-division, and object-division modes of parallel operation, within the same parallel graphics platform, which can be realized on PC-based computing system architectures.

Claims

1. A multi-mode parallel graphics rendering system (MMPGRS) embodied within a host computing system having a CPU for executing one or more graphics-based applications, CPU memory for storing said one or more graphics-based applications and a graphics library for generating graphics commands and data during the execution of each said graphics-based application, and a display device for displaying images of a 3D scene containing graphics during the execution of said one or more graphics-based applications, said MMPGRS supporting object, image and time division modes of parallel operation and comprising:

(1) a multi-mode parallel graphics rendering subsystem including

(i) three parallelization stages including a decompose module, a distribute module and a recompose module, and

(ii) a plurality of graphic processing pipelines (GPPLs) for supporting a graphics rendering process that employs two or more of modes of parallel graphics rendering operation during a single session of said one or more graphics-based applications in order to execute graphic commands and process graphics data and generate images of said 3D scene for display, wherein each said GPPL includes a primary GPPL having a frame buffer for recompositing images; and

(2) a profiling and control mechanism (PCM) having a profiling and control cycle for automatically and dynamically profiling, on a frame by frame basis, said one or more graphics-based applications being executed on said host computing system, and controlling the modes of parallel graphics rendering operation of said MMPGRS during run-time of said one or more graphics based applications;

wherein for each said mode of parallel graphics rendering operation supported by said MMPGRS, said MMPGRS has one state of corresponding operation;

wherein said PCM performs profiling and control functions using multiple data stores, including (i) a historical repository for continuously storing up acquired data having historical depth, to construct a behavioral profile of currently running graphics-based applications, and (ii) a behavioral profile database (DB) for storing an application profile library of prior-known graphics-based applications, and enriched by newly constructed profiles for said prior-known graphics-based applications using data accessed from said historical repository;

wherein said decompose module divides up the stream of graphic commands and data according to the mode of parallel graphics rendering operation determined by said PCM;

wherein said distribute module physically distributes the streams of graphics commands and data to said plurality of GPPLs;

wherein said GPPLs execute said graphics commands using said graphics data and generate partial pixel data sets associated with frames of pixel images to be composited by the primary GPPL in said MMPGRS; and

wherein said recompose module merges together the partial pixel data sets produced from said GPPLs, according to said mode of parallel operation at any instant in time, and producing a final pixel data set within the frame buffer of said primary GPPL, which is sent into said display device for display;

wherein said decompose module, said distribute module and said recompose module each have multiple sub-states of operation, and cooperate to carry out all functions required by the different modes of parallel graphics rendering operation supported on said MMPGRS;

wherein said PCM controls the sub-states of said decompose, distribute and recompose modules, and transitions of the sub-states of said decompose, said distribute and said recompose modules; and

wherein each of said decompose, distribute and recompose modules is induced into a sub-state by setting parameters, and the mode and also the state of parallel graphics rendering operation of said MMPGRS is established by the combination of said sub-states.

2. The MMPGRS of claim 1, wherein said plurality of GPPLs comprises a plurality of pairs of GPU and video memory pipelines;

and wherein only one of said GPPLs is designated as the primary GPPL and is responsible for driving said display device with a final pixel image composited within a frame buffer (FB) maintained by said primary GPPL, and all other GPPLs function as secondary GPPLs, supporting the pixel image recompositing process.

3. The MMPGRS of claim 1,

wherein the sub-states of said decompose module are selected from the group consisting of object decomposition, image decomposition, alternate decomposition, and single GPPL for the object division, image division, time division and single GPPL (non parallel) modes of operation, respectively;

wherein the sub-states of said distribute module are selected from the group consisting of divide and broadcast sub-states for object division and image division modes of operation, and single GPPL sub-state for the time division and single GPPL mode of operation; and

wherein the subs-states of said recompose module are selected from the group consisting of (i) test based sub-state which carries out re-composition based on predefined tests performed on pixels of partial frame buffers, (ii) screen based sub-state which combines together parts of the final frame buffers, and (iii) the None mode which makes no merges, just moves one of the pipeline frame buffers to the display device, as required in time division parallelism or in a single GPPL (non parallel) mode.

4. The MMPGRS of claim 1, wherein said display device comprises an LCD panel.

5. The MMPGRS of claim 1, wherein said host computing system is any computing machine supporting real-time generation and display of 3D graphics.

6. The MMPGRS of claim 2, wherein said GPU comprises a geometry processing subsystem and a pixel processing subsystem.

7. The MMPGRS of claim 1, wherein when a graphics-based application starts, said PCM tries identifying whether said graphics-based application is previously known to said MMPGRS and its profile recorded in said behavioral database;

and wherein in the case of a previously known graphics-based application, the optimal starting state of said MMPGRS is recommended by said behavioral profile database; and

during the course of profiling said one or more graphics-based applications, said behavioral database assists said PCM.

8. The MMPGRS of claim 1, wherein said profiling and control cycle comprises a periodical trial and error based control cycle.

9. The MMPGRS of claim 1, wherein said profiling and control cycle comprises an event driven trial and error control cycle.

10. The MMPGRS of claim 1, wherein said multiple modes of parallel graphics rendering operation are selected from the group consisting of an object division mode of parallel graphics rendering operation, an image division mode of parallel graphics rendering operation, and a time division mode of parallel graphics rendering operation.

11. The MMPGRS of claim 10, wherein while determining when a state transition should occur within said MMPGRS, said PCM considers one or more of the following parameters selected from the group consisting of:

(1) high texture volume, where a high value of this parameter will trigger a transition to the image division and/or time division mode of operation;

(2) high screen resolution, where a high value of this parameter will trigger a transition to the image division, and/or time division mode of operation;

(3) high pixel layer depth, where a high value of this parameter will trigger a transition to the image division mode of operation;

(4) high polygon volume, where a high value of this parameter will trigger a transition to the object division mode of operation;

(5) FPS drop, where this parameter will trigger a transition to a trial and error cycle;

(6) same frame buffer, where this parameter will trigger use in successive frames, as a preventive condition from the time division mode of operation; and

(7) high video memory footprint, where a high value of this parameter will trigger a transition to the object division mode of operation.

12. The MMPGRS of claim 1, wherein said PCM tests whether the graphics-based application is listed in said behavioral profile database;

if the graphics-based application is listed in said behavioral database, then the profile of said graphics-based application is taken from said behavioral profile database, a preferred state is set, N successive frames are rendered, performance data is collected, and added to said historical repository and analyzed for next optimal state; and

upon conclusion of said graphics-based application, said behavioral database is updated by the collected data from said historical repository.

13. The MMPGRS of claim 2, wherein each said GPPL comprises a GPU including a geometry engine, a pixel engine, and video memory, and wherein said PCM further comprises:

an application profiling and analysis module;

a parallel policy management module; and

a distributed graphics function control module; and

wherein said application profiling and analysis module monitors and analyzes profiling data on said one or more graphics-based applications, and performs analysis based on the data selected from the group consisting of;

(i) performance data collected from several sources selected from the group consisting of vendor'"'"'s driver, GPUs, a chipset, and a graphic hub device, for estimating the performance of said several sources and locating bottlenecks;

(ii) data stored in said historical repository; and

(iii) data stored in said behavioral profile database.

14. The MMPGRS of claim 13, wherein said performance data includes data elements selected from the group consisting of:

(i) texture count;

(ii) screen resolution;

(iii) polygon volume;

(iv) at each GPU, utilization of (a) a geometry engine, (b) a pixel engine, and (c) video memory;

(v) Utilization of CPU;

(vi) total pixels rendered;

(vii) total geometric data rendered;

(viii) workload of each GPU; and

(ix) volumes of transferred data.

15. The MMPGRS of claim 13, wherein said application profiling and analysis module processes said performance data for real time analysis and the performance of one or more tasks selected from the group consisting of:

(1) Recognition of said one or more graphics-based applications;

(2) Processing of trial &

error results;

(3) Utilization of application profiles from said behavioral profile database;

(4) Data Aggregation in said historical repository;

(5) Analysis of input performance data;

(6) Analysis based on integration of (a) frame-based “

atomic”

performance data, (b) aggregated data in said historical repository, and (c) data in said behavioral profile database;

(7) Detection of rendering algorithms used by said one or more graphics-based applications;

(8) Detection of use of frame buffer (FB) in next successive frame as a preventive condition for time division mode;

(9) Recognition of preventive conditions for other parallel modes;

(10) Evaluation of pixel layer depth at the pixel engine of each GPU;

(11) Frame/sec count;

(12) Detection of critical events (e.g. frame/sec drop);

(13) Detection of bottlenecks in said GPPLs;

(14) Measure and balance of load among GPUs;

(15) Update said behavioral profile database from said historical repository; and

(16) Selection of optimal parallel mode of operation.

16. The MMPGRS of claim 1, wherein said PCM, said decompose module, said distribute module and said recompose module are implemented as a software package within said CPU memory, and wherein said GGPLs include at least two GPUs supported on external graphics cards.

17. The MMPGRS of claim 1, wherein said PCM and said decompose module are implemented as a software package within said CPU memory, while said distribute and recompose modules are implemented in a hardware-based graphics hub device, and wherein said GGPLs include at least two GPUs supported on external graphics cards connected to said CPU via said graphics hub device and a bridge circuit;

and wherein said display device is connected to said graphics hub device.

18. The MMPGRS of claim 1, wherein said PCM and said decompose module are implemented as a software package within said CPU memory, while said distribute and recompose modules are implemented in a hardware-based graphics hub device supporting said GGPLs.

19. The MMPGRS of claim 18, wherein said graphics hub device is implemented as a single SOC-based graphics chip on a single graphics card to which said graphics hub device is connected.

20. The MMPGRS of claim 1, wherein said PCM and said decompose module are implemented as a software package within said CPU memory, while said distribute and recompose modules are implemented in a graphics hub device within an integrated graphics device (IGD) supporting a GPU, and wherein said GGPLs include at least one GPU on an external graphics card connected to said CPU via said IGD and a bridge circuit.

21. The MMPGRS of claim 1, where said PCM and said decompose module are implemented as a software package within said CPU memory, while said distribute and recompose modules are implemented in a graphics hub device within an integrated graphics device (IGD), and wherein said GGPLs include multiple graphics cards, each supporting at least one GPU, and being connected to said IGD, while said display device is connected to said IGD.