Multi-mode parallel graphics rendering system employing real-time automatic scene profiling and mode control

• US 20080117217A1
• Filed: 01/18/2007
• Published: 05/22/2008
• Est. Priority Date: 11/19/2003
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Lucid Information Technology, Ltd.
• Inventors: Leviatan, Yaniv, Bakalash, Reuven
• Granted Patent: US 8,085,273 B2
• Time in Patent Office: Days
• US Class Current: 345/502
• CPC Class Codes: G06F 9/5066 Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs mappping at compile time, see G06F8/451
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 2210/52 Parallel processing

Abstract

A multi-mode parallel 3-D graphics system having multiple graphics processing pipelines with multiple GPUs supporting a parallel graphics rendering process having time, frame and object division modes of operation, wherein each GPU comprises video memory, a geometry processing subsystem and a pixel processing subsystem, and wherein 3D scene profiling is performed in real-time, and the parallelization state/modes of the system are dynamically controlled to meet graphics application requirements. The multiple modes of parallel graphics rendering use real-time graphics application profiling, and dynamic control over time-division, frame-division, and object-division modes of parallel operation, within the same parallel graphics platform, which can be realized on PC-based computing system architectures.

Claims

1-51. -51. (canceled)

52. A multi-mode parallel graphics rendering system (MMPGRS) embodied within a host computing system having a CPU for executing graphics-based applications, CPU memory for storing one or more graphics-based applications and a graphics library for generating graphics commands and data during the execution of the graphics-based application, and a display device for displaying images containing graphics during the execution of said application, said MMPGRS supporting object, image and time division modes of parallel operation and comprising:

(1) a multi-mode parallel graphics rendering subsystem including

(i) three parallelization stages including a decompose module, a distribute module and a recompose module, and

(ii) a plurality of graphic processing pipelines (GPPLs) supporting a graphics rendering process that employs object division, image division and/or time division modes of parallel operation during a single session of said graphics-based application in order to execute graphic commands and process graphics data; and

(2) a profiling and control mechanism (PCM) for automatically and dynamically profiling said graphics-based application executing on said host computing system, and controlling the various modes of parallel operation of said MMPGRS;

wherein said decompose module, said distribute module and said recompose module cooperate to carry out all functions required by the different modes of parallel operation supported on said MMPGRS; and

wherein said PCM enables real-time graphics application profiling and automatic configuration of said multiple GPPLs.

53. The MMPGRS of claim 52, wherein said array of GPPLs comprises a plurality of pairs of GPU and video memory pipelines;

and wherein only one of said GPPLs is designated as the primary GPPL and is responsible for driving said display unit with a final pixel image composited within a frame buffer (FB) maintained by said primary GPPL, and all other GPPLs function as secondary GPPLs, supporting the pixel image recompositing process.

54. The MMPGRS of claim 53, wherein said decompose module divides up the stream of graphic commands and data according to the required mode of parallel operation determined by said PCM;

wherein said distribute module physically distributes the streams of graphics commands and data to said plurality of GPPLs;

wherein said GPPLs execute said graphics commands using said graphics data and generate partial pixel data sets associated with frames of pixel images to be composited by the primary GPPL in said MMPGRS; and

wherein said recompose module merges together the partial pixel data sets from produced from said GPPLs, according to mode of parallel operation at any instant in time, and producing a final pixel data set within the frame buffer of the primary GPPL, which is sent into said display device for display.

55. The MMPGRS of claim 52,

wherein said decompose module can be set to different decomposing sub-states selected from the group consisting of object decomposition, image decomposition, alternate decomposition, and single GPPL for the object division, image division, time division and single GPPL (non parallel) modes of operation, respectively;

wherein said distribute module can be set to different distributing sub-states selected from the group consisting of divide and broadcast sub-state for object division and image division modes of operation, and single GPPL sub-state for the time division and single GPPL (i.e. non parallel system) mode of operation; and

wherein said recompose module can be set to different sub-states selected from the group consisting of (i) test based sub-state which carries out re-composition based on predefined test performed on pixels of partial frame buffers (typically these are depth test, stencil test, or combination thereof), (ii) screen based sub-state combines together parts of the final frame buffers (e.g. in a puzzle like fashion, creating a single image), and (iii) the None mode which makes no merges, just moves one of the pipeline frame buffers to the display device, as required in time division parallelism or in single GPU (non parallel); and

wherein said PCM controls the substates of said decompose, distribute and recompose modules, and interstate transitions thereof.

56. The MMPGRS of claim 52, wherein each of said decompose, distribute and recompose modules is induced into a sub-state by setting parameters, and the mode (i.e. state) of parallel operation of said MMPGRS is established by the combination of such sub-states.

57. The MMPGRS of claim 52, wherein said display unit comprises an LCD panel.

58. The MMPGRS of claim 52, wherein said host computing system is any computing machine supporting real-time generation and display of 3D graphics.

59. The MMPGRS of claim 52, wherein said PCM performs real-time profiling of the graphics-based application, on a frame by frame basis, and dynamically controls the modes of parallel operation of said MMPGRS so as to meet graphics requirements of said graphics-based Application.

60. The MMPGRS of claim 53, wherein said GPU comprises a geometry processing subsystem and a pixel processing subsystem.

61. The MMPGRS of claim 53, wherein said PCM comprises:

an application profiling and analysis module;

a parallel policy management module; and

a distributed graphics function control module.

62. The MMPGRS of claim 59, wherein said PCM performs its profiling and control functions using multiple data stores, including:

a historical repository for continuously storing up acquired data (i.e. this data having historical depth, and being used for constructing behavioral profile of ongoing application); and

a behavioral profile database (DB) for storing an application profile library of prior-known graphics Applications, and further enriched by newly created profiles based on data from said historical depository.

63. The MMPGRS of claim 59, wherein said PCM comprises a Profiling and Control Cycle, wherein when a graphics-based application starts, said PCM tries identifying whether said Application is previously known to said MMPGRS and its profile recorded in said behavioral database;

and wherein in the case of a previously known Application, the optimal starting state of said MMPGRS is recommended by said behavioral profile database; and

during the course of profiling said Application, said behavioral database assists said PCM.

64. The MMPGRS of claim 63, wherein said profiling and control cycle comprises a periodical trial &

error based control cycle.

65. The MMPGRS of claim 63, wherein said profiling and control cycle comprises an event driven trial &

error control cycle.

66. The MMPGRS of claim 59, wherein said PCM comprises a profiling and control cycle, and when determining when a state transition should occur within said MMPGRS, said PCM considers one or more of the following parameters selected from the group consisting of:

(1) high texture volume, where a high value of this parameter will trigger (i.e. indicate) a transition to the image division and/or time division mode of operation;

(2) high screen resolution, where a high value of this parameter will trigger a transition to the image division, and/or time division mode of operation;

(3) high pixel layer depth, where a high value of this parameter will trigger a transition to the image division mode of operation;

(4) high polygon volume, where a high value of this parameter will trigger a transition to the object division state of operation;

(5) fps drop, where this parameter will trigger a transition to the trial &

error cycle;

(6) same frame buffer, where this parameter will trigger use in successive frames, as a preventive condition from time division mode of operation; and

(7) high video memory footprint, where a high value of this parameter will trigger a transition to the object division mode of operation.

67. The MMPGRS of claim 66 wherein said PCM supports state transition control using one or more of said parameters.

68. The MMPGRS of claim 62, wherein said PCM tests whether the graphics-based Application is listed in said behavioral profile database;

if the graphics-based Application is listed in said behavioral database, then the profile of said graphics-based Application is taken from said behavioral profile database, a preferred state is set, N successive frames are rendered, performance data collected, and added to said historical repository and analyzed for next optimal state; and

upon conclusion of said graphics-based Application, said behavioral database is updated by the collected data from said historical repository.

69. The MMPGRS of claim 61, wherein said PCM further comprises an application profiling and analysis module;

a parallel policy management module; and

a distributed graphics function control module; and

wherein said application profiling and analysis module monitors and analyzes profiling data of running application, and performs its analysis based on the data selected from the group consisting of;

(i) performance data collected from several sources selected from the group consisting of vendor'"'"'s driver, GPUs, chipset, and a graphic hub device) for estimating the performance and locating bottlenecks;

(ii) data stored in said historical repository; and

(iii) data stored in said behavioral profile database.

70. The MMPGRS of claim 69, wherein said performance data includes data elements selected from the group consisting of:

(i) texture count;

(ii) screen resolution;

(iii) polygon volume;

(iv) at each GPU utilization of (a) Geometry engine, (b) Pixel engine, and (c) Video memory;

(v) Utilization of CPU;

(vi) total pixels rendered;

(vii) total geometric data rendered;

(viii) workload of each GPU; and

(ix) volumes of transferred data.

71. The MMPGRS of claim 69, wherein said application profiling and analysis module processes said performance data for real time analysis and the performance of one or more tasks selected from the group consisting of:

(1) Recognition of application;

(2) Processing of trial &

error results;

(3) Utilization of application profile from said behavioral profile database;

(4) Data Aggregation in said historical repository;

(5) Analysis of input performance data;

(6) Analysis based on integration of (a) frame-based “

atomic”

performance data, (b) aggregated data in said historical repository, and (c) data in said behavioral profile database;

(7) Detection of rendering algorithms used by application;

(8) Detection of use of FB in next successive frame as a preventive condition for time division mode;

(9) Recognition of preventive conditions for other parallel modes;

(10) Evaluation of pixel layer depth at the pixel subsystem of each GPU;

(11) Frame/sec count;

(12) Detection of critical events (e.g. frame/sec drop);

(13) Detection of bottlenecks in said GPPLs;

(14) Measure and balance of load among GPUs;

(15) Update behavioral profile database from historical depository; and

(16) Selection of optimal parallel mode of operation.

72. The MMPGRS of claim 52 where said PCM, said decompose module, distribute module and recompose module are implemented as a software package within said CPU memory, and wherein said GGPLs include at least two GPUs supported on external graphics cards.

73. The MMPGRS of claim 52 wherein said PCM and said decompose module are implemented as a software package within said CPU memory, while said distribute and recompose modules are implemented in a hardware-based graphics hub device, and wherein said GGPLs include at least two GPUs supported on external graphics cards connected to said CPU via said graphics hub device and a bridge circuit;

and wherein said display device is connected to said graphics hub device.

74. The MMPGRS of claim 52, wherein said PCM and said decompose module are implemented as a software package within said CPU memory, while said distribute and recompose modules are implemented in a hardware-based graphics hub devices supporting said GGPLs.

75. The MMPGRS of claim 74, wherein said graphics hub device is implemented as a single SOC-based graphics chip on a single graphics card to which said graphics device is connected.

76. The MMPGRS of claim 52, wherein said PCM and said decompose module are implemented as a software package within said CPU memory, while said distribute and recompose modules are implemented in a graphics hub device within a integrated graphics device (IGD) supporting a GPU, and wherein said GGPLs include at least one GPU on external graphics card connected to said CPU via said IGD and a bridge circuit.

77. The MMPGRS of claim 52, where said PCM and said Decomposing Module are implemented as a software package within said CPU memory, while said distribute and recompose modules are implemented in a graphics hub device within a integrated graphics device (IGD), and wherein said GGPLs include multiple graphics cards, each supporting at least one GPU, and being connected to said IGD, while said display device is connected to said IGD.