Deferred shading graphics pipeline processor having advanced features

• US 6,717,576 B1
• Filed: 08/20/1999
• Issued: 04/06/2004
• Est. Priority Date: 08/20/1998
• Status: Expired due to Term

• Current Assignee: Apple Inc.
• Original Assignee: Apple Computer Incorporated (Apple Inc.)
• Inventors: Papakipos, Matthew N., Hong, Bo, Rashid, Abbas, Mammen, Abraham, Fang, Emerson S., Nguyen, Tuan D., Redgrave, Jason R., Tsay, Albert Suan-Wei, Dodgen, Stephen L., Li, Sidong, Grass, Joseph P., Ng, Sam, Gong, Zhaoyu, Tuck, Nathan D., Cuan, George, Hsu, Hengwei, Arnold, Vaughn T., Fung, Lindy, Ho, Thomas Y., Duluk, Jerome F. Jr., Bratt, Joseph P., Go, Shun Wai, Hessel, Richard E., Benkual, Jack, Trivedi, Sushma S.
• Primary Examiner(s): Padmanabhan, Mano
• Application Number:
  US09/377,503
• Time in Patent Office: 1,691 Days
• Field of Search: 345/501, 345/502, 345/506, 345/522, 345/509, 345/553, 345/419, 345/421, 345/422, 345/426, 345/427, 345/581, 345/582, 345/589, 345/596, 345/611, 345/612, 345/613, 345/614, 345/619, 345/620, 345/625, 345/628, 345/545, 345/546, 345/552, 345/691, 345/694
• US Class Current: 345/419
• CPC Class Codes: G06T 11/40 Filling a planar surface by adding surface attributes, e.g. colour or texture
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 15/30 Clipping
• CPC Class Codes: G06T 15/405 using Z-buffer
• CPC Class Codes: G06T 15/50 Lighting effects
• CPC Class Codes: G06T 15/83 Phong shading

Abstract

A deferred shading graphics pipeline processor and method are provided encompassing numerous substructures. Embodiments of the processor and method may include one or more of deferred shading, a tiled frame buffer, and multiple-stage hidden surface removal processing. In the deferred shading graphics pipeline, hidden surface removal is completed before pixel coloring is done. The pipeline processor comprises a command fetch and decode unit, a geometry unit, a mode extraction unit, a sort unit, a setup unit, a cull unit, a mode injection unit, a fragment unit, a texture unit, a Phong lighting unit, a pixel unit, and a backend unit.

Claims

1. A graphics rendering system for forming a finished rendered image, the graphics rendering system comprising:

(i) a host computer having host memory coupled thereto and at least one input/output bus, the host computer supplying graphics data, the graphics data comprising graphics primitives;

(ii) one or more front end blocks to handle communication with the host computer through the input/output bus, the front end blocks also converting the graphics data into a series of packets;

(iii) a plurality of processing blocks connected sequentially in a pipeline, a first of the processing blocks connected to the front end blocks, where each of the processing blocks comprises;

(a) at least one data input;

(b) at least one data output;

(c) a FIFO buffer at the at least one data input; and

(d) logic for a packetized data transfer protocol for transferring information from processing block to processing block in packets, the packets each including a header portion and a data portion, the protocol used to sequentially transfer different packets having different forms and various lengths over a single communication channel from a processing block to another processing block while maintaining sequential order of at least some of the transferred information;

(iv) a frame buffer;

(v) a backend block coupled to the frame buffer and last of said processing blocks, the backend block function comprising controlling the frame buffer and sending the finished rendered image to an output device; and

(vi) a communication path coupling said backend block to said first of the processing blocks such that packets sent on said communication path pass through fewer than all of said sequentially connected processing blocks.

2. The graphics rendering system in claim 1, wherein the plurality of processing blocks further comprise:

a geometry block, coupled to the front end blocks, comprising logic for transformation of vertex coordinates, transformation of vertex normals, and per-vertex lighting.

3. The graphics rendering system in claim 2, wherein the geometry block further comprises:

logic for receiving one or more types of geometry input packets to transfer information from the front end blocks to the geometry block, the geometry input packets transferring information comprising;

transform matrices, material parameters, light parameters, and vertex data.

4. The graphics rendering system of claim 1 further comprising:

a scene memory, comprised of one or more memory blocks, coupled to one or more of the processing blocks, the scene memory used to store pipeline data, the pipeline data comprising;

(1) primitive data; and

(2) pipeline state;

the scene memory being comprised of at least;

(1) a spatial memory block for storing (1a) the part of the primitive data needed for hidden surface removal and (1b) the part of the pipeline state needed for hidden surface removal; and

(2) polygon memory block for storing (2a) the part of the primitive data not needed for hidden surface removal and (2b) the part of the pipeline state not needed for hidden surface removal.

5. A graphics rendering system according to claim 1, wherein said packets sent on said communication path comprise prefetch packets such that prefetch packets arrive at said backend block earlier than other packets not sent on said communication path.

6. A graphics rendering system according to claim 1, wherein at least one of said plurality of processing units is configured to dispatch a first type of packet to another one of said processing units and a second type of packet to a different one of said plurality of processing units.

7. A graphics rendering system according to claim 6, wherein said at least one of said plurality of processing units is a unit configured to sort graphics primitives according to tiles, said another one of said processing units is a unit configured to retrieve stored mode information, and said different one of said plurality of processing units is a unit configured to send graphics primitives to one or more other units in tile order.

8. A graphics rendering system according to claim 6, wherein said at least one of said plurality of processing units is a unit configured to retrieve stored mode information, said another one of said processing units is a unit configured to interpolate color values, and said different one of said plurality of processing units is a unit configured to perform per-fragment operations.

9. A graphics rendering system according to claim 6 wherein said at least one of said plurality of processing units is a unit configured to interpolate color values, said another one of said processing units is a unit configured to perform shading, and said different one of said plurality of processing units is a unit configured to apply texture maps.

10. A graphics rendering system according to claim 1, wherein said communication path comprises an interface between two processing units located on the same chip, wherein packets sent on said interface bypass other processing units on said chip.

11. A graphics rendering system according to claim 1, wherein said plurality of processing blocks include a unit configured to sort graphics primitives according to tiles, a unit configured to send graphics primitives to one or more other units in tile order, a unit configured to perform hidden surface removal, and a unit configured to retrieve stored mode information, said system further comprising:

a first interface between said unit configured to sort and said unit configured to send;

a second interface between said unit configured to send and said unit configured to perform hidden surface removal;

a third interface between said unit configured to perform hidden surface removal and said unit configured to retrieve stored mode information; and

wherein said communication path comprises a fourth interface between said unit configured to sort and unit configured to retrieve stored mode information.

12. A graphics rendering system according to claim 11, wherein said unit configured to sort, said unit configured to send, said unit configured to perform hidden surface removal, and said unit configured to retrieve stored mode information are provided on a first semiconductor chip.

13. A graphics rendering system according to claim wherein said plurality of processing units are provided on a plurality of semiconductor chips, including said first semiconductor chip, said system further comprising:

an interchip communication ring coupling said plurality of chips.

14. A graphics rendering method for forming a finished rendered image, the graphics rendering method comprising the steps:

(i) receiving data comprising graphics primitives;

(ii) converting at least some of the graphics data into a series of packets;

(iii) processing the series of packets through a plurality of graphics processes including a backend process, the plurality of graphics processes being sequentially connected in a pipeline, including a first graphics process that receives the converted graphics data and a last graphics process that forms the finished rendered image; and

each graphics process comprising the steps;

(a) receiving a packet;

(b) generating a new packet for use in a packetized data transfer protocol for transferring information from graphics process in packets, the packets each including a header portion and a data portion, the protocol used to sequentially transmit different packets having different forms and various lenghts

(c) transmitting the new packet over single communication channel from a graphics process to another graphics process while maintaining sequential order of at least some of the transferred information

(d) generating a prefetch packet;

(e) transmitting said prefetch packet over a second communication channel to said backend process, wherein said second communication channel is shorter than said single communication channel;

(iv) storing the finished rendered image in a frame buffer; and

(v) sending the finished rendered image to an output device.

15. The graphics rendering method in claim 14, further comprising the steps:

receiving commands that stimulate the receiving of additional graphics data via direct memory access; and

receiving at least one type of geometry input packet, the geometry input packet transferring information comprising;

transform matrices, material parameters, light parameters, and vertex data;

storing pipeline data into one or more memories, the pipeline data comprising;

(1) primitive data; and

(2) pipeline state; and

each geometry process further comprising the step of receiving a plurality of types of vertex packets, the plurality of types of vertex packets being differing lengths that are processed at different performance levels; and

the storing step further comprising performing a three dimensional (3D) tile read, performing a three dimensional (3D) tile write using pixel ownership and performing a pixel ownership for write enables and overlay detection.

16. The graphics rendering method of claim 15 wherein the storing pipeline data step further comprises:

(1) storing first pipeline data into a spatial memory, the first pipeline data stored into spatial memory comprising;

(1a) the part of the primitive data needed for hidden surface removal and (1b) the part of the pipeline state needed for hidden surface removal; and

(2) storing second pipeline data into a polygon memory, the second pipeline data stored into polygon memory comprising;

(2a) the part of the primitive data not needed for hidden surface removal and (2b) the part of the pipeline state not needed for hidden surface removal.

17. The graphics rendering method in claim 15, wherein the plurality of graphics processes further comprise:

a sort process comprising the step;

storing vertex packets and mode packets into the spatial memory.

18. The graphics rendering method of claim 14, further comprising storing pipeline data into one or more memories, the pipeline data comprising (1) primitive data;

and pipeline state.

20. The graphics rendering method of claim 18 further comprising:

the step of storing the finished rendered image further comprising;

accessing a portion of the frame buffer as a window consisting of a rectangular grid of pixels, and the window being divided into tiles; and

at least some of the plurality of graphics processes comprising steps for performing per tile processing for forming the finished rendered image.

21. The graphics rendering method of claim 20, wherein the plurality of graphics processes further comprise:

a sort process comprising the steps;

(1) maintaining a list of vertices representing the graphic primitives;

(2) maintaining a set of tile pointer lists, one tile pointer list for each tile;

(3) sorting all the geometry in a frame, and (4) generating primitive packets, each primitive packet representing a complete primitive.

22. The graphics rendering method of claim 21, wherein the plurality of graphics processes further comprising:

a mode extraction process comprising the steps;

collecting temporally ordered state change data; and

saving temporally ordered state change in a polygon memory.

23. The graphics rendering method of claim 22, wherein the mode extraction process further comprises the steps:

accumulating two sets of material and texture data, one set for each of front and back faces of a primitive; and

storing, into the polygon memory, only one of the two sets based on a flag indicator for each primitive.

19. The graphics rendering method of claim wherein the plurality of graphics processes further comprise:

a cull process comprising the step;

performing a hidden surface removal process for culling out parts of the primitives that do not contribute to the finished rendered image and generating visible portions of the primitives; and

one or more graphics processes jointly comprising the steps;

fragment coloring and fragment blending, the steps performed on the generated visible portions of primitives.

24. A graphics rendering method comprising:

receiving graphics data;

converting at least some of said graphics data into a plurality of packets;

performing a mode extraction process comprising;

separating said plurality of packets into;

(i) spatial information comprising spatial packets, begin frame packets, end frame packets, and clear packets, and

(ii) shading information, the shading information comprising color packets, texture packets, and material packets;

sending said spatial information to a sorting process; and

storing said shading information in a polygon memory.

25. A method according to claim 24, wherein said sending comprises sending such that said sorting process receives only said spatial information.

26. A method according to claim 24, wherein said sending comprises sending such that said sorting process does not receive said shading information.

27. A method according to claim 24, wherein said sorting process is independent of said polygon memory.

28. A computer program for use in conjunction with a computer system, the computer program comprising a computer program mechanism embedded therein, the computer program mechanism, comprising:

a program module that directs the rendering of a digital representation of a final graphics image from a plurality of graphics primitives, to function in a specified manner, storing the final graphics image into a frame buffer memory, the program module including instructions for;

(i) receiving graphics data comprising graphics primitives;

(ii) converting at least some of the graphics data into a series of packets;

(iii) processing the series of packets through a plurality of graphics processes, the plurality of graphics processes being sequentially connected in a pipeline, including a first graphics process that receives the converted graphics data and a last graphics process that forms the finished rendered image; and

each graphics process comprising the steps;

(a) receiving a packet;

(b) generating a new packet for use in a packetized data transfer protocol for transferring information from graphics process to graphics process in packets, the packets each including a header portion and a data portion, the protocol used to sequentially transmit different packets having different forms and various lengths

(c) transmitting the new packet over a single communication channel from a graphics process to another graphics process while maintaining sequential order of at least some of the transferred information;

(d) generating a prefetch packet;

(e) transmitting said prefetch packet over a second communication channel to said backend process, wherein said second communication channel is shorter than said single communication channel;

(iv) storing the finished rendered image in a frame buffer; and

(v) sending the finished rendered image to an output device.

29. The computer program of claim 28, wherein the graphics processes further comprise:

(1) a cull process comprising the steps;

(a) performing a hidden surface removal process for culling out parts of the primitives that do not contribute to the finished rendered image; and

(b) generating visible portions of the primitives; and

(2) one or more graphics processes jointly comprising the steps;

(a) fragment coloring performed on the generated visible portions of primitives, to produce colored fragments; and

(b) fragment blending performed on the colored fragments.

30. The computer program of claim 28, wherein the graphics processes further comprise:

a pixel process comprising the steps;

(a) receiving the visible portions of the primitives, where each fragment has an independent color value;

(b) performing fragment operations on each sample, fragment operations comprising;

scissor test;

alpha test;

stencil test;

depth test; and

blending;

(c) blending the samples within each pixel to antialias the pixels; and

(d) outputting the antialiased pixels for use in the step of storing the finished rendered image.