Apparatus, systems and methods for controlling graphics and video data in multimedia data processing and display systems

• US RE39,898 E1
• Filed: 08/13/1999
• Issued: 10/30/2007
• Est. Priority Date: 01/23/1995
• Status: Expired due to Term

• Current Assignee: NVIDIA International Inc. (NVIDIA Corporation)
• Original Assignee: NVIDIA International Inc. (NVIDIA Corporation)
• Inventors: Nally, Robert M., Schafer, John C.
• Primary Examiner(s): Tung; Kee M.
• Application Number:
  US09/374,041
• Time in Patent Office: 3,000 Days
• Field of Search: 345501-506, 345/519, 345/520, 345/531, 345/535, 345545-548, 345/558, 345/629, 345/788, 345/790, 345/559, 345/589, 348/510, 348/714, 348/717, 348/719, 348/584
• US Class Current: 345/546
• CPC Class Codes: G06T 11/203 Drawing of straight lines or curves

Abstract

A graphics and video controller 105 is provided which includes a dual aperture interface 206 for receiving words of graphics and video pixel data, each word of such data associated with an address directing that word to be processed as either graphics or video data. Circuitry 200, 201, 202, 207, 208 is provided for writing a word of the pixel data received from the interface 206 to a one of the on- and off-screen memory areas corresponding to the address associated with the received word. Circuitry 201, 202 is provided for selectively retrieving graphics and video data from the on-screen and off-screen memory areas. A first pipeline 205 is provided for processing data received from the on-screen area of frame buffer 107 while a second pipeline 204 is provided for processing data retrieved from the off-screen area of the frame buffer.

Claims

1. A single integrated graphics and video controller adapted for driving a display sequentially comprising:

an interface for receiving words of pixel data, each said word associated with an address buffer;

circuitry for writing each said word of said pixel data received by said interface to a one of on-screen and off-screen memory areas of a frame buffer;

circuitry for selectively retrieving, as data is provided for display, said words from said on-screen and off-screen areas;

a first pipeline for substantially continuously processing words of graphics data retrieved from said frame buffer; and

a second pipeline for processing words of video data retrieved from said frame buffer so that the video data is ready for display once a display raster scan reaches a display position of a window.

2. The controller of claim 1 and further comprising output selection circuitry for selecting for output between graphics data received from said first pipeline and data received from said second pipeline, said selection circuitry operable to:

in a first mode, pass data from said first pipeline; and

in a second mode, pass data from said second pipeline when said data corresponds to a selected display position of a display window.

3. The controller of claim 2 wherein said selection circuitry is further operable to:

in a third mode, pass data from said second pipeline when said data corresponds to said selected display position of said display window and data from said first pipeline match a color key.

4. The controller of claim 3 wherein said selection circuitry is further operable in a fourth mode to pass data from said second pipeline when data from said first pipeline match a color key.

5. The controller of claim 1 wherein said circuitry for retrieving maintains a stream of graphics data to said first pipeline and provides video data to said second pipeline when a display raster scan reaches a display position of a window.

6. The controller of claim 1 and further comprising:

a video port for receiving real-time video data; and

circuitry for generating an address to said memory at which said real-time video data is to be stored.

7. The controller of claim 1 wherein said second pipeline includes a first first-in-first-out memory for receiving data for a first display line of pixels in memory and a second first-in-first-out memory for receiving data from a second display line of pixels in memory.

8. The controller of claim 7 wherein said first display line is adjacent in memory to said second display line.

9. The controller of claim 7 further comprising output selection circuitry, wherein said output selection circuitry comprises:

an output selector for selecting between data from said second pipeline and data from said first pipeline in response to a selection control signal;

a register for maintaining a plurality of overlay control bits;

window position control circuitry for selectively generating a position control signal when a word of said data stream from said second pipeline falls within a display window;

color comparison circuitry for comparing words of said data stream from said first pipeline with a color key and for providing in response a color comparison control signal; and

a control selector for selectively providing said selection control signal in response to said overlay control bits in said register and at least one of said position control and color comparison control signals.

10. The controller of claim 9 wherein said window position control circuitry comprises:

window position counters operable to increment from initial count values corresponding to a starting pixel of a display window as data representing each pixel in a display screen is pipelined through said overlay control circuitry;

screen position counters operable to count as data representing each pixel in said display screen is pipelined through said overlay control circuitry; and

comparison circuitry operable to compare a current count in said window position counters and a current count in said screen position counters and selectively generate said position control signal in response.

11. The controller of claim 9 wherein said color comparison circuitry comprises:

a color key register for storing bits composing said color key; and

a plurality of AND-gates for comparing said words of said graphics data stream with bits of said color key.

12. The controller of claim 1 wherein said interface includes a dual-aperture port.

13. A single integrated controller comprising:

circuitry for writing selectively, on a word by word basis, each word of received data into a selected one of on-screen and off-screen memory spaces of a frame buffer;

a first port for receiving video and graphics data, a word of said data received with an address of said memory spaces directing said word to be processed as a word of video data or a word of graphics data;

a second port for receiving real-time video data;

circuitry for generating an address associated with a selected one of said memory spaces for a word of said real-time video data;

circuitry for selectively retrieving said words of data on a word by word basis from said on-screen and off-screen memory spaces as data is rastered for driving a display in a sequential fashion;

a graphics backend pipeline for processing ones of said words of data representing graphics data retrieved from said frame buffer;

a video backend pipeline, separate from said graphics backend pipeline, for processing other ones of said words of data representing video data retrieved from said frame buffer, said circuitry for retrieving always rastering a stream of data from said frame buffer to said graphics backend pipeline and rastering video data to said video backend pipeline when so that the video data is ready for display once a display raster scan reaches a display position of a window; and

output selector circuitry for selecting for output between words of data output from said graphics backend pipeline and words of data output from said video backend pipeline.

14. The controller of claim 13 wherein said output selector circuitry is further operable to select between graphics data output from a color look-up table and true color data output from said graphics pipeline.

15. The controller of claim 13 wherein said output selector circuitry is operable to:

in a first mode, pass only a word of data output from said graphics pipeline;

in a second mode, pass a word of data output from said video pipeline when said display raster scan has reached a display position corresponding to a window and a word of data from said graphics pipeline when said display raster scan is in any other display position;

in a third mode, pass a word of data output from said video pipeline when said display raster scan has reached a display position corresponding to a window and a corresponding word of data from said graphics pipeline matches a color key and a word of data from said graphics pipeline when said display raster scan is in any other display position; and

in a fourth mode, pass a word of data from said video pipeline when said corresponding word of data from said graphics pipeline matches a color key and a word of data from said graphics pipeline when said display raster scan is in any other display position corresponding word does not match said color key.

16. The controller of claim 13 wherein said video pipeline includes a first first-in-first-out memory for receiving a plurality of words of data for a first display line of pixels in memory and a second first-in-first-out memory for receiving a plurality of words of data from for a second display line of pixels in memory.

17. The controller of claim 16 wherein said first display line is stored adjacent in memory to said second display line.

18. The controller of claim 13 wherein said output selector circuitry comprises:

a control selector having a plurality of control inputs coupled to a register, said register storing a plurality of overlay control bits;

window position control circuitry coupled to a first control input of said control selector, said window position control circuitry operable to selectively provide a first control signal to said first control input when a word of data being pipelined through said video pipeline falls within a display window;

color comparison circuitry operable to compare a word of data being pipelined through said graphics pipeline with a color key and provide in response a second control signal to a second control input of said control selector; and

wherein said control selector is operable to provide an output selection control signal in response to at least one of said first and second control signals and said overlay control bits being stored in said register.

19. The circuitry of claim 18 wherein said output selector circuitry further includes a third control input coupled to certain bits of said graphics pipeline, said output selector circuitry further operable to select between data on said respective video and graphics pipelines in response to said certain bits presented to said selector circuitry.

20. The circuitry of claim 18 wherein said window position control circuitry comprises:

a window x-position counter operable to count from a loaded x-position value in response to a video clock, said x-position counter reloading in response to a display horizontal synchronization signal;

a window y-position counter operable to count from a loaded y-position value in response to said horizontal synchronization signal, said y-position counter reloading in response to a display vertical synchronization signal;

CRT position circuitry operable to maintain counts corresponding to a current display pixel; and

comparison circuitry operable to compare current counts in said window counters with said current counts held in said CRT position circuitry and generate in response said first control signal.

21. The circuitry of claim 20 wherein said window position control circuitry further comprises an x-position register for holding said x-position value for loading into said x-position counter and a y-position register for holding said y-position value for loading into said y-position counter.

22. The circuitry of claim 13 wherein said color comparison circuitry comprises:

a color key register for storing a plurality of color key bits; and

a plurality of XNOR-gates each having at least one input coupled to a bit position in said color key register and at least one input coupled to said graphics data path.

23. The circuitry of claim 13 wherein said video pipeline comprises:

a first-in/first-out memory for receiving a first stream of words of data from said frame buffer;

a second first-in/first-out memory disposed in parallel with said first first-in/first-out memory for receiving a second stream of words of data from said frame buffer; and

interpolation circuitry for selectively generating an additional word of data by interpolating a word of said first stream and a word of said second stream data output from said first and second first-in/first-out memories.

24. The controller of claim 13 wherein said first port comprises a dual-aperture port.

25. A display system comprising:

a first backend pipeline for processing data;

a second backend pipeline for processing graphics data disposed in parallel to said first processing pipeline;

a multi-format frame buffer memory having on-screen and off-screen areas each operable to allow said frame buffer to simultaneously store data in graphics and video formats;

an input port for receiving both graphics and video data, each word of said data associated with an address directing said word to be processed as either graphics or video data;

circuitry for writing a word of said playback graphics and video data into a selected one of said areas of said multi-format memory;

memory control circuitry for controlling the transfer of data between said first backend pipeline and said frame buffer and between said second backend pipeline and said frame buffer;

a display unit; and

overlay control circuitry for selecting for output to said display unit between data provided by said first backend pipeline and data provided by said second backend pipeline.

26. The display system of claim 25 wherein said second backend pipeline includes:

a first first-in-first-out memory for receiving first selected data;

a second first-in-first-out memory disposed in parallel to said first first-in-first-out memory for receiving second selected data; and

interpolation data for generating additional data by interpolating data output from said respective first and second first-in-first-out memories.

27. The display system of claim 26 wherein said second backend pipeline further comprises color conversion circuitry for converting data received from said frame buffer in a video format to a graphics format.

28. The display system of claim 25 further comprising a video front-end pipeline for inputting video data into a selected one of on-screen and off-screen spaces of said frame buffer comprising:

a video data port for receiving video data from a real time data source; and

input control circuitry for receiving framing signals associated with said real time data and generating corresponding addresses to said selected one of said spaces in response.

29. The display system of claim 28 wherein said video front-end pipeline further comprises encoding circuitry for packing said video data prior to storage in said selected one of said spaces.

30. The display system of claim 28 wherein said video front-end pipeline further comprises multiplexing circuitry for selecting between video data received through said video data port and data received from a dual aperture port.

31. The display system of claim 28 wherein said video front end pipeline further comprises conversion circuitry for converting graphics data received through said dual-aperture port to a video format for storage in said selected one of said spaces.

32. The display system of claim 25 wherein said first backend pipeline processes playback video.

33. The display system of claim 25 wherein said input port comprises a dual-aperture input port.

34. A single integrated display data processing system comprising:

circuitry for writing data into an on-screen space of a frame buffer;

circuitry for writing data into an off-screen space of said frame buffer;

a video pipeline for processing data output from a selected one of said on-screen and off-screen spaces comprising;

a first first-in-first-out memory for receiving selected data from said selected space;

a second first-in-first-out memory disposed in parallel to said first first-in-first-out memory for receiving other selected data from said selected space; and

an interpolator for generating additional data by interpolating data output from said respective first and second first-in-first-out memories;

a graphics pipeline disposed in parallel to and separate from said video pipeline for processing data output from a selected one of said on-screen and off-screen spaces; and

an output selector for selecting between data output from said video pipeline and data output from said graphics pipeline.

35. The system of claim 34 further comprising selection control circuitry for generating an output control signal for controlling said output selector comprising:

a control selector having a plurality of data inputs coupled to a register, said register for storing a plurality of overlay control bits; and

color comparison circuitry operable to compare bits of data output from said graphics pipeline with a color key and provide in response a control signal to a control input of said control selector.

36. The system of claim 34 further comprising window position control circuitry operable to provide a second control signal to a second control input of said control selector when data from said video pipeline falls within a display window.

37. A single integrated display controller comprising:

circuitry for selectively retrieving data from an associated multi-format frame buffer, the frame buffer having separate storage locations respectively operable, for allowing simultaneously storing graphics and video data in said frame buffer;

a first pipeline for processing words of graphics data selectively retrieved from said frame buffer; and

a second pipeline, separate from the first pipeline, for processing words of video data selectively retrieved from said frame buffer.

38. The controller of claim 37 wherein said first and second pipelines are disposed in parallel and concurrently process data.

39. The controller of claim 38 further comprising output selection circuitry for selecting for output between graphics data received from said first pipeline and video data received from said second pipeline.

40. The controller of claim 37 wherein said frame buffer is partitioned into on-screen and off-screen areas, each of said on-screen and off-screen areas operable to allow the buffer to simultaneously store both graphics and video data.

41. The controller of claim 37 wherein said circuitry for selectively retrieving is operable to retrieve a substantially constant stream of graphics data from said frame buffer and provide said stream of graphics data to said first pipeline.

42. The controller of claim 41 wherein said circuitry for selectively retrieving is operable to retrieve at least one said word of video data from said frame buffer and provide said at least one word of said video data to said second pipeline, only when said display controller is generating a video display window.

48. The controller of claim 42 in which graphics data is substantially continuously retrieved for a display while video data is retrieved only for the video display window.

43. A display controller for interfacing a multi-format frame buffer and a display device, the multi-format frame buffer having on-screen and off-screen areas each operable for allowing simultaneously storing both graphics and video pixel data in the frame buffer, said display controller comprising:

circuitry for selectively retrieving pixel data from a selected one of said on-screen and off-screen areas of said frame buffer;

a graphics backend pipeline for processing graphics data retrieved from said selected one of said areas of said frame buffer;

a video backend pipeline for processing video data retrieved from said selected one of said areas of said frame buffer; and

an output selector for selectively passing to said display device data received from said graphics or video backend pipelines.

44. The display controller of claim 43 wherein said circuitry for selectively retrieving is operable to retrieve at least one said word of video data from said frame buffer and provide said at least one said word of video data to said second pipeline only when said display controller is generating a video display window.

45. The display controller of claim 43 wherein said output selector is operable to:

in a first mode pass data from said graphics pipeline; and

in a second mode pass data from said video pipeline when a display position corresponding to a video display window has been reached.

46. The display window of claim 43 wherein said output selector is operable to:

in a first mode, pass data from said graphics pipeline; and

in a second mode, pass data from said video pipeline when a display position corresponding to a video display window has been reached and data from said graphics pipeline match a color key.

47. The display controller of claim 43 wherein said output selector is operable to:

in a first mode, pass data from said graphics pipeline; and

in a second mode, pass data from said video pipeline when data from said graphics pipeline matches a color key.