Video and graphics system with parallel processing of graphics windows

• US 6,573,905 B1
• Filed: 08/18/2000
• Issued: 06/03/2003
• Est. Priority Date: 11/09/1999
• Status: Expired due to Term

• Current Assignee: Avago Technologies International Sales Pte Limited (Broadcom, Inc.)
• Original Assignee: Broadcom Corporation (Broadcom, Inc.)
• Inventors: MacInnis, Alexander G., Tang, Chengfuh Jeffrey, Xie, Xiaodong
• Primary Examiner(s): Bella, Matthew C.
• Assistant Examiner(s): Sajous, Wesner
• Application Number:
  US09/641,935
• Time in Patent Office: 1,019 Days
• Field of Search: 345/597, 345/601, 345/603, 345/604, 345/501, 345/502-503, 345/505-506, 345/519-522, 345/530-532, 345/536-538, 345/545-549, 345/555, 345/556, 345/564-567, 345/634, 345/543, 345/542, 345/619, 345/620-625, 345/629, 345/630, 345/635, 345/636, 345/592, 348/589, 348/599, 348/715, 348/597, 710/124, 710/244
• US Class Current: 345/629
• CPC Class Codes: G06F 13/00 Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units interface circuits for specific input/output devices G06F3/00 ; multiprogram control therefor G06F9/46; multiprocessor systems G06F15/16
• CPC Class Codes: G06F 15/167 using a common memory, e.g. mailbox
• CPC Class Codes: G06T 9/007 Transform coding, e.g. discrete cosine transform
• CPC Class Codes: G09G 2310/0224 Details of interlacing
• CPC Class Codes: G09G 2320/0247 Flicker reduction other than flicker reduction circuits used for single beam cathode-ray tubes
• CPC Class Codes: G09G 2340/02 Handling of images in compressed format, e.g. JPEG, MPEG
• CPC Class Codes: G09G 2340/0407 Resolution change, inclusive of the use of different resolutions for different screen areas
• CPC Class Codes: G09G 2340/10 Mixing of images, i.e. displayed pixel being the result of an operation, e.g. adding, on the corresponding input pixels
• CPC Class Codes: G09G 2340/125 wherein one of the images is motion video
• CPC Class Codes: G09G 2360/02 Graphics controller able to handle multiple formats, e.g. input or output formats
• CPC Class Codes: G09G 2360/121 using a cache memory
• CPC Class Codes: G09G 2360/125 using unified memory architecture [UMA]
• CPC Class Codes: G09G 2360/128 Frame memory using a Synchronous Dynamic RAM [SDRAM]
• CPC Class Codes: G09G 5/001 Arbitration of resources in a display system, e.g. control of access to frame buffer by video controller and/or main processor
• CPC Class Codes: G09G 5/02 characterised by the way in which colour is displayed details of colour display specific for CRTs G09G1/28; specific for flat matrix panels other than liquid crystal displays G09G3/2003; specific for liquid crystal displays G09G3/3607
• CPC Class Codes: G09G 5/024 using colour registers, e.g. to control background, foreground, surface filling G09G5/06 takes precedence
• CPC Class Codes: G09G 5/026 Control of mixing and/or overlay of colours in general G09G5/022 and G09G5/024 take precedence
• CPC Class Codes: G09G 5/06 using colour palettes, e.g. look-up tables
• CPC Class Codes: G09G 5/12 Synchronisation between the display unit and other units, e.g. other display units, video-disc players
• CPC Class Codes: G09G 5/14 Display of multiple viewports
• CPC Class Codes: G09G 5/28 : for enhancement of character form, e.g. smoothing
• CPC Class Codes: G09G 5/346 : for systems having a bit-mapped display memory
• CPC Class Codes: G09G 5/36 : characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
• CPC Class Codes: H04N 11/143 : Encoding means therefor H04N11/16 takes precedence
• CPC Class Codes: H04N 11/20 : Conversion of the manner in which the individual colour picture signal components are combined, e.g. conversion of colour television standards
• CPC Class Codes: H04N 19/42 : characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation H04N19/635 takes precedence
• CPC Class Codes: H04N 19/423 : characterised by memory arrangements H04N19/433 takes precedence
• CPC Class Codes: H04N 19/436 : using parallelised computational arrangements
• CPC Class Codes: H04N 19/44 : Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/61 : in combination with predictive coding
• CPC Class Codes: H04N 21/42204 : User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/426 : Internal components of the client ; Characteristics thereof H04N5/44 takes precedence
• CPC Class Codes: H04N 21/42653 : for processing graphics
• CPC Class Codes: H04N 21/4305 : Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets arrangements for synchronising receiver with transmitter by comparing receiver clock with transmitter clock H04L7/0012; arrangements for synchronising receiver with transmitter wherein the receiver takes measures against momentary loss of synchronisation H04L7/0083
• CPC Class Codes: H04N 21/4316 : for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/434 : Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream demultiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00
• CPC Class Codes: H04N 21/4438 : Window management, e.g. event handling following interaction with the user interface
• CPC Class Codes: H04N 21/47 : End-user applications
• CPC Class Codes: H04N 5/126 : whereby the synchronisation signal indirectly commands a frequency generator
• CPC Class Codes: H04N 5/14 : Picture signal circuitry for video frequency region cameras or camera modules comprising electronic image sensors, or control thereof H04N23/00
• CPC Class Codes: H04N 5/44504 : Circuit details of the additional information generator, e.g. details of the character or graphics signal generator, overlay mixing circuits
• CPC Class Codes: H04N 5/45 : Picture in picture , e.g. displaying simultaneously another television channel in a region of the screen
• CPC Class Codes: H04N 5/46 : for receiving on more than one standard at will deflecting circuits of multi-standard receivers H04N3/27
• CPC Class Codes: H04N 7/0135 : involving interpolation processes interpolation-based image scaling G06T3/4007; interpolation for video coding H04N19/587, H04N19/59
• CPC Class Codes: H04N 9/45 : Generation or recovery of colour sub-carriers
• CPC Class Codes: H04N 9/641 : Multi-purpose receivers, e.g. for auxiliary information H04N9/642 takes precedence
• CPC Class Codes: H04N 9/642 : Multi-standard receivers

Abstract

A display engine of a video and graphics system includes one or more processing elements and receives graphics from a memory. The graphics data define multiple graphics layers, and the processing elements process two or more graphics layers in parallel to generate blended graphics. Alpha values may be used while blending graphics. The processing elements may be integrated on an integrated circuit chip with an input for receiving the graphics data and other video and graphics components. The display engine may also include a graphics controller for receiving two or more graphics layers in parallel, for arranging the graphics layers in an order suitable for parallel processing, and for providing the arranged graphics layers to the processing elements. The blended graphics may be blended with HDTV video or SDTV video, which may be extracted from compressed data streams such as an MPEG Transport stream.

Claims

1. A display engine comprising:

an input for receiving data representing graphics from a memory, wherein the data representing graphics defines a plurality of graphics windows;

one or more processing elements capable of blending the plurality of graphics windows in parallel to generate blended graphics to be blended with video; and

a plurality of processing pipelines, each pipeline being capable of processing one of the plurality of graphics windows to place in a common format for blending in the processing elements.

2. The display engine of claim 1 wherein the input for receiving data representing graphics and said one or more processing elements are integrated on an integrated circuit chip.

3. The display engine of claim 1 further comprising:

a graphics controller for receiving the plurality of graphics windows in parallel from the processing pipelines, for arranging the graphics windows in an order suitable for parallel processing, and for providing the arranged graphics windows to the one or more processing elements.

4. The display engine of claim 1, wherein each pipeline monitors whether its graphics window is in range for blending, and if it is not in the range, puts out a pixel equivalent to a transparent one so that it will have no effect on a net output when blended with graphics windows from other pipelines.

5. The display engine of claim 1, wherein each graphics window comprises a plurality of pixels, and the graphics windows are processed in the processing pipelines in parallel and in synchronization such that the graphics windows are aligned to each other pixel by pixel in the processing pipelines.

6. The display engine of claim 1, wherein if a particular processing pipeline is not ready, then all pipelines stall and wait for the particular pipeline to become ready again.

7. The display engine of claim 3, wherein the graphics controller synchronizes the processing pipelines and stalls the pipelines if necessary so that the graphics windows processed in the processing pipelines are aligned in order to be blended together in the processing elements.

8. The display engine of claim 3, wherein the graphics controller orders the graphics windows in accordance with a window depth order.

9. The display engine of claim 3, wherein each processing pipeline comprises a graphics converter for converting the format of one of the plurality of graphics windows to the common format, and for providing it to the graphics controller.

10. The display engine of claim 9 wherein each processing pipeline comprises a CLUT for converting the format of said one of the plurality of graphics windows from a CLUT format to the common format.

11. The display engine of claim 1 further comprising:

a graphics filter for receiving and spatially processing the blended graphics prior to compositing the blended graphics with the video.

12. The display engine of claim 11 wherein the spatial processing includes filtering.

13. The display engine of claim 11 wherein the spatial processing includes scaling.

14. The display engine of claim 1 wherein a plurality of alpha values are used to blend the plurality of graphics windows to generate the blended graphics.

15. The display engine of claim 14 wherein the plurality of alpha values include at least one alpha value per each of the plurality of graphics windows.

16. The display engine of claim 15 wherein the plurality of alpha values include at least one alpha value per each of the plurality of pixels.

17. The display engine of claim 14 wherein the plurality of alpha values are blended to generate at least one composite alpha value.

18. The display engine of claim 11 further comprising a plurality of line buffers, and wherein the blended graphics include a plurality of lines of pixels, and each of the plurality of lines of pixels is loaded into one of the plurality of line buffers to be provided to the graphics filter.

19. The display engine of claim 1 wherein the one or more processing elements include three graphics blenders, and wherein at least two graphics windows are blended together in each of first two of the graphics blenders in parallel, and outputs of the first two of the graphics blenders are provided to the last graphics blender for blending to generate the blended graphics.

20. A method of blending a plurality of graphics windows to generate blended graphics comprising the steps of:

receiving data representing graphics, the data representing graphics defining the plurality of graphics windows, wherein each graphics window comprises a plurality of pixels;

processing the graphics windows, each graphics window in one of processing pipelines, in parallel and in synchronization such that the graphics windows are aligned to each other pixel by pixel in the processing pipelines; and

blending two or more of the plurality of graphics windows in parallel to generate the blended graphics to be blended with video.

21. The method of blending a plurality of graphics windows of claim 20 further comprising the step of:

arranging two or more graphics windows in an order suitable for parallel processing.

22. The method of blending a plurality of graphics windows of claim 21 further comprising the step of:

converting formats of the plurality of graphics windows to a common format.

23. The method of blending a plurality of graphics windows of claim 22 wherein the step of converting the format of the plurality of graphics windows to a common format comprises the step of converting the format of the plurality of graphics windows from one or more CLUT formats to the common format.

24. The method of blending a plurality of graphics windows of claim 20 further comprising the step of spatially processing the blended graphics.

25. The method of blending a plurality of graphics windows of claim 24 wherein the step of spatially processing the blended graphics comprises the step of filtering the blended graphics.

26. The method of blending a plurality of graphics windows of claim 24 wherein the step of spatially processing the blended graphics comprises the step of scaling the blended graphics.

27. The method of blending a plurality of graphics windows of claim 20 wherein the step of blending two or more of the plurality of graphics windows in parallel comprises the step of blending two or more of the plurality of graphics windows in parallel using a plurality of alpha values.

28. The method of blending a plurality of graphics windows of claim 27 wherein the plurality of alpha values include at least one alpha value per each of the plurality of graphics windows.

29. The method of blending a plurality of graphics windows of claim 28 wherein each of the plurality of graphics windows includes a plurality of pixels, and the plurality of alpha values include at least one alpha value per each of the plurality of pixels.

30. The method of blending a plurality of graphics windows of claim 28 further comprising the step of:

blending the plurality of alpha values to generate at least one composite alpha value.

31. A system comprising:

a transport processor for receiving a plurality of compressed data streams, at least one of the plurality of compressed data streams including video data;

a video decoder for decoding the video data to generate decoded video data;

a display engine for receiving a plurality of graphics windows and for blending them in parallel to generate blended graphics; and

a video compositor for blending the decoded video data with the blended graphics.

32. The system of claim 31 wherein the transport processor, the video decoder, the display engine and the video compositor are integrated on an integrated circuit chip.

33. The system of claim 31 wherein the plurality of compressed data streams include one or more MPEG Transport streams, and the transport processor includes an MPEG Transport processor.

34. The system of claim 31 wherein the video data include MPEG-2 video data and the video decoder includes an MPEG-2 video decoder.

35. The system of claim 31 wherein the video data include SDTV video data.

36. The system of claim 31 wherein the video data include HDTV video data.