Video, audio and graphics decode, composite and display system

• US 6,853,385 B1
• Filed: 08/18/2000
• Issued: 02/08/2005
• Est. Priority Date: 11/09/1999
• Status: Expired due to Term

• Current Assignee: Avago Technologies International Sales Pte Limited (Broadcom, Inc.)
• Original Assignee: Broadcom Corporation (Broadcom, Inc.)
• Inventors: MacInnis, Alexander G., Cheung, Francis, Hsiun, Vivian, Tang, Chengfuh Jeffrey, Bhatia, Sandeep, Kranawetter, Greg A., Xie, Xiaodong, Valmiki, Ramanujan, Kumar, Sathish
• Primary Examiner(s): Tung, Kee M.
• Assistant Examiner(s): SaJous, Wesner
• Application Number:
  US09/641,374
• Time in Patent Office: 1,635 Days
• Field of Search: 345/636, 345/694, 345/643, 345/698, 345/660, 345/671, 345/501, 345/503, 345505-506, 345/519, 345/549, 345545-546, 345554-556, 345/560, 345/567, 345629-630, 348/441, 348/446, 348447-448, 348455-457, 348459-461, 348/490, 348/492, 348/497, 348/701, 348624-630, 348666-668, 348/711, 348/581, 348/453, 348/384, 725 37- 41, 725/70, 725/105, 725/144, 725114-116, 725/138, 725/87, 725/63, 370/240.01, 370/389, 370/476, 386/95, 386/119, 386108-113, 710/52, 710/62, 710 68- 74, 711/104, 711/105, 711/118
• US Class Current: 345/629
• CPC Class Codes: G06T 9/007 Transform coding, e.g. discrete cosine transform
• CPC Class Codes: G09G 2310/0224 Details of interlacing
• CPC Class Codes: G09G 2320/0247 Flicker reduction other than flicker reduction circuits used for single beam cathode-ray tubes
• CPC Class Codes: G09G 2340/02 Handling of images in compressed format, e.g. JPEG, MPEG
• CPC Class Codes: G09G 2340/0407 Resolution change, inclusive of the use of different resolutions for different screen areas
• CPC Class Codes: G09G 2340/10 Mixing of images, i.e. displayed pixel being the result of an operation, e.g. adding, on the corresponding input pixels
• CPC Class Codes: G09G 2340/125 wherein one of the images is motion video
• CPC Class Codes: G09G 2350/00 Solving problems of bandwidth in display systems
• CPC Class Codes: G09G 2360/02 Graphics controller able to handle multiple formats, e.g. input or output formats
• CPC Class Codes: G09G 2360/121 using a cache memory
• CPC Class Codes: G09G 2360/125 using unified memory architecture [UMA]
• CPC Class Codes: G09G 2360/126 The frame memory having additional data ports, not inclusive of standard details of the output serial port of a VRAM
• CPC Class Codes: G09G 2360/128 Frame memory using a Synchronous Dynamic RAM [SDRAM]
• CPC Class Codes: G09G 5/001 Arbitration of resources in a display system, e.g. control of access to frame buffer by video controller and/or main processor
• CPC Class Codes: G09G 5/008 Clock recovery
• CPC Class Codes: G09G 5/02 characterised by the way in which colour is displayed details of colour display specific for CRTs G09G1/28; specific for flat matrix panels other than liquid crystal displays G09G3/2003; specific for liquid crystal displays G09G3/3607
• CPC Class Codes: G09G 5/024 using colour registers, e.g. to control background, foreground, surface filling G09G5/06 takes precedence
• CPC Class Codes: G09G 5/026 Control of mixing and/or overlay of colours in general G09G5/022 and G09G5/024 take precedence
• CPC Class Codes: G09G 5/06 using colour palettes, e.g. look-up tables
• CPC Class Codes: G09G 5/12 Synchronisation between the display unit and other units, e.g. other display units, video-disc players
• CPC Class Codes: G09G 5/14 : Display of multiple viewports
• CPC Class Codes: G09G 5/28 : for enhancement of character form, e.g. smoothing
• CPC Class Codes: G09G 5/346 : for systems having a bit-mapped display memory
• CPC Class Codes: G09G 5/36 : characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
• CPC Class Codes: G09G 5/363 : Graphics controllers
• CPC Class Codes: H04L 12/28 : characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks] wireless communication networks H04W ; arrangements for dividing the transmission path H04W40/00
• CPC Class Codes: H04N 11/00 : Colour television systems details H04N9/00
• CPC Class Codes: H04N 11/143 : Encoding means therefor H04N11/16 takes precedence
• CPC Class Codes: H04N 11/20 : Conversion of the manner in which the individual colour picture signal components are combined, e.g. conversion of colour television standards
• CPC Class Codes: H04N 11/22 : in which simultaneous signals are converted into sequential signals or vice versa
• CPC Class Codes: H04N 21/426 : Internal components of the client ; Characteristics thereof H04N5/44 takes precedence
• CPC Class Codes: H04N 21/42653 : for processing graphics
• CPC Class Codes: H04N 21/42692 : for reading from or writing on a volatile storage medium, e.g. Random Access Memory [RAM]
• CPC Class Codes: H04N 21/4305 : Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets arrangements for synchronising receiver with transmitter by comparing receiver clock with transmitter clock H04L7/0012; arrangements for synchronising receiver with transmitter wherein the receiver takes measures against momentary loss of synchronisation H04L7/0083
• CPC Class Codes: H04N 21/4316 : for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/434 : Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream demultiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00
• CPC Class Codes: H04N 21/4345 : Extraction or processing of SI, e.g. extracting service information from an MPEG stream
• CPC Class Codes: H04N 21/4348 : Demultiplexing of additional data and video streams
• CPC Class Codes: H04N 21/440218 : by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4 conversion of standards in analogue television systems H04N7/01
• CPC Class Codes: H04N 21/4435 : Memory management allocation of memory to service a request G06F9/5016; addressing or allocating within memory systems or architectures G06F12/02
• CPC Class Codes: H04N 21/8146 : involving graphical data, e.g. 3D object, 2D graphics
• CPC Class Codes: H04N 5/14 : Picture signal circuitry for video frequency region cameras or camera modules comprising electronic image sensors, or control thereof H04N23/00
• CPC Class Codes: H04N 5/44504 : Circuit details of the additional information generator, e.g. details of the character or graphics signal generator, overlay mixing circuits
• CPC Class Codes: H04N 5/45 : Picture in picture , e.g. displaying simultaneously another television channel in a region of the screen
• CPC Class Codes: H04N 5/46 : for receiving on more than one standard at will deflecting circuits of multi-standard receivers H04N3/27
• CPC Class Codes: H04N 7/0135 : involving interpolation processes interpolation-based image scaling G06T3/4007; interpolation for video coding H04N19/587, H04N19/59
• CPC Class Codes: H04N 9/45 : Generation or recovery of colour sub-carriers
• CPC Class Codes: H04N 9/641 : Multi-purpose receivers, e.g. for auxiliary information H04N9/642 takes precedence

Abstract

A video, audio and graphics system uses multiple transport processors to receive in-band and out-of-band MPEG Transport streams, to perform PID and section filtering as well as DVB and DES decryption and to de-multiplex them. The system processes the PES into digital audio, MPEG video and message data. The system is capable of decoding multiple MPEG SLICEs concurrently. Graphics windows are blended in parallel, and blended with video using alpha blending. During graphics processing, a single-port SRAM is used equivalently as a dual-port SRAM. The video may include both analog video, e.g., NTSC/PAL/SECAM/S-video, and digital video, e.g., MPEG-2 video in SDTV or HDTV format. The system has a reduced memory mode in which video images are reduced in half in horizontal direction only during decoding. The system is capable of receiving and processing digital audio signals such as MPEG Layer 1 and Layer 2 audio and Dolby AC-3 audio, as well as PCM audio signals. The system includes a memory controller. The system includes a system bridge controller to interface a CPU with devices internal to the system as well as peripheral devices including PCI devices and I/O devices such as RAM, ROM and flash memory devices. The system is capable of displaying video and graphics in both the high definition (HD) mode and the standard definition (SD) mode. The system may output an HDTV video while converting the HDTV video and providing as another output having an SDTV format or another HDTV format.

Claims

1. An integrated circuit comprising:

an input for receiving analog video;

an input for receiving data representing graphics;

an input for receiving a plurality of compressed data streams, the compressed data streams including video data;

an analog video decoder for decoding the analog video to generate digitized analog video;

a display engine for blending the data representing graphics to generate blended graphics;

a digital video decoder for processing the video data to generate decoded video data; and

a video compositor for blending the digitized analog video, the blended graphics and the decoded video data to provide an output of blended video and graphics,

wherein the plurality of compressed data streams include one or more MPEG Transport streams, and

wherein the one or more MPEG Transport streams include two in-band Transport streams and one out-of-band Transport stream.

2. An integrated circuit comprising:

an input for receiving analog video;

an input for receiving data representing graphics;

an input for receiving a plurality of compressed data streams, the compressed data streams including video data;

an analog video decoder for decoding the analog video to generate digitized analog video;

a display engine for blending the data representing graphics to generate blended graphics;

a digital video decoder for processing the video data to generate decoded video data;

a video compositor for blending the digitized analog video, the blended graphics and the decoded video data to provide an output of blended video and graphics; and

a system bridge controller for coupling a CPU to one or more peripheral devices,

wherein the one or more peripheral devices include one or more memory devices, and wherein the system bridge controller includes an I/O bus bridge to couple the CPU to the one or more memory devices.

3. An integrated circuit comprising:

an input for receiving analog video;

an input for receiving data representing graphics;

an input for receiving a plurality of compressed data streams, the compressed data streams including video data;

an analog video decoder for decoding the analog video to generate digitized analog video;

a display engine for blending the data representing graphics to generate blended graphics;

a digital video decoder for processing the video data to generate decoded video data;

a video compositor for blending the digitized analog video, the blended graphics and the decoded video data to provide an output of blended video and graphics; and

a system bridge controller for coupling a CPU to one or more peripheral devices,

wherein the system bridge controller includes a CPU interface block to couple the CPU to one or more devices internal to the integrated circuit.

4. An integrated circuit comprising:

an input for receiving analog video;

an input for receiving data representing graphics;

an input for receiving a plurality of compressed data streams, the compressed data streams including video data;

an analog video decoder for decoding the analog video to generate digitized analog video;

a display engine for blending the data representing graphics to generate blended graphics;

a digital video decoder for processing the video data to generate decoded video data;

a video compositor for blending the digitized analog video, the blended graphics and the decoded video data to provide an output of blended video and graphics; and

a core transport processor for accepting or rejecting data packets from the plurality of compressed data streams based on the value of a packet identification field of each data packet,

wherein the core transport processor is used to extract message data from the plurality of compressed data streams based on the value of message headers for the message data.

5. An integrated circuit comprising:

an input for receiving analog video;

an input for receiving data representing graphics;

an input for receiving a plurality of compressed data streams, the compressed data streams including video data;

an analog video decoder for decoding the analog video to generate digitized analog video;

a display engine for blending the data representing graphics to generate blended graphics;

a digital video decoder for processing the video data to generate decoded video data; and

a video compositor for blending the digitized analog video, the blended graphics and the decoded video data to provide an output of blended video and graphics,

wherein the video data include MPEG video data, and the digital video decoder scales the MPEG video data during a decoding process in a horizontal direction only.

6. The integrated circuit of claim 5 wherein the digital video decoder does not scale the MPEG video data during the decoding process when bi-directionally predicted pictures are not used.

7. An integrated circuit comprising:

an input for receiving analog video;

an input for receiving data representing graphics;

an input for receiving a plurality of compressed data streams, the compressed data streams including video data;

an analog video decoder for decoding the analog video to generate digitized analog video;

a display engine for blending the data representing graphics to generate blended graphics;

a digital video decoder for processing the video data to generate decoded video data; and

a video compositor for blending the digitized analog video, the blended graphics and the decoded video data to provide an output of blended video and graphics,

wherein the plurality of compressed data streams include one or more MPEG Transport streams,

wherein the video data include MPEG video data included in at least one MPEG Transport stream, and the integrated circuit further comprises a satellite transport processor to extract the MPEG video data, and a memory controller for accessing the MPEG video data in one of pre-defined orders that are efficient for MPEG decoding, and

wherein the digital video decoder is an MPEG video decoder for decoding the MPEG video data.

8. An integrated circuit comprising:

an input for receiving analog video;

an input for receiving data representing graphics;

an input for receiving a plurality of compressed data streams, the compressed data streams including video data;

an analog video decoder for decoding the analog video to generate digitized analog video;

a display engine for blending the data representing graphics to generate blended graphics;

a digital video decoder for processing the video data to generate decoded video data; and

a video compositor for blending the digitized analog video, the blended graphics and the decoded video data to provide an output of blended video and graphics,

wherein the data representing graphics include a plurality of graphics layers, and the display engine blends the plurality of graphics layers in parallel.

9. The integrated circuit of claim 8 further comprising a window controller to provide the plurality of graphics layers to the display engine in an order suitable for blending.

10. An integrated circuit comprising:

an input for receiving analog video;

an input for receiving data representing graphics;

an input for receiving a plurality of compressed data streams, the compressed data streams including video data;

an analog video decoder for decoding the analog video to generate digitized analog video;

a display engine for blending the data representing graphics to generate blended graphics;

a digital video decoder for processing the video data to generate decoded video data; and

a video compositor for blending the digitized analog video, the blended graphics and the decoded video data to provide an output of blended video and graphics,

wherein the single-port SRAM is used to implement one or more line buffers to store the blended graphics.

11. An integrated circuit comprising:

an input for receiving analog video;

an input for receiving data representing graphics;

an input for receiving a plurality of compressed data streams, the compressed data streams including video data;

an analog video decoder for decoding the analog video to generate digitized analog video;

a display engine for blending the data representing graphics to generate blended graphics;

a digital video decoder for processing the video data to generate decoded video data;

a video compositor for blending the digitized analog video, the blended graphics and the decoded video data to provide an output of blended video and graphics; and

a scaler,

wherein the video data include HDTV video data, and wherein the HDTV data is processed and provided in HDTV format as an HDTV output, while the scaler scales the processed HDTV data to have SDTV format and provides the processed and scaled HDTV data simultaneously as an SDTV output.

12. The integrated circuit of claim 11 wherein the processed and scaled HDTV data is stored in memory prior to being provided as the SDTV output.

13. A method of processing video and graphics in an integrated circuit comprising the steps of:

receiving an analog video;

receiving data representing graphics;

receiving a plurality of compressed data streams, the compressed data streams including video data;

decoding the analog video to generate digitized analog video;

blending the data representing graphics to generate blended graphics;

processing the video data to generate decoded video data; and

blending the digitized analog video, the blended graphics and the decoded video data to provide an output of blended video and graphics,

wherein the step of receiving a plurality of compressed data streams comprises the step of receiving one or more MPEG Transport streams, and

wherein the step of receiving one or more MPEG Transport streams comprises the step of receiving two in-band Transport streams and one out-of-band Transport stream.

14. A method of processing video and graphics in an integrated circuit comprising the steps of:

receiving an analog video;

receiving data representing graphics;

receiving a plurality of compressed data streams, the compressed data streams including video data;

decoding the analog video to generate digitized analog video;

blending the data representing graphics to generate blended graphics;

processing the video data to generate decoded video data;

blending the digitized analog video, the blended graphics and the decoded video data to provide an output of blended video and graphics; and

extracting message data from the plurality of compressed data streams based on the value of message headers for the message data.

15. A method of processing video and graphics in an integrated circuit comprising the steps of:

receiving an analog video;

receiving data representing graphics;

receiving a plurality of compressed data streams, the compressed data streams including video data;

decoding the analog video to generate digitized analog video;

blending the data representing graphics to generate blended graphics;

processing the video data to generate decoded video data; and

blending the digitized analog video, the blended graphics and the decoded video data to provide an output of blended video and graphics,

wherein the video data include MPEG video data, and wherein the step of processing the video data comprises the step of scaling the MPEG video data in a horizontal direction only.

16. The method of blending video and graphics in an integrated circuit of claim 15 wherein the MPEG video data is not scaled when B-pictures are not used.

17. A method of processing video and graphics in an integrated circuit comprising the steps of:

receiving an analog video;

receiving data representing graphics;

receiving a plurality of compressed data streams, the compressed data streams including video data;

decoding the analog video to generate digitized analog video;

blending the data representing graphics to generate blended graphics;

processing the video data to generate decoded video data; and

blending the digitized analog video, the blended graphics and the decoded video data to provide an output of blended video and graphics,

wherein the data representing graphics include a plurality of graphics layers, and wherein the step of blending the data representing graphics comprises the step of blending the plurality of graphics layers in parallel.

18. The method of blending video and graphics in an integrated circuit of claim 17 further comprising the step of ordering the plurality of graphics layers in an order suitable for blending.

19. A method of processing video and graphics in an integrated circuit comprising the steps of:

receiving an analog video;

receiving data representing graphics;

receiving a plurality of compressed data streams, the compressed data streams including video data;

decoding the analog video to generate digitized analog video;

blending the data representing graphics to generate blended graphics;

processing the video data to generate decoded video data;

blending the digitized analog video, the blended graphics and the decoded video data to provide an output of blended video and graphics;

configuring a single-port SRAM equivalently as a dual-port SRAM; and

storing the blended graphics in the single-port SRAM configured equivalently as a dual-port SRAM.

20. A method of processing video and graphics in an integrated circuit comprising the steps of:

receiving an analog video;

receiving data representing graphics;

receiving a plurality of compressed data streams, the compressed data streams including video data;

decoding the analog video to generate digitized analog video;

blending the data representing graphics to generate blended graphics;

processing the video data to generate decoded video data; and

blending the digitized analog video, the blended graphics and the decoded video data to provide an output of blended video and graphics,

wherein the video data include HDTV video data,

wherein the step of processing the video data comprises the step of processing the HDTV data, and

wherein the method further comprising the steps of;

providing the processed HDTV in HDTV format as an HDTV output;

scaling the processed HDTV data to have SDTV format; and

providing the processed and scaled HDTV data simultaneously as an SDTV output.

21. The method of blending video and graphics in an integrated circuit of claim 20 further comprising the step of storing the processed and scaled HDTV data in memory prior to providing it as the SDTV output.