Apparatus and method for blending graphics and video surfaces

• US 6,700,588 B1
• Filed: 11/09/1999
• Issued: 03/02/2004
• Est. Priority Date: 11/09/1998
• Status: Expired due to Term

• Current Assignee: Avago Technologies International Sales Pte Limited (Broadcom, Inc.)
• Original Assignee: Broadcom Corporation (Broadcom, Inc.)
• Inventors: MacInnis, Alexander G., Tang, Chengfuh Jeffrey, Kranawetter, Greg A., Patterson, James T., Xie, Xiaodong
• Primary Examiner(s): Luu, Matthew
• Application Number:
  US09/437,348
• Time in Patent Office: 1,575 Days
• Field of Search: 345/602, 345/549, 345/441, 345/629, 345/632, 345/641, 345/634, 345/619
• US Class Current: 345/629
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0683 Plurality of storage devices
• CPC Class Codes: G06T 1/20 Processor architectures; Processor configuration, e.g. pipelining
• CPC Class Codes: G06T 1/60 Memory management
• CPC Class Codes: G06T 9/00 Image coding bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00
• CPC Class Codes: G06T 9/007 Transform coding, e.g. discrete cosine transform
• CPC Class Codes: G09G 2310/0224 Details of interlacing
• CPC Class Codes: G09G 2320/0247 Flicker reduction other than flicker reduction circuits used for single beam cathode-ray tubes
• CPC Class Codes: G09G 2340/02 Handling of images in compressed format, e.g. JPEG, MPEG
• CPC Class Codes: G09G 2340/0407 Resolution change, inclusive of the use of different resolutions for different screen areas
• CPC Class Codes: G09G 2340/10 Mixing of images, i.e. displayed pixel being the result of an operation, e.g. adding, on the corresponding input pixels
• CPC Class Codes: G09G 2340/125 wherein one of the images is motion video
• CPC Class Codes: G09G 2360/02 Graphics controller able to handle multiple formats, e.g. input or output formats
• CPC Class Codes: G09G 2360/121 using a cache memory
• CPC Class Codes: G09G 2360/125 using unified memory architecture [UMA]
• CPC Class Codes: G09G 2360/126 The frame memory having additional data ports, not inclusive of standard details of the output serial port of a VRAM
• CPC Class Codes: G09G 2360/128 Frame memory using a Synchronous Dynamic RAM [SDRAM]
• CPC Class Codes: G09G 5/001 Arbitration of resources in a display system, e.g. control of access to frame buffer by video controller and/or main processor
• CPC Class Codes: G09G 5/02 : characterised by the way in which colour is displayed details of colour display specific for CRTs G09G1/28; specific for flat matrix panels other than liquid crystal displays G09G3/2003; specific for liquid crystal displays G09G3/3607
• CPC Class Codes: G09G 5/024 : using colour registers, e.g. to control background, foreground, surface filling G09G5/06 takes precedence
• CPC Class Codes: G09G 5/026 : Control of mixing and/or overlay of colours in general G09G5/022 and G09G5/024 take precedence
• CPC Class Codes: G09G 5/06 : using colour palettes, e.g. look-up tables
• CPC Class Codes: G09G 5/12 : Synchronisation between the display unit and other units, e.g. other display units, video-disc players
• CPC Class Codes: G09G 5/14 : Display of multiple viewports
• CPC Class Codes: G09G 5/28 : for enhancement of character form, e.g. smoothing
• CPC Class Codes: G09G 5/346 : for systems having a bit-mapped display memory
• CPC Class Codes: G09G 5/36 : characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
• CPC Class Codes: G09G 5/363 : Graphics controllers
• CPC Class Codes: H04N 11/143 : Encoding means therefor H04N11/16 takes precedence
• CPC Class Codes: H04N 11/20 : Conversion of the manner in which the individual colour picture signal components are combined, e.g. conversion of colour television standards
• CPC Class Codes: H04N 19/42 : characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation H04N19/635 takes precedence
• CPC Class Codes: H04N 19/423 : characterised by memory arrangements H04N19/433 takes precedence
• CPC Class Codes: H04N 19/436 : using parallelised computational arrangements
• CPC Class Codes: H04N 19/44 : Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/59 : involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
• CPC Class Codes: H04N 19/61 : in combination with predictive coding
• CPC Class Codes: H04N 21/42204 : User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/426 : Internal components of the client ; Characteristics thereof H04N5/44 takes precedence
• CPC Class Codes: H04N 21/42653 : for processing graphics
• CPC Class Codes: H04N 21/4305 : Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets arrangements for synchronising receiver with transmitter by comparing receiver clock with transmitter clock H04L7/0012; arrangements for synchronising receiver with transmitter wherein the receiver takes measures against momentary loss of synchronisation H04L7/0083
• CPC Class Codes: H04N 21/4316 : for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/434 : Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream demultiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00
• CPC Class Codes: H04N 21/440263 : by altering the spatial resolution, e.g. for displaying on a connected PDA
• CPC Class Codes: H04N 21/4438 : Window management, e.g. event handling following interaction with the user interface
• CPC Class Codes: H04N 21/47 : End-user applications
• CPC Class Codes: H04N 5/04 : Synchronising for television systems using pulse code modulation H04N7/56
• CPC Class Codes: H04N 5/126 : whereby the synchronisation signal indirectly commands a frequency generator
• CPC Class Codes: H04N 5/14 : Picture signal circuitry for video frequency region cameras or camera modules comprising electronic image sensors, or control thereof H04N23/00
• CPC Class Codes: H04N 5/44504 : Circuit details of the additional information generator, e.g. details of the character or graphics signal generator, overlay mixing circuits
• CPC Class Codes: H04N 5/45 : Picture in picture , e.g. displaying simultaneously another television channel in a region of the screen
• CPC Class Codes: H04N 5/46 : for receiving on more than one standard at will deflecting circuits of multi-standard receivers H04N3/27
• CPC Class Codes: H04N 7/0122 : the input and the output signals having different aspect ratios
• CPC Class Codes: H04N 7/0135 : involving interpolation processes interpolation-based image scaling G06T3/4007; interpolation for video coding H04N19/587, H04N19/59
• CPC Class Codes: H04N 9/45 : Generation or recovery of colour sub-carriers
• CPC Class Codes: H04N 9/641 : Multi-purpose receivers, e.g. for auxiliary information H04N9/642 takes precedence
• CPC Class Codes: H04N 9/642 : Multi-standard receivers

Abstract

A graphics integrated circuit chip is used in a set-top box for controlling a television display. The graphics chip processes analog video input, digital video input, and graphics input. The chip blends graphics and video information by various graphics windows using alpha values for the windows, alpha values per pixel, or both. The chip calculates a composite alpha value based on the window'"'"'s alpha values and the alpha values per pixel. Blended graphics and video may then be composited using the composite alpha value.

Claims

1. A method of compositing a plurality of graphics images and video comprising:

blending said graphics images into a blended graphics image using a plurality of alpha values, each graphics image comprising a plurality of pixels, and each pixel being associated with one of said alpha values, the graphics images being organized into layers, each graphics image being on and associated with a layer;

combining said alpha values into a plurality of composite alpha values by combining each alpha value of the graphics image on one layer with a corresponding alpha value of the graphics image on another layer; and

blending the blended graphics image and the video using the composite alpha values.

2. The method according to claim 1, wherein blending said graphics images comprises blending the graphics image on a second layer on top of the graphics image on a first layer to generate an intermediate image.

3. The method according to claim 2, wherein blending the graphics image on the second layer on top of the graphics image on the first layer comprises blending each pixel of the graphics image on the first layer with a corresponding pixel of the graphics image on the second layer, using the alpha values associated with the pixels of the graphics image on the second layer.

4. The method according to claim 3, wherein blending each pixel of the graphics image on the first layer with the corresponding pixel of the graphics image on the second layer comprises:

multiplying each pixel of the graphics image on the first layer by (1-alpha value) of the alpha value associated with the corresponding pixel of the graphics image on the second layer;

multiplying each pixel of the graphics image on the second layer by the associated alpha value; and

adding products of the two multiplications.

5. A method of compositing a plurality of graphics images and video comprising:

blending the plurality of graphics images into a blended graphics image using a plurality of alpha values, each graphics image comprising a plurality of pixels, and each pixel being associated with one of said alpha values, the graphics images being organized into layers, each graphics image being on and associated with a layer;

combining said alpha values into a plurality of composite alpha values by combining each alpha value of the graphics image on one layer with a corresponding alpha value of the graphics image on another layer; and

blending the blended graphics image arid the video using the composite alpha values,

wherein blending said graphics images into the blended graphics image comprises blending the graphics image on a second layer on top of the graphics image on a first layer to generate an intermediate image, and

blending a graphics image that has not been blended yet, on top of the intermediate image to generate a new intermediate image.

6. The method according to claim 5, wherein blending the graphics image that has not been blended yet comprises blending each pixel of the intermediate image with a corresponding pixel of the graphics image that has not been blended yet, using the alpha values associated with the pixels of the graphics image that has not beer blended yet.

7. The method according to claim 6, wherein blending each pixel of the intermediate image with the corresponding pixel of the graphics image that has not been blended yet comprises:

multiplying each pixel of the intermediate image by (1-alpha value) of the alpha value associated with the corresponding pixel of the graphics image that has not been blended yet;

multiplying each pixel of the graphics image that has not been blended yet by the associated alpha value; and

adding products of the two multiplications.

8. A method of compositing a plurality of graphics images and video comprising the steps of:

blending the plurality of graphics images into a blended graphics image;

combining a plurality of alpha values into a plurality of composite alpha values; and

blending the blended graphics image and the video using the plurality of composite alpha values,

wherein the plurality of graphics images are blended together using the plurality of alpha values,

wherein each of the plurality of graphics images includes a plurality of pixels, and each of the plurality of pixels is associated with one of the plurality of alpha values, and

wherein the step of combining a plurality of alpha values into a plurality of composite alpha values comprises the step of multiplying (1-alpha value) associated with each of the plurality of pixels of a graphics image with (1−

alpha value) associated with a corresponding pixel of each of all other graphics images to generate each of the plurality of composite alpha values.

9. A method of compositing a plurality of graphics images and video comprising the steps of:

blending the plurality of graphics images into a blended graphics image;

combining a plurality of alpha values into a plurality of composite alpha values; and

blending the blended graphics image and the video using the plurality of composite alpha values,

wherein the plurality of graphics images are blended together using the plurality of alpha values,

wherein each of the plurality of graphics images includes a plurality of pixels, and each of the plurality of pixels is associated with one of the plurality of alpha values,

wherein the plurality of graphics images are organized into layers, each of the plurality of graphics images being associated a layer, from the back most layer to the front most layer, and

wherein the step of combining a plurality of alpha values into a plurality of composite alpha values comprises the step of multiplying (1-alpha value) associated with each of the plurality of pixels of the graphics image on a first layer by (1-alpha value) associated with a corresponding one of the plurality of pixels of the graphics image on a second layer to generate a plurality of intermediate alpha values.

10. The method of compositing a plurality of graphics images and video of claim 9 wherein the step of combining a plurality of alpha values into) a plurality of composite alpha values further comprises the step of multiplying each of the plurality of intermediate alpha values with (1-alpha value) associated with a corresponding one of the plurality of pixels of the graphics image whose alpha values previously have not been multiplied into the plurality of intermediate alpha values, to newly generate a plurality of intermediate alpha values.

11. The method of compositing a plurality of graphics images and video of claim 10 wherein the step of combining a plurality of alpha values into a plurality of composite alpha values further comprises the step of repeating the step of multiplying each of the plurality of intermediate alpha values with (1-alpha value) associated with a corresponding one of the plurality of pixels of the graphics image whose alpha values previously have not been multiplied into the plurality of intermediate alpha values, to newly generate a plurality of intermediate alpha values, until alpha values of all graphics images have been multiplied, and wherein the newly generated plurality of intermediate alpha values, after the alpha values of all graphics images have keen multiplied, are the plurality of composite alpha values.

12. A method of compositing a plurality of graphics images and video comprising:

blending the plurality of graphics images into a blended graphics image using a plurality of alpha values, each graphics image comprising a plurality of pixels, and each pixel being associated with one of said alpha values, the graphics images being organized into layers, each graphics image being on and associated with a layer;

combining said alpha values into a plurality of composite alpha values by combining each alpha value of the graphics image on one layer with a corresponding alpha value of the graphics image on another layer;

blending the blended graphics image and the video using the composite alpha values; and

filtering the blended graphics image prior to blending the blended graphics image with the video,

wherein filtering comprises vertical scaling.

13. A method of compositing a plurality of graphics images and video comprising:

blending the plurality of graphics images into a blended graphics image using a plurality of alpha values, each graphics image comprising a plurality of pixels, and each pixel being associated with one of said alpha values, the graphics images being organized into layers, each graphics image being on and associated with a layer;

combining said alpha values into a plurality of composite alpha values by combining each alpha value of the graphics image on one layer with a corresponding alpha value of the graphics image on another layer;

blending the blended graphics image and the video using the composite alpha values; and

filtering the blended graphics image prior to blending the blended graphics image with the video,

wherein filtering comprises horizontal scaling.

14. A method of compositing a plurality of graphics images and video comprising:

blending the plurality of graphics images into a blended graphics image using a plurality of alpha values, each graphics image comprising a plurality of pixels, and each pixel being associated with one of said alpha values, the graphics images being organized into layers, each graphics image being on and associated with a layer;

combining said alpha values into a plurality of composite alpha values by combining each alpha value of the graphics image on one layer with a corresponding alpha value of the graphics image on another layer;

blending the blended graphics image and the video using the composite alpha values; and

filtering the blended graphics image prior to blending the blended graphics image with the video,

wherein filtering comprises anti-flutter filtering.

15. A method of compositing a plurality of graphics images and video comprising:

blending the plurality of graphics images into a blended graphics image using a plurality of alpha values, each graphics image comprising a plurality of pixels, and each pixel being associated with one of said alpha values, ;

he graphics images being organized into layers, each graphics image being on and associated with a layer;

combining said alpha values into a plurality of composite alpha values by combining each alpha value of the graphics image on one layer with a corresponding alpha value of the graphics image on another layer; and

blending the blended graphics image and the video using the composite alpha values,

wherein the video comprises a passthrough video and a video window.

16. The method according to claim 15 further comprising blending background color with the video.

17. The method according to claim 16 wherein the passthrough video is blended with the background color using alpha values to generate an intermediate video image.

18. The method according to claim 17 wherein the video window is blended with the intermediate video image using alpha values to generate a blended video image.

19. The method according to claim 18 wherein the blended video image is blended with the blended graphics image using the composite alpha values.

20. The method according to claim 19 wherein the blended video image comprises a plurality of video pixels and the blended graphics image comprises a plurality of graphics pixels, and each of the plurality of composite alpha values is associated with one of the video pixels and one of the graphics pixels.

21. The method according to claim 20 wherein blending the blended graphics image and the video comprises multiplying each video pixel by the associated composite alpha value;

and adding each graphics pixel to the product of the associated video pixel and the associated composite alpha value.

25. The graphics display system of claim 21 wherein the video compositor blends the passthrough video with the background color to generate an intermediate video image.

26. The graphics display system of claim 25 wherein the video compositor blends the intermediate video image with the video window to generate a blended video image having a plurality of video pixels.

27. The graphics display system of claim 26 wherein the blended graphics image includes a plurality of graphics pixels, each of the plurality of video pixels is associated with one of the plurality of composite alpha values and one of the plurality of graphics pixels, and video compositor blends the blended video image and the blended graphics by multiplying each of the plurality of video pixels by the associated one of the plurality of composite alpha values and adding the associated one of the plurality of graphics pixels.

22. A graphics display system comprising:

a display engine for blending a plurality of graphics images, each graphics image being on one of a plurality of layers, into a blended graphics image using a plurality of alpha values, and combining said alpha values into a plurality of composite alpha values by combining each alpha value of one graphics image with a corresponding alpha value of another graphics image; and

a video compositor for blending the blended graphics image with video using the composite alpha values.

23. The display system according to claim 22, wherein the video compositor also blends a background color with the blended graphics image and the video.

24. The display system according to claim 23, wherein the video comprises a passthrough video and a video window.

28. A graphics display system comprising:

a display engine for blending a plurality of graphics images into a blended graphics image and combining a plurality of alpha values into a plurality of composite alpha values; and

a video compositor for blending the blended graphics image with video,

wherein each of the plurality of graphics images includes a plurality of graphics pixels, and each of the plurality of aloha values is associated with one of the plurality of graphics pixels,

wherein the plurality of graphics images are organized into layers, wherein each of the plurality of graphics images is on and associated with a layer that is different from each other and the layers are stacked from the back most layer to the front most layer, and

wherein each of the plurality of graphics pixels of the graphics image on the back most layer is multiplied by (1-alpha value) of the alpha value associated with a corresponding one of the plurality of graphics pixels of the graphics image on the layer immediately in front of the back most layer, each of the plurality of graphics pixels of the graphics image on the layer immediately in front of the back most layer is multiplied by the associated alpha value, and the products of two multiplications are added to generate each of the plurality of graphics pixels of an intermediate graphics image.

29. The graphics display system of claim 28 wherein each of the plurality of graphics pixels of the intermediate graphics image is multiplied by (1-alpha value) of the alpha value associated with a corresponding one of the plurality of graphics pixels of the graphics image on the layer immediately in front of all layers for which the associated graphics image has been blended, each of the plurality of graphics pixels of the graphics image, on the layer immediately in front of all layers for which the associated graphics image has been blended, is multiplied by the associated alpha value, and the products of two multiplications are added to generate each of the plurality of graphics pixels of a new intermediate graphics image.

30. The graphics display system of claim 29 wherein the plurality of graphics pixels of a new intermediate graphics image are repeatedly generated until each of the plurality of graphics pixels of the intermediate graphics image is multiplied by (1-alpha value) of the alpha value associated with a corresponding one of the plurality of graphics pixels of the graphics image on the front most layer, each of the plurality of the graphics pixels of the graphics image on the front most layer has been multiplied with the associated alpha value, and the products of the two multiplications are added to generate each of the plurality of graphics pixels of the new intermediate graphics image,

and wherein the new intermediate graphics image is the blended graphics image after the graphics image on the front most layer has been blended.

31. A graphics display system comprising:

a display engine for blending a plurality, of graphics images into a blended graphics image and combining a plurality of alpha values into a plurality of composite alpha values; and

a video compositor for blending the blended graphics image with video,

wherein each of the plurality of graphics images includes a plurality of graphics pixels, and each of the plurality of alpha values is associated with one of the plurality of graphics pixels,

wherein the plurality of graphics images is organized into layers, wherein each of the plurality of graphics images is on and associated with a layer that is different from each other and the layers are stacked from the back most layer to the front most layer,

wherein (1-alpha value) of the alpha value associated with each of the plurality of the pixels of the graphics image on the back most layer is multiplied by (1-alpha value) associated with a corresponding one of the plurality of the pixels of the graphics image on the layer immediately in front of the back most layer to generate each of the plurality of intermediate alpha values, and

wherein each of the plurality of intermediate alpha values is multiplied by (1-alpha) of the alpha value associated with each of the plurality of the pixels of the graphics image on the layer that is immediately in front of all layers for which associated plurality of alpha values have been multiplied into the plurality of intermediate alpha values, to newly generate the plurality of intermediate alpha values.

32. The graphics display system of claim 31 wherein the plurality of intermediate alpha values are newly generated repeatedly until the plurality of alpha values associated with the plurality of pixels of the graphics image on the front most layer have been multiplied into the plurality of intermediate alpha values, and

wherein the plurality of intermediate alpha values after the plurality of alpha values associated with the plurality of pixels of the graphics image on the front most layer are the plurality of composite alpha values.

33. A method of compositing graphics images using alpha blend values comprising the steps of:

blending each line of a second graphics image on top of a corresponding Line of a first graphics image using alpha values, if the line is neither a first line nor a last line of the second graphics image; and

blending the first line and the last line of the second graphics image on top of the corresponding lines of the first graphics image using about 0.5*alpha blend values,

wherein the second graphics image includes a plurality of pixels, each pixel being associated with an alpha blend value, and the first graphics image includes a plurality of pixels,

wherein the step of blending each line of a second graphics image on top of a corresponding line of a first graphics image, if the line is neither a first line nor a last line of the second graphics image, comprises the steps of multiplying each of the plurality of pixels of the first graphics image by (1-alpha blend value) of the alpha blend value associated with a corresponding pixel of the second graphics image, multiplying the corresponding pixel of the second graphics image with the associated alpha blend value, and adding the products of the two multiplications.

34. A method of compositing graphics images using alpha blend values comprising the steps of:

blending each line of a second graphics image on top of a corresponding line of a first graphics image using aloha values, if the line is neither a first line nor a last line of the second graphics image;

ard

blending the first line and the last line of the second graphics image on top of the corresponding lines of the first graphics image using about 0.5*alpha blend values,

wherein the second graphics image includes a plurality of pixels, each pixel being associated with an alpha blend value, and the first graphics image includes a plurality of pixels,

wherein the step of blending the first line and the last line of the second graphics image on top of the corresponding lines of the first graphics image using about 0.5*alpha blend values comprises the steps of multiplying each of the plurality of pixels of the first graphics image by (1 B 0.5*alpha blend value) of the alpha blend value associated with a corresponding pixel of the second graphics image, multiplying the corresponding pixel of the second graphics image with the associated alpha blend value*0.5, and adding the products of the two multiplications.