Apparatus and method for mixing graphics with video images

• US 8,659,704 B2
• Filed: 03/16/2007
• Issued: 02/25/2014
• Est. Priority Date: 12/20/2005
• Status: Active Grant

• Current Assignee: Savant Systems, LLC
• Original Assignee: Savant Systems, LLC
• Inventors: Madonna, Robert P., Noonan, Michael E., Eschholz, Siegmar K., Allen, James F., Corsini, Peter H.
• Primary Examiner(s): Lee, Michael
• Application Number:
  US11/687,511
• Publication Number:
  US 20120188459A1
• Time in Patent Office: 2,538 Days
• Field of Search: 348/563, 348/564, 348/565, 348/566, 348/569, 348/580, 348/581, 348/843, 348/298, 348/629, 348/634, 348584-601, 345629-641
• US Class Current: 348/587
• CPC Class Codes: G09G 2340/10 Mixing of images, i.e. displayed pixel being the result of an operation, e.g. adding, on the corresponding input pixels
• CPC Class Codes: G09G 2340/125 wherein one of the images is motion video
• CPC Class Codes: G09G 2370/20 Details of the management of multiple sources of image data
• CPC Class Codes: G09G 5/026 Control of mixing and/or overlay of colours in general G09G5/022 and G09G5/024 take precedence
• CPC Class Codes: G09G 5/397 Arrangements specially adapted for transferring the contents of two or more bit-mapped memories to the screen simultaneously, e.g. for mixing or overlay G09G5/02 takes precedence
• CPC Class Codes: H04L 12/2803 Home automation networks
• CPC Class Codes: H04L 12/282 based on user interaction within the home receiver circuitry for displaying additional information being controlled by a remote control apparatus H04N21/42204
• CPC Class Codes: H04L 12/2838 Distribution of signals within a home automation network, e.g. involving splitting/multiplexing signals to/from different paths adaptations of television systems for transmission by electric cable for domestic distribution H04N7/106; hybrid transport H04L12/6418; home network arrangements specially adapted for distribution of digital video signals H04N7/24
• CPC Class Codes: H04L 2012/2849 Audio/video appliances
• CPC Class Codes: H04L 2012/285 Generic home appliances, e.g. refrigerators
• CPC Class Codes: H04N 21/4112 having fewer capabilities than the client, e.g. thin client having less processing power or no tuning capabilities
• CPC Class Codes: H04N 21/42646 for reading from or writing on a non-volatile solid state storage medium, e.g. DVD, CD-ROM
• CPC Class Codes: H04N 21/43615 Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals home Audio Video Interoperability [HAVI] data switching networks H04L12/2805
• CPC Class Codes: H04N 21/4438 Window management, e.g. event handling following interaction with the user interface
• CPC Class Codes: H04N 5/265 Mixing
• CPC Class Codes: H04N 5/272 Means for inserting a foreground image in a background image, i.e. inlay, outlay
• CPC Class Codes: H04N 5/765 Interface circuits between an apparatus for recording and another apparatus
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera
• CPC Class Codes: H04N 5/775 between a recording apparatus and a television receiver
• CPC Class Codes: H04N 5/85 on discs or drums
• CPC Class Codes: H04N 9/74 : for obtaining special effects H04N9/65 - H04N9/73 take precedence
• CPC Class Codes: H04N 9/76 : for mixing of colour signals H04N9/75 takes precedence

Abstract

In one embodiment, graphics from a general-purpose computer are mixed with video images from a video source. A first interface receives the graphics, the graphics including a plurality of graphics frames, at least some graphics frames including one or more regions having pixels set to one or more predetermined color values. A second interface receives video images from the video source, each video image corresponding to one of the plurality of graphics frames. A video mixer includes logic configured to recognize pixels in the at least some graphics frames that are set to the one or more predetermined color values, and in the one or more regions where the pixels are set to the one or more predetermined color values to mix pixels from the corresponding video image with the graphics frame, to create a plurality of mixed images. One or more output interfaces outputs the mixed images.

Claims

1. An apparatus for mixing graphics from a general-purpose computer with video images from a video source, comprising:

a first interface configured to receive graphics from the general-purpose computer, the graphics including a plurality of graphics frames composed of pixels that each have a color, the color of each pixel represented in a red, green, blue (RGB) color space, at least some graphics frames including one or more regions having pixels set to a repeating pattern of predetermined colors, the repeating pattern of predetermined colors defined by two or more nearby pixels where a first pixel has two or more of a red value, a green value, and a blue value that have a first relative difference and the first relative difference has a first particular less than or greater than relation to a first threshold, and a second pixel has two or more of a red value, a green value, and a blue value that have a second relative difference and the second relative difference has a second particular less than or greater than relation to a second threshold;

a second interface configured to receive video images from the video source, each video image corresponding to one of the plurality of graphics frames;

a video mixer including logic configured to recognize pixels in each of the at least some graphics frames that are set to the repeating pattern of predetermined colors, and in the one or more regions where the pixels are set to the repeating pattern of predetermined colors to mix pixels from the corresponding video image with the graphics frame, to create a mixed image; and

one or more output interfaces to output mixed images.

2. The apparatus of claim 1, wherein the at least some graphics frames include one or more other regions that do not have pixels set to the repeating pattern of predetermined colors, the one or more other regions including pixels of graphics intended to be displayed to a user.

3. The apparatus of claim 1, wherein the nearby pixels are adjacent pixels.

4. The apparatus of claim 3, wherein the repeating pattern of predetermined colors is a repeating pattern of two adjacent pixels.

5. The apparatus of claim 3, wherein the repeating pattern of predetermined colors is a repeating pattern of three or more adjacent pixels.

6. The apparatus of claim 1, wherein the video mixer is further configured to recognize pixels in the at least some graphics frames as set to the repeating pattern of predetermined colors by detecting pixels having two or more of a red value, a green value, and a blue value whose relative difference has the first particular relation to the first threshold and by detecting pixels having two or more of a red value, a green value, and a blue value whose relative difference has the second particular relation to the second threshold.

7. The apparatus of claim 1, further comprising:

an alpha blend value generator configured to generate blend values, the blend values to cause blending of the colors of pixels in one or more other regions that do not have pixels set to the pattern of predetermined colors to be blended with color values of pixels in the corresponding video image.

8. A method for mixing graphics from a general-purpose computer with video images from a video source, comprising:

receiving graphics at a video mixer from the general-purpose computer, the graphics including a plurality of graphics frames composed of pixels that each have a color, the color of each pixel represented in a red, green, blue (RGB) color space, at least some graphics frames including one or more regions having pixels set in a repeating pattern of predetermined colors, the repeating pattern of predetermined colors defined by two or more nearby pixels where a first pixel has two or more of a red value, a green value, and a blue value that have a first relative difference and the first relative difference has a first particular less than or greater than relation to a first threshold, and a second pixel has two or more of a red value, a green value, and a blue value that have a second relative difference and the second relative difference has a second particular less than or greater than relation to a second threshold;

receiving video images from the video source, each video image corresponding to one of the plurality of graphics frames;

recognizing pixels in each of the at least some graphics frames that are set to the repeating pattern of predetermined colors;

in the one or more regions where the pixels are set to the repeating pattern of predetermined colors, mixing pixels from the corresponding video image with the graphics frame to create a mixed image; and

outputting mixed images.

9. The method of claim 8, wherein the at least some graphics frames include one or more other regions that do not have pixels set to the repeating pattern of predetermined colors, the one or more other regions including pixels of graphics intended to be displayed to a user.

10. The method of claim 8, wherein the nearby pixels are adjacent pixels.

11. The method of claim 10, wherein the repeating pattern of predetermined colors is a repeating pattern of two adjacent pixels.

12. The method of claim 10, wherein the repeating pattern of predetermined colors is a repeating pattern of three or more adjacent pixels.

13. The method claim 8, further comprising:

generating blend values, the blend values to cause blending of the colors of pixels in one or more other regions that do not have pixels set to the pattern of predetermined colors to be blended with color values of pixels in the corresponding video image.

14. An apparatus for mixing graphics from a general-purpose computer with video images from a video source, comprising:

a first interface configured to receive graphics from the general-purpose computer, the graphics including a plurality of graphics frames composed of pixels that each have a color, the color of each pixel represented in a color space, at least some graphics frames including one or more regions having pixels set to a repeating pattern of predetermined colors, the repeating pattern of predetermined colors defined by two or more nearby pixels where a first pixel has two or more values whose difference has a first less than or greater than relation to a first threshold, and the second pixel has two or more values whose difference has a second particular less than or greater than relation to a second threshold;

a second interface configured to receive video images from the video source, each video image corresponding to one of the plurality of graphics frames;

a video mixer including logic configured to recognize pixels in each of the at least some graphics frames that are set to the repeating pattern of predetermined colors, and in the one or more regions where the pixels are set to the repeating pattern of predetermined colors to mix pixels from the corresponding video image with the graphics frame, to create a mixed image; and

one or more output interfaces to output mixed images.

15. The apparatus of claim 14, wherein the repeating pattern of predetermined colors is defined by three or more nearby pixels.

16. The apparatus of claim 14, wherein the two or more values of the first pixel and of the second pixel comprise two or more of a red value, a green value, and a blue value.

17. The apparatus of claim 14, wherein the first interface, the second interface, the video mixer, and the one or more output interfaces are included in a programmable multimedia controller interconnected to and configured to control a plurality of different types of devices, and the video source is an audio/video (AV) device interconnected to the programmable multimedia controller.

18. The apparatus of claim 16, wherein the general-purpose computer is included in a processing subsystem of the programmable multimedia controller.