Video compositor

• US 6,909,438 B1
• Filed: 01/23/2001
• Issued: 06/21/2005
• Est. Priority Date: 02/04/2000
• Status: Expired due to Term

• Current Assignee: Sportsmedia Technology Corporation
• Original Assignee: Sportvision, Inc. (Sportsmedia Technology Corporation)
• Inventors: McGuffin, James, Hsiao, Walter, White, Marvin S., Gloudemans, James R., Cavallaro, Richard H., Honey, Stanley K., Meier, Kevin R.
• Primary Examiner(s): Zimmerman, Mark
• Assistant Examiner(s): Santiago, Enrique L.
• Application Number:
  US09/767,777
• Time in Patent Office: 1,610 Days
• Field of Search: 345/549, 345/589, 345/591, 345/592, 345/593, 345/597, 345/600, 345/601, 345/602, 345/606, 345/629, 345/625, 345/626, 345/634, 345/639, 345/640, 345/572, 345/768
• US Class Current: 345/629
• CPC Class Codes: H04N 9/76 for mixing of colour signals H04N9/75 takes precedence

Abstract

A system is disclosed for blending two image that makes use of a color map which indicates colors in a foreground can be mixed with the background and how much of each source to mix. One embodiment of the invention restricts the use of the color map to only pixels in the foreground that correspond to a graphic (or effect) in the background. Another embodiment makes use of a gray scale matte which stores blending values for each pixel in the foreground.

Claims

1. A method for blending images, comprising:

receiving a first image, said first image includes a first set of pixels;

receiving a second image, said second image includes a second set of pixels and a third set of pixels, said third set of pixels has a predefined attribute indicating that said third set of pixels should not be mixed;

determining which pixels of said second image are part of said second set of pixels and which pixels of said second image are part of said third set of pixels based on said predefined attribute;

for each specific pixel of said first set of pixels that corresponds in position with a pixel from said second set of pixels;

determining a color of said specific pixel from said first video image,

accessing a color map after determining said color of said specific pixel, said color map capable of storing blending value information for a plurality of different colors, and

using said determined color of said specific pixel to determine a pixel blending value for said specific pixel from said color map; and

blending said first set of pixels with said second set of pixels based on said pixel blending values.

2. A method according to claim 1, wherein:

said predefined attribute is a transparent color.

4. A method according to claim 1, further including the step of:

adding additional visibly distinct colors to said color map.

5. A method according to claim 1, wherein:

said second image is part of a video signal;

said step of receiving a second image includes receiving said video signal.

6. A method according to claim 5, wherein:

said predefined attribute is a predefined color; and

said step of determining which pixels of said video signal are part of said second image comprises the step of identifying which pixels of said video signal are said predefined color.

7. A method according to claim 5, wherein:

said predefined attribute is a transparent color; and

said step of determining which pixels of said video signal are part of said second image comprises the step of identifying which pixels of said video signal are said transparent color.

8. A method according to claim 1, wherein:

said first image is part of a first video signal;

said step of receiving a first image includes receiving said first video signal;

said second image is part of a second video signal;

said step of receiving a second image includes receiving said second video signal.

9. A method according to claim 1, wherein:

said color map is a look-up table storing color values and blending information for said color values.

10. A method according to claim 1, wherein:

said color map is a look-up table storing ranges of color values and blending information for said ranges.

11. A method according to claim 1, wherein:

said steps of receiving a first image, receiving a second image, accessing and blending are performed by one apparatus.

12. A method according to claim 1, further including the step of:

receiving additional blending values that are not based on said color map, said step of blending is also based on said additional blending values.

13. A method according to claim 12, wherein:

said additional blending values indicate whether to ignore said pixel blending values.

14. A method according to claim 1, wherein said step of blending includes the steps of:

creating a matte of said second image;

performing a logical AND of said matte and values based on said pixel blending values; and

keying said second image with said first image using a key equal to said logical AND of said matte and said values based on blending values.

15. A method according to claim 1, wherein said step of blending includes the steps of:

creating a matte of said second image;

creating a first composited image by keying said second image with said first image using said a key equal to said matte; and

creating a second composited image by keying said first composited image with said first image.

16. A method according to claim 1, wherein:

said method is performed in real time during a live event.

17. A method according to claim 1, wherein said step of blending includes the steps of:

creating a matte based on said pixel blending values, said matte includes a fourth set of pixels, each pixel in said fourth set of pixels includes a data value, each data value is based on at least a corresponding pixel blending value; and

mixing said first image with said second image on a pixel by pixel basis using said matte.

18. A method according to claim 1, wherein:

said step of using said determined color of said specific pixel to determine a pixel blending value includes reading said pixel blending value from said color map.

19. A method according to claim 1, wherein:

said step of using said determined color of said specific pixel to determine a pixel blending value includes reading color data from said color map and calculating said pixel blending value from said color data.

3. A method for blending images, comprising:

receiving a first image, said first image includes a first set of pixels;

receiving a second image, said second image includes a second set of pixels and a third set of pixels, said third set of pixels has a predefined attribute indicating that said third set of pixels should not be mixed;

determining which pixels of said second image are part of said second set of pixels and which pixels of said second image are part of said third set of pixels based on said predefined attribute;

for each specific pixel of said first set of pixels that corresponds in position with a pixel from said second set of pixels, accessing a color map based on color of said specific pixel and determining a pixel blending value for said specific pixel based on said accessing said color map, said color map capable of storing blending value information for a plurality of different colors;

blending said first set of pixels with said second set of pixels based on said pixel blending values; and

updating said color map in real time, said step of blending is performed in real time.

20. An apparatus used for blending images, comprising:

an input device;

an output device;

memory; and

one or more processors, said one or more processors perform a method comprising the steps of;

receiving a first image, said first image includes a first set of pixels,

receiving a second image, said second image includes a second set of pixels and a third set of pixels, said third set of pixels has a predefined attribute indicating that said third set of pixels should not be mixed,

determining which pixels of said second image are part of said second set of pixels and which pixels of said second image are part of said third set of pixels based on said predefined attribute,

for each specific pixel of said first set of pixels that corresponds in position with a pixel from said second set of pixels;

determining a color of said specific pixel from said first video image,

accessing a color map after determining said color of said specific pixel, said color map capable of storing blending value information for a plurality of different colors, and

using said determined color of said specific pixel to determine a pixel blending value for said specific pixel from said color map, and

causing a blending of said first set of pixels with said second set of pixels based on said pixel blending values.

21. An apparatus according to claim 20, wherein

said predefined attribute is a transparent color.

22. An apparatus according to claim 20, wherein said method further includes the step of:

updating said color map in real time, said step of blending is performed in real time.

23. An apparatus according to claim 20, wherein said method further includes the step of:

adding additional visibly distinct colors to said color map.

24. An apparatus according to claim 20, wherein:

said step of determining which pixels of said video signal are part of said second image comprises the step of identifying which pixels of said video signal are a predefined color.

25. An apparatus according to claim 20, wherein said method further includes the step of:

receiving additional blending values that are not based on said color map, said step of blending is also based on said additional blending values, said additional blending values indicate whether to ignore said pixel blending values.

26. An apparatus used for blending images, comprising:

an input device;

an output device;

memory; and

one or more processors, said one or more processors perform a method comprising the steps of;

receiving a first image, said first image includes a first set of pixels,

receiving a second image, said second image includes a second set of pixels and a third set of pixels, said third set of pixels has a predefined attribute indicating that said third set of pixels should not be mixed,

determining which pixels of said second image are part of said second set of pixels and which pixels of said second image are part of said third set of pixels based on said predefined attribute,

for each specific pixel of said first set of pixels that corresponds in position with a pixel from said second set of pixels, accessing a color map based on color of said specific pixel and determining a pixel blending value for said specific pixel based on said accessing said color map, said color map capable of storing blending value information for a plurality of different colors,

causing a blending of said first set of pixels with said second set of pixels based on said pixel blending values, said step of causing a blending includes creating a matte based on said pixel blending values and mixing said first image with said second image on a pixel by pixel basis using said matte, said matte includes a fourth set of pixels, each pixel in said fourth set of pixels includes a data value, each data value is based on at least a corresponding pixel blending value.

27. A method for blending images, comprising the steps of:

storing blending information for a set of visibly distinct colors;

receiving a first image, said first image includes a first set of pixels, each pixel of said first set of pixels is associated with color data;

receiving a second image, said second image includes a second set of pixels;

for each pixel in said first set of pixels, identifying an appropriate subset of said stored blending information based on said color data;

creating a matte, said matte includes a third set of pixels, each pixel in said third set of pixels corresponds to at least one pixel in said first set of pixels, each pixel in said third set of pixels includes a data value based on said appropriate subset of said stored blending information for said corresponding at least one pixel in said first set of pixels; and

causing a blending of said first image with said second image based on said matte.

28. A method according to claim 27, wherein:

said matte is a gray scale image that is part of a video signal.

29. A method according to claim 27, wherein:

said first image is a video image; and

said second image is a video image.

30. A method according to claim 27, wherein:

said blending information includes a color map, said color map stores an identifications of colors and blending values for said colors.

31. A method according to claim 30, wherein:

each data value stores one of said blending values.

32. A method according to claim 31, wherein:

said step of causing a blending includes the steps of creating an alpha signal based on said matte and transmitting said alpha signal to a keyer;

said first image is a video image; and

said second image is a video image.

33. A method according to claim 27, wherein said step of causing a blending includes the steps of:

accessing a first pixel of said second image;

determining whether said first pixel of said second image is part of an effect;

outputting a first pixel of said first image, without blending, if said pixel is not part of said effect;

outputting a blend of said first pixel of said first image and said first pixel of said second image according to a first a data value in said matte if said first pixel of said second image is part of said effect, said first data value corresponds to a first pixel of said matte, said first pixel of said matte corresponds to said first pixel of said first image and said first pixel of said second image.

34. A method according to claim 27, further comprising the steps of:

receiving a third image;

displaying said third image;

receiving a selection of a portion of said third image, said portion of said third image includes a set of colors; and

receiving one or more blending values for said set of colors, said stored different blending information includes said blending values.

35. An apparatus used for blending images, comprising:

an input device;

an output device;

memory; and

one or more processors, said one or more processors, said one or more processors perform a method comprising the steps of;

storing blending information for a set of visibly distinct colors,

receiving a first image, said first image includes a first set of pixels, each pixel of said first set of pixels is associated with color data,

receiving a second image, said second image includes a second set of pixels,

for each pixel in said first set of pixels, identifying an appropriate subset of said stored blending information based on said color data,

creating a matte, said matte includes a third set of pixels, each pixel in said third set of pixels corresponds to at least one pixel in said first set of pixels, each pixel in said third set of pixels includes a data value based on said appropriate subset of said stored blending information for said corresponding at least one pixel in said first set of pixels, and

causing a blending of said first image with said second image based on said matte.

36. An apparatus according to claim 35, wherein said step of causing a blending includes the steps of:

accessing a first pixel of said second image;

determining whether said first pixel of said second image is part of an effect;

outputting a first pixel of said first image, without blending, if said pixel is not part of said effect;

outputting a blend of said first pixel of said first image and said first pixel of said second image according to a first a data value in said matte if said first pixel of said second image is part of said effect, said first data value corresponds to a first pixel of said matte, said first pixel of said matte corresponds to said first pixel of said first image and said first pixel of said second image.

37. An apparatus according to claim 35, wherein:

said matte is a gray scale image that is part of a video signal.

38. An apparatus according to claim 35, wherein:

said blending information includes a color map, said color map stores an identifications of colors and blending values for said colors.

39. An apparatus for blending a first video image with a second video image, comprising:

a pixel blending value calculation circuit receiving said first video image, said pixel blending value calculation circuit includes a color map, said color map stores information indicating blending values for multiple visibly distinct colors, said pixel blending value calculation circuit generates an output by;

determining color in said first video image,

accessing said color map after determining said color in said first video image, and

using said determined color in said first video image to determine a pixel

blending value from said color map for at least a portion of said first video image; and

a blending circuit receiving said first video image and said second video image, said pixel blending circuit in communication with said pixel blending value calculation circuit, said blending circuit mixes said first video image with said second video image based on said output of said pixel blending value calculation circuit.

40. An apparatus according to claim 39, wherein said blending circuit comprises:

control logic in communication with said pixel blending value calculation circuit;

a first video altering circuit receiving said first video image, said first video altering circuit is in communication with said control logic, said first video altering circuit alters said first video image based on a first set of one or more blending variables received from said control logic;

a second video altering circuit receiving said second video image, said second video altering circuit is in communication with said control logic, said second video altering circuit alters said second video image based on a second set of one or more blending variables received from said blending control logic; and

a mixing circuit in communication with said first video altering circuit and said second video altering circuit.

41. An apparatus according to claim 39, wherein said pixel blending value calculation circuit comprises:

a memory, said memory stores said color map; and

a processor in communication with said memory, said processor receives said first video image, and generates said output by determining said color in said first video image, accessing said color map, and using said determined color to determine a pixel blending value.

42. An apparatus according to claim 39, wherein:

said pixel blending value calculation circuit receives a geometric key signal, said pixel blending value calculation circuit selectively ignores said color map for a given pixel in said first video image in response to said geometric key signal.

43. An apparatus according to claim 39, wherein:

said pixel blending value calculation circuit selectively ignores said color map for a particular pixel in said first video image if said particular pixel is of a predetermined color.

44. An apparatus according to claim 39, wherein:

said pixel blending value calculation circuit accesses said color map for every pixel of said first video image and generates an output based on said color map for every pixel of said first video image.

45. An apparatus for blending a first video signal with a second video signal, comprising:

a memory, said memory stores a color map, said color map stores information indicating blending information for multiple colors;

a processor in communication with said memory, said processor receives said first video signal, determines colors in said first video signal, accesses said color map after determining said colors in said first video signal, and uses said determined colors in said first video signal to determine blending values from said color map; and

a video blending circuit in communication with said processor, said video blending circuit receives said first video signal and said second video signal, said video blending circuit mixes said first video signal with said second video signal based on said blending values.

46. An apparatus according to claim 45, wherein said blending circuit comprises:

control logic in communication with said processor;

a first video signal altering circuit receiving said first video signal, said first video signal altering circuit is in communication with said control logic, said first video signal altering circuit alters said first video signal based on a first set of one or more blending values received from said control logic;

a second video signal altering circuit receiving said second video signal, said second video signal altering circuit is in communication with said control logic, said second video signal altering circuit alters said second video signal based on a second set of one or more blending values received from said control logic; and

a video blending circuit in communication with said first video signal altering circuit and said second video signal altering circuit.

47. An apparatus according to claim 45, wherein:

said processor receives a geometric key signal, said processor selectively ignores said color map for a given pixel in said first video image in response to said geometric key signal.

48. An apparatus according to claim 45, wherein:

said processor selectively ignores said color map for a particular pixel in said first video image if said particular pixel is of a predetermined color.

49. An apparatus according to claim 45, wherein:

said processor accesses said color map for every pixel of a first image in said first video signal and generates an output based on said color map for every pixel of said first image.