System and method for producing video effects on live-action video

• US 6,069,668 A
• Filed: 04/07/1997
• Issued: 05/30/2000
• Est. Priority Date: 04/07/1997
• Status: Expired due to Fees

• Current Assignee: Pinnacle Systems Incorporated (Avid Technology Incorporated)
• Original Assignee: Pinnacle Systems Incorporated (Avid Technology Incorporated)
• Inventors: Dresdner, Daniel, Woodham, David J. Jr., Woodbury, William C.
• Primary Examiner(s): HSIA, SHERRIE Y
• Application Number:
  US08/833,503
• Time in Patent Office: 1,149 Days
• Field of Search: 348/578, 348/580, 348/583, 348/584, 348/585, 348/586, 348/587, 348/590, 348/239, 382/276, 382/277, 382/293, 382/285, 382/296, 382/298
• US Class Current: 348/578
• CPC Class Codes: H04N 9/74 for obtaining special effects H04N9/65 - H04N9/73 take precedence

Abstract

A digital video effects system for producing live-action video effects in real time includes an address generator having a pair of 2-D tables for providing x-coordinate and y-coordinate offset values. A third 2-D table contains warp activation factors (activators) which are scaled by a clip and gain processing technique to produce scaled activators. The scaled activators are used to scale the x- and y-offset values which are then combined with the original raster-order addresses to produce frame buffer readout addresses. An output video signal is produced by an inverse mapping of the input frame buffer to the output. The 2-D tables provide control of the image on a pixel-by-pixel basis. The degree of scaling by the clip and gain unit varies with time so that the resulting readout addresses also vary with time. A video effect, therefore, is produced simply by adjusting the scaling of the clip and gain unit. Moreover, a variety of effects can be achieved simply by changing the various 2-D tables.

Claims

1. A video effects device for producing live-action video, comprising:

input means for capturing a frame of live-action video and for storing said frame in a frame buffer as a set of pixels;

means for generating image addresses, each having an x-coordinate and a y-coordinate;

a vector offset table addressed by a first index and by a second index, including first means for receiving the x-coordinate and y-coordinate of a first image address respectively as the first index and second index;

means for mapping said first image address to a raw activation factor;

means for reducing or increasing a raw activation factor to produce an adjusted activation factor, including means for varying the amount of reduction or increase with time;

computation means for multiplying an adjusted activation factor with a vector offset to produce an activated vector offset and for adding said activated vector offset to a second image address to produce a frame buffer read address; and

means for outputting pixels, coupled to said computation means and to said frame buffer to generate a transformed video signal.

2. The video effects device of claim 1 wherein said input means includes a second frame buffer wherein first and second fields of a frame of video are stored in said frame buffers, and wherein said means for varying is synchronized with each said field.

3. The video effects device of claim 1 wherein said means for varying is synchronized with each frame of live-action video.

4. The video effects device of claim 1 wherein said means for mapping includes a warp activation table addressed by said first index and by said second index.

5. The video effects device of claim 1 wherein said means for reducing or increasing includes:

a clipper unit for outputting a first value or a second value if a raw activation factor is less than a minimum threshold or exceeds a maximum threshold; and

a gain unit for multiplying a raw activation factor by a gain value if said raw activation factor falls between said minimum and maximum thresholds.

6. The video effects device of claim 5 wherein said first value is equal to zero.

7. The video effects device of claim 5 wherein said means for varying includes changing said minimum and said maximum thresholds.

8. The video effects device of claim 5 wherein said means for varying includes changing said gain value.

9. The video effects device of claim 1 wherein said means for reducing or increasing includes a combination of piecewise linear functions.

10. The video effects device of claim 1 wherein said means for reducing or increasing includes means for computing said adjusted activation factor as a continuous function of said raw activation factor.

11. The video effects device of claim 1 further including a video mixing means for combining a video signal and said transformed video signal into a single output video signal.

12. The video effects device of claim 11 further including a key table indexed by the x-coordinates and y-coordinates of image addresses, thereby producing a key signal;

said video mixing means responsive to said key signal for blending together said video signal and said transformed video signal.

13. The video effects device of claim 1 wherein said means for outputting includes means for reading out pixels from said frame buffer addressed by said frame buffer read address and means for generating a video signal from such pixels.

14. The video effects device of claim 1 wherein said means for generating image addresses includes means for producing raster-order addresses and means for transforming said raster-order addresses through a 3-D transformer and a perspective transformer.

15. The video effects device of claim 14 wherein said 3-D transformer is capable of producing at least one of scaling, rotation, translation, and shear effects.

16. The video effects device of claim 1 wherein said first and second image addresses are the same address.

17. A method for producing live-action video effects, comprising the steps of:

defining a video effect, including initializing a vector offset table and an activation table, each table indexed by a first index and a second index;

inputting a first video signal and storing said video signal in a frame buffer as a plurality of pixels;

generating a stream of pixel addresses, each having an x-coordinate and a y-coordinate;

generating a stream of vector offsets from said vector offset table, including indexing said vector offset table by the x-coordinates and the y-coordinates of said stream of pixel addresses;

generating a stream of raw activation values, including indexing said activation table by the x-coordinates and the y-coordinates of said stream of pixel addresses;

generating a stream of adjusted activation values by performing a computation of said stream of raw activation values with an adjusting function;

generating a stream of scaled offsets, including scaling said stream of vector offsets by corresponding values in said stream of adjusted activation values;

varying said adjusting function; and

generating a stream of output pixels, including adding said stream of scaled offsets to a stream of pixel addresses and reading out said frame buffer with addresses produced thereby.

18. The method of claim 17 wherein said step of initializing a vector offset table includes initializing an x-offset table and a y-offset table, an x-offset and a y-offset together defining a vector offset.

19. The method of claim 17 wherein said step of generating a stream of pixel addresses includes applying a combination of scaling, translation, rotation, shear, and projection transformations on a set of raster-order addresses.

20. The method of claim 17 wherein said adjusting function includes a minimum threshold and a maximum threshold, said step of generating a stream of adjusted activation values includes (i) outputting a first activation value, if a raw activation value is less than said minimum threshold, (ii) outputting a second activation value, if a raw activation value is greater than said maximum threshold;

and (iii) outputting a value between said first activation value and said second activation value otherwise.

21. The method of claim 20 wherein said first activation value is equal to zero.

22. The method of claim 20 wherein said step of varying said adjusting function includes changing both said minimum and maximum thresholds.

23. The method of claim 17 further wherein said step of storing said video signal includes storing first and second fields of each frame of said video signal, and said step of varying said adjusting function is performed at a rate of once per field.

24. The method of claim 17 wherein:

said step of generating a stream of adjusted activation values includes computing the following;

##EQU3## where F is said adjusting function,

S is an adjusted activation value,

R is a raw activation value; and

said step of varying said adjusting function includes incrementing or decrementing Δ

R.

25. The method of claim 24 wherein said adjusting function F is a straight-line function having a slope G.

26. A video effects device comprising a video input buffer for storing an input image as a plurality of pixels, an address generator for producing pixel read out addresses, a video output means for producing an output video signal based on said pixel readout addresses, said address generator comprising:

means for generating an image address, said image address having an x-coordinate and a y-coordinate;

first means for mapping said image address to an offset vector;

second means for mapping said image address to an activation factor;

means for scaling said activation factor to produce a scaled activation factor, the degree of scaling varying with time;

means for combining said scaled activation factor and said offset vector to produce a scaled offset vector; and

means for adding said scaled offset vector to said image address to produce a pixel readout address;

whereby said output video signal is produced by accessing pixels from said video input buffer based on pixel readout addresses.

27. The video effects device of claim 26 wherein the degree of scaling by said means for scaling varies with each field of a frame of video.

28. The video effects device of claim 26 wherein the degree of scaling by said means for scaling varies with each frame of video.

29. The video effects device of claim 26 wherein said offset vector comprises an x-offset and a y-offset, said means for combining includes means for multiplying said scaled activation factor with each of said x-offset and said y-offset.

30. The video effects device of claim 26 wherein said means for scaling includes:

clipping means for setting said scaled activation factor equal to a first activation value or a second activation value if said activation factor falls outside of a scaling range defined by a maximum threshold and a minimum threshold;

gain means for computing said scaled activation factor as a function of said activation factor if said activation factor falls within said scaling range; and

range adjust means for adjusting said minimum and maximum thresholds of said scaling range.

31. The video effects device of claim 30 wherein said gain means includes means for multiplying said activation factor with a gain value.

32. The video effects device of claim 31 wherein said gain value varies with time.

33. The video effects device of claim 30 wherein said function in said gain means is one of a piecewise linear function, a piecewise continuous function, or a combination thereof.

34. The video effects device of claim 26 wherein said means for generating includes a generator of raster-order addresses and a 3-D and perspective effects transformer coupled to receive said raster-order addresses to produce image addresses.

35. The video effects device of claim 26 further including means for inputting a user-specified multiplication factor and means for multiplying said user-specified multiplication factor with said scaled activation factor.

36. The video effects device of claim 26 wherein said first and second means for mapping each includes a lookup table indexed by the x- and y-coordinates of said image address.

37. The video effects device of claim 26 wherein said first and second means for mapping each includes a lookup table indexed by a scalar value and a means for computing said scalar value based on the x- and y-coordinates of said image address.

38. A method for manipulating live-action video to produce video effects in real-time, comprising the steps of:

(a) inputting and storing live-action video in a memory store as a set of input pixels, each input pixel being addressed by an x-coordinate and a y-coordinate;

(b) providing a plurality of offset vectors;

(c) providing a plurality of warp activation values;

(d) providing a mapping relation for mapping a first warp activation factor to produce a second warp activation factor;

(e) selecting an offset vector from among said plurality of offset vectors;

(f) selecting a warp activation value from among said plurality of warp activation values;

(g) mapping a selected warp activation value to produce a mapped warp activation value;

(h) multiplying said mapped warp activation value with a selected offset vector to form a scaled offset vector;

(i) constructing a video signal, including adding said scaled offset vector to a raster-order address to produce a readout address and accessing a pixel from said memory store addressed by said readout address;

(j) altering said mapping relation; and

(k) repeating said steps (e) through (j).

39. The method of claim 38 wherein said step of altering said mapping relation is performed at a rate of once for each field of a video frame.

40. The method of claim 38 wherein said step of altering said mapping relation is performed at a rate of once for each video frame.

41. The method of claim 38 wherein said step of providing offset vectors includes initializing an x-offset table and a y-offset table, whereby each x-offset has a corresponding y-offset which together define an offset vector;

said step of selecting an offset vector includes selecting an x-offset and a y-offset; and

said step of multiplying includes multiplying said mapped warp activation value with a selected x-offset and a selected y-offset.

42. The method of claim 41 wherein said x-offset and y-offset tables are arranged in row and column order;

and said step of selecting an offset vector further includes selecting a pixel coordinate and indexing into the row and column of each of said x-offset and y-offset tables to produce said selected x-offset and said selected y-offset.

43. The method of claim 42 wherein said step of providing a mapping relation includes providing an x-mapping relation and a y-mapping relation, said step of mapping produces an x-mapped warp value and a y-mapped warp value, and said step of multiplying includes multiplying selected x-offset and selected y-offset values respectively by said x-mapped warp value and y-mapped warp value.

44. The method of claim 38 wherein:

said step of providing a mapping relation includes selecting a minimum warp activation threshold and a maximum warp activation threshold;

said step of mapping includes the substeps of (i) setting said mapped warp activation value equal to a min value, if said selected warp activation value is less than said minimum warp activation threshold;

(ii) setting said mapped warp activation value equal to a max value, if said selected warp activation value is greater than said maximum warp activation threshold; and

(iii) setting said mapped warp activation value to a value between said min value and said max value, otherwise; and

said step of altering said mapping relation includes adjusting said minimum warp activation value and said maximum warp activation value.

45. The method of claim 44 wherein said substep (iii) includes defining mapped warp activation values as a function, F, of selected warp activation values.

46. The method of claim 45 wherein said function F is a continuous function.

47. The method of claim 45 wherein said function F is a straight line function.

48. The method of claim 45 wherein said function F is a piecewise linear function, a piecewise continuous function, or a combination thereof.

49. A video effects device for producing live-action video effects on a video display, comprising:

video input means for obtaining a live-action video image;

a video memory to store said live-action video image as a plurality of input pixels, said video memory being addressed by an x-coordinate and a y-coordinate;

means for generating pixel coordinates;

an x-offset table having offset values which are referenced by a first index and a second index;

a y-offset table having offset values which are referenced by said first index and said second index;

means for selecting an x-offset and a y-offset respectively from said x-offset table and said y-offset table, including means for receiving, as said first index, the x-coordinate of a first generated pixel coordinate and for receiving, as said second index, the y-coordinate of said first generated pixel coordinate;

calculation means for computing a sum of a selected x-offset value and the x-coordinate of a second generated pixel coordinate and for computing a sum of a selected y-offset value and the y-coordinate of said second generated pixel coordinate, thereby producing a readout pixel coordinate; and

a video signal generator, coupled to said calculation means, for reading out said video memory to produce a video signal.

50. The video effects device of claim 49 wherein said means for selecting further includes scaling means for scaling each of a selected x-offset and a selected y-offset.

51. The video effects device of claim 49 wherein said means for generating pixel coordinates includes means for generating raster-order addresses and means for transforming said raster order addresses to produce a combination of translation, rotation, and perspective-projection transforms.

52. The video effects device of claim 49 wherein said means for generating includes a first translation, rotation, and perspective transformer and a second translation, rotation, and perspective transformer, and wherein said first generated pixel coordinate is produced by said first transformer and said second generated pixel coordinate is produced by said second transformer.

53. The video effects device of claim 49 wherein said first generated pixel coordinate and said second generated pixel coordinate are the same coordinate.

54. The video effects device of claim 49 further including a second video input means and a video mixing means for combining an image produced by said video output means with an image obtained by said second video input means.

55. A method of producing live-action video effects in real time, comprising the steps of:

selecting a visual effect;

initializing a vector table to contain a plurality of transform vectors for producing a selected visual effect, said vector table organized as rows and columns, each transform vector comprising an x-offset and a y-offset;

following said steps of selecting and initializing, inputting a video signal and storing said video signal as a frame of live-action video in a memory store as a set of pixels, each pixel being addressed by an x-coordinate and a y-coordinate;

producing a stream of pixel addresses;

accessing transform vectors from locations in said vector table addressed by said stream of pixel addresses to produce a stream of transform vectors;

combining said pixel addresses with accessed transform vectors to produce readout addresses; and

reading out pixels from locations in said memory store addressed by said readout addresses and outputting such pixels in a video output signal.

56. The method of claim 55 further including selecting a second visual effect and initializing said vector table to contain a second plurality of transform vectors, thereby producing second live-action video effects.

57. The method of claim 55 wherein said step of producing a stream of pixel addresses includes producing raster-order addresses and taking a 3-D perspective transformation of said raster-order addresses.

58. The method of claim 55 wherein said step of initializing said vector table includes initializing a table of x-offsets with offset values and a table of y-offsets with offset values, each table organized in rows and columns, each vector consisting of an x-coordinate offset paired with a corresponding y-coordinate offset.

59. The method of claim 55 wherein said step of combining further includes multiplying each transform vector with a scalar value.

60. The method of claim 59 wherein said step of combining further includes changing said scalar value over time.

61. The method of claim 59 wherein said step of combining further includes changing said scalar value with each successive field of a frame of live-action video.

62. The method of claim 55 further including combining said video output signal with a second video signal.