Programmable digital video processing system

• US 5,227,863 A
• Filed: 08/07/1990
• Issued: 07/13/1993
• Est. Priority Date: 11/14/1989
• Status: Expired due to Fees

• Current Assignee: Intelligent Resources Integrated Systems Incorporated
• Original Assignee: Intelligent Resources Integrated Systems Incorporated
• Inventors: Brooks, John M., Frederiksen, Jeffrey E., Fields, Craig, Bilbrey, Brett C., Jakobs, Thomas
• Primary Examiner(s): Groody, James J.
• Assistant Examiner(s): Metjahic, Safet
• Application Number:
  US07/564,148
• Time in Patent Office: 1,071 Days
• Field of Search: 358/22, 358/180, 358/181, 358/182, 358/183, 358/185, 358/903, 358/160, 358/148, 364/521, 395/118, 395/133, 395/153
• US Class Current: 348/578
• CPC Class Codes: G06T 15/005 General purpose rendering architectures
• CPC Class Codes: H04N 5/262 Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects

Abstract

Programmable apparatus for digital processing of video signals from multiple sources converted to digital format to provide real-time multiple simultaneous special video effects and suitable for direct interface to a conventional microcomputer bus such as an Apple Macintosh II NuBus. The apparatus includes a matrix of multipliers to do real-time video processing permitting special effects such as fading between at least two video image sources, as well as a priority resolver to control display on a pixel by pixel basis of more than ten sources based upon dynamically programmable priority. In addition, a programmable multiple range thresholder, a hardware window generator capable of generating multiple simultaneous windows, a color look up table and optional image capture capabilities are provided. The apparatus also provides for a light pen input, genlocking and a range of special video effects including zooming, mosaicing, panning and blending.

Claims

1. A digital video processor for processing video source components from a plurality of video sources, comprising:

input means for receiving a plurality of video sources;

source selection means coupled to the input means, for programmably selecting a plurality of video source components from the plurality of video sources to provide a plurality of selected video source components;

blend input means for input of a blend value;

pre-scale multiplier means, coupled to the blend input means, for multiplying the input blend value by a programmable pre-scale constant to provide a pre-scaled blend constant source;

constant selection means, coupled to the pre-scale multiplier means, for programmably selecting a plurality of constants from among a plurality of constant sources including the pre-scaled blend constant source to provide a plurality of selected constants;

multiplier means, coupled to the source selection means and constant selection means, for multiplying each selected source component by a respective selected constant to provide a plurality of multiplier outputs;

summing means, coupled to the multiplier means, for summing the plurality of outputs to provide at least one summed multiplier output.

2. The digital video processor of claim 1 wherein at least one summed multiplier output is coupled back to the source selection means as one of the plurality of video sources.

3. The digital video processor of claim 1 wherein the multiplier means comprises a multiplier array which is comprised of a plurality of multipliers at least one of which is adapted to perform at least two multiplications within a pixel time period.

4. The digital video processor of claim 1 wherein the multiplier means comprises a plurality of multipliers each of which is adapted to perform a plurality of multiplications within a pixel time period.

5. The digital video processor of claim 3 wherein the multiplier array is comprised of a plurality of multiplier arrays each having inputs and outputs, wherein the output of at least one of said plurality of multiplier arrays is coupled to the input of at least one other of said plurality of multiplier arrays.

6. The digital video processor of claim 3 wherein the pre-scale multiplier means and the multiplier means produce fading effects.

7. The digital video processor of claim 3 wherein the pre-scale multiplier means and the multiplier means produce dissolving effects.

8. The digital video processor of claim 1 wherein each video source comprises signals representative of a plurality of pixels, and wherein the blend input means comprises means for input on a pixel by pixel basis of a unique blend value for each pixel.

9. The digital video processor of claim 1 wherein the multiplier means comprises an array of multipliers.

10. The digital video processor of claim 1 wherein the multiplier means comprises a three by three array of multipliers adapted to multiply each of three components of up to three selected video sources by a set of up to nine selected constants to produce up to nine products.

11. The digital video processor of claim 10 wherein the summing means comprises three summing circuits each adapted to sum three products to produce three summed multiplier components, and means for combining the summed multiplier components to form a summed multiplier output.

12. The digital video processor of claim 11 wherein the summing means further comprises means for providing a total sum of the three summed multiplier components and for selecting predetermined most significant bits of the total sum to provide a partial sum multiplier output.

13. The digital video processor of claim 1 wherein the multiplier means comprises a four by four array of multipliers adapted to multiply each of up to four components of up to four selected video sources by a set of up to sixteen selected constants to produce up to sixteen products.

14. The digital video processor of claim 1 wherein the multiplier means comprises an n by m array of multipliers, where n and m are both positive integers, adapted to multiply each of n components of up to m selected video sources by a set of up to nxm selected constants to produce up to nxm products.

15. The digital video processor of claim 14 wherein n equals m.

16. The digital video processor of claim 14 wherein n does not equal m.

17. The digital video processor of claim 14 wherein the summing means comprises n summing circuits each adapted to sum m products to produce n summed multiplier components, and means for combining the summed multiplier components to form a summed multiplier output.

18. The digital video processor of claim 17 wherein the summing means further comprises means for providing a total sum of the m summed multiplier components and for selecting a subset of predetermined most significant bits of the total sum to provide a partial sum multiplier output.

19. The digital video processor of claim 14 wherein the array of multipliers can be configured to perform blending.

20. The digital video processor of claim 14, wherein at least one video source is comprised of multiple video components, and wherein the array of multipliers can be configured to perform matrix transformations using all components of one video source.

21. The digital video processor of claim 20 wherein said video components are comprised of R, G, and B components.

22. The digital video processor of claim 20 wherein said video components are comprised of Y, U, and V components.

23. The digital video processor of claim 20 wherein said video components are comprised of Y, I, and Q components.

24. The digital video processor of claim 14 wherein the array of multipliers is selectively configurable to perform frame averaging.

25. The digital video processor of claim 9 wherein the array of multipliers is selectively configurable to perform filtering using one of said video components of said video source.

26. The digital video processor of claim 25 wherein said filtering is comprised of image processing, comprised of at least one of softening, sharpening, focusing, blurring, edge detecting, and highlighting.

27. The digital video processor of claim 9 wherein the array of multipliers is selectively configurable to perform simultaneous blending and matrix transformations.

28. The digital video processor of claim 9 wherein the array of multipliers is selectively configurable to perform color transformations.

29. The digital video processor of claim 3 wherein at least one video source is comprised of multiple video components, and each of the pixels has associated therewith data for said multiple video components.

30. The digital video processor of claim 29 wherein said multiple video components is comprised of a control component.

31. The digital video processor of claim 30 wherein the control component is used to control the blend input means on a pixel by pixel basis.

32. The digital video processor of claim 30 wherein the control component is used to control the pre-scale multiplier means on a pixel by pixel basis.

33. The digital video processor of claim 30 wherein the control component is used to control the constant selection means on a pixel by pixel basis.

34. The digital video processor of claim 30 wherein the control component is used to control the multiplier means on a pixel by pixel basis.

35. The digital video processor of claim 30 wherein the control component is used to control the summing means on a pixel by pixel basis.

36. The digital video processor of claim 30 wherein the control component is used to control at least one of the tint, contrast, hue, and brightness of the individual pixels.

37. The digital video processor of claim 30 wherein the control component is used to control the expansion of the individual pixels from a single video component other than the control component to all of the multiple video components.

38. The digital video processor of claim 30 wherein the control component controls video processing effects on a pixel by pixel basis.

39. The digital video processor of claim 30 wherein said control component is further comprised of multiple subcomponents.

40. The digital video processor of claim 30 wherein said control component is comprised of control information and non-control data.

41. The digital video processor of claim 1 wherein the blend input means is responsive to a constant generator.

42. The digital video processor of claim 41 wherein the constant generator is comprised of a plurality of constant generators.

43. The digital video processor of claim 41 wherein said constant generator is a pseudo-random generator.

44. The digital video processor of claim 43 wherein the pseudo-random sequence generator produces a linear pseudorandom sequence.

45. The digital video processor of claim 43 wherein the pseudo-random sequence generator produces a gaussian pseudorandom sequence.

46. The digital video processor of claim 41 wherein said constant generator is an external noise source.

47. The digital video processor claim 41 wherein said constant generator provides an output, wherein said output can be used as one of a blend value, a prescale value, and a video source.

48. The digital video processor of claim 1 wherein the multiplier means include overflow and underflow prevention means for substituting predefined limits responsive to detecting a sum outside a range of predefined overflow and underflow conditions.

49. The digital video processor of claim 1 wherein the multiplier means comprises a plurality of multipliers each adapted to perform multiplications at least at a real-time video rate.

50. The digital video processor of claim 1 further comprising:

designation means for assigning a priority designation to each of the plurality of sources including the summed multiplier output; and

resolver means, coupled to the designation means, for generating a source selection signal in response to the priority designations.

51. The digital video processor of claim 50 wherein the designation is selectively configurable to perform mixing effects.

52. The digital video processor of claim 50 further comprising a threshold means, coupled to the multiplier means, input means and resolver means, for comparing a programmably selected video source including the summed multiplier output to a plurality of programmable threshold ranges, and generating a priority designation for an associated source in response thereto.

53. The digital video processor of claim 52 wherein the threshold means can be programmed to perform wipe effects.

54. The digital video processor of claim 52 further comprising means for allowing a user to modify the programmably selected video sources.

55. The digital video processor of claim 52 further comprising means for allowing a user to modify the programmable threshold ranges.

56. The digital video processor of claim 52 further comprising output means, coupled to the resolver means, input means and the multiplier means, for selecting and coupling one of the video sources to an output in response to the source selection signal.

57. The digital video processor of claim 56 further comprising window means, coupled to the resolver means, for generating a plurality of window source and window priority designations.

58. The digital video processor of claim 57 wherein the resolver means is responsive to the window source and window priority designations for selecting and coupling selected ones of the video sources for output by the output means.

59. The digital video processor of claim 56 further comprising:

a color look up table means, responsive to the input means, for providing a video signal output coupled to the output means as a video source, and for providing an associated priority designation coupled to the resolver means.

60. The digital video processor of claim 59 wherein the source selection signal generated by the resolver means provides for selection of one of the following sources for output to the output means:

the input means, the multiplier means, or the color look up table means.

61. The digital video processor of claim 60 wherein the source selection signal generated by the resolver means can select between any of a plurality of input sources.

62. The digital video processor of claim 59 wherein the color look up table also includes a control component for providing a priority designation coupled to the resolver means.

63. The digital video processor of claim 59 wherein the color look up table means can be configured to perform colorization (also known as false coloring, or pseudo-coloring).

64. The digital video processor of claim 56 wherein each video source is comprised of a plurality of horizontal lines of pixels, and further comprising a programmable line control means for providing a plurality of control signals associated with each horizontal line for programmably altering source selections and multiplier constant selections.

65. The digital video processor of claim 64 further comprising pixel control means for providing, on a pixel by pixel basis, a blend constant value, and control signals for altering threshold ranges, output source selections and programmable threshold ranges.

66. The digital video processor of claim 65 wherein each video source comprises signals representative of a plurality of pixels and at least one associated control component, and wherein the designation means for assigning a priority designation is responsive to the threshold means, line control means, input means, pixel control means, and the control component.

67. The digital video processor of claim 65 wherein the designation means in conjunction with the threshold means and the pixel control means, selectively produces chrominance key effects on a pixel by pixel basis.

68. The digital video processor of claim 65 wherein the designation means in conjunction with the threshold means and the pixel control means selectively produces luminance key effects on a pixel by pixel basis.

69. The digital video processor of claim 56 further comprising frame control means for providing, on a frame by frame basis, a blend constant value, and control signals for altering threshold ranges, output source selections and programmable threshold ranges.

70. The digital video processor of claim 59 wherein the color look up table means is further comprised of multiple color look up table means, each of which is independently and selectively configurable for different video sources, outputs, modes, and priorities.

71. The digital video processor of claim 70 wherein the multiple color look up table is further comprised of a programmable means to select modes.

72. The digital video processor of claim 52 wherein each video source comprises a plurality of pixels and further comprising means for providing control signals for altering threshold modes, selected video sources for output, and programmable threshold ranges on a pixel by pixel basis.

73. The digital video processor of claim 52 wherein the designation means can be used in conjunction with the threshold means to selectively produce at least one of luminance and chrominance key effects.

74. The digital video processor of claim 59 wherein the color look up table means provides for selectively remapping of individual video components responsive to the input video source.

75. The digital video processor of claim 1 further comprising memory means for storing selected video source signals in the form of digital data.

76. The digital video processor of claim 1 wherein the pre-scale multiplier means can perform anti-aliasing of computer text and graphics over a plurality of video sources.

77. A video processing system comprising:

input means for receiving signals from a plurality of video sources;

multiplier the signals from means coupled to the input means, for multiplying at least one programmably selected video source with programmably selected constants to generate a multiplier video source;

designation means for assigning a priority designation to each of of the video sources;

resolver means, coupled to the designation means, for generating a source selection signal in response to the priority designation;

output control means, coupled to the resolver means, input means and multiplier means, for coupling of a selected one of the video sources to an output in response to the source selection signal.

78. The video processing system of claim 77 further comprising threshold means for comparing a programmably selected video source to a plurality of programmable ranges and for generating a priority designation for an associated source in response thereto.

79. The video processing system of claim 78 further comprising window means for generating a plurality of window priority designations and designation of respective associated video sources.

80. The video processing system of claim 78 wherein each video source comprises a plurality of pixels and further comprising means for providing control signals for altering threshold range modes, selected video sources for output, and programmable threshold ranges on a pixel by pixel basis.

81. The video processing system of claim 78 further comprising of a functional look up table means for controlling the designation means, resolver means, and output control means, and responsive to the threshold means.

82. The video processing system of claim 77 further comprising a video bus, wherein a plurality of video processing systems can be connected together via the video bus.

83. The video processing system of claim 82 wherein the video bus is comprised of two, independent bi-directional channels.

84. The video processing system of claim 83 wherein each channel is comprised of four video components.

85. The video processing system of claim 84 wherein the bi-directionality is independent for each component of each channel.

86. The video processing system of claim 82 wherein a plurality of video processing systems are connected to the video bus wherein a first video processing system provides output to the next video processing system responsive to the input means, wherein, thereafter, each next video processing system provides output to its respective output means responsive to the input means and the output from the previous video processing system.

87. The video processing system of claim 82 wherein the video bus is comprised of a plurality of independent bi-directional channels.

88. The video processing system of claim 87 wherein each channel is comprised of a plurality of video components.

89. The video processing system of claim 77 further comprising threshold means for comparing one of a programmably selected video source, portion of a video source, and a control source, to a plurality of programmable ranges and for generating a priority designation in response thereto.

90. The video processing system of claim 77 further comprising color look up table means for providing a video source, and a priority designation for an associated source in response to input of a video source.

91. The video processing system of claim 90 wherein the color look up table means is further comprised of multiple color look up tables, each of which is selectively configurable independently for different video sources.

92. The video processing system of claim 91 wherein the multiple color look up tables are further comprised of a programmable means to reconfigure the multiple tables.

93. The video processing system of claim 92 wherein the multiple reconfigurable color look up tables can be combined into a larger color look up table.

94. The video processing system of claim 77 further comprising capture means for digitizing an analog video source to provide a digital video source and for coupling the digital video source to the input means to provide one of the video sources.

95. The video processing system of claim 94 further comprising memory means for storing selected video source signals in the form of digital data.

96. The video processing system of claim 95 wherein the memory means is capable of storing data representations of the control components associated with the video source signals.

97. The video processing system of claim 95 wherein the memory means can be coupled to the output means to perform zoom effects.

98. The video processing system of claim 95 wherein the memory means can be coupled to the output means to perform panning effects.

99. The video processing system of claim 95 wherein the memory means can be coupled to the output means to perform mosacing effects.

100. The video processing system of claim 77, further comprising memory means comprised of a plurality of memory banks, each of which can be programmably configured to store video source signals from any of a plurality of video sources.

101. The video processing system of claim 100 wherein the plurality of memory banks can be programmably combined to form a plurality of larger memory banks, each consisting of a plurality of memory banks.

102. The video processing system of claim 77 wherein each video source comprises signals representative of a plurality of pixels, and further comprising a blend input means for the input on a pixel by pixel basis of a unique blend value for each pixel.

103. The video processing system of claim 77 wherein video input means include at least one video source is in a 24 frames-per-second film format.

104. The video processing system of claim 77 wherein the input means can receive a plurality of video sources, each in different video formats.

105. The video processing system of claim 77 wherein the input means can receive a plurality of video sources, at least one of which is in an interlaced format.

106. The video processing system of claim 77 wherein the input means can receive a plurality of video sources, at least one of which is in a non-interlaced format.

107. The video processing system of claim 77 wherein at least one video source is responsive to a pseudo-random sequence generator.

108. A method for processing video source components from a plurality of video sources comprising the steps of:

programmably selecting a plurality of video source components from the plurality of video sources to provide a plurality of selected video source components;

providing an input blend value;

multiplying the input blend value by a programmable pre-scale ratio to provide a pre-scaled blend source;

programmably selecting a plurality of constants from among a plurality of constant sources including the prescaled blend source to provide a plurality of selected constants;

multiplying each selected source component by a respective selected constant to provide a plurality of multiplier outputs;

summing the plurality of outputs to provide at least one summed multiplier output.

109. The method of claim 108 further comprising the step of coupling at least one summed multiplier output back as one of the plurality of video sources for selection of selected video source components.

110. The method of claim 108 further comprising the steps of assigning a priority designation to each of the plurality of sources including the summed multiplier output and generating a source selection signal in response to the priority designation.

111. The method of claim 110 further comprising the steps of comparing programmably selected video sources including the summed multiplier output to a plurality of programmable ranges and generating a priority designation for an associated source in response thereto.

112. The method of claim 111 further comprising the step of selecting and coupling one of the video sources to an output in response to the source selection signal.

113. The method of claim 112 further comprising the step of displaying the output.

114. The method of claim 112 further comprising the step of generating a plurality of window priority designations each associated with a window source.

115. A video processing system comprising:

input means for receiving signals from a plurality of video sources;

multiplier means for multiplying signals from at least one selected video source with selected constants to generate a multiplied video source;

clock means for providing programmable frequency timing signals for the input means and multiplier means comprising means for dividing the frequency of a reference clock signal by a first programmable number to provide a divided reference signal;

means for dividing the frequency of a selected signal by a second programmable number to provide a divided signal;

means for comparing the divided reference signal with the divided signal to generate a control signal dependent upon the difference between the frequencies of divided signals;

voltage controlled oscillator for generating a VCO signal having a frequency controlled by the control signal;

means for selecting the VCO signal as the selected signal and wherein the selected signal is utilized as a timing signal.

116. The system of claim 115 further comprising a chroma divider means for dividing the frequency of the selected signal by a third programmable number to provide a chroma clock signal, and a pixel divider means for dividing the frequency of the selected signal by a fourth programmable number to provide a pixel clock signal.

117. The system of claim 116 wherein the means for selecting further comprises means for programmably selecting between the VCO signal and at least one external clock signal to provide the selected signal.

118. The system of claim 115 further comprising means for providing a variable rate pixel clock which allows variable screen resolutions on a continuously variable range.

119. The video processing system of claim 115 further comprising output gamma correction means as a means for correcting for non-linear response of the video monitor systems.

120. The video processing system to claim 115 wherein the output means can produce a video signal in a plurality of different video formats.

121. The video processing system of claim 115 wherein the output means can produce a video signal in a plurality of different video formats, at least one of which is in an interlaced format.

122. The video processing system of claim 115 wherein the output means can produce a video signal in a plurality of different video formats, at least one of which is in a non-interlaced format.

123. The video processing system of claim 115 wherein the output means can product a signal in a plurality of different screen resolutions.

124. The video processing system of claim 115, further comprised of clock timing circuitry comprising system registers, and which selectively provides for modification of the system registers responsive to a security code key.