System for spatially transforming images

• US 4,472,732 A
• Filed: 10/07/1981
• Issued: 09/18/1984
• Est. Priority Date: 04/10/1981
• Status: Expired due to Term

• Current Assignee: Ampex Corporation (Delta Information Systems, Inc.)
• Original Assignee: Ampex Corporation (Delta Information Systems, Inc.)
• Inventors: Gabriel, Steven A., Bennett, Phillip P.
• Primary Examiner(s): Masinick, Michael A.
• Application Number:
  US06/310,901
• Time in Patent Office: 1,077 Days
• Field of Search: 358/21 R, 358/22, 358/104, 358/105, 358/160, 340/146.3 H, 364/515, 364/521, 364/522
• US Class Current: 348/452
• CPC Class Codes: G06T 3/606 by memory addressing or mapping
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 9/74 for obtaining special effects H04N9/65 - H04N9/73 take precedence

Abstract

A system for spatially transforming images by separate transformation of each dimension of the image is exemplified by a raster scan television system which includes for each color component a transposing memory providing a change of scan direction from horizontal to vertical, a vertical transformation system transforming in the vertical direction the vertically scanned video information, a second transposing memory coupled to receive vertically transformed video information and provide a change of scan direction from vertical back to horizontal, and a horizontal transformation system coupled to horizontally transform the horizontally scanned video signal to produce a color component output signal. The transformation system is controlled by a transform composer factorizor which receive input commands designating X and Y pretranslations, X and Y size control, Z axis rotation angles, and X and Y post translations to produce a commanded composite transformation which is then factored into horizontal and vertical components.

Claims

1. A deinterlace filter comprising a motion detector coupled to receive simultaneously at least a most recent field and a third most recent field of interlaced video data and to compare corresponding pixels on corresponding lines of said two fields of data, the motion detector generating a motion signal if two compared pixels differ by a threshold amount;

and a deinterlacing circuit coupled to receive the most recent field and a next most recent field of data and the motion signal and output deinterlaced frames of data comprising the most recent field of data with the intermediate lines thereof completed by corresponding lines of the next most recent field in the absence of the motion signal and completed by the average of corresponding data points of the two adjacent lines of the most recent field in the presence of the motion signal.

2. The deinterlace filter according to claim 1 above, wherein the motion detector comprises a first subtractor coupled to subtract sequential pixels of data defining the third most recent field from corresponding pixels of data defining the most recent field;

a second subtractor coupled to subtract sequential pixels of data defining a fourth most recent field from corresponding pixels of data defining the second most recent field, first and second threshold detectors coupled to receive the respective difference outputs of the first and second subtractors and generate first and second preliminary motion signal outputs whenever a respective difference output exceeds a threshold amount;

a first delay circuit coupled to receive the first preliminary motion signal and delay the first preliminary motion signal by one pixel time to generate a delayed first preliminary motion signal;

a second delay circuit coupled to receive the second preliminary motion signal and delay the second preliminary motion signal by one pixel time to generate a delayed second preliminary motion signal; and

an OR gate coupled to generate the motion signal in response to the occurrence of the first preliminary motion signal, the delayed first preliminary motion signal or the delayed second preliminary motion signal.

3. A deinterlace filter for interlaced raster scanned video data receiving sequential fields of video data one pixel at a time with each pixel of video data being received during a pixel time and outputing a deinterlaced frame of video data for each field received, the deinterlace filter comprising:

a first threshold detection system coupled to simultaneously receive a most recent field of video data and a third most recent field of video data in raster scan line order, to compare corresponding pixels of the most and third most recent fields of video data and output a first preliminary motion signal whenever a difference between two corresponding pixels exceeds a given threshold;

a second threshold detection system coupled to simultaneously receive a second most recent field of video data and a fourth most recent field of video data as the most recent and third most recent fields of data are received synchronously therewith, to compare corresponding pixels of the second and fourth most recent fields of video data and to output a second preliminary motion signal whenever a difference between two corresponding pixels exceeds the given threshold;

a first delay circuit coupled to receive the first preliminary motion signal and output a delayed first preliminary motion signal which is delayed relative to the first preliminary motion signal by one pixel time;

a second delay circuit coupled to receive the second preliminary motion signal and output a delayed second preliminary motion signal which is delayed relative to the second preliminary motion signal by one pixel time;

an OR circuit coupled to generate a motion signal in response to the occurrence of a first preliminary motion signal, a delayed first preliminary motion signal, or a delayed second preliminary motion signal; and

a deinterlace circuit coupled to receive the mcst recent and second most recent fields of video data and the motion signal and to output the most recent field as a first output, the deinterlace filter completing the alternate missing lines of the most recent field by outputing the second most recent field as a second output in the absence of the motion signal, the output being generated as an average of corresponding pixels of the immediately preceding and succeeding lines of video data appearing in the most recent field in response to the occurrence of the motion signal.

4. The deinterlace filter according to claim 3 above, wherein the first and second threshold detection systems each include a subtractor receiving two field inputs and outputing the difference therebetween, a difference register coupled to store the subtractor difference output, a threshold detector coupled to receive the difference output and generate an output signal whenever the difference exceeds the given threshold and an output register coupled to receive the threshold detector output and generate the preliminary motion signal in response thereto.

5. The deinterlace filter according to claim 3 or 4 above, wherein the deinterlace circuit provides the first and second outputs phased relative to the first and second motion detection systems such that each first output pixel is synchronous with the delayed first preliminary motion signal corresponding thereto.

6. The deinterlace filter according to claim 5 above, wherein the deinterlace circuit includes first and second registers coupled to receive and store pixels of video data representing the most recent field;

third and fourth registers coupled to receive and store pixels of video data representing the second most recent field;

a third delay circuit coupled to receive the pixels of video data from the second register and delay the received data by at least one pixel time to provide the first output;

a fourth delay circuit coupled to receive the pixels of video data from the fourth register and the received data by at least one pixel time to generate second most recent field output video data;

an averaging circuit coupled to receive the input and output of the third delay circuit and generate the average thereof as averaged output video data; and

a multiplexer coupled to receive the second most recent field output video data, the averaged output video data and the motion signal and to output the second most recent field output video data as the second output in response to the absence of the motion signal and to output the averaged output video data as the second output in response to the occurrence of a motion signal.

7. The deinterlace filter according to claim 6 above, wherein the third and fourth delay circuits are coupled to alternatively provide a delay of one line time in response to a control signal indicating a mode of operation in which incoming fields of data have been scanned in a horizontal direction.

8. A raster scan television video signal deinterlace system comprising:

five field stores coupled to receive and store sequential fields of video signal as they are received, the fields being stored in the field stores on a sequential rotating basis with the oldest stored field being replaced by a newest incoming field as it is received, the field stores being coupled to output the first through fourth newest fields in parallel in raster scan order as each incoming field is received;

a motion detector coupled to receive the first through fourth newest fields and to compare the first and third newest fields and second and fourth newest fields on a pixel by pixel basis as they are received and to output a motion signal if the comparison indicates a difference of more than a threshold amount; and

a de-interlace circuit coupled to receive the first and second most recent fields of data and output a deinterlace frame of data for each field received in response to the motion signal, the de-interlace circuit outputing the first most recent field as a portion of the frame and completing the pixels of the intermediate lines of the frame with corresponding pixels from the second most recent field in the absence of the motion signal and as the average of the pixels in the first newest field immediately above and below the intermediate line in the presence of the motion signal, the motion signal being synchronized to indicate motion corresponding to a pixel of an output intermediate line when motion is detected at the intermediate line pixel position between the second fourth most recent fields and when motion is detected at the pixel positions immediately above or immediately below the intermediate line pixel position between the first and third most recent fields.

9. The deinterlace system according to claim 8 above, wherein the de-interlace circuit further comprises a delay disposed in series with the data for the second newest field and a selector circuit coupled to pass data from the output of the delay when the first most recent field is an even field containing even lines and to pass data from the input to the delay when the most recent field is an odd field containing odd numbered lines.

10. A transposing memory arranged in at least two dimensions for an array of data having a plurality of data points comprising:

a plurality of data storage elements, each operating on a memory cycle having a sequentially phased rotating relationship relative to the memory cycles of other data storage elements;

an address and control system coupled to operate the data storage elements on a sequentially phased rotating basis, the address and control system including a timing generator providing a timing and control enable signal and a timing and control circuit coupled to start a memory sequence for an array of data in response to the timing and control enable signal, the timing generator being responsive to a line signal providing a line start indication a predetermined time prior to the occurrence of valid data for each line of the array and being coupled to generate the timing and control enable signal in a revolving sequence time relationship to the line start indication such that for the first line the first phase memory cycle is synchronized to process the first valid data point for the array with the timing and control enable signal being generated one data point time earlier for each successive scan line until the data storage element operating on the last sequentially phased memory cycle process the first valid data of a scan line with the revolving sequence time relationship then being successively repeated until all data points of the scan line have been processed by the transposing memory.

11. A transposing memory for an array of data points having rows and columns, the memory comprising:

a timing generator responsive to a line blank signal indicating an end of a valid data time for each scan line of the array and the start of the next valid data time for a next scan line of the array with a predetermined time period occurring between the start indication and the actual occurrence of valid data for the array, and a second signal occurring between the end of data and start of next valid data indications of the line blank signal after the last valid data of one array and before the first valid data of a next array, the timing generator being coupled to be reset by the second signal and to cyclically generate a timing and control enable signal starting a predetermined time after the start of valid data indication for a first line of each array and one data point time earlier for each subsequent line of the array until a last segment of a sequentially segmented store processes a first valid data point for a line, the cycle then being repeated until all lines of an array have been processed;

a timing and control circuit coupled to generate timing signals for the transposing memory in response to the timing and control enable signals, the timing signals being synchronized to cause the processing of a first valid data point in each array of data by a first store segment;

a vertical counter coupled to be preset to a preset count in response to the timing generator, the preset count being an address start count in a horizontal scan mode and in a vertical scan mode being an address start count for a first scan line and being cyclically decremented for each scan line until the timing cycle of the timing and control enable signal repeats, the vertical counter being incremented for each scan line in a horizontal mode and for each data point during a vertical scan time in the vertical mode;

a horizontal counter coupled to be preset to a starting address count after each scan line in a horizontal mode and after each array in a vertical mode, the horizontal counter being incremented after each store segment has sequentially experienced a memory cycle in the horizontal mode and each time the timing and control enable signal recycles to its initial timing relationship in the vertical mode;

a first plurality of memory segments coupled to process sequential data points on a cyclic, sequential basis in response to timing signals from the timing and control generator; and

a second plurality of memory segments less than the first plurality, each segment of the second plurality being coupled to operate synchronously and in parallel with a different segment of the first plurality.

12. The transposing memory according to claim 11 above, further comprising an address map coupled to receive the count states of the vertical and horizontal counters and provide to the first and second plurality of memory segments actual address corresponding to the count states.

13. The transposing memory according to claim 12 above, further comprising a flip-flop coupled to be set in a carry state and generate a carry output signal in a vertical mode each time the plurality of memory segments recycle to a memory cycle by a first segment and to be set to a noncarry state in which a carry output signal is not generated at the address start count of the vertical counter and every nth increment of the vertical counter thereafter where n is the number of memory segments in the first plurality of memory segments.

14. A television video signal chroma predecimation and interpolation decimation filter system comprising:

a buffer storing at least three lines of pixel data, the buffer operating on a cyclic sequence of memory cycles in which during 1/8 of the memory cycles in each cyclic sequence are devoted to writing a first newest, incoming line of data into address locations occupied by a fourth newest line of data, 2/8 of the memory cycles in each cyclic sequence are devoted to reading pixel data for a second newest line of data and partially predecimated copies thereof;

1/8 of the memory cycles in each cycling sequence are devoted to writing predecimated data received from a predecimation filter and 4/8 of the memory cycles in each cyclic sequence are devoted to reading pixel data for a third newest line of video data;

a predecimation filter coupled to receive pixel data for the second newest line of data and predecimated copies thereof from the buffer and output to the buffer filtered smaller sized copies of the data received;

an interpolation decimation filter coupled to receive from the buffer a plurality of sequential pixels of data defining a portion of the third newest line of data and the predecimated copies thereof stored at a selected address and to generate a filtered pixel of data as a selected filter function of the plurality of sequential pixels.

15. The filter system according to claim 14 above, wherein the predecimation filter generates an output image which is half the size of the image impact thereto.

16. The filter system according to claim 15 above, wherein the predecimation filter has the weighting function 3/32, 8/32, 10/32, 8/32, 3/32 relative to input pixel data distributed about the image point of the filtered output pixel data.

17. The filter system according to claim 14, 15 or 16 above, wherein the interpolation decimation filter is a four point filter which receives pixel data, weights pixel data and accumulates weighted pixel data for four sequential pixels of data before outputing a filtered pixel of interpolated data as the accumulation of the four weighted pixels.

18. The filter system according to claim 17 above, wherein the interpolation decimation filter includes a coefficient store providing weighting coefficients in dependence upon the size of copy from which the source image is taken, the change of size imparted by the interpolation decimation filter and the location of a pixel of data being weighted relative to a source address point.

19. The filter system according to claim 18 above, wherein the buffer memory operates on a cyclic sequence of memory cycles in which 1/8 of the cycles are devoted to writing incoming new pixel data, 1/8 of the cycles are devoted to writing predecimated pixel data, 2/8 of the cycles are devoted to reading data for predecimation and 4/8 of the cycles are devoted to reading data for use by the decimation interpolation filter.

20. The filter system according to claim 19 above, wherein the buffer memory operates upon two pixels of data in parallel during each memory cycle.

21. An array transformation system comprising:

a first transposing memory coupled to receive a serial sequence of data points in a source array serialized by scanning in a first dimension;

the first transposing memory being selectively operable to output the data points of the source array in the serial sequence in which they are received or in a serial sequence representing a scan in a second dimension different from the first dimension;

a first predecimation filter coupled to receive the data points output from the first transposing memory, the first predecimation filter generating at least one less than full size filtered copy of each scan line from the data points therefor;

a first interpolation decimation filter coupled to receive the full size and at least one less than full size copies of the data points for first dimension scan lines and to output a serial sequence of data points scanned in a third dimension for a target array as a filtered function of the data points above a first source address point within a selected size copy;

a second transposing memory coupled to receive the serial sequence of data points from the first interpolation decimation filter scanned in the third dimension and to output the data points in scan line data point sequences scanned in the second dimension;

a second predecimation filter coupled to receive the data points output from the second transposing memory, the second predecimation filter generating at least one less than full size filtered copy of each third dimension scan line from the data points therefor;

a second interpolation decimation filter coupled to receive the full size and at least one less than full size copies of the data points for third dimension scan lines and to output a serial sequence of data points scanned in a fourth dimension for a target array as a filtered function of the data points about a second source address point within a selected size copy; and

first and second source address generators coupled to provide to the first and second interpolation decimation filters serial sequences of respectively first and second source address points corresponding to respectively serial sequences of third and fourth target dimension data points in accordance with a desired transformation between the source and target array.

22. The array transformation system according to claim 21 above wherein the source and target arrays are two-field interlaced raster scanned television images and further comprising a single deinterlace filter disposed between the first transposing memory and the first predecimation filter.

23. The array transformation system according to claim 22 above wherein the array transformation system transforms a luminance component of the images and further comprising second and third transformation systems as defined for the luminance transformation system and coupled to transform two chrominance components of the images, the first and second transposing memories of the second and third transformation systems sharing addressing circuitry with the first and second transposing memories respectively of the luminance transformation system.

24. The array transformation system according to claim 23 above, further comprising second, third and fourth array transformation systems as set forth therein and a switcher coupled to receive the transformed arrays of the four transformation system and to output said arrays in a selected combination.

25. An interpolation decimation filter for transforming an array of data in a single dimension, the filter comprising:

an eight segment line buffer coupled to receive and store a line of data of an array with any group of eight different sequential data points being stored in eight different line buffer segments;

a line buffer address circuit coupled to receive a source data point address and to address the eight line buffer segments to output eight source data points therefrom disposed about the source data point address;

a coefficient store storing filter function coefficients defining a magnitude relationship between the eight source data points and a filtered data point, the coefficient store outputing eight coefficients corresponding respectively to the eight data points in response to address input representative of array size changes and the relative position of source data points to the source data point address; and

a filter circuit coupled to generate a filtered data point as an output in response to the eight source data points and the eight coefficients corresponding thereto.

26. A predecimation filter coupled to receive sequential data points representing single dimension scan lines of a multidimensional array of data points, the predecimation filter comprising:

a plurality of line buffers coupled to receive data points for a scan line on a revolving basis with data points for an incoming scan line replacing data points for an oldest scan line; and

a filter circuit coupled to receive copies of data points for scan lines and output filtered half size copies in response thereto, the line buffers being operated in pairs in a swapping arrangement wherein data points are swapped back and forth as they are recurrently passed through the filter circuit from one of the pair to the other, the array being successively reduced to half its prior size each time it passes through the filter circuit, the predecimation filter outputing the original full size and successive half size copies of the data points of the array as they are passed through the filter circuit.