Combined intraframe and interframe transform coding system

• US 4,704,628 A
• Filed: 07/16/1986
• Issued: 11/03/1987
• Est. Priority Date: 07/16/1986
• Status: Expired due to Term

• Current Assignee: Magnitude Compression Systems Incorporated
• Original Assignee: Compression Labs Incorporated (Asure Software)
• Inventors: Chen, Wen-hsiung, Elliott, James P., Rackett, Albert E., Newell, Robert E. G., Nichols, Ralph E.
• Primary Examiner(s): Britton, Howard W.
• Application Number:
  US06/886,634
• Time in Patent Office: 475 Days
• Field of Search: 358/136, 358/135, 358/133, 375/27
• US Class Current: 348/400.1
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/126 Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
• CPC Class Codes: H04N 19/13 Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
• CPC Class Codes: H04N 19/60 using transform coding
• CPC Class Codes: H04N 19/619 the transform being operated outside the prediction loop
• CPC Class Codes: H04N 19/62 by frequency transforming in three dimensions H04N19/63 takes precedence
• CPC Class Codes: H04N 19/649 the transform being applied to non rectangular image segments
• CPC Class Codes: H04N 19/91 Entropy coding, e.g. variable length coding [VLC] or arithmetic coding

Abstract

An apparatus and method for signal processing employing intraframe and interframe variable prediction transform coding. Images are represented by sequential frames of two-dimensional arrays of digital signals. The digital signals are transformed to form transform coefficients for each frame. Predicted transform coefficients are formed using sets of variable prediction factors. The predicted transform coefficients for each frame are compared with corresponding actual transform coefficients for the frame to form transform coefficient difference signals. The difference signals are processed to control their range of values. The processed difference signals are statistically coded such that the more frequently occurring values are represented by shorter code lengths and the less frequently occurring values are represented by longer code lengths. The coded signals are stored in a buffer memory for transmission.

Claims

1. A method for processing image signals to reduce the amount of data utilized to represent images where the images are defined by a sequence of frames where each frame includes a two-dimensional array of digital signals representing the image, the steps comprising,

transforming the digital signals to form transform coefficients for each of a plurality of frames in the sequence of frames,

forming predicted transform coefficients for a first frame using variable prediction factors,

comparing said predicted transform coefficients with corresponding transform coefficients of a second frame to provide transform coefficient difference signals for each of the coefficients of said second frame,

processing said transform difference signals to form processed difference signals having a controlled range of values,

coding said processed difference signals to form statistically coded signals such that the more frequently occurring values of coded signals are represented by shorter code lengths and the less frequently occurring values of coded signals are represented by longer code lengths.

2. The method of claim 1 wherein said variable prediction factors are determined as a function of frame-to-frame differences in said images.

3. The method of claim 2 wherein said variable prediction factors are determined in response to frame-to-frame comparisons of said transform coefficients.

4. The method of claim 2 wherein said variable prediction factors are determined as a function of frame-to-frame comparisons of said digital signals.

5. A method for processing image signals to reduce the amount of data utilized to represent images where the images are defined by a sequence of frames where each frame includes a two-dimensional array of blocks where each block includes a two-dimensional array of digital signals representing the portion of an image within the block, the steps comprising,

transforming the digital signals to transform coefficients for each block for each of a plurality frames in the sequence of frames,

forming variable predicted transform coefficients for each block of a first frame using variable prediction factors determined for each block,

comparing said predicted transform coefficients with corresponding transform coefficients of a corresponding block of a second frame to provide transform coefficient difference signals for each of the coefficients in each of the blocks of said second frame,

processing said transform difference signals for each block to form processed difference signals having a controlled range of values,

coding said processed difference signals to form statistically coded signals such that the more frequently occurring values of coded signals are represented by shorter code lengths and the less frequently occurring values of coded signals are represented by longer code lengths,

storing said coded signals in a buffer memory.

6. The method of claim 5 wherein the step of forming predicted transform coefficients includes, for each block,

comparing data elements in a block of said first frame with data elements of a corresponding block in said second frame to form a data comparison signal,

weighting said comparison signal to form a weighted comparison signal,

accumulating said weighted comparison signal for each of the data elements in the block to form an accumulated count for the block,

selecting a set of said prediction factors for the block as a function of said accumulated count.

7. The method of claim 6 wherein said data elements are transform coefficients.

8. The method of claim 6 wherein said data elements are said digital signals.

9. The method of claim 5 wherein the step of forming predicted transform coefficients includes, for each block,

comparing each transform coefficient in a block of said first frame with a corresponding transform coefficient of a corresponding block in said second frame to form a coefficient comparison signal,

weighting said comparison signal to form a weighted comparison signal,

accumulating said weighted comparison signal for each of the coefficients in the block to form an accumulated count for the block,

selecting a set of said prediction factors for the block as a function of said accumulated count.

10. The method of claim 9 wherein said set of prediction factors is unity whenever the accumulated count is below a first accumulated count value and wherein said set of prediction factors is zero whenever said accumulated count exceeds a second accumulated count value.

11. The method of claim 10 wherein said set of prediction factors is bounded by unity and zero and wherein said factors vary for each transform coefficient difference in the block whenever said accumulated count is between said first and second accumulated count values.

12. The method of claim 11 wherein said factors vary non-linearly such that the factor is greater for coefficient differences corresponding to low frequencies and lower for coefficient differences corresponding to high frequencies.

13. The method of claim 9 wherein said set of prediction factors is unity whenever the accumulated count is below a first accumulated count value.

14. The method of claim 9 including loading a replenishment signal into said buffer memory in place of said coded signals whenever said accumulated count is below a first accumulated count value to signify that the coefficients for the block of the second frame are to be interpreted as the same as the coefficients for the block of the first frame.

15. The method of claim 5 wherein the step of storing said coded signals in a buffer memory increases the fullness of said buffer memory and further including the steps of,

unloading said coded signals from said buffer memory for transmission thereby reducing the fullness of said buffer memory, and

adaptively controlling the processing of said transform difference signals as a function of the fullness of said buffer memory to control the range of values of said processed difference signals whereby the amount of data representing the images in said coded signals is adaptively controlled.

16. The method of claim 15 wherein said processing step includes a normalization step whereby each of said difference signals is multiplied by a normalization factor 1/N_f to form a normalized processed difference signal and wherein said step of adaptively controlling varies N_f as a function of the fullness of said buffer.

17. The method of claim 16 wherein said processing step includes the step of reducing any normalized processed difference signal having a value less than a threshold, T_c, to zero value to form said processed difference signals and wherein said coding step includes a run-length coding step representing sequential ones of said processed difference signals with zero value in run-length code.

18. The method of claim 17 wherein said coding step includes a statistical coding step for coding said run-length code and said processed difference signals.

19. The method of claim 16 wherein the step of forming predicted transform coefficients includes, for each block, the steps of,

comparing each transform coefficient in a block of said first frame with a transform coefficient of a corresponding block in said second frame to form a coefficient difference signal,

accumulating said coefficient difference signals for each of the coefficients in the block to form an accumulated count for the block,

selecting a prediction factor as a function of said accumulated count,

multiplying said processed difference signals by the factor N_f to form a reconstituted difference signal and adding said reconstituted difference signal to said predicted transform coefficients to form reconstituted transform coefficients and multiplying said reconstituted transform coefficients by a prediction factor within said set to form a predicted transform coefficient for a subsequent frame.

20. The method of claim 16 wherein the step of forming predicted transform coefficients includes, for each block, the steps of,

comparing each transform coefficient in a block of said first frame with a transform coefficient of a corresponding block in said second frame to form a coefficient difference signal,

accumulating said coefficient difference signals for each of the coefficients in the block to form an accumulated count for the block,

selecting a prediction factor as a function of said accumulated count,

multiplying said processed difference signals by the factor N_f and adding T_c to non-zero values of said processed difference to form reconstituted difference signal and adding said reconstituted difference signal to said predicted transform coefficients,

selecting a prediction factor as a function of said accumulated count,

modifying representations of the transform coefficients for the block as a function of said prediction factor to form said predicted representations.

21. A method for processing image signals to reduce the amount of data utilized to represent images where the images are defined by a sequence of frames where each frame includes a two-dimensional array of blocks where each block includes a two-dimensional array of digital signals representing the portion of the image within the block, the steps comprising,

transforming the digital signals to transform coefficients for each block for each of a plurality frames in the sequence of frames,

forming predicted transform coefficients for each block of a first frame,

comparing said predicted transform coefficients with corresponding transform coefficients of a corresponding block of a second frame to provide transform coefficient difference signals for each of the coefficients in each of the blocks of said second frame,

processing said transform difference signals for each block to form processed difference signals having a controlled range of values, determined by a feedback signal said processing including selecting a subset of said difference signals for each block with a selection threshold controlled by a threshold signal,

coding said processed difference signals to form statistically coded signals such that the more frequently occurring values of coded signals are represented by shorter code lengths and the less frequently occurring values of coded signals are represented by longer code lengths,

storing said coded signals in a buffer memory, thereby increasing the fullness of said buffer memory up to the currently filled capacity which is equal to or less than the maximum capacity of said buffer memory,

unloading said coded signals from said buffer memory, thereby reducing the currently filled capacity of said memory,

adaptively determining said feedback signal as a function of the difference between the currently filled capacity and the maximum capacity of said buffer memory.

22. An apparatus for processing image signals to reduce the amount of data utilized to represent images where the images are defined by a sequence of frames where each frame includes a two-dimensional array of digital signals representing the image, said apparatus comprising,

means for transforming the digital signals to form transform coefficients for each of a plurality of frames in the sequence of frames,

means for forming predicted transform coefficients for a first frame using variable prediction factors,

means for comparing said predicted transform coefficients with corresponding transform coefficients of a second frame to provide transform coefficient difference signals for each of the coefficients of said second frame,

means for processing said transform difference signals to form processed difference signals having a controlled range of values,

means for coding said processed difference signals to form statistically coded signals such that the more frequently occurring values of coded signals are represented by shorter code lengths and the less frequently occurring values of coded signals are represented by longer code lengths.

23. The apparatus of claim 22 including means for determining said variable prediction factors as a function of frame-to-frame differences in said images.

24. The apparatus of claim 23 including means for determining said variable prediction factors in response to frame-to-frame comparisons of said transform coefficients.

25. The apparatus of claim 23 including means for determining said variable prediction factors as a function of frame-to-frame comparisons of said digital signals.

26. A apparatus for processing image signals to reduce the amount of data utilized to represent images where the images are defined by a sequence of frames where each frame includes a two-dimensional array of blocks where each block includes a two-dimensional array of digital signals representing the portion of an image within the block, said apparatus comprising,

means for transforming the digital signals to transform coefficients for each block for each of a plurality frames in the sequence of frames,

means for forming variable predicted transform coefficients for each block of a first frame using variable prediction factors determined for each block,

means for comparing said predicted transform coefficients with corresponding transform coefficients of a corresponding block of a second frame to provide transform coefficient difference signals for each of the coefficients in each of the blocks of said second frame,

means for processing said transform difference signals for each block to form processed difference signals having a controlled range of values,

means for coding said processed difference signals to form statistically coded signals such that the more frequently occurring values of coded signals are represented by shorter code lengths and the less frequently occurring values of coded signals are represented by longer code lengths,

means for storing said coded signals in a buffer memory.

27. The apparatus of claim 26 wherein said means for forming predicted transform coefficients includes, for each block,

means for comparing data elements in a block of said first frame with data elements of a corresponding block in said second frame to form a data comparison signal,

means for weighting said comparison signal to form a weighted comparison signal,

means for accumulating said weighted comparison signal for each of the data elements in the block to form an accumulated count for the block,

means for selecting a set of said prediction factors for the block as a function of said accumulated count.

28. The apparatus of claim 27 wherein said data elements are transform coefficients.

29. The apparatus of claim 27 wherein said data elements are said digital signals.

30. The apparatus of claim 26 wherein said means for forming predicted transform coefficients includes,

means for comparing each transform coefficient in a block of said first frame with a corresponding transform coefficient of a corresponding block in said second frame to form a coefficient comparison signal,

means for weighting said comparison signal to form a weighted comparison signal,

means for accumulating said weighted comparison signal for each of the coefficients in the block to form an accumulated count for the block,

means for selecting a set of said prediction factors for the block as a function of said accumulated count.

31. The apparatus of claim 30 wherein said set of prediction factors is unity whenever the accumulated count is below a first accumulated count value and wherein said set of prediction factors is zero whenever said accumulated count exceeds a second accumulated count value.

32. The apparatus of claim 31 wherein said set of prediction factors is bounded by unity and zero and wherein said factors vary for each transform coefficient difference in the block whenever said accumulated count is between said first and second accumulated count values.

33. The apparatus of claim 32 wherein said factors vary non-linearly such that the factor is greater for coefficient differences corresponding to low frequencies and lower for coefficient differences corresponding to high frequencies.

34. The apparatus of claim 30 wherein said set of prediction factors is unity whenever the accumulated count is below a first accumulated count value.

35. The apparatus of claim 30 means for including loading a replenishment signal into said buffer memory in place of said coded signals whenever said accumulated count is below a first accumulated count value to signify that the coefficients for the block of the second frame are to be interpreted as the same as the coefficients for the block of the first frame.

36. The apparatus of claim 26 wherein said means for storing said coded signals in a buffer memory increases the fullness of said buffer memory and further includes,

means for unloading said coded signals from said buffer memory for transmission thereby reducing the fullness of said buffer memory, and

means for adaptively controlling the processing of said transform difference signals as a function of the fullness of said buffer memory to control the range of values of said processed difference signals whereby the amount of data representing the images in said coded signals is adaptively controlled.

37. The apparatus of claim 36 wherein said means for processing includes normalization means whereby each of said difference signals is multiplied by a normalization factor 1/N_f to form a normalized processed difference signal and wherein said means for adaptively controlling varies N_f as a function of the fullness of said buffer.

38. The apparatus of claim 37 wherein said means for processing includes means for reducing any normalized processed difference signal having a value less than a threshold, T_c, to zero value to form said processed difference signals and wherein said means for coding includes a run-length coder representing sequential ones of said processed difference signals with zero value in run-length code.

39. The apparatus of claim 38 wherein said means for coding includes a statistical coder for coding said run-length code and said processed difference signals.