Fast signal transforms with lifting steps

• US RE40,081 E1
• Filed: 07/29/2003
• Issued: 02/19/2008
• Est. Priority Date: 12/16/1998
• Status: Expired due to Term

• Current Assignee: RPX Corporation
• Original Assignee: Fast VDO LLC
• Inventors: Topiwala, Pankaj, Tran, Trac D.
• Primary Examiner(s): Couso; Jose L.
• Application Number:
  US10/629,303
• Time in Patent Office: 1,666 Days
• Field of Search: 382/232, 382/233, 382/236, 382/238, 382/239, 382/240, 382/242, 382/248, 382/250, 358/426.13, 358/426.14, 348/384.1, 348/394.1, 348/395.1, 3484001-4041, 3484071-4161, 3484201-4211, 348/425.1, 348/430.1, 348/431.1, 348/699, 341/51, 341/63, 341/65, 341/67, 341/107, 364/724.011, 364/724.04, 364/724.05, 364/724.13, 364/724.14, 364/725.01, 364/725.02, 708/400, 375/240.11, 375/240.16
• US Class Current: 382/232
• CPC Class Codes: G06F 17/147 Discrete orthonormal transforms, e.g. discrete cosine transform, discrete sine transform, and variations therefrom, e.g. modified discrete cosine transform, integer transforms approximating the discrete cosine transform G06F17/145 takes precedence
• CPC Class Codes: H04N 19/114 Adapting the group of pictures [GOP] structure, e.g. number of B-frames between two anchor frames H04N19/107 takes precedence
• CPC Class Codes: H04N 19/42 characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation H04N19/635 takes precedence
• CPC Class Codes: H04N 19/63 using sub-band based transform, e.g. wavelets
• CPC Class Codes: H04N 19/635 characterised by filter definition or implementation details

Abstract

This invention introduces a class of multi-band linear phase lapped biorthogonal transforms with fast, VLSI-friendly implementations via lifting steps called the LiftLT. The transform is based on a lattice structure which robustly enforces both linear phase and perfect reconstruction properties. The lattice coefficients are parameterized as a series of lifting steps, providing fast, efficient in-place computation of the transform coefficients as well as the ability to map integers to integers. Our main motivation of the new transform is its application in image and video coding. Comparing to the popular 8×8 DCT, the 8×16 LiftLT only requires 1 more multiplication, 22 more additions, and 6 more shifting operations. However, image coding examples show that the LiftLT is far superior to the DCT in both objective and subjective coding performance. Thanks to properly designed overlapping basis functions, the LiftLT can completely eliminate annoying blocking artifacts. In fact, the novel LiftLT'"'"'s coding performance consistently surpasses that of the much more complex 9/7-tap biorthogonal wavelet with floating-point coefficients. More importantly, our transform'"'"'s block-based nature facilitates one-pass sequential block coding, region-of-interest coding/decoding as well as parallel processing.

Claims

1. An apparatus for coding, storing or transmitting, and decoding M×

M sized blocks of digitally represented images, where M is an even number, comprising

a. a forward transform comprising

i. a base transform having M channels numbered 0 through M−

1, half of said channel numbers being odd and half being even;

ii. an equal normalization factor in each of the M channels selected to be dyadic-rational;

iii. a full-scale butterfly implemented as a series of lifting steps with a first set of dyadic rational coefficients;

iv. M/2 delay lines in the odd numbered channels;

v. a full-scale butterfly implemented as a series of lifting steps with said first set of dyadic rational coefficients; and

vi. a series of lifting steps in the odd numbered channels with a second specifically selected set of dyadic-rational coefficients;

b. means for transmission or storage of the transform output coefficients; and

c. an inverse transform comprising

i. M channels numbered 0 through M−

1, half of said channel numbers being odd and half being even;

ii. a series of inverse lifting steps in the odd numbered channels with said second set of specifically selected dyadic-rational coefficients;

iii. a full-scale butterfly implemented as a series of lifting steps with said first set of specifically selected dyadic-rational coefficients;

iv. M/2 delay lines in the even numbered channels;

v. a full-scale butterfly implemented as a series of lifting steps with said first set of specifically selected dyadic-rational coefficients;

vi. an equal denormalization factor in each of the M channels specifically selected to be dyadic-rational; and

vii. a base inverse transform having M channels numbered 0 through M−

1.

2. The apparatus of claim 1 in which the normalizing factor takes the value 25/16 and simultaneously the denormalizing factor takes the value 16/25.

3. The apparatus of claim 1 in which the normalizing factor takes the value 5/4 and simultaneously the denormalizing factor takes the value 4/5.

4. The apparatus of claim 1 in which the first set of dyadic rational coefficients are all equal to 1.

5. The apparatus of claim 1 in which the second set of dyadic rational coefficients are all equal to ½

.

6. The apparatus of claim 1 in which the base transform is any M×

M invertible matrix of the form of a linear phase filter and the inverse base transform is the inverse of said M×

M invertible matrix.

7. The apparatus of claim 1 in which the base transform is the forward M×

M discrete cosine transform and the inverse base transform is the inverse M×

M discrete cosine transform.

11. The apparatus of claim 1 in which the constants coefficients are approximations chosen for rapid computing rather than exact constants coefficients.

8. An apparatus for coding, compressing, storing or transmitting, and decoding a block of M×

M intensities from a digital image selected by an M×

M window moving recursively over the image, comprising;

a. an M×

M block transform comprising;

i. an initial stage

ii. a normalizing factor in each channel

b. a cascade comprising a plurality of dyadic rational lifting transforms, each of said plurality of dyadic rational lifting transforms comprising

i. a first bank of pairs of butterfly lifting steps with unitary coefficients between adjacent lines of said transform;

ii. a bank of delay lines in a first group of M/2 alternating lines;

iii. a second bank of butterfly lifting steps with unitary coefficients, and

iv. a bank of pairs of butterfly lifting steps with coefficients of 1/2 between M/2−

1 pairs of said M/2 alternating lines;

c. means for transmission or storage of the output coefficients of said M×

M block transform; and

d. an inverse transform comprising

i. a cascade comprising a plurality of dyadic rational lifting transforms, each of said plurality of dyadic rational lifting transforms comprising

a) a bank of pairs of butterfly lifting steps with coefficients of 1/2 between said M/2−

1 pairs of said M/2 alternating lines;

b) a first bank of pairs of butterfly lifting steps with unitary coefficients between adjacent lines of said transform;

c) a bank of delay lines in a second group of M/2 alternating lines, ;

and

d) a second bank of pairs of butterfly lifting steps with unitary coefficients between adjacent lines of said transform;

ii. a de-scaling bank, ;

and

iii. an inverse initial stage.

9. A method of coding, storing or transmitting, and decoding M×

M sized blocks of digitally represented images, where M is an even number a power of 2, comprising

a. transmitting the original picture signals to a coder, which effects the steps of

i. converting the signals with a base transform having M channels numbered 0 through M−

1, half of said channel numbers being odd and half being even, ;

ii. normalizing the output of the preceding step with a dyadic rational normalization factor in each of said M channels;

iii. processing the output of the preceding step through two lifting steps with a first set of identical dyadic rational coefficients connecting each pair of adjacent numbered channels in a butterfly configuration;

iv. transmitting the resulting coefficients through M/2 delay lines in the odd numbered channels;

v. processing the output of the preceding step through two inverse lifting steps with the first set of dyadic rational coefficients connecting each pair of adjacent numbered channels in a butterfly configuration; and

vi. applying two lifting steps with a second set of identical dyadic rational coefficients connecting each pair of adjacent odd numbered channels to the output of the preceding step;

b. transmitting or storing the transform output coefficients;

c. receiving the transform output coefficients in a decoder; and

d. processing the output coefficients in a decoder, comprising the steps of

i. receiving the coefficients in M channels numbered 0 through M−

1, half of said channel numbers being odd and half being even;

ii. applying two inverse lifting steps with dyadic rational coefficients connecting each pair of adjacent odd numbered channels;

iii. applying two lifting steps with dyadic rational coefficients connecting each pair of adjacent numbered channels in a butterfly configuration;

iv. transmitting the result of the preceding step through M/2 delay lines in the even numbered channels;

v. applying two inverse lifting steps with dyadic rational coefficients connecting each pair of adjacent numbered channels in a butterfly configuration;

vi. denormalizing the result of the preceding step with a dyadic rational inverse normalization factor in each of said M channels; and

vii. processing the result of the preceding step through a base inverse transform having M channels numbered 0 through M−

1.

10. A method of coding, compressing, storing or transmitting, and decoding a block of M×

M intensities from a digital image selected by an M×

M window moving recursively over the image, comprising the steps of;

a. Processing the intensities in an M×

M block coder comprising the steps of;

i. processing the intensities through an initial stage;

ii. scaling the result of the preceding step in each channel;

b. processing the result of the preceding step through a cascade comprising a plurality of dyadic rational lifting transforms, each of said plurality of dyadic rational lifting transforms comprising

i. a first bank of pairs of butterfly lifting steps with unitary coefficients between adjacent lines of said transform;

ii. a bank of delay lines in a first group of M/2 alternating lines;

iii. a second bank of butterfly lifting steps with unitary coefficients, and

iv. a bank of pairs of butterfly lifting steps with coefficients of 1/2 between M/2−

1 pairs of said M/2 alternating lines;

c. transmitting or storing the output coefficients of said M×

M block coder;

d. receiving the output coefficients in a decoder; and

e. processing the output coefficients in the decoder, comprising the steps of

i. processing the output coefficients through a cascade comprising a plurality of dyadic rational lifting transforms, each of said plurality of dyadic rational lifting transforms comprising

a) a bank of pairs of butterfly lifting steps with coefficients of 1/2 between said M/2−

1 pairs of said M/2 alternating lines;

b) a first bank of pairs of butterfly lifting steps with unitary coefficients between adjacent lines of said transform;

c) a bank of delay lines in a second group of M/2 alternating lines;

d) a second bank of pairs of butterfly lifting steps with unitary coefficients between adjacent lines of said transform;

e) a de-scaling bank; and

f. processing the results of the preceding step in an inverse initial stage.

12. A method of coding, storing or transmitting, and decoding a block of M×

M intensities from a digital image selected by an M×

M window moving recursively over the image, comprising;

a. processing the intensities in an M×

M block coder comprising the steps of;

i. processing the intensities through an initial stage;

ii. scaling the result of the preceding step in each channel;

b. processing the result of the preceding step through a transform coder using a method of processing blocks of samples of digital signals of integer length M comprising processing the digital samples of length M with an invertible linear transform of dimension M, said transform being representable as a cascade, using the steps, in arbitrary order, of;

i) at least one +/−

1 butterfly step,

ii) at least one lifting step with rational complex coefficients, and

iii) at least one scaling factor;

c. transmitting or storing the output coefficients of said M×

M block coder;

d. receiving the output coefficients in a decoder; and

e. processing the output coefficients in the decoder into a reconstructed image using the inverse of the coder of steps a. and b.

13. The method of claim 12 wherein the method of processing blocks of samples of digital signals of integer length M additionally comprises the step of at least one time delay.

14. The method of claim 12, wherein the rational complex coefficients in the at least one lifting step are dyadic.

15. The method of claim 12, wherein

a) said invertible transform is an approximation of a biorthogonal transform;

b) said biorthogonal transformation comprises a representation as a cascade of at least one butterfly step, at least one orthogonal transform, and at least one scaling factor;

c) said at least one orthogonal transform comprises a cascade of

i) at least one ±

1 butterfly step,

ii) at least one planar rotation, and

iii) at least one scaling factor;

b) said at least one planar rotation being represented by equivalent lifting steps and scale factors; and

,

c) said approximation is obtained by replacing floating point coefficients in the lifting steps with rational coefficients.

16. The method of claim 15, wherein the coefficients of the lifting steps are chosen to be dyadic rational.

17. The method of claim 12, wherein the invertible transform is a unitary transform.

18. The method of claim 12, wherein

a) said invertible transform is an approximation of a unitary transform;

b) said approximation of the unitary transform comprises a representation of the unitary transform as a cascade of at least one butterfly step, at least one orthogonal transform, and at least one scale factor;

c) said at least one orthogonal transform being represented as a cascade of

(1) at least one ±

1 butterfly steps,

(2) at least one planar rotation, and

(3) at least one scaling factor;

d) said at least one planar rotation being represented by equivalent lifting steps and scale factors; and

,

e) said approximation being derived by using approximate rational values for the coefficients in the lifting steps.

19. The method of claim 18, wherein the invertible transform is an approximation of a transform selected from the group of special unitary transforms:

discrete cosine transform (DCT);

discrete Fourier transform (DFT);

discrete sine transform (DST).

20. The method of claim 18, wherein the coefficients of the lifting steps are dyadic rational.

21. The method of claim 18, wherein at least one of the following lifting steps is used, whose matrix representations take on the form:

$\left[\begin{array}{cc}1& a\\ 0& 1\end{array}\right],\left[\begin{array}{cc}1& 0\\ b& 1\end{array}\right],$

22. The method of claim 21, wherein the invertible transform is an approximation of a transform selected from the group:

discrete cosine transform (DCT);

discrete Fourier transform (DFT);

discrete sine transform (DST).

23. The method of claim 22, wherein the approximation of the 4 point DCT is selected from the group of matrices:

$\left\{\left[\begin{array}{cccc}1& 1& 1& 1\\ 2& 1& -1& -2\\ 1& -1& -1& 1\\ 1& -1& 1& -1\end{array}\right],\left[\begin{array}{cccc}1& 1& 1& 1\\ 2& 1& -1& -2\\ 1& -1& -1& 1\\ 1& -2& 2& -1\end{array}\right],\left[\begin{array}{cccc}1& 1& 1& 1\\ 2& 1& -1& -2\\ 1& -1& -1& 1\\ 5& -2& 2& -5\end{array}\right]\right\}.$

24. The method of claim 19 in which the invertible transform is an approximation of a transform selected from the group three point DCT, 4 point DCT, 8 point DCT, and 16 point DCT.

25. The method of claim 19 in which the invertible transform is an approximation of a transform selected from the group 512 point FFT, 1024 point FFT, 2048 point FFT, and 4096 point FFT.

26. A method of coding, storing or transmitting, and decoding sequences of intensities of integer length M recursively selected from a time ordered string of intensities arising from electrical signals, the method comprising the steps of

a) recursively processing the sequences of intensities of integer length M with an invertible forward linear transform of dimension M, said transform being representable as a cascade using the steps, in a preselected arbitrary order, of:

ii) at least one ±

1 butterfly step,

iii) at least one lifting step with rational complex coefficients, and

iv) applying at least one scaling factor;

b) compressing the resulting transform coefficients;

c) storing or transmitting the compressed transform coefficients;

d) receiving or recovering from storage the transmitted or stored compressed transform coefficients;

e) decompressing the received or recovered compressed transform coefficients; and

f) recursively processing the decompressed transform coefficients with the inverse of the forward linear transform of dimension M, said inverse transform being representable as a cascade using the steps, in the exact reverse order of the preselected arbitrary order, of;

ii) at least one inverse butterfly corresponding to each of the at least one ±

1 butterfly step;

iii) at least one inverse lifting step corresponding to each of the at least one lifting step with rational complex coefficients; and

,

iv) applying at least on inverse scaling factor corresponding to the at least one scaling factor.

27. The method of claim 26 wherein the method of processing blocks of samples of digital signals of integer length M additionally comprises the step of at least one time delay.

28. The method of claim 26, wherein the rational complex coefficients in the at least one lifting step are dyadic.

29. The method of claim 26, wherein

a) said invertible transform is an approximation of a biorthogonal transform;

b) said biorthogonal transformation comprises a representation as a cascade of at least one butterfly step, at least one orthogonal transform, and at least one scaling factor;

c) said at least one orthogonal transform comprising a cascade of

i) at least one ±

1 butterfly step,

ii) at least one planar rotation, and

iii) at least one scaling factor;

b) said at least one planar rotation being represented by equivalent lifting steps and scale factors; and

,

c) said approximation being obtained by replacing floating point coefficients in the lifting steps with rational coefficients.

30. The method of claim 29, wherein the coefficients of the lifting steps are chosen to be dyadic rational.

31. The method of claim 26, wherein the invertible transform is a unitary transform.

32. The method of claim 26, wherein

a) said invertible transform is an approximation of a unitary transform;

b) said approximation of the unitary transform comprises a representation of the unitary transform as a cascade of at least one butterfly step, at least one orthogonal transform, and at least one scale factor;

c) said at least one orthogonal transform being represented as a cascade of

(1) at least one ±

1 butterfly steps,

(2) at least one planar rotation, and

(3) at last one scaling factor;

d) said at least one planar rotation being represented by equivalent lifting steps and scale factors; and

,

e) said approximation being derived by using approximate rational values for the coefficients in the lifting steps.

33. The method of claim 32, wherein the invertible transform is an approximation of a transform selected from the group of special unitary transforms:

discrete cosine transform (DCT);

discrete Fourier transform (DFT);

discrete sine transform (DST).

34. The method of claim 32, wherein the coefficients of the lifting steps are dyadic rational.

35. The method of claim 32, wherein at least one of the following lifting steps is used, whose matrix representations take on the form:

$\left[\begin{array}{cc}1& a\\ 0& 1\end{array}\right],\left[\begin{array}{cc}1& 0\\ b& 1\end{array}\right],$

36. The method of claim 35, wherein the invertible transform is an approximation of a transform selected from the group:

discrete cosine transform (DCT);

discrete Fourier transform (DFT);

discrete sine transform (DST).

37. The method of claim 36, wherein the approximation of the 4 point DCT is selected from the group of matrices:

$\left\{\left[\begin{array}{cccc}1& 1& 1& 1\\ 2& 1& -1& -2\\ 1& -1& -1& 1\\ 1& -1& 1& -1\end{array}\right],\left[\begin{array}{cccc}1& 1& 1& 1\\ 2& 1& -1& -2\\ 1& -1& -1& 1\\ 1& -2& 2& -1\end{array}\right],\left[\begin{array}{cccc}1& 1& 1& 1\\ 2& 1& -1& -2\\ 1& -1& -1& 1\\ 5& -2& 2& -5\end{array}\right]\right\}.$

38. The method of claim 33 in which the invertible transform is an approximation of a transform selected from the group three point DCT, 4 point DCT, 8 point DCT, and 16 point DCT.

39. The method of claim 33 in which the invertible transform is an approximation of a transform selected from the group 512 point FFT, 1024 point FFT, 2048 point FFT, and 4096 point FFT.