Method and apparatus for reduction of image data compression noise

• US 5,629,778 A
• Filed: 05/15/1995
• Issued: 05/13/1997
• Est. Priority Date: 05/15/1995
• Status: Expired due to Term

• Current Assignee: Intellectual Ventures I LLC (Intellectual Ventures LLC)
• Original Assignee: Polaroid Corporation (PLR IP Holdings LLC)
• Inventors: Reuman, Steven R.
• Primary Examiner(s): Brinich, Stephen
• Application Number:
  US08/441,372
• Time in Patent Office: 729 Days
• Field of Search: 358/426-427, 358/261.1-261.3, 358/430-433, 382/232-233
• US Class Current: 382/252
• CPC Class Codes: H04N 19/527 Global motion vector estimation
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness

Abstract

An image encoding/decoding apparatus for performing transform coding by a method in which blocking artifacts are suppressed or eliminated is disclosed in which encoded data, transmitted by the encoding apparatus, is converted into received image data terms which are subsequently overlap transformed into frequency coefficients for modification by means of a filtering operation utilizing a quantization error matrix. The quantization error matrix can be derived from quantization error data generated in the encoding unit, or can be provided as a look-up table in the decoding unit. The modified frequency coefficients are converted into reduced-noise image data terms for reconstruction into a digital image.

Claims

1. A method for reducing the effects of blocking artifacts attributable to quantization incurred in the transform image coding and compression of a two-dimensional image signal into a digital image, the degree of compression determined by a scaling factor κ

and a quantization table, and the image signal provided as a series of electrical signals, each electrical signal corresponding to a characteristic of an element of a two-dimensional image, where the image elements have been configured as a two-dimensional H×

V array, said method comprising the steps of;

converting the series of electrical signals into a set of numerical values, each said numerical value quantitatively describing a feature characteristic of a corresponding image element, said numerical values further denoted by s₀ (z,y) where 0≦

y≦

H-1 and 0≦

z≦

V-1;

formatting said set of numerical values into a plurality of N×

N image data matrices identified by indices `p` and `q`, each said image-data matrix comprising image-data terms denoted by s_q,p (j,i), each said image data term comprised of a said numerical value determined in accordance with the relationship;

space="preserve" listing-type="equation">s.sub.q,p (j,i)=s.sub.0 (j+Nq,i+Np)

where 0≦

i,j≦

N-1;

transforming each said image-data matrix into an N×

N frequency-coefficient matrix comprising frequency-coefficient terms denoted by S_q,p (υ

,μ

), said step of transforming each said image-data matrix accomplished by means of an orthogonal transform basis matrix C in accordance with the expression,

space="preserve" listing-type="equation">S.sub.q,p (υ

,μ

)=C×

s.sub.q,p (j,i)×

C.sup.T ;

dividing each said frequency-coefficient term S_q,p (υ

,μ

) by a scaled quantization term denoted by Q(υ

,μ

), said scaled quantization term obtained from the quantization table and modified by the scaling factor, to yield a plurality of N×

N quotient-coefficient matrices comprising quotient terms denoted by Qu _q,p (υ

,μ

) derived in accordance with the expression, ##EQU16## rounding each said quotient term to a less precise value to yield a quantized-coefficient matrix comprising quantized-quotient terms denoted by Qc_q,p (υ

,μ

) in accordance with the expression, ##EQU17## subtracting each said quantized-coefficient term Qc_q,p (υ

,μ

) from a corresponding said quotient term Qu_q,p (υ

,μ

) to form a plurality of N×

N difference-coefficient matrices comprising difference terms Dc_q,p (υ

,μ

) derived in accordance with the expression, ##EQU18## deriving a quantization error matrix, comprising terms E₀ (υ

,μ

), by summing and averaging error terms, said error terms comprising functions of said difference-coefficient matrices;

selecting a set of filter parameters, denoted by α and

β

;

multiplying each said quantized coefficient term by a corresponding quantization term to yield a plurality of N×

N mask-multiplied transform coefficient matrices comprising terms denoted by R_q,p (υ

,μ

) derived in accordance with the expression,

space="preserve" listing-type="equation">R.sub.q,p (υ

,μ

)=Qc.sub.q,p (υ

,μ

)×

Q(υ

,μ

);

applying an inverse orthogonal transform to said mask-multiplied transform coefficient matrices to yield a plurality of N×

N received image-data matrices denoted by r_q,p (j,i), in accordance with the inverse transform equation,

space="preserve" listing-type="equation">r.sub.q,p (j,i)=C.sup.T ×

R.sub.q,p (υ

,μ

)×

C;

formatting said received image-data matrices into an H×

V received image-data set comprising terms denoted by s_R (z,y), in accordance with the equation,

space="preserve" listing-type="equation">s.sub.R (j+Nq,i+Np)=r.sub.q,p (j,i)

where 0≦

i+Np≦

H-1 and 0≦

j+Nq≦

V-1;

forming said received image data set into a plurality of N×

N overlapped image-data matrices comprising terms denoted by v_s,r (j,i), in accordance with the equation,

space="preserve" listing-type="equation">v.sub.s,r (j,i)=s.sub.R (j+ω

s, i+ω

r);

transforming said overlapped image-data matrices into modified coefficient matrices comprising terms denoted by Sv_s,r (υ

,μ

), in accordance with the equation,

space="preserve" listing-type="equation">Sv.sub.s,r (υ

,μ

)=C×

v.sub.s,r (j,i)×

C.sup.T ;

converting said modified coefficient matrices into filtered coefficient matrices by means of said quantization error matrix and said filter parameters, said modified coefficient matrix comprising terms denoted by Sf_s,r (υ

,μ

), said step of converting said modified coefficient matrices performed in accordance with the equation, ##EQU19## transforming said filtered coefficient matrices into filtered image-data matrices in accordance with the transform equation,

space="preserve" listing-type="equation">rf.sub.s,r (j,i)=C.sup.T ×

Sf.sub.s,r (υ

,μ

)×

C; and

converting said filtered image-data matrices into a series of filtered electrical signals such that said filtered electrical signals can be configured into a two-dimensional H×

V array of filtered image elements, each said filtered electrical signal corresponding to a characteristic of one said filtered image element.

2. The method of claim 1 wherein said quantization error matrix E₀ (υ

,μ

) is determined in accordance with the relationship ##EQU20##

3. The method of claim 1 wherein said quantization error matrix E₀ (υ

,μ

) is determined in accordance with the relationship ##EQU21##

4. The method of claim 1 wherein said orthogonal transform basis matrix C comprises a discrete cosine transform basis matrix.

5. The method of claim 1 wherein said step of converting said filtered image-data matrices comprises the steps of extracting a core submatrix from each said filtered image-data matrix and combining a plurality of said core submatrices to form said two-dimensional H×

V array of filtered image elements.

6. The method of claim 5 wherein said step of extracting a core submatrix comprises deleting image-data terms in rows 0 through ##EQU22## and ##EQU23## through (N-1) of said filtered image-data matrix and deleting filtered image-data terms in columns 0 through ##EQU24## and ##EQU25## through (N-1) of said filtered image-data matrix.

7. The method of claim 1 wherein ω

is set to ##EQU26##

8. The method of claim 1 wherein N is equal to 8.

9. A method for reducing the effects of blocking artifacts attributable to quantization incurred in the transform image coding and compression of a two-dimensional image signal into a digital image, the degree of compression determined by a scaling factor κ

and a quantization table, and the image signal provided as a series of electrical signals, each electrical signal corresponding to a characteristic of an element of a two-dimensional image, where the image elements have been configured as a two-dimensional H×

V array, said method comprising the steps of;

transform image coding the two-dimensional H×

V array of image elements to form a coded image-data set by means of an orthogonal transform basis matrix, said step of transform image coding comprising a quantization operation;

decoding said coded image-data set by means of said orthogonal transform basis matrix to form a decoded image-data set;

converting said decoded image-data set into a plurality of frequency coefficient terms by means of said orthogonal transform basis matrix;

filtering said plurality of frequency coefficient terms by means of a filtering computation, said filtering computation comprising the step of multiplying said frequency coefficient terms with a coefficient filtering term to form a plurality of filtered frequency coefficient terms; and

converting said filtered frequency coefficients into a reduced-noise digital image.

10. The method of claim 9 wherein said step of converting said decoded image-data set comprises the steps of forming said decoded image-data set into an array of overlapped image-data matrices and converting said array of overlapped image-data matrices into an array of frequency coefficient matrices by means of said orthogonal transform basis matrix, said frequency coefficient matrices comprising said frequency coefficient terms.

11. The method of claim 9 further comprising the step of deriving said coefficient filtering term.

12. The method of claim 11 wherein said step of deriving said coefficient filtering term comprises the step of deriving at least one filter parameter.

13. The method of claim 12 wherein said filter parameter is a function of said scaling factor κ

.

14. The method of claim 11 wherein said step of deriving said coefficient filtering term comprises the step of deriving a quantization error matrix.

15. The method of claim 9 wherein said orthogonal transform basis matrix comprises a discrete cosine transform basis matrix.

16. The method of claim 9 wherein said step of converting said filtered frequency coefficients comprises the steps of transforming said filtered frequency coefficients into filtered image-data terms by means of said orthogonal transform basis matrix and converting said filtered image-data terms into a series of filtered electrical signals such that said filtered electrical signals can be configured into a two-dimensional H×

V array of filtered image elements to form said reduced-noise digital image, each said filtered electrical signal corresponding to a characteristic of one said filtered image element.

17. The method of claim 16 wherein said step of converting said filtered image-data terms comprises the steps of extracting a portion of said filtered image-data terms and combining said portion to form a two-dimensional H×

V reduced-noise image-data set from which said reduced-noise digital image is formed.

18. A method for filtering a two-dimensional decoded image signal such that the effects of blocking artifacts attributable to quantization are reduced, the blocking artifacts arising in the decoded image signal as a consequence of there having been performed transform image coding, data compression, and quantization on an antecedent image signal, the antecedent image signal having been provided as a series of electrical signals, each electrical signal corresponding to a characteristic of an element of the antecedent image, where the compression operation utilized a scaling factor κ

and quantization terms derived from a quantization table, and where the decoded image signal has been configured as a two-dimensional H×

V set of image-data terms denoted by S_R (z,y), said method comprising the steps of;

defining an overlap parameter, denoted by ω

;

forming the decoded image signal set into a plurality of N×

N overlapped image-data matrices comprising terms denoted by v_s,r (j,i), in accordance with the relationship,

space="preserve" listing-type="equation">v.sub.s,r (j,i)=s.sub.R (j+ω

s, i+ω

r);

transforming said overlapped image-data matrices into modified coefficient matrices comprising terms denoted by Sv_s,r (υ

,μ

), by means of an orthogonal transform basis matrix C in accordance with the matrix equation,

space="preserve" listing-type="equation">Sv.sub.s,r (υ

,μ

)=C×

v.sub.s,r (j,i)×

C.sup.T ;

estimating the scaling factor κ

;

determining filter parameters as a function of said scaling factor, said filter parameters denoted by α

(κ

) and η

(κ

);

selecting a quantization error matrix, denoted by E₀ ;

converting said modified coefficient matrices into filtered coefficient matrices by means of said quantization error matrix and said filter parameters, said modified coefficient matrix comprising terms denoted by Sf_s,r (υ

,μ

), said step of converting said modified coefficient matrices performed in accordance with the equation, ##EQU27## transforming said filtered coefficient matrices into filtered image-data matrices, denoted by rf_s,r (j,i), in accordance with the matrix equation,

space="preserve" listing-type="equation">rf.sub.s,r (j,i)=C.sup.T ×

Sf.sub.s,r (υ

,μ

)×

C; and

converting said filtered image-data matrices into a series of filtered electrical signals such that said filtered electrical signals can be configured into a two-dimensional H×

V array of filtered image elements to form a reduced-noise digital image, each said filtered electrical signal corresponding to a characteristic of one said filtered image element.

19. The method of claim 18 wherein said orthogonal transform basis matrix C comprises a discrete cosine transform basis matrix.

20. The method of claim 18 wherein said step of converting said filtered image-data matrices comprises the steps of extracting a core submatrix from each said filtered image-data matrix and combining a plurality of said core submatrices to form a two-dimensional H×

V reduced-noise image-data set from which said reduced-noise digital image is formed.

21. The method of claim 20 wherein said step of extracting a core submatrix comprises deleting image-data terms in rows 0 through ##EQU28## and ##EQU29## through (N-1) of said filtered image-data matrix and deleting filtered image-data terms in columns 0 through ##EQU30## and ##EQU31## through (N-1) of said filtered image-data matrix.

22. The method of claim 18 wherein ω

is set to ##EQU32##

23. The method of claim 18 wherein said step of estimating a scaling factor K comprises the step of setting K equal to the DC scaled quantization term of said quantization table.

24. The method of claim 18 wherein said step of determining filter parameters comprises the step of obtaining at least one of said filter parameters from a numerical table.

25. The method of claim 18 wherein κ

≦

8 and α

(κ

)=β

(κ

)=0.

26. The method of claim 18 wherein 8≦

κ

≦

24 and α

(κ

), α

(78 )≧

1.

27. The method of claim 18 wherein 24≦

κ

≦

32 and α

(κ

), β

(κ

)≧

2.

28. The method of claim 18 wherein κ

>

32 and α

(κ

), β

(κ

)≧

3.

29. The method of claim 18 wherein said step of selecting a quantization error matrix comprises the step of obtaining at least one quantization error term from a numerical table.

30. A method for filtering a two-dimensional decoded image signal such that the effects of blocking artifacts attributable to quantization are reduced, the blocking artifacts arising in the decoded image signal as a consequence of there having been performed transform image coding, data compression, and quantization on an antecedent image signal, the antecedent image signal having been provided as a series of electrical signals, each electrical signal corresponding to a characteristic of an element of the antecedent image, where the compression operation utilized a scaling factor and quantization terms obtained from a quantization table, and where the decoded image signal has been configured as a two-dimensional H×

V set of image-data terms denoted by s_R (z,y), said method comprising the steps of;

converting said set of image-data terms into a plurality of frequency coefficient terms by means of an orthogonal transform basis matrix;

filtering said plurality of frequency coefficient terms by means of a filtering computation, said filtering computation comprising the step of multiplying said frequency coefficient terms with a coefficient filtering term to form a plurality of filtered frequency coefficient terms; and

converting said filtered frequency coefficients into a reduced-noise digital image.

31. The method of claim 30 wherein said step of converting said set of image-data terms comprises the steps of forming said image-data terms into an array of overlapped image-data matrices and converting said array of overlapped image-data matrices into an array of coefficient matrices comprising frequency coefficient terms by means of said orthogonal transform basis matrix.

32. The method of claim 30 further comprising the step of deriving said coefficient filtering term.

33. The method of claim 32 wherein said step of deriving said coefficient filtering term comprises the step of deriving at least one filter parameter.

34. The method of claim 33 wherein said filter parameter is a function of the scaling factor.

35. The method of claim 32 wherein said step of deriving said coefficient filtering term comprises the step of deriving a quantization error matrix.

36. The method of claim 30 wherein said step of converting said filtered frequency coefficients comprises the steps of transforming said filtered frequency coefficients into filtered image-data terms by means of said orthogonal transform basis matrix and converting said filtered image-data terms into a series of filtered electrical signals such that said filtered electrical signals can be configured into a two-dimensional H×

V array of filtered image elements to form said reduced-noise digital image, each said filtered electrical signal corresponding to a characteristic of one said filtered image element.

37. The method of claim 36 wherein said step of converting said filtered image-data terms comprises the steps of extracting a portion of said filtered image-data terms and combining said portion to form a two-dimensional H×

V reduced-noise image-data set from which said reduced-noise digital image is formed.

38. An image processing apparatus for reducing the effects of blocking artifacts attributable to quantization incurred in the transform image coding and compression of a two-dimensional image signal into a digital image, the degree of compression determined by a scaling factor and a quantization table, and the image signal provided as a series of electrical signals, each electrical signal corresponding to a characteristic of an element of a two-dimensional image, where the image elements have been configured as a two-dimensional H×

V array, said apparatus comprising;

means for applying an orthogonal transform basis matrix to the two-dimensional H×

V array of image elements so as to form a coded image-data set comprising quantized terms;

means for applying said orthogonal transform basis matrix to said coded image-data set so as to form a decoded image-data set;

means for transform image coding said decoded image-data set into a plurality of frequency coefficient terms;

filtering means, operative on said frequency coefficient terms such that said frequency coefficient terms are multiplied by a coefficient filtering term to form a plurality of filtered frequency coefficient terms; and

means for converting said filtered frequency coefficients into a reduced-noise digital image.

39. The image processing apparatus of claim 38 wherein said means for transform image coding said decoded image-data set comprises means for forming said decoded image-data set into an array of overlapped image-data matrices and means for transform image coding said array of overlapped image-data matrices into an array of frequency coefficient matrices comprising said frequency coefficient terms.

40. The image processing apparatus of claim 38 further comprising means for deriving said coefficient filtering term.

41. The image processing apparatus of claim 40 wherein said means for deriving said coefficient filtering term comprises means for providing a quantization error matrix.

42. The image processing apparatus of claim 41 wherein said means for providing a quantization error matrix comprises a numerical table.

43. The image processing apparatus of claim 38 wherein said orthogonal transform basis matrix comprises a discrete cosine transform basis matrix.

44. The image processing apparatus of claim 38 wherein said means for converting said filtered frequency coefficients comprises means for transform image coding said filtered frequency coefficients into filtered image-data terms and means for converting said filtered image-data terms into a series of filtered electrical signals such that said filtered electrical signals can be configured into a two-dimensional H×

V array of filtered image elements to form said reduced-noise digital image, each said filtered electrical signal corresponding to a characteristic of one said filtered image element.

45. The image processing apparatus of claim 44 wherein said means for converting said filtered image-data terms comprises means for extracting a portion of said filtered image-data terms and means for combining said portion to form a two-dimensional H×

V reduced-noise image-data set from which said reduced-noise digital image is formed.

46. An image processing apparatus for filtering a two-dimensional decoded image signal such that the effects of blocking artifacts attributable to quantization are reduced, the blocking artifacts arising in the decoded image signal as a consequence of there having been performed transform image coding, data compression, and quantization on an antecedent image signal, the antecedent image signal having been provided as a series of electrical signals, each electrical signal corresponding to a characteristic of an element of the antecedent image, where the compression operation utilized a scaling factor and quantization terms obtained from a quantization table, and where the decoded image signal has been configured as a two-dimensional H×

V set of image-data terms denoted by S_R (z,y), said apparatus comprising;

means for transform image coding said set of image-data terms into a plurality of frequency coefficient terms by means of an orthogonal transform basis matrix;

filtering means, operative on said frequency coefficient terms such that said frequency coefficient terms are multiplied by a coefficient filtering term to form a plurality of filtered frequency coefficient terms; and

means for converting said filtered frequency coefficients into a reduced-noise digital image.

47. The image processing apparatus of claim 46 wherein said means for transform image coding said set of image-data terms comprises means for forming said image-data terms into an array of overlapped image-data matrices and means for transform image coding said array of overlapped image-data matrices into an array of frequency coefficient matrices comprising said frequency coefficient terms.

48. The image processing apparatus of claim 46 further comprising means for deriving said coefficient filtering term.

49. The image processing apparatus of claim 48 wherein said means for deriving said coefficient filtering term comprises means for providing a quantization error matrix.

50. The image processing apparatus of claim 49 wherein said means for providing a quantization error matrix comprises a numerical table.

51. The image processing apparatus of claim 46 wherein said means for converting said filtered frequency coefficients comprises means for transform image coding said filtered frequency coefficients into filtered image-data terms and means for converting said filtered image-data terms into a series of filtered electrical signals such that said filtered electrical signals can be configured into a two-dimensional H×

V array of filtered image elements to form said reduced-noise digital image, each said filtered electrical signal corresponding to a characteristic of one said filtered image element.

52. The image processing apparatus of claim 51 wherein said means for converting said filtered image-data terms comprises means for extracting a portion of said filtered image-data terms and means for combining said portion to form a two-dimensional H×

V reduced-noise image-data set from which said reduced-noise digital image is formed.