Method and apparatus for blocking effect reduction in images by post-processing in the spatial domain

• US 5,933,542 A
• Filed: 04/24/1996
• Issued: 08/03/1999
• Est. Priority Date: 04/24/1996
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Lee, Chuen-Chien, Chang, Ching-Fang
• Primary Examiner(s): Rogers, Scott A.
• Application Number:
  US08/637,206
• Time in Patent Office: 1,196 Days
• Field of Search: 382/268, 382/266, 382/264, 382/275, 382/260, 382/254, 382/248, 382/250, 358/463, 358/433, 358/432, 348/420, 348/404, 348/403
• US Class Current: 382/264
• CPC Class Codes: H04N 19/527 Global motion vector estimation
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness

Abstract

An apparatus and method for post processing image data which previously was encoded using a discrete cosine transform in order to remove resulting blocking effects. Correction coefficients are generated by determining the least mean square error of the blocking error. The correction coefficients are adjusted to be within the quantization range of the quantization step size used during the coding process. The adjusted correction coefficients are then used to generate the correction terms which are subsequently combined with the original image data to generate an image exhibiting reduced blocking effects when the image is displayed.

Claims

1. A method for minimizing blocking effects in a blocking image caused by prior performance of encoding/decoding processes including a transform and quantization process, said blocking image described by blocking image data, said method comprising the steps of:

determining blocking errors in the blocking image data;

generating correction coefficients in accordance with the least mean square error of the blocking errors;

calculating correction terms using the correction coefficients;

combining the blocking image data and the correction terms to generate an adjusted image data.

2. The method as set forth in claim 1, further comprising the step of adjusting the correction coefficients to be within quantization range defined by the quantization step size utilized in the coding process;

wherein the step of calculating correction terms is performed using adjusted correction coefficients.

3. The method as set forth in claim 1, wherein the step of determining blocking errors comprises the steps of:

filtering the blocking image data to remove high frequency components above a first identified threshold to generate a filtered image data; and

subtracting the filtered image data from the blocking image data to generate the blocking errors.

4. The method as set forth in claim 3, wherein the first identified threshold is chosen such that most of the blocking effects are removed, at a cost of sharpness of the image.

5. The method as set forth in claim 1, further comprising the step of wide band filtering above a second identified threshold the combined image data to remove artifacts caused by the step of combining the blocking image data and the correction terms to generate an adjusted image data.

6. The method as set forth in claim 5, wherein the second identified threshold is chosen such that only very high frequency components are removed, while sharpness of the image is preserved as much as possible.

7. The method as set forth in claim 1, wherein the step of generating correction coefficients comprises the step of applying a Least Mean Square Error (LMSE) criterium to the Mean Square Error (MSE) of the blocking errors by solving the following to determine the correction coefficients a_kl :

##EQU13## where 0≦

k<

kx,0≦

l<

lx,kx*lx is equal to the number of correction coefficients used in the correction terms, and a_kl are the correction coefficients.

8. The method as set forth in claim 7, wherein the MSE is chosen to be the blocking errors of all pixels in the block as defined by the following equation:

##EQU14## and the resulting solutions of the correction coefficients a_kl are determined according to the following equations;

##EQU15## where E i! j! is the blocking error, A i! j! is the correction term, and M,N represent the dimensions of the block size.

9. The method as set forth in claim 7, wherein the MSE can be chosen to be the blocking errors of the pixels along the block boundaries as defined by the following equation:

##EQU16## where E ! !0 the blocking error, A ! ! represents the correction turn and M,N are the dimensions of the block size.

10. The method as set forth in claim 9 wherein if kx=2 and lx=2, the resulting solutions of the four correction coefficients, a₀₀ and a₀₁, a₁₀, a₁₁, are determined according to the following equation:

##EQU17## where E ! ! represents the blocking error and M,N are the dimensions of the block size.

11. The method as set forth in claim 2, wherein the stop of adjusting the correction coefficients comprises the step of if the correction coefficients is outside a quantization range specified by plus or minus 1/2 of a quantization step size used during the coding process, adjusting the correction coefficient toward the nearest boundary of the quantization range.

12. The method as set forth in claim 1, wherein the step of calculating the correction terms comprises the performing the following equation:

##EQU18## where M,N represent the dimensions of the block size, 0≦

i<

N,0≦

j<

M,0<

kx≦

N,<

;

0<

lx≦

M, and kx*lx is the number of correction coefficients used in the correction term.

13. The method as set forth in claim 1, wherein the step of combining further comprises the step of for each pixel in the block, computing, p'"'"' i! j!=p i! j!+A i! j! where p ! ! is the original blocking image and p'"'"' ! ! is the produced image exhibiting reduced blocking effects and A ! ! are correction terms.

14. A circuit for minimizing blocking effects in a blocking image caused by prior performance of coding processes including transform and quantization processes, said blocking image described by blocking image data, said circuit comprising:

a first low pass filter coupled to received the blocking image data and output filtered image data, said first low pass filter filtering the blocking image data to remove high frequency components above a first identified threshold;

a blocking error detection subcircuit coupled to receive the filtered image data and the blocking image data and generate blocking error data;

a correction coefficient subcircuit coupled to receive the blocking error data and generating correction coefficients in accordance with the least mean square error of the blocking error data;

correction term subcircuit coupled to receive the correction coefficients and calculating correction terms;

combination subcircuit coupled to receive the blocking image data and the correction terms, said combination subcircuit combining the correction terms and the blocking image data to generate combined image data that exhibits reduced blocking effects.

15. The circuit as set forth in claim 14, further comprising a second low pass filter coupled to receive the combined image data, said second low pass filter filtering out artifacts generated by the combination subcircuit.

16. The circuit as set forth in claim 15, wherein a cutoff frequency of the second low pass filter is higher than a cutoff frequency of the first low pass filter.

17. The circuit as set forth in claim 14, wherein a cutoff frequency of the first low pass filter is set to a frequency such that most of the blocking effects are removed, at the cost of sharpness of the image.

18. The circuit as set forth in claim 15, wherein a cutoff frequency of the second low pass filter is set to a frequency such that only very high frequency components are removed, while sharpness of the image is preserved as much as possible.

19. The circuit as set forth in claim 14, wherein the blocking error detection subcircuit generates blocking error data by subtracting the filtered image data from of the blocking image data.

20. The circuit as set forth in claim 14, wherein the correction coefficient subcircuit determines the correction coefficients a_kl according to following equations by applying least mean square errors to the blocking errors of all the pixels in the block:

##EQU19## where E i! j! is the blocking error, 0≦

k<

kx,0≦

l<

lx,kx*lx is the number of coefficients used in the correction term, M and N represent a first and second dimension of the block size.

21. The circuit as set forth in claim 14, wherein the correction coefficient subcircuit determines the correction coefficients a_kl by applying LMSE to the blocking errors of only the pixels along the block boundaries as defined by the following:

##EQU20## where E ! ! represents the blocking error, A ! ! represents the correction turn and M,N are the dimensions of the block size.

22. The circuit as set forth in claim 21, wherein, kx=2 and lx=2 as the correct coefficients comprise a₀₀ and a₀₁, a₁₀, a₁₁ as defined by the following equations:

##EQU21## where E ! ! represents the blocking error and M,N are the dimensions of the block size.

23. The circuit as set forth in claim 14, wherein correction term subcircuit calculates the correction terms using the following equation:

##EQU22## where M,N represent the dimensions of the block size, 0≦

i<

N,0≦

j<

M,0<

kx≦

N,<

;

0<

lx≦

M, and kx*lx is the number of correction coefficients used in the correction term.

24. The circuit as set forth in claim 14 further comprising a Q-level adjustment subcircuit to adjust the correction coefficients to be within the quantization range defined by the quantization step size utilized in the coding process;

wherein the correction term subcircuit receives adjusted correction coefficients to calculate correction terms.

25. The circuit as set forth in claim 14 wherein the Q-level adjustment subcircuit adjusts the correction coefficients towards the nearest boundary of a quantization range specified as plus or minus one half the quantization step size if the correction coefficients are outside the quantization range.

26. The circuit as set forth in claims 14, wherein the combination subcircuit combines the correction terms and blocking image data according to the following equation:

p'"'"' i! j!=p i! j!+A i! j!, where p ! ! is the original blocking image and p'"'"' ! ! is the produced image exhibiting reduced blocking effects and A ! ! are the correction terms.