Method and apparatus for spatially adaptive filtering for video encoding

• US 5,764,307 A
• Filed: 07/24/1995
• Issued: 06/09/1998
• Est. Priority Date: 07/24/1995
• Status: Expired due to Fees

• Current Assignee: Motorola, Inc. (Motorola Solutions, Inc.), Northwestern University
• Original Assignee: Motorola, Inc. (Motorola Solutions, Inc.), Northwestern University
• Inventors: Erdogan, Ozan, Katsaggelos, Aggelos K., Auyeung, Cheung, Brailean, James C., Ozcelik, Taner
• Primary Examiner(s): Grant, Christopher C.
• Application Number:
  US08/506,272
• Time in Patent Office: 1,051 Days
• Field of Search: 348/607, 348/608, 348/609, 348/701, 348/700, 348/620, 348/621, 348/618, 348/619, 348/384, 348/390, 382/254, 382/260, 382/261, 382/262, 382/263, 382/264
• US Class Current: 348/608
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence
• CPC Class Codes: H04N 5/21 Circuitry for suppressing or minimising disturbance, e.g. moiré or halo

Abstract

The present invention provides a method (200) and an apparatus (100) for spatially adaptive filtering for video encoding. The apparatus filters a video sequence prior the encoding process. The apparatus comprises a noise variance determiner (102), a local variance determiner (104), a noise visibility function determiner (106), a Gaussian kernel determiner (108), and a convolver (110). The apparatus removes noise directly from a Displaced Frame Difference, DFD, signal. This novel approach removes noise and miscellaneous high frequency components from the DFD signal without the introduction of the filtering artifacts characteristic of current techniques. By reducing the miscellaneous high frequency components, the present invention is capable of reducing the amount of information that must be encoded by the video encoder without substantially degrading the decoded video sequence.

Claims

1. A method of spatially adaptive filtering a displaced frame difference, a DFD, the DFD is a difference between a video frame and an estimated video frame, the method comprising:

determining, using a noise variance determiner, a noise variance of the DFD based on a predetermined signal-to-noise ratio;

determining, using a local variance determiner, a local variance of the DFD based on the noise variance and the predetermined signal-to-noise ratio;

determining, using a noise visibility determiner, a noise visibility function based on the noise variance, the local variance, and a predetermined image type;

determining, using a Gaussian kernel determiner, a Gaussian kernel for every pixel in the DFD based on the noise visibility function, a predetermined normalizing coefficient, and a predetermined base variance; and

convolving, using a convolver, the DFD with the Gaussian kernel to provide an adaptively filtered DFD.

2. The method of claim 1 wherein the noise variance is determined by calculating an energy contained in each predefined block using the following equation:

##EQU6## where M and N are the dimensions of the predefined block and i,j denotes a row and column of a pixel in the predefined block.

3. The method of claim 1 wherein the local variance of the DFD is determined using the following equation:

##EQU7## where M and N are the dimensions of the predefined block and i,j denotes a row and column of a pixel in the predefined block.

4. The method of claim 1 wherein the noise visibility function is determined using the following equation:

##EQU8## where σ

.sub.ƒ

² is the local variance, σ

_n² is the noise variance, and Θ

is the predetermined image type.

5. The method of claim 1 wherein the Gaussian kernel is determined using the following equation:

##EQU9## where σ

_g² is the predetermined base variance, Z is the predetermined normalizing coefficient, υ

(Θ

) is the noise visibility function, and m,n denotes a row and column of a pixel in the DFD.

6. The method of claim 1 wherein the DFD convolved with the Gaussian kernel is determined using the following equation:

##EQU10## where g(m,n) is the Gaussian kernel and m,n denotes a row and column of a pixel in the DFD.

7. The method of claim 1 wherein the steps of the method are embodied in a Digital Signal Processor, DSP, that has been programmed with said steps.

8. The method of claim 1 wherein the steps of the method are embodied in an Application Specific Integrated Circuit, ASIC, arranged to provide said steps.

9. The method of claim 1 wherein the steps of the method are embodied in a gate array that has been arranged to provide said steps.

10. The method of claim 1 wherein the steps of the method are stored in a computer memory.

11. The method of claim 10 wherein the computer memory is a computer diskette having a stored program to impart processing functionality for implementing the steps of the method by the computer.

12. The method of claim 10 wherein the computer memory is a memory unit of the computer that includes a program to impart processing functionality for implementing the steps of the method by the computer.

13. An apparatus for spatially adaptive filtering a displaced frame difference, a DFD, the DFD is a difference between a video frame and an estimated video frame, the apparatus comprising:

a noise variance determiner, operably coupled to receive the DFD and a predetermined signal-to-noise ratio, for determining a noise variance of the DFD based on the predetermined signal-to-noise ratio;

a local variance determiner, operably coupled to the noise variance determiner and operably coupled to receive the DFD, for determining a local variance of the DFD based on the noise variance and the predetermined signal-to-noise ratio;

a noise visibility determiner, operably coupled to the noise variance determiner and the local variance determiner and operably coupled to receive a predetermined image type, for determining a noise visibility function based on the noise variance, the local variance, and the predetermined image type;

a Gaussian kernel determiner, operably coupled to the noise visibility determiner and operably coupled to receive a predetermined base variance, for determining a Gaussian kernel for every pixel in the DFD based on the noise visibility function, a predetermined normalizing coefficient, and the predetermined base variance; and

a convolver, operably coupled to the Gaussian kernel determiner and operably coupled to receive the DFD, for convolving the DFD with the Gaussian kernel to provide an adaptively filtered DFD.

14. The apparatus of claim 13 wherein the noise variance is determined by calculating an energy contained in each predefined block using the following equation:

##EQU11## where M and N are the dimensions of the predefined block and i,j denotes a row and column of a pixel in the predefined block.

15. The apparatus of claim 13 wherein the local variance of the DFD is determined using the following equation:

##EQU12## where M and N are the dimensions pf the predefined block and i,j denotes a row and column of a pixel in the predefined block.

16. The apparatus of claim 13 wherein noise visibility function is determined using the following equation:

##EQU13## where σ

.sub.ƒ

² is the local variance, σ

_n² is the noise variance, and Θ

is the predetermined image type.

17. The apparatus of claim 13 wherein the Gaussian kernel is determined using the following equation:

##EQU14## where σ

_g² is the predetermined base variance, Z is the predetermined normalizing coefficient, υ

(Θ

) is the noise visibility function, and m,n denotes a row and column of a pixel in the DFD.

18. The apparatus of claim 13 wherein the DFD signal convolved with the Gaussian kernel is determined using the following equation:

##EQU15## where g(m,n) is the Gaussian kernel and m,n denotes a row and column of a pixel in the DFD.

19. The apparatus of claim 13 wherein the apparatus includes a Digital Signal Processor, DSP.

20. The apparatus of claim 13 wherein the apparatus includes an Application Specific Integrated Circuit, ASIC.

21. The apparatus of claim 13 wherein the apparatus includes a gate array.

22. The apparatus of claim 13 wherein the apparatus includes a computer usable medium having computer readable program codes.

23. The apparatus of claim 22 wherein the computer usable medium is a computer diskette.

24. The apparatus of claim 22 wherein the computer usable medium is a memory unit of the computer.