Apparatus and methods for smoothing images

• US 5,799,111 A
• Filed: 09/06/1995
• Issued: 08/25/1998
• Est. Priority Date: 06/14/1991
• Status: Expired due to Fees

• Current Assignee: DVP Technologies Ltd.
• Original Assignee: DVP Technologies Ltd.
• Inventors: Guissin, Rami
• Primary Examiner(s): ROGERS, SCOTT A
• Application Number:
  US08/454,239
• Time in Patent Office: 1,084 Days
• Field of Search: 382/254, 382/263, 382/264, 382/266, 382/275, 382/300, 358/447, 358/463, 358/448
• US Class Current: 382/254
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20012 Locally adaptive
• CPC Class Codes: G06T 2207/20182 Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/73 Deblurring; Sharpening
• CPC Class Codes: G06T 5/90 Dynamic range modification of images or parts thereof
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness

Abstract

Improved spatial, temporal, and spatio-temporal noise reduction apparatus (10) and also apparatus for noise reduction in conjunction with one or several of the following functions: edge enhancement, spatial interpolation, magnification adjustment by spatial interpolation, and dynamic range compression (DRC). An embodiment of apparatus (10) comprises an image preprocessing unit (12), an estimation gain parameter computation unit (14) and a two-directional processor (16).

Claims

1. A method for acuity-preserving image smoothing comprising:

proceeding along at least a portion of a first dimension of received image pixels in a first direction and computing a first sequence of estimated pixel values from the received image pixels defined along the first direction;

proceeding along at least a portion of the first dimension of received image pixels in a second direction and computing a second sequence of estimated pixel values from the received image pixels defined along the second direction; and

for each individual pixel along the first dimension, computing an improved estimated pixel value for the individual pixel based on a data dependent combination of at least estimated pixel values in the first and second sequences,

wherein at least one of the computing steps takes into account image derived information other than the pixel value of the pixel to be estimated.

2. A method according to claim 1 wherein at least one of the computing steps takes into account image derived information other than the signal to noise ratio.

3. A method according to claim 1 wherein said image derived information pertains to the image as a whole.

4. A method according to claim 1 wherein said image derived information comprises the location within the image of the pixel whose value is to be estimated.

5. A method according to claim 1 wherein said image derived information comprises the location of the pixel whose value is to be estimated relative to at least one user-designated location within the image.

6. A method according to claim 1 for enhancement of contrast in an image further comprising:

extracting the high frequency detail from the image signal;

amplifying the extracted high frequency detail signal to an extent at least partly determined as a nonlinear function of at least one of the following image features;

the amplitude of the high frequency detail signal; and

local image brightness; and

combining at least a portion of the amplified high frequency detail with at least a portion of the image signal.

7. A method according to claim 6 wherein said amplifying comprises amplifying the extracted high frequency detail signal to an extent which is partly determined by the user.

8. A method according to claim 6 also comprising reducing noise in the image so as to prevent enhancement of noise artifacts.

9. A method according to claim 8 wherein said noise reduction comprises the following:

proceeding along at least a portion of a first dimension of received image pixels in a first direction and computing a first sequence of estimated pixel values from the received image pixels defined along the first direction;

proceeding along at least a portion of the first dimension of received image pixels in a second direction and computing a second sequence of estimated pixel values from the received image pixels defined along the second direction; and

for each individual pixel along the first dimension, computing an improved estimated pixel value for the individual pixel based on a data dependent combination of at least estimated pixel values in the first and second sequences.

10. A method according to claim 1 for dynamic range compression of an image, the method further comprising:

reducing noise in the image to different extents in different portions of the image, such that noise is substantially reduced in a first portion of the image which is expected to become noisy as a result of dynamic range compression and noise is only mildly reduced in a second portion of the image which is expected to be less noisy than the first portion of the image as a result of dynamic range compression; and

compressing the dynamic range of the image.

11. A method according to claim 10 wherein said noise reduction comprises the following:

proceeding along at least a portion of a first dimension of received image pixels in a first direction and computing a first sequence of estimated pixel values from the received image pixels defined along the first direction;

proceeding along at least a portion of the first dimension of received image pixels in a second direction and computing a second sequence of estimated pixel values from the received image pixel defined along the second direction; and

for each individual pixel along the first dimension, computing an improved estimated pixel value for the individual pixel based on a data dependent combination of at least estimated pixel values in the first and second sequences.

12. A method according to claim 1 for enlarging an image further comprising:

reducing noise in the image; and

subsequently interpolating between pixels so as to enlarge the image,

wherein the noise reduction step comprises;

proceeding along at least a portion of a first dimension of received image pixels in a first direction and computing a first sequence of estimated pixel values from the received image pixels defined along the first direction;

proceeding along at least a portion of the first dimension of received image pixels in a second direction and computing a second sequence of estimated pixel values from the received image pixels defined along the second direction; and

for each individual pixel along the first dimension, computing an improved estimated pixel value for the individual pixel based on a data dependent combination of at least estimated pixel values in the first and second sequences.

13. A method according to claim 12 and also comprising, following the last of said noise reducing and interpolating steps, restoring to the enlarged image, high frequency detail which was removed from the image during the noise reducing, at location in the enlarged image which correspond to the locations of the high frequency detail in the original image.

14. A video telephone system comprising:

a video telephone codec for coding and decoding a video image; and

noise reduction apparatus for reducing noise in the video image comprising;

a first-direction pixel estimator proceeding along at least a portion of a first dimension of received image pixels in a first direction and computing a first sequence of estimated pixel values from the received image pixels defined along the first direction;

a second-direction pixel estimator proceeding along at least a portion of the first dimension of received image pixels in a second direction and computing a second sequence of estimated pixel values from the received image pixels defined along the second direction; and

a data-dependent pixel estimator operative, for each individual pixel along the first dimension, to compute an improved estimated pixel value for the individual pixel based on a data dependent combination of at least estimated pixel values in the first and second sequences.

15. Apparatus according to claim 14 further comprising:

an image processor operative to perform a plurality of image improvement operations; and

a sequencer receiving an external input and operative to control the image processor such that, for each value of the external input, the image processor performs at least some of the plurality of image improvement operations in a predetermined sequence.

16. Apparatus according to claim 15 wherein the plurality of image improvement operations includes at least some of VCR noise reduction, white noise reduction, fixed pattern noise reduction, interpolation, enhancement and dithering, and wherein, upon receipt of an external input indicating that an image to be improved is arriving from a VCR, the image processor performs said image improvement operations in the above sequence.

17. Apparatus according to claim 14 for enhancement of contrast in an image further comprising:

a detail extractor operative to extract high frequency detail from the image signal;

a nonlinear detail amplifier operative to amplify the extracted high frequency detail signal to an extent at least partly determined as a nonlinear function of at least one of the following image features;

the amplitude of the high frequency detail signal; and

local image brightness; and

a detail-image combiner operative to combine the amplified high frequency detail with at least a portion of the image signal.

18. Apparatus according to claim 14 for dynamic range compression of an image, further comprising:

a differential noise reducer operative to reduce noise in the image to different extents in different portions of the image, such that noise is substantially reduced in a first portion of the image which is expected to become noisy as a result of dynamic range compression and noise is only mildly reduced in a second portion of the image which is expected to be less noisy than the first portion of the image as a result of dynamic range compression; and

a dynamic range compression unit operative to compress the dynamic range of the image.

19. Apparatus according to claim 14 for dynamic range compression of an image, the apparatus further comprising:

a differential contrast enhancer operative to enhance contrast in the image to different extents in different portions of the image, such that contrast is substantially enhanced in a first portion of the image which is expected to lose considerable contrast as a result of dynamic range compression and contrast is only mildly enhanced in a second portion of the image which is expected to lose less contrast than the first portion of the image; and

a dynamic range compression unit operative to compress the dynamic range of the image.

20. A video system according to claim 14 comprising:

a video camera for acquiring said video image.