Quality image warper

• US 6,061,477 A
• Filed: 04/15/1997
• Issued: 05/09/2000
• Est. Priority Date: 04/18/1996
• Status: Expired due to Term

• Current Assignee: Sarnoff Corporation (SRI International, Inc.)
• Original Assignee: Sarnoff Corporation (SRI International, Inc.)
• Inventors: Burt, Peter Jeffrey, Lohmeyer, Mark S., van der Wal, Gooitzen Siemen
• Primary Examiner(s): Boudreau, Leo H.
• Assistant Examiner(s): SHERALI, ISHRAT I
• Application Number:
  US08/838,101
• Time in Patent Office: 1,120 Days
• Field of Search: 382/268, 382/293, 382/295, 382/296, 382/298, 382/297, 382/299, 382/300, 382/301, 358/451, 358/525, 358/428, 348/581, 348/263, 348/583, 348/399, 348/580, 348/420, 364/724.011, 364/724.1, 345/433, 345/436-439, 345/427, 345/127-130, 708/300, 708/313
• US Class Current: 382/300
• CPC Class Codes: G06T 3/4007 based on interpolation, e.g. bilinear interpolation image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403
• CPC Class Codes: G06T 3/4084 in the transform domain, e.g. fast Fourier transform [FFT] domain scaling

Abstract

A digital image warper system produces a warped output image from an input image. The warper system increases the sampling rate of a sampled image above the Nyquist rate or samples an analog image at a higher rate than the Nyquist rate and prefilters the upsampled image to counteract filtering which may occur during the warping operation. The upsampled image is warped using an interpolator for resampling to produce a warped upsampled image. The interpolator is, for example, a low quality interpolator such as a bilinear interpolator. The warped image is then down-sampled to the same resolution as the input image to produce the warped image. Down-sampling and warping can be combined into one step by modifying the geometric transformation function implemented when warping the upsampled image.

Claims

1. A image processing system comprising:

means for applying an input image signal to the image processing system, said input image signal having an input sampling rate;

means for uniformly upsampling and filtering all pixels in the input image to increase the sampling rate of the input image signal to at least two times the Nyquist rate, the upsampled image having interpolated values spaced uniformly at predetermined distances from each input sample;

means for warping the uniformly upsampled image signal to produce an upsampled warped image signal, and

means for down sampling the upsampled warped image signal to produce a warped image output signal having a sampling rate which is substantially the same as the input image sampling rate.

2. The image processing system according to claim 1, wherein the means for warping includes a bilinear interpolator.

3. The image processing system according to claim 1, wherein the upsampling and filtering means samples the input image signal at a rate of 2^N times the Nyquist rate where N is a positive integer.

4. The image processing system according to claim 1, wherein the input image signal has pixel values which are provided to the upsampling and filtering means at one pixel per clock cycle and the upsampling and filtering means produces four upsampled pixel values per clock cycle.

5. The image processing system according to claim 4, wherein the warping means includes four memories and the four upsampled pixel values are stored in respectively different ones of the four memories.

6. The image processing system according to claim 1, wherein the upsampling and filtering means includes a filter for processing the input image signal to counteract deterioration in spatial frequency characteristics caused by the warping means.

7. The image processing system according to claim 1, wherein:

the input image signal includes pixel values;

the warping means includes four memories; and

the upsampling and filtering means further produces four pixel values for each of one the pixel values in the input image signal and stores each one of the four pixel values in a respectively different one of the four memories.

8. The image processing system according to claim 1, wherein:

the input image signal includes pixel values; and

the upsampling and filtering means further produces four pixel values for each one of the pixel values in the input image signal where;

(a) a first one of the four pixel values corresponds to one pixel value in the input image signal,

(b) a second one of the four pixel values is produced by horizontally interpolating ones of the pixel values in the input image signal,

(c) a third one of the four pixel values is produced by vertically interpolating ones of the pixel values in the input image signal, and

(d) a fourth one of the four pixel values is produced by horizontally and vertically interpolating ones of the pixel values in the input image signal.

9. The image processing system according to claim 1, wherein the input image signal includes multiple image planes and the image processing means includes means for spatially interleaving the multiple image planes and wherein the warping means concurrently warps the multiple image planes in an interleaved operation.

10. A image processing system according to claim 1, wherein:

the means for warping and the means for down sampling operate at substantially the same time.

11. An integrated circuit comprising:

an input terminal to which an input image signal including pixel values are provided said input image signal having an input sampling rate;

upsampling and filtering circuitry, coupled to the input terminal, which upsamples the input image signal to at least two times the Nyquist rate to provide an increased sampling rate image signal including pixel values, the increased sampling rate image signal having interpolated pixel values spaced uniformly over the entire input image signal at predetermined distances from each input pixel value;

write address circuitry which generates a write address signal that specifies a location in a memory where the pixels values are to be stored;

read address circuitry which generates a read address signal that specifies which ones of the pixels values are to be retrieved from the memory;

warper circuitry which receives retrieved pixel values from the memory and produces a warped image signal; and

down sampling circuitry, coupled to the warper circuitry, which produces a warped image output signal, said output signal having an output sampling rate substantially equal to said input sampling rate.

12. The integrated circuit according to claim 11, wherein the down sampling circuitry and the warper circuitry are combined as one circuit.

13. A method for processing an input image signal having an input sampling rate comprising the steps of:

(a) increasing the sampling rate of the input image signal to at least two times the Nyquist rate by interpolating values spaced uniformly over the entire input image signal at predetermined distances from each input sample;

(b) warping the upsampled image signal, and

(c) down sampling the upsampled warped image signal to produce a warped image output signal having an output sampling rate substantially equal to said input sampling rate.

14. The method according to claim 13, wherein step (a) samples the input image signal at a rate of 2^N times the Nyquist rate where N is a positive integer.

15. The method according to claim 13, wherein step (a) includes the step of filtering the input image signal to counteract deterioration in the frequency characteristics of the upsampled warped image signal caused by step (b).

16. The method according to claim 13, wherein:

the input image signal includes pixel values;

step (a) produces four pixel values for each one of the pixel values in the input image signal, and

the method further comprises the step of storing each one of the four pixel values in a separate memory.

17. The method according to claim 13, wherein steps (b) and (c) are performed at substantially the same time.

18. An image processing system comprising:

means for applying an input image signal to the image processing system, said input image signal having an input sampling rate;

means for uniformly upsampling and filtering to increase the sampling rate of the input image signal, the upsampled image having interpolated values spaced uniformly at predetermined distances from each input sample;

means for warping the upsampled image signal, and

means for down sampling the warped image signal to produce a warped image output signal;

wherein;

(a) the input image signal is a digital signal sampled at the Nyquist rate and the upsampling and filtering means increases the sampling rate of the input image signal to at least two times the Nyquist rate, and

(b) the means for warping includes a bilinear interpolator.

19. An image processing system comprising:

means for applying an input image signal to the image processing system, and input image signal having an input sampling rate;

means for uniformly upsampling and filtering to increase the sampling rate of the input image signal, the upsampled image having interpolated values spaced uniformly at predetermined distances from each input sample;

means for warping the upsampled image signal, and

means for down sampling the warped image signal to produce a warped image output signal;

wherein the upsampling and filtering means samples the input image signal at a rate of 2^N times the Nyquist rate where N is a positive integer.

20. An image processing system comprising:

means for applying an input image signal to the image processing system, said input image signal having an input sampling rate;

means for uniformly upsampling and filtering to increase the sampling rate of the input image signal to at least two times the Nyquist rate, the upsampled image having interpolated values spaced uniformly at predetermined distances from each input sample;

means for warping the upsampled image signal, and

means for down sampling the warped image signal to produce a warped image output signal;

wherein

the input image signal includes pixel values; and

the upsampling and filtering means further produces four pixel values for each one of the pixel values in the input image signal where;

(a) a first one of the four pixel values corresponds to one pixel value in the input image signal,

(b) a second one of the four pixel values is produced by horizontally interpolating ones of the pixel values in the input image signal,

(c) a third one of the four pixel values is produced by vertically interpolating ones of the pixel values in the input image signal, and

(d) a fourth one of the four pixel values in produced by horizontally and vertically interpolating ones of the pixel values in the input image signal.

21. A method for processing an input image signal having an input sampling rate comprising the steps of:

(a) increasing the sampling rate of the input image signal by interpolating values spaced uniformly at predetermined distances from each input sample;

(b) warping the upsampled image signal, and

(c) down sampling the upsampled warped image signal to produce a warped image output signal having an output sampling rate substantially equal to said input sampling rate,

wherein step (a) samples the input image signal at a rate of 2^N times the Nyquist rate where N is a positive integer.