Process for interpolating progressive frames

• US 5,995,154 A
• Filed: 12/10/1996
• Issued: 11/30/1999
• Est. Priority Date: 12/22/1995
• Status: Expired due to Fees

• Current Assignee: Thomson Multimedia SA (Vantiva SA)
• Original Assignee: Thomson Multimedia SA (Vantiva SA)
• Inventors: Heimburger, Catherine
• Primary Examiner(s): Peng, John K.
• Assistant Examiner(s): Natnael, Paulos M.
• Application Number:
  US08/763,178
• Time in Patent Office: 1,085 Days
• Field of Search: 348/448, 348/449, 348/450, 348/451, 348/452, 348/443, 348/446, 348/447, 348/458, 348/910, 348/911
• US Class Current: 348/448
• CPC Class Codes: H04N 19/577 Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence
• CPC Class Codes: H04N 7/012 Conversion between an interlaced and a progressive signal for graphics images G09G2310/0229
• CPC Class Codes: H04N 7/0132 the field or frame frequency of the incoming video signal being multiplied by a positive integer, e.g. for flicker reduction
• CPC Class Codes: H04N 7/014 involving the use of motion vectors motion estimation and compensation in video coding H04N19/51

Abstract

A process for converting interlaced frames into progressive frames comprising a change of frame frequency by interpolation and motion compensation, wherein when a motion vector associated with a pixel to be interpolated is non-zero or when the motion vector is zero but the confidence accorded to this vector is less than a given threshold, the interpolation of the pixel of a frame situated temporally between two input frames is carried out by a median filtering pertaining to the values obtained by a first motion compensated linear temporal filter, a second motion compensated linear filter, and a motion compensated median temporal filter.

Claims

1. Process for converting interlaced frames into progressive frames comprising a change of frame frequency by interpolation and motion compensation wherein

when a motion vector associated with a pixel to be interpolated is non-zero or when this vector is zero, but the confidence accorded to this vector is less than a given threshold,

the interpolation of a pixel (x,y) of a frame situated temporally between two input frames is carried out by a combination of median filtering and motion compensation linear and median filtering comprising:

obtaining by a first motion-compensated linear temporal filter a first interpolated value based on a first set of pixels;

obtaining by a second motion-compensated linear temporal filter a second interpolated value based on a second set of pixels;

obtaining by a motion-compensated median temporal filter a third median value based on a third set of pixels; and

median filtering according to said first and second interpolated values and said third median value to obtain a value corresponding to said interpolated pixel (x,y).

2. Process according to claim 1, wherein the confidence accorded to a vector is a linear function of the error engendered by this vector.

3. Process according to claim 1 wherein the said linear filters are dependent on two pixels (a,b respectively b,c) situated respectively in the input frames temporally surrounding the frame to be interpolated, the linear interpolation coefficients of each pixel corresponding to the ratio of the time intervals separating the frame to be interpolated and the preceding input frame and the following input frame respectively.

4. Process according to claim 1 wherein when the preceding input frame is odd, the said median temporal filter pertains to the following three pixels:

For the even lines of the frame to be interpolated, ##EQU47## then;

P₂ (x,y)=med(ma+(1-m)b;

med(a,b,c);

mc+(1-m)b)

For the odd lines of the frame to be interpolated, ##EQU48## then;

P₂ (x,y)=med((1-m)a+mb;

med(a,b,c);

(1-m)c+mb)

where Front(i,j) and Rear(i,j) are the fractions of the motion vector associated with the pixel to be interpolated, this vector being scaled and rounded, where "0" and "1/2" respectively represent the input frame preceding respectively following the frame to be interpolated, and where m represents an interpolation coefficient depending on the position of the frame to be interpolated between the input frames.

5. Process according to claim 1 wherein when the preceding input frame is odd, the said median temporal filter pertains to the following three pixels:

For the odd lines of the frame to be interpolated;

##EQU49## then;

P₃ (x,y)=med((1-n)a+b;

med(a,b,c);

(1-n)c+b)

For the even lines of the frame to be interpolated;

##EQU50## then;

P3(x,y)=med(na+(1-n)b;

med(a,b,c)nc+(1-n)b)

where Front(i,j) and Rear(i,j) are the fractions of the motion vector associated with the pixel to be interpolated, this vector being scaled and rounded, where "1/2" and "1" respectively represent the input frame preceding respectively following the frame to be interpolated, and where n represents an interpolation coefficient depending on the position of the frame to be interpolated between the input frames.

6. Process according to claim 3 wherein the two pixels used for each linear filter are chosen from the group of three pixels used by the median temporal filter.

7. Process according to claim 1 wherein

the interlaced input frames emanate from progressive input images of frequency 25 Hz and that the frequency of the output frames is 75 Hz, a first output frame (F1) being temporally merged with the second frame emanating from a first input image, a second output frame (F2) being temporally situated between the said first input frame and the first frame (F2/3) emanating from a second input image, a third output frame (F3) being temporally situated between the two input frames (F2/3, F4) emanating from the said second input image,

the interpolation coefficients used by the said temporal filters are 1/2 for the said second output frame.

8. Process according to claim 7, wherein

the pixels of the said third output frame are the pixels of the second input image.

9. Process according to claim 1 wherein when the motion vector associated with a pixel to be interpolated is zero and the confidence in this vector exceeds a given threshold, then the value of the pixel to be interpolated is the final value obtained by median filtering between

firstly two pixel values, one of which is obtained by subtracting the motion vector confidence referred to the pixel and divided by a corrective factor from the value of the pixel with the same position as the pixel to be interpolated in, either, the preceding input frame or the following input frame, and the other obtained by adding the motion vector confidence referred to the pixel and divided by the corrective factor to the value of the pixel with the same position as the pixel to be interpolated in, either, the preceding input frame or the following input frame, and

secondly a fallback value which results from a temporal and horizontal linear filtering on the pixels of the lines of the preceding and following input frames with identical position to the line containing the pixel to be interpolated, a vertical spatial filtering being performed if appropriate on the undefined lines of the input frames.

10. Process according to claim 9, wherein the fallback value is equal to:

##EQU51## where "0" denotes the input frame preceding the frame to be interpolated and "1/2" the input frame following the frame to be interpolated, and where m represents a weighting coefficient depending on the temporal position of the frame to be interpolated between the input frames.

11. Process according to claim 9 wherein

the interpolation of a pixel (P(x,y)) of a line of given parity of a frame (1) whose lines of opposite parity are equal to the lines of an input frame (1) is carried out, when the associated motion vector is not zero or the confidence accorded to this vector is low, through median filtering (P1 (x,y)) pertaining to the values obtained

firstly by vertical linear filtering (F1),

secondly by vertical spatio-temporal median filtering on two pixels and motion-compensated on one pixel (F2) and

thirdly by spatio-temporal median filtering pertaining to four pixels in a cross and motion-compensated on one pixel (F3).

12. Process according to claim 11, wherein the said vertical linear filtering (F1) is as follows:

##EQU52## where "0" represents the temporal position of the frame to be interpolated.

13. Process according to claim 11 wherein the said vertical spatio-temporal filtering on two pixels and motion-compensated on a pixel (F2) is:

##EQU53## where Vx and Vy are the coordinates of a motion vector associated with the pixel to be interpolated, and where "0" represents the frame to be interpolated and "1/2" the following input frame.

14. Process according to claim 11 wherein the said spatio-temporal filtering pertaining to four pixels in a cross and to a motion-compensated pixel (F3) is:

##EQU54##

15. Process according to claim 11 wherein when the motion vector associated with a pixel to be interpolated is zero and the confidence in this vector is greater than a given threshold, then the said pixel takes the value of the pixel with the same position in the following input frame.

16. A process for converting interlaced frames into progressive frames comprising a change of frame frequency by interpolation and motion compensation wherein when a motion vector associated with a pixel to be interpolated is non-zero or when this vector is zero, but a confidence level accorded to this vector is less than a given threshold,

the interpolation of a pixel (x,y) of a frame situated temporally between two input frames is carried out by a first motion-compensated linear temporal filter for obtaining a first interpolated pixel value based on at least two pixels a, b associated with said two input frames, a second motion-compensated linear filter for obtaining a second interpolated pixel value based on at least two pixels b, c associated with said two input frames, a motion-compensated median temporal filter for obtaining a median pixel value based on at least three pixels a, b, c associated with said two input frames;

and median filtering according to said first and second interpolated pixel values and said median pixel value to obtain a value corresponding to the interpolation of said pixel (x,y).