Method and apparatus for frame interpolation

• US 8,243,194 B2
• Filed: 05/24/2007
• Issued: 08/14/2012
• Est. Priority Date: 05/31/2006
• Status: Active Grant

• Current Assignee: Vestel Elektronik Sanayi Ve Ticaret A.S. (Zorlu Holding AS)
• Original Assignee: Vestel Elektronik Sanayi Ve Ticaret A.S. (Zorlu Holding AS)
• Inventors: Ozdemir, Huseyin
• Primary Examiner(s): Harold, Jefferey
• Assistant Examiner(s): Khalid, Omer
• Application Number:
  US11/753,354
• Publication Number:
  US 20080008243A1
• Time in Patent Office: 1,909 Days
• Field of Search: 348/441
• US Class Current: 348/441
• CPC Class Codes: G06T 3/4007 based on interpolation, e.g. bilinear interpolation image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403
• CPC Class Codes: H04N 19/577 Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51
• CPC Class Codes: H04N 7/0127 by changing the field or frame frequency of the incoming video signal, e.g. frame rate converter
• CPC Class Codes: H04N 7/0137 dependent on presence/absence of motion, e.g. of motion zones H04N7/014 takes precedence; movement detection in television signals H04N5/144

Abstract

The invention provides a method of frame interpolation, the method comprising: receiving first and second frames from an input video stream; generating an interpolated frame for arranging between the first and second frames in a processed video stream so that the second video stream has a different higher frame rate than the input video stream, wherein generating the interpolated frame comprises: identifying one or more moving objects within the first frame; segmenting the or each of the identified moving objects; and, determining motion parameters for each of the segments of the segmented objects.

Claims

1. A method of frame interpolation for a video stream comprising generating an interpolated video frame for arrangement between first and second frames in a video stream, the second frame appearing after the first frame in the video stream, the method comprising:

(a) identifying one or more moving objects within the first frame;

(b) segmenting each moving object identified in step (a);

(c) determining motion parameters for each of the segments of the segmented objects;

(d) generating said interpolated video frame in dependence on said determined motion parameters; and

,

predicting the position of objects within a third frame, the third frame appearing after the second frame in the video stream, wherein said predictions are dependent on only the motion parameters determined in step (c) for the first interpolated frame, checking said predictions against the third frame for a predetermined accuracy, and if accurate, using only said motion parameters for the first interpolated frame in generating a second interpolated frame for arrangement between the second frame and the third frame in the video stream, else if not accurate repeating steps (a) to (d) with the second and third frames, wherein the first, second and third frames are consecutive.

2. A method according to claim 1, wherein the step of determining motion parameters for the or each of the segments comprises identifying for at least one of the segments of the first frame a match in the second frame.

3. A method according to claim 2, wherein once a matching pair of segments has been found, motion parameters for the segment are determined.

4. A method according to claim 3, wherein the position of a corresponding segment in the interpolated frame is determined in dependence on the determined motion parameters for the segment in the first frame.

5. A method according to claim 2, wherein identifying for at least one of the segments of the first frame a match in the second frame comprises identifying all possible matches and performing analysis on the possible pairs to determine an actual match.

6. A method according to claim 5, wherein the analysis includes determining a match coefficient defined as follows:

Match coefficient=ABC

in which A=Average luminance difference;

B=Average colour vector magnitude difference; and

C=Shape index difference.

7. A method according to claim 6, in which a match coefficient is determined for all possible matching pairs of segments and a match is formed between the two segments having the lowest match coefficient.

8. A method according to claim 4, wherein after a match has been established between all possible matching pairs of segments, values for picture parameters of the interpolated segments are smoothed.

9. A method according to claim 1 further comprising up-converting the frame rate of a first video stream thereby generating a second processed video stream, wherein said upconvertinq comprises:

receiving a first video stream at a first frame rate;

generating interpolated frames for arranging temporally between pairs of frames in the first video stream; and

collating the frames of first video stream with the generated interpolated frames to produce a second video stream having a frame rate higher than the frame rate of the first video stream.

10. A method according to claim 9, wherein the first video stream has a frame rate of NHz and the second video stream has a frame rate of 2 NHz.

11. Apparatus for frame interpolation of a video stream, the apparatus comprising:

a processor for extracting first and second frames from a first video stream, the second frame appearing after the first frame in the video stream and generating an interpolated frame for arrangement temporally between the first and second frames in a second video stream, the processor having

an identifier for identifying in a step (a) one or more objects within the first frame;

a segmenter for segmenting in a step (b) each object identified in step (a);

a determiner for determining in a step (c) motion parameters for each of the segmented objects;

a generator for generating in a step (d) said interpolated video frame in dependence on said determined motion parameters; and

,

an predictor for predicting the position of objects within a third frame, the third frame appearing after the second frame in the video stream, wherein said predictions are dependent on only the motion parameters determined in step (c) for the first interpolated frame, checking said predictions against the third frame for a predetermined accuracy, and if accurate, using only said parameters determined in step (c)for the first interpolated frame in generating a second interpolated frame for arrangement between the second frame and the third frame in the video stream, else if not accurate, repeating steps carried out by steps (b) to (d) with the second and third frames, wherein the first, second and third frames are consecutive.

12. An apparatus according to claim 11, wherein the apparatus comprises to a frame rate up-converter for up-converting the frame rate of a first video signal, the frame rate up-converter being arranged to receive the first video signal and generate interpolated frames for arranging with the frames of the first video signal.

13. An apparatus according to claim 12, wherein the apparatus corresponds to a television set having

a receiver for receiving a television signal at a first frame rate;

and

the frame rate up-converter for processing the received television signal.