Frame interpolation and apparatus using frame interpolation

• US 20040046891A1
• Filed: 09/09/2003
• Published: 03/11/2004
• Est. Priority Date: 09/10/2002
• Status: Active Grant

• Current Assignee: Flexiworld Technologies, Inc.
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Itoh, Goh, Mishima, Nao
• Granted Patent: US 7,098,959 B2
• Time in Patent Office: Days
• US Class Current: 348/459
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51
• CPC Class Codes: H04N 7/0112 one of the standards corresponding to a cinematograph film standard
• CPC Class Codes: H04N 7/0132 the field or frame frequency of the incoming video signal being multiplied by a positive integer, e.g. for flicker reduction
• CPC Class Codes: H04N 7/014 involving the use of motion vectors motion estimation and compensation in video coding H04N19/51

Abstract

A first motion vector is estimated by using a first frame and a second frame that follows the first frame. A support frame is generated from at least either the first or the second frame by using the first motion vector. The support frame is divided into a plurality of small blocks. Motion vector candidates are estimated by using the first and second frames, in relation to each of the small blocks. A small block on the first frame, a small block on the second frame and the small blocks on the support frame are examined. The small block on the first frame and the small block on the second frame correspond to each of the motion vector candidates. A second motion vector is selected from the motion vector candidates, and points the small block on the first frame and the small block on the second frame which have the highest correlation with each of the small blocks on the support frame. An interpolated frame is generated from at least either the first or the second frame by using the second motion vector.

Claims

1. A frame interpolation method comprising:

estimating a first motion vector by using a first frame and a second frame that follows the first frame;

generating a support frame from at least either the first or the second frame by using the first motion vector;

dividing the support frame into a plurality of small blocks;

estimating motion vector candidates by using the first and second frames, in relation to each of the small blocks;

examining a small block on the first frame, a small block on the second frame and the small blocks on the support frame, the small block on the first frame and the small block on the second frame corresponding to each of motion vector candidates;

selecting a second motion vector from the motion vector candidates, the second motion vector pointing the small block on the first frame and the small block on the second frame which have the highest correlation with each of the small blocks on the support frame; and

generating an interpolated frame from at least either the first or the second frame by using the second motion vector.

2. The frame interpolation method according to claim 1, wherein the generating of the support frame includes:

scale-converting a length of the first motion vector based on time at which the frame is interpolated; and

generating the support frame by motion compensation using the scale-converted first motion vector.

3. The frame interpolation method according to claim 1, wherein the estimating of motion-vector candidates employs forward motion estimation, backward motion estimation, or bi-directional motion estimation.

4. The frame interpolation method according to claim 1 further comprising:

detecting pixels with pixel value from the support frame,

wherein the examining of the image block computes correlation by using only the detected pixels.

5. The frame interpolation method according to claim 1, wherein the estimating of motion vector candidates includes:

dividing a third frame with no image data into a plurality of small blocks of square grid;

computing correlation between a first small block in the first frame and a second small block in the second frame, the first and second small blocks being located at symmetrical positions as a block pair with one of the small blocks of the third frame as a symmetrical center, correlation being computed for all block pairs in the first and second frames; and

selecting vectors connecting small blocks in the block pairs as the motion-vector candidates in descending order of correlation.

6. The frame interpolation method according to claim 1, wherein the examining of the image block includes:

obtaining a first small block from the first frame and a second small block from the second frame, the first and second small blocks determined by each motion-vector candidate;

obtaining a third small block from the support frame at a location corresponding to the motion vector candidates;

computing a first correlation between the first and third small blocks;

computing a second correlation between the second and third small blocks;

detecting the second motion vector from the motion-vector candidates, having maximum correlation that is sum of the first and second correlations.

7. A frame interpolation method comprising:

generating a support frame from by using a first frame and a second frame that follows the first frame;

dividing the support frame into a plurality of small blocks;

estimating at least one motion-vector candidate group by recursive searching for motion-vector candidates in each small block of the support frame;

examining the motion-vector candidate group by using the support frame, to obtain a first motion-vector group and a first unmatched-filter group; and

generating an interpolated frame by motion compensation using the first motion-vector group, the first unmatched-filter group, and the first and the second frames.

8. The frame interpolation method according to claim 7, wherein the producing of the support frame includes:

dividing the first frame into a plurality of first small blocks;

estimating a second motion-vector group and a second unmatched-filter group by recursive searching to each first small block;

scale-converting a length of the second motion-vector group based on time at which the frame is interpolated, to obtain a third motion-vector group; and

generating the support frame by motion compensation using the third motion-vector group, the second unmatched-filer group, and the first and second frames.

9. The frame interpolation method according to claim 8, wherein the estimating of at least one motion-vector candidates employs forward motion estimation, backward motion estimation, or bi-directional motion estimation.

10. The frame interpolation method according to claim 7, wherein the estimating of at least one motion-vector candidates includes:

computing correlation between a first small block in the first frame and a second small block in the second frame, the first and second small blocks being located at symmetrical positions as a block pair with one of small blocks in the third frame as a symmetrical center, correlation being computed for all block pairs in the first and second frames; and

estimating the motion vector candidates from vectors connecting small blocks in the block pairs in descending order of correlation.

11. A frame interpolation method comprising:

decoding a motion-compensated predictive signal to separate the signal into a video bitstream and motion-vector information;

extracting a first and a second frame and a corresponding first motion vector from the video bitstream, thus generating a support frame from the first and second frames by using the first motion vector;

estimating motion-vector candidates from the first and second frames in relation to each block in the support frame;

examining an image block in the first frame and an image block in the second frame, and an image block in the support frame, to select one of the motion vector candidates as a second motion vector having the highest correlation as a second motion vector; and

generating an interpolated frame from image blocks, in the first and second frames, determined by the second motion vector.

12. The frame interpolation method according to claim 11, the extracting of first and second frames includes obtaining a predictive error corresponding to decoded first and second frames and estimating the first motion vector from the decoded first and second frames if the predictive error is larger than a reference value.

13. A frame interpolation apparatus comprising:

a motion vector estimating unit configured to estimate a first motion vector by using a first frame and a second frame;

a support frame generating unit configured to produce a supplemental frame from at least either the first or the second frame by using the first motion vector;

a supplemental frame dividing unit configured to divide the support frame into a plurality of small blocks;

a motion vector candidate estimating unit configured to estimate motion vector candidates by using the first frame and second frames, in relation to the small blocks;

a examining unit configured to examine a first small block on the first frame, a second small block on the second frame and the small block on the supplemental frame, the first small block and the second small block corresponding to each of the motion vector candidates;

a selecting unit configured to select a second motion vector from the motion vector candidates, the second motion vector pointing the first and second small blocks which have the highest correlation with each of the small blocks; and

a interpolated frame generating unit configured to generate an interpolated frame from at least either the first or the second frame by using the second motion vector.

14. A television set comprising:

an interlaced-to-progressive converter configured to convert an input video signal into a progressive video signal; and

a frame interpolator configured to interpolate an interpolated frame, which is generated from the progressive video signal using the frame interpolation apparatus according to claim 13, into the progressive video signal.

15. A motion picture decoding apparatus comprising:

a retriever configured to retrieve coded data of motion pictures from a storage;

a decoder configured to decode the coded data of motion pictures; and

a frame interpolator configured to interpolate an interpolated frame, which is generated from the motion pictures using the frame interpolation apparatus according to claim 13, into the motion pictures; and

a display unit configured to display the frame-interpolated decoded data of motion pictures.

16. A medical display system comprising a frame interpolator configured to interpolate an interpolated frame of a motion-picture signal, which is generated using the frame interpolation apparatus according to claim 13, into the motion-picture signal when receiving the motion-picture signal sent via an X-ray detector.

17. A television conference system comprising:

a camera configured to film a motion picture, thus producing a video signal;

an encoder configured to encode the video signal, thus generating coded data;

a transmitter configured to transmit the coded data;

a receiver configured to receive the transmitted coded data;

a decoder configured to decode the received coded data, thus reproducing the video signal; and

a frame interpolator configured to interpolate an interpolated frame, which is generated from the video signal reproduced by the decoder using the frame interpolation apparatus according to claim 13, into the video signal.

18. A television set comprising:

a receiver configured to receive coded data of motion pictures;

a decoder configured to decode the received coded data, thus reproducing the motion pictures;

a frame interpolator configured to interpolate an interpolated frame, which is generated from the motion pictures using the frame interpolation apparatus according to claim 13, into the motion pictures; and

a display unit configured to display the frame-interpolated motion pictures.

19. A moving-picture recording apparatus comprising:

an read unit configured to read a video-motion picture;

a frame interpolator configured to interpolate an interpolated frame, which is generated from the video-motion picture using the frame interpolation apparatus according to claim 13, into the video-motion picture;

an encoder configured to encode the frame-interpolated motion picture, thus generating coded data; and

a recorder configured to record the coded data in a storage.