Apparatus and method of converting frame and/or field rate using adaptive motion compensation

• US 20030086498A1
• Filed: 10/22/2002
• Published: 05/08/2003
• Est. Priority Date: 10/25/2001
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Lee, Sung-Hee, Yang, Seung-joon
• Granted Patent: US 7,075,988 B2
• Time in Patent Office: Days
• US Class Current: 375/240.16
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51
• CPC Class Codes: H04N 7/0137 dependent on presence/absence of motion, e.g. of motion zones H04N7/014 takes precedence; movement detection in television signals H04N5/144
• CPC Class Codes: H04N 7/014 involving the use of motion vectors motion estimation and compensation in video coding H04N19/51

Abstract

An apparatus and method which includes estimating a motion vector of a frame and/or field to be interpolated and the accuracy of the motion vector using motion vectors of previous and next frames and/or fields; determining motion information by analyzing the estimated motion vector, and calculating a correlation between the motion vector of the current block and the motion vectors of neighboring blocks; generating pixels to be interpolated by performing motion compensation in a frame and/or field to be interpolated using the estimated motion vector and using pixels neighboring a pixel to be interpolated in the frame and/or field to be interpolated and pixels neighboring a pixel to be interpolated in the previous and next frame and/or field of the pixel to be interpolated; and outputting the generated pixels as interpolation values according to the accuracy of the estimated motion vector, the motion information, and the correlation.

Claims

1. A method of converting a frame and/or field rate of an input image signal, the method comprising:

estimating a motion vector of a frame and/or field to be interpolated and an accuracy of the motion vector using motion vectors of previous and next frames and/or fields;

determining motion information by analyzing the estimated motion vector estimated in the estimating the motion vector while calculating a correlation between a motion vector of a current block and motion vectors of neighboring blocks for the frame and/or field to be interpolated;

generating first pixels to be interpolated by performing motion compensation in the frame and/or field to be interpolated using the estimated motion vector;

generating second pixels to be interpolated using pixels neighboring a pixel to be interpolated in the frame and/or field to be interpolated and pixels neighboring the pixel to be interpolated in the previous and next frame and/or field of the pixel to be interpolated; and

outputting pixels generated in the generating the first and second pixels to be interpolated as interpolation values according to an accuracy of the estimated motion vector and the determined motion information and the calculated correlation.

2. The method of claim 1, wherein the estimating the motion vector includes estimating the motion vector using a value obtained by accumulating the differences among all pixels in the current block.

3. The method of claim 1, wherein the determining the motion information in comprises:

estimating a global motion vector in a current scene using a distribution of other estimated motion vectors of a plurality of frames and/or fields, each of the other estimated motion vectors being estimated as in the estimating the motion vector;

determining motion information on current block containing the pixels of the frame and/or field to be interpolated by comparing the motion vector of the current block with a vector value of the estimated global motion vector; and

calculating a correlation value by using a difference between the estimated motion vector of the current block and the other estimated motion vectors of neighboring blocks.

4. The method of claim 3, wherein the distribution of motion vectors is expressed by using a histogram.

5. The method of claim 3, wherein the global motion vector is estimated as a dominant value of a histogram in the horizontal direction of the motion vector, and a dominant value of a histogram in the vertical direction of the motion vector.

6. The method of claim 3, wherein the global motion vector is estimated by comparing the histogram value of the motion vector with a threshold.

7. The method of claim 3, wherein the determining the motion information comprises:

comparing the motion vector of the current block with the estimated global motion vector to obtain a difference;

if the difference is less than a predetermined threshold, setting the motion information of the current block as a global motion;

if the difference is greater than the predetermined threshold, setting the motion information of the current block as a local motion; and

if the difference is a zero vector, setting the motion information of the current block as a zero motion.

8. The method of claim 3, wherein the calculating the correlation value includes using a difference value between the motion vector of the current block and the motion vectors of neighboring blocks as a soft-switching value in motion compensation.

9. The method of claim 1, wherein the generating the first pixels to be interpolated by performing the motion compensation includes compensating the motions of the pixels of neighboring frames and/or fields using the motion vector of the current block containing the pixels of the frame and/or field to be interpolated.

10. The method of claim 1, wherein the generating the first pixels to be interpolated by performing the motion compensation includes compensating the motions of the pixels of neighboring frames and/or fields by using the motion vectors of blocks neighboring the current block containing the pixels of the frame and/or field to be interpolated.

11. The method of claim 1, wherein the generating the second pixels to be interpolated using neighboring pixels includes interpolating the pixels of the frame and/or field to be interpolated from the pixels in the neighboring frame/field corresponding to the pixels in the frame and/or field to be interpolated.

12. The method of claim 1, wherein the generating the first pixels to be interpolated includes interpolating the pixels of the frame and/or field to be interpolated using the neighboring pixels of the neighboring frames and/or fields.

13. The method of claim 1, wherein the outputting the pixels includes:

if the motion type of the current block containing the pixels of the frame and/or field to be interpolated is a global motion and the estimated accuracy estimated in the estimating the motion vector is higher than a predetermined threshold, the first pixels obtained in the generating first pixels to be interpolated are selected as the pixels to be interpolated, and

if the estimated accuracy estimated in the estimating the motion vector is lower than the predetermined threshold, the second pixels obtained in the generating the second pixels to be interpolated are selected as the pixels to be interpolated.

14. The method of claim 1, wherein the outputting the pixels includes, if the motion type of the current block containing the pixels of the frame and/or field to be interpolated is a local motion, performing interpolation from the first pixels generated in the generating the first pixels to be interpolated and the second pixels obtained in the generating the second pixels to be interpolated by soft-switching using a difference value of the motion vector of the current block and the motion vectors of the neighboring blocks.

15. The method of claim 14, wherein the interpolation by soft-switching includes assigning different weighted values to the first pixels generated in the generating the first pixels to be interpolated and the second pixels obtained in the generating the second pixels to be interpolated according to the difference value.

16. The method of claim 1, wherein the outputting the pixels includes interpolating using a first median value of the pixels of the previous frame and/or field and a second median value of the pixels of the next frame and/or field, based on the motion vector of the current block containing the pixels of the frame and/or field to be interpolated and the motion vectors of the neighboring blocks.

17. The method of claim 1, wherein the generating the first pixels to be interpolated includes generating the first pixels by using a first average value of the pixels of the previous frame and/or field and a second average value of the pixels of the next frame and/or field using the motion vector of the current block containing the pixels of the frame and/or field to be interpolated and the motion vectors of the neighboring blocks.

18. An apparatus which converts a frame and/or field rate of an image signal, comprising:

a motion estimation unit which estimates a motion vector for a frame and/or field to be interpolated using motion vectors of previous and next frames and/or fields;

a motion analyzing unit which determines motion information by analyzing the motion vector estimated in the motion estimation unit, and calculates a correlation between the motion vector of a current block and the motion vector of neighboring blocks;

a motion compensation interpolation unit which generates first pixels to be interpolated by performing motion compensation in the frame and/or field to be interpolated using the estimated motion vector estimated in the motion estimation unit;

a spatiotemporal interpolation unit which generates second pixels to be interpolated using pixels neighboring the pixels to be interpolated in the frame and/or field neighboring the frame and/or field to be interpolated; and

an adaptive motion compensation unit which outputs, as pixel values to be interpolated, the first and second pixels generated in the motion compensation interpolation unit and the spatiotemporal interpolation unit according to the motion information and correlation estimated in the motion analyzing unit.

19. The apparatus of claim 18, wherein the motion analyzing unit comprises:

a global motion estimation unit which estimates a global motion vector referring to a distribution of the estimated motion vectors;

a motion type determining unit which determines the motion information of the current block by comparing the global motion vector estimated in the global motion estimation unit and the estimated motion vector; and

a reliability calculating unit which adjusts weighted values of a motion compensation average value and a temporal average value by calculating the difference value of the motion vector of the current block and the motion vectors of neighboring blocks.

20. An apparatus which converts a frame and/or field rate of an image signal, comprising:

a motion estimation unit which estimates a motion vector for a frame and/or field to be interpolated using motion vectors of previous and next frames and/or fields;

a motion compensation interpolation unit which performs motion compensation in the frame and/or field to be interpolated using the estimated motion vector estimated in the motion estimation to generate first interpolated pixels;

a spatiotemporal interpolation unit which uses pixels neighboring the pixels to be interpolated in the frame and/or field neighboring the frame and/or field to be interpolated to generate second interpolated pixels; and

an adaptive motion compensation unit which receives the first and second interpolated pixels, uses the estimated motion vector to evaluate a type of motion for blocks in the frame/and or field to be interpolated which includes one block including the pixels to be interpolated, and selectively outputs alone or in combination the first and second interpolated pixels as pixel values for the pixels to be interpolated according to the evaluated type of motion.

21. The apparatus of claim 20, wherein, if the adaptive motion compensation unit determines the type of motion to be a first type of motion, the adaptive compensation unit adjusts a ratio of the first and second interpolated pixels to output the pixel values for the pixels to be interpolated.

22. The apparatus of claim 21, wherein if the type of motion is the first type of motion, the adaptive motion compensation unit determines a correlation between estimated motions for the one block and an adjacent one of the blocks in the frame and/or field to be interpolated, and adjusts the ratio of the first and second interpolated pixels according to a degree of the correlation.

23. The apparatus of claim 20, wherein, if the adaptive motion compensation unit determines the type of motion to be a first type of motion, the adaptive compensation unit outputs the first interpolated pixels alone to output the pixel values for the pixels to be interpolated.

24. The apparatus of claim 23, wherein if the type of motion is determined to be a second type of motion, the adaptive motion compensation unit determines a correlation between estimated motions for the one block and an adjacent one of the blocks in the frame and/or field to be interpolated, and adjusts a ratio of the first and second interpolated pixels according to a degree of the correlation to output the pixels values for the pixels to be interpolated.

25. The apparatus of claim 24, wherein if the type of motion is determined to be a third type of motion, the adaptive motion compensation unit outputs the second interpolated pixels as the pixel values for the pixels to be interpolated.

26. A computer readable medium encoded with processing instructions for implementing a method of converting a frame and/or field rate of an input image signal performed by a computer, the method comprising

estimating a motion vector of a frame and/or field to be interpolated using motion vectors of previous and next frames and/or fields;

determining motion information by analyzing the estimated motion vector;

generating first interpolated pixels by performing motion compensation in the frame and/or field to be interpolated using the estimated motion vector;

generating second interpolated pixels using pixels neighboring a pixel to be interpolated in the frame and/or field to be interpolated and pixels neighboring the pixel to be interpolated in the previous and next frame and/or field of the pixel to be interpolated; and

evaluating a type of motion for blocks in the frame/and or field to be interpolated which includes a current block including the pixels to be interpolated using the determined motion information; and

selectively outputting alone or in combination the first and second interpolated pixels as pixel values for the pixels to be interpolated according to the evaluated type of motion.

27. The computer readable medium of claim 26, wherein the evaluating the type of motion comprises:

estimating a global motion vector in a current scene using a distribution of other estimated motion vectors of a plurality of frames and/or fields, each of the other estimated motion vectors being estimated as in the estimating the motion vector;

determining the motion type on the current block containing the pixels of the frame and/or field to be interpolated by comparing the estimated motion vector of the current block with a vector value of the estimated global motion vector; and

calculating a correlation value by using a difference between the estimated motion vector of the current block and the estimated motion vectors of neighboring blocks.

28. The computer readable medium of claim 27, wherein a distribution of the estimated motion vectors is expressed by using a histogram.

29. The computer readable medium of claim 27, wherein the determining the motion type comprises:

comparing the estimated motion vector of the current block with the estimated global motion vector to obtain a difference;

if the difference is less than a predetermined threshold, setting the motion type of the current block as a global motion;

if the difference is greater than the predetermined threshold, setting the motion type of the current block as a local motion; and

if the difference is a zero vector, setting the motion type of the current block as a zero motion.

30. The computer readable medium of claim 26, wherein the selectively outputting comprises, if the evaluated type of motion is a first type of motion, adjusting a ratio of the first and second interpolated pixels to be output as the pixels to be interpolated.

31. The computer readable medium of claim 30, wherein the adjusting the ratio comprises determining a correlation between the estimated motions of the current block with the estimated motions of adjacent blocks, and adjusting the ratio according to a degree of the correlation.

32. The computer readable medium of claim 30, wherein if the evaluated type of motion is a second type of motion, the selectively outputting comprises outputting the first interpolated pixels as the pixels to be interpolated.

33. The computer readable medium of claim 32, wherein if the evaluated type of motion is a third type of motion, the selectively outputting comprises outputting the second interpolated pixels as the pixels to be interpolated.