SYSTEM AND METHOD FOR REDUCING VISIBLE HALO IN DIGITAL VIDEO WITH DUAL MOTION ESTIMATION

• US 20100045856A1
• Filed: 08/20/2008
• Published: 02/25/2010
• Est. Priority Date: 08/20/2008
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Chappalli, Mahesh B., Kim, Yeong Taeg
• Granted Patent: US 8,259,225 B2
• Time in Patent Office: Days
• US Class Current: 348/441
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51
• CPC Class Codes: H04N 7/014 involving the use of motion vectors motion estimation and compensation in video coding H04N19/51

Abstract

Embodiments include systems and methods for interpolating frames. In one embodiment, a system and method provides reduced halo and consequently a higher degree of accuracy/fidelity. In one embodiment, a method of video frame interpolation includes receiving forward and backward motion estimation data based on a first video frame and a second, subsequent, video frame. Each block in a plurality of blocks of pixels in an intermediate frame between the first and second frames is classified according to measures of covering and uncovering associated with the forward and backward motion estimation data. A block to be constructed is estimated based on the classification results of proximal ones of the plurality of blocks to the block to be constructed.

Claims

1. A method of video frame interpolation, comprising:

receiving forward and backward motion estimation data based on a first video frame and a second video frame, wherein the second video frame is subsequent to the first video frame;

classifying each block in a plurality of blocks of pixels in an intermediate frame between the first and second frames according to measures of covering and uncovering associated with the forward and backward motion estimation data;

selecting from the forward and backward motion estimation data based on the classification results; and

estimating a block to be constructed based on the selected motion estimation data.

2. The method of claim 1, wherein selecting from the forward and backward motion estimation data is based on classification results of ones of the plurality of blocks proximal to the block to be constructed.

3. The method of claim 2, wherein the proximal ones of the plurality of blocks comprises a group of blocks surrounding the block to be estimated.

4. The method of claim 2, wherein the selecting comprises:

determining respective counts of the proximal blocks classified as covering and uncovering;

comparing the counts to threshold levels; and

selecting from the forward and backward motion estimation data based on the comparison.

5. The method of claim 4, wherein the estimating is further based on a corresponding degree of occlusion.

6. The method of claim 1, wherein the classifying comprises:

translating blocks from the first video frame to the intermediate frame using the forward motion estimation data.

translating blocks from the second video frame to the intermediate frame using the backward motion estimation data.

calculating first measures of overlap area of the blocks translated using the forward motion estimation data with each of the blocks in the intermediate frame;

calculating second measures of overlap area of the blocks translated using the backward motion estimation data with each of the blocks in the intermediate frame;

classifying each of the blocks based on the first measures of overlap area; and

classifying each of the blocks based on the second measures of overlap area.

7. The method of claim 1, further comprising classifying the blocks of the intermediate frame to be one of normal, covering and uncovering with respect to each of the forward and backward motion estimation data.

8. The method of claim 1, wherein the classifying further comprises:

ranking the blocks in order of at least one of the first and second measures of overlap area; and

classifying the block of pixels to be constructed according to the degree of occlusion based at least in part on the ranking.

9. The method of claim 8, further comprising classifying the block of pixels to be constructed as normal if the largest ranked overlap area measure indicates that the overlap area is greater than a first threshold.

10. The method of claim 9, further comprising classifying the block of pixels to be constructed as normal if the two largest ranked overlap area measures correspond to blocks translating via the motion data from block positions separated by a distance that is less than a second threshold.

11. The method of claim 10, further comprising classifying the block of pixels to be constructed as uncovering if the two largest ranked overlap area indicators correspond to blocks translating via the motion data from block positions separated by a distance that is greater than a third threshold.

12. The method of claim 11, further comprising classifying the block of pixels to be constructed as covering if none of the measures of overlap area exceeds a fourth threshold.

13. The method of claim 1, further comprising generating the forward and backward motion estimation data based on the first and second frames.

14. The method of claim 13, wherein the generating comprises determining matching blocks in the first and second frame based on residual error measurements and determining a corresponding motion vector for each block.

15. The method of claim 1, further comprising outputting the estimated block of the intermediate frame for display.

16. A system for video frame interpolation, comprising:

a memory configured to store forward and backward motion estimation data based on a first video frame and a second video frame, wherein the second video frame is subsequent to the first video frame; and

at least one processor configured to;

classify each block in a plurality of blocks of pixels in an intermediate frame between the first and second frames according to measures of covering and uncovering associated with the forward and backward motion estimation data;

select from the forward and backward motion estimation data based on the classification results; and

estimate a block to be constructed based on the selected motion estimation data.

17. The system of claim 16, wherein the processor is configured to select from the forward and backward motion estimation data based on classification results of ones of the plurality of blocks proximal to the block to be constructed.

18. The system of claim 17, wherein the proximal ones of the plurality of blocks comprises a group of blocks surrounding the block to be estimated.

19. The system of claim 17, wherein the processor is configured to:

determine respective counts of the proximal blocks classified as covering and uncovering;

compare the counts to threshold levels;

select from the forward and backward motion estimation data based on the comparison.

20. The system of claim 16, wherein the processor is configured to:

translate blocks from the first frame to the intermediate frame using the forward motion estimation data.

translate blocks from the second frame to the intermediate frame using the backward motion estimation data.

calculate first measures of overlap area of the blocks translated using the forward motion estimation data with each of the blocks in the intermediate frame;

calculate second measures of overlap area of the blocks translated using the backward motion estimation data with each of the blocks in the intermediate frame;

classify each of the blocks based on the first measures of overlap area; and

classify each of the blocks based on the second measures of overlap area.

21. The system of claim 16, wherein the processor is configured to classify the blocks of the intermediate frame as one of normal, covering and uncovering with respect to each of the forward and backward motion estimation data.

22. The system of claim 16, wherein the processor is further configured to generate the forward and backward motion estimation data based on the first and second frames.

23. The system of claim 22, wherein the processor is configured to generate the motion estimation data by determining matching blocks in the first and second frame based on residual error measurements and determining a corresponding motion vector for each block.

24. The system of claim 16, wherein the system comprises at least one of a television, a set-top box, a digital-video recorder, a recorded media player.

25. A system for video frame interpolation, comprising:

means for storing forward and backward motion estimation data based on a first video frame and a second video frame, wherein the second video frame is subsequent to the first video frame; and

means for processing configured to;

classify each block in a plurality of blocks of pixels in an intermediate frame between the first and second frames according to measures of covering and uncovering associated with the forward and backward motion estimation data; and

select from the forward and backward motion estimation data based on the classification results; and

estimate a block to be constructed based on selected motion estimation data.

26. The system of claim 25, wherein the processing means is configured to select from the forward and backward motion estimation data based on classification results of ones of the plurality of blocks proximal to the block to be constructed.

27. A computer-program product for interpolating video frames, the product comprising:

a computer-readable medium having stored thereon codes executable by at least one processor to;

receive forward and backward motion estimation data based on a first video frame and a second video frame, wherein the second video frame is subsequent to the first video frame; and

classify each block in a plurality of blocks of pixels in an intermediate frame between the first and second frames according to measures of covering and uncovering associated with the forward and backward motion estimation data;

select from the forward and backward motion estimation data based on the classification results; and

estimate a block to be constructed based on selected motion estimation data.

28. The product of claim 27, further comprising codes executable by the processor to select from the forward and backward motion estimation data based on classification results of ones of the plurality of blocks proximal to the block to be constructed.

29. The product of claim 28, wherein the proximal ones of the plurality of blocks comprises a group of blocks surrounding the block to be estimated.

30. The product of claim 28, further comprising codes executable by the processor to:

determine respective counts of the proximal blocks classified as covering and uncovering;

compare the counts to threshold levels; and

select from the forward and backward motion estimation data based on the comparison.

31. The product of claim 27, further comprising codes executable by the processor to:

translate blocks from the first frame to the intermediate frame using the forward motion estimation data.

translate blocks from the second frame to the intermediate frame using the backward motion estimation data.

calculate first measures of overlap area of the blocks translated using the forward motion estimation data with each of the blocks in the intermediate frame;

calculate second measures of overlap area of the blocks translated using the backward motion estimation data with each of the blocks in the intermediate frame;

classify each of the blocks based on the first measures of overlap area; and

classify each of the blocks based on the second measures of overlap area.