System and method for introducing virtual zero motion vector candidates in areas of a video sequence involving overlays

• US 8,630,346 B2
• Filed: 02/20/2007
• Issued: 01/14/2014
• Est. Priority Date: 02/20/2007
• Status: Expired due to Fees

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Chappalli, Mahesh, Kim, Yeong-Taeg
• Primary Examiner(s): Rahman, Mohammad L
• Application Number:
  US11/709,047
• Publication Number:
  US 20080198931A1
• Time in Patent Office: 2,520 Days
• Field of Search: 375/240.16, 375/240.14
• US Class Current: 375/240.14
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/577 Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

Systems and methods for efficiently encoding and/or reformatting video data including transparent overlay portions are disclosed. In one embodiment, the method includes using two prediction regions for predicting the portion of the video including the transparent overlay. In one embodiment, a first of the two prediction regions is determined based on motion compensated prediction in reference to another video frame and a second of the two prediction regions is a collocated portion of video in another frame as referenced by a virtual zero motion vector. A mixing weight factor to be used for combining the two predictions is determined. In one embodiment, the mixing weight factor is determined based on the relative values of two error metrics, a first error metric related to the motion compensated prediction and a second error metric related to the collocated prediction of the virtual zero motion vector.

Claims

1. A method of compressing video data comprising:

determining a first prediction of one or more pixel values in a first video frame based on one or more non-collocated pixel values in a second video frame;

determining a second prediction of the one or more pixel values in the first video frame based on one or more collocated pixel values of a stationary transparent overlay in the second video frame;

determining a mixing weight factor to be used to combine the first and second predictions during reconstruction of the one or more pixel values;

calculating a first error metric between the first prediction pixel values and the pixel values in the first video frame; and

calculating a second error metric between the second prediction pixel values and the pixel values in the first video frame; and

wherein determining the mixing weight factor is based on the calculated first and second error metrics.

2. The method of claim 1, wherein determining the mixing weight factor results in a larger weight being applied to the first prediction than the second prediction when the first error metric is less than the second error metric.

3. The method of claim 1, wherein the determined mixing weight factor is in a range greater than or equal to zero and less than or equal to one, where a value of one results in only the first prediction contributing to the combined prediction and a value of zero results in only the second prediction contributing to the combined prediction.

4. A method of video processing comprising:

receiving a first frame of video data;

receiving a second frame of video data, wherein a first set of one or more pixel values of the second frame is predicted, using motion estimation, from a second set of one or more non-collocated pixel values of the first frame of video data;

determining a first prediction of a third set of one or more identified pixels of a third frame based on the motion estimation between the second frame and the first frame;

determining a second prediction of the third set of one or more pixels of the third frame based on collocated pixels of a transparent overlay of the first frame, the second frame, or a combination thereof;

calculating values of the one or more identified pixels of the third set by combining the first prediction and the second prediction;

calculating a first error metric between the pixel values of the first set of one or more pixel values of the second frame and the pixel values of the second set of non-collocated pixel values in the first frame;

calculating a second error metric between the collocated pixels of the transparent overlay of the first frame and the second frame; and

determining a mixing weight factor based on the calculated first and second error metrics;

wherein calculating values of the one or more identified pixels of the third set includes combining the first prediction and the second prediction using the determined mixing weight factor.

5. The method of claim 4, wherein the motion estimation includes a first motion vector indicating the relative positions of the first set of pixels of the second frame and the non-collocated second set of pixels of the first frame, the method further comprising determining a second motion vector between the third frame and the first frame, the second frame, or a combination thereof, the second motion vector being determined based on the first motion vector and temporal distances between the first, second and third frames and wherein the second motion vector identifies the third set of one or more pixels in the third frame.

6. The method of claim 5, wherein determining the first prediction of the third set of one or more pixels includes combining the non-collocated second set of one or more pixel values in the first frame and the first set of one or more pixel values in the second frame.

7. The method of claim 4 wherein the determined mixing weight factor results in the calculated values of the one or more identified pixels of the third set being substantially equal to the first prediction when the first error metric is less than a first threshold.

8. The method of claim 7 wherein the determined mixing weight factor results in the calculated values of the one or more identified pixels of the third set being substantially equal to the first prediction when the second error metric is greater than a second threshold.

9. The method of claim 8, wherein the determined mixing weight factor results in the calculated values of the one or more identified pixels of the third set being a varying combination of the first prediction and the second prediction when the first error metric is greater than the first threshold and the second error metric is less than the second threshold.

10. The method of claim 5, wherein the first frame and the second frame of video data are uncompressed video data, the method further comprising calculating the first motion vector using motion estimation.

11. A system for compressing video data comprising:

a motion estimation subsystem configured to determine a first prediction of one or more pixel values in a first video frame based on one or more non-collocated pixel values in a second video frame, and to determine a second prediction of the one or more pixel values in the first video frame based on one or more collocated pixel values of a stationary transparent overlay in the second video frame;

a mixing factor computation subsystem configured to determine a mixing weight factor to be used to combine the first and second predictions during reconstruction of the one or more pixel values;

an error metric calculation subsystem configured to calculate a first error metric between the first prediction pixel values and the pixel values in the first video frame, and to calculate a second error metric between the second prediction pixel values and the pixel values in the first video frame, wherein the mixing factor computation subsystem is configured to determine the mixing weight factor based on the calculated first and second error metrics.

12. The system of claim 11, wherein the determined mixing weight factor results in a larger weight being applied to the first prediction than the second prediction when the first error metric is less than the second error metric.

13. The system of claim 11, wherein the determined mixing weight factor is in a range greater than or equal to zero and less than or equal to one, where a value of one results in only the first prediction contributing to the combined prediction and a value of zero results in only the second prediction contributing to the combined prediction.

14. A system for video processing comprising:

a receiver configured to receive a first frame of video data, and to receive a second frame of video data, wherein a first set of one or more pixel values of the second frame is predicted, using motion estimation, from a second set of one or more non-collocated pixel values of the first frame of video data;

a reformatting subsystem configured to determine a first prediction of a third set of one or more identified pixels of a third frame based on the motion estimation between the second frame and the first frame, to determine a second prediction of the third set of one or more pixels of the third frame based on collocated pixels a transparent overlay of the first frame, the second frame, or a combination thereof and to calculate values of the one or more identified pixels of the third set by combining the first prediction and the second prediction;

an error metric calculation subsystem configured to calculate a first error metric between the pixel values of the first set of one or more pixel values of the second frame and the pixel values of the second set of non-collocated pixel values in the first frame, and to calculate a second error metric between the collocated pixels of the transparent overlay of the first frame and the second frame; and

wherein the reformatting subsystem is further configured to determine a mixing weight factor based on the calculated first and second error metrics, and to calculate values of the one or more identified pixels of the third set by combining the first prediction and the second prediction using the determined mixing weight factor.

15. The system of claim 14, wherein the motion estimation includes a first motion vector indicating the relative positions of the first set of pixels of the second frame and the non-collocated second set of pixels of the first frame, and the reformatting subsystem is further configured to determine a second motion vector between the third frame and the first frame, the second frame, or a combination thereof, the second motion vector being determined based on the first motion vector and temporal distances between the first, second and third frames and wherein the second motion vector identifies the third set of one or more pixels in the third frame.

16. The system of claim 14, wherein the reformatting subsystem is further configured to determine the first prediction of the third set of one or more pixels by combining the non-collocated second set of one or more pixel values in the first frame and the first set of one or more pixel values in the second frame.

17. The system of claim 14 wherein the determined mixing weight factor results in the calculated values of the one or more identified pixels of the third set being substantially equal to the first prediction when the first error metric is less than a first threshold.

18. The system of claim 17 wherein the determined mixing weight factor results in the calculated values of the one or more identified pixels of the third set being substantially equal to the first prediction when the second error metric is greater than a second threshold.

19. The system of claim 18, wherein the determined mixing weight factor results in the calculated values of the one or more identified pixels of the third set being a varying combination of the first prediction and the second prediction when the first error metric is greater than the first threshold and the second error metric is less than the second threshold.

20. The system of claim 15, wherein the first frame and the second frame of video data are uncompressed video data, the device further comprising a motion compensation subsystem configured to calculate the first motion vector using the motion estimation.