SELECTIVE VIDEO FRAME RATE UPCONVERSION

• US 20070242748A1
• Filed: 08/29/2006
• Published: 10/18/2007
• Est. Priority Date: 04/13/2006
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: El-Maleh, Khaled Helmi, Shi, Fang, Raveendran, Vijayalakshmi R., Mahadevan, Vijay, Pillai, Brijesh
• Granted Patent: US 8,582,660 B2
• Time in Patent Office: Days
• US Class Current: 375/240.140
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/587 involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression
• CPC Class Codes: H04N 7/0127 by changing the field or frame frequency of the incoming video signal, e.g. frame rate converter
• CPC Class Codes: H04N 7/0137 dependent on presence/absence of motion, e.g. of motion zones H04N7/014 takes precedence; movement detection in television signals H04N5/144

Abstract

This disclosure is directed to techniques for selective video frame rate upconversion (FRUC) in a video decoder. A video decoder selectively enables or disables FRUC based on one or more adaptive criteria. The adaptive criteria may be selected to indicate whether FRUC is likely to introduce spatial artifacts. Adaptive criteria may include a motion activity threshold, a mode decision threshold, or both. The criteria are adaptive, rather than fixed. When the criteria indicate that a frame includes excessive motion or new content, the decoder disables FRUC.

Claims

1. A method comprising:

selectively enabling and disabling video frame rate upconversion in a video decoder based on one or more criteria; and

adapting the criteria based on one or more characteristics of at least one previous video frame in proximity to a skipped frame.

2. The method of claim 1, wherein the criteria include a motion activity threshold, the method further comprising selectively disabling frame rate upconversion when a motion activity value for a current frame exceeds the motion activity threshold.

3. The method of claim 2, wherein the motion activity value includes an average motion vector value for the current frame, and wherein the motion activity threshold is derived from a histogram of motion vector values for the at least one previous frame.

4. The method of claim 3, wherein adapting the criteria includes adjusting the motion activity threshold based on the motion vector values for the at least one previous frame.

5. The method of claim 1, wherein the criteria includes a mode decision threshold, the method further comprising selectively disabling frame rate upconversion when a number of intra-coded macroblocks in a current frame exceeds the mode decision threshold.

6. The method of claim 5, further comprising determining the mode decision threshold based on a number of macroblocks in the previous frame having a number of coding bits above a threshold.

7. The method of claim 5, further comprising determining the mode decision threshold based on an approximate size of a contiguous object in at least one previous frame.

8. The method of claim 1, further comprising repeating the previous frame in place of the skipped frame when the frame rate upconversion module is disabled.

9. The method of claim 1, wherein the criteria includes a motion activity threshold and a mode decision threshold specifying a macroblock count, the method further comprising selectively disabling frame rate upconversion when a motion activity value for a current frame exceeds the motion activity threshold or when a number of intra-coded macroblocks in the current frame exceeds the mode decision threshold.

10. The method of claim 1, wherein the criteria includes a motion activity threshold and a mode decision threshold specifying a contiguous object size, the method further comprising selectively disabling frame rate upconversion when a motion activity value for a current frame exceeds the motion activity threshold or when a number of contiguous, intra-coded macroblocks in the current frame exceeds the mode decision threshold.

11. The method of claim 1, wherein the criteria includes a motion activity threshold, a macroblock count threshold, and a contiguous object size threshold, the method further comprising:

comparing the motion activity value for a current frame to the motion activity threshold to produce a first difference value;

comparing a number of intra-coded macroblocks in the current frame to the mode decision threshold to produce a second difference value;

comparing a number of contiguous, intra-coded macroblocks in the current frame to the contiguous object size threshold to produce a third difference value;

weighting the difference values;

summing the weighted difference values; and

selectively disabling frame rate upconversion when the sum exceeds a threshold value.

12. The method of claim 1, wherein the frame rate upconversion includes video frame interpolation, and the one or more criteria indicate whether the video frame interpolation is likely to produce a significant artifact.

14. The video decoder of claim 12, wherein the criteria include a motion activity threshold, and the control module selectively disables the frame rate upconversion module when a motion activity value for a current frame exceeds the motion activity threshold.

15. The video decoder of claim 14, wherein the motion activity value includes an average motion vector value for the current frame, and wherein the motion activity threshold is derived from a histogram of motion vector values for the at least one previous frame.

16. The video decoder of claim 15, wherein the control module adjusts the motion activity threshold based on the motion vector values for the at least one previous frame to adapt the criteria.

45. The video decoder of claim 1, wherein the frame rate upconversion includes video frame interpolation, and the one or more criteria indicate whether the video frame interpolation is likely to produce a significant artifact.

13. A video decoder comprising:

a frame rate upconversion module; and

a control module that selectively enables and disables the frame rate upconversion module based on one or more criteria, and adapts the criteria based on one or more characteristics of at least one previous video frame in proximity to a skipped frame.

17. The video decoder of claim 13, wherein the criteria includes a mode decision threshold, and the control module selectively disables the frame rate upconversion module when a number of intra-coded macroblocks in a current frame exceeds the mode decision threshold.

18. The video decoder of claim 17, wherein the control module determines the mode decision threshold based on a number of macroblocks in the previous frame having a number of coding bits above a threshold.

19. The video decoder of claim 17, wherein the control module determines the mode decision threshold based on an approximate size of a contiguous object in at least one previous frame.

20. The video decoder of claim 13, wherein the decoder repeats the previous frame in place of the skipped frame when the frame rate upconversion module is disabled.

21. The video decoder of claim 13, wherein the criteria includes a motion activity threshold and a mode decision threshold specifying a macroblock count, and the control module selectively disables the video frame rate upconversion module when a motion activity value for a current frame exceeds the motion activity threshold or when a number of intra-coded macroblocks in the current frame exceeds the mode decision threshold.

22. The video decoder of claim 13, wherein the criteria includes a motion activity threshold and a mode decision threshold specifying a contiguous object size, and the control module selectively disables the video frame rate upconversion module when a motion activity value for a current frame exceeds the motion activity threshold or when a number of contiguous, intra-coded macroblocks in the current frame exceeds the mode decision threshold.

23. The video decoder of claim 13, wherein the criteria includes a motion activity threshold, a macroblock count threshold, and a contiguous object size threshold, and the control module compares the motion activity value for a current frame to the motion activity threshold to produce a first difference value, compares a number of intra-coded macroblocks in the current frame to the mode decision threshold to produce a second difference value, compares a number of contiguous, intra-coded macroblocks in the current frame to the contiguous object size threshold to produce a third difference value, weights the difference values, sums the weighted difference values, and selectively disables the video frame rate upconversion module when the sum exceeds a threshold value.

24. The video decoder of claim 13, wherein frame rate upconversion module interpolates the skipped video frame, and the one or more criteria indicate whether the video frame interpolation is likely to produce a significant artifact.

25. The video decoder of claim 13, wherein the frame rate upconversion applies video frame interpolation, and the one or more criteria indicate whether the video frame interpolation is likely to produce a significant artifact.

26. A computer-readable medium comprising instructions to cause a processor to:

selectively enable and disable video frame rate upconversion in a video decoder based on one or more criteria; and

adapt the criteria based on one or more characteristics of at least one previous video frame in proximity to a skipped frame.

27. The computer-readable medium of claim 26, wherein the criteria include a motion activity threshold, and the instructions cause the processor to selectively disable video frame rate upconversion when a motion activity value for a current frame exceeds the motion activity threshold.

28. The computer-readable medium of claim 27, wherein the motion activity value includes an average motion vector value for the current frame, and wherein the motion activity threshold is derived from a histogram of motion vector values for the at least one previous frame.

29. The computer-readable medium of claim 28, wherein the instructions cause the processor to adjust the motion activity threshold based on the motion vector values for the at least one previous frame.

30. The computer-readable medium of claim 26, wherein the criteria includes a mode decision threshold, and wherein the instructions cause the processor to selectively disable video frame rate upconversion when a number of intra-coded macroblocks in a current frame exceeds the mode decision threshold.

31. The computer-readable medium of claim 30, wherein the instructions cause the processor to determine the mode decision threshold based on a number of macroblocks in the previous frame having a number of coding bits above a threshold.

32. The computer-readable medium of claim 30, wherein the instructions cause the processor to determine the mode decision threshold based on an approximate size of a contiguous object in at least one previous frame.

33. The computer-readable medium of claim 26, wherein the instructions cause the processor to repeat the previous frame in place of the skipped frame when frame rate upconversion is disabled.

34. The computer-readable medium of claim 26, wherein the criteria includes a motion activity threshold and a mode decision threshold specifying a macroblock count, wherein the instructions cause the processor to selectively disable video frame rate upconversion when a motion activity value for a current frame exceeds the motion activity threshold or when a number of intra-coded macroblocks in the current frame exceeds the mode decision threshold.

35. The computer-readable medium of claim 26, wherein the criteria includes a motion activity threshold and a mode decision threshold specifying a contiguous object size, wherein the instructions cause the processor to selectively disable video frame rate upconversion when a motion activity value for a current frame exceeds the motion activity threshold or when a number of contiguous, intra-coded macroblocks in the current frame exceeds the contiguous object size threshold.

36. The computer-readable medium of claim 26, wherein the criteria includes a motion activity threshold, a macroblock count threshold, and a contiguous object size threshold, wherein the instructions cause the processor to:

compare the motion activity value for a current frame to the motion activity threshold to produce a first difference value;

compare a number of intra-coded macroblocks in the current frame to the mode decision threshold to produce a second difference value;

compare a number of contiguous, intra-coded macroblocks in the current frame to the contiguous object size threshold to produce a third difference value;

weight the difference values;

sum the weighted difference values; and

selectively disable video frame rate upconversion when the sum exceeds a threshold value.

37. The computer-readable medium of claim 26, wherein frame rate upconversion includes video frame interpolation, and the one or more criteria indicate whether the video frame interpolation is likely to produce a significant artifact.

41. The video decoder of claim 36, wherein the means for adapting the criteria includes means for adjusting the motion activity threshold based on the motion vector values for the at least one previous frame.

42. The video decoder of claim 36, wherein the criteria includes a mode decision threshold, the decoder further comprising means for selectively disabling video frame rate upconversion when a number of intra-coded macroblocks in a current frame exceeds the mode decision threshold.

43. The video decoder of claim 42, further comprising means for determining the mode decision threshold based on a number of macroblocks in the previous frame having a number of coding bits above a threshold.

44. The video decoder of claim 42, further comprising means for determining the mode decision threshold based on an approximate size of a contiguous object in at least one previous frame.

38. A video decoder comprising:

means for selectively enabling and disabling video frame rate upconversion in a video decoder based on one or more criteria; and

means for adapting the criteria based on one or more characteristics of at least one previous video frame in proximity to a skipped frame.

39. The video decoder of claim 38, wherein the criteria include a motion activity threshold, the decoder further comprising means for selectively disabling video frame rate upconversion when a motion activity value for a current frame exceeds the motion activity threshold.

40. The video decoder of claim 39, wherein the motion activity value includes an average motion vector value for the current frame, and wherein the motion activity threshold is derived from a histogram of motion vector values for the at least one previous frame.