Method and apparatus for using frame rate up conversion techniques in scalable video coding

• US 20060002465A1
• Filed: 07/01/2005
• Published: 01/05/2006
• Est. Priority Date: 07/01/2004
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Walker, Gordon Kent, Raveendran, Vijayalakshmi R.
• Granted Patent: US 8,948,262 B2
• Time in Patent Office: Days
• US Class Current: 375/240.10
• CPC Class Codes: H04N 19/109 among a plurality of temporal predictive coding modes
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/147 according to rate distortion criteria rate-distortion as a criterion for motion estimation H04N19/567
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/19 using optimisation based on Lagrange multipliers
• CPC Class Codes: H04N 19/31 in the temporal domain
• CPC Class Codes: H04N 19/567 Motion estimation based on rate distortion criteria
• CPC Class Codes: H04N 19/577 Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• CPC Class Codes: H04N 19/587 involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence

Abstract

A method for encoding a video stream includes partitioning the video stream into a main layer having a plurality of main layer frames, and an interpolated layer having a plurality of interpolated layer frames; interpolating a frame rate up conversion (FRUC) frame; and encoding the plurality of main layer frames in the interpolated layer with the assistance of the main FRUC frame. A video encoder implementing the method is also described.

Claims

1. A method for encoding a video stream, the method comprising:

partitioning the video stream into a main layer having a plurality of main layer frames, and an interpolated layer having a plurality of interpolated layer frames;

interpolating a frame rate up conversion (FRUC) frame; and

encoding the plurality of main layer frames in the main layer with the assistance of the interpolated FRUC frame.

2. The method of claim 1, wherein there is a one-to-one relationship among each main layer frame and each potential FRUC frame.

3. The method of claim 1, wherein the FRUC frame depends on a reconstructed current frame and a stored previous frame.

4. The method of claim 1, wherein said interpolating the FRUC frame comprises interpolating the FRUC frame at the same time instance of the to-be-encoded main layer frame.

5. The method of claim 1, wherein said encoding plurality of main layer frames in the main layer with the assistance of the interpolated FRUC frame comprises performing a mode-selection process of a macroblock of one interpolated layer frame in the plurality of interpolated layer frames.

6. A computer-readable medium having instructions stored thereon, the stored instructions, when executed by a processor, cause the processor to perform a method for encoding a video stream, the method comprising:

partitioning the video stream into a main layer having a plurality of main layer frames, and an interpolated layer having a plurality of interpolated layer frames;

interpolating a frame rate up conversion (FRUC) frame; and

encoding the plurality of main layer frames in the main layer with the assistance of the interpolated FRUC frame.

7. The computer-readable medium of claim 6, wherein there is a one-to-one relationship among each main layer frame and each potential FRUC frame.

8. The computer-readable medium of claim 6, wherein the FRUC frame depends on a reconstructed current frame and a stored previous frame.

9. The computer-readable medium of claim 6, wherein said interpolating the FRUC frame comprises interpolating the FRUC frame at the same time instance of the to-be-encoded main layer frame.

10. The computer-readable medium of claim 6, wherein of encoding plurality of main layer frames in the main layer with the assistance of the interpolated FRUC frame comprises performing a mode-selection process of a macroblock of one interpolated layer frame in the plurality of interpolated layer frames.

11. An apparatus for encoding a video stream comprising:

means for partitioning the video stream into a main layer having a plurality of main layer frames, and an interpolated layer having a plurality of interpolated layer frames;

means for interpolating a frame rate up conversion (FRUC) frame; and

means for encoding the plurality of main layer frames in the main layer with the assistance of the interpolated FRUC frame.

12. The apparatus of claim 11, wherein there is a one-to-one relationship among each main layer frame and each potential FRUC frame.

13. The apparatus of claim 11, wherein the FRUC frame depends on a reconstructed current frame and a stored previous frame.

14. The apparatus of claim 11, wherein the means for interpolating the FRUC frame comprises means for interpolating the FRUC frame at the same time instance of the to-be-encoded main layer frame.

15. The apparatus of claim 11, wherein the means for encoding plurality of main layer frames in the main layer with the assistance of the interpolated FRUC frame comprises means for performing a mode-selection process of a macroblock of one interpolated layer frame in the plurality of interpolated layer frames.

16. A video-encoder processor receiving a video stream, the video encoder processor comprising:

a frame rate-up-conversion (FRUC) module, the FRUC module generating a plurality of FRUC frames from the received video stream;

a rate distortion cost (RD_cost) calculation module coupled to the FRUC module, the RD_cost calculation module receiving the plurality of FRUC frames and calculating a F frame macroblock RD_cost based on a macroblock in one of the plurality of FRUC frames; and

a mode-decision module coupled to the RD_cost calculation module, the mode decision module configured to compare the F frame macroblock RD_cost based on the macroblock in one of the plurality of FRUC frames to a B frame macroblock RD_cost for a corresponding macroblock in a corresponding B frame.

17. The video-encoder processor of claim 16, wherein the mode-decision module is further configured to select a F frame encoding mode for an output video stream when the F frame macroblock RD_cost based on the macroblock in one of the plurality of FRUC frames is lower than the B frame macroblock RD_cost for the corresponding macroblock in the corresponding B frame.

18. The video-encoder processor of claim 16, further comprising a B frame encoder, the B frame encoder generating the corresponding B frame.

19. The video-encoder processor of claim 16, further comprising a sum-of-absolute difference (SAD) calculation module coupled to the FRUC module, the SAD calculation module configured to calculate a SAD value based on the macroblock in one of the plurality of FRUC frames and a SAD value for the corresponding macroblock in the corresponding B frame.

20. The video-encoder processor of claim 19, wherein the SAD calculation module is configured to calculate a difference of the SAD value based on the macroblock in one of the plurality of FRUC frames and the SAD value for the corresponding macroblock in the corresponding B frame, and to compare the difference to a predetermined threshold, the SAD calculation module generating a single bit to indicate a F frame mode if the difference is lower than the predetermined threshold.

21. A method for encoding a video stream, the method comprising:

generating a plurality of a frame rate up conversion (FRUC) frames from a received video stream;

calculating a F frame macroblock RD_cost based on a macroblock in one of the plurality of FRUC frames;

comparing the calculated F frame macroblock RD_cost to a B frame macroblock RD_cost for a corresponding macroblock in a corresponding B frame; and

selecting an encoding scheme based on the result of the comparison.

22. An apparatus for encoding a video stream, comprising:

means for generating a plurality of a frame rate up conversion (FRUC) frames from a received video stream;

means for calculating a F frame macroblock RD_cost based on a macroblock in one of the plurality of FRUC frames;

means for comparing the calculated F frame macroblock RD_cost to a B frame macroblock RD_cost for a corresponding macroblock in a corresponding B frame; and

means for selecting an encoding scheme based on the result of the comparison.

23. A computer-readable medium having instructions stored thereon, the stored instructions, when executed by a processor, cause the processor to perform a method for encoding a video stream, the method comprising:

generating a plurality of a frame rate up conversion (FRUC) frames from a received video stream;

calculating a F frame macroblock RD_cost based on a macroblock in one of the plurality of FRUC frames;

comparing the calculated F frame macroblock RD_cost to a B frame macroblock RD_cost for a corresponding macroblock in a corresponding B frame; and

selecting an encoding scheme based on the result of the comparison.