System and method for frame interpolation for a compressed video bitstream
First Claim
1. A method for frame interpolation for a bitstream encoding a first source image and a second source image which is encoded subsequent to the first source image wherein a device receives the bitstream, the method comprising the steps of:
- decoding the first source image and the second source image from the bitstream;
performing a first motion estimation which uses the first source image and the second source image to create a first motion field wherein the first source image is a reference grid for the first motion estimation;
performing a first motion compensation which uses the first motion field to create a forward candidate interpolation picture;
performing a second motion estimation which uses the first source image and the second source image to create a second motion field which is a different motion field than the first motion field wherein the second source image is a reference grid for the second motion estimation;
performing a second motion compensation which uses the second motion field to create a backward candidate interpolation picture;
performing a third motion estimation which uses the first source image and the second source image to create a third motion field which is a different motion field than the first motion field and the second motion field wherein a bidirectional candidate interpolation picture is a reference grid for the third motion estimation;
performing a third motion compensation which uses the third motion field to create the bidirectional candidate interpolation picture;
determining an estimated visual quality of a final interpolated picture formed by a combination of the forward candidate interpolation picture, the backward candidate interpolation picture and the bidirectional candidate interpolation picture; and
displaying the final interpolated picture if the estimated visual quality exceeds a threshold.
0 Assignments
0 Petitions
Accused Products
Abstract
A system and a method perform frame interpolation for a compressed video bitstream. The system and the method may combine candidate pictures to generate an interpolated video picture inserted between two original video pictures. The system and the method may generate the candidate pictures from different motion fields. The candidate pictures may be generated partially or wholly from motion vectors extracted from the compressed video bitstream. The system and the method may reduce computation required for interpolation of video frames without a negative impact on visual quality of a video sequence.
-
Citations
21 Claims
-
1. A method for frame interpolation for a bitstream encoding a first source image and a second source image which is encoded subsequent to the first source image wherein a device receives the bitstream, the method comprising the steps of:
-
decoding the first source image and the second source image from the bitstream; performing a first motion estimation which uses the first source image and the second source image to create a first motion field wherein the first source image is a reference grid for the first motion estimation; performing a first motion compensation which uses the first motion field to create a forward candidate interpolation picture; performing a second motion estimation which uses the first source image and the second source image to create a second motion field which is a different motion field than the first motion field wherein the second source image is a reference grid for the second motion estimation; performing a second motion compensation which uses the second motion field to create a backward candidate interpolation picture; performing a third motion estimation which uses the first source image and the second source image to create a third motion field which is a different motion field than the first motion field and the second motion field wherein a bidirectional candidate interpolation picture is a reference grid for the third motion estimation; performing a third motion compensation which uses the third motion field to create the bidirectional candidate interpolation picture; determining an estimated visual quality of a final interpolated picture formed by a combination of the forward candidate interpolation picture, the backward candidate interpolation picture and the bidirectional candidate interpolation picture; and displaying the final interpolated picture if the estimated visual quality exceeds a threshold. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13)
-
-
14. A method for frame interpolation for a bitstream encoding a first source image and a second source image subsequent to the first source image wherein the first source image and the second source image are formed by macroblocks and further wherein motion vectors are encoded by the bitstream wherein each of the macroblocks is associated with at least one of the motion vectors and further wherein the bitstream encodes block mode information wherein a device receives the bitstream, the method comprising the steps of:
-
determining reliable motion vectors of the motion vectors encoded by the bitstream wherein the motion vectors and the block mode information are used to determine the reliable motion vectors; performing a first motion estimation which uses the first source image and the second source image to create a first motion field wherein the first source image is a reference grid for the first motion estimation and further wherein the first motion estimation uses the reliable motion vectors; performing a first motion compensation which uses the first motion field to create a forward candidate interpolation picture; performing a second motion estimation which uses the first source image and the second source image to create a second motion field which is a different motion field than the first motion field wherein the second source image is a reference grid for the second motion estimation and further wherein the second motion estimation uses the reliable motion vectors; performing a second motion compensation which uses the second motion field to create a backward candidate interpolation picture; performing a third motion estimation which uses the first source image and the second source image to create a third motion field which is a different motion field than the first motion field and the second motion field wherein a bidirectional candidate interpolation picture is a reference grid for the third motion estimation; performing a third motion compensation which uses the third motion field to create the bidirectional candidate interpolation picture; and displaying the first source image, the second source image and an interim image wherein the interim image is displayed after the first source image and before the second source image. - View Dependent Claims (15, 16, 17)
-
-
18. A system for frame interpolation for a bitstream encoding a first source image and a second source image, the system comprising:
-
a mobile device which receives the bitstream; a processor connected to the mobile device which decodes the first source image and the second source image from the bitstream; and an application executed by the mobile device which directs the processor to use the first source image and the second source image to generate at least three candidate interpolation pictures wherein the processor applies a sum of absolute difference operation to the at least three candidate interpolation pictures to estimate a number of blocks which are likely to have motion artifacts in a final interpolated picture formed by the at least three candidate interpolation pictures. - View Dependent Claims (19, 20, 21)
-
Specification