Method and apparatus for minimizing number of reference pictures used for inter-coding

• US 8,073,048 B2
• Filed: 08/24/2004
• Issued: 12/06/2011
• Est. Priority Date: 08/26/2003
• Status: Active Grant

• Current Assignee: InterDigital Madison Patent Holdings (InterDigital, Inc.)
• Original Assignee: Thomson Licensing (Vantiva SA)
• Inventors: Tourapis, Alexandros, Boyce, Jill MacDonald
• Primary Examiner(s): Bayard, Emmanuel
• Application Number:
  US10/569,168
• Publication Number:
  US 20070019724A1
• Time in Patent Office: 2,660 Days
• Field of Search: 375/240.12, 375/240.15, 375/240.16, 375/240.01, 348/700
• US Class Current: 375/240.01
• CPC Class Codes: H04N 19/105 Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
• CPC Class Codes: H04N 19/147 according to rate distortion criteria rate-distortion as a criterion for motion estimation H04N19/567
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/573 Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

A method for minimizing and optimizing a list of active reference pictures to be used for (forward or backward) inter coding of a current picture, or of a pixel region (e.g., video object) within the current picture. Setting the number of active reference pictures in a list equal to one based upon one or more decisions related to the pixel data and movement between the current picture and one or more reference pictures. Otherwise, minimizing the number of active reference pictures in the list based upon one or more other decisions related to the pixel data and movement between the current picture and one or more reference pictures. Minimizing the number of active reference pictures in a second reference picture list based upon the utilization of reference pictures in the first reference picture list. A video encoder processes data representing a two-dimensional video image which has been produced by a conventional commercially available video camera. The video encoder is adapted to determine, select, and to minimize the number of active reference pictures to be used to inter code a current picture. The bit stream transmitted by the encoder can be decoded by decoders of the related art.

Claims

1. In a video encoder, a method of inter coding a pixel region of a current picture in a video sequence of pictures, the sequence including a plurality of references listed in at least one reference list, the method comprising:

the step of selecting the first reference listed in a reference list to be used as the only reference to be used to encode the pixel region of the current picture, wherein the step of selecting the first listed reference comprises;

a first substep of computing the ratio MBR of the blocks in the pixel region of the current picture having a sum of absolute pixel differences value MBSAD relative to the first listed reference being below a second threshold value T₂ ;

wherein if ratio MBR is equal to or greater than a first threshold ratio TR_R then only the first listed reference is used to encode the pixel region.

2. In a video encoder, a method of inter coding a current picture in a video sequence of pictures, the sequence including a plurality of references listed in a reference list, the method comprising:

the step of selecting the first reference listed in the reference list to be used as the only reference to be used to encode the current picture, wherein the step of selecting the first listed reference comprises a substep of computing the ratio RK1 of blocks encoded in the first listed reference picture encoded with reference indices equal to zero;

wherein if ratio RK1 exceeds a predetermined a threshold ratio T_RK1, then only the first listed reference is used to encode the current picture.

3. The method of claim 2, wherein the substep of computing the ratio RK1 is performed only if the ratio MBR, of the blocks in the pixel region of the current picture having a block SAD value MBSAD relative to the first listed reference being below a fourth threshold value T₄, is equal to or greater than a second predetermined threshold ratio T_RH.

4. The method of claim 2, wherein the substep of computing the ratio RK1 is performed only of sum of absolute pixel differences within the corresponding pixel regions of the current piture and of first listed reference is less than a third predetermined threshold T₃.

5. In a video encoder, a method of inter coding a current picture in a video sequence of pictures, the sequence including a plurality of references listed in a reference list, the method comprising:

the step of selecting the first reference listed in the reference list to be used as the only reference to be used to encode the current picture, wherein the step of selecting the first listed reference comprises a substep of computing the ratio RK2 of blocks of the first reference picture having been encoded to have motion vector magnitude components MVx and MVy being equal to or less than predetermined threshold magnitudes TX and TY respectively, wherein if the ratio RK2 is greater than a predetermined threshold percentage T_RK2, then only the first listed reference shall be used to encode the current picture.

6. The method of claim 5, wherein threshold magnitude TX equals threshold magnitude TY.

7. The method of claim 5, wherein the substep of computing the ratio RK2 is performed only if the sum of absolute pixel differences within the corresponding pixel regions of the current picture and of first listed reference is not less than a first predetermined threshold T₁.

8. In a video encoder, a method of inter coding a current picture in a video sequence of pictures using at least one of a plurality of reference pictures listed in a list of active references, the method comprising:

the step of removing at least one of the listed references from the list of active references based upon a comparison of each of the removed references to the current picture.

9. The method of claim 8, further comprising the step of reducing num_ref_idx_—

1N_active_minus1 accordingly, wherein N represents the number of the list of active references.

10. The method of claim 8 wherein the comparison is a direct comparison.

11. The method of claim 10 wherein the step of removing listed references comprises a substep of measuring distortion by calculating the sum of absolute pixel differences between the current picture and each removed reference.

12. The method of claim 10 wherein the step of removing listed references comprises performing, for each reference removed from the list, the substep of computing the number of blocks having a sum of absolute difference values larger than a predetermined threshold T₆.

13. The method of claim 12 further comprising dynamically reducing the value of threshold T₆ for each reference picture that is not used for encoding another reference picture that is closer in time to the current picture.

14. The method of claim 8 wherein the comparison is an indirect comparison.

15. The method of claim 14 wherein the step of removing at least one of the listed references from the list of active references based upon a comparison of each of the removed references to the current picture includes removing a second reference picture from a list of active references if the second reference picture has high distortion compared to a first reference picture, and the first reference picture has low distortion compared to the current picture.

16. In a video encoder, a method of inter coding a current picture using at least one of a plurality of reference pictures listed in a list of active references, the method comprising:

the step of reordering the listed references so that reference pictures having smaller distortion relative to the current picture are listed with higher priority in the list of active references.

17. In a video encoder, a method of inter coding a video sequence of pictures, the method comprising:

performing a first coding step of encoding the current picture using all reference pictures listed in a reference picture list;

a step of selecting and removing one or more pictures from the reference list to create a new reference list; and

then performing a second coding step of re-coding the current picture using only the pictures listed in the new reference list.

18. The method of claim 17, wherein at least one of the selected pictures removed from the reference list is removed because it was not used as a reference in the first coding step for encoding any of the blocks of the current picture.

19. The method of claim 17, wherein at least one of the selected pictures removed from the reference list is removed because in the first step its reference index was used to encode less than a predetermined threshold number of blocks of the current picture.

20. The method of encoding of claim 17, further comprising removing from the active reference list all the pictures not used as reference pictures during the first pass.

21. The method of encoding of claim 17, further comprising computing the number of blocks in the current picture that used a particular indexed reference picture as coding reference in the first pass;

and if that number of blocks is less than a predetermined threshold number of blocks, then that particular indexed reference picture is not used to encode the current picture in the second pass.

22. In a video encoder, a method of encoding a video sequence of images, the method comprising:

the step of inter coding the current picture K times, using K permutations of M reference pictures, where K is equal to;

23. The method of encoding of claim 22, wherein the step of selecting one encoded current picture includes computing and comparing the distortion of each of the K encoded current pictures relative to the current picture to be encoded.

24. The method of encoding of claim 22, wherein the step of selecting one encoded current picture includes computing and comparing the bitrate of each of the K encoded current pictures.

25. The method of encoding of claim 22, wherein the step of selecting one encoded current picture includes computing and comparing the bitrate and distortion of each of the K encoded current pictures, wherein distortion is weighted against bitrate using a Lagrange multiplier.

26. In a video encoder, a method of inter coding a video sequence of pictures, the method comprising:

the step of performing motion estimation coding of a current picture using each of a plurality of permutations of available references and selecting the permutation of available references that minimizes a predetermined condition.

27. The method claim 26, wherein predetermined condition is selected from:

bitrate, distortion, or weighted combination of bitrate and distortion.

28. The method of claim 26, wherein the selected permutation of available references consists of one single reference, and wherein the predetermined condition minimized is bitrate.

29. The method of claim 26, wherein if the selected permutation of available references consists of one single reference, then further performing the step of recoding the current picture using only the single reference for motion estimation.

30. An encoder for encoding a video sequence of images, the encoder comprising a reference picture buffer, wherein the encoder is adapted to inter code a current picture in the sequence of pictures using at least one of a plurality of reference pictures stored in the reference picture buffer and listed in a reference list;

wherein the encoder is further adapted to dynamically select the first listed reference to be used as the only reference to be used to inter code the current picture based upon predetermined criteria, wherein the encoder is further adapted to dynamically select and remove one or more pictures from the list of references stored in the reference picture buffer, to create a new reference list; and

to inter code the current picture using only the pictures listed in the new reference list.

31. An encoder for encoding a sequence of images, the encoder comprising a reference buffer, wherein the encoder is adapted to inter code a current picture using at least one of a plurality of reference pictures listed the reference buffer, and adapted to dynamically reorder a list of references stored in the reference buffer so that reference pictures having smaller distortion relative to the current picture are listed with higher priority in the list of active references.

32. The encoder of claim 31, wherein the encoder is further adapted to inter code the current picture using only the first listed reference as the only reference to be used to inter code the current picture if encoding using only the first listed reference meets satisfies a predetermined criteria.

33. The encoder of claim 32, wherein the predetermined criteria includes minimizing the bitrate of the current picture to be encoded.

34. The encoder of claim 32, wherein the predetermined criteria includes minimizing the distortion of the encoded current picture.