Method, device and program for encoding and decoding a sequence of images using area-by-area loop filtering

• US 10,085,042 B2
• Filed: 11/07/2012
• Issued: 09/25/2018
• Est. Priority Date: 11/07/2011
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Laroche, Guillaume, Gisquet, Christophe, Francois, Edouard, Onno, Patrice
• Primary Examiner(s): Hallenbeck-Huber, Jeremiah C
• Assistant Examiner(s): Hodges, Susan E
• Application Number:
  US14/356,831
• Publication Number:
  US 20140301489A1
• Time in Patent Office: 2,148 Days
• Field of Search: 375240
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/13 Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/139 Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence
• CPC Class Codes: H04N 19/82 involving filtering within a prediction loop
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness

Abstract

A method of encoding a sequence of images comprises performing area-by-area loop filtering on at least one image of the sequence and disabling such area-by-area loop filtering on at least one other image of the sequence. The images of the sequence are associated with respective levels from a plurality of levels, the levels corresponding to different respective rate-distortion balances or to different image qualities, and the images on which the area-by-area loop filtering is disabled are determined in dependence upon the levels.

Claims

1. A method of encoding images, comprising:

receiving a sequence of images;

encoding the received sequence of images to produce a bitstream; and

processing at least one encoded image to generate a reconstructed image usable as a reference image for the encoding of another image, the processing comprising loop filtering;

wherein the loop filtering comprises;

determining which images of said sequence of images are enabled images on which area-by-area loop filtering is to be enabled, permitting different image areas within the same enabled image to have different loop filtering selections, and which images of said sequence of images are disabled images on which area-by-area loop filtering is to be disabled, different image areas within the same disabled image not being permitted to have different loop filtering selections;

performing area-by-area loop filtering on each said enabled image of the sequence and disabling such area-by-area loop filtering on each said disabled image of the sequence;

producing a measure of the area-by-area loop filtering activity in one or more images, said measure being dependent on the loop filtering selections for the image areas of said one or more images in the case of enabling area-by-area loop filtering thereon; and

determining that at least one image of said sequence is an enabled image or a disabled image in dependence upon the measure,

wherein producing the measure comprises determining a number of image areas within an image, including consecutive and non-consecutive image areas, in which use of loop filtering is selected.

2. A method according to claim 1, comprising:

associating with the images of the sequence respective levels from a plurality of levels;

producing a measure of area-by-area loop filtering activity in one or more images having a predetermined one of said levels; and

determining that at least one image of said sequence is an enabled image or a disabled image in dependence upon the measure.

3. A method according to claim 2, wherein the determining is carried out image-by-image and the predetermined level is a level higher than a level of a current image.

4. A method according to claim 2, wherein the predetermined level is the highest level.

5. A method according to claim 2, comprising:

producing respective such measures of the area-by-area loop filtering activity for two or more said levels, each such measure being dependent on the respective decisions made for the image areas of one or more images having the level concerned; and

determining that at least one image having a given level is an enabled image or a disabled image in dependence upon the measure produced for said given level.

6. A method according to claim 1, comprising comparing the measure with a threshold value and determining that at least one image is an enabled image or a disabled image according to the comparison result.

7. A method according to claim 5, comprising comparing the measure produced for said given level with a threshold value dependent on the given level and determining that at least one image having said given level is an enabled image or a disabled image according to the comparison result.

8. A method according to claim 2, wherein all images having the highest level are determined as such enabled images.

9. A method according to claim 2, wherein all images having the lowest level are determined as such disabled images.

10. A method according to claim 2, comprising determining images having a given level as such disabled images.

11. A method according to claim 10, further comprising after encoding a predetermined number of said disabled images having said given level determining subsequent images having said given level as such enabled images.

12. A method according to claim 1, wherein the images of said sequence have a plurality of color components, and the method comprises:

determining for each color component independently which images are such enabled images and which images are such disabled images.

29. A method according to claim 1, wherein area-by-area loop filtering comprises selecting image area-by-image area in the enabled image whether to use loop filtering or not to use loop filtering in the image area concerned.

30. A method according to claim 1, wherein disabling area-by-area loop filtering comprises selecting for the disabled image as a whole not to use loop filtering.

31. A method according to claim 1, wherein the area-by-area loop filtering is or comprises area-by-area sample adaptive offset loop filtering.

32. A method according to claim 1, wherein said one or more images for which said measure of area-by-area loop filtering activity is produced are previously-determined enabled images.

33. A method according to claim 2, wherein the levels correspond to one of:

(a) different respective values of a Lagrangian parameter in a rate-distortion criterion;

(b) different respective values of a quantization parameter; and

(c) different respective levels among a Group of Pictures (GoP) having a hierarchical coding structure.

39. A method according to claim 33, wherein the levels correspond to (a) different respective values of the Lagrangian parameter in said rate-distortion criterion and the higher the level the higher the value of the Lagrangian parameter.

40. A method according to claim 33, wherein

the levels correspond to (b) different respective values of said quantization parameter, and the lower the level the higher the quantization-parameter value.

41. A method according to claim 33, wherein the levels correspond to (c) different respective levels among a Group of Pictures (GoP) having a hierarchical coding structure, and the greater the number of reference images of said Group of Pictures on which encoding of a given image of said Group of Pictures depends the lower the level of said given image.

42. A method according to claim 31, wherein a type of the area-by-area sample adaptive offset loop filtering is switchable between edge-type and band-type, and the method comprises determining for each type independently which images are such enabled images and which images are such disabled images.

13. A method of decoding images comprising:

receiving a bitstream comprising encoded data of a sequence of images;

decoding the encoded data of the sequence of images to obtain decoded images, the decoding comprising loop filtering;

wherein the loop filtering comprises;

determining which images of said sequence of images are enabled images on which area-by-area loop filtering is to be enabled, permitting different image areas within the same enabled image to have different loop filtering selections, and which images of said sequence of images are disabled images on which area-by-area loop filtering is to be disabled, different image areas within the same disabled image not being permitted to have different loop filtering selections;

performing area-by-area loop filtering on each said enabled image of the sequence and disabling such area-by-area loop filtering on each said disabled image of the sequence;

producing a measure of the area-by-area loop filtering activity in one or more images, said measure being dependent on the loop filtering selections for the image areas of said one or more images in the case of enabling area-by-area loop filtering thereon; and

determining that at least one image of said sequence is an enabled image or a disabled image in dependence upon the measure,

wherein producing the measure comprises determining a number of image areas within an image, including consecutive and non-consecutive image areas, in which use of loop filtering is selected.

14. A method according to claim 13, comprising:

associating with the images of the sequence respective levels from a plurality of levels;

producing a measure of the area-by-area loop filtering activity in one or more images having a predetermined one of said levels; and

determining that at least one image of said sequence is an enabled image or a disabled image in dependence upon the measure.

15. A method according to claim 14, wherein the determining is carried out image-by-image and the predetermined level is a level higher than a level of a current image.

16. A method according to claim 14, wherein the predetermined level is the highest level.

17. A method according to claim 14, comprising:

producing respective such measures of the area-by-area loop filtering activity for two or more said levels, each such measure being dependent on the respective decisions made for the image areas of one or more images having the level concerned; and

determining that at least one image having a given level is an enabled image or a disabled image in dependence upon the measure produced for said given level.

18. A method according to claim 13, comprising comparing the measure with a threshold value and determining that at least one image is an enabled image or a disabled image according to the comparison result.

19. A method according to claim 17, comprising comparing the measure produced for said given level with a threshold value dependent on the given level and determining that at least one image having said given level is an enabled image or a disabled image according to the comparison result.

20. A method according to claim 14, all images having the highest level are determined as such enabled images.

21. A method according to claim 14, all images having the lowest level are determined as such disabled images.

22. A method according to claim 14, comprising determining images having a given level as such disabled images.

23. A method according to claim 22, further comprising after encoding a predetermined number of images having said given level determining subsequent images having said given level as such enabled images.

24. A method according to claim 13, wherein the images of said sequence have a plurality of color components, and the method comprises:

determining for each color component independently which images are such enabled images and which images are such disabled images.

34. A method according to claim 13, wherein area-by-area loop filtering comprises selecting image area-by-image area in the enabled image whether to use loop filtering or not to use loop filtering in the image area concerned.

35. A method according to claim 13, wherein disabling area-by-area loop filtering comprises selecting for the disabled image as a whole not to use loop filtering.

36. A method according to claim 13, wherein the area-by-area loop filtering is or comprises area-by-area sample adaptive offset loop filtering.

37. A method according to claim 13, wherein said one or more images for which said measure of area-by-area loop filtering activity is produced are previously-determined enabled images.

38. A method according to claim 14, wherein the levels correspond to one of:

(a) different respective values of a Lagrangian parameter in a rate-distortion criterion;

(b) different respective values of a quantization parameter; and

(c) different respective levels among a Group of Pictures (GoP) having a hierarchical coding structure.

43. A method according to claim 38, wherein the levels correspond to (a) different respective values of the Lagrangian parameter in said rate-distortion criterion and the higher the level the higher the value of the Lagrangian parameter.

44. A method according to claim 38, wherein

the levels correspond to (b) different respective values of said quantization parameter, and the lower the level the higher the quantization-parameter value.

45. A method according to claim 38, wherein the levels correspond to (c) different respective levels among a Group of Pictures (GoP) having a hierarchical coding structure, and the greater the number of reference images of said Group of Pictures on which encoding of a given image of said Group of Pictures depended the lower the level of said given image.

46. A method according to claim 36, wherein a type of the sample adaptive offset loop filtering is switchable between edge-area-by-area type and band-type, and the method comprises determining for each type independently which images are such enabled images and which images are such disabled images.

25. A device for encoding images, comprising:

a receiving unit which receives a sequence of images;

an encoding unit which encodes the received sequence of images to produce a bitstream; and

a processing unit which processes at least one encoded image to generate a reconstructed image usable as a reference image for the encoding of another image, the processing unit comprising a loop filter adapted to;

determine which images of said sequence of images are enabled images on which area-by-area loop filtering is to be enabled, permitting different image areas within the same enabled image to have different loop filtering selections, and which images of said sequence of images are disabled images on which area-by-area loop filtering is to be disabled, different image areas within the same disabled image not being permitted to have different loop filtering selections;

perform area-by-area loop filtering on each said enabled image of the sequence and not to perform such area-by-area loop filtering on each said disabled image of the sequence;

produce a measure of the area-by-area loop filtering activity in one or more images, said measure being dependent on the loop filtering selections for the image areas of said one or more images in the case of enabling area-by-area loop filtering thereon; and

determine that at least one image of said sequence is an enabled image or a disabled image in dependence upon the measure,

wherein producing the measure comprises determining a number of image areas within an image, including consecutive and non-consecutive image areas, in which use of loop filtering is selected.

26. A device for decoding images, comprising:

a receiving unit which receives a bitstream comprising encoded data of a sequence of images;

a decoding unit which decodes the encoded data of the sequence of images to obtain decoded images, the decoding unit comprising a loop filter adapted to;

determine which images of said sequence of images are enabled images on which area-by-area loop filtering is to be enabled, permitting different image areas within the same enabled image to have different loop filtering selections, and which images of said sequence of images are disabled images on which area-by-area loop filtering is to be disabled, different image areas within the same disabled image not being permitted to have different loop filtering selections;

perform area-by-area loop filtering on each said enabled image of the sequence and not to perform such area-by-area loop filtering on each said disabled image of the sequence;

produce a measure of the area-by-area loop filtering activity in one or more images, said measure being dependent on the loop filtering selections for the image areas of said one or more images in the case of enabling area-by-area loop filtering thereon; and

determine that at least one image of said sequence is an enabled image or a disabled image in dependence upon the measure,

wherein producing the measure comprises determining a number of image areas within an image, including consecutive and non-consecutive image areas, in which use of loop filtering is selected.

27. A non-transitory computer-readable storage medium storing a program which, when run on a processor or computer, causes the processor or computer to carry out a method of encoding images, the program comprising:

a receiving code portion which receives a sequence of images;

an encoding code portion which encodes the received sequence of images to produce a bitstream; and

a processing code portion which processes at least one encoded image to generate a reconstructed image usable as a reference image for the encoding of another image, the processing code portion comprising a loop filter code portion adapted to;

determine which images of said sequence of images are enabled images on which area-by-area loop filtering is to be enabled, permitting different image areas within the same enabled image to have different loop filtering selections, and which images of said sequence of images are disabled images on which area-by-area loop filtering is to be disabled, different image areas within the same disabled image not being permitted to have different loop filtering selections;

perform area-by-area loop filtering on each said enabled image of the sequence and not to perform such area-by-area loop filtering on each said disabled image of the sequence;

produce a measure of the area-by-area loop filtering activity in one or more images, said measure being dependent on the loop filtering selections for the image areas of said one or more images in the case of enabling area-by-area loop filtering thereon; and

determine that at least one image of said sequence is an enabled image or a disabled image in dependence upon the measure,

wherein producing the measure comprises determining a number of image areas within an image, including consecutive and non-consecutive image areas, in which use of loop filtering is selected.

28. A non-transitory computer-readable storage medium storing a program which, when run on a processor or computer, causes the processor or computer to carry out a method of decoding images, the program comprising:

a receiving code portion which receives a bitstream comprising encoded data of a sequence of images;

a decoding code portion which decodes the encoded data of the sequence of images to obtain decoded images, the decoding code portion comprising a loop filter code portion adapted to;

determine which images of said sequence of images are enabled images on which area-by-area loop filtering is to be enabled, permitting different image areas within the same enabled image to have different loop filtering selections, and which images of said sequence of images are disabled images on which area-by-area loop filtering is to be disabled, different image areas within the same disabled image not being permitted to have different loop filtering selections;

perform area-by-area loop filtering on each said enabled image of the sequence and which does not perform such area-by-area loop filtering on each said disabled image of the sequence;

produce a measure of the area-by-area loop filtering activity in one or more images, said measure being dependent on the loop filtering selections for the image areas of said one or more images in the case of enabling area-by-area loop filtering thereon; and

determine that at least one image of said sequence is an enabled image or a disabled image in dependence upon the measure,

wherein producing the measure comprises determining a number of image areas within an image, including consecutive and non-consecutive image areas, in which use of loop filtering is selected.