METHODS AND APPARATUS FOR ADAPTIVE REFERENCE FILTERING (ARF) OF BI-PREDICTIVE PICTURES IN MULTI-VIEW CODED VIDEO

• US 20090129465A1
• Filed: 11/10/2008
• Published: 05/21/2009
• Est. Priority Date: 11/09/2007
• Status: Active Grant

• Current Assignee: InterDigital Madison Patent Holdings (InterDigital, Inc.)
• Original Assignee: Thomson Licensing (Vantiva SA)
• Inventors: LAI, POLIN, Yin, Peng, Pandit, Purvin Bibhas
• Granted Patent: US 8,270,472 B2
• Time in Patent Office: Days
• US Class Current: 375/240.20
• CPC Class Codes: H04N 19/109 among a plurality of temporal predictive coding modes
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/134 characterised by the element, parameter or criterion affecting or controlling the adaptive coding
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/573 Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/82 involving filtering within a prediction loop

Abstract

There are provided methods and apparatus for adaptive reference filtering of bi-predictive pictures in multi-view coded video. An apparatus includes an encoder (100) for encoding a current picture as a bi-predictive picture. The encoder (100) performs adaptive filtering of at least one reference picture to respectively obtain at least one filtered reference picture, and bi-predictively codes the current picture using the at least one filtered reference picture. The at least one reference picture is a picture wherein at least one sample thereof is used for inter-prediction either of subsequent to the least one sample being applied to an in-loop filter or in an absence of the at least one sample being applied to an in-loop filter. Reference filter coefficients for the adaptive filtering are determined responsive to at least one of luma values and chroma values of pixels grouped based on at least one of depth and disparity and motion.

Claims

1. An apparatus, comprising:

an encoder (100) for encoding a current picture as a bi-predictive picture, wherein said encoder performs adaptive filtering of at least one reference picture to respectively obtain at least one filtered reference picture, and bi-predictively codes the current picture using the at least one filtered reference picture, the at least one reference picture being a picture wherein at least one sample thereof is used for inter-prediction either of subsequent to the least one sample being applied to an in-loop filter or in an absence of the at least one sample being applied to an in-loop filter, and wherein reference filter coefficients for the adaptive filtering are determined responsive to at least one of luma values and chroma values of pixels grouped based on at least one of depth and disparity and motion.

2. The apparatus of claim 1, wherein said encoder (100) estimates at least one set of filters for the adaptive filtering based on information from the current picture and two reference pictures, each of the two reference pictures corresponding to a respective one of two reference lists, the two reference pictures including the at least one reference picture to be adaptively filtered to obtain the at least one filtered reference picture.

3. The apparatus of claim 2, wherein the information from the current picture and the two reference pictures in the two reference lists comprises at least one of three sets of information, the first set of information comprising at least one prediction direction for different blocks within the current picture, the second set of information comprising at least one of motion information and disparity information between the current picture and the two reference pictures in the two reference lists, and the third set of information comprising at least one of intensity information and color information between the current picture and the two reference picture in the two reference lists.

4. The apparatus of claim 2, wherein said encoder (100) further determines a number of filters in each of the at least one set based on the information from the current picture and the two reference pictures in the two reference lists.

5. The apparatus of claim 1, wherein the adaptive filtering is used to generate at least one of integer pixels and sub-pixels in two filtered reference pictures corresponding to two reference lists, the two filtered reference pictures including the at least one filtered reference picture that is obtained from adaptively filtering the at least one reference picture.

6. The apparatus of claim 1, wherein said encoder (100) classifies blocks in the current picture based upon three classes, wherein the blocks in the current picture classified in a first one of the three classes select reference pictures from a first reference list, blocks in the current picture classified in a second one of the three classes select reference pictures from a second reference list, and blocks in the current picture classified in a third one of the three classes select an average predictor based on the reference pictures from both the first reference list and the second reference list.

7. The apparatus of claim 6, wherein said encoder (100) respectively estimates one of three sets of filters for each of the three classes, where each set consists of a single filter when picture-wise inter-view mismatches exist, and includes more than one filter when non-picture-wise inter-view mismatches exist.

8. The apparatus of claim 6, wherein at least one of the three classes is partitioned into multiple depth-levels based upon disparity information.

9. The apparatus of claim 6, wherein said encoder (100) respectively estimates two sets of filters for the three classes, where a first one of the two sets corresponds to the first one and the third one of the three classes, and a second one of the two sets corresponds to the second one and the third one of the three classes, the first one of the two sets being estimated with respect to the first reference list, and the second one of the two sets being estimated with respect to the second reference list.

10. The apparatus of claim 1, wherein said encoder (100) performs an initial disparity estimation using only a single reference list from among two reference lists when only the single reference list includes mismatches with respect to the current picture, partitions the current picture into regions suffering from different types of mismatches, estimates at least one filter for each of the regions, and applies the at least one filter with respect to the single reference list used in the initial disparity estimation, wherein the current picture is encoded as the bi-predictive picture using the single reference list having at least one filtered reference picture corresponding thereto and another reference list.

11. The apparatus of claim 1, wherein said encoder (100) signals, using at least one high level syntax element, at least one of filters that are used for a first reference list, filters that are used for a second reference list, and filters that are used for both the first and the second reference lists.

21. The method of claim 1, wherein said encoding step comprises:

performing an initial disparity estimation using only a single reference list when only the single reference list includes mismatches with respect to the current picture (710);

partitioning the current picture into regions suffering from different types of mismatches (720);

estimating at least one filter for each of the regions (725); and

applying the at least one filter to the single reference list used in the initial disparity estimation (730),

wherein the current picture is encoded as the bi-predictive picture using the single reference list having at least one filtered reference picture corresponding thereto and another reference list (735).

12. A method, comprising:

encoding a current picture as a bi-predictive picture, wherein said encoding step comprises performing adaptive filtering of at least one reference picture to respectively obtain at least one filtered reference picture, and bi-predictively coding the current picture using the at least one filtered reference picture, the at least one reference picture being a picture wherein at least one sample thereof is used for inter-prediction either of subsequent to the least one sample being applied to an in-loop filter or in an absence of the at least one sample being applied to an in-loop filter, and wherein reference filter coefficients for the adaptive filtering are determined responsive to at least one of luma values and chroma values of pixels grouped based on at least one of depth and disparity and motion (415, 420, 425).

13. The method of claim 12, wherein said encoding step comprises estimating at least one set of filters for the adaptive filtering based on information from the current picture and two reference pictures, each of the two reference pictures corresponding to a respective one of two reference lists, the two reference pictures including the at least one reference picture to be adaptively filtered to obtain the at least one filtered reference picture.

14. The method of claim 13, wherein the information from the current picture and the two reference pictures in the two reference lists comprises at least one of three sets of information, the first set of information comprising at least one prediction direction for different blocks within the current picture, the second set of information comprising at least one of motion information and disparity information between the current picture and the two reference pictures in the two reference lists, and the third set of information comprising at least one of intensity information and color information between the current picture and the two reference picture in the two reference lists (415).

15. The method of claim 13, wherein said encoding step further comprises determining a number of filters in each of the at least one set based on information from the current picture and the two reference pictures in the two reference lists (520, 530, 545, 620, 630, 645, 715, 725, 740).

16. The method of claim 12, wherein the adaptive filtering is used to generate at least one of integer pixels and sub-pixels in two filtered reference pictures corresponding to two reference lists, the two filtered reference pictures including the at least one filtered reference picture that is obtained from adaptively filtering the at least one reference picture.

17. The method of claim 12, wherein said encoding step comprises classifying blocks in the current picture based upon three classes, wherein the blocks in the current picture classified in a first one of the three classes select reference pictures from a first reference list, blocks in the current picture classified in a second one of the three classes select reference pictures from a second reference list, and blocks in the current picture classified in a third one of the three classes select an average predictor based on the reference pictures from both the first reference list and the second reference list (515, 615).

18. The method of claim 17, wherein said encoding step further comprises respectively estimating one of three sets of filters for each of the three classes, where each set consists of a single filter when picture-wise inter-view mismatches exist, and includes more than one filter when non-picture-wise inter-view mismatches exist (520, 535, 550).

19. The method of claim 17, wherein at least one of the three classes is partitioned into multiple depth-levels based upon disparity information (510, 610, 710).

20. The method of claim 17, wherein said encoding step further comprises respectively estimating two sets of filters for the three classes, where a first one of the two sets corresponds to the first one and the third one of the three classes, and a second one of the two sets corresponds to the second one and the third one of the three classes, the first one of the two sets being estimated with respect to the first reference list, and the second one of the two sets being estimated with respect to the second reference list (630, 650).

22. The method of claim 12, wherein said encoding step comprises signaling, using at least one high level syntax element, at least one of filters that are used for a first reference list, filters that are used for a second reference list, and filters that are used for both the first and the second reference lists (815).