Adaptive filtering based upon boundary strength

• US 7,450,641 B2
• Filed: 09/14/2001
• Issued: 11/11/2008
• Est. Priority Date: 09/14/2001
• Status: Active Grant

• Current Assignee: Dolby Laboratories Licensing Corporation (Dolby Laboratories Incorporated)
• Original Assignee: Sharp Laboratories of America Incorporated (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Sun, Shijun, Lei, Shawmin
• Primary Examiner(s): LEE, Y YOUNG
• Application Number:
  US09/953,329
• Publication Number:
  US 20030053541A1
• Time in Patent Office: 2,615 Days
• Field of Search: 375/240.16, 375/240.2, 375/240.24, 375/240.29
• US Class Current: 375/240.16
• CPC Class Codes: H04N 19/105 Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/139 Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/15 by monitoring actual compressed data size at the memory before deciding storage at the transmission buffer
• CPC Class Codes: H04N 19/159 Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
• CPC Class Codes: H04N 19/167 Position within a video image, e.g. region of interest [ROI]
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/196 being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters processing of motion vectors H04N19/513
• CPC Class Codes: H04N 19/197 including determination of the initial value of an encoding parameter H04N19/56 takes precedence
• CPC Class Codes: H04N 19/43 Hardware specially adapted for motion estimation or compensation
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/527 Global motion vector estimation
• CPC Class Codes: H04N 19/573 Motion compensation with multiple frame prediction using two or more reference frames in a given prediction direction
• CPC Class Codes: H04N 19/58 Motion compensation with long-term prediction, i.e. the reference frame for a current frame not being the temporally closest one H04N19/23 takes precedence
• CPC Class Codes: H04N 19/60 : using transform coding
• CPC Class Codes: H04N 19/61 : in combination with predictive coding
• CPC Class Codes: H04N 19/615 : using motion compensated temporal filtering [MCTF]
• CPC Class Codes: H04N 19/82 : involving filtering within a prediction loop
• CPC Class Codes: H04N 19/85 : using pre-processing or post-processing specially adapted for video compression
• CPC Class Codes: H04N 19/86 : involving reduction of coding artifacts, e.g. of blockiness

Abstract

Adjacent regions are identified in an image. Coding parameters for the adjacent regions are identified. Selective filtering is performed at the region between the identified adjacent regions.

Claims

1. A method for at least one of encoding and decoding a digital image, comprising:

(a) identifying a first region adjacent a second region in said image, wherein said first region and said second region are substantially the same size;

(b) identifying a first set of one or more motion vectors calculated using information only from said first region and a second set of one or more motion vectors calculated using information only from said second region; and

(c) irrespective of whether or not said first region has non-zero transform coefficients, selectively not filtering the boundary area between said first region and said second region based on a relationship between said first set of one or more motion vectors and only said second set of one or more motion vectors.

2. The method of claim 1 wherein said filtering reduces blocking artifacts.

3. The method of claim 2 wherein said selective filtering is based upon the similarity of said motion vectors.

4. The method of claim 1 including said selectively not filtering when adjacent regions have similar motion vectors pointing to a same reference image frame.

5. The method of claim 1 including identifying transform coefficients for adjacent regions and said selectively filtering based upon a similarity of said transform coefficients.

6. The method of claim 5 including identifying D.C. components in said transform coefficients and said selectively filtering based upon a similarity of said D.C. components.

7. The method of claim 6 including identifying A.C. components in said transform coefficients and said selectively filtering based upon a similarity of said D.C. and said A.C. components.

8. The method of claim 5 including transforming said adjacent regions using a Discrete Cosine Transform to generate said transform coefficients.

9. The method of claim 1 including:

(a) comparing regions in the image with reference frames;

(b) transforming the result of said comparison between said reference frames and said regions in the image into transformed regions having transform coefficients;

(c) comparing the similarities between the transform coefficients; and

(d) selectively filtering at least a portion of said region according to said comparing of step (c).

10. The method of claim 9 wherein said transform coefficients include D.C. transform coefficients.

11. The method of claim 1 including:

(a) identifying similarities between said motion vectors in a luminance channel of adjacent regions; and

(b) controlling filtering for both the luminance channel and a chrominance channel in said image according to similarities in the luminance channel.

12. The method of claim 1 wherein said selectively filtering is based upon determining whether at least one of adjacent regions are intra-coded.

13. The method of claim 12 wherein said selectively filtering is based upon determining whether at least one of said adjacent regions is encoded with residuals.

14. The method of claim 13 wherein said selectively filtering is based upon determining whether at least two of said adjacent regions are predicted based upon two different reference frames.

15. The method of claim 14 wherein said selectively filtering is based upon determining whether at least two of said adjacent regions have an absolute difference of said motion vectors of said at least two adjacent regions at least one of greater than or less than a threshold value.

16. The method of claim 15 wherein said absolute difference is in a first direction.

17. The method of claim 16 wherein said absolute difference is also in a second direction different than said first direction.

18. The method of claim 15 wherein said selective filtering uses a first filter when said adjacent regions are intra-coded, a second filter when said at least one of said adjacent regions is encoded with said residuals, a third filter when said at least two of said adjacent regions are predicted based upon said two different reference frames, and said third filter when said at least two of said adjacent regions have said absolute difference of said motion vectors of said at least two adjacent regions said at least one of greater than or less than a threshold value.

19. The method of claim 14 wherein said selective filtering uses a first filter when said adjacent regions are intra-coded, a second filter when said at least one of said adjacent regions is encoded with said residuals, and a third filter when said at least two of said adjacent regions are predicted based upon said two different reference frames.

20. The method of claim 13 wherein said selective filtering uses a first filter when said adjacent regions are intra-coded and a second filter when said at least one of said adjacent regions is encoded with said residuals.

21. The method of claim 12 wherein said selective filtering uses a first filter when said adjacent regions are intra-coded.

22. An encoder for encoding an image comprising:

(a) a processor adapted to identify a first region adjacent at least one other region in said image, wherein said first region and said at least one other region are substantially the same size;

(b) said processor comparing at least one motion vector calculated using information from only said first region only with at least one motion vector calculated using information from only a respective one of said at least one other region; and

(c) irrespective of whether or not said first region has non-zero transform coefficients, said processor selectively not filtering the region proximate the boundary between said first region and said respective one of said at least one other region based on said comparing.

23. The encoder of claim 22 wherein said filtering reduces blocking artifacts.

24. The encoder of claim 23 wherein said selective filtering is based upon the similarity of said motion vectors.

25. The encoder of claim 22 including said filtering when adjacent regions have similar motion vectors pointing to a same reference image frame.

26. The encoder of claim 22 including identifying transform coefficients for adjacent regions and said selectively filtering based upon a similarity of said transform coefficients.

27. The encoder of claim 26 including identifying D.C. components in said transform coefficients and said selectively filtering based upon a similarity of said D.C. components.

28. The encoder of claim 27 including identifying A.C. components in said transform coefficients and said selectively filtering based upon a similarity of said D.C. and said A.C. components.

29. The encoder of claim 26 including transforming said adjacent regions using a Discrete Cosine Transform to generate said transform coefficients.

30. The encoder of claim 22 further including said processor:

(a) comparing regions in the image with reference frames;

(b) transforming the result of said comparison between said reference frames and said regions in the image into transformed regions having transform coefficients;

(c) comparing the similarities between the transform coefficients; and

(d) selectively filtering at least a portion of said region according to said comparing of step (c).

31. The encoder of claim 30 wherein said transform coefficients include D.C. transform coefficients.

32. The encoder of claim 22 further comprising said processor:

(a) identifying similarities between said motion vectors in a luminance channel of said adjacent regions; and

(b) controlling filtering for both the luminance channel and a chrominance channel in said image according to similarities in the luminance channel.

33. The encoder of claim 22 wherein said selectively filtering is based upon determining whether at least one of adjacent regions are intra-coded.

34. The encoder of claim 33 wherein said selectively filtering is based upon determining whether at least one of said adjacent regions is encoded with residuals.

35. The encoder of claim 34 wherein said selectively filtering is based upon determining whether at least two of said adjacent regions are predicted based upon two different reference frames.

36. The encoder of claim 35 wherein said selectively filtering is based upon determining whether at least two of said adjacent regions have an absolute difference of said motion vectors of said at least two adjacent regions at least one of greater than or less than a threshold value.

37. The encoder of claim 36 wherein said absolute difference is in a first direction.

38. The encoder of claim 37 wherein said absolute difference is also in a second direction different than said first direction.

39. The encoder of claim 36 wherein said selective filtering uses a first filter when said adjacent regions are intra-coded, a second filter when said at least one of said adjacent regions is encoded with said residuals, a third filter when said at least two of said adjacent regions are predicted based upon said two different reference frames, and said third filter when said at least two of said adjacent regions have said absolute difference of said motion vectors of said at least two adjacent regions said at least one of greater than or less than a threshold value.

40. The encoder of claim 35 wherein said selective filtering uses a first filter when said adjacent regions are intra-coded, a second filter when said at least one of said adjacent regions is encoded with said residuals, and a third filter when said at least two of said adjacent regions are predicted based upon said two different reference frames.

41. The encoder of claim 34 wherein said selective filtering uses a first filter when said adjacent regions are intra-coded and a second filter when said at least one of said adjacent regions is encoded with said residuals.

42. The encoder of claim 33 wherein said selective filtering uses a first filter when said adjacent regions are intra-coded.

43. A decoder for decoding an image comprising:

(a) a processor adapted to identify a first region adjacent at least one other region in said image, wherein said first region and said at least one other region are substantially the same size;

(b) said processor comparing at least one motion vector calculated using information from only said first region only with at least one motion vector calculated using information from only a respective one of said at least one other region; and

(c) irrespective of whether or not said first region has non-zero transform coefficients, said processor selectively not filtering the region proximate the boundary between said first region and said respective one of said at least one other region based on said comparing.

44. The decoder of claim 43 wherein said filtering reduces blocking artifacts.

45. The decoder of claim 44 wherein said selective filtering is based upon the similarity of said motion vectors.

46. The decoder of claim 43 including said filtering when adjacent regions have similar motion vectors pointing to a same reference image frame.

47. The decoder of claim 43 including identifying transform coefficients for adjacent regions and said selectively filtering based upon a similarity of said transform coefficients.

48. The decoder of claim 47 including identifying D.C. components in said transform coefficients and said selectively filtering based upon a similarity of said D.C. components.

49. The decoder of claim 48 including identifying A.C. components in said transform coefficients and said selectively filtering based upon a similarity of said D.C. and said A.C. components.

50. The decoder of claim 47 Discrete Cosine Transform to generate said transform coefficients.

51. The decoder of claim 43 further including said processor:

(a) comparing regions in the image with reference frames;

(b) transforming the result of said comparison between said reference frames and said regions in the image into transformed regions having transform coefficients;

(c) comparing the similarities between the transform coefficients; and

(d) selectively filtering at least a portion of said region according to said comparing of step (c).

52. The decoder of claim 51 wherein said transform coefficients include D.C. transform coefficients.

53. The decoder of claim 43 further comprising said processor:

(a) identifying similarities between motion vectors in a luminance channel of adjacent regions; and

(b) controlling filtering for both the luminance channel and a chrominance channel in said image according to similarities in the luminance channel.

54. The decoder of claim 43 wherein said selectively filtering is based upon determining whether at least one of adjacent regions are intra-coded.

55. The decoder of claim 54 wherein said selectively filtering is based upon determining whether at least one of said adjacent regions is encoded with residuals.

56. The decoder of claim 55 wherein said selectively filtering is based upon determining whether at least two of said adjacent regions are predicted based upon two different reference frames.

57. The decoder of claim 56 wherein said selectively filtering is based upon determining whether at least two of said adjacent regions have an absolute difference of said motion vectors of said at least two adjacent regions at least one of greater than or less than a threshold value.

58. The decoder of claim 57 wherein said absolute difference is in a first direction.

59. The decoder of claim 58 wherein said absolute difference is also in a second direction different than said first direction.

60. The decoder of claim 57 wherein said selective filtering uses a first filter when said adjacent regions are intra-coded, a second filter when said at least one of said adjacent regions is encoded with said residuals, a third filter when said at least two of said adjacent regions are predicted based upon said two different reference frames, and said third filter when said at least two of said adjacent regions have said absolute difference of said motion vectors of said at least two adjacent regions said at least one of greater than or less than a threshold value.

61. The decoder of claim 56 wherein said selective filtering uses a first filter when said adjacent regions are intra-coded, a second filter when said at least one of said adjacent regions is encoded with said residuals, and a third filter when said at least two of said adjacent regions are predicted based upon said two different reference frames.

62. The decoder of claim 55 wherein said selective filtering uses a first filter when said adjacent regions are intra-coded and a second filter when said at least one of said adjacent regions is encoded with said residuals.

63. The decoder of claim 54 wherein said selective filtering uses a first filter when said adjacent regions are intra-coded.