Method and device for processing of an image by regularization of total variation

• US 9,292,905 B2
• Filed: 06/19/2012
• Issued: 03/22/2016
• Est. Priority Date: 06/24/2011
• Status: Active Grant

• Current Assignee: InterDigital Madison Patent Holdings (InterDigital, Inc.)
• Original Assignee: Thomson Licensing (Vantiva SA)
• Inventors: Jiang, Wen Fei, Jin, Jian, Chen, Zhibo
• Primary Examiner(s): Chen, Wenpeng
• Application Number:
  US14/128,626
• Publication Number:
  US 20140126834A1
• Time in Patent Office: 1,372 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/20192 Edge enhancement; Edge preservation
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: H04N 1/4092 Edge or detail enhancement

Abstract

The invention addresses the problem that traditional Total Variation only enforces the horizontal and vertical intensity continuity and thus fails to reconstruct oblique edges well. In an embodiment, Directional Total Variation is defined which supports multiple gradient directions. It first pre-processes the image and determines the direction of edges/texture; Directional TV is calculated based on the gradients along the determined direction and its orthogonal direction. By utilizing adaptive weights to different directions in the regularization, Directional TV is capable of preserving edges independent from their orientation nicely. Thus, Directional TV regularization based image denoising, compression or super resolution tend to obtain better quality.

Claims

1. A method for processing of an image comprising:

pre-processing the image for determining at least one predominant direction of at least one of edges and texture;

determining a total variation of the image using a weighted sum of variations along the at least one predominant direction and along a direction orthogonal to the at least one predominant direction; and

processing the image using regularization of the total variation.

2. The method of claim 1, wherein said pre-processing comprises dividing the image into patches and determining a predominant direction for each patch.

3. The method of claim 1, wherein said pre-processing further comprises:

calculating gradients for each patch,

calculating an energy function for each gradient,

calculating a ratio between energy functions of orthogonal gradients, and

determining the predominant direction according to the ratio between the energy functions.

4. The method of claim 3, wherein said pre-processing further comprises determining that the at least one predominant direction has an energy function ratio further exceeding, by at least a predetermined positive threshold, each energy function ratio of gradients in different directions.

5. The method of claim 3, further comprising using a maximal energy function ratio for determining weights of said weighted sum.

6. The method of claim 2, wherein the determining a predominant direction further comprises:

detecting that a current patch is in a flat region,

detecting that a predominant direction has already been determined for an upper and/or left neighbor patch of the current patch, and

selecting among available possible predominant directions the most similar direction for the current patch.

7. The method according to claim 1, wherein for determining a total variation of the image, the variation along the at least one predominant direction is given higher weight than the variation along a direction orthogonal to the at least one predominant direction.

8. The method according to claim 1, further comprising calculating adaptive weights for said weighted sum of variations along the at least one predominant direction and along a direction orthogonal to the at least one predominant direction, the weights being calculated based on a ratio of energy sums.

9. The method according to claim 8, wherein a first weight α

_i,j that is applied to the variations along the at least one predominant direction x is reverse proportional to a respective energy sum E_x, and a second weight β

_i,j that is applied to the variations orthogonal to the at least one predominant direction is reverse proportional to said energy sum E_x.

10. The method according to claim 1, further comprising:

determining for at least one patch that not all of the required gradients are available; and

calculating a Directional Total Variation by scaling and averaging the determined available gradients for the at least one patch.

11. A device for processing of an image, comprising

a pre-processing unit adapted to pre-process the image in order to determine at least one predominant direction of at least one of edges and texture;

a Total Variation determining unit adapted to determine a total variation of the image using a weighted sum of variations along the at least one predominant direction and along a direction orthogonal to the at least one predominant direction; and

a processing unit adapted to process the image using regularization of the total variation.

12. The device according to claim 11, further comprising dividing means for dividing the image into patches, wherein for each patch a predominant direction is determined.

13. The device according to claim 12, wherein the pre-processing unit comprises a calculating and determination unit adapted to:

calculate gradients for each patch,

calculate an energy function for each gradient,

calculate a ratio between energy functions of orthogonal gradients, and

determine the predominant direction as the one that has a maximum energy function ratio.

14. The device according to claim 13, wherein said calculating and determination unit of said pre-processing unit is further adapted to determine that the at least one predominant direction has an energy function ratio further exceeding, by at least a predetermined positive threshold, each energy function ratio of gradients in different directions.

15. The device according to claim 13, wherein the processing unit is adapted to use a maximal energy function ratio for determining weights of said weighted sum.

16. The device according to claim 12, further comprising a detection and selection unit adapted to:

detect that a current patch is in a flat region,

detect that a predominant direction has already been determined for at least one of the upper and left neighbor patch of the current patch, and

select among available possible predominant directions the most similar direction for the current patch.

17. The device according to claim 11, wherein the Total Variation determining unit comprises calculation means for calculating a Directional Total Variation based on the gradients along the determined at least one predominant direction and its orthogonal direction.