METHOD AND SYSTEM FOR DETERMINING A QUALITY MEASURE FOR AN IMAGE USING A VARIABLE NUMBER OF MULTI-LEVEL DECOMPOSITIONS

• US 20110033119A1
• Filed: 10/17/2010
• Published: 02/10/2011
• Est. Priority Date: 02/11/2009
• Status: Active Grant

• Current Assignee: Ecole de technologie superieure
• Original Assignee: Ecole de technologie superieure
• Inventors: Rezazadeh, Soroosh, Coulombe, Stephane
• Granted Patent: US 8,515,181 B2
• Time in Patent Office: Days
• US Class Current: 382/199
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/20064 Wavelet transform [DWT]
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 7/0004 Industrial image inspection

Abstract

Method and system for determining a measure of quality for images are presented. Multi-level decomposition of images in the wavelet domain using a variable number of levels of decomposition and aggregation of selected subbands is performed to obtain an accurate measure of quality. The processing time is reduced in comparison to that required by other methods for generating measures of quality.

Claims

1. A method for determining a measure of quality for a distorted image Y, characterizing a similarity between the image Y and an undistorted reference image X, having the same number of rows and columns of pixels as the image Y, the method comprising:

(a1) determining a number of levels of decomposition, N, to be applied to the image X and the image Y as a function of one or more of a resolution of the image Y, a viewing distance for the image Y, and an image quality metric (IQM) applied to the image Y;

(b1) applying a N level multiresolution decomposition, comprising levels 1, 2, . . . i, i+1, . . . N, to the image X, producing an approximation subband containing main content of the image X and detail subbands containing edges of the image X;

(c1) applying said N level multiresolution decomposition, comprising levels 1, 2, . . . i, i+1, . . . N, to the image Y, producing an approximation subband containing main content of the image Y and detail subbands containing edges of the image Y;

(d1) applying the IQM to the approximation subband of the image X and the approximation subband of the image Y to produce an approximation quality measure, characterizing similarity between the main content of the image X and the main content of the image Y;

(e1) processing the detail subbands of the image X producing an edge map of the image X, characterizing the edges of the image X;

(f1) processing the detail subbands of the image Y producing an edge map of the image Y, characterizing the edges of image Y;

(g1) applying the IQM between the edge map of the image X and the edge map of the image Y to produce an edge quality measure, characterizing similarity between the edges of the image X and the edges of the image Y; and

(h1) processing the approximation quality measure and the edge quality measure to determine the measure of quality.

2. The method of claim 1, wherein the step (a1) further comprises determining the number of levels of decomposition N as a function of a minimum size of the approximation subband, S, which produces a substantially peak response for human visual system.

3. The method of claim 2, wherein the step (a1) further comprises determining the S as follows:

(a3) determining k, a ratio between the viewing distance and a height of a device on which the image Y is displayed; and

(b3) determining the S as a ratio between a predetermined number and the k.

4. The method of claim 2, wherein the step (a1) further comprises:

(a4) determining a ratio between a minimum of a height H of the image Y and a width W of the image Y, and the S;

(b4) applying a logarithm function to the ratio;

(c4) applying a round function to a result obtained in the step (b4) of applying said logarithm function; and

(d4) determining the N as a maximum of 0 and a result obtained in the step (c4) of applying said round function.

5. The method of claim 3, wherein the predetermined number is about 344.

6. The method of claim 2, further comprising one or more of the following:

(a6) at the step (b1);

(a6i) for the level 1, applying the multiresolution decomposition to the Image X producing intermediate subbands at the level 1 for processing at level 2; and

(a6ii) for the level i, with i ranging from 2 to N, applying the multiresolution decomposition to only selected intermediate subbands produced by the multiresolution decomposition performed at the level i−

1; and

(b6) at the step (c1);

(b6i) for the level 1, applying the multiresolution decomposition to the Image Y producing intermediate subbands at the level 1 for processing at level 2; and

(b6ii) for the level i, with i ranging from 2 to N, applying the multiresolution decomposition to only selected intermediate subbands produced by the multiresolution decomposition performed at the level i−

1.

7. The method of claim 6, wherein the steps (a6ii) and (b6ii) further comprise determining the selected intermediate subbands based on an accuracy to be achieved in determining the measure of quality.

8. The method of claim 7, further comprising selecting the intermediate subbands based on a number of the intermediate subbands required for achieving the accuracy.

9. The method of claim 6, further comprising one or more of the following:

(a9) at the step (e1), aggregating one or more of the selected intermediate subbands of the image X produced by the multiresolution decomposition performed at each level j, with j ranging from 1 to N−

1; and

(b9) at the step (f1), aggregating one or more of the selected intermediate subbands of the image Y produced by the multiresolution decomposition performed at each level j, with j ranging from 1 to N−

1.

10. The method of claim 9, wherein:

(a10) the step (a9) further comprises aggregating one or more of selected detail subbands of the image X produced at level N with said one or more of the selected intermediate subbands produced at said each level j; and

(b10) the step (b9) comprises aggregating one or more of selected detail subbands of the image Y produced at level N with said one or more of the selected intermediate subbands produced at said each level j.

11. The method of claim 10, further comprising:

(a11) at the step (a9), determining an edge map at the level j for the image X as a function of said one or more of the selected intermediate subbands produced at said each level j;

(b11) at the step (a10);

(b11i) determining an edge map at the level N for the image X as a function of said one or more of the selected detailed subbands of the image X produced at the level N; and

(b11ii) determining the edge map of the image X as a weighted sum of edge maps at the level j and the edge map at the level N for the image X;

(c11) at the step (b9), determining an edge map at the level j for the image Y as a function of said one or more of the selected intermediate subbands produced at said each level j; and

(d11) at the step (b10);

(d11i) determining an edge map at the level N for the image Y as a function of said one or more of the selected detailed subbands of the image Y produced at the level N; and

(d11ii) determining the edge map of the image Y as a weighted sum of edge maps at the level j and the edge map at the level N for the image Y.

12. The method of claim 2, wherein the approximation quality measure is an approximation quality map, and the edge quality measure is an edge quality map, and the step (h1) further comprises:

(a12) generating a contrast map, including assigning corresponding values to the pixels of the approximation subband and the edge map of the image X and the image Y according to their respective importance to human visual system;

(b12) performing weighted pooling of the approximation quality map using the contrast map to produce an approximation quality score;

(c12) performing weighted pooling of the edge quality map using the contrast map to produce an edge quality score; and

(d12) combining the approximation similarity score from the step (b12) with the edge similarity score from the step (c12) to determine the measure of quality.

13. The method of claim 2, wherein:

(a13) the step (d1) comprises applying a structural similarity (SSIM) IQM to the approximation subband of the image X and the approximation subband of the image Y to produce an approximation SSIM map, SSIM_A;

(b13) the step (g1) comprises applying the SSIM IQM between the edge map of the image X and the edge map of the image Y to produce an edge SSIM map, SSIM_E; and

(c13) the step (h1) comprises processing the SSIM_A and the SSIM_E to determine a SSIM_DWT score as the measure of quality.

14. The method of claim 2, wherein:

(a14) the step (d1) comprises applying an Absolute Difference (AD) IQM to the approximation subband of the image X and the approximation subband of the image Y to produce an approximation AD map, AD_A;

(b14) the step (g1) comprises applying the AD IQM between the edge map of the image X and the edge map of the image Y to produce an edge AD map, AD_E; and

(c14) the step (h1) comprises processing the AD_A and the AD_E to determine an AD_DWT score as the measure of quality.

15. The method of claim 2, wherein:

(a15) the step (d1) comprises applying a peak-signal-to-noise ratio (PSNR) IQM to the approximation subband of the image X and the approximation subband of the image Y to produce a PSNR approximation quality score, PSNR_A;

(b15) the step (g1) comprises applying the PSNR IQM between the edge map of the image X and the edge map of the image Y to produce a PSNR edge quality score, PSNR_E; and

(c15) step (h1) comprises processing the PSNR_A and the PSNR_E to determine a PSNR_DWT score as the measure of quality.

16. The method of claim 2, wherein:

(a16) the step (d1) comprises applying a Visual Information Fidelity (VIF) IQM to the approximation subband of the image X and the approximation subband of the image Y to produce a VIF approximation quality score, VIF_A;

(b16) the step (g1) comprises applying the VIF IQM between the edge map of the image X and the edge map of the image Y to produce a VIF edge quality score, VIF_E; and

(c16) the step (h1) comprises processing the VIF_A and the VIF_E to determine a VIF_DWT score as the measure of quality.

17. The method of claim 1, wherein the steps (b1) and (c1) comprise applying a N level discrete wavelet transform (DWT).

18. The method of claim 17, wherein the discrete wavelet transform is a Haar transform.

19. The method of claim 17, wherein the discrete wavelet transform is one of a Newland transform, or wavelet transform using a Daubechies filter.

20. The method of claim 6, wherein the detail subbands of the image X include one or more of a horizontal subband containing horizontal edges, one or more of a vertical subband containing vertical edges and one or more of a diagonal subband containing diagonal edges of the image X, and wherein the detail subbands of the image Y include one or more of a horizontal subband containing horizontal edges, one or more of a vertical subband containing vertical edges, and one or more of a diagonal subband containing diagonal edges of the image Y.

21. The method of claim 10, wherein:

(a21) the step (a10) further comprises choosing the selected intermediate subbands at said each level j and the selected detail subbands for the image X having substantially same resolution for aggregating; and

(b21) the step (b10) comprises choosing the selected intermediate subbands at said each level j and the selected detail subbands for the image Y having substantially same resolution for aggregating.

22. A system for determining a measure of quality for a distorted image Y, characterizing a similarity between the image Y and an undistorted reference image X, having the same number of rows and columns of pixels as the image Y, the system comprising:

a processor, and a computer readable storage medium having computer readable instructions stored thereon, which, when executed by the processor, form the following;

(a22) a number of levels unit determining a number of levels of decomposition, N, to be applied to the image X and the image Y as a function of one or more of a resolution of the image Y, a viewing distance for the image Y, and an image quality metric (IQM) applied to the image Y;

(b22) a first decomposition unit applying a N level multiresolution decomposition, comprising levels 1, 2, . . . i, i+1, . . . N, to the image X, producing an approximation subband containing main content of the image X and detail subbands containing edges of the image X;

(c22) a second decomposition unit applying said N level multiresolution decomposition, comprising levels 1, 2, . . . i, i+1, . . . N, to the image Y, producing an approximation subband containing main content of the image Y and detail subbands containing edges of the image Y;

(d22) an approximation quality measure unit applying the IQM to the approximation subband of the image X and the approximation subband of the image Y to produce an approximation quality measure characterizing similarity between the main content of the image X and the main content of the image Y;

(e22) a first detail subbands unit processing the detail subbands of the image X producing an edge map of the image X characterizing the edges of the image X;

(f22) a second detail subbands unit processing the detail subbands of the image Y producing an edge map of the image Y characterizing the edges of image Y;

(g22) an edge quality measure unit applying the IQM between the edge map of the image X and the edge map of the image Y to produce an edge quality measure characterizing similarity between the edges of the image X and the edges of the image Y; and

(h22) a quality measure unit processing the approximation quality measure and the edge quality measure to determine the measure of quality.

23. The system of claim 22, wherein the number of levels unit further comprises computational means for determining the number of levels of decomposition N as a function of a minimum size of the approximation subband, S, which produces a substantially peak response for human visual system.

24. The system of claim 23, wherein the number of levels unit (a22) further comprises a minimum size unit for:

determining k, a ratio between the viewing distance and a height of a device on which the image Y is displayed; and

determining the S as a ratio between a predetermined number and the k.

25. The system of claim 23, wherein the number of levels unit (a22) further comprises a computation unit for:

determining a ratio between a minimum of a height H of the image Y and a width W of the image Y, and the S;

applying a logarithm function to the ratio;

applying a round function to a result produced by said logarithm function; and

determining the N as a maximum of 0 and a result of applying said round function.

26. The system of claim 24, wherein the predetermined number is about 344.

27. The system of claim 23, wherein:

(a27) the first decomposition unit (b22) comprises one or more of the following;

(a27i) a first level 1 decomposition unit applying the multiresolution decomposition to the image X producing intermediate subbands at the level 1 for processing at level 2; and

(a27ii) a first level i decomposition unit applying the multiresolution decomposition to only selected intermediate subbands produced by the multiresolution decomposition performed at the level i−

1, with i ranging from 2 to N; and

(b27) the second decomposition unit (c22) comprises one or more of the following;

(b27i) a second level 1 decomposition unit applying the multiresolution decomposition to the Image Y producing intermediate subbands at the level 1 for processing at level 2; and

(b27ii) a second level i decomposition unit applying the multiresolution decomposition to only selected intermediate subbands produced by the multiresolution decomposition performed at the level i−

1, with i ranging from 2 to N.

28. The system of claim 27, wherein:

(a28) the first decomposition unit (b22) comprises a first subband selection unit determining selected intermediate subbands based on an accuracy to be achieved in determining the measure of quality; and

(b28) the second decomposition unit (c22) comprises a second subband selection unit determining selected intermediate subbands based on an accuracy to be achieved in determining the measure of quality.

29. The system of claim 28, wherein:

(a29) the first subband selection unit (a28) further comprises a first subband selection sub-unit determining the selected intermediate subbands based on a number of the intermediate subbands required for achieving the accuracy; and

(b29) the second subband selection unit (b28) further comprises a second subband selection sub-unit determining the selected intermediate subbands based on a number of the intermediate subbands required for achieving the accuracy.

30. The system of claim 27, wherein:

(a30) the first detail subbands unit (e22) comprises a first aggregation unit aggregating one or more of the selected intermediate subbands of the image X produced by the multiresolution decomposition performed at each level j, with j ranging from 1 to N−

1; and

(b30) the second detail subbands unit (f22) comprises a second aggregation unit, aggregating one or more of the selected intermediate subbands of the image Y subbands produced by the multiresolution decomposition performed at each level j, with j ranging from 1 to N−

1.

31. The system of claim 30, wherein:

(a31) the first aggregation unit (a30) comprises a first detail subbands aggregation unit aggregating one or more of selected detail subbands of the image X produced at level N with said one or more of the selected intermediate subbands produced at said each level j; and

(b31) the second aggregation unit (b30) comprises a second detail subbands aggregation unit aggregating one or more of selected detail subbands of the image Y produced at level N with said one or more of the selected intermediate subbands produced at said each level J.

32. The system of claim 31, wherein:

(a32) the first aggregation unit (a30) comprises a first level j aggregation unit determining an edge map at the level j for the image X as a function of the square of said one or more of the selected intermediate subbands produced at said each level j;

(b32) the first detail subbands aggregation unit (a31) comprises;

(b32i) a first level N aggregation unit determining an edge map at the level N for the image X as a function of the square of said one or more of the selected detailed subbands of the image X produced at the level N; and

(b32ii) a first edge map unit determining the edge map of the image X as a weighted sum of edge maps at the level j and the edge map at the level N for the image X;

(c32) the second aggregation unit (b30), comprises a second level j aggregation unit determining an edge map at the level j for the image Y as a function of the square of said one or more of the selected intermediate subbands produced at said each level j; and

(d32) the second detail subbands aggregation unit (b31) comprises;

(d32i) a second level N aggregation unit determining an edge map at the level N for the image Y as a function of the square of said one or more of the selected detailed subbands of the image Y produced at the level N; and

(d32ii) a second edge map unit determining the edge map of the image Y as a weighted sum of edge maps at the level j and the edge map at the level N for the image Y.

33. The system of claim 23, wherein the approximation quality measure is an approximation quality map, and the edge quality measure is an edge quality map, and the quality measure unit (h22) further comprises:

(a33) a contrast map unit generating a contrast map, including assigning corresponding values to the pixels of the approximation subband and the edge map of the image X and the image Y according to their respective importance to human visual system;

(b33) a first pooling unit performing weighted pooling of the approximation quality map using the contrast map to produce an approximation quality score;

(c33) a second pooling unit performing weighted pooling of the edge quality map using the contrast map to produce an edge quality score; and

(d33) a score combination unit combining the approximation similarity score produced by the first poling unit (b33) with the edge similarity score produced by the second poling unit (c33) to determine the measure of quality.

34. The system of claim 23, wherein:

(a34) the approximation quality measure unit (d22) comprises an approximation SSIM map unit applying a SSIM IQM to the approximation subband of the image X and the approximation subband of the image Y to produce an approximation structural similarity (SSIM) map, SSIM_A;

(b34) the edge quality measure unit (g22) comprises an edge SSIM map unit applying the SSIM IQM between the edge map of the image X and the edge map of the image Y to produce an edge SSIM map, SSIM_E; and

(c34) the quality measure unit (h22) comprises a SSIM_DWT score unit processing the SSIM_A and the SSIM_E to determine a SSIM_DWT score as the measure of quality.

35. The system of claim 23, wherein:

(a35) the approximation quality measure unit (d22) comprises an approximation AD map unit applying an AD IQM to the approximation subband of the image X and the approximation subband of the image Y to produce an approximation AD map, AD_A;

(b35) the edge quality measure unit (g22) comprises an edge AD map unit applying the AD IQM between the edge map of the image X and the edge map of the image Y to produce an edge AD map, AD_E; and

(c35) the quality measure unit (h22) comprises an AD_DWT score unit processing the AD_A and the AD_E to determine an AD_DWT score as the measure of quality.

36. The system of claim 23, wherein:

(a36) the approximation quality measure unit (d22) comprises a PSNR approximation quality score unit applying a PSNR IQM to the approximation subband of the image X and the approximation subband of the image Y to produce a PSNR approximation quality score, PSNR_A;

(b36) the edge quality measure unit (g22) comprises a PSNR edge quality score determination unit applying the PSNR IQM between the edge map of the image X and the edge map of the image Y to produce a PSNR edge quality score, PSNR_E; and

(c36) the quality measure unit (h22) comprises a PSNR_DWT score unit processing the PSNR_A and the PSNR_E to determine a PSNR_DWT score as the measure of quality.

37. The system of claim 23, wherein:

(a37) the approximation quality measure unit (d22) comprises a VIF approximation quality score unit applying a VIF IQM to the approximation subband of the image X and the approximation subband of the image Y to produce a VIF approximation quality score, VIF_A;

(b37) the edge quality measure unit (g22) comprises a VIF edge quality score determination unit applying the VIF IQM between the edge map of the image X and the edge map of the image Y to produce a VIF edge quality score, VIF_E; and

(c37) the quality measure unit (h22) comprises a VIF_DWT score unit processing the VIF_A and the VIF_E to determine a VIF_DWT score as the measure of quality.

38. The system of claim 22, wherein the first decomposition unit (b22) and the second decomposition unit (c22) comprise computational means for applying a N level discrete wavelet transform.

39. The system of claim 38, wherein the discrete wavelet transform is a Haar transform.

40. The system of claim 38, wherein the discrete wavelet transform is one of a Newland transform, or wavelet transform using a Daubechies filter.

41. The system of claim 27, wherein the detail subbands of the image X include one or more of a horizontal subband containing horizontal edges, one or more of a vertical subband containing vertical edges and one or more of a diagonal subband containing diagonal edges of the image X, and wherein the detail subbands of the image Y include one or more of a horizontal subband containing horizontal edges, one or more of a vertical subband containing vertical edges, and one or more of a diagonal subband containing diagonal edges of the image Y.

42. The system of claim 31, wherein:

(a42) the first aggregation unit (a30) further comprises computational means for choosing the selected intermediate subbands at said each level j and the selected detail subbands for the image X having substantially same resolution for aggregating; and

(b42) the second aggregation unit (b30) further comprises computational means for choosing the selected intermediate subbands at said each level j and the selected detail subbands for the image Y having substantially same resolution for aggregating.

43. A computer readable storage medium, having computer readable program code instructions stored thereon, which, when executed by a processor, perform the following:

(a43) determining a number of levels of decomposition, N, to be applied to the image X and the image Y as a function of one or more of a resolution of the image Y, a viewing distance for the image Y, and an image quality metric (IQM) applied to the image Y;

(b43) applying a N level multiresolution decomposition, comprising levels 1, 2, . . . i, i+1, . . . N, to the image X, producing an approximation subband containing main content of the image X and detail subbands containing edges of the image X;

(c43) applying said N level multiresolution decomposition, comprising levels 1, 2, . . . i, i+1, . . . N, to the image Y, producing an approximation subband containing main content of the image Y and detail subbands containing edges of the image Y;

(d43) applying the IQM to the approximation subband of the image X and the approximation subband of the image Y to produce an approximation quality measure, characterizing similarity between the main content of the image X and the main content of the image Y;

(e43) processing the detail subbands of the image X producing an edge map of the image X, characterizing the edges of the image X;

(f43) processing the detail subbands of the image Y producing an edge map of the image Y, characterizing the edges of image Y;

(g43) applying the IQM between the edge map of the image X and the edge map of the image Y to produce an edge quality measure, characterizing similarity between the edges of the image X and the edges of the image Y; and

(h43) processing the approximation quality measure and the edge quality measure to determine the measure of quality.