METHOD AND SYSTEM FOR DETERMINING STRUCTURAL SIMILARITY BETWEEN IMAGES

• US 20130094769A1
• Filed: 12/03/2012
• Published: 04/18/2013
• Est. Priority Date: 02/11/2009
• Status: Active Grant

• Current Assignee: Ecole de technologie superieure
• Original Assignee: Ecole de technologie superieure
• Inventors: REZAZADEH, Soroosh, COULOMBE, Stphane
• Granted Patent: US 8,532,396 B2
• Time in Patent Office: Days
• US Class Current: 382/199
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/20052 Discrete cosine transform [DCT]
• CPC Class Codes: G06T 2207/20064 Wavelet transform [DWT]
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection

Abstract

Method and system for low complexity assessment of quality of an image are presented. By performing multiresolution decomposition of images using, for example, a discrete wavelet transform, and determining a metric based on a structural similarity index or a structural similarity map, a structural similarity score, characterizing similarity between images with a high degree of accuracy, is produced. The processing time is much smaller in comparison to that required by other methods producing image quality metrics of comparable accuracy.

Claims

1. A system for determining a measure of similarity between an image X and an image Y, the system comprising:

a processor; and

a non-transitory computer readable storage medium having computer readable instructions stored thereon, causing the processor to;

(a) apply a multiresolution decomposition to the image X, producing an approximation subband containing main content of the image X and detail subbands containing fine edges of the image X;

(b) apply said multiresolution decomposition to the image Y, producing an approximation subband containing main content of the image Y and detail subbands containing fine edges of the image Y;

(c) determine an approximation similarity measure, characterizing similarity between the main content of the image X and the main content of the image Y, comprising processing the approximation subband of the image X and the approximation subband of the image Y;

(d) aggregate the detail subbands of the image X, producing an aggregate for the image X;

(e) aggregate the detail subbands of the image Y, producing an aggregate for the image Y;

(f) determine an edge similarity measure, characterizing similarity between the fine edges of the image X and the fine edges of the image Y, comprising processing the aggregate for the image X and the aggregate for the image Y; and

(g) determine the measure of similarity between the image X and the image Y, comprising processing the approximation similarity measure and the edge similarity measure.

2. The system of claim 1, wherein the measure of similarity comprises a Structural Similarity (SSIM) score between the image X and the image Y.

3. The system of claim 1, wherein the computer readable instructions further cause the processor to apply a discrete wavelet transform (DWT) or a discrete cosine transform (DCT).

4. The system of claim 3, wherein the computer readable instructions further cause the processor to determine a Wavelet Structural Similarity (WSSIM) score between the image X and the image Y provided the DWT has been applied, or a Cosine Structural Similarity (CSSIM) score provided the DCT has been applied.

5. The system of claim 2, wherein the computer readable instructions further cause the processor to determine a structural similarity map SSIM_A characterizing a difference between pixels in the approximation subband of the image X and corresponding pixels in the approximation subband of the image Y.

6. The system of claim 1, wherein the computer readable instructions further cause the processor to determine the approximation similarity measure, comprising a structural similarity index using Visual Information Fidelity (VIF) information, the structural similarity index characterizing a difference between pixels in the approximate subband of the image X and corresponding pixels in the approximation subband of the image Y.

7. The system of claim 5, wherein the computer readable instructions further cause the processor to determine the edge similarity measure, comprising an edge structural similarity map SSIM_E characterizing a difference between pixels in the aggregate for the image X and corresponding pixels in the aggregate for the image Y.

8. The system of claim 4, wherein the computer readable instructions further cause the processor to determine the edge similarity measure, comprising an edge structural similarity index using Visual Information Fidelity (VIF) information, the edge structural similarity index characterizing a difference between pixels in the aggregate for the image X and corresponding pixels in the aggregate for the image Y.

9. The system of claim 7, wherein the computer readable instructions further cause the processor to process the SSIM_A and the SSIM_E to obtain the SSIM score.

10. The system of claim 1, wherein the detail subbands of the image X include one or more of a horizontal subband containing horizontal fine edges, a vertical subband containing vertical fine edges and a diagonal subband containing diagonal fine edges of the image X, and wherein the detail subbands of the image Y include one or more of a horizontal subband containing horizontal fine edges, a vertical subband containing vertical fine edges, and a diagonal subband containing diagonal fine edges of the image Y.

11. The system of claim 9, wherein the detail subbands of the image X include one or more of a horizontal subband containing horizontal fine edges, a vertical subband containing vertical fine edges and a diagonal subband containing diagonal fine edges of the image X, and wherein the detail subbands of the image Y include one or more of a horizontal subband containing horizontal fine edges, a vertical subband containing vertical fine edges, and a diagonal subband containing diagonal fine edges of the image Y.

12. The system of claim 1, wherein the computer readable instructions further cause the processor to:

generate an edge map for the image X, including calculating weighted norm of the detail subbands of the image X; and

generate an edge map for the image Y, including calculating weighted norm of the detail subbands of the image Y.

13. The system of claim 1, wherein the computer readable instructions further cause the processor to:

generate a contrast map, including assigning corresponding values to the pixels of the image X and the image Y according to their respective importance to human visual system;

perform weighted pooling of the approximation similarity measure by using the contrast map to produce an approximation similarity index;

perform weighted pooling of the edge similarity measure by using the contrast map to produce an edge similarity index; and

combine the approximation similarity index with the edge similarity index to determine the measure of similarity.

14. The system of claim 7, wherein the computer readable instructions further cause the processor to:

generate a contrast map, including assigning corresponding values to the pixels of the image X and the image Y according to their respective importance to human visual system;

perform weighted pooling of the SSIM_A by using the contrast map to produce an approximation similarity index;

perform weighted pooling of the SSIM_E by using the contrast map to produce an edge similarity index; and

combine the approximation similarity index with the edge similarity index to determine the SSIM score.

15. The system of claim 1, wherein the computer readable instructions further cause the processor to:

apply a low-pass filter and a high-pass filter on the image X; and

apply a low-pass filter and a high-pass filter on the image Y.

16. The system of claim 13, wherein the computer readable instructions further cause the processor to perform one or more of the following:

to apply a low-pass filter horizontally and a high-pass filter vertically on the image X;

to apply a high-pass filter vertically and a high-pass filter horizontally on the image X;

to apply a low-pass filter vertically and a high-pass filter horizontally on the image X; and

to apply a low-pass filter horizontally and a low-pass filter vertically on the image X.

17. The system of claim 15, wherein the computer readable instructions further cause the processor to:

apply a low-pass filter horizontally and a high-pass filter vertically on the image Y;

apply a high-pass filter vertically and a high-pass filter horizontally on the image Y;

apply a low-pass filter vertically and a high-pass filter horizontally on the image Y; and

apply a low-pass filter horizontally and a low-pass filter vertically on the image Y.

18. The system of claim 2, wherein the discrete wavelet transform is one of a Haar transform, a Newland transform, or a wavelet transform using a Daubechies filter.

19. A method for determining a measure of similarity between an image X and an image Y, the method comprising:

(a) applying a multiresolution decomposition to the image X, producing an approximation subband containing main content of the image X and detail subbands containing fine edges of the image X;

(b) applying the multiresolution decomposition to the image Y, producing an approximation subband containing main content of the image Y and detail subbands containing fine edges of the image Y;

(c) determining an approximation similarity measure, characterizing similarity between the main content of the image X and the main content of the image Y, comprising processing the approximation subband of the image X and the approximation subband of the image Y;

(d) aggregating the detail subbands of the image X, producing an aggregate for the image X;

(e) aggregating the detail subbands of the image Y, producing a two dimensional discrete wavelet decomposition aggregate for the image Y;

(f) determining an edge similarity measure, characterizing similarity between the fine edges of the image X and the fine edges of the image Y, comprising processing the aggregate for the image X and the aggregate for the image Y; and

(g) determining the measure of similarity between the image X and the image Y, comprising processing the approximation similarity measure and the edge similarity measure.

20. The method of claim 19, wherein the step (a) comprises applying a two dimensional discrete wavelet decomposition, and the step (g) comprises determining a Wavelet Structural Similarity (WSSIM) score as the measure of similarity.