Method and system for increasing robustness of visual quality metrics using spatial shifting

• US 8,983,206 B2
• Filed: 05/03/2012
• Issued: 03/17/2015
• Est. Priority Date: 05/04/2011
• Status: Active Grant

• Current Assignee: Ecole de technologie superieure
• Original Assignee: Ecole De Techbologie Superieure
• Inventors: Coulombe, Stephane, Rezazadeh, Soroosh
• Primary Examiner(s): Bella, Matthew
• Assistant Examiner(s): TORRES, JOSE
• Application Number:
  US13/463,733
• Publication Number:
  US 20120281924A1
• Time in Patent Office: 1,048 Days
• Field of Search: None
• US Class Current: 382/209
• CPC Class Codes: G06T 2207/20064 Wavelet transform [DWT]
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression

Abstract

A visual quality assessment method and system are based on deriving a quality metric by comparing sub-band approximations of a distorted image and an undistorted version of the same image, providing a good compromise between computational complexity and accuracy. The sub-band approximations are derived from Discrete Wavelet (Haar) transforms of small image blocks of each image. Due to inherent symmetries, the wavelet transform is “blind” to certain types of distortions. But the accuracy of the method is enhanced, and the blindness of the transform is overcome, by computing quality metrics for the distorted image as well as computing quality metrics for a shifted version of the distorted image and combining the results.

Claims

1. A method of computing a measure of quality for a distorted image Y, the measure characterizing a similarity between the image Y and an undistorted reference image X having the same number of rows and columns of pixels as the image Y, the method comprising:

employing at least one processor for;

(a) transforming the distorted image Y and the undistorted reference image X by a transformation process to generate a transform of the distorted image Y and a transform of the undistorted reference image X respectively, the transformation process having a blindness property of not containing all information regarding an image to be transformed in a transform of the image, thus resulting in certain pixel patterns of the image leading to zero transform coefficients not to be recognized during the transformation process;

(b) spatially shifting pixels in the reference image X and the distorted image Y by a spatial shift, having the same predefined amount of pixels, to generate a spatially shifted version X2 of the reference image X and a spatially shifted version Y2 of the image Y respectively, the spatial shift being chosen so that the certain pixel patterns in images X and Y leading to zero transform coefficients upon applying the transformation process to the images X and Y would lead to non zero values for the transform coefficients upon applying the transformation process to the images X2 and Y2, thereby making the certain patterns detectable by using the transformation process applied to the spatially shifted versions X2 and Y2;

(c) transforming the spatially shifted version Y2 and the spatially shifted version X2 by the transformation process to generate a transform of the spatially shifted version Y2 and a transform of the spatially shifted version X2 respectively; and

(d) determining the measure of quality for the distorted image Y as a function of the transform of the distorted image Y, the transform of the undistorted reference image X, the transform of the spatially shifted version Y2, and the transform of the spatially shifted version X2.

2. The method of claim 1, wherein the step (b) further comprises:

dividing the pixels of the reference image X into blocks of W by W pixels; and

dividing the pixels of the distorted image Y into corresponding blocks of W by W pixels;

W being a smaller number than the number of rows and columns of pixels.

3. The method of claim 1, wherein the step (b) further comprises:

spatially shifting horizontally by a first predefined amount;

or

spatially shifting vertically by a second predefined amount;

or

spatially shifting horizontally and vertically by first and second predefined amounts respectively.

4. The method of claim 3, wherein the first predefined amount and the second predefined amount are the same.

5. The method of claim 2, wherein the W is equal to 2 ^ N, wherein an exponent N is a decomposition level of the transformation process.

6. The method of claim 3, wherein each of the first and second predefined amounts are the larger of 1 and (W/2−

1).

7. The method of claim 3, wherein the first and second predefined amounts are odd numbers less than W.

8. The method of claim 1, wherein the transformation process is one of the following:

a cosine transform;

a sine transform;

a wavelet transform.

9. The method of claim 1, wherein the transformation process is a Haar transform or a Daubechies transform.

10. The method of claim 1, wherein the step (d) further comprises:

computing a first approximation image quality metric IQMA using the transform of the distorted image Y and the transform of the undistorted reference image X;

computing a second approximation image quality metric IQM′

A using the transform of the spatially shifted version Y2 and the transform of the spatially shifted version X2; and

processing the first and second approximation image quality metrics IQMA and IQM′

A to generate the measure of quality for the distorted image Y.

11. The method of claim 10, wherein the first and second approximation quality metrics IQMA and IQM′

A respectively are first and second approximation quality maps, and the step (d) further comprises;

creating first and second contrast maps of the reference image X and of the spatially shifted version X2 respectively;

performing weighted pooling of the first and second approximation quality maps using the first and second contrast maps respectively to produce an approximation quality score SA; and

determining the measure of quality using the approximation quality score SA.

12. The method of claim 10, wherein the step (d) further comprises:

(i) computing a first edge image quality metric IQME using the transform of the distorted image Y and the transform of the undistorted reference image X;

(ii) computing a second edge image quality metric IQM′

E using the transform of the spatially shifted version Y2 and the transform of the spatially shifted version X2; and

(iii) determining the measure of quality for the distorted image Y as the function of IQMA, IQM′

A, IQME and IQM′

E.

13. The method of claim 12, wherein the step (iii) further comprises:

processing the first and second approximation image quality metrics IQMA and IQM′

A to generate an approximation quality score SA;

processing the first and second edge quality metrics IQME and IQM′

E to generate an edge quality score SE; and

determining the measure of quality for the distorted image Y as a function of the approximation quality score SA and the edge quality score SE.

14. The method of claim 13, wherein the measure of quality is computed as a final score SF=beta * SA+(1−

beta) * SE, with beta value ranging from 0 to 1.

15. The method of claim 12, wherein the first and second approximation quality metrics IQMA and IQM′

A respectively are first and second approximation quality maps, wherein further the first and second edge quality metrics IQME and IQM′

E respectively are first and second edge quality maps, and the step (d) further comprises;

(d1) creating first and second contrast maps of the reference image X and of the spatially shifted version X2 respectively;

(d2) performing weighted pooling of the first and second approximation quality maps using the first and second contrast maps respectively to produce an approximation quality score SA;

(d3) performing weighted pooling of the first and second edge quality maps using the first and second contrast maps respectively to produce an edge quality score SE; and

(d4) processing the approximation quality score SA and the edge quality score SE to determine the measure of quality.

16. The method of claim 15, the step of creating first and second contrast maps further comprises assigning values to pixels of the first and second approximation quality maps, and the first and second edge maps of the reference image X and the spatially shifted version X2 according to their respective importance to a human visual system.

17. The method of claim 15, wherein the step (d2) further comprises one of averaging the first approximation quality map and the second approximation quality map, and of taking the minimum of the first approximation quality map and the second approximation quality map to produce the approximation quality score SA.

18. A system for computing a measure of quality for a distorted image Y, the measure characterizing a similarity between the image Y and an undistorted reference image X having the same number of rows and columns of pixels as the image Y, the system comprising:

a processor;

a non-transitory computer readable storage medium having computer readable instructions stored thereon for execution by the processor, causing the processor to;

(a) transform the distorted image Y and the undistorted reference image X by a transformation process to generate a transform of the distorted image Y and a transform of the undistorted reference image X respectively, the transformation process having a blindness property of not containing all information regarding an image to be transformed in a transform of the image, thus resulting in certain patterns of the image leading to zero transform coefficients not to be recognized during the transformation process;

(b) spatially shift pixels in the reference image X and the distorted image Y by a spatial shift, having the same predefined amount of pixels, to generate a spatially shifted version X2 of the reference image X and a spatially shifted version Y2 of the image Y respectively, the spatial shift being chosen so that the certain pixel patterns in images X and Y leading to zero transform coefficients upon applying the transformation process to the images X and Y would lead to non zero values for the transform coefficients upon applying the transformation process to the images X2 and Y2, thereby making, the certain patterns detectable by using the transformation process applied to the spatially shifted versions X2 and Y2;

(c) transform the spatially shifted version Y2 and the spatially shifted version X2 by the transformation process to generate a transform of the spatially shifted version Y2 and a transform of the spatially shifted version X2 respectively; and

(d) determine the measure of quality for the distorted image Y as a function of the transform of the distorted image Y, the transform of the undistorted reference image X, the transform of the spatially shifted version Y2, and the transform of the spatially shifted version X2.

19. The system of claim 18, wherein the computer readable instructions further cause the processor to:

divide the pixels of the reference image X into blocks of W by W pixels, W being a smaller number than the number of rows and columns of pixels; and

divide the pixels of the distorted image Y into corresponding blocks of W by W pixels.

20. The system of claim 19, wherein the W is equal to 2 ^ N, wherein an exponent N is a decomposition level of the transformation process.

21. The system of claim 18, wherein the computer readable instructions further cause the processor to:

spatially shift horizontally by a first predefined amount;

or

spatially shift vertically by a second predefined amount;

or

spatially shift horizontally and vertically by first and second predefined amounts respectively.

22. The system of claim 21, wherein the first predefined amount and the second predefined amount are the same.

23. The system of claim 21, wherein each of the first and second predefined amounts are the larger of 1 and (W/2−

1).

24. The system of claim 21, wherein the first and second predefined amounts are odd numbers less than W.

25. The system of claim 18, wherein the transformation process is one of the following:

a cosine transform;

a sine transform;

a wavelet transform.

26. The system of claim 18, wherein the transformation process is a Haar transform or a Daubechies transform.

27. The system of claim 18, wherein the computer readable instructions further cause the processor to:

compute a first approximation image quality metric IQMA using the transform of the distorted image Y and the transform of the undistorted reference image X;

compute a second approximation image quality metric IQM′

A using the transform of the spatially shifted version Y2 and the transform of the spatially shifted version X2; and

process the first and second approximation image quality metrics IQMA and IQM′

A to generate the measure of quality for the distorted image Y.

28. The system of claim 27, wherein the first and second approximation quality metrics IQMA and IQM′

A respectively are first and second approximation quality maps, and wherein the computer readable instructions further cause the processor to;

create first and second contrast maps of the reference image X and of the spatially shifted version X2 respectively;

perform weighted pooling of the first and second approximation quality maps using the first and second contrast maps respectively to produce an approximation quality score SA; and

determine the measure of quality using the approximation quality score SA.

29. The system of claim 27, wherein the computer readable instructions further cause the processor to:

(i) compute a first edge image quality metric IQME using the transform of the distorted image Y and the transform of the undistorted reference image X;

(ii) compute a second edge image quality metric IQM′

E using the transform of the spatially shifted version Y2 and the transform of the spatially shifted version X2; and

(iii) determine the measure of quality for the distorted image Y as the function of IQMA, IQM′

A, IQME and IQM′

E.

30. The system of claim 29, wherein the computer readable instructions further cause the processor to:

process the first and second approximation image quality metrics IQMA and IQM′

A to generate an approximation quality score SA;

process the first and second edge quality metrics IQME and IQM′

E to generate an edge quality score SE; and

determine the measure of quality for the distorted image Y as a function of the approximation quality score SA and the edge quality score SE.

31. The system of claim 30, wherein the measure of quality is computed as a final score SF=beta * SA+(1−

beta) * SE , with beta value ranging from 0 to 1.

32. The system of claim 29, wherein the first and second approximation quality metrics IQMA and IQM′

A respectively are first and second approximation quality maps, wherein further the first and second edge quality metrics IQME and IQM′

E respectively are first and second edge quality maps, and wherein the computer readable instructions further cause the processor to;

create first and second contrast maps of the reference image X and of the spatially shifted version X2 respectively;

perform weighted pooling of the first and second approximation quality maps using the first and second contrast maps respectively to produce an approximation quality score SA;

perform weighted pooling of the first and second edge quality maps using the first and second contrast maps respectively to produce an edge quality score SE; and

process the approximation quality score SA and the edge quality score SE to determine the measure of quality.

33. The system of claim 32, wherein the computer readable instructions further cause the processor to assign values to pixels of the first and second approximation quality maps, and the first and second edge maps of the reference image X and the spatially shifted version X2 according to their respective importance to a human visual system.

34. The system of claim 32, wherein the computer readable instructions further cause the processor to perform one of averaging the first approximation quality map and the second approximation quality map, and of taking the minimum of the first approximation quality map and the second approximation quality map to produce the approximation quality score SA.