System and method of measuring fidelity of decompressed video signals and images

• US 5,329,379 A
• Filed: 10/22/1992
• Issued: 07/12/1994
• Est. Priority Date: 10/22/1992
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Rodriguez, Arturo A., Pietras, Mark A., Saenz, Andres J.
• Primary Examiner(s): Coles, Sr., Edward L.
• Assistant Examiner(s): VU, KIM Y
• Application Number:
  US07/965,649
• Time in Patent Office: 628 Days
• Field of Search: 358/454, 358/455, 358/456, 358/457, 358/458, 358/462, 358/533, 358/534, 358/535, 358/538, 358/433, 358/428, 382/19, 382/28, 382/56, 395/128, 395/131, 395/132, 348/396
• US Class Current: 382/167
• CPC Class Codes: H04N 17/02 for colour television signals
• CPC Class Codes: H04N 19/00 Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
• CPC Class Codes: H04N 19/154 Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion use of rate-distortion criteria H04N19/147
• CPC Class Codes: H04N 19/60 using transform coding

Abstract

A method of quantitatively measuring fidelity of a reproduced image reconstructed from a compressed data representation of an original image is disclosed. The method comprises, responsive to user selection, for establishing a global assessment mode or a local assessment mode. In the global assessment mode changes in luminance of the reproduced image from the original image and changes in color in first and second color difference values of the reproduced image from the original image are used score fidelity. Changes in luminance are measured using a dynamic range, nonlinear transform equation. In the local assessment mode, and responsive to user selection, the reproduced image and the original image are segmented and corresponding pairs of segments from the reproduced image and the original image are identified. Scoring of fidelity of the reproduced image to the original image is done by comparing corresponding pairs of segments in color, luminance, shape, displacement and texture.

Claims

1. A method of quantitatively measuring fidelity of a reproduced image to an original image, the method comprising the steps of:

capturing and digitizing the original image;

compressing the original image into a compressed data representation;

decompressing the compressed data representation to produce the reproduced image;

responsive to user selection, establishing a global assessment mode or a local assessment mode;

in the global assessment mode, determining luminance fidelity of the reproduced image to the original image and determining color fidelity over first and second color difference values of the reproduced image to the original image;

in the local assessment mode and responsive to user selection, segmenting the reproduced image and the original image and establishing corresponding pairs of segments having one segment each from the reproduced image and the original image; and

in the local assessment mode, scoring fidelity of segments of the reproduced image by comparing corresponding pairs of segments from the reproduced image and original image in color, luminance, shape and displacement.

2. A method of quantitatively measuring fidelity of a reproduced image reconstructed from a compressed data representation of an original image as set forth in claim 1, wherein the step in the local assessment mode of segmenting the reproduced image and the original image and establishing corresponding pairs of segments having one segment from the reproduced image and the original image further includes:

prompting the user to select segments by drawing an enclosure around areas in the images; and

executing an automatic edge detection process in a neighborhood around user drawn enclosures and setting the border of the selected segments equal to a detected edge.

3. A method of quantitatively measuring fidelity of a reproduced image reconstructed from a compressed data representation of an original image as set forth in claim 1, wherein the step in the local assessment mode of scoring fidelity of the reproduced image to the original image by corresponding pairs of segments further includes comparing texture of the segment from the reproduced image to a corresponding segment in the original image.

4. A method of quantitatively measuring fidelity of a reproduced image reconstructed from a compressed data representation of an original image as set in claim 3, and further comprising the steps of:

responsive to user selection, characterizing the segments of a corresponding pair of segments as textured or nontextured.

5. A method of quantitatively measuring fidelity of a reproduced image reconstructed from a compressed data representation of an original image as set forth in claim 4, wherein the step in the local assessment mode of scoring fidelity of the reproduced image to the original image by comparing corresponding pairs of segments in color and luminance further includes:

for a textured corresponding pair of segments, determining an average luminance difference between the segments and determining an average change in color for first and second color difference components;

for a nontextured corresponding pair of segments, determining luminance change between a pair of segments by computing the root mean square value over the luminance changes between pixels in the respective segments; and

for a nontextured corresponding pair of segments, determining color change between a pair of segments by filtering color difference values for subgroups of pixels within the segments and computing the root mean square value of the changes in the color difference values between filtered subgroups of pixels.

6. A method of quantitatively measuring fidelity of a reproduced image reconstructed from a compressed data representation of an original image, as set forth in claim 1, wherein in the global assessment mode the step of determining luminance fidelity of the reproduced image to the original image and determining color fidelity over first and second color difference values of the reproduced image to the original image includes:

determining luminance change between the reproduced image and the original image by computing the root mean square value over the differences between pixels in the respective images; and

determining color change between the reproduced image and the original image by filtering color difference values for subgroups of pixels in the images and computing the root mean square value of the changes in the color difference values by comparing filtered subgroups of pixels.

7. A method of quantitatively measuring fidelity of a reproduced image reconstructed from a compressed data representation of an original image as set forth in claim 5, wherein luminance is determined over a luminance range extending from minimum to maximum luminance intensity in the corresponding pair of segments and wherein the transformation is logarithmic of the form:

##EQU2## where [Y_min, Y_max] is the dynamic luminance scale of the data and where P gives L(Y) a resolution equal to a fraction of the original dynamic range and thus controls how luminance differences between the original image frame and the reconstructed frame are penalized.

8. method of quantitatively measuring fidelity of a reproduced image reconstructed from a compressed data representation of an original image as set forth in claim 6, wherein luminance is determined over a luminance range extending from minimum to maximum luminance intensity in the reproduced and original images and wherein the transformation is logarithmic of the form:

##EQU3## where [Y_min, Y_max] is the dynamic luminance scale of the data and where P gives L(Y) a resolution equal to a fraction of the original dynamic range and thus control how luminance differences between the original image frame and the reconstructed frame are penalized.

9. A method of quantitatively measuring fidelity of a reproduced image reconstructed from a compressed data representation of an original image as set forth in claim 7 and further including:

generating distortion curves based on fidelity scores.

19. A method as set forth in claim 1, and further comprising the steps of:

in the local assessment mode, weighing fidelity scoring for the segments by location in the reproduced image; and

summing the weighted scores to generate an overall fidelity score.

10. Apparatus for quantitatively measuring fidelity of a reproduced image with an original image, comprising:

means for capturing and digitizing the original image;

means for compressing the original image into a compressed data representation;

means for decompressing the compressed data representation to produce the reproduced image;

means for determining luminance fidelity of the reproduced image to the original image and for determining color fidelity from changes in first and second color difference values in the reproduced image from the original image;

means responsive to user selection for segmenting the reproduced image and the original image and establishing corresponding pairs of segments having one segment each from the reproduced image and the original image; and

means for scoring fidelity of the reproduced image to the original image by comparing corresponding pairs of segments in color, luminance, shape and displacement of a segment from the reproduced image compared to a corresponding segment in the original image to score the segments in the reproduced image.

11. Apparatus for quantitatively measuring fidelity of a reproduced image reconstructed from a compressed data representation of an original image as set forth in claim 10, wherein the means for segmenting the reproduced image and the original image and for establishing corresponding pairs of segments having one segment from the reproduced image and from the original image further include:

means for prompting the user to select segments by drawing an enclosure around the segments; and

means for automatically detecting and edge in a neighborhood around user drawn enclosure and setting the border of a selected segment equal to a detected edge.

12. Apparatus for quantitatively measuring fidelity of a reproduced image reconstructed from a compressed data representation of an original image as set forth in claim 10, wherein means for scoring fidelity of the reproduced image to the original image by corresponding pairs of segments further include means for comparing texture of the segment from the reproduced image to a corresponding segment in the original image.

13. Apparatus for quantitatively measuring fidelity of a reproduced image reconstructed from a compressed data representation of an original image as set in claim 12, and further comprising:

means responsive to user selection for identifying the segments of a corresponding pair of segments as textured or nontextured.

14. Apparatus for quantitatively measuring fidelity of a reproduced image reconstructed from a compressed data representation of an original image as set forth in claim 13, wherein the means for scoring fidelity of the reproduced image to the original image by comparing corresponding pairs of segments in color and by luminance further includes:

means for determining for a textured corresponding pair of segments an average luminance difference between the segments and for determining an average change in color for first and second color difference components;

means for determining for a nontextured corresponding pair of segments luminance change between a pair of segments by computing the root mean square value over the luminance changes between pixels in the respective segments; and

means for determining for a nontextured corresponding pair of segments color change between a pair of segments by filtering color difference values for subgroups of pixels within the segments and for computing the root means square value of the changes in the color difference values between filtered subgroups of pixels.

15. Apparatus for quantitatively measuring fidelity of a reproduced image reconstructed from a compressed data representation of an original image, as set forth in claim 10, wherein the means for determining luminance fidelity of the reproduced image to the original image and the means for determining color fidelity over first and second color difference values of the reproduced image to the original image include:

means for finding luminance change between the reproduced image and the original image by computing the root means square value over the differences between pixels in the respective images; and

means for finding color change between the reproduced image and the original image by filtering color difference values for subgroups of pixels in the image and for computing the root means square value of the changes in the color difference values by comparing filtered subgroups of pixels.

16. Apparatus for quantitatively measuring fidelity of a reproduced image reconstructed from a compressed data representation of an original image as set forth in claim 14, wherein luminance is determined over a luminance range extending from minimum to maximum luminance intensity in the corresponding pair of segments and wherein the transformation is logarithmic of the form:

##EQU4## where [Y_min, Y_max] is the dynamic luminance scale of the data and where P gives L(Y) a resolution equal to a fraction of the original dynamic range and thus controls how luminance differences between the original image frame and the reconstructed frame are penalized.

17. Apparatus for quantitatively measuring fidelity of a reproduced image reconstructed from a compressed data representation of an original image as set forth in claim 15, wherein luminance is determined over a luminance range extending from minimum to maximum luminance intensity in the reproduced and original images and wherein the transformation is logarithmic of the form:

##EQU5## where [Y_min, Y_max] is the dynamic luminance scale of the data and where P gives L(Y) a resolution equal to a fraction of the original dynamic range and thus controls how luminance differences between the original image frame and the reconstructed frame are penalized.

18. Apparatus for quantitatively measuring fidelity of a reproduced image reconstructed from a compressed data representation of an original image as set forth in claim 16 and further including:

means for generating distortion curves based on fidelity scores.

20. Apparatus as set forth in claim 10, and further comprising:

means for weighing fidelity scores for segments by location in the reproduced image; and

means for summing the weighted fidelity scores.