Method and apparatus for evaluating the visual quality of processed digital video sequences

• US 6,493,023 B1
• Filed: 03/12/1999
• Issued: 12/10/2002
• Est. Priority Date: 03/12/1999
• Status: Expired due to Fees

• Current Assignee: Administrator of The National Aeronautics and Space Administration
• Original Assignee: United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration
• Inventors: Watson, Andrew B.
• Primary Examiner(s): Miller, John
• Assistant Examiner(s): Natnael, Paulos M.
• Application Number:
  US09/266,962
• Time in Patent Office: 1,369 Days
• Field of Search: 348/180, 348/184, 348/192, 348/438.1, 348/455, 348/416.1, 348/422.1, 382/162, 382/167, 382/254, 382/260, 382/276, 382/282, 382/286, 702/69, 702/81
• US Class Current: 348/180
• CPC Class Codes: H04N 17/00 Diagnosis, testing or measuring for television systems or their details
• CPC Class Codes: H04N 17/004 for digital television systems
• CPC Class Codes: H04N 17/02 for colour television signals

Abstract

A Digital Video Quality (DVQ) apparatus and method that incorporate a model of human visual sensitivity to predict the visibility of artifacts. The DVQ method and apparatus are used for the evaluation of the visual quality of processed digital video sequences and for adaptively controlling the bit rate of the processed digital video sequences without compromising the visual quality. The DVQ apparatus minimizes the required amount of memory and computation. The input to the DVQ apparatus is a pair of color image sequences: an original (R) non-compressed sequence, and a processed (T) sequence. Both sequences (R) and (T) are sampled, cropped, and subjected to color transformations. The sequences are then subjected to blocking and discrete cosine transformation, and the results are transformed to local contrast. The next step is a time filtering operation which implements the human sensitivity to different time frequencies. The results are converted to threshold units by dividing each discrete cosine transform coefficient by its respective visual threshold. At the next stage the two sequences are subtracted to produce an error sequence. The error sequence is subjected to a contrast masking operation, which also depends upon the reference sequence (R). The masked errors can be pooled in various ways to illustrate the perceptual error over various dimensions, and the pooled error can be converted to a visual quality measure.

Claims

1. A digital video quality method for evaluating the visual quality of a processed (T) video sequence relative to an original (R) video sequence, the method comprising:

sampling the original and processed video sequences to generate sampled sequences (d₁) therefrom;

limiting the processing of said sampled sequences (d₁) to a region of interest and generating region of interest sequences (d₂) therefrom;

transforming said region of interest sequences (d₂) to local contrast coefficients (d₁₇);

filtering said local contrast coefficients (d₁₇) to generate filtered components (d₁₈) therefrom;

converting said filtered components (d₁₈) to threshold units (d₁₉);

subtracting said threshold units (d₁₉) corresponding to the original (R) and processed (T) sequences to obtain an error sequence (d₂₀);

subjecting said error sequence (d₂₀) to a contrast masking operation to generate a masked error sequence (d₂₄) therefrom; and

pooling said masked error sequence (d₂₄) to generate a perceptual error (E_Ω

).

2. A method according to claim 1, further including converting said perceptual error (E_Ω

) to a visual quality measure (Q_Ω

), to provide an output in terms of quality.

3. A method according to claim 1, further including feeding back said perceptual error (E_Ω

) to a codec, for regulating a compression bit rate to correspond to a desired image visual quality.

4. A method according to claim 2, further including feeding back said visual quality measure (Q_Ω

) to a codec, for regulating a compression bit rate to correspond to a desired image visual quality.

5. A method according to claim 1, wherein each of said processed (T) video sequence and said original (R) video sequence includes color channels;

and

wherein said color channels are converted by a color transformer to a perceptually relevant color space, to generate color transformed sequences (d₉) from said region of interest sequences (d₂).

6. A method according to claim 5, further including subjecting said color transformed sequences (d₉) to blocking to generates blocks.

7. A method according to claim 6, further including converting said blocks to a block of frequency coefficients (d₁₀) by means of a discrete cosine transformer.

8. A method according to claim 7, wherein said block of frequency coefficients (d₁₀) are converted to a local contrast signal (d₁₇) by means of a local contrast converter;

and

wherein said local contrast signal (d₁₇) includes a combination of AC coefficients (d_17a) and DC coefficients (d_17b).

9. A method according to claim 8, wherein contrast masking is accomplished by rectifying said threshold units (d₁₉).

10. A method according to claim 5, wherein said region of interest sequences (d₂) are transformed from their native color space to gamma-corrected color channels R′

, G′

, and B′

by a R′

G′

B′

transformer.

11. A method according to claim 10, further including converting said color channels R′

, G′

, and B′

to RGB color channels by a RGB transformer.

12. A method according to claim 11, further including converting said RGB color channels to XYZ color coordinates by a XYZ transformer.

13. A method according to claim 12, further including converting said XYZ color coordinates to YOZ color coordinates by a YOZ transformer.

14. A method according to claim 13, wherein if any of the processed (T) video sequence or the original (R) video sequence contains interlaced video fields, then de-interlacing said interlaced fields to a progressive sequence (d₇) by means of a de-interlacer.

15. A method according to claim 14, wherein de-interlacing is implemented by inserting blank lines into even numbered lines in odd fields, and odd numbered lines in even fields.

16. A method according to claim 14, wherein de-interlacing is implemented by inserting blank lines into even numbered lines in even fields, and odd numbered lines in odd fields.

17. A method according to claim 14, wherein de-interlacing is implemented by each pair of odd and even video fields as an image.

18. A method according to claim 14, further including adding a veiling light to said progressive sequence (d₇) by means of a veiling light combiner.

19. A method according to claim 1, wherein sampling includes pixel-replication.

20. A digital video quality apparatus with an original (R) video sequence and a processed (T) video sequence being fed thereto, the apparatus comprising:

a sampler for sampling the original and processed video sequences to generate sampled sequences (d₁) therefrom;

a region-of-interest processor for limiting the processing of said sampled sequences (d₁) to a region of interest and for generating region of interest sequences (d₂) therefrom;

a local contrast converter for transforming said region of interest sequences (d₂) to local contrast coefficients (d₁₇);

a time filter for filtering said local contrast coefficients (d₁₇) and for generating filtered components (d₁₈) therefrom;

a threshold scaler for converting said filtered components (d₁₈) to threshold units (d₁₉);

a subtractor for subtracting said threshold units (d₁₉) corresponding to the original (R) and processed (T) sequences to obtain an error sequence (d₂₀);

a contrast masking processor for subjecting said error sequence (d₂₀) to a contrast masking operation and for generating a masked error sequence (d₂₄) therefrom; and

a pooling processor for pooling said masked error sequence (d₂₄) to generate a perceptual error (E_Ω

).

21. An apparatus according to claim 20, further including a visual quality converter that converts said perceptual error (E_Ω

) to a visual quality measure (Q_Ω

), for providing an output in terms of quality.

22. An apparatus according to claim 21, further including a codec to which said perceptual error (E_Ω

) is fed back for regulating a compression bit rate to correspond to a desired image visual quality.

23. An apparatus according to claim 21, further including a codec to which said visual quality measure (Q_Ω

) is fed back for regulating a compression bit rate to correspond to a desired image visual quality.

24. An apparatus according to claim 20, wherein each of said processed (T) video sequence and said original (R) video sequence includes color channels;

and

a color transformer that converts said color channels to a perceptually relevant color space, for generating color transformed sequences (d₉) from said region of interest sequences (d₂).

25. An apparatus according to claim 24, further including a block constructor that subjects said color transformed sequences (d₉) to blocking, in order to generate blocks.

26. An apparatus according to claim 25, further including a discrete cosine transformer for converting said blocks to a block of frequency coefficients (d₁₀).

27. An apparatus according to claim 26, wherein if any of the processed (T) video sequence or the original (R) video sequence contains interlaced video fields, then de-interlacing said interlaced fields to a progressive sequence (d₇) by means of a de-interlacer.

28. An apparatus according to claim 27, further including a veiling light combiner for adding a veiling light to said progressive sequence (d₇).

29. An apparatus according to claim 28, further including a local contrast converter for converting said block of frequency coefficients (d₁₀) to a local contrast signal (d₁₇);

and

wherein said local contrast signal (d₁₇) includes a combination of AC coefficients (d_17a) and DC coefficients (d_17b).

30. A digital video quality apparatus with original (R) video sequence and a processed (T) video signal being fed thereto, the apparatus comprising:

a sampler for sampling the original and processed video sequences and for generating sampled sequences (d₁) therefrom;

a region-of-interest processor for limiting the processing of said sampled sequences (d₁) to a region of interest and for generating region of interest sequences (d₂) therefrom;

a local contrast converter for transforming said region of interest sequences (d₂) to local contrast coefficients (d₁₇);

a time filter for filtering said local contrast coefficients (d₁₇) and for generating filtered components (d₁₈) therefrom;

a threshold scaler for converting said said filtered components (d₁₈) to threshold units (d₁₉);

a subtractor for subtracting said threshold units (d₁₉) corresponding to the original (R) and processed (T) sequences to obtain an error sequence (d₂₀);

a contrast masking processor for subjecting said subtracted error sequence (d₂₀) to a contrast masking operation and generating a masked error sequence (d₂₄); and

a pooling processor for pooling said error sequence (d₂₀) to generate a perceptual error (E_Ω

).

31. An apparatus according to claim 30 further including a visual quality convertor for converting said perceptual error (E_Ω

) to visual quality measure (Q_Ω

) to provide output in terms of quality.

32. A digital quality method for evaluating the visual quality of a processed (T) video sequence relative to an original (R) video sequence, the method comprising:

sampling the original and processed video sequences and for generating sampled sequences (d₁) therefrom;

limiting the processing of said sampled sequences (d₁) to a region of interest and for generating region of interest sequences (d₂) therefrom;

transferring said region of interest sequences (d₂) to local contrast coefficients (d₁₇);

filtering said local contrast coefficients (d₁₇) and for generating components (d₁₈) therefrom;

subtracting said threshold units (d₁₉) to obtain an error sequence (d₂₀);

subjecting said error sequence (d₂₀) to a contrast masking operation to obtain a masked error sequence (d₂₄); and

pooling said error sequence (d₂₀) to generate a perceptual (E_Ω

).