Methods and systems for detecting video artifacts

• US 9,232,118 B1
• Filed: 01/23/2015
• Issued: 01/05/2016
• Est. Priority Date: 01/23/2015
• Status: Active Grant

• Current Assignee: Interra Systems Inc., US
• Original Assignee: Interra Systems, Inc.
• Inventors: Mogre, Advait Madhav, Kumar, Bhupender, Madnani, Shekhar
• Primary Examiner(s): Tran, Trang U
• Application Number:
  US14/604,290
• Time in Patent Office: 347 Days
• Field of Search: 348/607, 348/620, 348/622, 348/625, 348/701
• US Class Current: 1/1
• CPC Class Codes: H04N 19/142 Detection of scene cut or scene change
• CPC Class Codes: H04N 19/154 Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion use of rate-distortion criteria H04N19/147
• CPC Class Codes: H04N 5/21 Circuitry for suppressing or minimising disturbance, e.g. moiré or halo

Abstract

A system and method for detecting artifacts in a video having a sequence of pictures is described. The method comprises the steps of calculating a motion vector cost and a variable threshold for each of the pictures. In case, the motion vector cost for the picture is greater than the variable threshold of the picture, then a first scene change profile and a second scene change profile for the picture is computed and analyzed for determining the character of the variation in the motion vector cost. An artifact metric for the picture is also calculated and the artifact metric for the picture is compared with a programmable artifact reporting threshold for ascertaining if an artifact is present in the video. The described approach is generic in nature and takes into account various exceptions that may be present within a picture that may be similar in character as an artifact.

Claims

1. A method for detecting artifacts in a video having a sequence of consecutive one or more pictures, comprising the steps of:

calculating a motion vector cost between a current picture and at least one previously filtered picture from the sequence of consecutive one or more pictures;

computing a filter coefficient based on the motion vector cost;

applying a filter using the filter coefficient on the current picture to create a filtered picture to be used for next iteration;

calculating a variable threshold for each of the one or more pictures, wherein the variable threshold for each picture is based on the variable threshold of the corresponding previous picture;

monitoring the sequence of one or more pictures for a scene change, wherein the step of monitoring comprises of

comparing the motion vector cost of each picture with the corresponding variable threshold of the picture, and if the motion vector cost for the picture is greater than the variable threshold of the picture, then

calculating a first confidence value corresponding to a first scene change profile for the picture and a second confidence value corresponding to a second scene change profile for the picture, wherein the first confidence value and the second confidence value are based on the motion vector cost of the picture, motion vector cost of one or more previous pictures and motion vector cost of one or more subsequent pictures;

comparing the first confidence value with the second confidence value for the picture, and setting a scene change metric for the picture to be equal to the first confidence value if the first confidence value is higher than the second confidence value, else setting the scene change metric for the picture to be equal to the second confidence value;

calculating an artifact metric for the picture, wherein the artifact metric is an exponential function of the motion vector cost of the pictures, variable threshold of the picture, and the scene change metric of the picture; and

comparing the artifact metric for the picture with a programmable artifact reporting threshold, if the artifact metric for the picture is greater than the artifact reporting threshold then reporting the video as having an artifact.

2. The method as claimed in claim 1, wherein the one or more pictures is one or more frames of the video sequence.

3. The method as claimed in claim 1, wherein the one or more pictures is one or more top fields or one or more bottom fields of the video sequence.

4. The method as claimed in claim 1, wherein the filtered pixel for each picture is calculated by calculating a weighted sum of each pixel of the current picture and filtered pixel of the previous picture wherein the weight is based on the motion estimation of one or more previous pictures.

5. A method for detecting artifacts in a video having a sequence of consecutive one or more pictures, comprising the steps of:

calculating a motion vector cost between a current picture and at least one previously filtered picture from the sequence of consecutive one or more pictures;

computing a filter coefficient based on the motion vector cost;

applying a filter using the filter coefficient on the current picture to create a filtered picture to be used for next iteration;

calculating a variable threshold for each of the one or more pictures, wherein the variable threshold for each picture is based on the variable threshold of the corresponding previous picture;

monitoring the sequence of one or more pictures for a scene change, wherein the step of monitoring comprises of

comparing the motion vector cost of each picture with the corresponding variable threshold of the picture, and if the motion vector cost for the picture is greater than the variable threshold of the picture, then

calculating a first confidence value corresponding to a first scene change profile for the picture and a second confidence value corresponding to a second scene change profile for the picture, wherein the first confidence value and the second confidence value are based on the motion vector cost of the picture, motion vector cost of one or more previous pictures and motion vector cost of one or more subsequent pictures;

comparing the first confidence value with the second confidence value for the picture, and setting a scene change metric for the picture to be equal to the first confidence value if the first confidence value is higher than the second confidence value, else setting the scene change metric for the picture to be equal to the second confidence value;

calculating an artifact metric for the picture, wherein the artifact metric is an exponential function of the motion vector cost of the pictures, variable threshold of the picture, and the scene change metric of the picture;

selectively calculating one or more artifact exception weights for determining impact of one or more exceptions to the artifacts in the video;

calculating an artifact reporting metric, wherein the artifact reporting metric equals the product of the artifact metric and one or more selected artifact exception weights;

comparing the artifact reporting metric for the picture with a programmable artifact reporting threshold, if the artifact reporting metric for the picture is greater than the artifact reporting threshold then reporting the video as having an artifact.

6. The method as claimed in claim 5, wherein the filtered pixel for each picture is calculated by calculating a weighted sum of each pixel of the current picture and filtered pixel of the previous picture wherein the weight is based on the motion estimation of one or more previous pictures.

7. The method as claimed in claim 5, wherein the step of calculating one or more artifact exception weights comprises of

counting a number of scene changes for a predetermined set of consecutive pictures;

calculating a scene change density metric, wherein the scene change density metric equals the output of a leaky accumulator Infinite Impulse Response filter taking the number of scene changes for the predetermined set of consecutive pictures as input;

calculating a rapid motion exception weight wherein the rapid motion exception weight equals an exponential function of the scene change density metric.

8. The method as claimed in claim 5, wherein the step of calculating one or more artifact exception weights comprises of

determining an autocorrelation between motion vector costs of a predetermined set of consecutive pictures;

determining one or more zero crossings of the auto correlation;

determining an average zero crossing of the auto correlation;

determining a first cadence confidence metric and a second cadence confidence metric;

determining an overall cadence confidence value, wherein the overall cadence confidence value equals the maximum of either first cadence confidence metric or the second cadence confidence metric clamped between zero and a predetermined cadence threshold; and

calculating a cadence exception weight wherein the cadence exception weight equals an exponential function of the product of the overall cadence confidence value and the predetermined cadence threshold.

9. The method as claimed in claim 5, wherein the step of calculating one or more artifact exception weights comprises of

determining a luminance histogram for the picture and a predetermined number of consecutive previous pictures and a predetermined number of consecutive subsequent picture;

calculating area of luminance histogram;

calculating area under the picture for the luminance histogram;

determining a skewness value between the area of luminance histogram and area under the picture for the luminance histogram, wherein the skewness value equals the result of the division of the area of luminance histogram by the area under the picture for the luminance histogram;

determining a skewness metric, wherein the skewness metric equals the difference between the skewness value of the previous picture and the skewness value of the current picture; and

comparing the skewness metric with a predetermined skewness threshold, and if the skewness metric is greater than the skewness threshold then

calculating an blurriness exception weight wherein the blurriness exception weight equals an exponential function of the skewness metric.

10. The method as claimed in claim 5, wherein the step of calculating one or more artifact exception weights comprises of

determining a first histogram of oriented gradient computed over the x direction and a second histogram of oriented gradient computed over the y direction for the entire picture;

calculating the area of the first histogram of oriented gradient computed over the x direction and the area of the second histogram of oriented gradient computed over the y direction;

calculating a first entropy of the histogram of oriented gradient computed over the x direction and calculating a second entropy of the histogram of oriented gradient computed over the y direction;

calculating a total histogram of oriented gradient entropy, wherein the total histogram of oriented gradient entropy equals the summation of the first entropy of the histogram of oriented gradient computed over the x direction and the second entropy of the histogram of oriented gradient computed over the y direction;

calculating a sum of entropy deltas for the picture, wherein the sum of entropy deltas equals the summation of the difference between two consecutive pictures for a predetermined set of consecutive pictures;

comparing the sum of entropy deltas for the picture with a predetermined entropy threshold, and if the sum of entropy deltas for the picture is greater than the predetermined entropy threshold then setting a sum of entropy deltas metric to zero, else the sum of entropy deltas metric equals a function of the difference of the predetermined entropy threshold and the sum of entropy deltas; and

calculating a flashiness exception weight, wherein the flashiness exception weight equals an exponential function of the sum of entropy deltas metric.

11. A system for detecting artifacts in a video having a sequence of consecutive one or more pictures comprising

an internal memory storing one or more program modules;

a reference picture unit configured for

calculating a motion vector cost between a current picture and at least one previously filtered picture from the sequence of consecutive one or more pictures;

computing a filter coefficient based on the motion vector cost; and

applying a filter using the filter coefficient on the current picture to create a filtered picture to be used for next iteration;

a filter for calculating a variable threshold for each of the one or more pictures, wherein the variable threshold for each picture is based on the variable threshold of the corresponding previous picture; and

a processor configured for executing the program modules stored in the internal memory;

wherein the processor is configured for

monitoring the sequence of one or more pictures for a scene change, wherein the step of monitoring comprises of

comparing the motion vector cost of each picture with the corresponding variable threshold of the picture, and if the motion vector cost for the picture is greater than the variable threshold of the picture, then

calculating a first confidence value and a second confidence value for the picture, wherein the first confidence value and the second confidence value are based on the motion vector cost of the picture, motion vector cost of one or more previous pictures and motion vector cost of one or more subsequent pictures;

comparing the first confidence value with the second confidence value for the picture, and setting a scene change metric for the picture to be equal to the first confidence value if the first confidence value is higher than the second confidence value, else setting the scene change metric for the picture to be equal to the second confidence value;

calculating an artifact metric for the picture, wherein the artifact metric is an exponential function of the motion vector cost of the pictures, variable threshold of the picture, and the scene change metric of the picture;

selectively calculating one or more artifact exception weights for determining impact of one or more exceptions to the artifacts in the video;

calculating an artifact reporting metric, wherein the artifact reporting metric equals the product of the artifact metric and one or more selected artifact exception weights; and

comparing the artifact reporting metric for the picture with a programmable artifact reporting threshold, if the artifact reporting metric for the picture is greater than the artifact reporting threshold then reporting the video as having an artifact.

12. The system as claimed in claim 11, wherein the reference picture unit is configured for calculating filtered pixels for each picture by calculating a weighted sum of each pixel of the current picture and filtered pixel of the previous picture wherein the weight is based on the motion estimation of one or more previous pictures.

13. The system as claimed in claim 11, wherein the filter is a Finite impulse response filter.

14. The system as claimed in claim 11, wherein the filter is an Infinite impulse response filter.

15. The system as claimed in claim 11, wherein the processor for calculating one or more artifact exception weights is configured for

counting a number of scene changes for a predetermined set of consecutive pictures;

calculating a scene change density metric, wherein the scene change density metric equals the output of a leaky accumulator Infinite Impulse Response filter taking the number of scene changes for the predetermined set of consecutive pictures as input;

calculating an artifact exception weight wherein the artifact exception weight equals an exponential function of the scene change density metric.

16. The system as claimed in claim 11, wherein the processor for calculating one or more artifact exception weights is configured for

determining an autocorrelation between motion vector costs of a predetermined set of consecutive pictures;

determining one or more zero crossings of the auto correlation;

determining an average zero crossing of the auto correlation;

determining a first cadence confidence metric and a second cadence confidence metric;

determining an overall cadence confidence value, wherein the overall cadence confidence value equals the maximum of either first cadence confidence metric or the second cadence confidence metric clamped between zero and a predetermined cadence threshold; and

calculating an artifact exception weight wherein the artifact exception weight equals an exponential function of the product of the overall cadence confidence value and the predetermined cadence threshold.

17. The system as claimed in claim 11, wherein the processor for calculating one or more artifact exception weights is configured for

determining a luminance histogram for the picture and a predetermined number of consecutive previous pictures and a predetermined number of consecutive subsequent picture;

calculating area of luminance histogram;

calculating area under the picture for the luminance histogram;

determining a skewness value between the area of luminance histogram and area under the picture for the luminance histogram, wherein the skewness value equals the result of the division of the area of luminance histogram by the area under the picture for the luminance histogram;

determining a skewness metric, wherein the skewness metric equals the difference between the skewness value of the previous picture and the skewness value of the current picture; and

comparing the skewness metric with a predetermined skewness threshold, and if the skewness metric is greater than the skewness threshold then

calculating an artifact exception weight wherein the artifact exception weight equals an exponential function of the skewness metric.

18. The system as claimed in claim 11, wherein the processor for calculating one or more artifact exception weights is configured for

determining a first histogram of oriented gradient computed over the x direction and a second histogram of oriented gradient computed over the y direction for the entire picture;

calculating the area of the first histogram of oriented gradient computed over the x direction and the area of the second histogram of oriented gradient computed over the y direction;

calculating a first entropy of the histogram of oriented gradient computed over the x direction and calculating a second entropy of the histogram of oriented gradient computed over the y direction;

calculating a total histogram of oriented gradient entropy, wherein the total histogram of oriented gradient entropy equals the summation of the first entropy of the histogram of oriented gradient computed over the x direction and the second entropy of the histogram of oriented gradient computed over the y direction;

calculating a sum of entropy deltas for the picture, wherein the sum of entropy deltas equals the summation of the difference between two consecutive pictures for a predetermined set of consecutive pictures;

comparing the sum of entropy deltas for the picture with a predetermined entropy threshold, and if the sum of entropy deltas for the picture is greater than the predetermined entropy threshold then setting a sum of entropy deltas metric to zero, else the sum of entropy deltas metric equals a function of the difference of the predetermined entropy threshold and the sum of entropy deltas; and

calculating an artifact exception weight, wherein the artifact exception weight equals an exponential function of the sum of entropy deltas metric.