Artifact reduction in a digital video

• US 7,643,088 B2
• Filed: 12/01/2004
• Issued: 01/05/2010
• Est. Priority Date: 12/01/2004
• Status: Expired due to Fees

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Silverstein, D. Amnon, Fan, Jian
• Primary Examiner(s): Tran; Thai
• Assistant Examiner(s): DUNN, MISHAWN N
• Application Number:
  US11/000,766
• Publication Number:
  US 20060114358A1
• Time in Patent Office: 1,861 Days
• Field of Search: 386/46, 348/606, 348/607, 348624-630, 348/441, 348/443, 348/448, 348/452, 348/459, 382260-266
• US Class Current: 348/441
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20036 Morphological image processing
• CPC Class Codes: G06T 5/00 Image enhancement or restoration
• CPC Class Codes: G06T 5/005 Retouching; Inpainting; Scratch removal detecting, correction, reducing or removing defects, e.g. non-responsive pixels of solid state image sensors H04N25/68, scratch removal for cinematographic films scanned by electronic image sensor H04N5/253
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/77 Retouching; Inpainting; Scratch removal
• CPC Class Codes: H04N 5/21 Circuitry for suppressing or minimising disturbance, e.g. moiré or halo
• CPC Class Codes: H04N 5/253 Picture signal generating by scanning motion picture films or slide opaques, e.g. for telecine scanning details therefor H04N3/36 ; standard conversion therefor H04N7/0112

Abstract

A method performed by a processing system is provided. The method comprises detecting a first pixel value associated with an artifact in a current frame of a digital video using a previous frame and a next frame and replacing the first pixel value in the first frame with a replacement value.

Claims

1. A method performed by a processing system, the method comprising:

detecting a first set of pixel values in a first region of a current frame of a digital video that differs from at least one of a corresponding second set of pixel values in a previous frame and a corresponding third set of pixel values in a next frame;

determining whether the first region is in a range of sizes of known artifacts; and

replacing the first set of pixel values in the first frame with a set of replacement values in response to determining that the first region is in the range of sizes of known artifacts.

2. The method of claim 1 further comprising:

detecting that the first set of pixel values are in a range of colors of the known artifacts.

3. The method of claim 1 further comprising:

interpolating the set of replacement values using at least the corresponding second set of pixel values in the previous frame and the corresponding third set of pixel values in the next frame.

4. The method of claim 1 further comprising:

comparing the corresponding second set of pixel values in the previous frame with the corresponding third set of pixel values in the next frame.

5. The method of claim 4 further comprising:

identifying a set of pixels corresponding to the first set of pixel values in a non-temporal mask in response to the corresponding second set of pixel values in the previous frame differing from the corresponding third set of pixel values in the next frame.

6. The method of claim 5 further comprising:

interpolating the set of replacement values using a fourth set of pixel values in the current frame in response to the corresponding second set of pixel values in the previous frame differing from the corresponding third set of pixel values in the next frame.

7. The method of claim 6 wherein the first set of pixel values are from a first color plane of the digital video, and wherein the fourth set of pixel values are from at least a second color plane of the digital video that differs from the first color plane.

8. The method of claim 4 further comprising:

comparing the first set of pixel values with a fourth set of pixel values in the current frame in response to the corresponding second set of pixel values in the previous frame differing from the corresponding third set of pixel values.

9. The method of claim 1 further comprising:

performing block matching between the first region in the current frame and a second region from the previous frame; and

replacing the first region with the second region from the previous frame, wherein the second region comprises the set of replacement values.

10. The method of claim 1 further comprising:

performing block matching between the first region in the current frame and a second region from the next frame; and

replacing the first region with the second region from the next frame, wherein the second region comprises the set of replacement values.

11. The method of claim 1 further comprising:

performing motion compensation on the previous frame with respect to the current frame prior to detecting the first set of pixel values in the first region in the current frame.

12. The method of claim 1 further comprising:

performing motion compensation on the next frame with respect to the current frame prior to detecting the first set of pixel values in the first region in the current frame.

13. A system comprising:

a processor; and

a memory system for storing a digital video and an artifact reduction module;

wherein the processor is configured to execute the artifact reduction module to;

detect a first set of pixel values in a first region of a current frame of the digital video that differs from at least one of a corresponding second set of pixel values in a previous frame and a corresponding third set of pixel values in a next frame;

determine whether the first region is in a range of sizes of known artifacts; and

replace the first set of pixel values in the first frame with a set of replacement values.

14. The system of claim 13 wherein the memory system comprises an artifact detection module, wherein the processor is configured to execute the artifact detection module to:

detect that the first set of pixel values in the current frame differs significantly from the corresponding second set of pixel values in previous frame and the corresponding third set of pixel values in the next frame.

15. The system of claim 14 wherein the memory system comprises an artifact removal module, wherein the processor is configured to execute the artifact removal module to:

calculate the set of replacement values from the corresponding second set of pixel values and the corresponding third set of pixel values.

16. The system of claim 14 wherein the processor is configured to execute the artifact detection module to:

perform motion compensation on the previous frame with respect to the current frame to align a first set of pixels associated with the first set of pixel values with a second set of pixels associated with the corresponding second set of pixel values.

17. The system of claim 16 wherein the processor is configured to execute the artifact detection module to:

perform motion compensation on the next frame with respect to the current frame to align a first set of pixels associated with the first set of pixel values with a third set of pixels associated with the corresponding third set of pixel values.

18. The system of claim 13 wherein the processor is configured to execute the artifact reduction module to:

identify a set of pixels associated with the first set of pixel values in an artifact mask.

19. The system of claim 13 wherein the processor is configured to execute the artifact reduction module to:

identify a set of pixels corresponding to the first set of pixel values in a non-temporal mask in response to the corresponding second set of pixel values in the previous frame differing from the corresponding third set of pixel values in the next frame.

20. The system of claim 13 wherein the processor is configured to execute the artifact reduction module to:

interpolate the set of replacement values using a fourth set of pixel values in the current frame in response to the corresponding second set of pixel values in the previous frame differing from the corresponding third set of pixel values in the next frame.

21. The system of claim 13 wherein the processor is configured to execute the artifact reduction module to:

compare the first set of pixel values with colors a fourth set of pixel values in the current frame in response to the corresponding second set of pixel values in the previous frame differing from the corresponding third set of pixel values in the next frame.

22. The system of claim 13 wherein the processor is configured to execute the artifact reduction module to:

perform block matching between the first region in the current frame and a second region from the previous frame; and

replace the first region that comprises the first pixel value with the second region from the previous frame, wherein the second region comprises the set of replacement values.

23. The system of claim 13 wherein the processor is configured to execute the artifact detection module to:

compare the corresponding second set of pixel values in the previous frame with the corresponding third set of pixel values in the next frame;

identify a set of pixels corresponding to the first set of pixel values in a non-temporal mask in response to the corresponding second set of pixel values in the previous frame differing from the corresponding third set of pixel values in the next frame; and

interpolate the set of replacement values using a fourth set of pixel values in the current frame in response to the corresponding second set of pixel values in the previous frame differing from the corresponding third set of pixel values in the next frame;

wherein the first set of pixel values are from a first color plane of the digital video, and wherein the fourth set of pixel values are from at least a second color plane of the digital video that differs from the first color plane.

24. A system comprising:

means for detecting a first set of pixel values in a first region of a current frame of a digital video that differs from at least one of a corresponding second set of pixel values in a previous frame and a corresponding third set of pixel values in a next frame;

means for determining whether the first region is in a range of sizes of known artifacts; and

;

means for replacing the first set of pixel values in the first frame with a set of replacement values in response to determining that the first region is in the range of sizes of known artifacts.

25. The system of claim 24 further comprising:

means for performing motion compensation on the previous frame and the next frame with respect to the current frame.

26. The system of claim 24 further comprising:

means for comparing the corresponding second set of pixel values in the previous frame with the corresponding third set of pixel values in the next frame;

means for identifying a set of pixels corresponding to the first set of pixel values in a non-temporal mask in response to the corresponding second set of pixel values in the previous frame differing from the corresponding third set of pixel values in the next frame; and

means for interpolating the set of replacement values using a fourth set of pixel values in the current frame in response to the corresponding second set of pixel values in the previous frame differing from the corresponding third set of pixel values in the next frame;

wherein the first set of pixel values are from a first color plane of the digital video, and wherein the fourth set of pixel values are from at least a second color plane of the digital video that differs from the first color plane.

27. A program product comprising a computer-readable medium including instructions executable by a processing system for:

detecting a first set of pixel values in a first region of a current frame of a digital video that differs from at least one of a corresponding second set of pixel values in a previous frame and a corresponding third set of pixel values in a next frame;

determining whether the first region is in a range of sizes of known artifacts; and

replacing the first set of pixel values in the first frame with a set of replacement values in response to determining that the first region is in the range of sizes of known artifacts.

28. The program product of claim 27 wherein the computer-readable medium includes instructions executable by the processing system for:

detecting that the first set of pixel values are in a range of colors of the known artifacts.

29. The program product of claim 27 wherein the computer-readable medium includes instructions executable by the processing system for:

interpolating the set of replacement values using at least the corresponding second set of pixel values in the previous frame and the corresponding third set of pixel values in the next frame.

30. The program product of claim 27 wherein the computer-readable medium includes instructions executable by the processing system for:

in a set of pixels corresponding to the first set of pixel values in a non-temporal mask in response to the corresponding second set of pixel values in the previous frame differing from the corresponding third set of pixel values in the next frame.

31. The program product of claim 27 wherein the computer-readable medium includes instructions executable by the processing system for:

interpolating the set of replacement values using a fourth set of pixel values in the current frame in response to the corresponding second set of pixel values in the previous frame differing from the third set of pixel values in the next frame.

32. The program product of claim 27 wherein the computer-readable medium includes instructions executable by the processing system for:

comparing the first set of pixel values with fourth set of pixel values in the current frame in response to the corresponding second set of pixel values in the previous frame differing from the corresponding third set of pixel values.

33. The program product of claim 27 wherein the computer-readable medium includes instructions executable by the processing system for:

performing block matching between the first region in the current frame, a second region in the previous frame, and a third region in the next frame; and

replacing the first region with either the second region from the previous frame that includes the set of replacement values or the third region from the next frame that includes the set of replacement values.

34. The program product of claim 27 wherein the computer-readable medium includes instructions executable by the processing system for:

performing motion compensation on the previous frame and the next frame with respect to the current frame prior to detecting the first set of pixel values in the first region in the current frame.

35. A system comprising:

an artifact detection module configured to detect a first set of pixel values in a first region of a current frame of a digital video that differs from at least one of a corresponding second set of pixel values in a previous frame and a corresponding third set of pixel values in a next frame and configured to determine whether the first region is in a range of sizes of known artifacts; and

an artifact removal module configured to replace the first set of pixel values in the first frame with a set of replacement values.