Pre-processing method and system for data reduction of video sequences and bit rate reduction of compressed video sequences using spatial filtering

• US 7,430,335 B2
• Filed: 08/13/2003
• Issued: 09/30/2008
• Est. Priority Date: 08/13/2003
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Dumitras, Adriana, Salsbury, Ryan R., Normile, James Oliver
• Primary Examiner(s): Patel; Kanji
• Application Number:
  US10/640,944
• Publication Number:
  US 20050036704A1
• Time in Patent Office: 1,875 Days
• Field of Search: 382232-233, 382260-264, 382274-275, 348/14.08, 348/14.13, 345/691, 345/694, 375/240.29, 375/254
• US Class Current: 382/260
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/192 the adaptation method, adaptation tool or adaptation type being iterative or recursive
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence
• CPC Class Codes: H04N 5/21 Circuitry for suppressing or minimising disturbance, e.g. moiré or halo
• CPC Class Codes: H04N 9/8042 involving data reduction

Abstract

Methods for pre-processing video sequences prior to compression to provide data reduction of the video sequence. Also, after compression of the pre-processed video sequence, the bit rate of the pre-processed and compressed video sequence will be lower than the bit rate of the video sequence after compression but without pre-processing. Pre-processing may include spatial anisotropic diffusion filtering such as Perona-Malik filtering, Fallah-Ford filtering, or omni-directional filtering that extends Perona-Malik filtering to perform filtering in at least one diagonal direction. Pre-processing may also include performing filtering differently on a foreground region than on a background region of a video frame. This method includes identifying pixel locations having pixel values matching characteristics of human skin and determining a bounding shape for each contiguous grouping of matching pixel locations. The foreground region is comprised of pixel locations contained in a bounding shape and the background region is comprised of all other pixel locations.

Claims

1. A method of pre-filtering an original video sequence, the original video sequence comprising a plurality of frames, each frame comprising a plurality of pixel locations where each pixel location comprises a pixel value, the method comprising:

a) setting a current frame of the original video sequence;

b) identifying a region-of-interest in the current frame;

c) specifying a bounding shape that encloses at least a portion of the region-of-interest; and

d) filtering pixel locations in the bounding shape differently than other pixel locations in the current frame.

2. The method of claim 1, wherein the bounding shape comprises one of at least four-sided, three-sided, and circular form.

3. The method of claim 1 wherein each pixel location in the region-of-interest has a chrominance value within a predetermined low chrominance threshold value and a predetermined high chrominance threshold value.

4. The method of claim 1 wherein the bounding shape encloses over ½

of the pixel locations in the region-of-interest.

5. The method of claim 1 wherein:

a region of the current frame within the bounding shape is referred to as a foreground region; and

a region of the current frame not within the bounding shape is referred to as a background region.

6. The method of claim 5 wherein the filtering comprises applying anisotropic diffusion to the pixel locations in the foreground and background regions where different parameter values are used for the foreground and background regions.

7. The method of claim 5 wherein the filtering comprises applying anisotropic diffusion to the pixel locations in the foreground and background regions where different edge stopping functions are used for the foreground and background regions.

8. The method of claim 5 wherein the filtering comprises applying Fallah-Ford diffusion filtering to the pixel locations in the foreground region differently than to the pixel locations in the background region.

9. The method of claim 5 wherein the filtering comprises applying Perona-Malik diffusion filtering to the pixel locations in the foreground region differently than to the pixel locations in the background region.

10. The method of claim 5 wherein the filtering comprises applying anisotropic diffusion to the pixel locations in the foreground region differently than to the pixel locations in the background region, the anisotropic diffusion performing filtering on a pixel location in at least one diagonal direction with respect to the pixel location.

11. The method of claim 1 farther comprising:

e) setting a next frame in the original video sequence as the current frame;

f) repeating steps b), c), and d) for each frame of the plurality of frames in the original video sequence;

g) outputting a pre-filtered video sequence comprising a plurality of pre-filtered video frames; and

h) compressing the pre-filtered video sequence using a first compression method to produce a pre-filtered and compressed video sequence, wherein a bit rate associated with the pre-filtered and compressed video sequence is lower than a bit rate that would result from compressing the original video sequence using the first compression method without performing steps a) through f).

12. A computer readable medium storing a computer program for pre-filtering an original video sequence, the original video sequence comprising a plurality of frames, each frame comprising a plurality of pixel locations where each pixel location comprises a pixel value, the computer program executable by at least one processor, the computer program comprising sets of instructions for:

setting a current frame of the original video sequence;

identifying a region-of-interest in the current frame;

specifying a bounding shape that encloses at least a portion of the region-of-interest; and

filtering pixel locations in the bounding shape differently than other pixel locations in the current frame.

13. The computer readable medium of claim 12, wherein each pixel location in the region-of-interest has a chrominance value within a predetermined low chrominance threshold value and a predetermined high chrominance threshold value.

14. The computer readable medium of claim 12, wherein the filtering comprises applying anisotropic diffusion to the pixel locations in the bounding shape differently than other pixel locations in the current frame.

15. A system for pre-filtering an original video sequence, the original video sequence comprising a plurality of frames, each frame comprising a plurality of pixel locations where each pixel location comprises a pixel value, the system comprising:

a pre-processing component that;

sets a current frame of the original video sequence;

identifies a region-of-interest in the current frame;

specifies a bounding shape that encloses at least a portion of the region-of interest; and

filters pixel locations in the bounding shape differently than other pixel locations in the current frame.

16. The system of claim 15 wherein each pixel location in the region-of-interest has a chrominance value within a predetermined low chrominance threshold value and a predetermined high chrominance threshold value.

17. The system of claim 15 wherein the filtering comprises applying anisotropic diffusion to the pixel locations in the bounding shape differently than other pixel locations in the current frame.

18. A system for pre-filtering an original video sequence, the original video sequence comprising a plurality of frames, each frame comprising a plurality of pixel locations where each pixel location comprises a pixel value, the system comprising:

means for setting a current frame of the original video sequence;

means for identifying a region-of-interest in the current frame;

means for specifying a bounding shape that encloses at least a portion of the region-of-interest; and

means for filtering pixel locations in the bounding shape differently than other pixel locations in the current frame.

19. The system of claim 18 wherein the filtering comprises applying anisotropic diffusion to the pixel locations in the bounding shape differently than other pixel locations in the current frame.

20. A computer readable medium storing a computer program that is executable by at least one processor, the computer program comprising sets of instructions for:

a) identifying a region-of-interest in an original video picture, the region-of-interest comprising a plurality of pixel locations;

b) specifying a bounding shape that encloses at least a portion of the region-of-interest; and

c) filtering pixel locations in the bounding shape differently than other pixel locations in the video picture.

21. The computer readable medium of claim 20, wherein each pixel location in the region-of-interest has a chrominance value within a predetermined low chrominance threshold value and a predetermined high chrominance threshold value.

22. The computer readable medium of claim 20, wherein the region of the current frame within the bounding shape comprises a foreground region and the region of the current frame not within the bounding shape comprises a background region.

23. The computer readable medium of claim 22, wherein the set of instructions for filtering comprises a set of instructions for applying anisotropic diffusion to the pixel locations in the foreground and background regions where different parameter values are used for the foreground and background regions.

24. The computer readable medium of claim 22, wherein the set of instructions for filtering comprises a set of instructions for applying anisotropic diffusion to the pixel locations in the foreground and background regions where different edge stopping functions are used for the foreground and background regions.

25. The computer readable medium of claim 22, wherein the set of instructions for filtering comprises a set of instructions for applying Fallah-Ford diffusion filtering to the pixel locations in the foreground region differently than to the pixel locations in the background region.

26. The computer readable medium of claim 22, wherein the set of instructions for filtering comprises a set of instructions for applying Perona-Malik diffusion filtering to the pixel locations in the foreground region differently than to the pixel locations in the background region.

27. The computer readable medium of claim 22, wherein the set of instructions for filtering comprises a set of instructions for applying anisotropic diffusion to the pixel locations in the foreground region differently than to the pixel locations in the background region, the anisotropic diffusion performing filtering on a pixel location in at least one diagonal direction with respect to the pixel location.

28. The computer readable medium of claim 20, wherein said filtering reduces data in the video picture.

29. The computer readable medium of claim 20, wherein the computer program further comprises sets of instructions for compressing a filtered video sequence comprising said filtered video picture, wherein filtering said video picture is for reducing a bit rate associated with the compressed and filtered video sequence.

30. A computer readable medium storing a computer program that is executable by at least one processor, the computer program comprising sets of instructions for:

a) identifying a set of matching pixel locations in an original video picture, the pixel locations comprising chrominance values within predetermined thresholds;

b) specifying a bounding shape for each contiguous grouping of matching pixel locations in the set of matching pixel locations; and

c) filtering the pixel locations enclosed within the bounding shapes differently than the pixel locations not enclosed within the bounding shapes.

31. The computer readable medium of claim 30, wherein said filtering reduces a size of the video picture.

32. The computer readable medium of claim 30, wherein said predetermined thresholds comprises a low chrominance (U) threshold value and a high chrominance (U) threshold value for a chrominance (U) value associated with a pixel location, and a low chrominance (V) threshold value and a high chrominance (V) threshold value for a chrominance (V) value associated with a pixel location.

33. The computer readable medium of claim 32, wherein the low and high chrominance (U) threshold values and the low and high chrominance (V) threshold values reflect the chrominance (U, V) ranges of human skin.

34. The computer readable medium of claim 32, wherein the set of instructions for filtering comprises a set of instructions for applying anisotropic diffusion to the pixel locations within the bounding shapes differently than to the pixel locations not enclosed within the bounding shapes.

35. A method comprising:

a) identifying a region-of-interest in an original video picture, the region-of interest comprising a plurality of pixel locations;

b) specifying a bounding shape that encloses at least a portion of the region-of-interest; and

c) filtering pixel locations in the bounding shape differently than other pixel locations in the video picture.

36. The method of claim 35, wherein each pixel location in the region-of-interest has a chrominance value within a predetermined low chrominance threshold value and a predetermined high chrominance threshold value.

37. The method of claim 35, wherein the region of the current frame within the bounding shape comprises a foreground region and the region of the current frame not within the bounding shape comprises a background region.

38. The method of claim 37, wherein the filtering comprises applying anisotropic diffusion to the pixel locations in the foreground and background regions where different parameter values are used for the foreground and background regions.

39. The method of claim 37, wherein the filtering comprises applying anisotropic diffusion to the pixel locations in the foreground and background regions where different edge stopping functions are used for the foreground and background regions.

40. The method of claim 37, wherein the filtering comprises applying Fallah-Ford diffusion filtering to the pixel locations in the foreground region differently than to the pixel locations in the background region.

41. The method of claim 37, wherein the filtering comprises applying Perona-Malik diffusion filtering to the pixel locations in the foreground region differently than to the pixel locations in the background region.

42. The method of claim 37, wherein the filtering comprises applying anisotropic diffusion to the pixel locations in the foreground region differently than to the pixel locations in the background region, the anisotropic diffusion performing filtering on a pixel location in at least one diagonal direction with respect to the pixel location.

43. A method comprising:

a) identifying a set of matching pixel locations in an original video picture, the pixel locations comprising chrominance values within predetermined thresholds;

b) specifying a bounding shape for each contiguous grouping of matching pixel locations in the set of matching pixel locations; and

c) filtering the pixel locations enclosed within the bounding shapes differently than the pixel locations not enclosed within the bounding shapes.

44. The method of claim 43, wherein said filtering reduces a size of the video picture.

45. The method of claim 43, wherein said predetermined thresholds comprises a low chrominance (U) threshold value and a high chrominance (U) threshold value for a chrominance (U) value associated with a pixel location, and a low chrominance (V) threshold value and a high chrominance (V) threshold value for a chrominance (V) value associated with a pixel location.

46. The method of claim 45, wherein the low and high chrominance (U) threshold values and the low and high chrominance (V) threshold values reflect the chrominance (U, V) ranges of human skin.

47. The method of claim 45, wherein the filtering comprises applying anisotropic diffusion to the pixel locations within the bounding shapes differently than to the pixel locations not enclosed within the bounding shapes.