PRE-PROCESSING METHOD AND SYSTEM FOR DATA REDUCTION OF VIDEO SEQUENCES AND BIT RATE REDUCTION OF COMPRESSED VIDEO SEQUENCES USING SPATIAL FILTERING

• US 20080292201A1
• Filed: 08/04/2008
• Published: 11/27/2008
• Est. Priority Date: 08/13/2003
• Status: Active Grant

• Current Assignee: Adriana Dumitras, James Oliver Normile, Ryan R. Salsbury
• Original Assignee: Adriana Dumitras, James Oliver Normile, Ryan R. Salsbury
• Inventors: Dumitras, Adriana, Salsbury, Ryan R., Normile, James Oliver
• Granted Patent: US 7,809,207 B2
• Time in Patent Office: Days
• US Class Current: 382/260
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/192 the adaptation method, adaptation tool or adaptation type being iterative or recursive
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence
• CPC Class Codes: H04N 5/21 Circuitry for suppressing or minimising disturbance, e.g. moiré or halo
• CPC Class Codes: H04N 9/8042 involving data reduction

Abstract

Methods for pre-processing video sequences prior to compression to provide data reduction of the video sequence. Also, after compression of the pre-processed video sequence, the bit rate of the pre-processed and compressed video sequence will be lower than the bit rate of the video sequence after compression but without pre-processing. Pre-processing may include spatial anisotropic diffusion filtering such as Perona-Malik filtering, Fallah-Ford filtering, or omni-directional filtering that extends Perona-Malik filtering to perform filtering in at least one diagonal direction. Pre-processing may also include performing filtering differently on a foreground region than on a background region of a video frame. This method includes identifying pixel locations having pixel values matching characteristics of human skin and determining a bounding shape for each contiguous grouping of matching pixel locations. The foreground region is comprised of pixel locations contained in a bounding shape and the background region is comprised of all other pixel locations.

Claims

1-7. -7. (canceled)

8. A method for pre-filtering an original video sequence, the original video sequence comprising a plurality of frames, each frame comprising a plurality of pixel locations where each pixel location comprises a pixel value, the method comprising:

specifying a bounding shape that encloses at least a portion of a region-of-interest in a particular frame from the plurality of frames of the original video sequence; and

filtering pixel locations in the bounding shape differently than other pixel locations in the particular frame.

9. The method of claim 8, wherein the bounding shape comprises one of at least four-sided, three-sided, and circular form.

10. The method of claim 8 wherein each pixel location in the region-of-interest has a chrominance value within a predetermined low chrominance threshold value and a predetermined high chrominance threshold value.

11. The method of claim 8, wherein the bounding shape encloses over ½

of the pixel locations in the region-of-interest.

12. The method of claim 8 wherein, a particular region of the particular frame within the bounding shape is referred to as a foreground region.

18. The method of claim 8 further comprising:

repeating specifying the bounding shape and filtering pixel locations for each frame of the plurality of frames in the original video sequence;

outputting a pre-filtered video sequence comprising a plurality of pre-filtered video frames; and

compressing the pre-filtered video sequence using a first compression method to produce a pre-filtered and compressed video sequence, wherein a bit rate associated with the pre-filtered and compressed video sequence is lower than a bit rate that would result from compressing the original video sequence using the first compression method without performing the specifying of the bounding shape and filtering of pixel locations.

13-17. -17. (canceled)

19-27. -27. (canceled)

28. A computer readable storage medium storing a computer program for pre-filtering an original video sequence, the original video sequence comprising a plurality of frames, each frame comprising a plurality of pixel locations where each pixel location comprises a pixel value, the computer program executable by at least one processor, the computer program comprising sets of instructions for:

specifying a bounding shape that encloses at least a portion of a region-of-interest of a particular frame from the plurality of frames of the original video sequence; and

filtering pixel locations in the bounding shape differently than other pixel locations in the particular frame.

29. The computer readable storage medium of claim 28, wherein each pixel location in the region-of-interest has a chrominance value within a predetermined low chrominance threshold value and a predetermined high chrominance threshold value.

30-43. -43. (canceled)

44. A method for pre-filtering a plurality of frames, each frame comprising a plurality of pixel locations, each pixel location comprising a pixel value, the method comprising:

automatically identifying a foreground region for a particular frame by identifying a region of interest in the particular frame, the region of interest being associated with the foreground region; and

identifying a background region of the particular frame; and

filtering the particular frame by filtering the foreground region and the background region differently.

45. The method of claim 44, wherein filtering the foreground region comprises filtering a set of pixel locations.

46. The method of claim 44, wherein filtering the background region comprises filtering a set of pixel locations.

47. The method of claim 44, wherein filtering the foreground region comprises defining a first binary mask for the foreground region, wherein filtering the background region comprises defining a second binary mask for the background region that is different than the first binary mask.

48. The method of claim 44, wherein automatically identifying the foreground region comprises identifying a set of pixel locations that comprises pixel values between a minimum chrominance value and a maximum chrominance value.

49. The method of claim 48, wherein the minimum and maximum chrominance values are defined by a chrominance range for human face or skin.

50. The method of claim 44, wherein each pixel location comprises first and second pixel values, wherein automatically identifying the foreground region comprises identifying a set of pixel locations, each particular pixel location from the set of pixel locations comprising (i) a particular first pixel value between a first minimum chrominance value and a first maximum chrominance value, and (ii) a particular second pixel value between a second minimum chrominance value and a second maximum chrominance value.

51. The method of claim 44, wherein automatically identifying the foreground region comprises specifying a bounding shape for at least a portion of the region of interest of the particular frame.

52. The method of claim 44, wherein automatically identifying the foreground region comprises identifying a plurality of foreground regions for the particular frame, wherein identifying the region of interest comprises identifying a plurality of region of interests.

53. The method of claim 44 further comprising combining the filtered background region and the filtered foreground region into a single filtered frame.

54. A method for pre-filtering a plurality of frames, the method comprising:

defining a first binary mask for a foreground region of a particular frame;

defining a second binary mask for a background region for the particular frame, the second binary mask being different than the first binary mask; and

filtering the particular frame by using the first binary mask to filter the foreground region and using the second binary mask to filter the background region.

55. The method of claim 54, wherein each frame comprises a plurality of pixel locations, the foreground region comprising a first set of pixel locations, the background region comprising a second set of pixel locations.

56. The method of claim 55, wherein the first binary mask comprises a plurality of values, each particular value for a particular pixel location of the particular frame, wherein the particular value is a value of 1 for each pixel location from the first set of pixel locations, wherein the particular value is a value of 0 for each pixel location from the second set of pixel locations.

57. The method of claim 55, wherein the second binary mask comprises a plurality of values, each particular value for a particular pixel location of the particular frame, wherein the particular value is a value of 1 for each pixel location from the second set of pixel locations, wherein the particular value is a value of 0 for each pixel location from the first set of pixel locations.

58. The method of claim 54, wherein identifying the first binary mask comprises specifying a bounding shape for a region of interest of the particular frame.

59. The method of claim 54, wherein the foreground region is filtered differently than the background region.