PRE-PROCESSING METHOD AND SYSTEM FOR DATA REDUCTION OF VIDEO SEQUENCES AND BIT RATE REDUCTION OF COMPRESSED VIDEO SEQUENCES USING SPATIAL FILTERING

• US 20080284904A1
• Filed: 04/21/2008
• Published: 11/20/2008
• Est. Priority Date: 08/13/2003
• Status: Active Grant

• Current Assignee: Adriana Dumitras, James Oliver Normile, Ryan R. Salsbury
• Original Assignee: Adriana Dumitras, James Oliver Normile, Ryan R. Salsbury
• Inventors: Dumitras, Adriana, Salsbury, Ryan R., Normile, James Oliver
• Granted Patent: US 8,615,042 B2
• Time in Patent Office: Days
• US Class Current: 348/399.1
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/192 the adaptation method, adaptation tool or adaptation type being iterative or recursive
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence
• CPC Class Codes: H04N 5/21 Circuitry for suppressing or minimising disturbance, e.g. moiré or halo
• CPC Class Codes: H04N 9/8042 involving data reduction

Abstract

Methods for pre-processing video sequences prior to compression to provide data reduction of the video sequence. Also, after compression of the pre-processed video sequence, the bit rate of the pre-processed and compressed video sequence will be lower than the bit rate of the video sequence after compression but without pre-processing. Pre-processing may include spatial anisotropic diffusion filtering such as Perona-Malik filtering, Fallah-Ford filtering, or omni-directional filtering that extends Perona-Malik filtering to perform filtering in at least one diagonal direction. Pre-processing may also include performing filtering differently on a foreground region than on a background region of a video frame. This method includes identifying pixel locations having pixel values matching characteristics of human skin and determining a bounding shape for each contiguous grouping of matching pixel locations. The foreground region is comprised of pixel locations contained in a bounding shape and the background region is comprised of all other pixel locations.

Claims

1. A method of pre-filtering an original video sequence, the method comprising:

a) receiving the original video sequence;

b) pre-filtering the original video sequence using anisotropic diffusion filtering;

c) outputting a pre-filtered video sequence; and

d) compressing the pre-filtered video sequence using a first compression method to produce a pre-filtered and compressed video sequence.

2. The method of claim 1 wherein the anisotropic diffusion filtering is Fallah-Ford diffusion filtering.

3. The method of claim 1 wherein the anisotropic diffusion filtering is Perona-Malik diffusion filtering.

4. The method of claim 1 wherein the anisotropic diffusion filtering performs diffusion filtering in at least one diagonal direction with respect to a pixel location of a video frame of the original video sequence.

5. A method of pre-processing a video frame having a plurality of pixels at pixel locations where each pixel location contains one or more pixel values, the method comprising:

performing anisotropic diffusion filtering on a pixel in the plurality of pixels, the pixel having a pixel location wherein the diffusion filtering is performed in at least one diagonal direction with respect to the pixel location.

6. The method of claim 5 wherein the diffusion filtering is performed in four horizontal or vertical directions (north, south, east and west) and four diagonal directions (north-east, north-west, south-east, and south-west) with respect to the pixel location.

7. The method of claim 6 wherein the diffusion filtering is expressed by the equation:

$I<br><br>\left(x,y,t+1\right)=I<br><br>\left(x,y,t\right)+\mathrm{\lambda <br><br>}<br><br>\left[\begin{array}{c}\underset{N,S,E,W}{\sum <br><br>}<br><br>c_m<br><br>\left(x,y,t\right)<br><br>\nabla <br><br>I_m<br><br>\left(x,y,t\right)+\\ \mathrm{\beta <br><br>}<br><br>\underset{\mathrm{NE},\mathrm{SE},\mathrm{SW},\mathrm{NW}}{\sum <br><br>}<br><br>c_n<br><br>\left(x,y,t\right)<br><br>\nabla <br><br>I_n<br><br>\left(x,y,t\right)\end{array}\right]$

where;

I(x, y, t) is a discrete image;

Δ

I(x, y, t) is the image gradient;

(x, y) specifies the pixel location in a discrete, two dimensional grid covering the video frame;

t denotes discrete time steps;

scalar constant λ

determines a rate of diffusion;

subscripts m and n correspond to eight directions of diffusion with respect to the pixel location (x, y), where m is a horizontal or vertical direction (north, south, east and west) and n is a diagonal direction (north-east, north-west, south-east, and south-west); and

notations c_m and c_n are diffusion coefficients where horizontal or vertical directions (north, south, east and west) are indexed by subscript m and diagonal directions (north-east, north-west, south-east, and south-west) are indexed by subscript n, each diffusion coefficient being referred to as an edge stopping function g(x) of ∇

I(x,y,t) in the corresponding direction as expressed in the following equations;

c_m(x,y,t)=g(∇

I_m(x,y,t))

c_n(x,y,t)=g(∇

I_n(x,y,t))

where g(x) satisfies the condition g(x)→

0 when x→

∞

such that the diffusion filtering is stopped across edges that are present in the video frame.

8-21. -21. (canceled)

22. A computer program product having a computer readable medium having computer program instructions recorded thereon, the computer program product comprising:

instructions for receiving the original video sequence;

instructions for pre-filtering the original video sequence using anisotropic diffusion filtering;

instructions for outputting a pre-filtered video sequence; and

instructions for compressing the pre-filtered video sequence.

23. The computer program product of claim 22 wherein the anisotropic diffusion filtering is Fallah-Ford diffusion filtering.

24. The computer program product of claim 22 wherein the anisotropic diffusion filtering is Perona-Malik diffusion filtering.

25. The computer program product of claim 22 wherein the anisotropic diffusion filtering performs diffusion filtering in at least one diagonal direction with respect to a pixel location of a video frame of the original video sequence.

26. A computer program product having a computer readable medium having computer program instructions recorded thereon for pre-processing a video frame having a plurality of pixels at pixel locations where each pixel location contains one or more pixel values, the computer program product comprising:

instructions for performing anisotropic diffusion filtering on a pixel in the plurality of pixels, the pixel having a pixel location wherein the diffusion filtering is performed in at least one diagonal direction with respect to the pixel location.

27. The computer program product of claim 26 wherein the diffusion filtering is performed in four horizontal or vertical directions (north, south, east and west) and four diagonal directions (north-east, north-west, south-east, and south-west) with respect to the pixel location.

28-30. -30. (canceled)

31. A system for processing an original video sequence, the original video sequence being comprised of a plurality of frames, each frame having a plurality of pixel locations where each pixel location contains a pixel value, the system comprising:

a pre-processing component that;

receives the original video sequence;

pre-filters the original video sequence using anisotropic diffusion filtering; and

outputs a pre-filtered video sequence; and

an encoder component coupled to pre-processing component, wherein the encoder component compresses the pre-filtered video sequence.

32. The system of claim 31 wherein the anisotropic diffusion filtering is Fallah-Ford diffusion filtering.

33. The system of claim 31 wherein the anisotropic diffusion filtering is Perona-Malik diffusion filtering.

34. The system of claim 31 wherein the anisotropic diffusion filtering performs diffusion filtering in at least one diagonal direction with respect to a pixel location of a video frame of the original video sequence.

35. A system for pre-processing a video frame having a plurality of pixels at pixel locations where each pixel location contains one or more pixel values, the system comprising:

a pre-processing component that performs anisotropic diffusion filtering on a pixel in the plurality of pixels, the pixel having a pixel location wherein the diffusion filtering is performed in at least one diagonal direction with respect to the pixel location.

36. The system of claim 35 wherein the diffusion filtering is performed in four horizontal or vertical directions (north, south, east and west) and four diagonal directions (north-east, north-west, south-east, and south-west) with respect to the pixel location.

37-39. -39. (canceled)

40. A system for pre-processing a video frame having a plurality of pixels at pixel locations where each pixel location contains one or more pixel values, the system comprising:

means for performing anisotropic diffusion filtering on a pixel in the plurality of pixels, the pixel having a pixel location wherein the diffusion filtering is performed in at least one diagonal direction with respect to the pixel location.

41. The system of claim 40 wherein the diffusion filtering is performed in four horizontal or vertical directions (north, south, east and west) and four diagonal directions (north-east, north-west, south-east, and south-west) with respect to the pixel location.

42-43. -43. (canceled)