Pre-processing method and system for data reduction of video sequences and bit rate reduction of compressed video sequences using spatial filtering

• US 8,615,042 B2
• Filed: 04/21/2008
• Issued: 12/24/2013
• Est. Priority Date: 08/13/2003
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Dumitras, Adriana, Normile, James Oliver, Salsbury, Ryan R.
• Primary Examiner(s): Vaughn, Jr., William C
• Assistant Examiner(s): PEREZ FUENTES, LUIS M
• Application Number:
  US12/107,072
• Publication Number:
  US 20080284904A1
• Time in Patent Office: 2,073 Days
• Field of Search: None
• US Class Current: 375/240.1
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/192 the adaptation method, adaptation tool or adaptation type being iterative or recursive
• CPC Class Codes: H04N 19/80 Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation H04N19/635, H04N19/86 take precedence
• CPC Class Codes: H04N 5/21 Circuitry for suppressing or minimising disturbance, e.g. moiré or halo
• CPC Class Codes: H04N 9/8042 involving data reduction

Abstract

Methods for pre-processing video sequences prior to compression to provide data reduction of the video sequence. Also, after compression of the pre-processed video sequence, the bit rate of the pre-processed and compressed video sequence will be lower than the bit rate of the video sequence after compression but without pre-processing. Pre-processing may include spatial anisotropic diffusion filtering such as Perona-Malik filtering, Fallah-Ford filtering, or omni-directional filtering that extends Perona-Malik filtering to perform filtering in at least one diagonal direction. Pre-processing may also include performing filtering differently on a foreground region than on a background region of a video frame. This method includes identifying pixel locations having pixel values matching characteristics of human skin and determining a bounding shape for each contiguous grouping of matching pixel locations. The foreground region is comprised of pixel locations contained in a bounding shape and the background region is comprised of all other pixel locations.

Claims

1. A method of pre-filtering video comprising:

receiving an original video sequence comprising a set of video frames, each video frame having a set of pixels;

pre-filtering each of a plurality of video frames to generate a pre-filtered video sequence, said pre-filtering comprising applying anisotropic diffusion filtering to each of a plurality of pixels in each of the plurality of video frames in order to reduce the differences between a plurality of neighboring pixels in the plurality of video frames, wherein the anisotropic diffusion filtering is performed for each pixel in the plurality of pixels in four horizontal and vertical directions comprising north, south, east and west and four diagonal directions comprising north-east, north-west, south-east, and south-west with respect to the pixel location of the pixel; and

encoding the pre-filtered video sequence using an encoding method to produce a pre-filtered and encoded video sequence that has a smaller bit rate than an encoded video sequence that would result from encoding the original video sequence using the same encoding method.

2. The method of claim 1, wherein the anisotropic diffusion filtering is Fallah-Ford diffusion filtering.

3. The method of claim 1, wherein the anisotropic diffusion filtering is Perona-Malik diffusion filtering.

4. A method of pre-processing a video frame comprising:

receiving an original video frame having a set of pixels, each pixel located at a pixel location;

pre-processing the original video frame to generate a pre-processed video frame that has less data to encode than the original video frame, the pre-processing comprising;

performing anisotropic diffusion filtering on each of a plurality of pixels of the original video frame in an orthogonal direction with respect to the pixel location of the pixel;

performing the anisotropic diffusion filtering on the pixel in at least one diagonal direction with respect to the pixel location of the pixel by scaling, by a factor that is a function of a dimension of the video frame, a difference in pixel values of the pixel and another pixel that is neighboring the pixel in the diagonal direction; and

outputting the pre-processed video frame to an encoder to encode the pre-processed video frame.

5. The method of claim 4, wherein the anisotropic diffusion filtering is performed in four horizontal or vertical directions comprising north, south, east and west and four diagonal directions comprising north-east, north-west, south-east, and south-west with respect to the pixel location of the pixel.

6. The method of claim 5, wherein the anisotropic diffusion filtering is expressed by the equation:

7. A non-transitory computer readable medium storing a program for pre-filtering an original video sequence, the program comprising sets of instructions for:

receiving the original video sequence comprising a set of video frames, each video frame having a set of pixels;

pre-filtering each of a plurality of video frames to generate a pre-filtered video sequence, said pre-filtering comprising applying anisotropic diffusion filtering to each of a plurality of pixels in each of the plurality of video frames in order to reduce the differences between a plurality of neighboring pixels in the plurality of video frames, wherein the anisotropic diffusion filtering is performed for each pixel in the plurality of pixels in four horizontal and vertical directions comprising north, south, east and west and four diagonal directions comprising north-east, north-west, south-east, and south-west with respect to the pixel location of the pixel; and

encoding the pre-filtered video sequence using an encoding method to produce a pre-filtered and encoded video sequence that has a smaller bit rate than an encoded video sequence that would result from encoding the original video sequence using the same encoding method.

8. The non-transitory computer readable medium of claim 7, wherein the anisotropic diffusion filtering is Fallah-Ford diffusion filtering.

9. The non-transitory computer readable medium of claim 7, wherein the anisotropic diffusion filtering is Perona-Malik diffusion filtering.

10. A non-transitory computer readable medium storing a program for pre-processing a video frame:

receiving an original video frame having a set of pixels, each pixel located at a pixel location;

pre-processing the original video frame to generate a pre-processed video frame, the set of instructions for pre-processing comprising sets of instructions for;

performing anisotropic diffusion filtering on each of a plurality of pixels of the original video frame in an orthogonal direction with respect to the pixel location of the pixel in order to reduce smoothing of edges in the pre-processed video frame; and

performing the anisotropic diffusion filtering on the pixel in at least one diagonal direction with respect to the pixel location of the pixel by scaling, by a factor that is a function of a dimension of the video frame, a difference in pixel values of the pixel and another pixel that is neighboring the pixel in the diagonal direction in order to further reduce smoothing of edges in the pre-processed video frame; and

outputting the pre-processed video frame to an encoder to encode the pre-processed video frame.

11. The non-transitory computer readable medium of claim 10, wherein the anisotropic diffusion filtering is performed in four horizontal or vertical directions comprising north, south, east and west and four diagonal directions comprising north-east, north-west, south-east, and south-west with respect to the pixel location of the pixel.

12. A system to process video comprising:

a pre-processing component to receive an original video sequence comprising a set of video frames, each video frame having a set of pixels, and to pre-filter each of a plurality of video frames to generate a pre-filtered video sequence, wherein said pre-processing component pre-filters each of the plurality of video frames by applying anisotropic diffusion filtering to each of a plurality of pixels in each of the plurality of video frames in order to reduce the differences between a plurality of neighboring pixels in the plurality of video frames, wherein the anisotropic diffusion filtering is performed for each pixel in the plurality of pixels in four horizontal and vertical directions comprising north, south, east and west and four diagonal directions comprising north-east, north-west, south-east, and south-west with respect to the pixel location of the pixel; and

an encoder component to encode the pre-filtered video sequence using an encoding method to produce a pre-filtered and encoded video sequence that has a smaller bit rate than an encoded video sequence that would result from encoding the original video sequence using the same encoding method, the encoder component coupled to pre-processing component.

13. The system of claim 12, wherein the anisotropic diffusion filtering is Fallah-Ford diffusion filtering.

14. The system of claim 12, wherein the anisotropic diffusion filtering is Perona-Malik diffusion filtering.

15. A system to pre-process a video frame, the system comprising:

a pre-processing component to receive an original video frame having a set of pixels, each pixel located at a pixel location, and to pre-process the original video frame to generate a pre-processed video frame that has less data to encode than the original video frame, wherein the pre-processing component pre-processes the original video frame by performing anisotropic diffusion filtering on each of a plurality of pixels of the original video frame in an orthogonal direction with respect to the pixel location of the pixel and by performing the anisotropic diffusion filtering on the pixel in at least one diagonal direction with respect to the pixel location of the pixel by scaling, by a factor that is a function of a dimension of the video frame, a difference in pixel values of the pixel and another pixel that is neighboring the pixel in the diagonal direction; and

an encoder component to encode the pre-processed video frame, the encoder component coupled to the pre-processing component.

16. The system of claim 15, wherein the anisotropic diffusion filtering is performed in four horizontal or vertical directions comprising north, south, east and west and four diagonal directions comprising north-east, north-west, south-east, and south-west with respect to the pixel location of the pixel.