Adaptive preprocessing method using feature-extracted video maps

• US 8,792,564 B2
• Filed: 10/28/2008
• Issued: 07/29/2014
• Est. Priority Date: 10/28/2008
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Youn, Jeongnam, Cho, Yushin, Hiranaka, Daisuke
• Primary Examiner(s): HOLDER, BRADLEY W
• Application Number:
  US12/259,649
• Publication Number:
  US 20100104027A1
• Time in Patent Office: 2,100 Days
• Field of Search: 375/240.29, 375/240, 375/240.01, 375/240.26
• US Class Current: 375/240.29
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/30 using hierarchical techniques, e.g. scalability H04N19/63 takes precedence
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression

Abstract

A method and apparatus for preprocessing a video signal prior to its being encoded. During preprocessing, features are extracted from the video signal into feature planes which are each adaptively filtered in response to estimations of feature activity. After feature extraction the signal which remains is referred to as a “remaining signal”, which is then noise filtered to remove a substantial portion of the noise. The feature planes and remaining signal are recombined into a preprocessed output signal configured for receipt and processing by a video encoder. A video encoder receiving the preprocessed video signal can generate higher quality and/or lower bit rate encoded video than one receiving the original source video. Preprocessing according to the invention is particularly well suited for use with encoders adapted to remove temporal and/or spatial redundancy, such as those implemented according to the MPEG2/4, or AVC/H.264 standards.

Claims

1. An apparatus for preprocessing a video signal prior to encoding, comprising:

means for extracting features from a source video into feature planes which are adaptively filtered;

partitioning feature planes into blocks, which are categorized into one of multiple filter parameters selected from a plurality of predefined filter parameters;

wherein a group of said features are extracted from the group of features consisting of edges, texture complexity, gradient characteristic, and activity;

means for noise reduction filtering of a remaining signal, which remains after extracting features from said source video;

means for adaptively filtering feature planes based on a level of feature activity; and

means for recombining adaptively filtered feature planes with a noise reduced version of said remaining signal into a preprocessed video output configured for receipt by a video encoder;

wherein said apparatus performs preprocessing of video signals toward increasing coding efficiency at a recipient video encoder by providing an increase in video quality and/or a lowered bit rate for a given encoded video quality.

2. A method of preprocessing a source video signal prior to encoding, comprising:

extracting features from a source video and producing feature planes and a remaining signal without the extracted features within a preprocessor;

partitioning each of said feature planes into blocks, said blocks being categorized into one of multiple filter parameters selected from a plurality of predefined filter parameters;

wherein a group of said features are extracted from the group of features consisting of edges, texture complexity, gradient characteristic, and activity;

noise reduction filtering of a remaining signal, which remains after extracting features from said source video;

estimation of feature activity for said feature planes;

adaptive strength filtering on said feature planes based on a level of feature activity; and

combining adaptively filtered feature planes with a noise reduced version of said remaining signal to generate a preprocessed video output configured for receipt by a video encoder;

wherein said method performs preprocessing of a source video signal toward increasing coding efficiency at a recipient video encoder by providing an increase in video quality and/or a lowered bit rate for a given encoded video quality.

3. A method as recited in claim 2, wherein the features to be extracted are selected in response to external control, programming and/or data available to said preprocessor.

4. A method as recited in claim 2, wherein said noise reduction filtering is configured to remove at least a portion of random noise within said remaining signal.

5. A method as recited in claim 2, wherein said noise reduction filtering comprises a low pass filter.

6. A method as recited in claim 2, wherein said extracted features are processed adaptively based on feature activity within each plane of extracted features.

7. A method as recited in claim 2, wherein said preprocessing aggressively processes areas of complicated texture that are not sensitive to a human eye while other feature areas are processed, from delicately to aggressively, in response to their level of activity.

8. A method as recited in claim 2, wherein said level of feature activity is estimated

within each block in response to a human perceptual model to output a degree of activity.

9. A method as recited in claim 2, wherein only the preprocessed video output need be communicated to an encoder, and no additional information needs to be sent from the preprocessor to an encoder.

10. A method as recited in claim 2, wherein said preprocessing output is compatible with encoders operating according to MPEG2/4 standards.

11. A method as recited in claim 2, wherein said preprocessing output is compatible with encoders operating according to AVC/H.264 standards.

12. A method as recited in claim 2, wherein said preprocessing method is performed in response to programming, retrieved from a memory, and executing on at least one computer processor.

13. A method as recited in claim 12, wherein said at least one computer processor is integrated within a preprocessing module configured for connection to an encoder device.

14. A method as recited in claim 12, wherein said at least one computer processor is integrated within an encoder device configured to perform preprocessing and encoding.

15. A method as recited in claim 14, wherein at least a first computer processor performs preprocessing while at least a second computer processor performs encoding of the preprocessed video output.

16. A method as recited in claim 14, wherein said at least one computer processor performs both preprocessing functions and encoding functions.

17. An apparatus for preprocessing video signals in preparation for encoding, comprising:

a computer configured for processing received video signals; and

programming executable on said computer for,

extracting features from said video signals and producing feature planes and a remaining signal without the extracted features,

partitioning each feature plane of said feature planes into blocks, wherein said blocks are categorized into one of multiple filter parameters selected from a plurality of predefined filter parameters;

wherein a group of said features are extracted from the group of features consisting of edges, texture complexity, gradient characteristic, and activity;

noise reduction filtering of a remaining signal, which remains after extracting features from said video signals,

estimation of feature activity for said feature planes,

adaptive strength filtering on said feature planes based on a level of feature activity, and

combining adaptively filtered feature planes with a noise reduced version of said remaining signal to generate a preprocessed video output configured for receipt by a video encoder;

wherein preprocessing the video signals increases coding efficiency at a recipient video encoder providing an increase in video quality and/or a lowered bit rate for a given encoded video quality.

18. A non-transitory computer-readable media containing a computer program executable on a computer configured for processing a video signal and causing the computer to preprocess the video signal in preparation for receipt by an encoder, in response to steps, comprising:

extracting features from a video signal and producing feature planes and a remaining signal without the extracted features within a preprocessor;

partitioning each feature plane of said feature planes into blocks, wherein said blocks are categorized into one of multiple filter parameters selected from a plurality of predefined filter parameters;

wherein a group of said features are extracted from the group of features consisting of edges, texture complexity, gradient characteristic, and activity;

noise reduction filtering of the remaining signal, which remains after extracting features from said video signal;

estimation of feature activity for said feature planes;

adaptive strength filtering on said feature planes based on a level of feature activity; and

combining adaptively filtered feature planes with a noise reduced version of said remaining signal to generate a preprocessed video output configured for receipt by a video encoder;

wherein preprocessing the video signal increases coding efficiency at a recipient video encoder by providing an increase in video quality and/or a lowered bit rate for a given encoded video quality.