Method of encoding video with film grain

• US 11,451,798 B2
• Filed: 12/08/2015
• Issued: 09/20/2022
• Est. Priority Date: 01/05/2015
• Status: Active Grant

• Current Assignee: ARRIS Enterprises LLC (CommScope Holding Company, Inc.)
• Original Assignee: ARRIS Enterprises LLC (CommScope Holding Company, Inc.)
• Inventors: Zhou, Wei, Perera, Jane, Michelsen, Wayne D., Chanda, Swaroop
• Primary Examiner(s): Nirjhar, Nasim N
• Application Number:
  US14/962,814
• Publication Number:
  US 20160198165A1
• Time in Patent Office: 2,478 Days
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/117 Filters, e.g. for pre-processing or post-processing sub-band filter banks H04N19/635
• CPC Class Codes: H04N 19/119 Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/179 the unit being a scene or a shot
• CPC Class Codes: H04N 19/184 the unit being bits, e.g. of the compressed video stream
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/593 involving spatial prediction techniques
• CPC Class Codes: H04N 19/70 characterised by syntax aspects related to video coding, e.g. related to compression standards
• CPC Class Codes: H04N 19/85 using pre-processing or post-processing specially adapted for video compression
• CPC Class Codes: H04N 19/86 involving reduction of coding artifacts, e.g. of blockiness

Abstract

A system for providing improved video quality and compression efficiency during encoding by detecting video segments having film grain approaching the “Red Lady” problem. The system detects when film grain approaches the level of the “Red Lady” problem by measuring frame-by-frame temporal differences (ME scores). From the ME scores, two key indicators are identified: (1) The average temporal difference in frames with an intermediate motion level higher than frames of non-noisy video; and (2) The fluctuation of the temporal differences between frames in a group is very small. When these indicators identify a high film video, a signal is provided to an encoder which allocates less bits to I frames and more bits to P and B frames than for other frames of video without comparable film grain.

Claims

1. A method for encoding video comprising:

obtaining video frames provided to an encoder configured to compresses the video frames;

obtaining motion estimation (ME) data for the video frames;

calculating a ME score for the video frames, wherein the ME score provides a measurement of plotted frame-by-frame temporal differences;

calculating an ME stability value measuring a variance of the ME score over time; and

providing a high film grain indication signal to the encoder indicating a particular segment of the video frames contains high film grain when the ME score for the segment exceeds a first threshold and the ME stability value for the segment is below a second threshold,

wherein when the high film grain indication signal is provided, the encoder compresses the video frames by allocating less bits to I frames and more bits to P and B type frames than to other frames of the video.

2. The method of claim 1, wherein when the high film grain indication is not provided, the encoder allocates more bits to the I frame than when the high film grain indication is provided.

3. The method of claim 1, wherein the first threshold is greater than 20.

4. An apparatus to encode video frames, the apparatus comprising:

an encoder configured to compresses the video frames and having a first input for receiving video frames to be processed and a second input for receiving parameter data to enable the encoder to allocate bits for frames for encoding;

a frame buffer having an input receiving the video frames and an output providing the first input to the encoder;

a preprocessor for receiving the video frames and having an output providing the second input to the controller;

a preprocessor memory connected to the processor for storing code that is executable by the preprocessor to determine the parameter data to enable the encoder to allocate bits, the code causing the preprocessor to perform the following steps;

obtaining motion estimation (ME) data for the video frames;

calculating a ME score for the video frames, wherein the ME score provides a measurement of plotted frame-by-frame temporal differences;

calculating an ME stability value measuring a variance of the ME score over time; and

providing a high film grain indication signal to the encoder indicating a particular segment contains high film grain when the ME score exceeds a first threshold and the ME stability value is below a second threshold,

wherein when the high film grain signal is provided, the encoder compresses the video frames by allocating less bits to I frames and more bits to P and B type frames than to other frames of the video.

5. The apparatus of claim 4, wherein when the high film grain indication is not provided, the encoder allocates more bits to the I frame than when the high film grain indication is provided.

6. The apparatus of claim 4, wherein the first threshold is greater than 20.