Method and apparatus for data compression using content-based features

• US 8,787,454 B1
• Filed: 07/13/2011
• Issued: 07/22/2014
• Est. Priority Date: 07/13/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Chechik, Gal, Yi, Xiaoquan, Aradhye, Hrishikesh
• Primary Examiner(s): Rao, Andy
• Assistant Examiner(s): ELAHI, SHAN E
• Application Number:
  US13/182,136
• Time in Patent Office: 1,105 Days
• Field of Search: 375/240.08, 375/240.09, 375/240.01, 375/240.02, 375/240.04
• US Class Current: 375/240.08
• CPC Class Codes: G06F 16/433 using audio data
• CPC Class Codes: G06F 21/10 Protecting distributed programs or content, e.g. vending or licensing of copyrighted material protection in video systems or pay television H04N7/16 ; Digital rights management [DRM]
• CPC Class Codes: G06T 9/20 Contour coding, e.g. using detection of edges
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/136 Incoming video signal characteristics or properties
• CPC Class Codes: H04N 19/154 Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion use of rate-distortion criteria H04N19/147
• CPC Class Codes: H04N 19/23 with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic

Abstract

Disclosed herein are methods and apparatuses for compressing a video signal. In one embodiment, the method includes storing a function derived from a set of human ratings in a memory, identifying within at least a portion of the video signal at least one content-based feature, inputting the at least one identified content-based feature into the stored function, determining a compression ratio based on the function using a processor and generating a compressed video signal at the determined compression ratio.

Claims

1. A method for encoding a video stream, the method comprising:

identifying a current frame from the video stream;

identifying, within a portion of the current frame, a content-based feature;

determining, by a processor in response to instructions stored on a tangible non-transitory storage medium, an encoding quality parameter based on the content-based feature, wherein determining the encoding quality parameter includes;

identifying a first candidate encoding quality parameter,

generating a first predicted perceptual quality for the current frame based on the first candidate encoding quality parameter and the content-based feature,

identifying a second candidate encoding quality parameter,

generating a second predicted perceptual quality for the current frame based on the second candidate encoding quality parameter and the content-based feature, and

identifying one of the first candidate encoding quality parameter or the second candidate encoding quality parameter as the encoding quality parameter based on the first predicted perceptual quality and the second predicted perceptual quality; and

generating an encoded frame by encoding the current frame using the encoding quality parameter.

2. The method of claim 1, wherein identifying one of the first candidate encoding quality parameter or the second candidate encoding quality parameter as the encoding quality parameter includes:

identifying the first candidate encoding quality parameter as the encoding quality parameter wherein the second predicted perceptual quality is less than a minimum quality threshold; and

identifying the first candidate encoding quality parameter as the encoding quality parameter wherein the second predicted perceptual quality is at least a minimum quality threshold.

3. The method of claim 1, wherein determining the encoding quality parameter based on the content-based feature includes identifying a result of a function of the content-based feature and a candidate encoding quality parameter.

4. The method of claim 3, further comprising:

updating the function based on a plurality of video streams other than the video stream.

5. The method of claim 3, wherein determining the encoding quality parameter based on the content-based feature includes updating the function based on the current frame.

6. The method of claim 1, wherein identifying the portion of the current frame including the content-based feature includes identifying a plurality of portions of the current frame, each portion from the plurality of portions including a respective content-based feature from a plurality of content-based features.

7. The method of claim 6, wherein determining the encoding quality parameter based on the content-based feature includes identifying a result of a function of the plurality of content-based features and a candidate encoding quality parameter.

8. The method of claim 7, wherein identifying the result of the function includes weighting each content-based feature from the plurality of content-based features.

9. The method of claim 7, wherein identifying the result of the function includes weighting a combination of content-based features from the plurality of content-based features.

10. The method of claim 1, wherein the content-based feature includes high frequency content, highly textured content, or a defined object.

11. The method of claim 1, wherein the content-based feature includes high frequency content and wherein the portion includes a small area of pixels having a high inter-pixel variance relative to a second portion of the current frame, the second portion including pixels having relatively low inter-pixel variance.

12. The method of claim 1, wherein a magnitude of motion between a location of the content-based feature in the current frame and corresponding location of the content-based feature in a reference frame is large relative to a magnitude of motion between a location of a second content-based feature in the current frame and corresponding location of the second content-based feature in the reference frame.

13. The method of claim 1, wherein identifying the portion of the current frame includes determining whether the video stream is associated with audio content.

14. The method of claim 1, wherein identifying the portion of the current frame including the content-based feature includes using a local binary pattern corresponding with the portion and representing the content-based feature, motion vector information corresponding with the portion and representing the content-based feature, transform coefficient frequency information corresponding with the portion and representing the content-based feature, or color histogram information corresponding with the portion and representing the content-based feature.

15. A method for encoding a video stream, the method comprising:

identifying a current frame from the video stream;

identifying, within a portion of the current frame, a content-based feature;

determining, by a processor in response to instructions stored on a non-transitory storage medium, an encoding quality parameter based on a function of the content-based feature and a plurality of candidate encoding quality parameters, wherein the function is based on a perceived encoding quality of a frame of another video stream, the frame of the other video stream including a content-based feature corresponding to the content-based feature of the current frame, the perceived encoding quality associated with a candidate encoding quality parameter from the plurality of candidate encoding quality parameters; and

generating an encoded frame by encoding the current frame using the encoding quality parameter.

16. The method of claim 15, wherein determining the encoding quality parameter includes:

identifying a first candidate encoding quality parameter from the plurality of candidate encoding quality parameters;

generating a first predicted perceptual quality for the current frame by determining a result of the function based on the first candidate encoding quality parameter and the content-based feature;

identifying a second candidate encoding quality parameter from the plurality of candidate encoding quality parameters;

generating a second predicted perceptual quality for the current frame by determining a result of the function based on the second candidate encoding quality parameter and the content-based feature; and

identifying one of the first candidate encoding quality parameter or the second candidate encoding quality parameter as the encoding quality parameter based on the first predicted perceptual quality and the second predicted perceptual quality.

17. The method of claim 16, wherein identifying one of the first candidate encoding quality parameter or the second candidate encoding quality parameter as the encoding quality parameter includes:

identifying the first candidate encoding quality parameter as the encoding quality parameter wherein the second predicted perceptual quality is less than a minimum quality threshold; and

identifying the first candidate encoding quality parameter as the encoding quality parameter wherein the second predicted perceptual quality is at least a minimum quality threshold.

18. The method of claim 15, wherein identifying the portion of the current frame including the content-based feature includes identifying a plurality of portions of the current frame, each portion from the plurality of portions including a respective content-based feature from a plurality of content-based features.

19. A method for encoding a video stream, the method comprising:

identifying a current frame from the video stream;

identifying a plurality of content-based features, such that a respective portion from a plurality of portions of the current frame includes each respective content-based feature from the plurality of content-based features;

determining, by a processor in response to instructions stored on a non-transitory storage medium, an encoding quality parameter based on a function of the plurality of content-based features and a plurality of candidate encoding quality parameters; and

generating an encoded frame by encoding the current frame using the encoding quality parameter.