Content classification for multimedia processing

• US 9,197,912 B2
• Filed: 03/10/2006
• Issued: 11/24/2015
• Est. Priority Date: 03/10/2005
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Raveendran, Vijayalakshmi R., Bhamidipati, Phanikumar, Walker, Gordon Kent
• Primary Examiner(s): NGUYEN, STEVEN H D
• Application Number:
  US11/373,577
• Publication Number:
  US 20060222078A1
• Time in Patent Office: 3,546 Days
• US Class Current: 1/1
• CPC Class Codes: H04N 19/103 Selection of coding mode or of prediction mode
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/139 Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/142 Detection of scene cut or scene change
• CPC Class Codes: H04N 19/147 according to rate distortion criteria rate-distortion as a criterion for motion estimation H04N19/567
• CPC Class Codes: H04N 19/149 by estimating the code amount by means of a model, e.g. mathematical model or statistical model
• CPC Class Codes: H04N 19/154 Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion use of rate-distortion criteria H04N19/147
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/177 the unit being a group of pictures [GOP]
• CPC Class Codes: H04N 19/196 being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters processing of motion vectors H04N19/513
• CPC Class Codes: H04N 19/198 including smoothing of a sequence of encoding parameters, e.g. by averaging, by choice of the maximum, minimum or median value
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/57 Motion estimation characterised by a search window with variable size or shape
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/87 involving scene cut or scene change detection in combination with video compression
• CPC Class Codes: H04N 21/2365 Multiplexing of several video streams
• CPC Class Codes: H04N 21/4347 : Demultiplexing of several video streams

Abstract

An apparatus and method for processing multimedia data, such as, for example, video data, audio data, or both video and audio data for encoding utilizing a determined content classification is claimed. Processing the multimedia data includes determining complexity of multimedia data, classifying the multimedia data based on the determined complexity, and, determining a bit rate for encoding the multimedia data based on its classification. The complexity can include a spatial complexity component and a temporal complexity component of the multimedia data. The multimedia data is classified using content classifications, which are based on a visual quality value for viewing multimedia data, using the spatial complexity, the temporal complexity, or both the spatial complexity and temporal complexity.

Claims

1. A method of processing multimedia data comprising:

retrieving, from a memory device, preprocessed information related to the multimedia data;

determining a complexity of the multimedia data from the retrieved information, wherein determining the complexity comprises determining spatial complexity and temporal complexity of the multimedia data; and

classifying the multimedia data based on the determined complexity, wherein classifying the multimedia data comprises;

associating the spatial complexity with a texture value;

associating the temporal complexity with a motion value; and

assigning a content classification to the multimedia data based on the texture value and the motion value;

determining a content classification curve based on the assigned content classification; and

based on a target quality level for the multimedia data and the determined content classification curve, determining a bit rate for the multimedia data.

2. The method of claim 1, wherein determining spatial complexity comprises determining at least one human visual sensitivity value for at least one block of the multimedia data.

3. The method of claim 1, wherein determining spatial complexity comprises determining at least one contrast ratio value for at least one block of the multimedia data.

4. The method of claim 1, wherein determining temporal complexity comprises determining at least one motion vector (MV) for at least one block of the multimedia data.

5. The method of claim 1, wherein determining temporal complexity further comprises associating a sum of absolute pixel differences associated with at least one block of the multimedia data and a reference block.

6. The method of claim 1, wherein determining temporal complexity comprises determining at least one frame difference value for at least one block of the multimedia data.

7. The method of claim 6, wherein the at least one frame difference value includes a SAD_MV value determined for the at least one block of the multimedia data.

8. The method of claim 1, wherein classifying the multimedia data comprises using a plurality of content classifications.

9. An apparatus for processing multimedia data comprising:

means for retrieving, from a memory device, preprocessed information related to the multimedia data;

means for determining a complexity of the multimedia data from the retrieved information, wherein the means for determining the complexity comprises means for determining spatial complexity and temporal complexity of the multimedia data; and

means for classifying the multimedia data content based on the determined complexity, wherein the means for classifying the multimedia data comprises;

means for associating the spatial complexity with a texture value;

means for associating the temporal complexity with a motion value; and

means for assigning a content classification to the multimedia data based on the texture value and the motion value;

means for determining a content classification curve based on the assigned content classification;

means for determining a bit rate for the multimedia data based on a target quality level for the multimedia data and the determined content classification curve.

10. The apparatus of claim 9, wherein the means for determining spatial complexity comprises means for determining at least one human visual sensitivity value for at least one block of the multimedia data.

11. The apparatus of claim 9, wherein the means for determining spatial complexity comprises means for determining at least one contrast ratio value for at least one block of the multimedia data.

12. The apparatus of claim 9, wherein the means for determining temporal complexity comprises means for determining at least one motion vector (MV) for at least one block of the multimedia data.

13. The apparatus of claim 12, wherein the means for determining temporal complexity further comprises means for determining a sum of absolute pixel differences associated with at least one block of the multimedia data and a reference block.

14. The apparatus of claim 9, wherein the means for determining temporal complexity comprises means for determining at least one frame difference value determined for at least one block of the multimedia data.

15. The apparatus of claim 14, wherein the at least one frame difference value is a SAD_MV value determined for at least one block of the multimedia data.

16. The apparatus of claim 9, wherein means for classifying the multimedia data content comprises a plurality of means for content classifications.

17. A non-transitory computer readable storage medium storing instructions that when executed by one or more processors cause the one or more processors to:

determine complexity of multimedia data, wherein the one or more processors determine the complexity by determining spatial complexity and temporal complexity of the multimedia data;

classify multimedia data based on the determined complexity, wherein classifying the multimedia data comprises;

associating the spatial complexity with a texture value;

associating the temporal complexity with a motion value; and

assigning a content classification to the multimedia data based on the texture value and the motion value;

determine a content classification curve based on the assigned content classification; and

based on a target quality level for the multimedia data and the determined content classification curve, determine a bit rate for the multimedia data.

18. The non-transitory computer-readable storage medium of claim 17, wherein the one or more processors determine temporal complexity by determining at least one human visual sensitivity value for at least one block of the multimedia data.

19. The non-transitory computer-readable storage medium of claim 17, wherein the one or more processors determine temporal complexity by determining at least one contrast ratio value for at least one block of the multimedia data.

20. The non-transitory computer-readable storage medium of claim 17, wherein the one or more processors determine temporal complexity by determining at least one motion vector (MV) for at least one block of the multimedia data.

21. The non-transitory computer-readable storage medium of claim 17, wherein the one or more processors determine temporal complexity by associating a sum of absolute pixel differences associated with at least one block of the multimedia data and a reference block.

22. The non-transitory computer-readable storage medium of claim 17, wherein the one or more processors determine temporal complexity by determining at least one frame difference value for at least one block of the multimedia data.

23. The non-transitory computer-readable storage medium of claim 22, wherein the at least one frame difference value includes a SAD_MV value determined for the at least one block of the multimedia data.

24. The non-transitory computer-readable storage medium of claim 17, wherein the one or more processors classify the multimedia data by using a plurality of content classifications.

25. A device for processing multimedia data, the device comprising:

a memory;

one or more processors configured to;

retrieve, from the memory, preprocessed information related to the multimedia data;

determine a complexity of the multimedia data from the retrieved information, wherein determining the complexity comprises determining spatial complexity and temporal complexity of the multimedia data;

classify the multimedia data based on the determined complexity, wherein classifying the multimedia data comprises;

associating the spatial complexity with a texture value;

associating the temporal complexity with a motion value; and

assigning a content classification to the multimedia data based on the texture value and the motion value;

determine a content classification curve based on the assigned content classification; and

based on a target quality level for the multimedia data and the determined content classification curve, determine a bit rate for the multimedia data.

26. The device of claim 25, wherein the one or more processors determine temporal complexity by determining at least one human visual sensitivity value for at least one block of the multimedia data.

27. The device of claim 25, wherein the one or more processors determine temporal complexity by determining at least one contrast ratio value for at least one block of the multimedia data.

28. The device of claim 25, wherein the one or more processors determine temporal complexity by determining at least one motion vector (MV) for at least one block of the multimedia data.

29. The device of claim 25, wherein the one or more processors determine temporal complexity by associating a sum of absolute pixel differences associated with at least one block of the multimedia data and a reference block.

30. The device of claim 25, wherein the one or more processors determine temporal complexity by determining at least one frame difference value for at least one block of the multimedia data.

31. The device of claim 30, wherein the at least one frame difference value includes a SAD_MV value determined for the at least one block of the multimedia data.

32. The device of claim 25, wherein the one or more processors classify the multimedia data by using a plurality of content classifications.