Scalability techniques based on content information

• US 20070081586A1
• Filed: 09/26/2006
• Published: 04/12/2007
• Est. Priority Date: 09/27/2005
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Chen, Peisong, Shi, Fang, Raveendran, Vijayalakshmi, Tian, Tao, Bhamidipati, Phanikumar, Subramania, Sitaraman, Walker, Gordon, Oguz, Seyfullah
• Granted Patent: US 9,113,147 B2
• Time in Patent Office: Days
• US Class Current: 375/240.100
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/114 Adapting the group of pictures [GOP] structure, e.g. number of B-frames between two anchor frames H04N19/107 takes precedence
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/142 Detection of scene cut or scene change
• CPC Class Codes: H04N 19/147 according to rate distortion criteria rate-distortion as a criterion for motion estimation H04N19/567
• CPC Class Codes: H04N 19/149 by estimating the code amount by means of a model, e.g. mathematical model or statistical model
• CPC Class Codes: H04N 19/154 Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion use of rate-distortion criteria H04N19/147
• CPC Class Codes: H04N 19/159 Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/18 the unit being a set of transform coefficients
• CPC Class Codes: H04N 19/187 the unit being a scalable video layer
• CPC Class Codes: H04N 19/194 involving only two passes
• CPC Class Codes: H04N 19/30 using hierarchical techniques, e.g. scalability H04N19/63 takes precedence
• CPC Class Codes: H04N 19/36 Scalability techniques involving formatting the layers as a function of picture distortion after decoding, e.g. signal-to-noise [SNR] scalability
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/577 Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 19/65 : using error resilience
• CPC Class Codes: H04N 19/86 : involving reduction of coding artifacts, e.g. of blockiness
• CPC Class Codes: H04N 19/87 : involving scene cut or scene change detection in combination with video compression
• CPC Class Codes: H04N 19/89 : involving methods or arrangements for detection of transmission errors at the decoder
• CPC Class Codes: H04N 21/234309 : by transcoding between formats or standards, e.g. from MPEG-2 to MPEG-4 or from Quicktime to Realvideo conversion of standards in analog television systems H04N7/01
• CPC Class Codes: H04N 21/2353 : specifically adapted to content descriptors, e.g. coding, compressing or processing of metadata
• CPC Class Codes: H04N 21/2389 : Multiplex stream processing, e.g. multiplex stream encrypting
• CPC Class Codes: H04N 5/144 : Movement detection for video coding H04N19/503; analysis of motion in general G06T7/20
• CPC Class Codes: H04N 5/147 : Scene change detection
• CPC Class Codes: H04N 5/21 : Circuitry for suppressing or minimising disturbance, e.g. moiré or halo
• CPC Class Codes: H04N 7/0115 : with details on the detection of a particular field or frame pattern in the incoming video signal, e.g. 3:2 pull-down pattern
• CPC Class Codes: H04N 7/012 : Conversion between an interlaced and a progressive signal for graphics images G09G2310/0229

Abstract

Apparatus and methods of using content information for encoding multimedia data are described. A method of processing multimedia data includes classifying content of multimedia data, and encoding the multimedia data in a first data group and in a second data group based on the content classification, wherein the first data group comprises a coefficient and the second data group comprises a first differential refinement associated with the first data group coefficient. An apparatus for using content information for encoding multimedia data includes a content classifying module configured to classify content of multimedia data and provide content classification data, and an encoder configured to encode the multimedia data in a first data group and in a second data group based on the content classification, wherein the first data group comprises a coefficient and the second data group comprises a first differential refinement associated with the first data group coefficient.

Claims

1. A method of encoding multimedia data, comprising:

classifying content of multimedia data; and

encoding the multimedia data in a first data group and in a second data group based on the content classification, wherein the first data group comprises a coefficient and the second data group comprises a first differential refinement associated with the first data group coefficient.

2. The method of claim 1, wherein said encoding comprises determining a bit rate based on the content classification of the multimedia data, and encoding the multimedia data based on the bit rate.

3. The method of claim 1, wherein classifying content comprises determining complexity of the multimedia data, and wherein the selected multimedia data is encoded based on the complexity of the multimedia data.

4. The method of claim 3, wherein the complexity comprises temporal complexity or spatial complexity.

5. The method of claim 3, wherein the complexity comprises temporal complexity and spatial complexity.

6. The method of claim 1, wherein encoding comprises encoding the multimedia data so as to allow decoding of only the first data group or the first data group and the second data group into a single combined data group.

7. The method of claim 1, wherein the first differential refinement indicates a difference between a selected video frame and frame data resulting from decoding the first data group.

8. The method of claim 1, wherein the first data group is a base layer and the second data group is an enhancement layer.

9. The method of claim 8, further comprising:

selecting the coefficient from one of an original base layer residual error coefficient or an original enhancement layer residual error coefficient; and

calculating the first differential refinement based on the coefficient and the original enhancement layer residual error coefficient.

10. The method of claim 1, wherein encoding further comprises encoding macroblock header information and motion vector information in the first data group.

11. The method of claim 1, wherein encoding further comprises quantizing the first data group at a first step size, and quantizing the second data group at a second step size, wherein the first step size and second step size are related by a scale factor.

12. The method of claim 1, wherein encoding further comprises determining a first quantization parameter having a first quantization step size for use in encoding the first data group, and determining a second quantization parameter having a second quantization step size for use in encoding the second data group, wherein the first and second quantization parameters are determined based on content information of a selected frame data, and wherein said first quantization step size is coarser than said second quantization step size.

13. The method of claim 1, wherein encoding comprises encoding the first data group using I-frames, and P-frames or any combination thereof and encoding the second data group using I-frames, P-frames, and B-frames or any combination thereof.

14. An apparatus for encoding multimedia data, comprising:

means for classifying content of multimedia data;

means for encoding the multimedia data in a first data group and in a second data group based on the content classification, wherein the first data group comprises a coefficient and the second data group comprises a first differential refinement associated with the first data group coefficient.

15. The apparatus of claim 14, wherein said encoding means comprise means for determining a bit rate based on the content classification of the multimedia data, and encoding the multimedia data based on the bit rate.

16. The apparatus of claim 14, wherein said classifying content means comprise means for determining complexity of the multimedia data, and wherein the selected multimedia data is encoded based on the complexity of the multimedia data.

17. The apparatus of claim 16, wherein the complexity comprises temporal complexity or spatial complexity.

18. The apparatus of claim 16, wherein the complexity comprises temporal complexity and spatial complexity.

19. The apparatus of claim 14, wherein said encoding means comprises means for encoding the multimedia data so as to allow decoding of only the first data group or the first data group and the second data group into a single combined data group.

20. The apparatus of claim 14, wherein the first differential refinement indicates a difference between a selected video frame and frame data resulting from decoding the first data group.

21. The apparatus of claim 14, wherein the first data group is a base layer and the second data group is an enhancement layer.

22. The apparatus of claim 14, wherein the encoding means comprises means for encoding macroblock header information and motion vector information in the first data group.

23. The apparatus of claim 14, wherein said encoding means further comprise means for quantizing the first data group at a first step size, and quantizing the second data group at a second step size, wherein the first step size and second step size are related by a scale factor.

24. The apparatus of claim 14, wherein said decoding means comprise means for determining a first quantization parameter having a first quantization step size for use in encoding the first data group, and determining a second quantization parameter having a second quantization step size for use in encoding the second data group, wherein the first and second quantization parameters are determined based on content information of a selected frame data, and wherein said first quantization step size is coarser than said second quantization step size.

25. The apparatus of claim 14, wherein said encoding means comprises:

means for encoding the first data group using I-frames and P-frames; and

means for encoding the second data group using I-frames, P-frames, and B-frames.

26. The apparatus of claim 21, wherein said encoding means comprises:

means for selecting the coefficient from one of an original base layer residual error coefficient or an original enhancement layer residual error coefficient; and

means for calculating the first differential refinement based on the coefficient and the original enhancement layer residual error coefficient.

27. An apparatus configured to encode multimedia data, comprising:

a content classifying module configured to classify content of multimedia data and provide content classification data; and

an encoder configured to encode the multimedia data in a first data group and in a second data group based on the content classification, wherein the first data group comprises a coefficient and the second data group comprises a first differential refinement associated with the first data group coefficient.

28. The apparatus of claim 27, wherein the encoder comprises a bit rate component configured to determine a bit allocation based on the content classification, and wherein the encoding component is further configured to encode the selected multimedia data using the bit allocation.

29. The apparatus of claim 27, wherein classifying content comprises determining complexity of the multimedia data, and wherein the selected multimedia data is encoded based on the complexity of the multimedia data.

30. The apparatus of claim 29, wherein the complexity comprises temporal complexity or spatial complexity.

31. The apparatus of claim 29, wherein the complexity comprises temporal complexity and spatial complexity.

32. The apparatus of claim 27, wherein encoding comprises encoding the multimedia data so as to allow decoding of only the first data group or the first data group and the second data group into a single combined data group.

33. The apparatus of claim 27, wherein the first differential refinement indicates a difference between a selected video frame and frame data resulting from decoding the first data group.

34. The apparatus of claim 27, wherein the first data group is a base layer and the second data group is an enhancement layer.

35. A machine-readable medium comprising instructions that upon execution cause a machine to:

classify content of multimedia data; and

encode the multimedia data in a first data group and in a second data group based on the content classification, wherein the first data group comprises a coefficient and the second data group comprises a first differential refinement associated with the first data group coefficient.

36. The machine-readable medium of claim 35, wherein the instructions to encode comprise instructions to determine a bit allocation based on the content classification, and wherein the encoding component is further configured to encode the selected multimedia data using the bit allocation.

37. The machine-readable medium of claim 35, wherein classifying content comprises determining complexity of the multimedia data, and wherein the selected multimedia data is encoded based on the complexity of the multimedia data.

38. The machine readable instructions of claim 37, wherein the complexity comprises temporal complexity or spatial complexity.

39. The machine-readable instructions of claim 37, wherein the complexity comprises temporal complexity and spatial complexity.

40. A processor being configured to:

classify content of multimedia data; and

encode the multimedia data in a first data group and in a second data group based on the content classification, wherein the first data group comprises a coefficient and the second data group comprises a first differential refinement associated with the first data group coefficient.

41. The processor of claim 40, wherein the processor is further configured to determine a bit allocation based on the content classification, and wherein the encoding component is further configured to encode the selected multimedia data using the bit allocation.

42. The processor of claim 40, wherein the processor is further configured to determine complexity of the multimedia data, and wherein the content classification is based on the complexity of the multimedia data.

43. The processor of claim 42, wherein the complexity comprises temporal complexity or spatial complexity.

44. The processor of claim 42, wherein the complexity comprises temporal complexity and spatial complexity.