MULTIMEDIA ENCODING METHOD AND DEVICE BASED ON MULTIMEDIA CONTENT CHARACTERISTICS, AND A MULTIMEDIA DECODING METHOD AND DEVICE BASED ON MULTIMEDIA

• US 20110047155A1
• Filed: 04/16/2009
• Published: 02/24/2011
• Est. Priority Date: 04/17/2008
• Status: Abandoned Application

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Jung, Hae-Kyung, Sohn, Yu-Mi, Lee, Young-Yoon, Moon, Han-Gil
• Application Number:
  US12/988,426
• Publication Number:
  US 20110047155A1
• Time in Patent Office: Days
• US Class Current: 707/736
• CPC Class Codes: G10H 2220/086 Beats per minute [BPM] indicator, i.e. displaying a tempo value, e.g. in words or as numerical value in beats per minute
• CPC Class Codes: G10H 2240/131 Library retrieval, i.e. searching a database or selecting a specific musical piece, segment, pattern, rule or parameter set
• CPC Class Codes: G10L 19/022 Blocking, i.e. grouping of samples in time; Choice of analysis windows; Overlap factoring
• CPC Class Codes: G10L 19/167 Audio streaming, i.e. formatting and decoding of an encoded audio signal representation into a data stream for transmission or storage purposes
• CPC Class Codes: H04N 19/109 among a plurality of temporal predictive coding modes
• CPC Class Codes: H04N 19/11 among a plurality of spatial predictive coding modes
• CPC Class Codes: H04N 19/134 characterised by the element, parameter or criterion affecting or controlling the adaptive coding
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

Metadata includes information for effectively presenting content, and the information included in the metadata includes some information useful for encoding or decoding of multimedia data. Thus, although syntax information of the metadata is provided for an information search, an increase of encoding or decoding efficiency of data can be achieved by using strong connection between the syntax information and the data.

Claims

1. A method of encoding multimedia data based on attributes of multimedia content, the method comprising:

receiving the multimedia data;

detecting attribute information of the multimedia data based on the attributes of the multimedia content; and

determining an encoding scheme of encoding the multimedia data based on the detected attribute information.

2. The method of claim 1, further comprising:

encoding the multimedia data according to the encoding scheme; and

generating a bitstream comprising the encoded multimedia data.

3. The method of claim 2, further comprising encoding the attribute information of the multimedia data as a descriptor for management or search of the multimedia data,

wherein the generating of the bitstream comprises generating a bitstream comprising the encoded multimedia data and the descriptor.

4. The method of claim 1, wherein the predetermined attributes comprise at least one of color attributes of image data, texture attributes of image data, and speed attributes of sound data, and

wherein the detecting of the attribute information comprises detecting at least one of the color attributes of image data, the texture attributes of image data, and the speed attributes of sound data.

5. The method of claim 4, wherein the color attributes of image data comprises at least one of a color layout of an image and an accumulated distribution per color bin.

6. The method of claim 4, wherein the determining the encoding scheme comprises measuring a variation between a pixel value of current image data and a pixel value of reference image data by using the color attributes of the image data.

7. The method of claim 6, wherein the determining the encoding scheme further comprises compensating for the pixel value of the current image data by using the variation between the pixel value of the current image data and the pixel value of the reference image data.

8. The method of claim 7, further comprising compensating for the variation of the pixel values for the current image data for which motion compensation has been performed and encoding the current image data.

9. The method of claim 4, wherein the texture attributes of the image data comprises at least one of homogeneity, smoothness, regularity, edge orientation, and coarseness of image texture.

10. The method of claim 9, wherein the determining of the encoding scheme comprises determining a size of a data processing unit for motion estimation of current image data by using the texture attributes of the image data.

11. The method of claim 10, wherein the determining the encoding scheme comprises determining the size of the data processing unit based on at least one of the homogeneity, the smoothness, and the regularity of the texture attributes of the image data so that a texture change of the current image decreases as the size of the data processing unit increases.

12. The method of claim 10, further comprising performing motion estimation or motion compensation for the current image data by using the data processing unit of which the size is determined for the image data.

13. The method of claim 9, wherein the determining the encoding scheme comprises determining a predictable intra prediction mode for the current image data by using the texture attributes of the image data.

14. The method of claim 13, wherein the determining the encoding scheme comprises determining a type and a priority of a predictable intra prediction mode for the current image data based on the edge orientation of the texture attributes of the image data.

15. The method of claim 13, further comprising performing motion estimation for the current image data by using the intra prediction mode determined for the current image data.

16. The method of claim 4, wherein the determining the encoding scheme comprises determining a length of a data processing unit for frequency transform of current sound data by using the speed attributes of the sound data.

17. The method of claim 16, wherein the determining the encoding scheme comprises determining the length of the data processing unit to decrease as the current sound data increases, based on the tempo information of the speed attributes of the sound data.

18. The method of claim 17, further comprising performing frequency transform for the current sound data by using the data processing unit of which the length is determined for the sound data.

19. The method of claim 4, further comprising:

encoding at least one of metadata regarding a color layout, metadata regarding a color structure, and metadata regarding a scalable color as a descriptor for management or search of the multimedia based on the multimedia content if the predetermined attributes of the multimedia content are the color attributes of the image data;

encoding at least one of metadata regarding an edge histogram, metadata for texture browsing, and metadata regarding homogeneity of texture as the descriptor for management or search of the multimedia based on the multimedia content if the predetermined attributes of the multimedia content are the texture attributes of the image data; and

encoding at least one of metadata regarding audio tempo, semantic description information, and side information as the descriptor for management or search of the multimedia based on the multimedia content if the predetermined attributes of the multimedia content are the speed attributes of the sound data.

45. A computer readable recording medium storing a computer readable program for executing the method of claim 1.

20. A method of decoding multimedia data based on attributes of multimedia content, the method comprising:

receiving a bitstream of encoded multimedia data;

parsing the received bitstream;

classifying encoded data of the multimedia data and information regarding the multimedia data based on the parsed bitstream;

extracting attribute information for management or search of the multimedia data from the information regarding the multimedia; and

determining a decoding scheme of decoding the multimedia data based on the extracted.

21. The method of claim 20, further comprising:

decoding the encoded data of the multimedia according to the decoding scheme; and

restoring the decoded multimedia data as the multimedia data.

22. The method of claim 20, wherein the extracting the attribute information comprises:

extracting a descriptor for management or search of the multimedia based on the multimedia content; and

extracting the attribute information from the descriptor.

23. The method of claim 20, wherein the predetermined attributes comprise at least one of color attributes of image data, texture attributes of image data, and speed attributes of sound data, and

wherein the extracting of the attribute information comprises extracting at least one of the color attributes of image data, the texture attributes of image data, and the speed attributes of sound data.

24. The method of claim 23, wherein the color attributes of image data comprises at least one of a color layout of an image and an accumulated distribution per color bin.

25. The method of claim 23, wherein the determining of the decoding scheme comprises measuring a variation between a pixel value of current image data and a pixel value of reference image data by using the color attributes of the image data.

26. The method of claim 25, further comprising:

performing motion compensation of inverse-frequency-transformed current image data; and

compensating for the pixel value of the current image data for which the motion compensation has been performed by using the variation between the pixel value of the current image data and the pixel value of the reference image data.

27. The method of claim 23, wherein the texture attributes of the image data comprises at least one of homogeneity, smoothness, regularity, edge orientation, and coarseness of image texture.

28. The method of claim 27, wherein the determining of the decoding scheme comprises determining a size of a data processing unit for motion estimation of current image data by using the texture attributes of the image data.

29. The method of claim 28, wherein the determining the decoding scheme comprises determining the size of the data processing unit based on at least one of the homogeneity, the smoothness, and the regularity of the texture attributes of the image data so that a texture change of the current image decreases as the more the size of the data processing unit increases.

30. The method of claim 28, further comprising performing motion estimation or motion compensation for the current image data by using the data processing unit of which the size is determined for the image data.

31. The method of claim 23, wherein the determining the decoding scheme comprises determining a predictable intra prediction mode for the current image data by using the texture attributes of the image data.

32. The method of claim 31, wherein the determining the decoding scheme comprises determining a type and a priority of a predictable intra prediction mode for the current image data based on edge orientation of the texture attributes of the image data.

33. The method of claim 31, further comprising performing motion estimation for the current image data by using the intra prediction mode determined for the current image data.

34. The method of claim 22, wherein speed attributes of sound data comprises tempo information of sound.

35. The method of claim 22, wherein the determining of the decoding scheme comprises determining a length of a data processing unit for inverse frequency transform of current sound data by using the speed attributes of the sound data.

36. The method of claim 35, wherein the determining the decoding scheme comprises determining the length of the data processing unit to decrease as the current sound data increases, based on the tempo information of the speed attributes of the sound data.

37. The method of claim 35, further comprising performing inverse frequency transform for the current sound data by using the data processing unit of which the length is determined for the sound data.

38. The method of claim 31, wherein the extracting of the attribute information comprises:

extracting at least one of metadata regarding a color layout, metadata regarding a color structure, metadata regarding a scalable color, metadata regarding an edge histogram, metadata for texture browsing, metadata regarding homogeneity of texture, metadata regarding audio tempo, semantic description information, and side information from the descriptor by parsing the bitstream; and

if the extracted descriptor is at least one of the metadata regarding a color layout, the metadata regarding a color structure, and the metadata regarding a scalable color, extracting the color attributes of the image data from the extracted descriptor,

if the extracted descriptor is at least one of the metadata regarding an edge histogram, the metadata for texture browsing, and the metadata regarding homogeneity of texture, extracting the texture attributes of the image data from the extracted descriptor, and

if the extracted descriptor is at least one of the metadata regarding audio tempo, the semantic description information, and the side information, extracting the speed attributes of the sound data from the extracted descriptor.

46. A computer readable recording medium storing a computer readable program for executing the method of claim 20.

39. An apparatus that encodes multimedia data based on attributes of multimedia content, the apparatus comprising:

an input unit that receives the multimedia data;

an attribute information detector that detects attribute information of the multimedia data based on the attributes of the multimedia content;

an encoding scheme determiner that determines an encoding scheme of encoding the multimedia data based on the detected attribute information; and

a multimedia data encoder that encodes the multimedia data according to the encoding scheme.

40. The apparatus of claim 39, further comprising a descriptor encoder that encodes the attribute information for management or search of the multimedia into a descriptor.

41. The apparatus of claim 40, wherein the attribute information includes at least one of color attributes of image data, texture attributes of image data, and speed attributes of sound data, and

the descriptor includes at least one of metadata regarding a color layout, metadata regarding a color structure, metadata regarding a scalable color, metadata regarding an edge histogram, metadata for texture browsing, and metadata regarding homogeneity of texture of the image data and the speed attributes of sound data.

42. An apparatus for decoding multimedia data based on attributes of multimedia content, the apparatus comprising:

a receiver that receives a bitstream of encoded multimedia data, parses the received bitstream, and classifies encoded multimedia data and information regarding the multimedia based on the parsed bitstream;

an attribute information extractor that extracts attribute information for management or search of the multimedia data from the information regarding the multimedia;

a decoding scheme determiner that determines a decoding scheme of decoding the multimedia data based on the extracted attribute information; and

a multimedia data decoder that decodes the encoded multimedia data according to the decoding scheme.

43. The apparatus of claim 42, further comprising a restorer that restores the decoded multimedia data as the multimedia data.

44. The apparatus of claim 42, wherein the attribute information extractor extracts a descriptor for management or search of the multimedia by parsing the bitstream and extracts the attribute information from the descriptor,

the attribute information includes at least one of color attributes of image data, texture attributes of image data, and speed attributes of sound data, and

the descriptor includes at least one of metadata regarding a color layout, metadata regarding a color structure, metadata regarding a scalable color, metadata regarding an edge histogram, metadata for texture browsing, and metadata regarding homogeneity of texture of the image data and the speed attributes of sound data.