Video transcoding using syntactic and semantic clues

• US 6,574,279 B1
• Filed: 04/11/2000
• Issued: 06/03/2003
• Est. Priority Date: 02/02/2000
• Status: Expired due to Term

• Current Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: Vetro, Anthony, Sun, Huifang, Divakaran, Ajay
• Primary Examiner(s): Rao, Andy
• Application Number:
  US09/547,159
• Time in Patent Office: 1,148 Days
• Field of Search: 375/240-1-2
• US Class Current: 375/240.23
• CPC Class Codes: G06T 9/001 Model-based coding, e.g. wire frame
• CPC Class Codes: H04L 47/10 Flow control; Congestion control
• CPC Class Codes: H04L 65/1101 Session protocols
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04L 65/765 intermediate
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04L 69/04 Protocols for data compression, e.g. ROHC
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/147 according to rate distortion criteria rate-distortion as a criterion for motion estimation H04N19/567
• CPC Class Codes: H04N 19/152 by measuring the fullness of the transmission buffer
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/19 using optimisation based on Lagrange multipliers
• CPC Class Codes: H04N 19/20 using video object coding
• CPC Class Codes: H04N 19/25 with scene description coding, e.g. binary format for scenes [BIFS] compression
• CPC Class Codes: H04N 19/29 involving scalability at the object level, e.g. video object layer [VOL]
• CPC Class Codes: H04N 19/33 in the spatial domain
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/436 using parallelised computational arrangements
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/2343 : involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/234318 : by decomposing into objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/23439 : for generating different versions
• CPC Class Codes: H04N 21/2662 : Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/84 : Generation or processing of descriptive data, e.g. content descriptors systems specially adapted for using meta-information in broadcast systems H04H60/73
• CPC Class Codes: H04N 21/8549 : Creating video summaries, e.g. movie trailer retrieval in video databases by using presentations in form of a video summary G06F16/739

Abstract

A method for transcoding a compressed video partitions the compressed video into hierarchical levels, and extracts features from each of the hierarchical levels. One of a number of conversion modes of a transcoder is selected dependent on the features extracted from the hierarchical levels. The compressed video is then transcoded according to the selected conversion mode.

Claims

1. A method for transcoding a compressed video, comprising the steps of:

partitioning the compressed video into a plurality of hierarchical levels;

extracting features from each of the plurality of hierarchical levels;

selecting one of a plurality of conversion modes of a transcoder dependent on the features extracted from each of the plurality of hierarchical levels; and

transcoding the compressed video according to the selected conversion mode.

2. The method of claim 1 wherein the plurality of hierarchical levels include a program level, a shot level, a frame level, and sub-region level.

3. The method of claim 1 wherein the frame level includes video object planes.

4. The method of claim 1 wherein the features depend on semantics and syntactics of the compressed video.

5. The method of claim 1 wherein the compressed video includes low-level meta-data, and further comprising the step of:

mapping the low-level meta-data to rate-quality characteristics depending on content of the video.

6. The method of claim 5 wherein the rate-quality characteristics are partitioned into a plurality of classes.

7. The method of claim 6 including a first class where spatial quality is less than temporal quality, a second class where spatial quality is greater than temporal quality, and a third class where spatial quality is equal to temporal quality.

8. The method of claim 7 wherein content of the first class is transcoded with a discrete-summary transcoder, content of the second class is transcoded with a continuous-conversion transcoder at a relatively low frame rate, and content of the third class is transcoded with the continuous-conversion transcoder at a relatively high frame rate.

9. The method of claim 8 wherein the relatively low frame rate is less than ten frames per second, and the relatively high frame rate is greater than twenty frames per second.

10. The method of claim 6 including a first class where content is easy to compress, a second class where content is moderately difficult to compress, and a third class where content is difficult to compress.

11. The method of claim 10 wherein content of the first class is transcoded with a continuous-conversion transcoder at a relatively high frame rate, content of the second class is transcoded with a continuous-conversion transcoder at a moderate frame rate, and content of the third class is transcoded with the continuous-conversion transcoder at a relatively low high frame rate.

12. The method of claim 11 where the relatively high frame is greater than twenty frames per second, the moderate frame rate is between twenty and ten frames per second, the relatively low fame rate is less than ten frames per second.

13. The method of claim 5 wherein the low-level meta-data is based on a priority of content of the video.

14. The method of claim 5 wherein the low-level meta-data is based on a shape parameters of content of the video.

15. The method of claim 5 wherein the low-level meta-data is based on a texture of content of the video.

16. The method of claim 5 wherein the low-level meta-data is based on motion activity of content of the video.

17. The method of claim 5 wherein the low-level meta-data is based on a DCT coefficients of content of the video.

18. The method of claim 5 wherein high-level meta-data are encoded from low-level meta-data.

19. The method of claim 6 wherein the plurality of classes are ranked on a numeric scale.

20. The method of claim 1 wherein the transcoding produces a CBR bit stream.

21. The method of claim 1 wherein the transcoding produces a VBR bit stream.

22. The method of claim 1 further comprising steps of:

generating a descriptor scheme from the low-level meta-data; and

generating a variation of the compressed input bitstream according to the descriptor scheme.

23. The method of claim 1 wherein the transcoding is based on features extracted from a selected one of the hierarchical levels.

24. The method claim 23 wherein the features depend on spatial importance.

25. The method claim 23 wherein the features depend on temporal importance.

26. The method claim 23 wherein the features depend on compression difficulty.