Method and system for optimal video transcoding based on utility function descriptors

• US 20090290635A1
• Filed: 10/14/2004
• Published: 11/26/2009
• Est. Priority Date: 04/26/2002
• Status: Active Grant

• Current Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Original Assignee: Trustees Of Columbia University In The City Of New York (Columbia University)
• Inventors: Wang, Yong, Kang, Kyeongok, Chang, Shih-Fu, Kim, Jae-Gon, Kim, Jinwoong
• Granted Patent: US 8,218,617 B2
• Time in Patent Office: Days
• US Class Current: 375/240.02
• CPC Class Codes: H04N 19/115 Selection of the code volume for a coding unit prior to coding
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/147 according to rate distortion criteria rate-distortion as a criterion for motion estimation H04N19/567
• CPC Class Codes: H04N 19/159 Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
• CPC Class Codes: H04N 19/164 Feedback from the receiver or from the transmission channel
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/18 the unit being a set of transform coefficients
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/587 involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

Techniques for generating utility-based descriptors from compressed multimedia information are disclosed. A preferred method includes the steps of receiving least a segment of compressed multimedia information, determining two or more portions of utility based descriptor information based on one or more adaptation operations, each corresponding to a unique target rate, adapting the compressed multimedia segment by each the portions of utility based descriptor information to generate adapted multimedia segments, using a quality management method to generate measurement for each adapted multimedia segment, and generating a utility based descriptors based on the portions of utility based descriptor information and corresponding quality measurements.

Claims

1. A method for generating a utility-based descriptor from compressed multimedia information, comprising the steps of:

(a) receiving at least a segment of said compressed multimedia information;

(b) determining two or more portions of utility based descriptor information based on one or more adaptation operations, each of which corresponds to one or more resource constraints for a particular one of said one or more adaptation operations;

(c) adapting said received compressed multimedia segment by each of said two or more portions of said utility based descriptor information to generate two or more adapted multimedia segments, each corresponding to a different one of said two or more portions;

(d) using a quality measurement method to generate a quality measurement for each of said two or more adapted multimedia segments; and

(e) generating a utility based descriptor based on said two or more portions of utility based descriptor information and said corresponding quality measurements.

2. The method of claim 1, wherein said compressed multimedia information comprises MPEG-4 data.

3. The method of claim 1, wherein said compressed multimedia information comprises MPEG-2 data.

4. The method of claim 1, wherein said compressed multimedia information comprises MPEG-1 data.

5. The method of claim 1, wherein said resource constraints comprise target rate information.

6. The method of claim 1, wherein said determining step comprises determining from five to one hundred portions of utility based descriptor information based on one or more adaptation operations.

7. The method of claim 6, wherein said portions of utility-based descriptor information comprise uniformly sampled data.

8. The method of claim 6, wherein said portions of utility-based descriptor information comprise non-uniformly sampled data.

9. The method of claim 6, wherein said determining step comprises determining ten or more portions of utility based descriptor information based on two or more adaptation operations.

10. The method of claim 1, wherein said determining step comprises determining two or more portions of utility based descriptor information based on two or more adaptation operations.

11. The method of claim 1, wherein said one or more adaptation operations includes dropping frames.

12. The method of claim 11, wherein said frames comprise first B frames in each sub group of pictures of said received segment.

13. The method of claim 11, wherein said frames comprise all B frames in each group of pictures of said received segment.

14. The method of claim 11, wherein said one or more adaptation operations further includes DCT coefficient dropping.

15. The method of claim 1, wherein said one or more adaptation operations further includes truncation of an MPEG-4 Fine Granularity Scalability stream.

16. The method of claim 15, wherein said one or more adaptation operations further includes specifying a number of bitplanes of MPEG-4 FSG frames to be truncated.

17. The method of claim 15, wherein said one or more adaptation operations further includes specifying a number of bitplanes of FGST frames to be truncated.

18. The method of claim 15, wherein said one or more adaptation operations further includes specifying a number of bitplanes of FSG frames and FGST frames to be truncated.

19. The method of claim 1, further comprising the step of selecting said quality measurement method from two or more available measurement methods prior to step (c).

20. The method of claim 1, wherein said compressed multimedia information comprises stored video information.

21. The method of claim 1, wherein said compressed multimedia information comprises live video information.

22. The method of claim 1, further comprising the step of ranking said quality measurements prior to step (e).

23. A method for delivering compressed multimedia information to one or more users, each having one or more resource constraints, comprising the steps of:

(a) receiving at least a segment of said compressed multimedia information and a corresponding utility based descriptor;

(b) parsing said utility based descriptor into two or more portions of descriptor information, each portion corresponding to a different one or more resource constraints;

(c) for each of said one or more users, selecting one of said two or more utility based descriptor information portions, if any, which corresponds to said resource constraints for said user; and

(d) for each of said one or more users, adapting said received compressed multimedia segment by said selected utility based descriptor information portion.

24. The method of claim 23, wherein said compressed multimedia information comprises MPEG-4 data.

25. The method of claim 23, wherein said compressed multimedia information comprises MPEG-2 data.

26. The method of claim 23, wherein said compressed multimedia information comprises MPEG-1 data.

27. The method of claim 23, wherein said resource constraints comprise target rate information.

28. The method of claim 23, further comprising the step of interpolating, for each of said one or more users for which one of said two or more utility based descriptor information portions were not selected in step (c), between two consecutive utility based descriptor information portions from said two or more utility based descriptor information portions to generate an interpolated utility based descriptor which approximately corresponds to said resource constraints for said user.

29. The method of claim 23, wherein said parsing step comprises parsing said received utility based descriptor into five to one hundred portions, each portion corresponding to a unique set of one or more resource constraints.

30. The method of claim 23, wherein said portions of utility based descriptor information are uniformly sampled.

31. The method of claim 23, wherein said portions of utility based descriptor information are non-uniformly sampled.

32. The method of claim 23, wherein said adaptation step includes dropping frames.

33. The method of claim 32, wherein said frames comprise first B frames in each sub group of pictures of said received segment.

34. The method of claim 32, wherein said frames comprise all B frames in each group of pictures of said received segment.

35. The method of claim 32, wherein said frames include a P frame at an end of each group of pictures of said received segment.

36. The method of claim 23, wherein said adaptation step further includes DCT coefficient dropping.

37. The method of claim 23, wherein said adaptation step further includes truncation of an MPEG-4 Fine Granularity Scalability stream.

38. The method of claim 23, wherein said resource constraints comprise target rate information, and further comprising the step of receiving user preference information from at least one of said one or more users, and wherein said adaptation step further includes using said user preference information for said corresponding user.

39. The method of claim 23, wherein said resource constraints comprise target rate information, and further comprising the step of receiving target bit rate feedback from a network corresponding to at least one of said one or more users, and wherein said adaptation step further includes using said feedback for said corresponding user.

40. The method of claim 23, and wherein said adaptation step further includes using a predetermined network bandwidth parameter for adapting said received compressed multimedia segment.

41. The method of claim 23, wherein said compressed multimedia information comprises stored video information.

42. The method of claim 23, wherein said compressed multimedia information comprises live video information.

43. A system for delivering compressed multimedia information to one or more user terminals, each having different target bit rates, comprising:

(a) a content delivery computer, adapted to;

(i) receive at least a segment of said compressed multimedia information and corresponding utility based descriptor;

(ii) parse said received utility based descriptor into two or more portions of descriptor information, each portion corresponding to a unique target bit rate;

(iii) for each of said one or more users, select one of said two or more utility based descriptor information portions, if any, which corresponds to said unique target bit rate for said user;

(iv) for each of said one or more users, adapt said received compressed multimedia segment by a corresponding selected utility based descriptor information portion; and

(v) for each of said one or more users, output said adapted corresponding compressed multimedia segment

(b) a network, coupled to said content delivery computer and receiving each of said output adapted compressed multimedia segments, to transmit said output adapted compressed multimedia segments to said one or more user terminals; and

(c) one or more user terminals, each coupled to said network and adapted to receive a corresponding adapted compressed multimedia segment.

44. The system of claim 43, wherein said content delivery computer is further adapted to interpolate, for each of said one or more users for which one of said two or more utility based descriptor information portions cannot be selected, between two consecutive utility based descriptor information portions from said two or more utility based descriptor information portions to generate an interpolated utility based descriptor which approximately corresponds to said resource constraints for said user.

45. A utility-based descriptor for specifying the delivery of compressed multimedia information to one or more user terminals, each subject to one or more resource constraints, comprising:

(a) one or more adaptation descriptors, each of which describes a utility function associated with an adaptation method; and

(b) one or more utility function descriptors, each of which represents a set of adaptation operators indicative of adaptation for the delivery of compressed multimedia information to one or more users subject to a corresponding one or more resource constraints.

46. The descriptor of claim 45, wherein said resource constraints comprise target rate information.

47. The descriptor of claim 45, further comprising one or more resource descriptors defining at least one corresponding constrained resource associated with said one or more utility function descriptors.

48. The descriptor of claim 45, further comprising one or more utility descriptors defining at least one corresponding constrained utility associated with said one or more utility function descriptors.