Method and apparatus for transcoding multimedia using content analysis

• US 6,970,602 B1
• Filed: 10/06/1999
• Issued: 11/29/2005
• Est. Priority Date: 10/06/1998
• Status: Expired due to Term

• Current Assignee: SINOEAST CONCEPT LIMITED (Tencent Holdings Limited)
• Original Assignee: International Business Machines Corporation
• Inventors: Smith, John R., Mohan, Rakesh, Li, Chung-Sheng
• Primary Examiner(s): Do, Anh Hong
• Application Number:
  US09/413,515
• Time in Patent Office: 2,246 Days
• Field of Search: 382/232, 382/100, 707/104.1, 707/513, 707/1, 707/10, 709/221, 709/220, 709/231, 370/466, 341/50, 715/716, 700/83
• US Class Current: 382/232
• CPC Class Codes: H04N 19/12 Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 21/234336 by media transcoding, e.g. video is transformed into a slideshow of still pictures or audio is converted into text
• CPC Class Codes: H04N 21/25808 Management of client data terminal profiles in network data switching protocols H04L67/303
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities

Abstract

A method and apparatus for selecting at least one transcoding method for manipulating multimedia data for delivery on the basis of analysis of the content of the multimedia data. Many possible transcoding operations can be performed on multimedia data to adapt it to constraints in delivery and display, processing and storage of client devices. The selection of specific transcoding operations can be made by first analyzing the features, purposes and relevances of the individual multimedia objects within the multimedia documents, then by selecting the transcoding alternatives according to the results of the analysis. Based on the analysis, different transcoding algorithms can be applied to different content, less than all of the content can be transcoded, groups of multimedia objects can be transcoded, etc.

Claims

1. A computerized method for transcoding a multimedia presentation for delivery and display comprising the steps of:

analyzing the content of the multimedia presentation by separating a multimedia document into individual multimedia objects; and

analyzing each multimedia object individually; and

performing transcoding based on said analyzing.

2. The method of claim 1 wherein said performing transcoding comprises the steps of:

selecting at least one transcoding alternative based on the results of said analyzing; and

transcoding the content according to said at least one transcoding alternative.

3. The method of claim 1 wherein said performing transcoding comprises the steps of:

selecting less than all of said content for transcoding based on said analyzing; and

transcoding less than all of said content.

4. The method of claim 1 further comprising the steps of:

separating the multimedia objects into individual modal elements; and

analyzing each modal element of each multimedia object independently.

5. The method of claim 1 further comprising the steps of:

identifying relationships between individual multimedia objects within a multimedia document; and

transcoding related multimedia objects as a group.

6. The method of claim 4 further comprising the steps of:

identifying relationships between individual modal elements of multimedia objects; and

transcoding the related modal elements as a group.

7. The method of claim 1, wherein the multimedia content is a document published on the World-Wide Web.

8. The method of claim 1, wherein the multimedia content comprises visual content.

9. The method of claim 8, wherein the content analysis classifies the visual content into at least one of image type, purpose and semantic classes.

10. The method of claim 9, wherein the content analysis utilizes a decision-tree for classifying images into image type classes.

11. The method of claim 10 wherein the image type classes comprise color photos, color graphics, gray photos, gray graphics, black and white photos, and black and white graphics.

12. The method of claim 11, wherein the content analysis procedure extracts color and texture features from the images.

13. The method of claim 12, wherein image type classification is used to select from different methods for compression, size reduction, color reduction, substitution, and removal.

14. The method of claim 12, wherein image purpose classification is used to select from different methods for compression, size reduction, color reduction, substitution and removal.

15. The method of claim 1, wherein the transcoder adapts the content to the display, processing and storage constraints of the client devices.

16. The method of claim 1, wherein the transcoder adapts the content to the bandwidth and connectivity constraints of the network.

17. The method of claim 15, wherein the client device is a speech browser in an automotive vehicle.

18. The method of claim 15 wherein the client device is a hand-held computer.

19. The method of claim 15 wherein the client device is a smart phone.

20. The method of claim 16, wherein the network connection uses a wireless link to the client device.

21. The method of claim 20, wherein the client and network provides intermittent connectivity between the transcoder and client device.

22. A method as in claim 1, wherein the transcoding operation manipulates the data to generate an alternative version of it.

23. Method as in claim 1, wherein the transcoding operation selects an alternative version of data.

24. The method of claim 1, wherein the content analysis is performed off-line and the results are stored embedded in or along with the multimedia content.

25. A system for providing transcoding of the content of a multimedia presentation comprising:

a content analysis component for analyzing the content of the multimedia presentation by separating a multimedia document into individual multimedia objects; and

analyzing each multimedia object individually; and

at least one transcoding component for performing transcoding of the content based on analyzing output provided by said content analysis component.

26. The system of claim 25 further comprising a content selection component connected to received input from the content analysis component and to select at least one transcoding option based on the input;

and to instruct said at least one transcoding component to perform the at least one transcoding option.

27. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps for transcoding a multimedia presentation for delivery and display, said method comprising the steps of:

analyzing the content of the multimedia presentation by separating a multimedia document into individual multimedia objects; and

analyzing each multimedia object individually; and

performing transcoding based on said analyzing.

28. The program storage device of claim 27 wherein said performing transcoding comprises the steps of:

selecting at least one transcoding alternative based on the results of said analyzing; and

transcoding the content according to said at least one transcoding alternative.