METHOD AND APPARATUS FOR HYBRID TAGGING AND BROWSING ANNOTATION FOR MULTIMEDIA CONTENT

• US 20110173141A1
• Filed: 02/04/2008
• Published: 07/14/2011
• Est. Priority Date: 02/04/2008
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Campbell, Murray S., Natsev, Apostol I., Yan, Rong
• Granted Patent: US 8,229,865 B2
• Time in Patent Office: Days
• US Class Current: 706/12
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually

Abstract

A computer program product and embodiments of systems are provided for annotating multimedia documents. The computer program product and embodiments of the systems provide for performing manual and automatic annotation.

Claims

1. A computer program product comprising machine executable instructions stored on machine readable media, the product for at least one of tagging and browsing multimedia content, the instructions comprising instructions for:

providing a tagging annotation interface adapted for allowing at least one user to manually associate at least one keyword with at least one multimedia document;

providing a browsing annotation interface adapted for allowing the user to judge a relevance of at least one keyword and at least one automatically associated multimedia document;

providing an annotation candidate selection component that is adapted for automatically associating at least one annotation keyword and at least one multimedia document, and manually associating the at least one selected annotation keyword with the at least one multimedia document; and

a selection tool for permitting the user to select at least one of the tagging annotation interface and the browsing annotation interface.

2. The computer program product as in claim 1, wherein the multimedia document comprises at least one modality comprised of audio, visual, text and speech.

3. The computer program product as in claim 1, further comprising instructions for providing output to at least one of a monitor, a cell phone, and a personal digital assistant (PDA).

4. The computer program product as in claim 1, further comprising instructions for receiving input from at least one of a keyboard, a mouse, a touch screen, a stylus, a microphone, and a speech interface.

5. The computer program product as in claim 1, wherein the at least one keyword has a relationship to at least one of:

a visual topic comprising information derived from a setting that is at least one of nature, sky, urban, studio;

an event topic comprising information derived from an event involving at least one of sports, news and entertainment; and

a genre topic comprising information derived from at least one of a cartoon and a drama.

6. The computer program product as in claim 1, wherein the selection tool selects an interface according to at least one of a learning model and an input from the user.

7. The computer program product as in claim 6, wherein the learning model learns from multi-modal information comprising at least one of visual, audio and text information.

8. The computer program product as in claim 6, wherein the learning model is related to at least one of word frequency, average annotation times per word and image visual similarity.

9. The computer program product as in claim 6, wherein the learning model is based on at least one of a decision tree, a k-nearest neighbor classification, a support vector machine and a Gaussian mixture model.

10. The computer program product as in claim 6, wherein the learning model is updated based on user annotation results.

11. The computer program product as in claim 6, wherein the learning model is adapted for minimizing time required for the user to associate the at least one keyword with the multimedia document.

12. The computer program product as in claim 1, wherein the annotation candidate selection component is adapted for terminating a process of automatically associating the at least one keyword and the at least one multimedia document.

13. The computer program product as in claim 1, wherein the automatically associating and the manually associating provides for at least one of annotating training data for automatic learning;

annotating at least one of image and video collections for at least one of search and retrieval;

amusement;

personal media management and archival of the at least one multimedia document.

14. A system for annotating multimedia documents, the system comprising:

a processing system for implementing machine executable instructions stored on machine readable media; and

a computer program product comprising machine executable instructions stored on machine readable media coupled to the processing system, the product for at least one of tagging and browsing multimedia content, the instructions comprising instructions for;

providing a tagging annotation interface adapted for allowing at least one user to manually associate at least one keyword with at least one multimedia document;

providing a browsing annotation interface adapted for allowing the user to judge a relevance of at least one keyword and at least one automatically associated multimedia document;

providing an annotation candidate selection component that is adapted for automatically associating at least one annotation keyword and at least one multimedia document, and manually associating the at least one selected annotation keyword with the at least one multimedia document; and

a selection tool for permitting the user to select at least one of the tagging annotation interface and the browsing annotation interface.

15. The system as in claim 14, further comprising at least one adapter for communication with a network.

16. A system for annotating multimedia documents, the system comprising:

at least one input device and at least one output device, the input device and the output device adapted for interacting with machine executable instructions for annotating the multimedia documents through an interface;

the interface communicating the interaction to a processing system comprising a computer program product comprising machine executable instructions stored on machine readable media, the product for at least one of tagging and browsing multimedia content, the instructions comprising instructions for;

providing a tagging annotation interface adapted for allowing at least one user to manually associate at least one keyword with at least one multimedia document;

providing a browsing annotation interface adapted for allowing the user to judge a relevance of at least one keyword and at least one automatically associated multimedia document;

providing an annotation candidate selection component that is adapted for automatically associating at least one annotation keyword and at least one multimedia document, and manually associating the at least one selected annotation keyword with the at least one multimedia document; and

a selection tool for permitting the user to select at least one of the tagging annotation interface and the browsing annotation interface.