METHOD AND APPARATUS FOR CLASSIFYING MULTIMEDIA ARTIFACTS USING ONTOLOGY SELECTION AND SEMANTIC CLASSIFICATION

• US 20080168070A1
• Filed: 01/08/2007
• Published: 07/10/2008
• Est. Priority Date: 01/08/2007
• Status: Active Grant

• Current Assignee: SINOEAST CONCEPT LIMITED (Tencent Holdings Limited)
• Original Assignee: International Business Machines Corporation
• Inventors: Tasic, Jelena, Naphade, Milind R., Smith, John Richard
• Granted Patent: US 7,707,162 B2
• Time in Patent Office: Days
• US Class Current: 707/100
• CPC Class Codes: G06F 16/353 into predefined classes
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: Y10S 707/99944 Object-oriented database structure

Abstract

A method and apparatus is provided for automatically classifying a multimedia artifact based on scoring, and selecting the appropriate set of ontologies from among all possible sets of ontologies, preferably using a recursive routing selection technique. The semantic tagging of the multimedia artifact is enhanced by applying only classifiers from the selected ontology, for use in classifying the multimedia artifact, wherein the classifiers are selected based on the context of the multimedia artifact. One embodiment of the invention, directed to a method for classifying a multimedia artifact, uses a specified criteria to select one or more ontologies, wherein the specified criteria indicates the comparative similarity between specified characteristics of the multimedia artifact and each ontology. The method further comprises scoring and selecting one or more classifiers from a plurality of classifiers that respectively correspond to semantic element of the selected ontologies, and evaluating the multimedia artifact using the selected classifiers to determine a classification for the multimedia artifact.

Claims

1. A method for classifying a multimedia artifact comprising the steps of:

using specified criteria to select one or more ontologies, wherein said specified criteria indicates the comparative similarity between specified characteristics of said multimedia artifact and each ontology;

scoring and selecting one or more classifiers from a plurality of classifiers that respectively correspond to semantic elements of said selected ontologies; and

evaluating said multimedia artifact using said selected classifiers to determine a classification for said multimedia artifact.

2. The method of claim 1, wherein:

said criteria is used to select one or more ontologies from a plurality of available ontologies, and said criteria selectively comprises either metadata associated with said multimedia artifact, or a score derived for each ontology that pertains to content information of said multimedia artifact.

3. The method of claim 1, wherein:

said evaluation using said selected classifiers enriches said multimedia artifact with semantic metadata, wherein said semantic metadata is stored in a specified database, or packaged with said multimedia artifact, selectively.

4. The method of claim 2, wherein:

said ontology scores and classifier scores are each derived by analyzing context information related to said multimedia artifact, wherein said context information includes at least one element selected from a group of elements comprising time, place, content provider, person, event, object, scene and content acquisition parameters.

5. The method of claim 1, wherein:

said evaluating step comprises using said selected classifiers to detect the presence or absence of specified semantic elements that characterize said multimedia artifact.

6. The method of claim 5, wherein:

said selected classifiers are used in an automatic classification procedure that uses a method selected from a group of methods that includes at least neural networks, decision trees, Gaussian mixture models, hidden Markov models, and support vector machines.

7. The method of claim 1, wherein:

a recursive router is used with said multimedia artifact, to derive said ontology and classifier scores, and to select said ontologies and classifiers from respective ontology and classifier databases.

8. The method of claim 7, wherein:

said recursive router supplies said multimedia artifact with enriched metadata.

9. The method of claim 1, wherein:

said ontologies are scored and selected based on the number of semantic elements from said ontologies that are found to be relevant to said multimedia artifact.

10. The method of claim 9, wherein:

the relevance of said ontologies is determined by respectively selecting one or more semantic elements from said ontologies, and applying corresponding classifiers to said multimedia artifact.

11. A computer program product in a computer readable medium for classifying a multimedia artifact comprising the steps of:

first instructions for using specified criteria to select one or more ontologies, wherein said specified criteria indicates the comparative similarity between specified characteristics of said multimedia artifact and each ontology;

second instructions for scoring and selecting one or more classifiers from a plurality of classifiers that respectively correspond to semantic elements of said selected ontologies; and

third instructions for evaluating said multimedia artifact using said selected classifiers to determine a classification for said multimedia artifact.

12. The computer program product of claim 11, wherein:

said criteria is used to select one or more ontologies from a plurality of available ontologies, and said criteria selectively comprises either metadata associated with said multimedia artifact, or a score derived for each ontology that pertains to content information of said multimedia artifact.

13. The computer program product of claim 11, wherein:

said evaluation using said selected classifiers enriches said multimedia artifact with semantic metadata, wherein said semantic metadata is stored in a specified database, or packaged with said multimedia artifact, selectively.

14. The computer program product of claim 12, wherein:

said ontology scores and classifier scores are each derived by analyzing context information related to said multimedia artifact, wherein said context information includes at least one element selected from a group of elements comprising time, place, content provider, person, event, object, scene and content acquisition parameters.

15. The computer program product of claim 11, wherein:

said evaluating step comprises using said selected classifiers to detect the presence or absence of specified semantic elements that characterize said multimedia artifact.

16. The computer program product of claim 15, wherein:

said selected classifiers are used in an automatic classification procedure that uses a method selected from a group of methods that includes at least neural networks, decision trees, Gaussian mixture models, hidden Markov models, and support vector machines.

17. The computer program product of claim 11, wherein:

a recursive router is used with said multimedia artifact, to derive said ontology and classifier scores, and to select said ontologies and classifiers from respective ontology and classifier databases.

18. The computer program product of claim 17, wherein:

said recursive router supplies said multimedia artifact with enriched metadata.

19. The computer program product of claim 11, wherein:

said ontologies are scored and selected based on the number of semantic elements from said ontologies that are found to be relevant to said multimedia artifact.

20. The computer program product of claim 19, wherein:

the relevance of said ontologies is determined by respectively selecting one or more semantic elements from said ontologies, and applying corresponding classifiers to said multimedia artifact.

21. An apparatus for classifying a multimedia artifact comprising:

a first processing component for using specified criteria to select one or more ontologies, wherein said specified criteria indicates the comparative similarity between specified characteristics of said multimedia artifact and each ontology;

a second processing component for scoring and selecting one or more classifiers from a plurality of classifiers that respectively correspond to semantic elements of said selected ontologies; and

a third processing component for evaluating said multimedia artifact using said selected classifiers to determine a classification for said multimedia artifact.

22. The apparatus of claim 21, wherein:

said first component uses said criteria to select one or more ontologies from a plurality of available ontologies, and said criteria selectively comprises either metadata associated with said multimedia artifact, or a score derived for each ontology that pertains to content information of said multimedia artifact.

23. The apparatus of claim 21, wherein:

said third component performs said evaluation using said selected classifiers to enrich said multimedia artifact with semantic metadata, wherein said semantic metadata is stored in a specified database, or packaged with said multimedia artifact, selectively.

24. The apparatus of claim 22, wherein:

said ontology scores and classifier scores are each derived by analyzing context information related to said multimedia artifact, wherein said context information includes at least one element selected from a group of elements comprising time, place, content provider, person, event, object, scene and content acquisition parameters.

25. The apparatus of claim 21, wherein:

said third component uses said selected classifiers to detect the presence or absence of specified semantic elements that characterize said multimedia artifact.

26. The apparatus of claim 25, wherein:

said selected classifiers are used in an automatic classification procedure that uses a method selected from a group of methods that includes at least neural networks, decision trees, Gaussian mixture models, hidden Markov models, and support vector machines.

27. The apparatus of claim 21, wherein:

a recursive router is used with said multimedia artifact, to derive said ontology and classifier scores, and to select said ontologies and classifiers from respective ontology and classifier databases.

28. The apparatus of claim 27, wherein:

said recursive router supplies said multimedia artifact with enriched metadata.

29. The apparatus of claim 21, wherein:

said ontologies are scored and selected based on the number of semantic elements from said ontologies that are found to be relevant to said multimedia artifact.

30. The apparatus of claim 29, wherein:

the relevance of said ontologies is determined by respectively selecting one or more semantic elements from said ontologies, and applying corresponding classifiers to said multimedia artifact.