Application of category theory and cognitive science to design of semantic descriptions for content data

• US 7,319,951 B2
• Filed: 03/14/2001
• Issued: 01/15/2008
• Est. Priority Date: 03/14/2000
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.), Sony Electronics Inc. (Sony Group Corp.)
• Inventors: Rising, Hawley III, Tabatabai, Ali
• Primary Examiner(s): Dorvil; Richemond
• Assistant Examiner(s): Han; Qi
• Application Number:
  US09/809,684
• Publication Number:
  US 20010047254A1
• Time in Patent Office: 2,498 Days
• Field of Search: 704 1- 10, 707/2, 707/100, 709/245, 715/723, 382/284
• US Class Current: 704/9
• CPC Class Codes: G06F 16/319 Inverted lists
• CPC Class Codes: G06F 16/367 Ontology
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70

Abstract

Instead of focusing on specific, static semantic description schemes, emphasis and focus is placed on determining what is necessary and needed to create any type or kind of semantic description for content data in various applications such as MPEG-7. In particular, numerous semantic description tools are selected after examining the principles of cognitive science and category theory. These semantic description tools provide sufficient flexibility and power to create any type or kind of semantic description. Semantic entity tools and categorical structure tools were identified as necessary and needed to create any type or kind of semantic description. Semantic entity tools are tools that represent entities in a semantic description. Categorical structure tools are tools that represent categorical structures of semantic entities and relations among these categorical structures. The semantic entity tools and the categorical structure tools facilitate creation of a semantic mosaic description for content data from multiple semantic descriptions.

Claims

1. A method of representing a semantic description for content data, comprising the steps of:

a) using a core semantic entity for expressing an underlying meaning of said content data instead of expressing what is in said content data to represent said semantic description, wherein said content data comprises one of picture, graphics, 3D model, audio, video, and any combination thereof, wherein said core semantic entity comprises a semantic object which semantically describes an object and which includes a plurality of attributes associated with said object, wherein said semantic object comprises semantic sub-objects, a semantic state which is a collection of attributes of said semantic object, wherein said semantic state comprises semantic sub-states, a semantic event which is a change in said semantic state, wherein said semantic event comprises semantic sub-events, and a semantic episode which represents semantic events over a period of time, wherein said semantic episode comprises semantic sub-episodes;

b) using a constructed semantic entity having at least one core semantic entity to represent said semantic description, wherein said constructed semantic entity expresses an underlying meaning of said content data instead of expressing what is in said content data, wherein said constructed semantic entity has a plurality of operational properties resembling cognitive operations;

c) using a categorical structure to represent said semantic description, wherein said categorical structure comprises a morphism being a mapping between entities, a graph including a set of morphisms between entities where said morphisms are implemented as edges and a set of entities where said entities are implemented as vertices, a category including a plurality of entities and at least one morphism between said entities, a functor being a mapping between categories and obeys a plurality of categorical constraints, a natural transformation being a mapping between functors and obeys said plurality of categorical constraints, and a relation; and

d) storing said semantic description, wherein said semantic description describes an underlying meaning of said content data instead of describing what is in said content data.

2. A method as recited in claim 1 further comprising the steps of:

using a context semantic entity to represent said semantic description.

3. A method as recited in claim 2 wherein said context semantic entity comprises a frame.

4. A method as recited in claim 1 wherein said constructed semantic entity comprises a mental space.

5. A method as recited in claim 1 wherein said constructed semantic entity comprises a descriptive structure.

6. A method of representing a semantic description for content data, comprising the steps of:

a) using a core semantic entity to represent said semantic description, wherein said content data comprises one of picture, graphics, 3D model, audio, video, and any combination thereof, wherein said core semantic entity comprises a semantic object which semantically describes an object and which includes a plurality of attributes associated with said object, a semantic state which is a collection of attributes of said semantic object, a semantic event which is a change in said semantic state, and a semantic episode which represents semantic events over a period of time;

b) using a categorical structure to represent said semantic description, wherein said categorical structure comprises a morphism being a mapping between entities, a graph including a set of morphisms between entities where said morphisms are implemented as edges and a set of entities where said entities are implemented as vertices, a category including a plurality of entities and at least one morphism between said entities, a functor being a mapping between categories and obeys a plurality of categorical constraints, a natural transformation being a mapping between functors and obeys said plurality of categorical constraints, and a relation; and

c) storing said semantic description.

7. A method as recited in claim 6 further comprising the steps of:

using a constructed semantic entity having at least one core semantic entity to represent said semantic description, wherein said constructed semantic entity has a plurality of operational properties resembling cognitive operations.

8. A method as recited in claim 7 further comprising the steps of:

using a context semantic entity to represent said semantic description.

9. A method as recited in claim 8 wherein said context semantic entity comprises a frame.

10. A method as recited in claim 7 wherein said constructed semantic entity comprises a mental space.

11. A method as recited in claim 7 wherein said constructed semantic entity comprises a descriptive structure.

12. A computer system comprising:

a bus;

a memory device coupled to said bus and having computer-executable instructions; and

a processor coupled to said bus, wherein said processor executes said computer executable instructions to use a core semantic entity for expressing an underlying meaning of content data instead of expressing what is in said content data to represent a semantic description for said content data, wherein said content data comprises one of picture, graphics, 3D model, audio, video, and any combination thereof, wherein said core semantic entity comprises a semantic object which semantically describes an object and which includes a plurality of attributes associated with said object, wherein said semantic object comprises semantic sub-objects, a semantic state which is a collection of attributes of said semantic object, wherein said semantic state comprises semantic sub-states, a semantic event which is a change in said semantic state, wherein said semantic event comprises semantic sub-events, and a semantic episode which represents semantic events over a period of time, wherein said semantic episode comprises semantic sub-episodes, wherein said processor executes said computer executable instructions to use a constructed semantic entity having at least one core semantic entity to represent said semantic description, wherein said constructed semantic entity expresses an underlying meaning of said content data instead of expressing what is in said content data, wherein said constructed semantic entity has a plurality of operational properties resembling cognitive operations, and wherein said processor executes said computer executable instructions to store said semantic description, wherein said semantic description describes an underlying meaning of said content data instead of describing what is in said content data, wherein said processor executes said computer executable instructions to use a categorical structure to represent said semantic description, and wherein said categorical structure comprises a morphism being a mapping between entities, a graph including a set of morphisms between entities where said morphisms are implemented as edges and a set of entities where said entities are implemented as vertices, a category including a plurality of entities and at least one morphism between said entities, a functor being a mapping between categories and obeys a plurality of categorical constraints, a natural transformation being a mapping between functors and obeys said plurality of categorical constraints, and a relation.

13. A computer system as recited in claim 12 wherein said processor executes said computer executable instructions to use a context semantic entity to represent said semantic description.

14. A computer system as recited in claim 13 wherein said context semantic entity comprises a frame.

15. A computer system as recited in claim 12 wherein said constructed semantic entity comprises a mental space.

16. A computer system as recited in claim 12 wherein said constructed semantic entity comprises a descriptive structure.

17. A computer system comprising:

a bus;

a memory device coupled to said bus and having computer-executable instructions; and

a processor coupled to said bus, wherein said processor executes said computer executable instructions to use a core semantic entity to represent a semantic description for content data, wherein said core semantic entity comprises a semantic object which semantically describes an object and which includes a plurality of attributes associated with said object, a semantic state which is a collection of attributes of said semantic object, a semantic event which is a change in said semantic state, and a semantic episode which represents semantic events over a period of time, wherein said processor executes said computer executable instructions to use a categorical structure to represent said semantic description for said content data, wherein said categorical structure comprises a morphism being a mapping between entities, a graph including a set of morphisms between entities where said morphisms are implemented as edges and a set of entities where said entities are implemented as vertices, a category including a plurality of entities and at least one morphism between said entities, a functor being a mapping between categories and obeys a plurality of categorical constraints, a natural transformation being a mapping between functors and obeys said plurality of categorical constraints, and a relation, and wherein said processor executes said computer executable instructions to store said semantic description, wherein said content data comprises one of picture, graphics, 3D model, audio, video, and any combination thereof.

18. A computer system as recited in claim 17 wherein said processor executes said computer executable instructions to use a constructed semantic entity having at least one core semantic entity to represent said semantic description, wherein said constructed semantic entity has a plurality of operational properties resembling cognitive operations.

19. A computer system as recited in claim 18 wherein said processor executes said computer executable instructions to use a context semantic entity to represent said semantic description.

20. A computer system as recited in claim 19 wherein said context semantic entity comprises a frame.

21. A computer system as recited in claim 18 wherein said constructed semantic entity comprises a mental space.

22. A computer-readable medium comprising a data structure representing a semantic description for content data, wherein said data structure stored therein comprises

a core semantic entity for expressing an underlying meaning of said content data instead of expressing what is in said content data to represent said semantic description, wherein said content data comprises one of picture, graphics, 3D model, audio, video, and any combination thereof, wherein said core semantic entity comprises a semantic object which semantically describes an object and which includes a plurality of attributes associated with said object, wherein said semantic object comprises semantic sub-objects, a semantic state which is a collection of attributes of said semantic object, wherein said semantic state comprises semantic sub-states, a semantic event which is a change in said semantic state, wherein said semantic event comprises semantic sub-events, and a semantic episode which represents semantic events over a period of time, wherein said semantic episode comprises semantic sub-episodes;

a constructed semantic entity having at least one core semantic entity to represent said semantic description, wherein said constructed semantic entity expresses an underlying meaning of said content data instead of expressing what is in said content data, wherein said constructed semantic entity has a plurality of operational properties resembling cognitive operations; and

a categorical structure to represent said semantic description, wherein said categorical structure comprises a morphism being a mapping between entities, a graph including a set of morphisms between entities where said morphisms are implemented as edges and a set of entities where said entities are implemented as vertices, a category including a plurality of entities and at least one morphism between said entities, a functor being a mapping between categories and obeys a plurality of categorical constraints, a natural transformation being a mapping between functors and obeys said plurality of categorical constraints, and a relation.

23. A computer-readable medium as recited in claim 22 wherein said constructed semantic entity comprises a mental space.

24. A computer-readable medium as recited in claim 22 wherein said data structure further comprises a context semantic entity to represent said semantic description, wherein said context semantic entity comprises a frame.

25. A method of representing a semantic mosaic description for content data, comprising the steps of:

a) using a plurality of semantic descriptions to represent said semantic mosaic description, wherein each semantic description describes an underlying meaning of respective content data instead of describing what is in said respective content data, wherein said content data comprises one of picture, graphics, 3D model, audio, video, and any combination thereof, wherein each semantic description comprises a semantic object which semantically describes an object and which includes a plurality of attributes associated with said object, wherein said semantic object comprises semantic sub-objects, a semantic state which is a collection of attributes of said semantic object, wherein said semantic state comprises semantic sub-states, a semantic event which is a change in said semantic state, wherein said semantic event comprises semantic sub-events, and a semantic episode which represents semantic events over a period of time, wherein said semantic episode comprises semantic sub-episodes, wherein each semantic description includes a categorical structure comprising a morphism being a mapping between entities, a graph including a set of morphisms between entities where said morphisms are implemented as edges and a set of entities where said entities are implemented as vertices, a category including a plurality of entities and at least one morphism between said entities, a functor being a mapping between categories and obeys a plurality of categorical constraints, a natural transformation being a mapping between functors and obeys said plurality of categorical constraints, and a relation; and

b) storing said semantic mosaic description.

26. A method as recited in claim 25 wherein said semantic mosaic description facilitates navigation through said content data and said plurality of semantic descriptions.

27. A method as recited in claim 25 wherein said semantic descriptions have common semantic elements.

28. A computer system comprising:

a bus;

a memory device coupled to said bus and having computer-executable instructions; and

a processor coupled to said bus, wherein said processor executes said computer executable instructions to use a plurality of semantic descriptions to represent a semantic mosaic description for content data, wherein each semantic description describes an underlying meaning of respective content data instead of describing what is in said respective content data, wherein said content data comprises one of picture, graphics, 3D model, audio, video, and any combination thereof, wherein each semantic description comprises a semantic object which semantically describes an object and which includes a plurality of attributes associated with said object, wherein said semantic object comprises semantic sub-objects, a semantic state which is a collection of attributes of said semantic object, wherein said semantic state comprises semantic sub-states, a semantic event which is a change in said semantic state, wherein said semantic event comprises semantic sub-events, and a semantic episode which represents semantic events over a period of time, wherein said semantic episode comprises semantic sub-episodes, and wherein said processor executes said computer executable instructions to store said semantic mosaic description, wherein each semantic description includes a categorical structure comprising a morphism being a mapping between entities, a graph including a set of morphisms between entities where said morphisms are implemented as edges and a set of entities where said entities are implemented as vertices, a category including a plurality of entities and at least one morphism between said entities, a functor being a mapping between categories and obeys a plurality of categorical constraints, a natural transformation being a mapping between functors and obeys said plurality of categorical constraints, and a relation.

29. A computer system as recited in claim 28 wherein said semantic mosaic description facilitates navigation through said content data and said plurality of semantic descriptions.

30. A computer system as recited in claim 28 wherein said semantic descriptions have common semantic elements.

31. A computer-readable medium comprising a data structure representing a semantic mosaic description for content data, wherein said data structure stored therein comprises

a plurality of semantic descriptions, wherein each semantic description describes an underlying meaning of respective content data instead of describing what is in said respective content data, wherein said content data comprises one of picture, graphics, 3D model, audio, video, and any combination thereof, wherein each semantic description comprises a semantic object which semantically describes an object and which includes a plurality of attributes associated with said object, wherein said semantic object comprises semantic sub-objects, a semantic state which is a collection of attributes of said semantic object, wherein said semantic state comprises semantic sub-states, a semantic event which is a change in said semantic state, wherein said semantic event comprises semantic sub-events, and a semantic episode which represents semantic events over a period of time, wherein said semantic episode comprises semantic sub-episodes, wherein each semantic description includes a categorical structure comprising a morphism being a mapping between entities, a graph including a set of morphisms between entities where said morphisms are implemented as edges and a set of entities where said entities are implemented as vertices, a category including a plurality of entities and at least one morphism between said entities, a functor being a mapping between categories and obeys a plurality of categorical constraints, a natural transformation being a mapping between functors and obeys said plurality of categorical constraints, and a relation.

32. A computer-readable medium as recited in claim 31 wherein said semantic mosaic description facilitates navigation through said content data and said plurality of semantic descriptions.

33. A computer-readable medium as recited in claim 31 wherein said semantic descriptions have common semantic elements.