Recording medium with video index information recorded therein video information management method which uses the video index information, recording medium with audio index information recorded therein, audio information management method which uses the audio index information, video retrieval method which uses video index information, audio retrieval method which uses the audio index information and a video retrieval system

• US 20040184774A1
• Filed: 01/30/2004
• Published: 09/23/2004
• Est. Priority Date: 09/03/1998
• Status: Active Grant

• Current Assignee: Masajiro Iwasaki, Takayuki Kunieda, Yuki Wakita
• Original Assignee: Masajiro Iwasaki, Takayuki Kunieda, Yuki Wakita
• Inventors: Iwasaki, Masajiro, Kunieda, Takayuki, Wakita, Yuki
• Granted Patent: US 7,277,621 B2
• Time in Patent Office: Days
• US Class Current: 386/69
• CPC Class Codes: G06F 16/71 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/7343 Query language or query format
• CPC Class Codes: G06F 16/78 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G11B 2220/20 Disc-shaped record carriers
• CPC Class Codes: G11B 2220/2545 CDs
• CPC Class Codes: G11B 2220/41 Flat as opposed to hierarchical combination, e.g. library of tapes or discs, CD changer, or groups of record carriers that together store one title
• CPC Class Codes: G11B 27/105 of operating discs
• CPC Class Codes: G11B 27/11 by using information not detectable on the record carrier
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence

Abstract

The video index information has a tree structure comprising frame information, sound information, segment information, and package information each as a structure element object. The segment information manages package information for managing a plurality of other segment information as a group, and in the tree structure, the package information is allocated in addition to the frame information and sound information under one segment information, and the video information is managed by using the tree structure of the video index information and structure element objects therein.

Claims

1. A computer-readable recording medium with video index information prepared by analyzing the video information recorded therein,

the video index information having a data structure wherein the video index information has a tree structure comprising, as structure element objects, at least one frame information for managing a given number of successive frames in the video information as one unit of information, at least one sound information for managing voices corresponding to a given range in the video information as one unit of information, at least one segment information for classifying the frame information and sound information each as one unit of information and managing the information as a desired significant collection of data, and package information for collectively managing a plurality of segment information as a group;

further the segment information manages the package information for collectively managing a plurality of other segment information generated by using the same frame as that of the frame information and sound information managed by the segment information and voices in the frame; and

the package information is arranged, in addition to the frame information and sound information, under one segment information within said tree structure.

2. The recording medium with video index information recorded therein according to claim 1;

wherein, of the structure element objects, at least one type of structure element object has retrieval information for retrieving contents of frames and/or voices directly and indirectly managed by the structure element object.

3. The recording medium with video index information recorded therein according to claim 1;

wherein the structure element object has retrieval information for retrieving frames and/or voices directly or indirectly managed by each structure element object.

4. A video information management method which uses a video index information for managing the video information by analyzing at least video including a plurality of frames, generating, apart from the video information, video index information for managing the video information, and referring to the generated video index information;

wherein

the video index information has a tree structure comprising, as structure element objects, at least one frame information for managing a given number of successive frames in the video information as one unit of information, at least one sound information for managing voices corresponding to a given range in the video information as one unit of information, at least one segment information for classifying the frame information and sound information each as one unit of information and managing the information as a desired significant collection of data, and package information for collectively managing a plurality of segment information as a group;

further the segment information manages the package information for collectively managing a plurality of other segment information generated by using the same frame as that of the frame information and sound information managed by the segment information and voices in the frame;

the package information is arranged, in addition to the frame information and sound information, under one segment information within said tree structure; and

the video information is managed by using said tree structure and structure element objects of the video index information.

5. The video information management method which uses the video index information according to claim 4;

wherein a particular structure element object in said tree structure of video index is specified to make the specified structure element object visible and/or audible.

6. The video information management method which uses the video index information according to claim 4;

wherein the frame information has a node ID in said tree structure, a range of successive frames managed according to the frame information, and a pointer indicating a position in the video information.

7. The video information management method which uses the video index information according to claim 4;

wherein the sound information has a node ID in said tree structure, a given range to be managed according to the frame information, and a pointer indicating a position in the video information.

8. The video information management method which uses the video index information according claim 4;

wherein the segment information has a node ID in said tree structure, identifying information for identifying frame information, sound information and package information to be managed according to the segment information, and a pointer to upper package information with the segment information belonging thereto.

9. The video information management method which uses the video index information according to claim 4;

wherein the package information has a node ID in said tree structure, a list of segment information managed according to the package information, and a pointer to upper segment information with the package information belonging thereto.

10. The video information management method which uses the video index information according to claim 4;

wherein the video index information has a given attribute object (106) allocated in said tree structure, all of structure element objects in the video index information each have a pointer to the attribute object (106), and given additional information can be added to a given structure element object using the attribute object (106) and the pointer to the attribute object (106).

11. The video information management method which uses the video index information according to claim 4;

wherein the video index information is separated from the video information and is stored in the separated state.

12. A video information management method which uses a video index information for managing the video information by analyzing video information including at least a plurality of frames, generating, apart from the video information, video index information for managing the video information, and referring to the generated video index information;

wherein

the video index information has at least a plurality of segment information for managing a range of a given number of successive frames, and a link list for indicating links between segment information for indicating in what order the plurality of segment information should be followed;

a plurality of link lists each indicating in what a desired order of segment information are to be followed of the plurality of segment information each constituting the video information, a plurality of view information corresponding to the link lists are set, and the video information is managed by using the video index information having the plurality of view information.

13. The video information management method which uses the video index information according to claim 12;

wherein, of the plurality of view Information in the video index information, particular view information is specified, and a portion of the video information is made visible and/or audible by using a link list for the specified view information.

14. The video information management method which uses the video index information according to claim 12;

wherein the video index information is'"'"'separated from the video information and is stored in the separated state.

15. A computer-readable recording medium with audio index information prepared by analyzing audio information recorded therein;

wherein

the audio index information has a tree structure comprising at least one sound information for managing voices corresponding to a given range in the audio information as one unit of information, at least one segment information for classifying one the sound information equivalent to one unit of information and managing the sound information as a collection of information having desired meaning, package information for collectively managing the segment information as a group each as a structure element object;

the segment information manages package information for collectively managing a plurality-of other segment information generated by using the same range of sounds as the sound information managed according to the segment information and managing the segment information as a group; and

the package information is allocated in addition to the sound information, under one segment information within said tree structure.

16. The recording medium with audio index information recorded therein according to claim 15;

wherein, of the structure element objects, at least one type of structure element object has retrieval information for retrieving contents of sounds directly or indirectly managed by the structure element object.

17. The recording medium with audio index information recorded therein according to claim 15;

wherein the structure element object has retrieval information for retrieving contents of sounds directly or indirectly managed by each structure element object.

18. An audio information management method which uses an audio index information for managing the audio information by analyzing audio information, generating, apart from the audio information, audio index information for managing the audio information, and referring to the generated audio index information;

wherein

the audio index information has a tree structure comprising at least one sound information for managing sounds corresponding to a given range in the audio information as one unit of information, at least one segment information for classifying the sound information equivalent to one unit of information and managing the sound information as a group having desired meaning, package information for collectively managing the group of segment information each as a structure element object;

the segment information has package information for collectively managing a plurality of other segment information generated using sounds in the same range as that of the sound information managed by the segment information as a group and managing the group of information;

the package information is allocated, in addition to the sound information, under one segment information within said tree structure; and

the audio information is managed by using said tree structure of the audio index information and the structure element objects.

19. The audio information management method which uses the audio index information (100) according to claim 18;

wherein the audio index information (100) is separated from the audio information and stored in the separated state.

20. A video retrieval method which uses a video index information for retrieving video information satisfying desired retrieval conditions from a plurality of video information by referring video index information prepared in correlation to each video information;

wherein

the video index information has a tree structure comprising at least one frame information for managing a given number of successive frames in the video information as one unit of information, at least one sound information for managing sounds corresponding to a given range in the video information as one unit of information, at least one segment information for classifying the sound information equivalent to one unit of information and managing the sound information as a group having desired meaning, package information for collectively managing the group of segment information each as a structure element object;

at least one type of structure element object of the structure element objects has retrieval information for retrieving contents of a frame and/or sounds directly or indirectly managed by the structure element object;

the segment information manages package information for collectively managing a plurality of segment information prepared by using the frame information managed by the segment information as well as a frame and sounds in the same range as that of the sound information and managing the group of segment information;

the package information is located, in addition to the frame information and sound information, under one segment information within said tree structure; and

retrieval conditions for desired vide information are inputted, a plurality of video index information prepared from a plurality of different video information to identify video index information including retrieval information satisfying the retrieval conditions, and a list of video information corresponding to the identified video index information is outputted as a result of retrieval.

21. The video retrieval method which uses the video index information according to claim 20;

wherein the frame information further has retrieval information for managing a representative frame freely selected from successive frames in the video information managed as one unit of information and retrieving contents of the representative frame; and

the sound information has retrieval information for managing a representative noise and sound in a given range selected from sounds corresponding to a given range in the video information managed as one unit of information and retrieving contents of the representative voice and sound.

22. The video retrieval method which uses the video index information according to claim 20;

wherein further a hierarchy level of the video index information is inputted as a retrieval range together with the retrieval conditions, video index information including retrieval information satisfying the retrieval conditions is identified by retrieving only retrieval information for structure element objects present in the retrieval range in the plurality of video index information, and a list for video information corresponding to the identified video index information is outputted as a result of retrieval.

23. The video retrieval method which uses the video index information according to claim 20;

wherein only retrieval information in video index information prepared in correlation to video information is used when retrieving desired video information from the plurality of video information.

24. The video retrieval method which uses the video index information according to claim 20;

wherein the retrieval conditions include a plurality of retrieval items each for a specified type of structure element objects.

25. The video retrieval method which uses the video index information according to claim 20;

wherein the retrieval conditions include a plurality of retrieval items each for the same type of structure element object.

26. The video retrieval method which uses the video index information according to claim 24;

wherein the retrieval conditions can logically express relations among the plurality of retrieval items.

27. The video retrieval method which uses the video index information according to claim 24;

wherein the retrieval item has the same data structure as that of the retrieval information present in a structure element object as an object for retrieval.

28. The video retrieval method which uses the video index information according to claim 25;

wherein the retrieval item has the same data structure as that of the retrieval information present in a structure element object as an object for retrieval.

29. The video retrieval method which uses the video index information according to claim 20;

wherein a list of structure element objects having retrieval information satisfying the retrieval conditions is outputted together with the list for video information for each video information on a list outputted as a result of retrieval.

30. The video retrieval method which uses the video index information according to claim 20;

wherein the retrieval conditions comprise a plurality of retrieval items capable of being weighted independently, and a score assessed according to weighting of the retrieval items is given to each video information on the list outputted as a result of retrieval.

31. The video retrieval method which uses the image index information according to claim 30;

wherein each video information on the list outputted as a result of retrieval is outputted according to the assessment value in the ranking order.

32. A video retrieval method which uses a video index information for retrieving a video satisfying desired retrieval conditions from image information by referring video index information previously prepared from video information;

wherein

the video index information has a tree structure comprising at least one frame information for managing a given number of successive frames in the video information as one unit of information, at least one sound information for managing sounds corresponding to a given range in the video information as one unit of information, at least one segment information for classifying the frame information and sound information each as one unit of information as a group having desired meaning and managing the group of information, and package information for managing a plurality of segment information as a group each as a structure element object;

the structure element object has retrieval information for retrieving contents of frames and/or sounds directly and indirectly managed by the structure element object;

the segment information manages package information for managing a plurality of other segment information generated using frames, sounds in the same range as that of the frame information and sound information managed by the segment information;

the package information is located, in addition to the frame information and sound information, under one segment information within said tree structure;

retrieval conditions for a desired video-image are inputted, a structure element object including retrieval information satisfying the retrieval conditions is identified by retrieving the video index information, and a list for the identified structure element objects is outputted as a result of retrieval.

33. The video retrieval method which uses the video index information according to claim 32;

wherein a hierarchy level in a direction of depth in said tree structure is inputted as a retrieval range together with the retrieval conditions, a structure element object including retrieval information satisfying the retrieval conditions is identified by retrieving only retrieval information for structure element objects present in the retrieval range in the video index information, and a list of identified structure element objects is outputted as a result of retrieval.

34. The video retrieval method which uses the video index information according to claim 32;

wherein the retrieval conditions comprises a plurality of retrieval items capable of being weighted independently, and a score assessed according to weighting of the retrieval items is assigned to each structure element object on the list outputted as a result of retrieval.

35. The video retrieval method which uses the video index information according to claim 34;

wherein each of the structure element objects on the list outputted as a result of retrieval is outputted according to the assessment value in the ranking order.

36. An audio retrieval method which uses an audio index information for retrieving audio information satisfying desired retrieval conditions from a plurality of audio information by referring to audio index information previously prepared in correlation to each audio information;

wherein

the audio index information has a tree structure comprising at least one sound information for managing sounds corresponding to a given range in the audio information, at least one segment information for classifying the sound information as one unit of information and managing the sound information as a group having desired meaning, package information for managing the plurality of segment information each as a structure element object;

at least one type of structure element objects of the structure element objects have retrieval information for retrieving contents of sounds directly or indirectly managed by the structure element object;

the segment information manages package information for collectively managing other segment information prepared by using sounds in the same range as that of the sound information managed by the segment information and managing the segment information as a group;

the package information is allocated in addition to the sound information, under one segment information within said tree structure; and

retrieval conditions as conditions for desired sounds are inputted, audio index information satisfying the retrieval conditions is identified by retrieving a plurality of audio index information previously prepared from a plurality of different audio information, and a list of audio information corresponding to the identified audio index information is outputted as a result of retrieval.

37. The audio retrieval method which uses the audio index information according to claim 36;

wherein positional information identifying a position of a desired voice is added according to a structure element object satisfying the retrieval conditions to each audio information on a list outputted as a result of retrieval and the audio information with the positional information added thereto is outputted.

38. The audio retrieval method which uses the audio index information according to claim 36;

wherein the retrieval conditions comprises a plurality of retrieval items each capable of being weighted independently, and a score assessed according to weighting of each retrieval item is added to each audio information on the list outputted as a result of retrieval.

39. The audio retrieval method which uses the audio index information according to claim 38;

wherein each of audio information on the list outputted as a result of retrieval is outputted according to the assessment order in the ranking order.

40. An audio retrieval method which uses an audio index information for retrieving sounds satisfying a desired retrieval condition from audio information by referring to audio index information previously prepared from audio information;

wherein

the audio index information has a tree structure comprising at least one sound information for managing voiced and sounds corresponding to a given range in the audio information as one unit of information, at least one segment information for classifying the sound information as a group and managing the sound information as a group having desired meanings, and package information for collectively managing the group of segment information;

the structure element object has retrieval information for retrieving contents of sounds directly or indirectly managed by the structure element object;

the segment information has package information for collectively managing a plurality of other segment information prepared using sounds in the same range as that the sound information managed by the segment information as a group;

the package information is allocated in addition to the sound information under one segment information within said tree structure; and

retrieval conditions for desired sounds are inputted, a structure element object including retrieval information satisfying the retrieval conditions is identified by retrieving the audio index information, and a list for the identified structure elements is outputted as a result of retrieval.

41. The audio retrieval method which uses the audio index information according to claim 40;

wherein a hierarchy level in a direction of depth in said tree structure is inputted together with the retrieval conditions, a structure element object including retrieval information satisfying the retrieval conditions is identified by retrieving only retrieval information for structure element objects present in the retrieval range in the audio index information, and a list for the identified structure element objects is outputted as a result of retrieval.

42. The audio retrieval method which uses the audio index information according to claim 40;

wherein the retrieval conditions consist of a plurality of retrieval items each capable of being weighted independently, and a score assessed according to weighting of the retrieval item is given to each of the structure element objects on the list outputted as a result of retrieval.

43. The audio retrieval method which uses the audio index information according to claim 42;

wherein each of structure elements on the list outputted as a result of retrieval is outputted according to the assessment value in the ranking order.

44. A video retrieval system comprising:

a plurality of video information databases each storing therein a plurality of video information;

a plurality of video index information databases each storing video index information previously prepared in correlation to each video information;

a retrieval information delivery file generated by extracting retrieval information for retrieving video information stored in said video information databases from the video index information stored in said plurality of index information databases; and

a retrieval device for retrieving video information completely or almost satisfying a desired retrieval condition from said video information databases by using said retrieval information delivery file;

wherein

the video index information in said video index information databases has a tree structure comprising at least one frame information for managing a given number of successive frames in the video information as one unit of information, at least one sound information for managing sounds corresponding to a given range in the video information as one unit of information, at least one segment information for classifying the frame information and sound information each as one unit of information and managing the information as a group having desired meanings, and package information for managing a plurality of segment information as a group each as a structure element object;

of the structure element objects, at least one type of structure element objects have retrieval information for retrieving contents of frames and/or sounds directly or indirectly managed by the structure element object, the segment information manages package information for managing a plurality of other segment information prepared by using the frame information managed by the segment information and sounds in the same frame as that of the sound information as a group, and further the package information is allocated in addition to the frame information and sound information under one segment information within said tree structure;

said retrieval information delivery file stores therein in a correlated form the retrieval information extracted from said plurality of video index information databases, and the retrieval information classified into a plurality of categories with a structure element object as an extraction source of the retrieval information and video index information having the structure element object present; and

said retrieval device identifies, when retrieval conditions for a desired video-image are inputted, retrieval information completely or almost satisfying the retrieval conditions using each retrieval information stored in said retrieval information delivery file and information relating to and a category of the retrieval information, and outputs the video information as a result of retrieval according to the video index information including the retrieval information.

45. The video retrieval system according to claim 44;

wherein said video information databases, video index information databases, retrieval information delivery files, and retrieval devices are provided on a network, and

any retrieval device on the network can be connected via a given retrieval information delivery file to any of all of the video index information databases and all of the video information databases.

46. The video retrieval system according to claim 44;

wherein said video information databases, video index information databases, retrieval information delivery files, and retrieval devices are provided on a network, and

any retrieval device on the network can be connected via a given retrieval information delivery file on the network to any of all of the video index information databases and all of the video information databases.

47. A video retrieval system comprising:

a plurality of video information databases each storing therein a plurality of video information;

a plurality of video index information databases each storing video index information previously prepared in correlation to each video information;

a retrieval information delivery file generated by extracting retrieval information for retrieving video information stored in said video information databases from the video index information stored in said plurality of index information databases; and

a retrieval device for retrieving video information completely or almost satisfying a desired retrieval condition from said video information databases by using said retrieval information delivery file;

wherein

the video index information in said video index information databases has a tree structure comprising at least one frame information for managing a given number of successive frames in the video information as one unit of information, at least one sound information for managing sounds corresponding to a given range in the video information as one unit of information, at least one segment information for classifying the frame information and sound information each as one unit of information and managing the information as a group having desired meanings, and package information for managing a plurality of segment information as a group each as a structure element object;

the segment information manages package information for managing a plurality of other segment information prepared by using the frame information managed by the segment information and frames and sounds in the same range as that of the sound information as a group, and further the package information is allocated in addition to the frame information and sound information under one segment information in said tree structure;

said retrieval information delivery file stores therein the plurality of retrieval information classified into a plurality of categories, each retrieval information correlated to a structure element object retrieved according to the retrieval information and video index information with the structure element object present therein; and

said retrieval device identifies, when retrieval conditions for a desired video-image are inputted, retrieval information completely or almost satisfying the retrieval conditions using each retrieval information stored in said retrieval information delivery file and information relating to and a category of the retrieval information, and outputs the video information as a result of retrieval according to the video index information including the retrieval information.

48. The video retrieval system according to claim 47;

wherein said video information databases, video index information databases, retrieval information delivery files, and retrieval devices are provided on a network, and

any retrieval device on the network can be connected via a given retrieval information delivery file to any of all of the video index information databases and all of the video information databases.

49. The video retrieval system according to claim 47;

wherein said video information databases, video index information databases, retrieval information delivery files, and retrieval devices are provided on a network, and

any retrieval device on the network can be connected via a given retrieval information delivery file on the network to any of all of the video index information databases and all of the video information databases.