Multimedia information arranging apparatus and arranging method

• US 6,948,123 B2
• Filed: 03/20/2002
• Issued: 09/20/2005
• Est. Priority Date: 10/27/1999
• Status: Expired due to Term

• Current Assignee: Fujitsu Limited
• Original Assignee: Fujitsu Limited
• Inventors: Endou, Susumu, Masumoto, Daiki, Nagata, Shigemi, Sashida, Naoki
• Primary Examiner(s): Cabeca, John
• Assistant Examiner(s): LEE, TING ZHOU
• Application Number:
  US10/101,272
• Publication Number:
  US 20020105541A1
• Time in Patent Office: 1,280 Days
• Field of Search: 715/700, 715/968, 707/3, 707/6, 707/7, 707/102, 707/104.1, 382/305
• US Class Current: 715/700
• CPC Class Codes: G06F 16/438 Presentation of query results
• CPC Class Codes: G06F 16/54 Browsing; Visualisation therefor
• CPC Class Codes: G06F 18/2137 based on criteria of topology preservation, e.g. multidimensional scaling or self-organising maps
• CPC Class Codes: Y10S 707/99936 Pattern matching access
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing
• CPC Class Codes: Y10S 715/968 interface for database querying and retrieval

Abstract

A multimedia information arranging apparatus that utilizes various feature values contained in multimedia information, carries out a flexible retrieval and classification and displays the information in an information set arrangement space is provided.

An information set obtaining portion 10 obtains information sets. An axis setting portion 20 assigns the feature value of the media information to an information set arrangement space axis so as to set the information set arrangement space. A feature value extracting portion 30 extracts the feature value from each of the media information of the obtained information set, and an information set arranging portion 40 arranges the information sets in the information set arrangement space according to their feature values. An information displaying portion 50 displays the information set arrangement space in such a manner as to be seen from a predetermined display viewpoint. By seeing the display result, as necessary, the axis setting portion 20 resets the information set arrangement space axis using a different feature value, so that the information sets are rearranged and redisplayed in the information set arrangement space, and retrievals of different aspects are repeated.

Claims

1. A multimedia information arranging apparatus comprising:

a computer, comprising an information set obtaining portion for obtaining pieces of media information in units of information sets, which are formed by grouping together the pieces of media information that are related to a same target, the media information being of a same kind and different kinds, from among a group of pieces of media information including image information, text information and audio information;

an axis setting portion for assigning an attribute of a feature value extracted from the obtained pieces of media information contained in the information sets to an axis of a space in which a group of the information sets is arranged and setting an information set arrangement space with one or more axes;

a feature value extracting portion for extracting a component of the feature value from the pieces of media information in the information sets;

an information set arranging portion for arranging the information sets in the information set arrangement space based on the attribute of the feature value of the pieces of media information contained in the information sets and the component of his feature value; and

an information displaying portion for displaying pieces of media information corresponding to a viewpoint with respect to the information set arrangement space, from among the pieces of media information of the information sets arranged in the information set arrangement space.

2. The multimedia information arranging apparatus according to claim 1, wherein, in the axis setting portion, a plurality of the attributes of the feature values are assigned in combination to one axis of the space or one attribute of the feature value is assigned to a plurality of the axes.

3. The multimedia information arranging apparatus according to claim 2, wherein the axis setting portion has an axis resetting function of resetting an assignment of the attribute of the feature value to each of the axes of the information set arrangement space and resetting the information set arrangement space with one or more axes, and

The feature value extracting portion extracts the component of the feature value based an the axis-resetting by the axis setting portion, the information set arranging portion arranged the information sets in the information set arrangement space based on the component of the extracted feature value, and the information displaying portion displays the pieces of media information corresponding to the viewpoint with respect to the reset information set arrangement space.

4. The multimedia information arranging apparatus according to claim 1 or, wherein the axis setting portion has an axis resetting function of resetting an assignment of the attribute of the feature value to each of the axes of the information set arrangement space and resetting the information set arrangement space with one or more axes, and

the feature value extracting portion extracts the component of the feature value based on the axis-resetting by the axis setting portion, the information set arranging portion arranges the information sets in the information set arrangement space based on the component of the extracted feature value, and the information displaying portion displays the pieces of media information corresponding to toe viewpoint with respect to the reset information set arrangement space.

5. The multimedia information arranging apparatus according to claim 1, wherein the information set obtaining portion comprises an information collecting portion for collecting the pieces of media information including the image information, the text information and the audio information, a relationship analyzing portion for analyzing a relationship between the collected pieces of media information and an information set generating portion for grouping and editing the related pieces of media information, which is of the same kind or the different kinds, as the information sets.

6. The multimedia information arranging apparatus according to claim 5, wherein the information collecting portion collects the pieces of media information according to a specified selection criterion when collecting the pieces of media information from a multimedia information group, and the selection criterion is specified by being selected from a criterion group including keyword information, site information, link information and similarity information with respect to a specific information set.

7. The multimedia information arranging apparatus according to claim 1, wherein the feature value is selected from a DCT coefficient feature value with respect to the image information, a wavelet transform coefficient feature value with respect to the image information, a HSI color histogram feature value with respect to the image information, a feature value representing a presence of a specific word with respect to the text information, a feature value of how many times a specific word is used with respect to the text information, a voice frequency feature value with respect to the audio information, an amplitude feature value with respect to the audio information and a time change feature value with respect to the audio information.

8. The multimedia information arranging apparatus according to claim 1, wherein the information set arranging portion comprises a self-organizing map processing portion for utilizing a local interaction and performing a self-organization by learning, and

the information set arranging portion arranges the information sets based on the feature value extracted by the feature value extracting portion using a self-organizing map processing by the self-organizing map processing portion.

9. The multimedia information arranging apparatus according to claim 1, wherein the information displaying portion comprises a display viewpoint moving portion having a function of moving a position of setting the viewpoint for displaying the information set and the information set arrangement space, and

the information displaying portion displays the information set arrangement space, in which the information sets are arranged, based on the position of the viewpoint set by the display viewpoint moving portion.

10. The multimedia information arranging apparatus according to claim 1, wherein the axis setting portion resets an assignment of the attribute of the feature value to each of the axis of the information set arrangement space and resets the information set arrangement space that has been displayed already,

the information set arranging portion rearranges the information sets in the reset information set arrangement space, and

when displaying how the information sets are rearranged, the information displaying portion moves the displayed pieces of media information at predetermined intervals from a position at which the information sets have been located before the rearrangement to a position at which they are to be located thereafter.

11. The multimedia information arranging apparatus according to claim 10, wherein the information set arranging portion has

a function of fixing an information set selected by a user to a specific position in an information set arrangement space specified by the user, and

a function of fixing the information set selected by the user to the specific position while rearranging only the other information sets according to the information set arrangement space, when rearranging the information sets with respect to the information set arrangement space with reset axis.

12. The multimedia information arranging apparatus according to claim 1, wherein the information set arranging portion has

a function of fixing an information set selected by a user to a specific position in an information set arrangement space specified by the user, and

a function of fixing the information set selected by the user to the specific position while rearranging only the other information sets according to the information set arrangement space, when rearranging the information sets with respect to the information set arrangement space with reset axis.

13. A computer-readable recording medium storing a processing program for realizing a multimedia information arranging apparatus for arranging and displaying information sets, each of which is a group of related pieces of media information including image information, text information and audio information, in an information set arrangement space, the processing program comprising:

an information set obtaining operation for obtaining the pieces of media information in units of the information sets, each formed by associating with each other the related pieces of media information including the image information, the text information and the audio information, the media information being of a same kind and different kinds;

an axis setting operation for assigning an attribute of a feature value extracted from the obtained pieces of media information contained in the information sets to an axis of a space in which a group of the information sets is arranged and setting an information set arrangement space with one or more axes;

a feature value extracting operation for extracting a component of the feature value from the pieces of media information in the information sets;

an information set arranging operation for arranging the information sets in the information set arrangement space based on the extracted component of the feature value; and

an information displaying operation for displaying pieces of media information corresponding to a viewpoint with respect to the information set arrangement space, from among the pieces of media information of the information sets arranged in the information set arrangement space.

14. An information set computer-implemented arranging method comprising:

obtaining pieces of media information in units of information sets, each formed by associating with each other related pieces of media information including image information, text information and audio information, the media information being of a same kind and different kinds;

assigning an attribute of a feature value extracted from the obtained pieces of media information contained in the information sets to an axis of a space in which a group of the information sets is arranged and setting an information set arrangement space with one or more axes;

extracting a component of the feature value from the pieces of media information in the information sets;

arranging the information sets in the information set arrangement space based on the attribute of the feature value of the pieces of media information contained in the information sets and the component of this feature value;

specifying a feature value that is different from the feature value with respect to the arranged information sets, resetting an assignment to each of the axis of the information set arrangement space, and rearranging the information sets in the information set arrangement space based on the resetting; and

setting the axis of the information set arrangement space and arranging the information sets in the information set arrangement space repeatedly while switching feature values to be used.